From erno at iki.fi  Tue Apr  3 08:47:36 2007
From: erno at iki.fi (Erno Kuusela)
Date: Tue, 3 Apr 2007 09:47:36 +0300
Subject: [Python-3000] IPv6 address tuple
In-Reply-To: <20070323220556.8C8A21C047@coconut.itojun.org>
References: <20070323093628.BC6A61C04E@coconut.itojun.org>
	<20070323220556.8C8A21C047@coconut.itojun.org>
Message-ID: <20070403064736.GC24629@erno.iki.fi>

hello,

On Sat, 24 Mar 2007, Jun-ichiro itojun Hagino wrote:

| > 	i understand your concern.  current 4-tuple is direct mapping from
| > 	struct sockaddr_in6 (which has those two additional members).
| 
| 	note however, you must rewrite DNS resolving from gethostby* to
| 	get*info anyways, so i'm not too sure if the number of tuple members in
| 	sockaddr matters that much.

This is true when using explicit DNS resolving, but Python socket API
allows you to pass in a string in place of the address in the tuple,
and then resolves it to an address behind your back. This
is probably the most common way of using DNS in Python.

It seems that when you have an AF_INET6 socket, it looks for
an AAAA record, and when you have an AF_INET socket, it goes for A.

Unfortunately this bumps the address family-agnostic get*info API
to second place in the convenience-of-use competition
(getaddrinfo example at <http://docs.python.org/lib/socket-example.html>).

  -- erno

From itojun at itojun.org  Tue Apr  3 08:54:57 2007
From: itojun at itojun.org (Jun-ichiro itojun Hagino)
Date: Tue,  3 Apr 2007 15:54:57 +0900 (JST)
Subject: [Python-3000] IPv6 address tuple
In-Reply-To: Your message of "Tue, 3 Apr 2007 09:47:36 +0300"
	<20070403064736.GC24629@erno.iki.fi>
References: <20070403064736.GC24629@erno.iki.fi>
Message-ID: <20070403065457.13D8A1C047@coconut.itojun.org>

> | > 	i understand your concern.  current 4-tuple is direct mapping from
> | > 	struct sockaddr_in6 (which has those two additional members).
> | 
> | 	note however, you must rewrite DNS resolving from gethostby* to
> | 	get*info anyways, so i'm not too sure if the number of tuple members in
> | 	sockaddr matters that much.
> 
> This is true when using explicit DNS resolving, but Python socket API
> allows you to pass in a string in place of the address in the tuple,
> and then resolves it to an address behind your back. This
> is probably the most common way of using DNS in Python.
> 
> It seems that when you have an AF_INET6 socket, it looks for
> an AAAA record, and when you have an AF_INET socket, it goes for A.

	hmm.  how would Python behave when there are multiple AAAA (or A)
	records?  just pick the first one?

> Unfortunately this bumps the address family-agnostic get*info API
> to second place in the convenience-of-use competition
> (getaddrinfo example at <http://docs.python.org/lib/socket-example.html>).

	with mixed IPv4/v6 environment (this would be the network environment
	for today and coming decade(s)), a program should try to connect both
	A and AAAA records, since connectivity situation varies by
	configuration such as network configuration, kernel compilation options
	and so forth.
	for this reason it would be better (or i might say we have to)
	for all libraries (like HTTP, FTP, and stuff) as well as user
	applications to use get*info API, instead of just building a tuple and
	then use AF_INET/6 socket.

itojun

From aahz at pythoncraft.com  Tue Apr  3 21:08:48 2007
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 3 Apr 2007 12:08:48 -0700
Subject: [Python-3000] Proposed changes to PEP3101 advanced string
	formatting -- please discuss and vote!
In-Reply-To: <d09829f50703140821t3f65db4bg3598a62b979a60cf@mail.gmail.com>
References: <d09829f50703121947y42cd6ad3nd4af96fc3ab361c7@mail.gmail.com>
	<45F7D7FC.5080908@gmail.com>
	<d09829f50703140821t3f65db4bg3598a62b979a60cf@mail.gmail.com>
Message-ID: <20070403190847.GA24460@panix.com>

[Taking the first day of Pesach as vacation gives me time to respond to
some old threads]

On Wed, Mar 14, 2007, Patrick Maupin wrote:
> On 3/14/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>> I detest the name 'flag_format', though - the function doesn't format a
>> flag!
> 
> I don't like it that much myself, either.  I considered
> "extended_format" and a few others, but wasn't happy with any of them.
>  Name suggestions are certainly welcome.

Why not just string.format()?  There's precedence for module-level
functions to have a different API from a method.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Need a book?  Use your library!

From aahz at pythoncraft.com  Tue Apr  3 21:15:35 2007
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 3 Apr 2007 12:15:35 -0700
Subject: [Python-3000] Proposed changes to PEP3101 advanced string
	formatting -- please discuss and vote!
In-Reply-To: <d09829f50703121947y42cd6ad3nd4af96fc3ab361c7@mail.gmail.com>
References: <d09829f50703121947y42cd6ad3nd4af96fc3ab361c7@mail.gmail.com>
Message-ID: <20070403191535.GB24460@panix.com>

On Mon, Mar 12, 2007, Patrick Maupin wrote:
>
> Feature:  Alternate syntaxes for escape to markup.
> 
> The second method is the well-understood ${} syntax.  The $ is easy to
> find in a sea of { characters, and the only special handling required
> is that every $ must be doubled.

-0

> The third method is something I came up with a couple of years ago
> when I was generating a lot of C code from templates.  It would make
> any non-Python programmer blanch, because it relies on significant
> whitespace, but it made for very readable technical templates.  WIth
> this method "{foo}" escapes to markup, but when there is whitespace
> after the leading "{",  e.g. "{ foo}", the brace is not an escape to
> markup.  If the whitespace is a space, it is removed from the output,
> but if it is '\r', '\n', or '\t', then it is left in the output.  The
> result is that, for example, most braces in most C texts do not need
> to have spaces inserted after them, because they immediately precede a
> newline.

-1

> Feature:  Automatic search of locals() and globals() for name lookups
> if no parameters are given.

-1 -- make it a flag for string.format() or use a different function
name.

> Feature:  Placement of a dummy record on the traceback stack for
> underlying errors.

+1

> Changed feature: By default, not using all arguments is not an exception

-0 for *args, +1 for **kwargs

While I understand your reasoning, I still think that mismatched strings
and arguments is likely to be a significant problem; instead I favor
allowing this feature to be turned off.

> Feature:  Exception raised if attribute with leading underscore accessed.

+1
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Need a book?  Use your library!

From oliphant.travis at ieee.org  Wed Apr  4 01:03:05 2007
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Tue, 03 Apr 2007 17:03:05 -0600
Subject: [Python-3000] PEP for extended buffer protocol
Message-ID: <euum29$vip$1@sea.gmane.org>


I know everybody is busy, but I would appreciate some kind of feedback 
from Python developers about the attached extended buffer protocol which 
I am proposing for Python 3k.

I am working on the changes to the struct module needed to accomodate 
the enhanced syntax.

I'd like to know if I these changes will be accepted into Python and if 
the remaining extended buffer protocol changes will be allowed as well.

Thank you very much,

-Travis


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pep_buffer.txt
Url: http://mail.python.org/pipermail/python-3000/attachments/20070403/d2d72d76/attachment-0001.txt 

From guido at python.org  Tue Apr 10 19:53:40 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Apr 2007 10:53:40 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
Message-ID: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>

As I stated in my PyCon keynote and repeated (though somewhat hidden)
in PEP 3000, all PEPs for Py3k are due by April. The end of April is
only a few weeks away, so now is the time to start drafting that PEP
if you care about Python 3000! This is so that we will be able to
concentrate on *implementing* the accepted proposals by June, in time
for the planned release 3.0a1.

Your PEP doesn't have to be *accepted* by April 30; but it should be
in good shape, have seen some discussion on python-3000 (or
python-ideas), and have popular support. We can do the final
discussion about acceptance of individual PEPs (including some
revisions) after the deadline; but no PEP submitted after the deadline
has a chance.

This applies to my own pet projects as well; in particular, if I want
Abstract Base Classes in Py3k, I will have to write up a PEP by April
30. If you want interfaces or generic functions, likewise, you need to
have a decent PEP submitted by April 30!

I'm currently working on the new I/O library, and am confident that
this will be in a usable shape by the end of this week. fter that, I
plan to concentrate on the str/unicode unification, which is by far
the riskiest project still to be done.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Tue Apr 10 20:28:02 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 10 Apr 2007 12:28:02 -0600
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
Message-ID: <d11dcfba0704101128s6860db8cudc4e8aee9ba6a19d@mail.gmail.com>

On 4/10/07, Guido van Rossum <guido at python.org> wrote:
> As I stated in my PyCon keynote and repeated (though somewhat hidden)
> in PEP 3000, all PEPs for Py3k are due by April. The end of April is
> only a few weeks away, so now is the time to start drafting that PEP
> if you care about Python 3000! This is so that we will be able to
> concentrate on *implementing* the accepted proposals by June, in time
> for the planned release 3.0a1.

Does this also include proposals for parts of the Python 3000 stdlib,
e.g. changing or replacing a particular module API?

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From python at rcn.com  Tue Apr 10 20:42:10 2007
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Apr 2007 14:42:10 -0400 (EDT)
Subject: [Python-3000] Reminder: Py3k PEPs due by April
Message-ID: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>

Thanks for the reminder.  I expect to write two:

* PEP to eliminate __del__ in favor of weakref idioms

* PEP on simpler alternative to abstract-base-classes
suggesting an occasional method special attribute
(so for instance a __getitem__ method can tell you
whether it thinks it is a sequence or mapping).


Here's that I think might not need a PEP:

* Eliminate implicit string concatenation:   "abc" "def"
in favor of an explicit + operation.  That simplifies
the grammar just a bit and the compiler already is 
smart enough to do constant fold this operation at 
compile time.  When there are multi-line concats, I think
the parenthesis serves us much better than a trailing \
which is ugly and relies on having no trailing whitespace.
Replace:
   'hello         ' \
   'world         '
With:
   ('hello        ' +
    'world        ')



Raymond

From guido at python.org  Tue Apr 10 20:50:30 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Apr 2007 11:50:30 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <d11dcfba0704101128s6860db8cudc4e8aee9ba6a19d@mail.gmail.com>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<d11dcfba0704101128s6860db8cudc4e8aee9ba6a19d@mail.gmail.com>
Message-ID: <ca471dc20704101150m1a6918dan6636019e592aeb7@mail.gmail.com>

On 4/10/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/10/07, Guido van Rossum <guido at python.org> wrote:
> > As I stated in my PyCon keynote and repeated (though somewhat hidden)
> > in PEP 3000, all PEPs for Py3k are due by April. The end of April is
> > only a few weeks away, so now is the time to start drafting that PEP
> > if you care about Python 3000! This is so that we will be able to
> > concentrate on *implementing* the accepted proposals by June, in time
> > for the planned release 3.0a1.
>
> Does this also include proposals for parts of the Python 3000 stdlib,
> e.g. changing or replacing a particular module API?

Good question. I think it is wholly unrealistic to expect much
movement in the stdlib before July; let's plan to do the stdlib
upgrade after 3.0a1 is released, and exempt it from the PEP deadline.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Apr 10 20:53:27 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Apr 2007 11:53:27 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
Message-ID: <ca471dc20704101153n5df63f26m80e21c5eabbe11b8@mail.gmail.com>

On 4/10/07, Raymond Hettinger <python at rcn.com> wrote:
> Thanks for the reminder.  I expect to write two:
>
> * PEP to eliminate __del__ in favor of weakref idioms
>
> * PEP on simpler alternative to abstract-base-classes
> suggesting an occasional method special attribute
> (so for instance a __getitem__ method can tell you
> whether it thinks it is a sequence or mapping).
>
> Here's that I think might not need a PEP:
>
> * Eliminate implicit string concatenation:   "abc" "def"
> in favor of an explicit + operation.  That simplifies
> the grammar just a bit and the compiler already is
> smart enough to do constant fold this operation at
> compile time.  When there are multi-line concats, I think
> the parenthesis serves us much better than a trailing \
> which is ugly and relies on having no trailing whitespace.
> Replace:
>    'hello         ' \
>    'world         '
> With:
>    ('hello        ' +
>     'world        ')

That needs a PEP too; there are subtle issues like

   ('hello %s'
    'world' % 42)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Tue Apr 10 21:03:46 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Apr 2007 12:03:46 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
Message-ID: <bbaeab100704101203m4ee60f85x734c957cb7d702b2@mail.gmail.com>

On 4/10/07, Guido van Rossum <guido at python.org> wrote:
>
> As I stated in my PyCon keynote and repeated (though somewhat hidden)
> in PEP 3000, all PEPs for Py3k are due by April. The end of April is
> only a few weeks away, so now is the time to start drafting that PEP
> if you care about Python 3000! This is so that we will be able to
> concentrate on *implementing* the accepted proposals by June, in time
> for the planned release 3.0a1.



Just so people know what I have in the pipeline, I have two PEPs planned:

1. Add a built-in named __main__ so as to fix the problem of executing a
script that exists in a package and uses new relative imports (see
http://sayspy.blogspot.com/2007/03/possible-change-to-handle-main-code-in.htmlfor
a more detailed explanation).

2. Remove file.__init__, code.__init__, and object.__subclasses__ for
security reasons.  I am trying to get approval from my supervisor to make
public a doc I wrote for a class that explains my security work that I
presented on at PyCon to vet the design one more time on python-dev before I
push for these removals.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/ac79adb7/attachment.htm 

From brett at python.org  Tue Apr 10 21:05:06 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Apr 2007 12:05:06 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
Message-ID: <bbaeab100704101205n22ba44f4g6dedb7f1841ab1c5@mail.gmail.com>

On 4/10/07, Raymond Hettinger <python at rcn.com> wrote:
>
> Thanks for the reminder.  I expect to write two:
>
> * PEP to eliminate __del__ in favor of weakref idioms



Woohoo!

And I still expect a named tuple implementation in the collections module.
=)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/630e3297/attachment.html 

From python at rcn.com  Tue Apr 10 21:05:18 2007
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 10 Apr 2007 15:05:18 -0400 (EDT)
Subject: [Python-3000] Fwd: Re:  Reminder: Py3k PEPs due by April
Message-ID: <20070410150518.BEU90548@ms09.lnh.mail.rcn.net>

> there are subtle issues like
>
>   ('hello %s'
>    'world' % 42)

Thank goodness the % formatting operator is going away.  Having the same precendence as the modulo-operation has not been a good thing.


Raymond
-------------- next part --------------
An embedded message was scrubbed...
From: "Guido van Rossum" <guido at python.org>
Subject: Re: [Python-3000] Reminder: Py3k PEPs due by April
Date: Tue, 10 Apr 2007 11:53:27 -0700
Size: 3647
Url: http://mail.python.org/pipermail/python-3000/attachments/20070410/a9c63f7b/attachment.eml 

From pje at telecommunity.com  Tue Apr 10 21:23:22 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 10 Apr 2007 15:23:22 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <bbaeab100704101203m4ee60f85x734c957cb7d702b2@mail.gmail.co
 m>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>

At 12:03 PM 4/10/2007 -0700, Brett Cannon wrote:
>2. Remove file.__init__, code.__init__, and object.__subclasses__ for 
>security reasons.

The __subclasses__ method is useful -- even more so in 3.x than in 2.x, 
because in 3.x there are no classic classes.

I was planning to make use of this in my generic function libraries to 
allow automatic checking for completeness and ambiguity of a ruleset, given 
the classes that are currently defined.  Currently, this is limited by the 
inability to access classic classes' subclasses, but in 3.x __subclasses__ 
is an ideal way to locate active classes.


From brett at python.org  Tue Apr 10 21:25:29 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Apr 2007 12:25:29 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
Message-ID: <bbaeab100704101225t42818dc6rd2c169e9f0cbed1c@mail.gmail.com>

On 4/10/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>
> At 12:03 PM 4/10/2007 -0700, Brett Cannon wrote:
> >2. Remove file.__init__, code.__init__, and object.__subclasses__ for
> >security reasons.
>
> The __subclasses__ method is useful -- even more so in 3.x than in 2.x,
> because in 3.x there are no classic classes.
>
> I was planning to make use of this in my generic function libraries to
> allow automatic checking for completeness and ambiguity of a ruleset,
> given
> the classes that are currently defined.  Currently, this is limited by the
> inability to access classic classes' subclasses, but in 3.x __subclasses__
> is an ideal way to locate active classes.
>
>

The problem with it, though, is it does expose *so* much.  If I have a class
defined that is not exposed within the interpreter but is live in the
process, I can still get to it with this method.  That's a problem if the
class has a security component to it where there is a class attribute that
should not be exposed within the interpreter.

This can be discussed more if/when I get my security paper out in the public
and get a PEP written to actually push for the removals.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/be95fe46/attachment.html 

From steven.bethard at gmail.com  Tue Apr 10 21:31:45 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 10 Apr 2007 13:31:45 -0600
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <bbaeab100704101205n22ba44f4g6dedb7f1841ab1c5@mail.gmail.com>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<bbaeab100704101205n22ba44f4g6dedb7f1841ab1c5@mail.gmail.com>
Message-ID: <d11dcfba0704101231l57408f33v35bc7847ce926894@mail.gmail.com>

On 4/10/07, Brett Cannon <brett at python.org> wrote:
> And I still expect a named tuple implementation in the collections module.

Fortunately, Raymond borrowed the time machine already::

Python 2.6a0 (trunk:54711M, Apr  6 2007, 23:48:33) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import collections
>>> collections.NamedTuple
<function NamedTuple at 0x00B47980>

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From pje at telecommunity.com  Tue Apr 10 22:47:44 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 10 Apr 2007 16:47:44 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <bbaeab100704101225t42818dc6rd2c169e9f0cbed1c@mail.gmail.co
 m>
References: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
	<ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070410164341.02c47678@sparrow.telecommunity.com>

At 12:25 PM 4/10/2007 -0700, Brett Cannon wrote:
>On 4/10/07, Phillip J. Eby 
><<mailto:pje at telecommunity.com>pje at telecommunity.com> wrote:
>>At 12:03 PM 4/10/2007 -0700, Brett Cannon wrote:
>> >2. Remove file.__init__, code.__init__, and object.__subclasses__ for
>> >security reasons.
>>
>>The __subclasses__ method is useful -- even more so in 3.x than in 2.x,
>>because in 3.x there are no classic classes.
>>
>>I was planning to make use of this in my generic function libraries to
>>allow automatic checking for completeness and ambiguity of a ruleset, given
>>the classes that are currently defined.  Currently, this is limited by the
>>inability to access classic classes' subclasses, but in 3.x __subclasses__
>>is an ideal way to locate active classes.
>
>The problem with it, though, is it does expose *so* much.

 From my POV, that's a feature, not a bug.  :)


>   If I have a class defined that is not exposed within the interpreter 
> but is live in the process, I can still get to it with this 
> method.  That's a problem if the class has a security component to it 
> where there is a class attribute that should not be exposed within the 
> interpreter.
>
>This can be discussed more if/when I get my security paper out in the 
>public and get a PEP written to actually push for the removals.

If you're using a capabilities-based system (or a multiparadigm system like 
Zope 3's zope.security), it's easy enough to restrict access to special 
attributes like __subclasses__ (or for that matter, any other sensitive 
attributes), so I don't see why the feature itself needs to be removed.


From brett at python.org  Tue Apr 10 23:27:21 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Apr 2007 14:27:21 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <d11dcfba0704101231l57408f33v35bc7847ce926894@mail.gmail.com>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<bbaeab100704101205n22ba44f4g6dedb7f1841ab1c5@mail.gmail.com>
	<d11dcfba0704101231l57408f33v35bc7847ce926894@mail.gmail.com>
Message-ID: <bbaeab100704101427m150c1f97g105e96b916c4aeea@mail.gmail.com>

On 4/10/07, Steven Bethard <steven.bethard at gmail.com> wrote:
>
> On 4/10/07, Brett Cannon <brett at python.org> wrote:
> > And I still expect a named tuple implementation in the collections
> module.
>
> Fortunately, Raymond borrowed the time machine already::
>
> Python 2.6a0 (trunk:54711M, Apr  6 2007, 23:48:33) [MSC v.1310 32 bit
> (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import collections
> >>> collections.NamedTuple
> <function NamedTuple at 0x00B47980>



Ah, now I see the NEWS entry for it.  I guess my expectations have been met
then.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/f91d84bb/attachment.htm 

From jcarlson at uci.edu  Tue Apr 10 23:47:43 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 10 Apr 2007 14:47:43 -0700
Subject: [Python-3000] Fwd: Re:  Reminder: Py3k PEPs due by April
In-Reply-To: <20070410150518.BEU90548@ms09.lnh.mail.rcn.net>
References: <20070410150518.BEU90548@ms09.lnh.mail.rcn.net>
Message-ID: <20070410143944.62A5.JCARLSON@uci.edu>


Raymond Hettinger <python at rcn.com> wrote:
> > there are subtle issues like
> >
> >   ('hello %s'
> >    'world' % 42)
> 
> Thank goodness the % formatting operator is going away.  Having the
> same precendence as the modulo-operation has not been a good thing.

Did I miss a PEP somewhere?

 - Josiah


From greg.ewing at canterbury.ac.nz  Wed Apr 11 02:24:39 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Apr 2007 12:24:39 +1200
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
Message-ID: <461C2AC7.4090908@canterbury.ac.nz>

Raymond Hettinger wrote:

> Here's that I think might not need a PEP:
> 
> * Eliminate implicit string concatenation:   "abc" "def"
> in favor of an explicit + operation.

-0.707. I've used languages where the only way of
splitting a string over lines is to use concatenation,
and it's tedious.

> When there are multi-line concats, I think
> the parenthesis serves us much better than a trailing \

Which can be done perfectly well as things are:

     ('hello        '
      'world        ')

So no, I don't think this is uncontroversial enough
to get by without a PEP.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Apr 11 02:38:03 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Apr 2007 12:38:03 +1200
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
Message-ID: <461C2DEB.4070907@canterbury.ac.nz>

Phillip J. Eby wrote:

> The __subclasses__ method is useful -- even more so in 3.x than in 2.x, 
> because in 3.x there are no classic classes.

For security purposes, I think it would be better to
adopt a principle that it shouldn't be possible to
do anything dangerous merely by instantiating a
class (e.g. open() opens files, but file() doesn't).
Then __subclasses__ would be harmless.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Apr 11 02:54:01 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 11 Apr 2007 12:54:01 +1200
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
Message-ID: <461C31A9.5090204@canterbury.ac.nz>

Guido van Rossum wrote:
> all PEPs for Py3k are due by April.

Can we nominate one of our 2.x PEPs for promotion
to a 3000 PEP?

If so, I'd like to nominate PEP 335 - Overloadable
Boolean Operators.

--
Greg

From brett at python.org  Wed Apr 11 03:22:56 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 10 Apr 2007 18:22:56 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <461C2DEB.4070907@canterbury.ac.nz>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<5.1.1.6.0.20070410151803.02c2e440@sparrow.telecommunity.com>
	<461C2DEB.4070907@canterbury.ac.nz>
Message-ID: <bbaeab100704101822g2a2295f3hb5c37eabb499efa@mail.gmail.com>

On 4/10/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> Phillip J. Eby wrote:
>
> > The __subclasses__ method is useful -- even more so in 3.x than in 2.x,
> > because in 3.x there are no classic classes.
>
> For security purposes, I think it would be better to
> adopt a principle that it shouldn't be possible to
> do anything dangerous merely by instantiating a
> class (e.g. open() opens files, but file() doesn't).
> Then __subclasses__ would be harmless.



True.  As long as you don't use class attributes or have dangerous default
arguments (and that is the kicker usually) then that policy is doable.

And I should clarify that the method does not need to disappear as it can
move to an extension module somewhere.   I just don't want it exposed right
off of object and thus in the built-in namespace.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070410/cfbcd37e/attachment.html 

From guido at python.org  Wed Apr 11 06:50:19 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Apr 2007 21:50:19 -0700
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <461C31A9.5090204@canterbury.ac.nz>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<461C31A9.5090204@canterbury.ac.nz>
Message-ID: <ca471dc20704102150g72e3a884jf4d540c96bee8a2c@mail.gmail.com>

On 4/10/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
> > all PEPs for Py3k are due by April.
>
> Can we nominate one of our 2.x PEPs for promotion
> to a 3000 PEP?
>
> If so, I'd like to nominate PEP 335 - Overloadable
> Boolean Operators.

Sure. That looks like something I recall intending to do and never
getting to. Or maybe I'm imagining it. Anyway, it sounds like a good
idea if it can be done efficiently.

 I'm kind of worried about the performance though -- won't you have to
pay at least for the lookup of __and1__ or __or1__ in the MRO of the
first argument?

Also, given the plans for backporting significant portions of 3.0 to
2.6, why not propose this for 2.6? I see no incompatibilities that
would prevent this. If it is checked in soon enough it should be
straightforward to merge it into 3.0. Backporting after first
implementing it in 3.0 would be more work.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From walter at livinglogic.de  Wed Apr 11 10:10:20 2007
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 11 Apr 2007 10:10:20 +0200
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
 python/branches/p3yk/Lib: io.py	test/test_io.py
In-Reply-To: <20070411010907.226411E400D@bag.python.org>
References: <20070411010907.226411E400D@bag.python.org>
Message-ID: <461C97EC.8040903@livinglogic.de>

guido.van.rossum wrote:

> Author: guido.van.rossum
> Date: Wed Apr 11 03:09:03 2007
> New Revision: 54742
> 
> Modified:
>    python/branches/p3yk/Lib/io.py
>    python/branches/p3yk/Lib/test/test_io.py
> Log:
> Checkpoint so I can continue to work on this at a different box.
> There is somewhat working (but slow) code supporting seek/tell for text files,
> but extensive testing exposes a bug I can't nail down.
> 
> 
> Modified: python/branches/p3yk/Lib/io.py
> [...]
> +        decoder_state = pickle.dumps(self._decoder, 2)

Would it make sense to make the state of the decoder public, e.g. by 
adding setstate() and getstate() methods? This would give a cleaner API.

Should I work on a patch?

Servus,
    Walter

From guido at python.org  Wed Apr 11 16:47:25 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Apr 2007 07:47:25 -0700
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
	python/branches/p3yk/Lib: io.py test/test_io.py
In-Reply-To: <461C97EC.8040903@livinglogic.de>
References: <20070411010907.226411E400D@bag.python.org>
	<461C97EC.8040903@livinglogic.de>
Message-ID: <ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>

On 4/11/07, Walter D?rwald <walter at livinglogic.de> wrote:
> Would it make sense to make the state of the decoder public, e.g. by
> adding setstate() and getstate() methods? This would give a cleaner API.

I've been thinking of the same thing!

I wonder if it would be possible to return the state as a pair
(unread, flags) where unread is a (byte) string of unprocessed bytes
and flags is some other state, with the constraint that in the initial
state the flags must be zero. Then I can optimize the case where flags
is returned as zero by subtracting len(unread) from the current
position and that'd be the correct seek position. I imagine most
decoders have only very few flags they care about. (The worst might be
the utf-16 decoder which must have a flag to remember whether it
already saw a byte order marker, and another indicating the byte
order. Maybe we'll have to special-case that one, so don't worry too
much about it.)

> Should I work on a patch?

That would be great!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From talin at acm.org  Wed Apr 11 17:01:45 2007
From: talin at acm.org (Talin)
Date: Wed, 11 Apr 2007 08:01:45 -0700
Subject: [Python-3000] Line continuation using ellipsis
Message-ID: <461CF859.6050003@acm.org>

I'm not sure that I am the first to bring this up, but I might as well 
get it written down.

I've never liked using the backslash operator for line continuation, for 
two reasons:

1) It requires that it be the last character of a line, meaning that you 
have to insure that there's no trailing whitespace, which can be 
difficult with some editors.

2) As a wise sage once said, "it looks like line noise on my monitor".

I'm not proposing to remove the feature, however I'd like to see an 
alternative method for declaring statements that cross a line boundary. 
I seem to recall at one point someone suggesting the use of ellipsis, 
which makes a lot of sense to me:

    sorted_result = partition_lower( input_list, pivot ) ...
       + pivot ...
       + partition_upper( input_list, pivot )

Parsing-wise, an ellipsis operator would have the following meaning: 
Inside a parenthesized or bracketed expression, it would have the same 
meaning it does now. Otherwise, it has the meaning "consume all 
whitespace, including newlines".

-- Talin

From fdrake at acm.org  Wed Apr 11 17:30:29 2007
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed, 11 Apr 2007 11:30:29 -0400
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <461CF859.6050003@acm.org>
References: <461CF859.6050003@acm.org>
Message-ID: <200704111130.30019.fdrake@acm.org>

On Wednesday 11 April 2007 11:01, Talin wrote:
 > Parsing-wise, an ellipsis operator would have the following meaning:
 > Inside a parenthesized or bracketed expression, it would have the same
 > meaning it does now. Otherwise, it has the meaning "consume all
 > whitespace, including newlines".

Really?  What about this:

    -----------------------------------
    print "strings are cool", ...


    "really!"
    -----------------------------------

(Note that there are two blank lines between the two non-blank lines.)

Would this be a single print statement, or a print statement and an expression 
statement?


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From pje at telecommunity.com  Wed Apr 11 18:04:58 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 11 Apr 2007 12:04:58 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <ca471dc20704102150g72e3a884jf4d540c96bee8a2c@mail.gmail.co
 m>
References: <461C31A9.5090204@canterbury.ac.nz>
	<ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<461C31A9.5090204@canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20070411115258.02ac4588@sparrow.telecommunity.com>

At 09:50 PM 4/10/2007 -0700, Guido van Rossum wrote:
>I'm kind of worried about the performance though -- won't you have to
>pay at least for the lookup of __and1__ or __or1__ in the MRO of the
>first argument?

Not if they're tp_* slots.  And we could introduce the feature only for 
new-style classes, thereby preventing any performance hit for existing 
classic classes.  (Classic classes' tp_and1 et al would simply invoke 
normal boolean testing.)

What I find interesting about this proposal is that it comes very close to 
allowing abstract interpretation of arbitrary Python expressions.  If there 
were a couple of other operator pairs like 'if1/if2', you'd be able to take 
a generator expression and transform it to SQL without parsing or bytecode 
decompiling (the two techniques currently used by today's Python ORM's like 
DejaVu).

Unfortunately, to get full abstract interpretation (ala PyPy, but without 
looking at any bytecode), there would need to be a way to handle 
try/except/finally and while loops, and I don't have any ideas about how 
one would implement those.


From g.brandl at gmx.net  Wed Apr 11 18:08:36 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 11 Apr 2007 18:08:36 +0200
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <461CF859.6050003@acm.org>
References: <461CF859.6050003@acm.org>
Message-ID: <evj164$e41$1@sea.gmane.org>

Talin schrieb:
> I'm not sure that I am the first to bring this up, but I might as well 
> get it written down.
> 
> I've never liked using the backslash operator for line continuation, for 
> two reasons:
> 
> 1) It requires that it be the last character of a line, meaning that you 
> have to insure that there's no trailing whitespace, which can be 
> difficult with some editors.

Then use parentheses. That should be possible almost everywhere.
Also, if we have to have a line-continuation token, the backslash is a
good choice because it's present in so many other languages.

> 2) As a wise sage once said, "it looks like line noise on my monitor".

But it's end-of-line noise ;)

> I'm not proposing to remove the feature, however I'd like to see an 
> alternative method for declaring statements that cross a line boundary. 
> I seem to recall at one point someone suggesting the use of ellipsis, 
> which makes a lot of sense to me:
> 
>     sorted_result = partition_lower( input_list, pivot ) ...
>        + pivot ...
>        + partition_upper( input_list, pivot )
> 
> Parsing-wise, an ellipsis operator would have the following meaning: 
> Inside a parenthesized or bracketed expression, it would have the same 
> meaning it does now. Otherwise, it has the meaning "consume all 
> whitespace, including newlines".

This doesn't work with Py3k as the ellipsis is a general expression element
there. You can already write

...

and it will be short for

Ellipsis

cheers,
Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From eoghan at qatano.com  Wed Apr 11 18:42:08 2007
From: eoghan at qatano.com (Eoghan Murray)
Date: Wed, 11 Apr 2007 17:42:08 +0100
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
Message-ID: <ddb3f080704110942j7f11b77bt695023779da9b99c@mail.gmail.com>

On 10/04/07, Raymond Hettinger <python at rcn.com> wrote:
>
> [snip]
>
> Here's that I think might not need a PEP:
>
> * Eliminate implicit string concatenation:   "abc" "def"
> in favor of an explicit + operation.  That simplifies
> the grammar just a bit and the compiler already is
> smart enough to do constant fold this operation at
> compile time.  When there are multi-line concats, I think
> the parenthesis serves us much better than a trailing \
> which is ugly and relies on having no trailing whitespace.
> Replace:
>    'hello         ' \
>    'world         '
> With:
>    ('hello        ' +
>     'world        ')
>
>
>
> Raymond
> ______________________________________________


I've started a discussion on python-ideas proposing to allow implicit
concatenation of string variables as well as string literals.
-1 for elimination!

Eoghan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070411/764822d5/attachment.html 

From jason.orendorff at gmail.com  Wed Apr 11 19:03:52 2007
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 11 Apr 2007 13:03:52 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
Message-ID: <bb8868b90704111003g3a7bb808xd3a0376a9d9fe9ad@mail.gmail.com>

On 4/10/07, Raymond Hettinger <python at rcn.com> wrote:
> * Eliminate implicit string concatenation:   "abc" "def"
> in favor of an explicit + operation.  That simplifies
> the grammar just a bit and the compiler already is
> smart enough to do constant fold this operation at
> compile time. [...]

My gut instinct was "whoa, don't change that".  But:

I think this behavior comes from C, where it's useful mainly because
of the preprocessor.  In Python it's not as useful.  And C experience
is not as universal as it once was.  I bet this looks pretty confusing
the first time you run into it.

Sure, I use it, but if it went away, I would type the plus sign.  Not
a problem.  And it would be one less thing for newcomers to learn, and
explicit is better, right?

Maybe this little slice of the complexity budget could be better
invested elsewhere.

-j

From pje at telecommunity.com  Wed Apr 11 19:17:39 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 11 Apr 2007 13:17:39 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <bb8868b90704111003g3a7bb808xd3a0376a9d9fe9ad@mail.gmail.co
 m>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
Message-ID: <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com>

At 01:03 PM 4/11/2007 -0400, Jason Orendorff wrote:
>On 4/10/07, Raymond Hettinger <python at rcn.com> wrote:
> > * Eliminate implicit string concatenation:   "abc" "def"
> > in favor of an explicit + operation.  That simplifies
> > the grammar just a bit and the compiler already is
> > smart enough to do constant fold this operation at
> > compile time. [...]
>
>My gut instinct was "whoa, don't change that".  But:
>
>I think this behavior comes from C, where it's useful mainly because
>of the preprocessor.  In Python it's not as useful.  And C experience
>is not as universal as it once was.  I bet this looks pretty confusing
>the first time you run into it.
>
>Sure, I use it, but if it went away, I would type the plus sign.  Not
>a problem.  And it would be one less thing for newcomers to learn, and
>explicit is better, right?

But there's another Python principle here, I think...  complexity of 
computation should be represented by complexity of syntax.  We don't 
generally like to use properties for expensive computation, or methods for 
simple field access, for example.  Putting in a '+' sign makes the code 
*feel* like there's more computation going on, even if the computation gets 
optimized away.


From jason.orendorff at gmail.com  Wed Apr 11 19:53:22 2007
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 11 Apr 2007 13:53:22 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com>
Message-ID: <bb8868b90704111053k625da756q6b4599f81dbd576a@mail.gmail.com>

On 4/11/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> But there's another Python principle here, I think...  complexity of
> computation should be represented by complexity of syntax.  We don't
> generally like to use properties for expensive computation, or methods for
> simple field access, for example.  Putting in a '+' sign makes the code
> *feel* like there's more computation going on, even if the computation gets
> optimized away.

Hmm.  It sounds like what you're saying is, if we're going to do
string constant folding, then it makes sense to have a special case in
the grammar to reflect that visually.

It's a nice principle, but I wouldn't take it that far.

-j

From steven.bethard at gmail.com  Wed Apr 11 20:00:34 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 11 Apr 2007 12:00:34 -0600
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <evj164$e41$1@sea.gmane.org>
References: <461CF859.6050003@acm.org> <evj164$e41$1@sea.gmane.org>
Message-ID: <d11dcfba0704111100i5d7e1cfcvaae7b7f70705a9cc@mail.gmail.com>

On 4/11/07, Georg Brandl <g.brandl at gmx.net> wrote:
> Talin schrieb:
> > I'm not sure that I am the first to bring this up, but I might as well
> > get it written down.
> >
> > I've never liked using the backslash operator for line continuation, for
> > two reasons:
> >
> > 1) It requires that it be the last character of a line, meaning that you
> > have to insure that there's no trailing whitespace, which can be
> > difficult with some editors.
>
> Then use parentheses. That should be possible almost everywhere.

Yes, please. There's currently no need for the backslash in pretty
much anything but triple-quoted strings. Since they're not needed now,
I see no reason to introduce a replacement for them. If you're going
to propose something, propose removing them.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From bwinton at latte.ca  Wed Apr 11 20:34:15 2007
From: bwinton at latte.ca (Blake Winton)
Date: Wed, 11 Apr 2007 14:34:15 -0400
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>	<20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com>
Message-ID: <461D2A27.8060200@latte.ca>

Phillip J. Eby wrote:
>>> * Eliminate implicit string concatenation:   "abc" "def"
>> Sure, I use it, but if it went away, I would type the plus sign.  Not
>> a problem.  And it would be one less thing for newcomers to learn, and
>> explicit is better, right?
> But there's another Python principle here, I think...  complexity of 
> computation should be represented by complexity of syntax.  We don't 
> generally like to use properties for expensive computation, or methods for 
> simple field access, for example.  Putting in a '+' sign makes the code 
> *feel* like there's more computation going on, even if the computation gets 
> optimized away.

Given that the set of things the compiler can optimize grows at a faster 
rate than the syntax changes to the language, I'm not sure that that's 
really a principle.  Yeah, sure, you don't want to hide, say, a 
reverse-dns-lookup-with-associated-timeout behind the creation of a 
socket (to take a horrible example from Java that's just finished biting 
me), but + isn't really a heavyweight operator, especially if you're 
thinking of (small-ish) integers, and if the compiler optimizes it into 
less than one instruction, then great.

I suppose I see the + case for strings, in particular, as being more 
similar to the + case for numbers than calling methods or properties...

Later,
Blake.

From greg.ewing at canterbury.ac.nz  Thu Apr 12 00:55:55 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 12 Apr 2007 10:55:55 +1200
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <bb8868b90704111053k625da756q6b4599f81dbd576a@mail.gmail.com>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<5.1.1.6.0.20070411131550.02b985c8@sparrow.telecommunity.com>
	<bb8868b90704111053k625da756q6b4599f81dbd576a@mail.gmail.com>
Message-ID: <461D677B.7000604@canterbury.ac.nz>

Jason Orendorff wrote:

> Hmm.  It sounds like what you're saying is, if we're going to do
> string constant folding, then it makes sense to have a special case in
> the grammar to reflect that visually.

I think it's more a matter of wanting to write things
declaratively rather than procedurally. You want to
say "print this long string" rather than "join these
strings together and print the result".

--
Greg

From walter at livinglogic.de  Thu Apr 12 10:10:13 2007
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 12 Apr 2007 10:10:13 +0200
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
 python/branches/p3yk/Lib: io.py test/test_io.py
In-Reply-To: <ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>
References: <20070411010907.226411E400D@bag.python.org>	
	<461C97EC.8040903@livinglogic.de>
	<ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>
Message-ID: <461DE965.3050302@livinglogic.de>

Guido van Rossum wrote:
> On 4/11/07, Walter D?rwald <walter at livinglogic.de> wrote:
>> Would it make sense to make the state of the decoder public, e.g. by
>> adding setstate() and getstate() methods? This would give a cleaner API.
> 
> I've been thinking of the same thing!
> 
> I wonder if it would be possible to return the state as a pair
> (unread, flags) where unread is a (byte) string of unprocessed bytes
> and flags is some other state, with the constraint that in the initial
> state the flags must be zero. Then I can optimize the case where flags
> is returned as zero by subtracting len(unread) from the current
> position and that'd be the correct seek position.

I'd say that bytestream.tell() is the correct position.

Or should seek() return to the last position where the codec was in a
default state without anything buffered? (This can't work for UTF-16,
because the codec almost never is in the default state.)

> I imagine most
> decoders have only very few flags they care about. (The worst might be
> the utf-16 decoder which must have a flag to remember whether it
> already saw a byte order marker, and another indicating the byte
> order. Maybe we'll have to special-case that one, so don't worry too
> much about it.)
> 
>> Should I work on a patch?
> 
> That would be great!

OK, here's the patch: http://bugs.python.org/1698994

The state returned from getstate() should be treated as an opaque value
(e.g. for the buffered incremental codecs it is the buffered string, for
the UTF-16 encoder it's the flag indicating whether a BOM has been
written etc.). The codecs try to return None, if they are in some kind
of default state (e.g. there's nothing buffered).

I'm going to add tests next.

Servus,
   Walter


From python at rcn.com  Thu Apr 12 18:04:40 2007
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 12 Apr 2007 12:04:40 -0400 (EDT)
Subject: [Python-3000] Marshal Version and Pickle Protocols
Message-ID: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net>

Can we drop the old protocols?


Raymond

From guido at python.org  Thu Apr 12 18:57:26 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Apr 2007 09:57:26 -0700
Subject: [Python-3000] Marshal Version and Pickle Protocols
In-Reply-To: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net>
References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net>
Message-ID: <ca471dc20704120957q3afd48b8p840d87519cff1eeb@mail.gmail.com>

On 4/12/07, Raymond Hettinger <python at rcn.com> wrote:
> Can we drop the old protocols?

I'd like to keep supporting the old pickle for data
interchangeability, but default to pickle protocol 2, which has been
supported by Python for several versions.

I'm not aware of Marshal protocol versions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fumanchu at amor.org  Thu Apr 12 19:25:05 2007
From: fumanchu at amor.org (Robert Brewer)
Date: Thu, 12 Apr 2007 10:25:05 -0700
Subject: [Python-3000] Marshal Version and Pickle Protocols
In-Reply-To: <ca471dc20704120957q3afd48b8p840d87519cff1eeb@mail.gmail.com>
Message-ID: <435DF58A933BA74397B42CDEB8145A860AF33CE8@ex9.hostedexchange.local>

Guido van Rossum wrote:
> On 4/12/07, Raymond Hettinger <python at rcn.com> wrote:
> > Can we drop the old protocols?
> 
> I'd like to keep supporting the old pickle for data
> interchangeability, but default to pickle protocol 2, which has been
> supported by Python for several versions.

I still use pickle protocol 0, since it does not output \x00 like
protocols 1 and 2. Some databases either don't provide a way to escape
\x00 in SQL statements, or have nasty bugs when comparing or operating
on string values with \x00 in them.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org

From barry at python.org  Thu Apr 12 19:26:20 2007
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Apr 2007 13:26:20 -0400
Subject: [Python-3000] Marshal Version and Pickle Protocols
In-Reply-To: <ca471dc20704120957q3afd48b8p840d87519cff1eeb@mail.gmail.com>
References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net>
	<ca471dc20704120957q3afd48b8p840d87519cff1eeb@mail.gmail.com>
Message-ID: <FBC9E8F8-DBC1-4144-B921-8F440586CEEF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 12, 2007, at 12:57 PM, Guido van Rossum wrote:

> On 4/12/07, Raymond Hettinger <python at rcn.com> wrote:
>> Can we drop the old protocols?
>
> I'd like to keep supporting the old pickle for data
> interchangeability, but default to pickle protocol 2, which has been
> supported by Python for several versions.

+1, certainly for reading.  I wouldn't care too much if only protocol  
2 is supported for writing new pickles, but I'm probably -0 on  
removing protocol 0 and 1 for writing.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRh5rw3EjvBPtnXfVAQIMuAP+MzSVxI8/23FsK4xb+TqnLqJgF6r6HLXQ
NToWp/WyWgIeAe9eP2g/zyItRF+WxNGG26wtY01sa5JH0d9SUYQ1E1XMug8PAyg+
53apmhd+pJe2MhD5R+KRbwc6i3LloDl/b7eef7HqpFXWdavD2P7OFPakMR+tzGks
pi+agCJWjPM=
=DJq4
-----END PGP SIGNATURE-----

From guido at python.org  Thu Apr 12 19:30:04 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Apr 2007 10:30:04 -0700
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
	python/branches/p3yk/Lib: io.py test/test_io.py
In-Reply-To: <461DE965.3050302@livinglogic.de>
References: <20070411010907.226411E400D@bag.python.org>
	<461C97EC.8040903@livinglogic.de>
	<ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>
	<461DE965.3050302@livinglogic.de>
Message-ID: <ca471dc20704121030l65fab48ct18ed98da330ed372@mail.gmail.com>

On 4/12/07, Walter D?rwald <walter at livinglogic.de> wrote:
> Guido van Rossum wrote:
> > On 4/11/07, Walter D?rwald <walter at livinglogic.de> wrote:
> >> Would it make sense to make the state of the decoder public, e.g. by
> >> adding setstate() and getstate() methods? This would give a cleaner API.
> >
> > I've been thinking of the same thing!
> >
> > I wonder if it would be possible to return the state as a pair
> > (unread, flags) where unread is a (byte) string of unprocessed bytes
> > and flags is some other state, with the constraint that in the initial
> > state the flags must be zero. Then I can optimize the case where flags
> > is returned as zero by subtracting len(unread) from the current
> > position and that'd be the correct seek position.
>
> I'd say that bytestream.tell() is the correct position.
>
> Or should seek() return to the last position where the codec was in a
> default state without anything buffered? (This can't work for UTF-16,
> because the codec almost never is in the default state.)

That was my hope, yes (and I realize that UTF-16 is an exception).
Consider UTF-8 though. If the chunk we read from the byte stream ended
in the middle of a multi-byte character, the codec will have the first
part of that character buffered. In general we want to subtract
buffered data from the byte stream's position when reporting the
position of the text stream. The idea is that if we later seek to the
reported position, we should be reading the same character data. This
can be accomplished in two ways: by backing up the byte stream to the
previous character boundary, and resetting the decoder to neutral; or
by positioning the byte stream to where it was originally and setting
the state of the decoder to what it was before. However, backing up
the byte stream has the advantage that no decoder state needs to be
encoded in the position cookie.

> > I imagine most
> > decoders have only very few flags they care about. (The worst might be
> > the utf-16 decoder which must have a flag to remember whether it
> > already saw a byte order marker, and another indicating the byte
> > order. Maybe we'll have to special-case that one, so don't worry too
> > much about it.)
> >
> >> Should I work on a patch?
> >
> > That would be great!
>
> OK, here's the patch: http://bugs.python.org/1698994
>
> The state returned from getstate() should be treated as an opaque value
> (e.g. for the buffered incremental codecs it is the buffered string, for
> the UTF-16 encoder it's the flag indicating whether a BOM has been
> written etc.). The codecs try to return None, if they are in some kind
> of default state (e.g. there's nothing buffered).

I would like to await completion of those unit tests; there seem to be
some subtle issues. I wonder if setstate() should call self.reset()
first. I'd also like to ask if setstate() could default to "" only if
the argument is None, not if it is empty; I'd like to use it to change
the buffer to be a bytes object. (And yes, I need to maintain more
hacks for that, alas).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 12 19:34:52 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Apr 2007 10:34:52 -0700
Subject: [Python-3000] Marshal Version and Pickle Protocols
In-Reply-To: <FBC9E8F8-DBC1-4144-B921-8F440586CEEF@python.org>
References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net>
	<ca471dc20704120957q3afd48b8p840d87519cff1eeb@mail.gmail.com>
	<FBC9E8F8-DBC1-4144-B921-8F440586CEEF@python.org>
Message-ID: <ca471dc20704121034i188925f7p9882e40f77bc55d0@mail.gmail.com>

On 4/12/07, Barry Warsaw <barry at python.org> wrote:
> On Apr 12, 2007, at 12:57 PM, Guido van Rossum wrote:
>
> > On 4/12/07, Raymond Hettinger <python at rcn.com> wrote:
> >> Can we drop the old protocols?
> >
> > I'd like to keep supporting the old pickle for data
> > interchangeability, but default to pickle protocol 2, which has been
> > supported by Python for several versions.
>
> +1, certainly for reading.

Duh. For reading you don't need to specify a protocol, it finds out
based on what it reads. The default is only necessary for writing.

> I wouldn't care too much if only protocol
> 2 is supported for writing new pickles, but I'm probably -0 on
> removing protocol 0 and 1 for writing.

That is exactly my proposal.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Thu Apr 12 19:38:57 2007
From: barry at python.org (Barry Warsaw)
Date: Thu, 12 Apr 2007 13:38:57 -0400
Subject: [Python-3000] Marshal Version and Pickle Protocols
In-Reply-To: <ca471dc20704121034i188925f7p9882e40f77bc55d0@mail.gmail.com>
References: <20070412120440.BFD12018@ms09.lnh.mail.rcn.net>
	<ca471dc20704120957q3afd48b8p840d87519cff1eeb@mail.gmail.com>
	<FBC9E8F8-DBC1-4144-B921-8F440586CEEF@python.org>
	<ca471dc20704121034i188925f7p9882e40f77bc55d0@mail.gmail.com>
Message-ID: <C1F11EAA-B92A-4A8E-9B65-46287683842B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 12, 2007, at 1:34 PM, Guido van Rossum wrote:

> On 4/12/07, Barry Warsaw <barry at python.org> wrote:
>> On Apr 12, 2007, at 12:57 PM, Guido van Rossum wrote:
>>
>> > On 4/12/07, Raymond Hettinger <python at rcn.com> wrote:
>> >> Can we drop the old protocols?
>> >
>> > I'd like to keep supporting the old pickle for data
>> > interchangeability, but default to pickle protocol 2, which has  
>> been
>> > supported by Python for several versions.
>>
>> +1, certainly for reading.
>
> Duh. For reading you don't need to specify a protocol, it finds out
> based on what it reads. The default is only necessary for writing.

I was actually responding to what I thought Raymond was suggesting,  
but never mind, because I agree with you!

>> I wouldn't care too much if only protocol
>> 2 is supported for writing new pickles, but I'm probably -0 on
>> removing protocol 0 and 1 for writing.
>
> That is exactly my proposal.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRh5usnEjvBPtnXfVAQIWTAQApdlQAEORnFF3G6qswpFECC34EH+IsnVA
kqr1m/PlpLLxAc7PBKtUMuanxHb/i2XjGwkB2d9DOsya9vTg4KNAyDCIC9rFMkz2
B7db40JfyA63tIekZaiOO4llYIr18VFQHxy0Uuo1aQFKQ3vqyDIuS2k0EZ59c+wq
oJzUT2JlqbU=
=hHUb
-----END PGP SIGNATURE-----

From walter at livinglogic.de  Thu Apr 12 21:32:08 2007
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 12 Apr 2007 21:32:08 +0200
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
 python/branches/p3yk/Lib: io.py test/test_io.py
In-Reply-To: <ca471dc20704121030l65fab48ct18ed98da330ed372@mail.gmail.com>
References: <20070411010907.226411E400D@bag.python.org>	
	<461C97EC.8040903@livinglogic.de>	
	<ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>	
	<461DE965.3050302@livinglogic.de>
	<ca471dc20704121030l65fab48ct18ed98da330ed372@mail.gmail.com>
Message-ID: <461E8938.80304@livinglogic.de>

Guido van Rossum wrote:

> On 4/12/07, Walter D?rwald <walter at livinglogic.de> wrote:
>> Guido van Rossum wrote:
>> > On 4/11/07, Walter D?rwald <walter at livinglogic.de> wrote:
>> >> Would it make sense to make the state of the decoder public, e.g. by
>> >> adding setstate() and getstate() methods? This would give a cleaner 
>> API.
>> >
>> > I've been thinking of the same thing!
>> >
>> > I wonder if it would be possible to return the state as a pair
>> > (unread, flags) where unread is a (byte) string of unprocessed bytes
>> > and flags is some other state, with the constraint that in the initial
>> > state the flags must be zero. Then I can optimize the case where flags
>> > is returned as zero by subtracting len(unread) from the current
>> > position and that'd be the correct seek position.
>>
>> I'd say that bytestream.tell() is the correct position.
>>
>> Or should seek() return to the last position where the codec was in a
>> default state without anything buffered? (This can't work for UTF-16,
>> because the codec almost never is in the default state.)
> 
> That was my hope, yes (and I realize that UTF-16 is an exception).

We could designate natural endianness as the default state, but that 
would mean that a codec state can't be transferred to a different 
machine (or we could declare little (or big) endianness to be the 
default state).

> Consider UTF-8 though. If the chunk we read from the byte stream ended
> in the middle of a multi-byte character, the codec will have the first
> part of that character buffered. In general we want to subtract
> buffered data from the byte stream's position when reporting the
> position of the text stream. The idea is that if we later seek to the
> reported position, we should be reading the same character data. This
> can be accomplished in two ways: by backing up the byte stream to the
> previous character boundary, and resetting the decoder to neutral; or
> by positioning the byte stream to where it was originally and setting
> the state of the decoder to what it was before. However, backing up
> the byte stream has the advantage that no decoder state needs to be
> encoded in the position cookie.

OK, so for decoders getstate() should always return a tuple, with the 
first entry being the buffered byte string (or bytes object?) and the 
second being additional state data.

Do we need any specification for encoders?

>> > I imagine most
>> > decoders have only very few flags they care about. (The worst might be
>> > the utf-16 decoder which must have a flag to remember whether it
>> > already saw a byte order marker, and another indicating the byte
>> > order. Maybe we'll have to special-case that one, so don't worry too
>> > much about it.)
>> >
>> >> Should I work on a patch?
>> >
>> > That would be great!
>>
>> OK, here's the patch: http://bugs.python.org/1698994
>>
>> The state returned from getstate() should be treated as an opaque value
>> (e.g. for the buffered incremental codecs it is the buffered string, for
>> the UTF-16 encoder it's the flag indicating whether a BOM has been
>> written etc.). The codecs try to return None, if they are in some kind
>> of default state (e.g. there's nothing buffered).
> 
> I would like to await completion of those unit tests;

The second version of the patch includes the unit tests (and fixes the 
utf-8-sig codec).

> there seem to be
> some subtle issues.

Can you be more concrete?

> I wonder if setstate() should call self.reset()
> first.

Calling reset() and calling setstate() with the initial state should 
have the same effect.

> I'd also like to ask if setstate() could default to "" only if
> the argument is None, not if it is empty; I'd like to use it to change
> the buffer to be a bytes object.

I'd say for Python 3000 it should always be a bytes object. Will this 
interoperate seamlessly with the C part of the codec machinery?

> (And yes, I need to maintain more
> hacks for that, alas).

I'l try to update the patch tomorrow or over the weekend.

Servus,
    Walter

From guido at python.org  Thu Apr 12 23:48:21 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 12 Apr 2007 14:48:21 -0700
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
	python/branches/p3yk/Lib: io.py test/test_io.py
In-Reply-To: <461E8938.80304@livinglogic.de>
References: <20070411010907.226411E400D@bag.python.org>
	<461C97EC.8040903@livinglogic.de>
	<ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>
	<461DE965.3050302@livinglogic.de>
	<ca471dc20704121030l65fab48ct18ed98da330ed372@mail.gmail.com>
	<461E8938.80304@livinglogic.de>
Message-ID: <ca471dc20704121448x42e40fa9h1d97dac4ece0e18c@mail.gmail.com>

> >> > I wonder if it would be possible to return the state as a pair
> >> > (unread, flags) where unread is a (byte) string of unprocessed bytes
> >> > and flags is some other state, with the constraint that in the initial
> >> > state the flags must be zero. Then I can optimize the case where flags
> >> > is returned as zero by subtracting len(unread) from the current
> >> > position and that'd be the correct seek position.
> >>
> >> I'd say that bytestream.tell() is the correct position.
> >>
> >> Or should seek() return to the last position where the codec was in a
> >> default state without anything buffered? (This can't work for UTF-16,
> >> because the codec almost never is in the default state.)
> >
> > That was my hope, yes (and I realize that UTF-16 is an exception).
>
> We could designate natural endianness as the default state, but that
> would mean that a codec state can't be transferred to a different
> machine (or we could declare little (or big) endianness to be the
> default state).

I think it's okay for file positions involving codec states not to be
tranferable between platforms. I think they wouldn't even be
guaranteed between subsequent runs of the same program.

> > Consider UTF-8 though. If the chunk we read from the byte stream ended
> > in the middle of a multi-byte character, the codec will have the first
> > part of that character buffered. In general we want to subtract
> > buffered data from the byte stream's position when reporting the
> > position of the text stream. The idea is that if we later seek to the
> > reported position, we should be reading the same character data. This
> > can be accomplished in two ways: by backing up the byte stream to the
> > previous character boundary, and resetting the decoder to neutral; or
> > by positioning the byte stream to where it was originally and setting
> > the state of the decoder to what it was before. However, backing up
> > the byte stream has the advantage that no decoder state needs to be
> > encoded in the position cookie.
>
> OK, so for decoders getstate() should always return a tuple, with the
> first entry being the buffered byte string (or bytes object?) and the
> second being additional state data.
>
> Do we need any specification for encoders?

I don't need this for encoders at all -- we don't use incremental
encoders, only incremental decoders.

> >> The state returned from getstate() should be treated as an opaque value
> >> (e.g. for the buffered incremental codecs it is the buffered string, for
> >> the UTF-16 encoder it's the flag indicating whether a BOM has been
> >> written etc.). The codecs try to return None, if they are in some kind
> >> of default state (e.g. there's nothing buffered).
> >
> > I would like to await completion of those unit tests;
>
> The second version of the patch includes the unit tests (and fixes the
> utf-8-sig codec).
>
> > there seem to be
> > some subtle issues.
>
> Can you be more concrete?

I think I just meant the str/bytes issue I already mentioned.

> > I wonder if setstate() should call self.reset()
> > first.
>
> Calling reset() and calling setstate() with the initial state should
> have the same effect.

OK, I should do that anyway. (I wasn't aware of reset() until I saw
your patch. ;-)

> > I'd also like to ask if setstate() could default to "" only if
> > the argument is None, not if it is empty; I'd like to use it to change
> > the buffer to be a bytes object.
>
> I'd say for Python 3000 it should always be a bytes object.

Eventually, yes. But right now we're in a world where sometimes there
are bytes and sometimes there are (8-bit) strings -- and I'd like to
get as many tests passing with the new IO library without making it
the default first.

> Will this
> interoperate seamlessly with the C part of the codec machinery?

It should if it uses the buffer API as it should. When I encounter
places where it requires 8-bit strings I'll fix them
opportunistically.

> > (And yes, I need to maintain more
> > hacks for that, alas).
>
> I'l try to update the patch tomorrow or over the weekend.

Thanks!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Fri Apr 13 01:24:59 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 13 Apr 2007 11:24:59 +1200
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <461CF859.6050003@acm.org>
References: <461CF859.6050003@acm.org>
Message-ID: <461EBFCB.6050404@canterbury.ac.nz>

Talin wrote:

> I've never liked using the backslash operator for line continuation

How about regarding a line as a continuation if it's
indented more than the one which started the statement?

> 1) It requires that it be the last character of a line,

That requirement could be removed, couldn't it? There's
no other meaning for a backslash followed by nothing
except whitespace.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Apr 13 01:27:44 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 13 Apr 2007 11:27:44 +1200
Subject: [Python-3000] [Python-ideas] Implicit String Concatenation
In-Reply-To: <BCDF57D3-8555-4230-8ABD-8419A41A8E1C@atlas.st>
References: <ddb3f080704110238q2fde4a17j4635956ed369ad48@mail.gmail.com>
	<fb6fbf560704111315u7b006d14t48d669dcbf21ee6b@mail.gmail.com>
	<BCDF57D3-8555-4230-8ABD-8419A41A8E1C@atlas.st>
Message-ID: <461EC070.9060802@canterbury.ac.nz>

For Py3k, how about changing the definition of triple
quoted strings so that indentation is stripped up
to the level of the line where the string began?

In other words, apply an implicit dedent() to it
in the parser.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Apr 13 02:05:29 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 13 Apr 2007 12:05:29 +1200
Subject: [Python-3000] [Python-Dev] PEP 3118: Extended buffer protocol
	(new version)
In-Reply-To: <e7ba66e40704120740y1b92fe36odeb4f0758c38488c@mail.gmail.com>
References: <eve7kn$1g3$2@sea.gmane.org>
	<e7ba66e40704111721m6c5cb5d0gff7c6d2ea8aeb36e@mail.gmail.com>
	<461D8B5F.2020208@canterbury.ac.nz>
	<e7ba66e40704120740y1b92fe36odeb4f0758c38488c@mail.gmail.com>
Message-ID: <461EC949.5080900@canterbury.ac.nz>

Lisandro Dalcin wrote:
> What about keeping flags it provides instead of the 
> requested ones?

It's hard to guess what kind of info the exporter
might want to keep.

May there could just be a general-purpose void *
field in the bufferinfo for the exporter to use
however it wants.

--
Greg

From collinw at gmail.com  Fri Apr 13 05:16:26 2007
From: collinw at gmail.com (Collin Winter)
Date: Thu, 12 Apr 2007 20:16:26 -0700
Subject: [Python-3000] PEP 344: attaching tracebacks to exceptions + pickle
Message-ID: <43aa6ff70704122016r252ecce7v6bcf39f759036d31@mail.gmail.com>

In 2.x, exceptions are pickle-able; tracebacks, however, are not. If
tracebacks are attached to exceptions in Python 3 (as specified in PEP
344), exceptions will no longer be pickle-able.

There are of course options:

1. Discard an exception's __traceback__ attribute when pickling.

2. Include __traceback__ in the pickle, but in a more limited form.
That is: the hard parts of exceptions (frames, code objects) aren't
fully needed to, say, print out a traceback. The following is all you
need:

class Code(object):
    def __init__(self, code):
        self.co_filename = code.co_filename
        self.co_name = code.co_name

class Frame(object):
    def __init__(self, frame):
        self.f_globals = {"__file__": frame.f_globals["__file__"]}
        self.f_code = Code(frame.f_code)

class Traceback(object):
    def __init__(self, tb):
        self.tb_frame = Frame(tb.tb_frame)
        self.tb_lineno = tb.tb_lineno
        if tb.tb_next is None:
            self.tb_next = None
        else:
            self.tb_next = Traceback(tb.tb_next)

When pickling an exception, __traceback__ could be replaced with the
limited Traceback implementation above.

3. Reject the proposal to attach tracebacks to exceptions. (This is my
least favorite.)

Thoughts?

Collin Winter

From jcarlson at uci.edu  Fri Apr 13 07:28:04 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 12 Apr 2007 22:28:04 -0700
Subject: [Python-3000] PEP 344: attaching tracebacks to exceptions +
	pickle
In-Reply-To: <43aa6ff70704122016r252ecce7v6bcf39f759036d31@mail.gmail.com>
References: <43aa6ff70704122016r252ecce7v6bcf39f759036d31@mail.gmail.com>
Message-ID: <20070412222659.62EF.JCARLSON@uci.edu>


"Collin Winter" <collinw at gmail.com> wrote:
> In 2.x, exceptions are pickle-able; tracebacks, however, are not. If
> tracebacks are attached to exceptions in Python 3 (as specified in PEP
> 344), exceptions will no longer be pickle-able.

I know that idlelib packages tracebacks for transfer across a socket
connection, it may offer more complete pickled traceback information.

 - Josiah


From walter at livinglogic.de  Fri Apr 13 13:49:47 2007
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Fri, 13 Apr 2007 13:49:47 +0200
Subject: [Python-3000] [Python-3000-checkins] r54742 - in
 python/branches/p3yk/Lib: io.py test/test_io.py
In-Reply-To: <ca471dc20704121448x42e40fa9h1d97dac4ece0e18c@mail.gmail.com>
References: <20070411010907.226411E400D@bag.python.org>	
	<461C97EC.8040903@livinglogic.de>	
	<ca471dc20704110747w6c1db20dy1cf1c7cb12a7e7d5@mail.gmail.com>	
	<461DE965.3050302@livinglogic.de>	
	<ca471dc20704121030l65fab48ct18ed98da330ed372@mail.gmail.com>	
	<461E8938.80304@livinglogic.de>
	<ca471dc20704121448x42e40fa9h1d97dac4ece0e18c@mail.gmail.com>
Message-ID: <461F6E5B.2090908@livinglogic.de>

Guido van Rossum wrote:

>> >> > I wonder if it would be possible to return the state as a pair
>> >> > (unread, flags) where unread is a (byte) string of unprocessed bytes
>> >> > and flags is some other state, with the constraint that in the
>> initial
>> >> > state the flags must be zero. Then I can optimize the case where
>> flags
>> >> > is returned as zero by subtracting len(unread) from the current
>> >> > position and that'd be the correct seek position.
>> >>
>> >> I'd say that bytestream.tell() is the correct position.
>> >>
>> >> Or should seek() return to the last position where the codec was in a
>> >> default state without anything buffered? (This can't work for UTF-16,
>> >> because the codec almost never is in the default state.)
>> >
>> > That was my hope, yes (and I realize that UTF-16 is an exception).
>>
>> We could designate natural endianness as the default state, but that
>> would mean that a codec state can't be transferred to a different
>> machine (or we could declare little (or big) endianness to be the
>> default state).
> 
> I think it's okay for file positions involving codec states not to be
> tranferable between platforms. I think they wouldn't even be
> guaranteed between subsequent runs of the same program.

OK, done in the third version of the patch.

>> > Consider UTF-8 though. If the chunk we read from the byte stream ended
>> > in the middle of a multi-byte character, the codec will have the first
>> > part of that character buffered. In general we want to subtract
>> > buffered data from the byte stream's position when reporting the
>> > position of the text stream. The idea is that if we later seek to the
>> > reported position, we should be reading the same character data. This
>> > can be accomplished in two ways: by backing up the byte stream to the
>> > previous character boundary, and resetting the decoder to neutral; or
>> > by positioning the byte stream to where it was originally and setting
>> > the state of the decoder to what it was before. However, backing up
>> > the byte stream has the advantage that no decoder state needs to be
>> > encoded in the position cookie.
>>
>> OK, so for decoders getstate() should always return a tuple, with the
>> first entry being the buffered byte string (or bytes object?) and the
>> second being additional state data.
>>
>> Do we need any specification for encoders?
> 
> I don't need this for encoders at all -- we don't use incremental
> encoders, only incremental decoders.

True for reading, but what about writing?

>> >> The state returned from getstate() should be treated as an opaque
>> value
>> >> (e.g. for the buffered incremental codecs it is the buffered
>> string, for
>> >> the UTF-16 encoder it's the flag indicating whether a BOM has been
>> >> written etc.). The codecs try to return None, if they are in some kind
>> >> of default state (e.g. there's nothing buffered).
>> >
>> > I would like to await completion of those unit tests;
>>
>> The second version of the patch includes the unit tests (and fixes the
>> utf-8-sig codec).
>>
>> > there seem to be
>> > some subtle issues.
>>
>> Can you be more concrete?
> 
> I think I just meant the str/bytes issue I already mentioned.

Since the new version never sets the buffer to an explicit value except
in the constructor this problem should have disappeared.

>> > I wonder if setstate() should call self.reset()
>> > first.
>>
>> Calling reset() and calling setstate() with the initial state should
>> have the same effect.
> 
> OK, I should do that anyway. (I wasn't aware of reset() until I saw
> your patch. ;-)
> 
>> > I'd also like to ask if setstate() could default to "" only if
>> > the argument is None, not if it is empty; I'd like to use it to change
>> > the buffer to be a bytes object.
>>
>> I'd say for Python 3000 it should always be a bytes object.
> 
> Eventually, yes. But right now we're in a world where sometimes there
> are bytes and sometimes there are (8-bit) strings -- and I'd like to
> get as many tests passing with the new IO library without making it
> the default first.

OK.

>> Will this
>> interoperate seamlessly with the C part of the codec machinery?
> 
> It should if it uses the buffer API as it should. When I encounter
> places where it requires 8-bit strings I'll fix them
> opportunistically.
> 
>> > (And yes, I need to maintain more
>> > hacks for that, alas).
>>
>> I'l try to update the patch tomorrow or over the weekend.
> 
> Thanks!

Done. I've also added documentation (The description of the constraints
on the decoder state sounds quite esoteric ;)).

Servus,
   Walter


From paul-python at svensson.org  Fri Apr 13 15:24:49 2007
From: paul-python at svensson.org (Paul Svensson)
Date: Fri, 13 Apr 2007 09:24:49 -0400 (EDT)
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <461EBFCB.6050404@canterbury.ac.nz>
References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz>
Message-ID: <20070413091624.H68198@familjen.svensson.org>


Talin wrote:
> I've never liked using the backslash operator for line continuation

Me neither, but the suggested replacements don't appeal to me either.
Is there any reason we can't just remove them ?
Is backslash continuation still needed anywhere,
or does the grammar now allow for sufficient parenteses
anywhere a line might be 'too long' ?

 	/Paul

From jimjjewett at gmail.com  Fri Apr 13 16:19:33 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 13 Apr 2007 10:19:33 -0400
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <20070413091624.H68198@familjen.svensson.org>
References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz>
	<20070413091624.H68198@familjen.svensson.org>
Message-ID: <fb6fbf560704130719o66e4d25fuedda91b309ff4a79@mail.gmail.com>

On 4/13/07, Paul Svensson <paul-python at svensson.org> wrote:
> Talin wrote:
> > I've never liked using the backslash operator for line continuation

> Me neither, but the suggested replacements don't appeal to me either.
> Is there any reason we can't just remove them ?
> Is backslash continuation still needed anywhere,


    >>> assert True,
    SyntaxError: invalid syntax
    >>> assert True, \
           "ASDF"
    >>>

From steven.bethard at gmail.com  Fri Apr 13 17:33:33 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 13 Apr 2007 09:33:33 -0600
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <fb6fbf560704130719o66e4d25fuedda91b309ff4a79@mail.gmail.com>
References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz>
	<20070413091624.H68198@familjen.svensson.org>
	<fb6fbf560704130719o66e4d25fuedda91b309ff4a79@mail.gmail.com>
Message-ID: <d11dcfba0704130833kf355faehf35a0f3b0023bc62@mail.gmail.com>

On 4/13/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/13/07, Paul Svensson <paul-python at svensson.org> wrote:
> > Talin wrote:
> > > I've never liked using the backslash operator for line continuation
>
> > Me neither, but the suggested replacements don't appeal to me either.
> > Is there any reason we can't just remove them ?
> > Is backslash continuation still needed anywhere,
>
>
>     >>> assert True,
>     SyntaxError: invalid syntax
>     >>> assert True, \
>            "ASDF"
>     >>>

Just like everywhere else, you can use parentheses:

>>> assert True, (
...     'ASDF')


STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From ark-mlist at att.net  Fri Apr 13 21:07:41 2007
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri, 13 Apr 2007 15:07:41 -0400
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <461CF859.6050003@acm.org>
Message-ID: <002b01c77dff$03121e40$6402a8c0@arkdesktop>

> I'm not proposing to remove the feature, however I'd like to see an
> alternative method for declaring statements that cross a line boundary.
> I seem to recall at one point someone suggesting the use of ellipsis,
> which makes a lot of sense to me:
> 
>     sorted_result = partition_lower( input_list, pivot ) ...
>        + pivot ...
>        + partition_upper( input_list, pivot )

I once used a programming language named EFL that had what I think is a nice
convention: If the last token on a line is an operator (i.e. something that
cannot syntactically be the last token of a statement), the next line is
deemed a continuation.  So your example would be able to be written this
way:

	sorted_result = partition_lower( input_list, pivot) +
	   pivot +
	   partition_upper( input_list, pivot )

Interestingly, if I remember correctly, EFL did *not* assume a continuation
for unmatched parentheses -- it looked only at the last token on the line.

For Python, I guess I'd like it if either unmatched parentheses or ending
with an operator were to signal continuation.



From oliphant.travis at ieee.org  Fri Apr 13 23:59:59 2007
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Fri, 13 Apr 2007 15:59:59 -0600
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <461C31A9.5090204@canterbury.ac.nz>
References: <ca471dc20704101053s26e861b4qae8aeda7f39deac1@mail.gmail.com>
	<461C31A9.5090204@canterbury.ac.nz>
Message-ID: <evouh0$dsi$1@sea.gmane.org>

Greg Ewing wrote:
> Guido van Rossum wrote:
> 
>>all PEPs for Py3k are due by April.
> 
> 
> Can we nominate one of our 2.x PEPs for promotion
> to a 3000 PEP?
> 
> If so, I'd like to nominate PEP 335 - Overloadable
> Boolean Operators.
> 

That one would sure improve a lot of trouble we currently have with 
NumPy users not understanding why

x[x>3 and x<5]

doesn't give them what they think it should.


I'm

+1

on this proposal for Python 2.6 and Python 3.0


-Travis


From rrr at ronadam.com  Sat Apr 14 03:17:03 2007
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 13 Apr 2007 20:17:03 -0500
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <002b01c77dff$03121e40$6402a8c0@arkdesktop>
References: <461CF859.6050003@acm.org>
	<002b01c77dff$03121e40$6402a8c0@arkdesktop>
Message-ID: <evpa2d$e6b$1@sea.gmane.org>

Andrew Koenig wrote:
>> I'm not proposing to remove the feature, however I'd like to see an
>> alternative method for declaring statements that cross a line boundary.
>> I seem to recall at one point someone suggesting the use of ellipsis,
>> which makes a lot of sense to me:
>>
>>     sorted_result = partition_lower( input_list, pivot ) ...
>>        + pivot ...
>>        + partition_upper( input_list, pivot )
> 
> I once used a programming language named EFL that had what I think is a nice
> convention: If the last token on a line is an operator (i.e. something that
> cannot syntactically be the last token of a statement), the next line is
> deemed a continuation.  So your example would be able to be written this
> way:
> 
> 	sorted_result = partition_lower( input_list, pivot) +
> 	   pivot +
> 	   partition_upper( input_list, pivot )
> 
> Interestingly, if I remember correctly, EFL did *not* assume a continuation
> for unmatched parentheses -- it looked only at the last token on the line.
> 
> For Python, I guess I'd like it if either unmatched parentheses or ending
> with an operator were to signal continuation.

I tend to do it the other way around because starting off each continued 
line with an operator makes for more readable code.  It's even nicer with 
syntax highlighting.

Using either a backslash or parentheses is fine for me in those situations.

The attractiveness of the ellipsis is it is already used in an interactive 
console to indicate continued lines, but you still need the slash or 
parentheses to get that. ;-)



The only problem I have with the '\' is I don't like it's behavior in 
strings.  I would prefer that a single back slash in regular strings to 
always be an escape introducer and cause a syntax error if it's an invalid 
escape character or it is not at the end of a line, and for it to always be 
a single back slash in raw strings, except at the end of a physical line, 
where it can be a continue line indicator as usual.


 >>> r"\"
   File "<stdin>", line 1
     r"\"
        ^
SyntaxError: EOL while scanning single-quoted string

 >>> "\""
'"'
 >>> r"\""
'\\"'

Shouldn't r"\" result in "\\" ?


 >>> print '1 \ 2'
1 \ 2
 >>> print '1 \\ 2'
1 \ 2
 >>> print '1 \\\ 2'
1 \\ 2
 >>> print '1 \\\\ 2'
1 \\ 2


 >>> """ hello
... world"""
' hello\nworld'

 >>> """ hello \
... world"""
' hello world'

 >>> """hello\
... world"""
'hello\\ \nworld'


I think this does result in silent bugs at times, but more often result in 
the output not being what was expected.

Cheers,
     Ron



From greg.ewing at canterbury.ac.nz  Sat Apr 14 04:18:42 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 14 Apr 2007 14:18:42 +1200
Subject: [Python-3000] Line continuation using ellipsis
In-Reply-To: <d11dcfba0704130833kf355faehf35a0f3b0023bc62@mail.gmail.com>
References: <461CF859.6050003@acm.org> <461EBFCB.6050404@canterbury.ac.nz>
	<20070413091624.H68198@familjen.svensson.org>
	<fb6fbf560704130719o66e4d25fuedda91b309ff4a79@mail.gmail.com>
	<d11dcfba0704130833kf355faehf35a0f3b0023bc62@mail.gmail.com>
Message-ID: <46203A02.2010600@canterbury.ac.nz>

Steven Bethard wrote:

> Just like everywhere else, you can use parentheses:
> 
>>>>assert True, (
> ...     'ASDF')

Yes, but you have to put the opening paren in
a somewhat awkward place to get it to work.

Seems to me it would be much more intuitive to
use what Haskell calls the "offside rule", i.e.
anything indented more than the beginning of a
statement is part of that statement.

The only drawback might be that an indentation
error could lead to something being interpreted
in an unintended way. But I suspect it would
result in a syntax error in most situations.

--
Greg

From talin at acm.org  Sat Apr 14 19:57:28 2007
From: talin at acm.org (Talin)
Date: Sat, 14 Apr 2007 10:57:28 -0700
Subject: [Python-3000] An introduction to ABC's
Message-ID: <46211608.7010604@acm.org>

Part of the reason why I haven't volunteered to write a PEP for ABC's is 
because I don't feel that I understand the various proposals and the 
background discussion well enough. However, it occurs to me that writing 
the rationale section of a PEP can often be the hardest part, and I 
think I understand the issues well enough to write that preface. So 
here's my contribution:

---

In the domain of object-oriented programming, the usage patterns for 
interacting with an object can be divided into two basic categories, 
which are 'invocation' and 'inspection'.

Invocation means interacting with an object by invoking its methods. 
Usually this is combined with polymorphism, so that invoking a given 
method may run different code depending on the type of an object.

Inspection means the ability for external code (outside of the object's 
methods) to examine the type or properties of that object, and make 
decisions on how to treat that object based on that information.

Both usage patterns serve the same general end, which is to be able to 
support the processing of diverse and potentially novel objects in a 
uniform way, but at the same time allowing processing decisions to be 
customized for each different type of object.

In classical OOP theory, invocation is the preferred usage pattern, and 
inspection is actively discouraged, being considered a relic of an 
earlier, procedural programming style. However, in practice this view is 
simply too dogmatic and inflexible, and leads to a kind of design 
rigidity that is very much at odds with the dynamic nature of a language 
like Python.

In particular, there is often a need to process objects in a way that 
wasn't anticipated by the creator of the object class. It is not always 
the best solution to build in to every object methods that satisfy the 
needs of every possible user of that object. Moreover, there are many 
powerful dispatch philosophies that are in direct contrast to the 
classic OOP requirement of behavior being strictly encapsulated within 
an object, examples being rule or pattern-match driven logic.

On the the other hand, one of the criticisms of inspection by classic 
OOP theorists is the lack of formalisms and the ad hoc nature of what is 
being inspected. In a language such as Python, in which almost any 
aspect of an object can be reflected and directly accessed by external 
code, there are many different ways to test whether an object conforms 
to a particular protocol or not. For example, if asking 'is this object 
a mutable sequence container?', one can look for a base class of 'list', 
or one can look for a method named '__getitem__'. But note that although 
these tests may seem obvious, neither of them are correct, as one 
generates false negatives, and the other false positives.

The generally agreed-upon remedy is to standardize the tests, and group 
them into a formal arrangement. This is most easily done by associating 
with each class a set of standard testable properties, either via the 
inheritance mechanism or some other means. Each test carries with it a 
set of promises: it contains a promise about the general behavior of the 
class, and a promise as to what other class methods will be available.

This PEP proposes a particular strategy for organizing these tests known 
as Abstract Base Classes, or ABC. ABCs are simply Python classes that 
are added into an object's inheritance tree to signal certain features 
of that object to an external inspector. Tests are done using 
isinstance(), and the presence of a particular ABC means that the test 
has passed.

Like all other things in Python, these promises are in the nature of a 
gentlemen's agreement - which means that the language does not attempt 
to enforce that these promises are kept.


From martin at v.loewis.de  Sun Apr 15 11:46:50 2007
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Apr 2007 11:46:50 +0200
Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source encoding
Message-ID: <4621F48A.1090308@v.loewis.de>

I'd like to submit the following PEP for Python 3.

Regards,
Martin


PEP: xxx
Title: Using UTF-8 as the default source encoding
Version: $Revision $
Last-Modified: $Date $
Author: Martin v. L?wis <martin at v.loewis.de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 15-Apr-2007
Python-Version: 3.0
Post-History:


Specification
=============

This PEP proposes to change the default source encoding from ASCII to
UTF-8. Support for alternative source encodings [#pep263]_ continues to
exist; an explicit encoding declaration takes precedence over the
default.


A Bit of History
================

In Python 1, the source encoding was unspecified, except that the
source encoding had to be a superset of the system's basic execution
character set (i.e. an ASCII superset, on most systems).  The source
encoding was only relevant for the lexis itself (bytes representing
letters for keywords, identifiers, punctuation, line breaks, etc).
The contents of a string literal was copied literally from the file
on source.

In Python 2.0, the source encoding changed to Latin-1 as a side effect
of introducing Unicode. For Unicode string literals, the characters
were still copied literally from the source file, but widened on a
character-by-character basis. As Unicode gives a fixed interpretation
to code points, this algorithm effectively fixed a source encoding, at
least for files containing non-ASCII characters in Unicode literals.

PEP 263 identified the problem that you can use only those Unicode
characters in a Unicode literal which are also in Latin-1, and
introduced a syntax for declaring the source encoding. If no source
encoding was given, the default should be ASCII. For compatibility
with Python 2.0 and 2.1, files were interpreted as Latin-1 for a
transitional period. This transition ended with Python 2.5, which
gives an error if non-ASCII characters are encountered and no source
encoding is declared.

Rationale
=========

With PEP 263, using arbitrary non-ASCII characters in a Python file is
possible, but tedious. One has to explicitly add an encoding
declaration. Even though some editors (like IDLE and Emacs) support
the declarations of PEP 263, many editors still do not (and never
will); users have to explicitly adjust the encoding which the editor
assumes on a file-by-file basis.

When the default encoding is changed to UTF-8, adding non-ASCII text
to Python files becomes easier and more portable: On some systems,
editors will automatically choose UTF-8 when saving text (e.g. on Unix
systems where the locale uses UTF-8). On other systems, editors will
guess the encoding when reading the file, and UTF-8 is easy to
guess. Yet other editors support associating a default encoding with a
file extension, allowing users to associate .py with UTF-8.

For Python 2, an important reason for using non-UTF-8 encodings was
that byte string literals would be in the source encoding at run-time,
allowing then to output them to a file or render them to the user
as-is. With Python 3, all strings will be Unicode strings, so the
original encoding of the source will have no impact at run-time.

Implementation
==============

The parser needs to be changed to accept bytes > 127 if no source
encoding is specified; instead of giving an error, it needs to check
that the bytes are well-formed UTF-8 (decoding is not necessary,
as the parser converts all source code to UTF-8, anyway).

IDLE needs to be changed to use UTF-8 as the default encoding.


References
==========

.. [#pep263]
   http://www.python.org/dev/peps/pep-0263/



Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From g.brandl at gmx.net  Sun Apr 15 18:53:25 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 15 Apr 2007 18:53:25 +0200
Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source
	encoding
In-Reply-To: <4621F48A.1090308@v.loewis.de>
References: <4621F48A.1090308@v.loewis.de>
Message-ID: <evtla5$6hf$1@sea.gmane.org>

Martin v. L?wis schrieb:
> I'd like to submit the following PEP for Python 3.
> 
> Regards,
> Martin
> 
> 
> PEP: xxx
> Title: Using UTF-8 as the default source encoding
> Version: $Revision $
> Last-Modified: $Date $
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 15-Apr-2007
> Python-Version: 3.0
> Post-History:
> 
> 
> Specification
> =============
> 
> This PEP proposes to change the default source encoding from ASCII to
> UTF-8. Support for alternative source encodings [#pep263]_ continues to
> exist; an explicit encoding declaration takes precedence over the
> default.

+1 in general. Do you want to add a sentence or two about the infamous
UTF-8 "BOM"?


Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sun Apr 15 19:20:19 2007
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Apr 2007 19:20:19 +0200
Subject: [Python-3000] Draft PEP: Using UTF-8 as the default
	source	encoding
In-Reply-To: <evtla5$6hf$1@sea.gmane.org>
References: <4621F48A.1090308@v.loewis.de> <evtla5$6hf$1@sea.gmane.org>
Message-ID: <46225ED3.9010809@v.loewis.de>

> +1 in general. Do you want to add a sentence or two about the infamous
> UTF-8 "BOM"?

Can do. Strictly speaking, it's covered by PEP 263 already, which makes
presence of the BOM equivalent to an encoding declaration.

Regards,
Martin

From nnorwitz at gmail.com  Mon Apr 16 09:29:30 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 16 Apr 2007 00:29:30 -0700
Subject: [Python-3000] removing functions from string module
Message-ID: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>

Can we remove all the functions from the string module that are also
existing methods on string?  Then we can figure out what to do with
what's left over?

I'm tempted to ask the same thing about the types module.

TOOWTDI.

n

From nevillegrech at hotmail.com  Mon Apr 16 13:12:11 2007
From: nevillegrech at hotmail.com (Neville Grech)
Date: Mon, 16 Apr 2007 13:12:11 +0200
Subject: [Python-3000] Empty set and empty dictionary
Message-ID: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>

This is just some syntax sugar:

Since set literals will change to for example {1,2,3} from set([1,2,3])  and set comprehensions will be specified inside {} I feel that {} will be more naturally associated with sets than dicts (or at least as much).

What if the empty set literal is changed to {} and an empty dict literal changed to {:}. Performing the conversion automatically wouldn't be so complex and also un-ambiguous. The hardest thing to change would be the mentality then.

i.e:
{} ::= set([])
{1,2,3} ::= set([1,2,3])
{x for x in y} ::= set(x for x in y)
{:} ::= dict()

{a:b, h:j} is a dict since it contains colons.

I hope this isn't a stupid suggestion (I'm new here).

-Neville
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070416/6cf865e9/attachment.html 

From guido at python.org  Mon Apr 16 18:33:45 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Apr 2007 09:33:45 -0700
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
Message-ID: <ca471dc20704160933i44cb674by20de6c2922484051@mail.gmail.com>

It's not stupid, but it's been brought up before (exactly like you
propose) and rejected, on the basis that dicts are still much more
common than sets in most code.

On 4/16/07, Neville Grech <nevillegrech at hotmail.com> wrote:
>
>
> This is just some syntax sugar:
>
> Since set literals will change to for example {1,2,3} from set([1,2,3])  and
> set comprehensions will be specified inside {} I feel that {} will be more
> naturally associated with sets than dicts (or at least as much).
>
> What if the empty set literal is changed to {} and an empty dict literal
> changed to {:}. Performing the conversion automatically wouldn't be so
> complex and also un-ambiguous. The hardest thing to change would be the
> mentality then.
>
> i.e:
> {} ::= set([])
> {1,2,3} ::= set([1,2,3])
> {x for x in y} ::= set(x for x in y)
> {:} ::= dict()
>
> {a:b, h:j} is a dict since it contains colons.
>
> I hope this isn't a stupid suggestion (I'm new here).
>
> -Neville
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 16 18:34:12 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Apr 2007 09:34:12 -0700
Subject: [Python-3000] removing functions from string module
In-Reply-To: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
References: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
Message-ID: <ca471dc20704160934s6cc020d7id41221f3f0908803@mail.gmail.com>

On 4/16/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> Can we remove all the functions from the string module that are also
> existing methods on string?  Then we can figure out what to do with
> what's left over?

+1

> I'm tempted to ask the same thing about the types module.

+1

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Mon Apr 16 18:16:35 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 16 Apr 2007 09:16:35 -0700
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
Message-ID: <20070416090725.6311.JCARLSON@uci.edu>


"Neville Grech" <nevillegrech at hotmail.com> wrote:
> This is just some syntax sugar:
> 
> Since set literals will change to for example {1,2,3} from set([1,2,3])
>  and set comprehensions will be specified inside {} I feel that {}
> will be more naturally associated with sets than dicts (or at least
> as much).
> 
> What if the empty set literal is changed to {} and an empty dict
> literal changed to {:}. Performing the conversion automatically
> wouldn't be so complex and also un-ambiguous. The hardest thing to
> change would be the mentality then.

Because the mentality would be so difficult to change, and because sets
are still used less frequently than dicts, it doesn't make sense to
alter dictionary syntax (that people have used for over a decade) for
sets (which don't yet have a literal syntax in a released Python).

Your heart is in the right place, but your logic is jumping the gun by
a while.  Offer the suggestion after the syntax has been available in a
released Python for at least a year, when everyone has a chance to use
set syntax.

 - Josiah


From brett at python.org  Mon Apr 16 21:00:22 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Apr 2007 12:00:22 -0700
Subject: [Python-3000] removing functions from string module
In-Reply-To: <ca471dc20704160934s6cc020d7id41221f3f0908803@mail.gmail.com>
References: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
	<ca471dc20704160934s6cc020d7id41221f3f0908803@mail.gmail.com>
Message-ID: <bbaeab100704161200m4c80e727i778d1af5393a262c@mail.gmail.com>

On 4/16/07, Guido van Rossum <guido at python.org> wrote:
>
> On 4/16/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> > Can we remove all the functions from the string module that are also
> > existing methods on string?  Then we can figure out what to do with
> > what's left over?
>
> +1
>
> > I'm tempted to ask the same thing about the types module.
>
> +1



What Guido said.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070416/e43e5311/attachment.html 

From dillonco at comcast.net  Mon Apr 16 19:08:23 2007
From: dillonco at comcast.net (DillonCo)
Date: Mon, 16 Apr 2007 13:08:23 -0400
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
Message-ID: <200704161308.23924.dillonco@comcast.net>

On Monday 16 April 2007, Neville Grech wrote:
> Since set literals will change to for example {1,2,3} from set([1,2,3]) 
> and set comprehensions will be specified inside {} I feel that {} will be
> more naturally associated with sets than dicts (or at least as much).

While the topic of set literals is around, I figure I ought to ask something 
that's been on my mind:

Why not use "<>" for sets?  As far as I can tell, they're only currently used 
in "expr '<' | '>' " contexts, so it'd be easy to distinguish usages. (I 
think.  The "[]" operators are used similarly.)

I know the traditional mathematical notation uses "{}" and having it 
overloaded isn't a big deal; I'm simply curious. 


From nevillegrech at hotmail.com  Mon Apr 16 21:49:17 2007
From: nevillegrech at hotmail.com (Neville Grech)
Date: Mon, 16 Apr 2007 21:49:17 +0200
Subject: [Python-3000] Empty set and empty dictionary
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
Message-ID: <BAY129-DAV168DDB88798B07E84F7B0EC5520@phx.gbl>

That would probably be very difficult to parse because it introduces 
ambiguities. Take a look at this situation:

<8,True, 3>=4, 5 >

----- Original Message ----- 
From: "DillonCo" <dillonco at comcast.net>
To: <python-3000 at python.org>
Sent: Monday, April 16, 2007 7:08 PM
Subject: Re: [Python-3000] Empty set and empty dictionary


> On Monday 16 April 2007, Neville Grech wrote:
>> Since set literals will change to for example {1,2,3} from set([1,2,3])
>> and set comprehensions will be specified inside {} I feel that {} will be
>> more naturally associated with sets than dicts (or at least as much).
>
> While the topic of set literals is around, I figure I ought to ask 
> something
> that's been on my mind:
>
> Why not use "<>" for sets?  As far as I can tell, they're only currently 
> used
> in "expr '<' | '>' " contexts, so it'd be easy to distinguish usages. (I
> think.  The "[]" operators are used similarly.)
>
> I know the traditional mathematical notation uses "{}" and having it
> overloaded isn't a big deal; I'm simply curious.
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-3000/nevillegrech%40hotmail.com
> 


From greg.ewing at canterbury.ac.nz  Tue Apr 17 02:22:29 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Apr 2007 12:22:29 +1200
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <200704161308.23924.dillonco@comcast.net>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
Message-ID: <46241345.6070605@canterbury.ac.nz>

DillonCo wrote:

> Why not use "<>" for sets?

Some possible reasons:

* It would look ugly

* There could be visual confusion with comparison operators

* There could be parsing difficulties distinguishing
   nested set bracketing from << and >> operators

--
Greg

From eoghan at qatano.com  Tue Apr 17 02:31:56 2007
From: eoghan at qatano.com (Eoghan Murray)
Date: Tue, 17 Apr 2007 01:31:56 +0100
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <46241345.6070605@canterbury.ac.nz>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
	<46241345.6070605@canterbury.ac.nz>
Message-ID: <ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>

I had another idea on this theme.. going by unicode and raw string literals,
how about
s[1, 2, 3, 4]
for a set?

Eoghan

On 17/04/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> DillonCo wrote:
>
> > Why not use "<>" for sets?
>
> Some possible reasons:
>
> * It would look ugly
>
> * There could be visual confusion with comparison operators
>
> * There could be parsing difficulties distinguishing
>    nested set bracketing from << and >> operators
>
> --
> Greg
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/eoghan%40qatano.com
>



-- 
http://qatano.com/
http://eoghan.qatano.org/
+353 86 236 1190
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070417/472d19b9/attachment-0001.html 

From eoghan at qatano.com  Tue Apr 17 02:35:49 2007
From: eoghan at qatano.com (Eoghan Murray)
Date: Tue, 17 Apr 2007 01:35:49 +0100
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
	<46241345.6070605@canterbury.ac.nz>
	<ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
Message-ID: <ddb3f080704161735l5ac10614n44ec8c2dfcbd5fd2@mail.gmail.com>

s[1] being a list index of the list 's'
s[1,] being a set literal with one element, '1'

On 17/04/07, Eoghan Murray <eoghan at qatano.com> wrote:
>
> I had another idea on this theme.. going by unicode and raw string
> literals, how about
> s[1, 2, 3, 4]
> for a set?
>
> Eoghan
>
> On 17/04/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> >
> > DillonCo wrote:
> >
> > > Why not use "<>" for sets?
> >
> > Some possible reasons:
> >
> > * It would look ugly
> >
> > * There could be visual confusion with comparison operators
> >
> > * There could be parsing difficulties distinguishing
> >    nested set bracketing from << and >> operators
> >
> > --
> > Greg
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-3000/eoghan%40qatano.com
> >
>
>
>
> --
> http://qatano.com/
> http://eoghan.qatano.org/
> +353 86 236 1190




-- 
http://qatano.com/
http://eoghan.qatano.org/
+353 86 236 1190
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070417/4b9dc874/attachment.htm 

From greg.ewing at canterbury.ac.nz  Tue Apr 17 02:36:01 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Apr 2007 12:36:01 +1200
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
	<46241345.6070605@canterbury.ac.nz>
	<ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
Message-ID: <46241671.10108@canterbury.ac.nz>

Eoghan Murray wrote:
> how about
> s[1, 2, 3, 4]
> for a set?

Already meaningful syntax. Try this:

   d = {}
   d[1, 2, 3, 4] = 'spam'
   print d

--
Greg

From cvrebert at gmail.com  Tue Apr 17 03:08:22 2007
From: cvrebert at gmail.com (Chris Rebert)
Date: Mon, 16 Apr 2007 18:08:22 -0700
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>	<200704161308.23924.dillonco@comcast.net>	<46241345.6070605@canterbury.ac.nz>
	<ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
Message-ID: <46241E06.2010907@gmail.com>

I think someone has probably proposed this before, but why not use "{,}" 
as the empty set literal? It's somewhat analogous to the 1-element tuple 
syntax and fairly similar to the syntax that the original set PEP 
proposed ("{-}"). From what I understand, {-} was nixed for being too 
hard to parse because the minus sign could be the start of an 
expression. I don't think this would apply to the comma as it's 
basically used as an expression separator; however I'm no expert on 
Python's grammar and parser, so I can't be sure.

Thoughts?

- Chris Rebert

Eoghan Murray wrote:
> I had another idea on this theme.. going by unicode and raw string 
> literals,
> how about
> s[1, 2, 3, 4]
> for a set?
> 
> Eoghan
> 
> On 17/04/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>>
>> DillonCo wrote:
>>
>> > Why not use "<>" for sets?
>>
>> Some possible reasons:
>>
>> * It would look ugly
>>
>> * There could be visual confusion with comparison operators
>>
>> * There could be parsing difficulties distinguishing
>>    nested set bracketing from << and >> operators
>>
>> -- 
>> Greg
>> _______________________________________________
>> Python-3000 mailing list
>> Python-3000 at python.org
>> http://mail.python.org/mailman/listinfo/python-3000
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-3000/eoghan%40qatano.com
>>
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/cvrebert%40gmail.com

From mike.klaas at gmail.com  Tue Apr 17 03:27:02 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Mon, 16 Apr 2007 18:27:02 -0700
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <46241E06.2010907@gmail.com>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
	<46241345.6070605@canterbury.ac.nz>
	<ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
	<46241E06.2010907@gmail.com>
Message-ID: <3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com>

On 4/16/07, Chris Rebert <cvrebert at gmail.com> wrote:
> I think someone has probably proposed this before, but why not use "{,}"
> as the empty set literal?

If you're sure, did you look for it first?

http://mail.python.org/pipermail/python-3000/2006-April/001400.html

-Mike

From cvrebert at gmail.com  Tue Apr 17 04:03:35 2007
From: cvrebert at gmail.com (Chris Rebert)
Date: Mon, 16 Apr 2007 19:03:35 -0700
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>	
	<200704161308.23924.dillonco@comcast.net>	
	<46241345.6070605@canterbury.ac.nz>	
	<ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>	
	<46241E06.2010907@gmail.com>
	<3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com>
Message-ID: <46242AF7.6060303@gmail.com>

Ah, that must have been it! Never mind then ;-).

- Chris Rebert

Mike Klaas wrote:
> On 4/16/07, Chris Rebert <cvrebert at gmail.com> wrote:
>> I think someone has probably proposed this before, but why not use "{,}"
>> as the empty set literal?
> 
> If you're sure, did you look for it first?
> 
> http://mail.python.org/pipermail/python-3000/2006-April/001400.html
> 
> -Mike

From guido at python.org  Tue Apr 17 05:24:09 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Apr 2007 20:24:09 -0700
Subject: [Python-3000] Empty set and empty dictionary
In-Reply-To: <46242AF7.6060303@gmail.com>
References: <BAY129-DAV9447185D811A70A0CFE8AC5520@phx.gbl>
	<200704161308.23924.dillonco@comcast.net>
	<46241345.6070605@canterbury.ac.nz>
	<ddb3f080704161731x69a1b22m1cbbee1523a2a4df@mail.gmail.com>
	<46241E06.2010907@gmail.com>
	<3d2ce8cb0704161827p48acdd5avffe157112999d830@mail.gmail.com>
	<46242AF7.6060303@gmail.com>
Message-ID: <ca471dc20704162024n6fbabfcai1208ba2ef0759f62@mail.gmail.com>

Can we move this thread out of the python-3000? This issue has already
been decided and is unlikely to be reopened.

On 4/16/07, Chris Rebert <cvrebert at gmail.com> wrote:
> Ah, that must have been it! Never mind then ;-).
>
> - Chris Rebert
>
> Mike Klaas wrote:
> > On 4/16/07, Chris Rebert <cvrebert at gmail.com> wrote:
> >> I think someone has probably proposed this before, but why not use "{,}"
> >> as the empty set literal?
> >
> > If you're sure, did you look for it first?
> >
> > http://mail.python.org/pipermail/python-3000/2006-April/001400.html
> >
> > -Mike
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From oliphant.travis at ieee.org  Tue Apr 17 06:39:07 2007
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Mon, 16 Apr 2007 22:39:07 -0600
Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source
	encoding
In-Reply-To: <4621F48A.1090308@v.loewis.de>
References: <4621F48A.1090308@v.loewis.de>
Message-ID: <f01iip$be9$1@sea.gmane.org>

Martin v. L?wis wrote:
> I'd like to submit the following PEP for Python 3.
> 
> Regards,
> Martin
> 
> 
> PEP: xxx
> Title: Using UTF-8 as the default source encoding
> Version: $Revision $
> Last-Modified: $Date $
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 15-Apr-2007
> Python-Version: 3.0
> Post-History:
> 

+1


From humberto at digi.com.br  Tue Apr 17 07:49:49 2007
From: humberto at digi.com.br (Humberto Diogenes)
Date: Tue, 17 Apr 2007 02:49:49 -0300
Subject: [Python-3000] Reminder: Py3k PEPs due by April
In-Reply-To: <461C2AC7.4090908@canterbury.ac.nz>
References: <20070410144210.BEU81714@ms09.lnh.mail.rcn.net>
	<461C2AC7.4090908@canterbury.ac.nz>
Message-ID: <605D1AB0-DFDC-43FD-9283-527AC6149B41@digi.com.br>


On 10/04/2007, at 21:24, Greg Ewing wrote:

> Raymond Hettinger wrote:
>
>> Here's that I think might not need a PEP:
>>
>> * Eliminate implicit string concatenation:   "abc" "def"
>> in favor of an explicit + operation.
>
> -0.707. I've used languages where the only way of
> splitting a string over lines is to use concatenation,
> and it's tedious.
>
>> When there are multi-line concats, I think
>> the parenthesis serves us much better than a trailing \
>
> Which can be done perfectly well as things are:
>
>      ('hello        '
>       'world        ')

   Yes, but this causes subtle bugs when you forget a comma: what  
should be the second parameter ends up in the first one. You run the  
program with no syntax errors and later you'll find that e-mails are  
being sent to 'me at domain.comHello!' or something like that.


As they say: just my 2 cents.
--
Humberto Di?genes
http://humberto.digi.com.br


From humberto at digi.com.br  Tue Apr 17 07:35:26 2007
From: humberto at digi.com.br (Humberto Diogenes)
Date: Tue, 17 Apr 2007 02:35:26 -0300
Subject: [Python-3000] i18n prefix
Message-ID: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br>

Hi, folks!!

   This is my first post to python-3000. After Guido PEPs-deadline  
announcement, I decided to discuss an old idea while it still has  
some chance of being accepted. :)

   No, I do not want i18n identifiers. What I'd like to propose is a  
standard syntax for i18n strings. Something like:
   print i"my string"

   My points are:
   * i18n is everywhere (or is it the other way around? :-)
   * Python has a tradition of incorporating good practices
   * _ = gettext.gettext; print _("my string") isn't very nice

   So, do you think that something like i"my string" would be better  
than _("my string")? Would it add unnecessary complexity to Python?

--
Humberto Di?genes
http://humberto.digi.com.br

From greg.ewing at canterbury.ac.nz  Tue Apr 17 08:44:56 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 17 Apr 2007 18:44:56 +1200
Subject: [Python-3000] i18n prefix
In-Reply-To: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br>
References: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br>
Message-ID: <46246CE8.7090904@canterbury.ac.nz>

Humberto Diogenes wrote:

>    * _ = gettext.gettext; print _("my string") isn't very nice

If you don't like _, you're free to use something
else, e.g.

   from gettext import gettext as i

   i("This is a an i18m string")

which is not very far from an i prefix.

--
Greg

From talin at acm.org  Tue Apr 17 09:18:05 2007
From: talin at acm.org (Talin)
Date: Tue, 17 Apr 2007 00:18:05 -0700
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <20070416150746.GE17621@crater.logilab.fr>
References: <46211608.7010604@acm.org>
	<20070416150746.GE17621@crater.logilab.fr>
Message-ID: <462474AD.2050807@acm.org>

Aur?lien Camp?as wrote:
 > On Sat, Apr 14, 2007 at 10:57:28AM -0700, Talin wrote:
 >> Part of the reason why I haven't volunteered to write a PEP for 
ABC's is because I don't feel that I understand the various proposals 
and the background discussion well enough. However, it occurs to me that 
writing the rationale section of a PEP can often be the hardest part, 
and I think I understand the issues well enough to write that preface. 
So here's my contribution:
 >>
 >> ---
 >>
 >> In the domain of object-oriented programming, the usage patterns for 
interacting with an object can be divided into two basic categories, 
which are 'invocation' and 'inspection'.
 >
 > I'd put 'definition' there. Especially in a language like Python
 > (dynamic isn't it ?), where additional definitions can be introduced
 > at run-time, or existing definitions be subtituted to existing ones
 > (aka 'monkey-patching'). At least where I work, this is an important
 > category. Java/C++ monkeys can't really understand the importance of
 > this, tough.
 >
 >> Invocation means interacting with an object by invoking its methods. 
Usually this is combined with polymorphism, so that invoking a given 
method may run different code depending on the type of an object.
 >>
 >> Inspection means the ability for external code (outside of the 
object's methods) to examine the type or properties of that object, and 
make decisions on how to treat that object based on that information.
 >>
 >> Both usage patterns serve the same general end, which is to be able 
to support the processing of diverse and potentially novel objects in a 
uniform way, but at the same time allowing processing decisions to be 
customized for each different type of object.
 >>
 >> In classical OOP theory, invocation is the preferred usage pattern,
 >> and
 >
 > uh, well, but what is classical OOP theory ? What is defined by
 > SmallTalk, C++ and Java ? Feh !

Since my knowledge of programming is almost entirely self-taught, I 
don't have much experience as to what are the formalisms that are being 
taught to young programmers today. However, based on my interactions 
with programmers who have gone through a formal CS education, I can 
infer the existence of a "school of thought" of object decomposition and 
software architecture that follows rules very similar to what I have 
described. What I lack, however, is bibliographic references for it.

 >> inspection is actively discouraged, being considered a relic of an 
earlier, procedural programming style. However, in practice this view is 
simply too dogmatic and inflexible, and leads to a kind of design 
rigidity that is very much at odds with the dynamic nature of a language 
like Python.
 >
 > imho it's not a too dogmatic & inflexible view (it is quite correct
 > that inspection-based dispatch, aka duck-typing (I call it
 > {L,S,F}uck-typing sometimes, in anger ...), is an evil hack that
 > should avoided from production software, for obious maintenance
 > problems) ; but it is very useful in the context of rapid, interactive
 > developement, when one has to explore the shape of things to come ...

I think it's important to understand that 'duck-typing' is not the same 
as inspection-based dispatch - it is an entirely orthogonal concept. 
'Duck-typing' simply means that two classes can be polymorphic without 
having a common base class or interface - all they have to do is support 
the same methods. If I want to create a new kind of widget, I don't have 
to inherit from the base Widget class, all I have to do is make sure 
that my custom widget class supports all of the proper Widget methods. 
This is not possible in Java or C++.

Duck typing can apply to both invocation ('use this object as a widget') 
and inspection ('is this object a widget?'). That is why I claim it is 
orthogonal to the issue of inspection.

And I don't think that inspection-based dispatch is an evil hack, but it 
certainly can be misused. Inspection-based dispatch can be something as 
simple as "if this is a list, do A, else do B". The evil comes when you 
use it in a way that lacks discipline - what you might call 'spaghetti 
inspection'. But it's hard to use proper discipline, because we haven't 
yet defined what the proper discipline *is* - we haven't set the 
standard. That is what the ABCs stuff is attempting to establish - to 
make it possible to use inspection in a way that is 'not evil' because 
it conforms to a well-understood pattern.

 >> In particular, there is often a need to process objects in a way 
that wasn't anticipated by the creator of the object class. It is not 
always the best solution to build in to every object methods that satisfy
 >> the
 >
 > it is just not possible
 >
 >> needs of every possible user of that object. Moreover, there are 
many powerful dispatch philosophies that are in direct contrast to the 
classic OOP requirement of behavior being strictly encapsulated
 >> within
 >
 > 'classic OOP requirement' is clearly the dogma there :)
 > Python developpers should never feel obliged towards it ; especially
 > since Python owes (almost) nothing to this dogma.
 >
 >> an object, examples being rule or pattern-match driven logic.
 >>
 >> On the the other hand, one of the criticisms of inspection by 
classic OOP theorists is the lack of formalisms and the ad hoc nature of 
what is being inspected. In a language such as Python, in which almost any
 >
 > The school of static verification of simple program properties will
 > never cease to come after us, programmers, to try to sell its highly
 > unpractical theoretical belief about ways to ensure programs
 > correctness ; let's not fall in the traps of their evil ways^H^H^H
 > hidden academic agenda ...
 >
 >> aspect of an object can be reflected and directly accessed by 
external code, there are many different ways to test whether an object 
conforms to a particular protocol or not. For example, if asking 'is 
this object a mutable sequence container?', one can look for a base 
class of 'list', or one can look for a method named '__getitem__'. But 
note that although these tests may seem obvious, neither of them are 
correct, as one generates false negatives, and the other false positives.
 >>
 >> The generally agreed-upon remedy is to standardize the tests, and 
group them into a formal arrangement. This is most easily done by 
associating with each class a set of standard testable properties, 
either via the inheritance mechanism or some other means. Each test 
carries with it a set of promises: it contains a promise about the 
general behavior of the class, and a promise as to what other class 
methods will be available.
 >>
 >> This PEP proposes a particular strategy for organizing these tests 
known as Abstract Base Classes, or ABC. ABCs are simply Python classes 
that are added into an object's inheritance tree to signal certain 
features of that object to an external inspector. Tests are done using 
isinstance(), and the presence of a particular ABC means that the test 
has passed.
 >>
 >> Like all other things in Python, these promises are in the nature of 
a gentlemen's agreement - which means that the language does not attempt 
to enforce that these promises are kept.
 >
 > bleh
 >
 > What's wrong with associating behaviour to methods ? Currently, the
 > wrongness lies in the strong coupling between class definition and
 > method definition. Hence you (also in the name of others) propose an
 > ABC, for instance, in oder to assess that object conforms to a
 > protocol. But suppose you work with a class whose definition you don't
 > control (because it's a piece of Plone/Archetype for instance) and you
 > want to make it indexable. With ABC, you'd need to basically add a
 > super class to add behaviour, at runtime, monkey-patching things up
 > till you make it work, most of the time. This is a terrible thing to
 > do, even in Python. Or maybe I miss some wonderful well_known recipe
 > about it ...
 >
 > Let's use generic functions instead of ABC's, for god's sake !
 > Need to define a common protocol for item access in an object :

There's nothing in what I said that prevents the use of generic 
functions. Part of the reason for defining the ABC's is to give the 
generic functions something to operate on.

If you look at the research material that's been done on generic 
functions, you'll notice that there's a generalization of generic 
functions called "predicated-based dispatch". That's simply a fancy way 
of saying that we do some tests on the calling arguments, and depending 
on how those tests come out we decide which actual piece of code to run.

A subset of predicate-based dispatch is called "type-based dispatch" -- 
what most people think of as generic functions. Type based dispatch is a 
specialized form of predicate-based dispatch where all of the tests are 
type tests, i.e. "if the first argument is of type A, then use this code..."

The point is, regardless of whether you are using the full 
predicate-based dispatch or the more specialized type-based dispatch, 
the generic function dispatch is nothing more than a series of boolean 
tests. In many cases, we can optimize the tests by creating lookup 
tables and such, compressing multiple tests into a single jump 
calculation so that they may not "look like" sequential boolean tests, 
but nevertheless that's what they are.

The controversy over ABCs is really a controversy over what kinds of 
tests we should use for dispatching. Should the tests be limited to 
"is-a" (isinstance) tests, or should we also allow "has-a" (hasattr) 
tests as well?

But generic functions can easily use either kind of test, or any other 
kind of boolean test for that matter. So nothing about ABCs is 
incompatible with the use of generic functions.

Now, you bring up the point that in some cases, you might want to use 
tests in a way that allows you to override the classification of an 
object. In other words, the object itself may have some "native" 
classification (e.g. 'this is a list, this is a string') that is 
inherent in its construction, but instead you want to might want to 
impose a different set of categories (e.g. 'this is a 
marshall-by-reference, this is a marshall-by-value'), so that objects 
which are unrelated get grouped together, and vice versa.

It seems to me that this is a good argument to have - it's really the 
crux of much of the previous discussion - but it will need more concrete 
examples and use cases.

You see, even if this "override" feature isn't supported in the ABC 
system, there's nothing to prevent someone from adding it to the generic 
dispatch mechanism. Generic functions need not be limited to only using 
'isinstance' tests and need not be limited to only using ABCs. An 
independent taxonomy of types and categories can be built using an 
entirely parallel data structure, without touching the original classes 
at all, and dispatching can be based on that.

The real question to ask is - what is the 90% solution? In other words, 
what is the simplest mechanism that covers 90% of the use cases, while 
allowing the other 10% of use cases to handled by allowing the 
application programmer to extend the system or add a layer on top of it?

Here's how the debate should go: Various proponents will submit designs 
for an object inspection/classification system that cover those 90% of 
cases. Ideally, the simplest design will be used as the starting point. 
Various people will then put forward use cases that aren't included by 
the candidate design. However, merely putting forward a use case isn't 
enough, one has to make a convincing argument that such a use case is 
significant - meaning that it is likely to be commonly occurring, and 
has no easy workaround.

Thus, if someone has a use case that requires that objects be 
dynamically reclassified at runtime, they should be prepared to show 
that (a) its a common use case, and (b) there's no obvious way to do it 
other than to build that feature into the system from the beginning.

 > use a 'getitem' generic function ...
 >
 > Need to implement it for classes you didn't write (or even for your
 > own) :
 > @method(FooWidget, int)
 > def getitem(widget, pos):
 >     # just do it in a FooWidget-y- way
 >
 > Well, I just don't dare post this on the py3k list ... I also just
 > don't get why people are so reluctant to understand what generic
 > functions are, and how they would solve all these non-problems :)
 >
 > Of course, for py3k there should be some bridge between old-style
 > generic functions (__len__, __getitem__, ...) and something more
 > general ; something that, above all, DECOUPLES method definition from
 > class definition. We already know that Python is not about
 > encapsulation (in the classical dogmatic way), but merely about puting
 > stuff into namespaces. So let's drop entirely the pseudo encapsulation
 > thing, and let's go with the one truly modular way. Generic functions
 > are namespaces, of course, and can be further put into any modules or
 > namespace kind you want to.
 >
 > Cheers,
 > Aur?lien.
 >
 > PS : note how such a getitem gf could generalize also to
 > hashtable/dict-like access, with something like :
 >
 > @method(dict, str):
 > def getitem(adict, astr):
 >     # just do it ...
 >
 > Double dispatch is not just a fancy complicated feature. It allows to
 > build much more leagible, maintainable code. Ok that might be a little
 > harder for the folks that maintain the language runtime. But these are
 > supposed the real men, not the day-to-day users that currently
 > struggle with a broken 'object model'.
 >
 > Wich is broken because inspired by the aforementioned, mindless,
 > dogmas. Too many people in Python-land have been exposed to Java, alas
 > ...
 >
 > hum, I stop now.

By the way, there's something odd about the "reply-to" headers on your 
email, when I hit "reply all", it replied only to you, not the list.

-- Talin

From talin at acm.org  Tue Apr 17 09:23:39 2007
From: talin at acm.org (Talin)
Date: Tue, 17 Apr 2007 00:23:39 -0700
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <462474AD.2050807@acm.org>
References: <46211608.7010604@acm.org>	<20070416150746.GE17621@crater.logilab.fr>
	<462474AD.2050807@acm.org>
Message-ID: <462475FB.6060405@acm.org>

Talin wrote:
> By the way, there's something odd about the "reply-to" headers on your 
> email, when I hit "reply all", it replied only to you, not the list.

Oh, duh. I'm an idiot - I didn't notice that the reply was only to me, 
not the list.

Aur?lien, my apologies for publicly posting material that you sent to me 
privately, that was my fault.

However, I still think that the questions you raise are worth 
discussing, and I don't think you need be concerned about raising them 
in public. At least, you won't look as dumb as I do right now :)

-- Talin

From aurelien.campeas at logilab.fr  Tue Apr 17 10:02:31 2007
From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=)
Date: Tue, 17 Apr 2007 10:02:31 +0200
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <462475FB.6060405@acm.org>
References: <46211608.7010604@acm.org>
	<20070416150746.GE17621@crater.logilab.fr>
	<462474AD.2050807@acm.org> <462475FB.6060405@acm.org>
Message-ID: <20070417080231.GA4600@crater.logilab.fr>

On Tue, Apr 17, 2007 at 12:23:39AM -0700, Talin wrote:
> Talin wrote:
> > By the way, there's something odd about the "reply-to" headers on your 
> > email, when I hit "reply all", it replied only to you, not the list.
> 
> Oh, duh. I'm an idiot - I didn't notice that the reply was only to me, 
> not the list.
> 
> Aur?lien, my apologies for publicly posting material that you sent to me 
> privately, that was my fault.

No problem.

> 
> However, I still think that the questions you raise are worth 
> discussing, and I don't think you need be concerned about raising them 
> in public. At least, you won't look as dumb as I do right now :)

Well, sorry :). Two points: my feeling is that right now these
discussions belong to python-ideas ; also I lack the time to write
clean and well-laid contributions that are up to level/quality of
these lists (py3k, python-ideas).

But the unconclusiveness of all the discussions related to ABCs,
interfaces and generic functions only show the immaturity of many on
these topics. Including mine, of course, as in: I lack the experience
required to back up all my claims about the generic function school of
OO.  ... (although P. Eby did a great work on this front, he looks a bit
lonely imho).

> 
> -- Talin
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/aurelien.campeas%40logilab.fr

From nnorwitz at gmail.com  Tue Apr 17 10:49:02 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 17 Apr 2007 01:49:02 -0700
Subject: [Python-3000] some test failures on 64-bit
Message-ID: <ee2a432c0704170149g79a48b45md9e40bcf2438c329@mail.gmail.com>

test_cpickle (probably all the same issue here):
test_xpickle, too

======================================================================
FAIL: test_ints (test.test_cpickle.cPickleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line
503, in test_ints
    self.assertEqual(expected, n2)
AssertionError: -9223372036854775807 != 1

======================================================================
FAIL: test_ints (test.test_cpickle.cPicklePicklerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line
503, in test_ints
    self.assertEqual(expected, n2)
AssertionError: -9223372036854775807 != 1

======================================================================
FAIL: test_ints (test.test_cpickle.cPickleListPicklerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line
503, in test_ints
    self.assertEqual(expected, n2)
AssertionError: -9223372036854775807 != 1

======================================================================
FAIL: test_ints (test.test_cpickle.cPickleFastPicklerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line
503, in test_ints
    self.assertEqual(expected, n2)
AssertionError: -9223372036854775807 != 1


test_gzip:
test test_gzip failed -- Traceback (most recent call last):
  File "/home/neal/python/dev/py3k/Lib/test/test_gzip.py", line 77, in
test_many_append
    ztxt = zgfile.read(8192)
  File "/home/neal/python/dev/py3k/Lib/gzip.py", line 235, in read
    self._read(readsize)
  File "/home/neal/python/dev/py3k/Lib/gzip.py", line 300, in _read
    self._read_eof()
  File "/home/neal/python/dev/py3k/Lib/gzip.py", line 316, in _read_eof
    crc32 = read32(self.fileobj)
  File "/home/neal/python/dev/py3k/Lib/gzip.py", line 40, in read32
    return struct.unpack("<l", input.read(4))[0]
  File "/home/neal/python/dev/py3k/Lib/struct.py", line 87, in unpack
    return o.unpack(s)
struct.error: unpack requires a string argument of length 4

From nnorwitz at gmail.com  Tue Apr 17 11:03:05 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 17 Apr 2007 02:03:05 -0700
Subject: [Python-3000] string module trimming
Message-ID: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>

I whacked almost everything in the string (and strop) module.  Here
are some things left to do:

 * capwords is defined and used only in one place:
      ./idlelib/testcode.py
    ISTM, this can go.

 * maketrans remains (also implemented in stropmodule.c).  It is used 11 times.
 * ascii_letters is used 17 times
 * ascii_lowercase is used 2 times
 * digits is used 15 times
 * hexdigits is used 2 times
 * letters is used 9 times
 * whitespace is used 4 times

Some of these uses are in test code.

There are a bunch of other constants that aren't used at all in the
stdlib.  Below is the full list from my grep (which probably missed a
few uses due to from string import).

n
--
Lib/idlelib/UndoDelegator.py:   string.ascii_letters + string.digits
Lib/idlelib/AutoComplete.py:    string.ascii_letters + string.digits
Lib/idlelib/AutoComplete.py:    string.ascii_letters + string.digits
Lib/idlelib/HyperParser.py:     string.ascii_letters + string.digits
Lib/idlelib/HyperParser.py:     string.ascii_letters
Lib/idlelib/keybindingDialog.py:        string.ascii_lowercase + string.digits
Lib/idlelib/keybindingDialog.py:        string.ascii_lowercase
Lib/idlelib/PyShell.py:         string.ascii_letters + string.digits
Lib/idlelib/ScriptBinding.py:   string.ascii_letters + string.digits
Lib/idlelib/AutoExpand.py:      string.ascii_letters + string.digits
Lib/distutils/util.py:          string.whitespace
Lib/distutils/util.py:          string.whitespace
Lib/distutils/fancy_getopt.py:  string.maketrans('-', '_')
Lib/distutils/fancy_getopt.py:  string.maketrans(string.whitespace, '
' * len(string.whitespace))
Lib/bsddb/test/test_dbshelve.py:        string.letters
Lib/bsddb/test/test_get_none.py:        string.letters
Lib/bsddb/test/test_get_none.py:        string.letters
Lib/bsddb/test/test_basics.py:          string.letters
Lib/bsddb/test/test_queue.py:           string.letters
Lib/bsddb/test/test_queue.py:           string.letters
Lib/msilib/__init__.py:                 string.digits
Lib/plat-riscos/riscospath.py:          string.ascii_letters
Lib/plat-riscos/rourl2path.py:          string.maketrans("/.", "./")
Lib/test/test_csv.py:                   string.letters
Lib/test/test_pkgimport.py:             string.letters
Lib/test/test_pkgimport.py:             string.letters
Lib/test/test_md5.py:                   string.hexdigits
Lib/test/test_mimetools.py:             string.ascii_letters + string.digits
Lib/test/test_pkg.py:                   "print(__name__, 'loading');
from . import string; print(string.spam)"),
Lib/test/test_hashlib.py:               string.hexdigits
Lib/test/string_tests.py:               string.maketrans('abc', 'xyz')
Lib/test/string_tests.py:               string.maketrans
Lib/test/string_tests.py:               string.maketrans('abc', 'xyz')
Lib/test/string_tests.py:               string.maketrans('a', 'A')
Lib/test/test_urllib2.py:               string.maketrans("/.", "./"))
Lib/test/test_bigmem.py:                string.maketrans('.aZ', '-!$')
Lib/plat-mac/gensuitemodule.py:         string.ascii_letters + '_'
Lib/plat-mac/gensuitemodule.py:         string.digits
Lib/nturl2path.py:                      string.ascii_letters
Lib/pipes.py:                           string.ascii_letters + string.digits
Lib/textwrap.py:                        string.maketrans(_whitespace,
' ' * len(_whitespace))
Lib/textwrap.py:                        string.lowercase
Lib/tokenize.py:                        string.ascii_letters
Lib/ntpath.py:                          string.ascii_letters + string.digits
Lib/Cookie.py:                          string.ascii_letters + string.digits
Lib/urllib.py:                          string.maketrans("", "")
Lib/cmd.py:                             string.ascii_letters + string.digits

From barry at python.org  Tue Apr 17 14:28:04 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Apr 2007 08:28:04 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
Message-ID: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 17, 2007, at 5:03 AM, Neal Norwitz wrote:

>  * maketrans remains (also implemented in stropmodule.c).  It is  
> used 11 times.
>  * ascii_letters is used 17 times
>  * ascii_lowercase is used 2 times
>  * digits is used 15 times
>  * hexdigits is used 2 times
>  * letters is used 9 times
>  * whitespace is used 4 times

You're not planning on removing these too are you?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRiS9VHEjvBPtnXfVAQJUIQP9Fn+5jclYPtaRwp+W/j78XARk5iIKIi2j
EEaOYrtRiJtzsQX743L8hT3x8MGnHWN4D803DFTE7n+gaooGdhBzzHw6jQNIwuIv
ZYEjRn7+1kUAYpONB9iepqVhk7KuLMOwRe+VyBRftnxh1bIaXPQxGRCU7ZRM6R+2
s7CiFH0Elc0=
=zsiL
-----END PGP SIGNATURE-----

From guido at python.org  Tue Apr 17 15:53:13 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Apr 2007 06:53:13 -0700
Subject: [Python-3000] Draft PEP: Using UTF-8 as the default source
	encoding
In-Reply-To: <f01iip$be9$1@sea.gmane.org>
References: <4621F48A.1090308@v.loewis.de> <f01iip$be9$1@sea.gmane.org>
Message-ID: <ca471dc20704170653o61ebeba3x7b12e83d8fa4f45@mail.gmail.com>

I don't see how I can fight this overwhelming support, so I'll give it +1 too.

On 4/16/07, Travis E. Oliphant <oliphant.travis at ieee.org> wrote:
> Martin v. L?wis wrote:
> > I'd like to submit the following PEP for Python 3.
> >
> > Regards,
> > Martin
> >
> >
> > PEP: xxx
> > Title: Using UTF-8 as the default source encoding
> > Version: $Revision $
> > Last-Modified: $Date $
> > Author: Martin v. L?wis <martin at v.loewis.de>
> > Status: Draft
> > Type: Standards Track
> > Content-Type: text/x-rst
> > Created: 15-Apr-2007
> > Python-Version: 3.0
> > Post-History:
> >
>
> +1
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Apr 17 16:02:58 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Apr 2007 10:02:58 -0400
Subject: [Python-3000] i18n prefix
In-Reply-To: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br>
References: <662B0B21-F14D-4EFC-BB57-9ACC22E879E6@digi.com.br>
Message-ID: <51F9EA46-4562-44DD-8B4E-440E206B0697@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 17, 2007, at 1:35 AM, Humberto Diogenes wrote:

>    This is my first post to python-3000. After Guido PEPs-deadline
> announcement, I decided to discuss an old idea while it still has
> some chance of being accepted. :)
>
>    No, I do not want i18n identifiers. What I'd like to propose is a
> standard syntax for i18n strings. Something like:
>    print i"my string"
>
>    My points are:
>    * i18n is everywhere (or is it the other way around? :-)
>    * Python has a tradition of incorporating good practices
>    * _ = gettext.gettext; print _("my string") isn't very nice
>
>    So, do you think that something like i"my string" would be better
> than _("my string")? Would it add unnecessary complexity to Python?

I've never understood the desire for an i-prefix, though it comes up  
often.  In py3k you won't have u-prefixes, but you'll still have r- 
prefixes (I'm assuming), so while you'll have less prefix-explosion,  
I still don't see much benefit in adding a prefix for something that  
is perfectly well handled by a function call.  Besides, if you used a  
prefix you'd have to explain how the run-time aspects of translations  
will get handled.  Definitely PEP material.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRiTTknEjvBPtnXfVAQKfiwP/WIxeq+iG1hXVYikH4vy6pkHHT96zsxRv
tJ9aYuhLOFDmWkhsXpiRsSnPzPMXvth3UHiHCYo+D1aOEUyHyqCAZ97iHcvD+N7T
PDycmWGl5dEaOBJ3ieXVV+0Bbv3i+J6QpA1Aj1QdS5o6pkB31STdt+3OPG2eKEiz
SQKG+YrpgAI=
=cR8m
-----END PGP SIGNATURE-----

From guido at python.org  Tue Apr 17 17:39:19 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Apr 2007 08:39:19 -0700
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <46211608.7010604@acm.org>
References: <46211608.7010604@acm.org>
Message-ID: <ca471dc20704170839q36406e47o74ae5aa5fd4f80b1@mail.gmail.com>

Have you seen the (*very* *early* *experimental*) set of ABCs that I
started in sandbox/abc/abc.py ?

On 4/14/07, Talin <talin at acm.org> wrote:
> Part of the reason why I haven't volunteered to write a PEP for ABC's is
> because I don't feel that I understand the various proposals and the
> background discussion well enough. However, it occurs to me that writing
> the rationale section of a PEP can often be the hardest part, and I
> think I understand the issues well enough to write that preface. So
> here's my contribution:
>
> ---
>
> In the domain of object-oriented programming, the usage patterns for
> interacting with an object can be divided into two basic categories,
> which are 'invocation' and 'inspection'.
>
> Invocation means interacting with an object by invoking its methods.
> Usually this is combined with polymorphism, so that invoking a given
> method may run different code depending on the type of an object.
>
> Inspection means the ability for external code (outside of the object's
> methods) to examine the type or properties of that object, and make
> decisions on how to treat that object based on that information.
>
> Both usage patterns serve the same general end, which is to be able to
> support the processing of diverse and potentially novel objects in a
> uniform way, but at the same time allowing processing decisions to be
> customized for each different type of object.
>
> In classical OOP theory, invocation is the preferred usage pattern, and
> inspection is actively discouraged, being considered a relic of an
> earlier, procedural programming style. However, in practice this view is
> simply too dogmatic and inflexible, and leads to a kind of design
> rigidity that is very much at odds with the dynamic nature of a language
> like Python.
>
> In particular, there is often a need to process objects in a way that
> wasn't anticipated by the creator of the object class. It is not always
> the best solution to build in to every object methods that satisfy the
> needs of every possible user of that object. Moreover, there are many
> powerful dispatch philosophies that are in direct contrast to the
> classic OOP requirement of behavior being strictly encapsulated within
> an object, examples being rule or pattern-match driven logic.
>
> On the the other hand, one of the criticisms of inspection by classic
> OOP theorists is the lack of formalisms and the ad hoc nature of what is
> being inspected. In a language such as Python, in which almost any
> aspect of an object can be reflected and directly accessed by external
> code, there are many different ways to test whether an object conforms
> to a particular protocol or not. For example, if asking 'is this object
> a mutable sequence container?', one can look for a base class of 'list',
> or one can look for a method named '__getitem__'. But note that although
> these tests may seem obvious, neither of them are correct, as one
> generates false negatives, and the other false positives.
>
> The generally agreed-upon remedy is to standardize the tests, and group
> them into a formal arrangement. This is most easily done by associating
> with each class a set of standard testable properties, either via the
> inheritance mechanism or some other means. Each test carries with it a
> set of promises: it contains a promise about the general behavior of the
> class, and a promise as to what other class methods will be available.
>
> This PEP proposes a particular strategy for organizing these tests known
> as Abstract Base Classes, or ABC. ABCs are simply Python classes that
> are added into an object's inheritance tree to signal certain features
> of that object to an external inspector. Tests are done using
> isinstance(), and the presence of a particular ABC means that the test
> has passed.
>
> Like all other things in Python, these promises are in the nature of a
> gentlemen's agreement - which means that the language does not attempt
> to enforce that these promises are kept.
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Tue Apr 17 18:18:38 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 17 Apr 2007 12:18:38 -0400
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <20070417103337.C1402@tribble.ilrt.bris.ac.uk>
References: <20070417080231.GA4600@crater.logilab.fr>
	<46211608.7010604@acm.org>
	<20070416150746.GE17621@crater.logilab.fr>
	<462474AD.2050807@acm.org> <462475FB.6060405@acm.org>
	<20070417080231.GA4600@crater.logilab.fr>
Message-ID: <5.1.1.6.0.20070417115834.02b207e8@sparrow.telecommunity.com>

At 10:47 AM 4/17/2007 +0100, Jan Grant wrote:
>On Tue, 17 Apr 2007, Aur?lien Camp?as wrote:
>
> > But the unconclusiveness of all the discussions related to ABCs,
> > interfaces and generic functions only show the immaturity of many on
> > these topics. Including mine, of course, as in: I lack the experience
> > required to back up all my claims about the generic function school of
> > OO.  ... (although P. Eby did a great work on this front, he looks a bit
> > lonely imho).
>
>FWIW Phillip's GFs look more and more like being the pythonic way to do
>things. ABCs and interfaces work in a java world, where arguments to a
>method arrive with a "preferred" type (ie, the type in the parameter
>declaraion) and that type is used to interpret method calls when it
>comes to method overloading. Hence Phillip's comment that interfaces are
>"receiver-oriented".

Yes...  and in *Java*, it tends to work out that way because a lot of 
factors conspire to make it work, like the fact that mere method names 
aren't enough to make an object acceptable, combined with overloading, lack 
of really useful generic types, etc.


>We don't have that in python, which makes me question the utility of
>creating a complex ABC universe prior to having a mechanism for method
>dispatch.
>
>Of course, GF implementations _do_ exist which would give the ABCs some
>utility. It's just a shame that there doesn't appear to be one that's
>going to be the default way to do it in py3k.

FYI, there's actually one tucked away in the pkgutil module since Python 
2.5... single dispatch only, but it was the easiest way to support asking 
future PEP 302 importers what modules are in a package.  It's more 
primitive than the "simplegeneric" package on the Cheeseshop, but would 
certainly be functional enough to work with ABC's, if it were documented 
(at the moment it's technically an implementation detail of pkgutil.

However, if there's interest, I'd be willing to create a *small* (i.e. 
about 1000 lines, give or take a few hundred) Py3K/2.6 
multi-dispatch/interface/adapters module, written such that it would be 
extensible for third-party dispatch and interface systems.  I'm not sure I 
could deliver a complete PEP before the deadline, however, and my 
impression is that Guido feels (or at least felt) that such a thing would 
be premature.  If that has changed, however, I'm willing to write a PEP and 
take a stab at getting a draft in this week.

Anyway, if we do that, we would then have interfaces, adapters, generic 
functions, *and* ABCs in Py3K, so everybody should be happy...  except for 
those who want only one Obvious Way, of course.  :)


From pje at telecommunity.com  Tue Apr 17 18:46:38 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 17 Apr 2007 12:46:38 -0400
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <462474AD.2050807@acm.org>
References: <20070416150746.GE17621@crater.logilab.fr>
	<46211608.7010604@acm.org>
	<20070416150746.GE17621@crater.logilab.fr>
Message-ID: <5.1.1.6.0.20070417122259.02b206a0@sparrow.telecommunity.com>

At 12:18 AM 4/17/2007 -0700, Talin wrote:
>The real question to ask is - what is the 90% solution? In other words,
>what is the simplest mechanism that covers 90% of the use cases, while
>allowing the other 10% of use cases to handled by allowing the
>application programmer to extend the system or add a layer on top of it?

That's easy, actually.  If you have a generic function, then you can add an 
overload for your specific type that does whatever you need it to do.

In other words, the proper discipline for inspection is to place the 
inspection within a generic function that your clients can add overloads to.

Note, by the way, that even fairly sophisticated generic function 
implementations are pretty trivial in Python.  Take a look at Guido's 
multi-dispatch prototype for Py3K, or the simplegeneric package on the 
Cheeseshop, or even the generic function implementation in pkgutil (Python 
2.5 and up).

That makes them a pretty good 90% solution, especially since they don't 
even *need* ABC's for this to work.  It's quite okay to define methods for 
concrete types and let the user decide whether their implementation 
suffices, doing something like:

     thefunction[mytype] = thefunction[othertype]

to say that *in the context of thefunction*, mytype is an adequate 
replacement for othertype.

(ABCs and interfaces have the problem, however, of being 
context-free.  Thus, the proper discipline for interfaces is to use 
adaptation and to have "one interface per use case", rather than "one 
interface per concept".)


>Thus, if someone has a use case that requires that objects be
>dynamically reclassified at runtime,

Dynamic reclassification isn't the real problem, it's just as much a 
problem for static classifications that *disagree with the receiver's 
choice of inspection*.  Pydoc and earlier Zope versions have this problem 
all over the place, no dynamic reclassification necessary.

The key principle here is that the *client* should get to decide what 
happens to their objects, because the author of the client is the only one 
who knows what he or she *intends*.  The author of the library cannot 
*know* the use cases of his or her users, and should therefore refuse the 
temptation to guess.

Generic functions allow library authors to provide *overrideable defaults*, 
in place of *unchangeable guesses*.  (Which is what if/then tests are, 
regardless of whether they use ABCs, concrete types, interfaces, or even 
hasattr().)


From nnorwitz at gmail.com  Tue Apr 17 19:27:44 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 17 Apr 2007 10:27:44 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
Message-ID: <ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>

On 4/17/07, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Apr 17, 2007, at 5:03 AM, Neal Norwitz wrote:
>
> >  * maketrans remains (also implemented in stropmodule.c).  It is
> > used 11 times.
> >  * ascii_letters is used 17 times
> >  * ascii_lowercase is used 2 times
> >  * digits is used 15 times
> >  * hexdigits is used 2 times
> >  * letters is used 9 times
> >  * whitespace is used 4 times
>
> You're not planning on removing these too are you?

I don't have any plans, just considering options.  Move them
somewhere?  Perhaps, trim the ones that are unused.  In a unicode
world, I'm not sure how much some of these make sense.  letters stands
out more than others.  I don't know enough about unicode to know if
digits or whitespace can be diff.

The Template class is in string, which seems pretty appropriate.
Although with the naming re-org that will be coming, should string be
renamed to stringlib (or whatever)?

What would you like to see for things like maketrans, capwords, etc?

n

From barry at python.org  Tue Apr 17 22:58:19 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Apr 2007 16:58:19 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
Message-ID: <D49C50A2-55AD-409E-B040-3870B7EA0F53@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 17, 2007, at 1:27 PM, Neal Norwitz wrote:

> On 4/17/07, Barry Warsaw <barry at python.org> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Apr 17, 2007, at 5:03 AM, Neal Norwitz wrote:
>>
>> >  * maketrans remains (also implemented in stropmodule.c).  It is
>> > used 11 times.
>> >  * ascii_letters is used 17 times
>> >  * ascii_lowercase is used 2 times
>> >  * digits is used 15 times
>> >  * hexdigits is used 2 times
>> >  * letters is used 9 times
>> >  * whitespace is used 4 times
>>
>> You're not planning on removing these too are you?
>
> I don't have any plans, just considering options.  Move them
> somewhere?  Perhaps, trim the ones that are unused.  In a unicode
> world, I'm not sure how much some of these make sense.  letters stands
> out more than others.  I don't know enough about unicode to know if
> digits or whitespace can be diff.

Certainly it makes sense to keep the ascii_* constants.  The docs say  
letters and lowercase are locale-aware, so I guess it makes sense to  
keep those too.

> The Template class is in string, which seems pretty appropriate.
> Although with the naming re-org that will be coming, should string be
> renamed to stringlib (or whatever)?

I don't know, I think 'string' still works fine for me.

> What would you like to see for things like maketrans, capwords, etc?

I've used them both a few times, so I guess I'd be fine with keeping  
them where they are.  capwords() seems easy enough to write the one  
or two times I'd need it, so I don't have strong feelings about them.

I know at one point there was talk about a 'text' or 'textlib'  
package, which might serve as a way to organize such other modules as  
textwrap, *StringIO, etc.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRiU07HEjvBPtnXfVAQKcOQQAt/1r05QGcDJ5VYKQ6xo9v5oA7vGpxtCp
oAZJJxreQOlXbxYjcupEzaX9pfFOTng3hGBmrSSspxivOK5v8kcKZvBfxf3Ndt2j
DKAH8rH79IoZO4TCQ8ehuEBxA6v912YUJtLmcwIWxsgO+ZgEBeTDV8UaddxEWRWS
mQRow+zP7ew=
=CkXk
-----END PGP SIGNATURE-----

From lists at cheimes.de  Tue Apr 17 22:12:10 2007
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 17 Apr 2007 22:12:10 +0200
Subject: [Python-3000] string module trimming
In-Reply-To: <ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
Message-ID: <f039of$ih$1@sea.gmane.org>

Neal Norwitz schrieb:
> I don't have any plans, just considering options.  Move them
> somewhere?  Perhaps, trim the ones that are unused.  In a unicode
> world, I'm not sure how much some of these make sense.  letters stands
> out more than others.  I don't know enough about unicode to know if
> digits or whitespace can be diff.

What do you think about replacing the definitions by information from
the unicode character properties database. The information are available
somewhere in Python:

http://docs.python.org/lib/re-syntax.html

\w ... With LOCALE, it will match the set [0-9_] plus whatever
characters are defined as alphanumeric for the current locale. If
UNICODE is set, this will match the characters [0-9_] plus whatever is
classified as alphanumeric in the Unicode character properties database.

Christian


From guido at python.org  Tue Apr 17 23:23:15 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Apr 2007 14:23:15 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <D49C50A2-55AD-409E-B040-3870B7EA0F53@python.org>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<D49C50A2-55AD-409E-B040-3870B7EA0F53@python.org>
Message-ID: <ca471dc20704171423xe6fff15s3b4e2760c0d3d64f@mail.gmail.com>

On 4/17/07, Barry Warsaw <barry at python.org> wrote:
> Certainly it makes sense to keep the ascii_* constants.  The docs say
> letters and lowercase are locale-aware, so I guess it makes sense to
> keep those too.

Really? Remember that Py3k won't be using the locale for anything
character/string-related -- strings will all be unicode and the
islower() methods and friends just use the Unicode database.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Apr 17 23:27:25 2007
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Apr 2007 17:27:25 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <ca471dc20704171423xe6fff15s3b4e2760c0d3d64f@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<D49C50A2-55AD-409E-B040-3870B7EA0F53@python.org>
	<ca471dc20704171423xe6fff15s3b4e2760c0d3d64f@mail.gmail.com>
Message-ID: <B9514FAE-46A8-403A-9BDF-691E923C07FA@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 17, 2007, at 5:23 PM, Guido van Rossum wrote:

> On 4/17/07, Barry Warsaw <barry at python.org> wrote:
>> Certainly it makes sense to keep the ascii_* constants.  The docs say
>> letters and lowercase are locale-aware, so I guess it makes sense to
>> keep those too.
>
> Really? Remember that Py3k won't be using the locale for anything
> character/string-related -- strings will all be unicode and the
> islower() methods and friends just use the Unicode database.

Well then in in that case, zap 'em! :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRiU7vXEjvBPtnXfVAQJO6gQAkZgv+nKJgZI0Vkf5Ykw7k0fLKWjPi+mp
bJdkoyZaD/vcS5Katy+okoDCvYy/aK2q4xt1zhppJjhn1wxOPA+2xYKNOU9vvLhP
RaH/aZwd09fiJAQLsXiaPeb21KWbW6kPoinVV+V5DokOXGdvZ3AnoC/BPERwkOc2
6Q6ocAKjcvU=
=wTup
-----END PGP SIGNATURE-----

From guido at python.org  Tue Apr 17 23:28:00 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Apr 2007 14:28:00 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <f039of$ih$1@sea.gmane.org>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
Message-ID: <ca471dc20704171428q63c71d6i3083c8302f8e60e8@mail.gmail.com>

On 4/17/07, Christian Heimes <lists at cheimes.de> wrote:
> Neal Norwitz schrieb:
> > I don't have any plans, just considering options.  Move them
> > somewhere?  Perhaps, trim the ones that are unused.  In a unicode
> > world, I'm not sure how much some of these make sense.  letters stands
> > out more than others.  I don't know enough about unicode to know if
> > digits or whitespace can be diff.
>
> What do you think about replacing the definitions by information from
> the unicode character properties database. The information are available
> somewhere in Python:
>
> http://docs.python.org/lib/re-syntax.html
>
> \w ... With LOCALE, it will match the set [0-9_] plus whatever
> characters are defined as alphanumeric for the current locale. If
> UNICODE is set, this will match the characters [0-9_] plus whatever is
> classified as alphanumeric in the Unicode character properties database.

Yes, unicode.islower() and friends have this information.

It would be silly to set e.g. letters to a string of all unicode
letters -- that would be a string of 46618 characters! Similar, there
are 304 unicode digits. (And this is in a narrow Unicode build, only
supporting the basic Unicode plane, 0--2**16!)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Wed Apr 18 00:55:46 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 17 Apr 2007 18:55:46 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <f039of$ih$1@sea.gmane.org>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
Message-ID: <fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>

On 4/17/07, Christian Heimes <lists at cheimes.de> wrote:
> Neal Norwitz schrieb:
> > I don't have any plans, just considering options.  Move them
> > somewhere?  Perhaps, trim the ones that are unused.  In a unicode
> > world, I'm not sure how much some of these make sense.  letters stands
> > out more than others.  I don't know enough about unicode to know if
> > digits or whitespace can be diff.

There are several additional characters in both sets, and plenty of
reasons that a given program might want to use a restricted set.
(Probably those already in string, or else a letters grouping set by
locale.)

> What do you think about replacing the definitions by information from
> the unicode character properties database. The information are available
> somewhere in Python:

> http://docs.python.org/lib/re-syntax.html

> \w ... With LOCALE, it will match the set [0-9_] plus whatever
> characters are defined as alphanumeric for the current locale. If
> UNICODE is set, this will match the characters [0-9_] plus whatever is
> classified as alphanumeric in the Unicode character properties database.

There are reasons to want exactly ASCII.

There are also reasons to want only "local" letters.  For example, in
a French interface, I might want to include the extra French letters,
but not the Greek.

Also note that regex isn't quite the only use of those letters groupings.

-jJ

From guido at python.org  Wed Apr 18 01:07:09 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Apr 2007 16:07:09 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
Message-ID: <ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>

On 4/17/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> There are also reasons to want only "local" letters.  For example, in
> a French interface, I might want to include the extra French letters,
> but not the Greek.

The Unicode world doesn't support this directly AFAIK. The locale
module doesn't deal with Unicode, only with 8-bit characters (not
multi-byte characters). You'll lose this anyway. Certainly
string.letters is not going to provide this functionality.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at cheimes.de  Wed Apr 18 01:53:48 2007
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 18 Apr 2007 01:53:48 +0200
Subject: [Python-3000] removing functions from string module
In-Reply-To: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
References: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
Message-ID: <f03mo4$382$1@sea.gmane.org>

Neal Norwitz schrieb:
> I'm tempted to ask the same thing about the types module.

Don't forget the 'new' module. It's a candidate for 'svn rm'.

"""Create new objects of various types.  Deprecated.

This module is no longer required except for backward compatibility.
Objects of most types can now be created by calling the type object.
"""

Christian


From jcarlson at uci.edu  Wed Apr 18 03:19:04 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 17 Apr 2007 18:19:04 -0700
Subject: [Python-3000] removing functions from string module
In-Reply-To: <f03mo4$382$1@sea.gmane.org>
References: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
	<f03mo4$382$1@sea.gmane.org>
Message-ID: <20070417181112.6320.JCARLSON@uci.edu>


Christian Heimes <lists at cheimes.de> wrote:
> 
> Neal Norwitz schrieb:
> > I'm tempted to ask the same thing about the types module.
> 
> Don't forget the 'new' module. It's a candidate for 'svn rm'.
> 
> """Create new objects of various types.  Deprecated.
> 
> This module is no longer required except for backward compatibility.
> Objects of most types can now be created by calling the type object.
> """

I would generally agree with you, except that when monkey patching
instances...
    self.method = new.instancemethod(other.method.im_func,
                                     self,
                                     self.__class__)

... is a bit more understandable than...
    self.metbod = type(other.method)(other.method.im_func,
                                     self,
                                     self.__class__)

... at least in the examples that I've seen.  Never mind that new is
documented in the standard library documentation, but the types are not.


 - Josiah


From lists at cheimes.de  Wed Apr 18 03:49:16 2007
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 18 Apr 2007 03:49:16 +0200
Subject: [Python-3000] removing functions from string module
In-Reply-To: <20070417181112.6320.JCARLSON@uci.edu>
References: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>	<f03mo4$382$1@sea.gmane.org>
	<20070417181112.6320.JCARLSON@uci.edu>
Message-ID: <f03tgl$iq5$1@sea.gmane.org>

Josiah Carlson schrieb:
> I would generally agree with you, except that when monkey patching
> instances...
>     self.method = new.instancemethod(other.method.im_func,
>                                      self,
>                                      self.__class__)
> 
> .... is a bit more understandable than...
>     self.metbod = type(other.method)(other.method.im_func,
>                                      self,
>                                      self.__class__)
> 
> .... at least in the examples that I've seen.  Never mind that new is
> documented in the standard library documentation, but the types are not.

I strongly emphasize on "a bit more". One needs a lot of inside
knowledge about internal representation to rebind a method to another
instance. The following idea is much easier to understand:

   self.method = boundmethod(self, method_or_function)


Christian


From steven.bethard at gmail.com  Wed Apr 18 04:26:05 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 17 Apr 2007 20:26:05 -0600
Subject: [Python-3000] removing functions from string module
In-Reply-To: <20070417181112.6320.JCARLSON@uci.edu>
References: <ee2a432c0704160029q1a0c493bpf8011e5bb54eda8e@mail.gmail.com>
	<f03mo4$382$1@sea.gmane.org> <20070417181112.6320.JCARLSON@uci.edu>
Message-ID: <d11dcfba0704171926je491971t6eb5683e8694fb0@mail.gmail.com>

On 4/17/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> Christian Heimes <lists at cheimes.de> wrote:
> >
> > Neal Norwitz schrieb:
> > > I'm tempted to ask the same thing about the types module.
> >
> > Don't forget the 'new' module. It's a candidate for 'svn rm'.
> >
> > """Create new objects of various types.  Deprecated.
> >
> > This module is no longer required except for backward compatibility.
> > Objects of most types can now be created by calling the type object.
> > """
>
> I would generally agree with you, except that when monkey patching
> instances...
>     self.method = new.instancemethod(other.method.im_func,
>                                      self,
>                                      self.__class__)
>
> ... is a bit more understandable than...
>     self.metbod = type(other.method)(other.method.im_func,
>                                      self,
>                                      self.__class__)
>
> ... at least in the examples that I've seen.

Yes, but you could just as easily write this as::

    self.method = other.method.im_func.__get__(self)

which I'd contend is better than the original new.instancemethod version.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Wed Apr 18 05:45:18 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Apr 2007 20:45:18 -0700
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <5.1.1.6.0.20070417115834.02b207e8@sparrow.telecommunity.com>
References: <46211608.7010604@acm.org>
	<20070416150746.GE17621@crater.logilab.fr> <462474AD.2050807@acm.org>
	<462475FB.6060405@acm.org> <20070417080231.GA4600@crater.logilab.fr>
	<20070417103337.C1402@tribble.ilrt.bris.ac.uk>
	<5.1.1.6.0.20070417115834.02b207e8@sparrow.telecommunity.com>
Message-ID: <ca471dc20704172045o4915e1dch6085d6aadc69c16e@mail.gmail.com>

On 4/17/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:47 AM 4/17/2007 +0100, Jan Grant wrote:
> >On Tue, 17 Apr 2007, Aur?lien Camp?as wrote:
> >
> > > But the unconclusiveness of all the discussions related to ABCs,
> > > interfaces and generic functions only show the immaturity of many on
> > > these topics. Including mine, of course, as in: I lack the experience
> > > required to back up all my claims about the generic function school of
> > > OO.  ... (although P. Eby did a great work on this front, he looks a bit
> > > lonely imho).
> >
> >FWIW Phillip's GFs look more and more like being the pythonic way to do
> >things. ABCs and interfaces work in a java world, where arguments to a
> >method arrive with a "preferred" type (ie, the type in the parameter
> >declaraion) and that type is used to interpret method calls when it
> >comes to method overloading. Hence Phillip's comment that interfaces are
> >"receiver-oriented".
>
> Yes...  and in *Java*, it tends to work out that way because a lot of
> factors conspire to make it work, like the fact that mere method names
> aren't enough to make an object acceptable, combined with overloading, lack
> of really useful generic types, etc.
>
>
> >We don't have that in python, which makes me question the utility of
> >creating a complex ABC universe prior to having a mechanism for method
> >dispatch.
> >
> >Of course, GF implementations _do_ exist which would give the ABCs some
> >utility. It's just a shame that there doesn't appear to be one that's
> >going to be the default way to do it in py3k.
>
> FYI, there's actually one tucked away in the pkgutil module since Python
> 2.5... single dispatch only, but it was the easiest way to support asking
> future PEP 302 importers what modules are in a package.  It's more
> primitive than the "simplegeneric" package on the Cheeseshop, but would
> certainly be functional enough to work with ABC's, if it were documented
> (at the moment it's technically an implementation detail of pkgutil.
>
> However, if there's interest, I'd be willing to create a *small* (i.e.
> about 1000 lines, give or take a few hundred) Py3K/2.6
> multi-dispatch/interface/adapters module, written such that it would be
> extensible for third-party dispatch and interface systems.  I'm not sure I
> could deliver a complete PEP before the deadline, however, and my
> impression is that Guido feels (or at least felt) that such a thing would
> be premature.  If that has changed, however, I'm willing to write a PEP and
> take a stab at getting a draft in this week.
>
> Anyway, if we do that, we would then have interfaces, adapters, generic
> functions, *and* ABCs in Py3K, so everybody should be happy...  except for
> those who want only one Obvious Way, of course.  :)

I'd like to see that PEP and that implementation.  I am planning to
make time to write my own PEP promoting ABCs. I think it would be
great if your code and mine could coexist and cooperate instead of
competing -- it's much easier to accept two PEPs than to have to
choose between two competing proposals. I don't take the TOOWTDI
concern too seriously; surely while there will be some overlap, for
many use cases it will be obvious which approach to take, and covering
more use cases means fewer times where applications or frameworks will
feel the need to roll their own.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Wed Apr 18 06:41:49 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 17 Apr 2007 21:41:49 -0700
Subject: [Python-3000] removing functions from string module
In-Reply-To: <d11dcfba0704171926je491971t6eb5683e8694fb0@mail.gmail.com>
References: <20070417181112.6320.JCARLSON@uci.edu>
	<d11dcfba0704171926je491971t6eb5683e8694fb0@mail.gmail.com>
Message-ID: <20070417213402.6325.JCARLSON@uci.edu>


"Steven Bethard" <steven.bethard at gmail.com> wrote:
> On 4/17/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > Christian Heimes <lists at cheimes.de> wrote:
> > > Neal Norwitz schrieb:
> > > > I'm tempted to ask the same thing about the types module.
> > >
> > > Don't forget the 'new' module. It's a candidate for 'svn rm'.
> > >
> > > """Create new objects of various types.  Deprecated.
> > >
> > > This module is no longer required except for backward compatibility.
> > > Objects of most types can now be created by calling the type object.
> > > """
> >
> > I would generally agree with you, except that when monkey patching
> > instances...
> >     self.method = new.instancemethod(other.method.im_func,
> >                                      self,
> >                                      self.__class__)
> 
> Yes, but you could just as easily write this as::
> 
>     self.method = other.method.im_func.__get__(self)
> 
> which I'd contend is better than the original new.instancemethod version.

It is certainly easier to read, but I don't believe it is as
self-documenting as the new.instancemethod variant.  What do you get
when calling 'new.instancemethod'?  A new instance method.

Is func.__get__ documented anywhere?  I've never seen it before (which
is why I use new).


Christian Heimes <lists at cheimes.de> wrote:
> I strongly emphasize on "a bit more". One needs a lot of inside
> knowledge about internal representation to rebind a method to another
> instance. The following idea is much easier to understand:
> 
>    self.method = boundmethod(self, method_or_function)

I agree that the above is easier to understand, but boundmethod is not a
builtin in 2.3, 2.4, or 2.5.  Even with my own use-case of monkey
patching instances with functions or methods, I would resist its
inclusion as a builtin.


 - Josiah


From steven.bethard at gmail.com  Wed Apr 18 07:11:20 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 17 Apr 2007 23:11:20 -0600
Subject: [Python-3000] removing functions from string module
In-Reply-To: <20070417213402.6325.JCARLSON@uci.edu>
References: <20070417181112.6320.JCARLSON@uci.edu>
	<d11dcfba0704171926je491971t6eb5683e8694fb0@mail.gmail.com>
	<20070417213402.6325.JCARLSON@uci.edu>
Message-ID: <d11dcfba0704172211i2d3ef251le618ea07caeba958@mail.gmail.com>

On 4/17/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "Steven Bethard" <steven.bethard at gmail.com> wrote:
> > On 4/17/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > > Christian Heimes <lists at cheimes.de> wrote:
> > > > Neal Norwitz schrieb:
> > > > > I'm tempted to ask the same thing about the types module.
> > > >
> > > > Don't forget the 'new' module. It's a candidate for 'svn rm'.
> > > >
> > > > """Create new objects of various types.  Deprecated.
> > > >
> > > > This module is no longer required except for backward compatibility.
> > > > Objects of most types can now be created by calling the type object.
> > > > """
> > >
> > > I would generally agree with you, except that when monkey patching
> > > instances...
> > >     self.method = new.instancemethod(other.method.im_func,
> > >                                      self,
> > >                                      self.__class__)
> >
> > Yes, but you could just as easily write this as::
> >
> >     self.method = other.method.im_func.__get__(self)
> >
> > which I'd contend is better than the original new.instancemethod version.
>
> It is certainly easier to read, but I don't believe it is as
> self-documenting as the new.instancemethod variant.  What do you get
> when calling 'new.instancemethod'?  A new instance method.
>
> Is func.__get__ documented anywhere?

In the Reference:

http://docs.python.org/ref/descriptor-invocation.html

But I agree it's not as prominent or as clear as it should be.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From erno at iki.fi  Wed Apr 18 11:23:40 2007
From: erno at iki.fi (Erno Kuusela)
Date: 18 Apr 2007 12:23:40 +0300
Subject: [Python-3000] IPv6 address tuple
References: <gu4r6rh5fw9.fsf@ee.oulu.fi.DONT_SPAM>
	<4602E9D8.6080803@v.loewis.de>
	<20070323095711.GO14803@erno.iki.fi> <46050350.600@v.loewis.de>
Message-ID: <gu4abx63ujn.fsf@ee.oulu.fi.DONT_SPAM>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> > I first suggested just chopping off the flowid and scopeid parts of the
> > tuple. Itojun's reply seems to indicate we could potentially get away
> > with merging the scopeid to the IP address part in a standard fashion,
> > and discarding the flowid part.
> 
> Thanks for this research? As a next step, it would be interesting to
> see how it is done elsewhere (besides C and Python).
> 
> As you may see, I would be really uncomfortable changing this in
> a way that locks out interesting applications, or deviates too far
> from the IPv6 community. So compatibility with the rest of the
> world is of utmost importance.
> 
> If you could collect options, established tradition, pros and cons
> in a document, that would be good.

I've not managed to get a survey of other environments done
so I'd like to ask others on this list to do a little
digging on other languages they are familiar with.

Hackers with knowledge about socket interfaces in Ruby, Perl, Java and
C# or any other high-level socket mapping, I'd like
answers to the following questions:

 * How is the socket addres (or equivalent) abstracted,
   what mechanisms are there to make it convenient for
   code to handle both IPv4 and IPv6 sockets?

 * If there's a high-level semi-automatic DNS interface,
   how does that deal with v4 vs v6? Eg, can I ask
   "make a TCP connection to www.ripe.net port 80, I don't care
   v4 or v6"

 * In what way (if any) are the flowinfo and scopeid 
   parts carried along the with the object doing the
   part of struct sockaddr?

  -- erno


From jimjjewett at gmail.com  Wed Apr 18 21:24:54 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 18 Apr 2007 15:24:54 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
Message-ID: <fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>

On 4/17/07, Guido van Rossum <guido at python.org> wrote:
> On 4/17/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > There are also reasons to want only "local" letters.  For example, in
> > a French interface, I might want to include the extra French letters,
> > but not the Greek.

> The Unicode world doesn't support this directly AFAIK.

Alphabets do tend to be contiguous, but ... not directly, no.

> The locale module doesn't deal with Unicode, only with 8-bit characters (not
> multi-byte characters). You'll lose this anyway. Certainly
> string.letters is not going to provide this functionality.

But for languages in Latin1, 8-bit characters are sufficient --
anything with more than 8 bits is by definition not a (local) letter.

I won't swear that localizations currently replace string.letters with
the appropriately ordered (slight) superset, but it is a valid use
case, and string* (or text*) is clearly the right place.

-jJ

From guido at python.org  Wed Apr 18 23:18:47 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 14:18:47 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
Message-ID: <ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/17/07, Guido van Rossum <guido at python.org> wrote:
> > The locale module doesn't deal with Unicode, only with 8-bit characters (not
> > multi-byte characters). You'll lose this anyway. Certainly
> > string.letters is not going to provide this functionality.
>
> But for languages in Latin1, 8-bit characters are sufficient --
> anything with more than 8 bits is by definition not a (local) letter.

Latin-1 is just another encoding (and not a very useful one given that
it can't encode all of Unicode). I don't want to define a feature that
only works for Latin-1.

> I won't swear that localizations currently replace string.letters with
> the appropriately ordered (slight) superset, but it is a valid use
> case, and string* (or text*) is clearly the right place.

The right solution for locale-dependent collation for sure isn't
having a string containing all the letters in the right order. There
are plenty of languages where that approach doesn't even work.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Apr 18 23:57:57 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 14:57:57 -0700
Subject: [Python-3000] Fixing super anyone?
Message-ID: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>

Is anyone available to write up a PEP on how to turn super into a
keyword? Inside regular and class methods, super.foo(args) should be
equivalent to super(ThisClass, self).foo(args). I think there are ways
to make the old syntax work too, but maybe that's only necessary for
2.6.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Wed Apr 18 23:59:40 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 18 Apr 2007 17:59:40 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
Message-ID: <fb6fbf560704181459x39429901x5f3aa700c57547b9@mail.gmail.com>

On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/17/07, Guido van Rossum <guido at python.org> wrote:
> > > The locale module doesn't deal with Unicode, only with 8-bit characters (not
> > > multi-byte characters). You'll lose this anyway. Certainly
> > > string.letters is not going to provide this functionality.

> > But for languages in Latin1, 8-bit characters are sufficient --
> > anything with more than 8 bits is by definition not a (local) letter.

> Latin-1 is just another encoding (and not a very useful one given that
> it can't encode all of Unicode). I don't want to define a feature that
> only works for Latin-1.

Today, string.letters works most easily with ASCII supersets, and is
effectively limited to 8-bit encodings.  Once everything is unicode, I
don't think that 8-bit restriction should apply any more.

> > I won't swear that localizations currently replace string.letters with
> > the appropriately ordered (slight) superset, but it is a valid use
> > case, and string* (or text*) is clearly the right place.

> The right solution for locale-dependent collation for sure isn't
> having a string containing all the letters in the right order. There
> are plenty of languages where that approach doesn't even work.

Theoretically, English is one of those non-working languages.   (Names
in bibliographic entries are supposed to be alphabetized according to
language of origin.)

In practice, ordered-list-of-chars works well enough, often enough.
It often works better than sorting by code point, which is the only
obvious alternative.

Unless I missed it (and I may have), unicode itself sort of ducks the
question about how to sort strings.  Python really needs to provide
*an* answer, but I'm not sure it is possible to provide the (single)
correct answer.

string.letters is one workaround, and I don't think we should remove
it until a better solution (or workaround) is available.

-jJ

From jimjjewett at gmail.com  Thu Apr 19 00:03:47 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 18 Apr 2007 18:03:47 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
Message-ID: <fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>

On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> Is anyone available to write up a PEP on how to turn super into a
> keyword? Inside regular and class methods, super.foo(args) should be
> equivalent to super(ThisClass, self).foo(args). I think there are ways
> to make the old syntax work too, but maybe that's only necessary for
> 2.6.

Does this mean it should find the super of the current class (which
isn't yet defined when the method is being defined)?

That would be a slight change (though probably an improvement) against
today's lookup-by-name.

If that change is OK, and no one else volunteers, I'll try to take a
go at it this weekend.

-jJ

From guido at python.org  Thu Apr 19 00:08:29 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 15:08:29 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
Message-ID: <ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > On 4/17/07, Guido van Rossum <guido at python.org> wrote:
> > > > The locale module doesn't deal with Unicode, only with 8-bit characters (not
> > > > multi-byte characters). You'll lose this anyway. Certainly
> > > > string.letters is not going to provide this functionality.
>
> > > But for languages in Latin1, 8-bit characters are sufficient --
> > > anything with more than 8 bits is by definition not a (local) letter.
>
> > Latin-1 is just another encoding (and not a very useful one given that
> > it can't encode all of Unicode). I don't want to define a feature that
> > only works for Latin-1.
>
> Today, string.letters works most easily with ASCII supersets, and is
> effectively limited to 8-bit encodings.  Once everything is unicode, I
> don't think that 8-bit restriction should apply any more.

But we already went over this. There are over 40K letters in Unicode.
It simply makes no sense to have a string.letters approaching that
size.

> > > I won't swear that localizations currently replace string.letters with
> > > the appropriately ordered (slight) superset, but it is a valid use
> > > case, and string* (or text*) is clearly the right place.
>
> > The right solution for locale-dependent collation for sure isn't
> > having a string containing all the letters in the right order. There
> > are plenty of languages where that approach doesn't even work.
>
> Theoretically, English is one of those non-working languages.   (Names
> in bibliographic entries are supposed to be alphabetized according to
> language of origin.)
>
> In practice, ordered-list-of-chars works well enough, often enough.
> It often works better than sorting by code point, which is the only
> obvious alternative.
>
> Unless I missed it (and I may have), unicode itself sort of ducks the
> question about how to sort strings.  Python really needs to provide
> *an* answer, but I'm not sure it is possible to provide the (single)
> correct answer.

The Unicode standard certainly has a solution, but it is complicated
and I don't believe it is currently implemented in core Python.

> string.letters is one workaround, and I don't think we should remove
> it until a better solution (or workaround) is available.

I disagree. The correct solution is to implement the Unicode support
for locale-specific sorting.

Remember that the locale module supports only a single, global locale
at a time. This renders it totally useless in many apps requiring
locale support (such as web servers).

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 19 00:15:49 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 15:15:49 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
Message-ID: <ca471dc20704181515l30273010xccab92b3e6caa876@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > Is anyone available to write up a PEP on how to turn super into a
> > keyword? Inside regular and class methods, super.foo(args) should be
> > equivalent to super(ThisClass, self).foo(args). I think there are ways
> > to make the old syntax work too, but maybe that's only necessary for
> > 2.6.
>
> Does this mean it should find the super of the current class (which
> isn't yet defined when the method is being defined)?
>
> That would be a slight change (though probably an improvement) against
> today's lookup-by-name.

Right. It should definitely use the current class and not the name of
the current class. I hope you'll be able to find a way to implement
this! Perhaps the class could be gotten from a hidden cell reference,
where the cell is filled in by the metaclass whe the class is created.
(The cell can't represent the result of the super call, because that
also incorporates the current value of 'self', for which you can use
the first argument.)

So the syntax "super" could be translated into "super(__thisclass__,
self)" where __thisclass__ is the cell reference, self refers to the
(current or original?) value of the first argument, and super is
exactly the same as the 2.x super function. (Though it could be
referenced via an opcode.)

> If that change is OK, and no one else volunteers, I'll try to take a
> go at it this weekend.

Thanks!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Apr 19 00:22:56 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 19 Apr 2007 10:22:56 +1200
Subject: [Python-3000] string module trimming
In-Reply-To: <f039of$ih$1@sea.gmane.org>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
Message-ID: <46269A40.3030602@canterbury.ac.nz>

Christian Heimes wrote:

> What do you think about replacing the definitions by information from
> the unicode character properties database.

I'm not sure I like the idea of pulling a rather large
database into the basic string module.

Also, even in a unicode world, it can be useful to
deal just with ascii letters and digits, e.g. when
processing a programming language.

--
Greg

From guido at python.org  Thu Apr 19 00:34:06 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 15:34:06 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <46269A40.3030602@canterbury.ac.nz>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<173F3FBB-3473-4DA1-8E5A-826C6554DB66@python.org>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org> <46269A40.3030602@canterbury.ac.nz>
Message-ID: <ca471dc20704181534v3160f030tdbdf2e0967bf9f6@mail.gmail.com>

On 4/18/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Christian Heimes wrote:
>
> > What do you think about replacing the definitions by information from
> > the unicode character properties database.
>
> I'm not sure I like the idea of pulling a rather large
> database into the basic string module.

Agreed. Though most of the database is loaded as part of the unicode
string object anyway, to support isupper() etc.

> Also, even in a unicode world, it can be useful to
> deal just with ascii letters and digits, e.g. when
> processing a programming language.

Right, I like having ascii_letters etc. (though I don't care about
where it lives or this particular name -- only about the
functionality).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Apr 19 00:41:40 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 19 Apr 2007 10:41:40 +1200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
Message-ID: <46269EA4.3070500@canterbury.ac.nz>

Guido van Rossum wrote:
> Is anyone available to write up a PEP on how to turn super into a
> keyword? Inside regular and class methods, super.foo(args) should be
> equivalent to super(ThisClass, self).foo(args).

As I'm still not convinced that super() is very
useful in the first place, I'm not sure I want
it made into a keyword at all.

I'd rather see syntactic support for making a
normal inherited method call without danger of
using the wrong class, e.g. if you change a
base class but forget to update inherited calls
to match.

--
Greg

From guido at python.org  Thu Apr 19 00:54:09 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 15:54:09 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <46269EA4.3070500@canterbury.ac.nz>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<46269EA4.3070500@canterbury.ac.nz>
Message-ID: <ca471dc20704181554j44567889v50b5a0f3f5418d13@mail.gmail.com>

On 4/18/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
> > Is anyone available to write up a PEP on how to turn super into a
> > keyword? Inside regular and class methods, super.foo(args) should be
> > equivalent to super(ThisClass, self).foo(args).
>
> As I'm still not convinced that super() is very
> useful in the first place, I'm not sure I want
> it made into a keyword at all.

Well, I and others *do* find it useful (indispensible, in fact) for
writing classes that support cooperative multiple inheritance. That's
not a sport for casual users, but if you need it, super is the only
game in town.

> I'd rather see syntactic support for making a
> normal inherited method call without danger of
> using the wrong class, e.g. if you change a
> base class but forget to update inherited calls
> to match.

You will get that too with the proposal.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Thu Apr 19 01:08:59 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 18 Apr 2007 19:08:59 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<ee2a432c0704171027l412cbe9y664acf114d737c6c@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
Message-ID: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>

On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:

> > Today, string.letters works most easily with ASCII supersets, and is
> > effectively limited to 8-bit encodings.  Once everything is unicode, I
> > don't think that 8-bit restriction should apply any more.

> But we already went over this. There are over 40K letters in Unicode.
> It simply makes no sense to have a string.letters approaching that
> size.

Agreed.  But there aren't 40K (alphabetic) letters in any particular
locale.  Most individual languages will have less than 100.

As a proxy for measuring "local" characters, I'll note that during
some optimization drives for Pango (e.g.,
http://primates.ximian.com/~federico/news-2005-11.html#04 ) it turned
out that there were only two non C-J-K languages that needed more than
256 cache positions in their character glyph tables.

> > Unless I missed it (and I may have), unicode itself sort of ducks the
> > question about how to sort strings.  Python really needs to provide
> > *an* answer, but I'm not sure it is possible to provide the (single)
> > correct answer.

> The Unicode standard certainly has a solution, but it is complicated
> and I don't believe it is currently implemented in core Python.

I guess you're right; I saw too many alternatives the last time I
looked, and must have stopped reading http://unicode.org/reports/tr10/
after section 1, where it becomes obvious that there is no
context-free right answer.

> > string.letters is one workaround, and I don't think we should remove
> > it until a better solution (or workaround) is available.

> I disagree. The correct solution is to implement the Unicode support
> for locale-specific sorting.

And set-inclusion.

I'm not convinced that waiting for such a heavyweight solution is
really the best choice, particularly since the spec itself warns
against using the strictest forms (too inefficient).

> Remember that the locale module supports only a single, global locale
> at a time. This renders it totally useless in many apps requiring
> locale support (such as web servers).

Fair enough.

-jJ

From guido at python.org  Thu Apr 19 01:16:43 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Apr 2007 16:16:43 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
Message-ID: <ca471dc20704181616x4898fb74w17f731440c96b282@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> > > Today, string.letters works most easily with ASCII supersets, and is
> > > effectively limited to 8-bit encodings.  Once everything is unicode, I
> > > don't think that 8-bit restriction should apply any more.
>
> > But we already went over this. There are over 40K letters in Unicode.
> > It simply makes no sense to have a string.letters approaching that
> > size.
>
> Agreed.  But there aren't 40K (alphabetic) letters in any particular
> locale.  Most individual languages will have less than 100.

Isn't that excluding the written language of half the world population
(at least China, Korea and Japan)?

> As a proxy for measuring "local" characters, I'll note that during
> some optimization drives for Pango (e.g.,
> http://primates.ximian.com/~federico/news-2005-11.html#04 ) it turned
> out that there were only two non C-J-K languages that needed more than
> 256 cache positions in their character glyph tables.

But here we're talking features, not optimizations. I really don't
think it's a good idea to propose a feature that can't be used
reasonably for CJK languages.

> > > Unless I missed it (and I may have), unicode itself sort of ducks the
> > > question about how to sort strings.  Python really needs to provide
> > > *an* answer, but I'm not sure it is possible to provide the (single)
> > > correct answer.
>
> > The Unicode standard certainly has a solution, but it is complicated
> > and I don't believe it is currently implemented in core Python.
>
> I guess you're right; I saw too many alternatives the last time I
> looked, and must have stopped reading http://unicode.org/reports/tr10/
> after section 1, where it becomes obvious that there is no
> context-free right answer.
>
> > > string.letters is one workaround, and I don't think we should remove
> > > it until a better solution (or workaround) is available.
>
> > I disagree. The correct solution is to implement the Unicode support
> > for locale-specific sorting.
>
> And set-inclusion.

For set-inclusion we already have isalpha() etc. That should be
enough. I really don't see much of a use case for inquiries of the
type "is this a letter in my locale" -- by the time you are doing
that, you probably are only thinking of one specific locale, and then
you should just reject non-locale charaters altogether rather than
treating them as punctuation.

> I'm not convinced that waiting for such a heavyweight solution is
> really the best choice, particularly since the spec itself warns
> against using the strictest forms (too inefficient).
>
> > Remember that the locale module supports only a single, global locale
> > at a time. This renders it totally useless in many apps requiring
> > locale support (such as web servers).
>
> Fair enough.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jyasskin at gmail.com  Thu Apr 19 08:19:26 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 18 Apr 2007 23:19:26 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
Message-ID: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > But we already went over this. There are over 40K letters in Unicode.
> > It simply makes no sense to have a string.letters approaching that
> > size.
>
> Agreed.  But there aren't 40K (alphabetic) letters in any particular
> locale.  Most individual languages will have less than 100.

I missed the beginning of this discussion, so sorry if you've already
covered this. Are you saying that in your app, just because I've set
the en_US locale, I won't be able to type "????"? Or that those
characters won't be recognized as letters?

The Unicode character database (http://www.unicode.org/ucd/) seems
like the obvious way to handle character properties if you want to get
the right answers.

Namast?,
Jeffrey Yasskin

From jcarlson at uci.edu  Thu Apr 19 08:50:17 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 18 Apr 2007 23:50:17 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com>
References: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com>
Message-ID: <20070418233622.6344.JCARLSON@uci.edu>


"Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > But we already went over this. There are over 40K letters in Unicode.
> > > It simply makes no sense to have a string.letters approaching that
> > > size.
> >
> > Agreed.  But there aren't 40K (alphabetic) letters in any particular
> > locale.  Most individual languages will have less than 100.
> 
> I missed the beginning of this discussion, so sorry if you've already
> covered this. Are you saying that in your app, just because I've set
> the en_US locale, I won't be able to type "????"?  Or that those
> characters won't be recognized as letters?

If I understand the conversation correctly, the discussion is what will
be in string.letters, and what will be handled in str.upper(), etc.,
when a locale is set.


> The Unicode character database (http://www.unicode.org/ucd/) seems
> like the obvious way to handle character properties if you want to get
> the right answers.

Certainly, but having 40k characters in string.letters seems like a bit
of overkill, for *any* locale.  It seems as though it only makes sense
to include the letters for the current locale as string.letters, and to
handle str.upper(), etc., as determined by the locale.

In terms of sorting, since all (unicode) strings should be comparable to
one another, using the unicode-specified ordering would seem to make
sense, unless it is something other than code point values.  If it isn't
code point values (which seems to be the implication), then we need to
decide if we want to check a 128kbyte table (for UCS-2 builds) in order
to sort strings (though cache lookup locality may make this a moot point
for most comparisons).

 - Josiah


From jyasskin at gmail.com  Thu Apr 19 17:14:15 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Thu, 19 Apr 2007 08:14:15 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <20070418233622.6344.JCARLSON@uci.edu>
References: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com>
	<20070418233622.6344.JCARLSON@uci.edu>
Message-ID: <5d44f72f0704190814t729d4149p43847449db523f14@mail.gmail.com>

On 4/18/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> "Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> > I missed the beginning of this discussion, so sorry if you've already
> > covered this. Are you saying that in your app, just because I've set
> > the en_US locale, I won't be able to type "????"?  Or that those
> > characters won't be recognized as letters?
>
> If I understand the conversation correctly, the discussion is what will
> be in string.letters, and what will be handled in str.upper(), etc.,
> when a locale is set.

string.letters should go away because I don't know of any correct uses
of it, and as you say 40K letters is too long. Searching a list is the
wrong way to decide whether a character is a letter, and case
transformations don't work a character at a time (consider what
happens with "?".upper() (That is, U+00DF, German Small Sharp S)).
http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt defines the
mappings that aren't 1-1. There are some that are locale-specific, but
you can do a pretty good job ignoring the language, as long as you
allow strings to change length.

> > The Unicode character database (http://www.unicode.org/ucd/) seems
> > like the obvious way to handle character properties if you want to get
> > the right answers.
>
> Certainly, but having 40k characters in string.letters seems like a bit
> of overkill, for *any* locale.  It seems as though it only makes sense
> to include the letters for the current locale as string.letters, and to
> handle str.upper(), etc., as determined by the locale.

As far as I understand, "letters for the current locale" is the same
as "letters" in Unicode. Can you point me to a character that is a
letter in one locale but not in another? (The third column of
http://www.unicode.org/Public/UNIDATA/UnicodeData.txt defines the
character's category, and
http://www.unicode.org/Public/UNIDATA/UCD.html#General_Category_Values
says what it means.)

> In terms of sorting, since all (unicode) strings should be comparable to
> one another, using the unicode-specified ordering would seem to make
> sense, unless it is something other than code point values.  If it isn't
> code point values (which seems to be the implication), then we need to
> decide if we want to check a 128kbyte table (for UCS-2 builds) in order
> to sort strings (though cache lookup locality may make this a moot point
> for most comparisons).

If you just need to store strings in an order-based data structure
(which I guess is moot for python with its hashes), then codepoint
order is fine. If you intend to show users a sorted list, then you
have to use the real collation algorithm or you'll produce the wrong
answer. I don't understand the algorithm's details, but ICU has an
implementation, and http://icu-project.org/charts/icu4c_footprint.html
claims that the data for all languages fits in 354K.

UCS-2 is an old and broken fixed-width encoding that cannot represent
characters above U+FFFF. Nobody should ever use it. You probably meant
UTF-16.

-- 
Namast?,
Jeffrey Yasskin

From jcarlson at uci.edu  Thu Apr 19 19:22:13 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 19 Apr 2007 10:22:13 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <5d44f72f0704190814t729d4149p43847449db523f14@mail.gmail.com>
References: <20070418233622.6344.JCARLSON@uci.edu>
	<5d44f72f0704190814t729d4149p43847449db523f14@mail.gmail.com>
Message-ID: <20070419091246.6347.JCARLSON@uci.edu>


"Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> On 4/18/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > "Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> > > I missed the beginning of this discussion, so sorry if you've already
> > > covered this. Are you saying that in your app, just because I've set
> > > the en_US locale, I won't be able to type "????"?  Or that those
> > > characters won't be recognized as letters?
> >
> > If I understand the conversation correctly, the discussion is what will
> > be in string.letters, and what will be handled in str.upper(), etc.,
> > when a locale is set.
> 
> string.letters should go away because I don't know of any correct uses
> of it, and as you say 40K letters is too long. Searching a list is the
> wrong way to decide whether a character is a letter, and case
> transformations don't work a character at a time (consider what
> happens with "??".upper() (That is, U+00DF, German Small Sharp S)).
> http://www.unicode.org/Public/UNIDATA/SpecialCasing.txt defines the
> mappings that aren't 1-1. There are some that are locale-specific, but
> you can do a pretty good job ignoring the language, as long as you
> allow strings to change length.

Because we aren't mutating unicode strings, this isn't an issue.  I
respond below regarding string.letters .


> > > The Unicode character database (http://www.unicode.org/ucd/) seems
> > > like the obvious way to handle character properties if you want to get
> > > the right answers.
> >
> > Certainly, but having 40k characters in string.letters seems like a bit
> > of overkill, for *any* locale.  It seems as though it only makes sense
> > to include the letters for the current locale as string.letters, and to
> > handle str.upper(), etc., as determined by the locale.
> 
> As far as I understand, "letters for the current locale" is the same
> as "letters" in Unicode. Can you point me to a character that is a
> letter in one locale but not in another? (The third column of
> http://www.unicode.org/Public/UNIDATA/UnicodeData.txt defines the
> character's category, and
> http://www.unicode.org/Public/UNIDATA/UCD.html#General_Category_Values
> says what it means.)

Neither I, nor I believe Python mean 'letters' in the general sense, but
the 'alphabet' of a particular locale.  For example, en_US compared to
sv_SE .


> > In terms of sorting, since all (unicode) strings should be comparable to
> > one another, using the unicode-specified ordering would seem to make
> > sense, unless it is something other than code point values.  If it isn't
> > code point values (which seems to be the implication), then we need to
> > decide if we want to check a 128kbyte table (for UCS-2 builds) in order
> > to sort strings (though cache lookup locality may make this a moot point
> > for most comparisons).
> 
> If you just need to store strings in an order-based data structure
> (which I guess is moot for python with its hashes), then codepoint
> order is fine. If you intend to show users a sorted list, then you
> have to use the real collation algorithm or you'll produce the wrong
> answer. I don't understand the algorithm's details, but ICU has an
> implementation, and http://icu-project.org/charts/icu4c_footprint.html
> claims that the data for all languages fits in 354K.

It could probably even be reduced lower than 354K with two tables and a
comparison function that knows how to handle surrogates.


> UCS-2 is an old and broken fixed-width encoding that cannot represent
> characters above U+FFFF. Nobody should ever use it. You probably meant
> UTF-16.

You are more or less right.  Earlier versions of Windows were limited to
UCS-2, and I believe earlier versions of Python on Windows were also
limited to UCS-2.  For narrow builds we use UTF-16, with surrogate pairs
and everything (though a unicode string consisting of a single surrogate
pair will have length 2, not 1 as would be expected).


 - Josiah


From guido at python.org  Thu Apr 19 19:40:41 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Apr 2007 10:40:41 -0700
Subject: [Python-3000] [Python-Dev] python3k change to slicing
In-Reply-To: <f0895c$s0o$1@sea.gmane.org>
References: <f0895c$s0o$1@sea.gmane.org>
Message-ID: <ca471dc20704191040w1619e287kddf35abf42e316bd@mail.gmail.com>

[+python-3000; followups please remove python-dev]

-1

While this may be theoretically preferable, I believe that in practice
changing this would be a major pain for very little gain. I don't
recall ever finding a bug related to this feature, and I believe it's
occasionally useful.

Here's something that would be much more cumbersome with your proposed
change: suppose I have a string of unknown length and I want to get
the first three characters, or less if it's not that long. Today I can
write s[:3]. With your proposal I would have to write s[:min(3,
len(s))].

--Guido

On 4/19/07, Neal Becker <ndbecker2 at gmail.com> wrote:
> There is one thing I'd like to see changed in a future python.  I always
> found it surprising, that
> >>> x = [1,2,3,4,5]
> >>> x[1:10]
> [2, 3, 4, 5]
>
> is not an error.  This is perhaps the only case (but a fundamental one!)
> where an error is silently ignored.
>
> I really can't think of a good justification for it.  If I really meant
> x[1:]
> I would have said so.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jason.orendorff at gmail.com  Thu Apr 19 20:51:24 2007
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Thu, 19 Apr 2007 14:51:24 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
Message-ID: <bb8868b90704191151u1ea9e28al93a00fa711d01378@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> > > Today, string.letters works most easily with ASCII supersets, and is
> > > effectively limited to 8-bit encodings.  Once everything is unicode, I
> > > don't think that 8-bit restriction should apply any more.
>
> > But we already went over this. There are over 40K letters in Unicode.
> > It simply makes no sense to have a string.letters approaching that
> > size.
>
> Agreed.  But there aren't 40K (alphabetic) letters in any particular
> locale.  Most individual languages will have less than 100.

But isn't the hip thing these days to use UTF-8 as the character set,
as in "LC_ALL=en_US.UTF-8"?  I picked a random Linux machine
and this happened to be the default LANG.  A quick C test revealed
that iswalpha() thinks there are 45,974 letters in that locale.

Anyway, let's discuss your use cases.


One: UIs that involve displaying all the letters.

> There are also reasons to want only "local" letters.  For example, in
> a French interface, I might want to include the extra French letters,
> but not the Greek.

But POSIX locales don't claim to provide this information,
as far as I can tell.

It seems like for a quick throwaway program, you're better off
ignoring languages.  And for even a slightly serious program,
string.letters is wrong in a dozen ways.

Seriously, a table of alphabets that's saner than string.letters
is pretty trivial to write:

  alphabets = {
      'en': list("ABCDEFGHIJKLMNOPQRSTUVWXYZ"),
      'es': ("A B C Ch D E F G H I J K L Ll M "
          + "N \u00d1 O P Q R S T U V W X Y Z").split(),
      ...
      }

...and you can go from there.


Two: Collation.

Collation can be done right: provide a function text.sort_key()
that converts a str into an opaque thing that has the desired
ordering.  You would use it like so:

  names.sort(key=text.sort_key)
  records.sort(key=lambda rec: text.sort_key(rec.title))

Jython can implement this using java.text.Collator.  IronPython can
use CultureInfo.CompareInfo.GetSortKey().  CPython can call
LCMapString() on Windows and wcsxfrm() on POSIX, falling back on just
returning the string itself.

-j

From jimjjewett at gmail.com  Thu Apr 19 20:52:00 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 19 Apr 2007 14:52:00 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <20070418233622.6344.JCARLSON@uci.edu>
References: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com>
	<20070418233622.6344.JCARLSON@uci.edu>
Message-ID: <fb6fbf560704191152u4d1d0494rd6ca2730fad86694@mail.gmail.com>

On 4/19/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > > > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:

> > > Agreed.  But there aren't 40K (alphabetic) letters in any particular
> > > locale.  Most individual languages will have less than 100.

> > ... Are you saying that in your app, just because I've set
> > the en_US locale, I won't be able to type "????"?  Or that those
> > characters won't be recognized as letters?

The latter.  Some applications may reject them for that reason; for
example some domain registrars have policies to prevent domain name
spoofing with similar-looking characters.  One way to do that is to
say that a character used in a domain name (under that registrar) is
limited to those letters used by the appropriate national language.

> In terms of sorting, since all (unicode) strings should be comparable to
> one another, using the unicode-specified ordering would seem to make
> sense, unless it is something other than code point values.

It is definately something other than code-point values.

In particular, see section 1.8 (common misconceptions) of
http://unicode.org/reports/tr10/

The sorting isn't fully defined without locale-specific tailoring and
a Unicode Element Collation Table (default 4 bytes/char, though
compressible).  There is a default tailoring and default Unicode
Element Collation Table; it looks (but I haven't proven to myself) as
if  these defaults are sufficient for most use, but certainly not all
usage.

Unicode sorting (even with your own collation table) definately
requires normalization, which is something Python has been careful not
to promise.  (There were some arguments over whether normalization was
even possible to do in a strictly correct fashion.  I didn't
understand them well enough to remember the summary.)  Unless the
"repetoire of supported character sequences" is (unnaturally)
restricted, normalization is only an intermediate step; a third
representation is constructed for the actual comparison.  This third
form can be done a few characters at a time, but then you have to redo
it for the next comparison.

As best I can easily tell about the default settings, there are
distinct strings which are equal, unequal strings which are not
ordered, and strings for which you must compare multiple characters at
once ("x"<"y", but "xz">"yz")

-jJ

From jimjjewett at gmail.com  Thu Apr 19 21:34:25 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 19 Apr 2007 15:34:25 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <bb8868b90704191151u1ea9e28al93a00fa711d01378@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<bb8868b90704191151u1ea9e28al93a00fa711d01378@mail.gmail.com>
Message-ID: <fb6fbf560704191234h5b6a8214p1f17b8588ba730e1@mail.gmail.com>

On 4/19/07, Jason Orendorff <jason.orendorff at gmail.com> wrote:
> On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> Seriously, a table of alphabets that's saner than string.letters
> is pretty trivial to write:

>   alphabets = {
>       'en': list("ABCDEFGHIJKLMNOPQRSTUVWXYZ"),
>       'es': ("A B C Ch D E F G H I J K L Ll M "
>           + "N \u00d1 O P Q R S T U V W X Y Z").split(),
>       ...
>       }

I suspect you could do a bit better with properties already in the
unicode database -- but offhand, I'm not sure how.  If setting locale
put the correct one in string.letters for me, that would be great.

> Two: Collation.

> Collation can be done right: provide a function text.sort_key()
> that converts a str into an opaque thing that has the desired
> ordering.

If this function is context-free (depending on only the input string),
it will violate the unicode standard (and, apparently, do the wrong
thing for some languages -- usually including French).

Also, that key is probably larger than the original string, and they
warn against trying to create it in a single pass.

I'm not saying relying strictly on unicode properties *can't* be done
right -- I'm just saying that it would be very difficult and very
inefficient, so it probably won't happen soon -- which is an argument
for keeping the half-measures around meanwhile.

-jJ

From jimjjewett at gmail.com  Thu Apr 19 21:48:00 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 19 Apr 2007 15:48:00 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <46269EA4.3070500@canterbury.ac.nz>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<46269EA4.3070500@canterbury.ac.nz>
Message-ID: <fb6fbf560704191248w17164e49k7299818b49228b9f@mail.gmail.com>

On 4/18/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> I'd rather see syntactic support for making a
> normal inherited method call without danger of
> using the wrong class, e.g. if you change a
> base class but forget to update inherited calls
> to match.

Could you give an example?  My first three readings of this were

(1)  Huh?
(2)  You mean, if the __bases__ are changed, super doesn't get
updated?  (It does, as I just verified.)
(3)  You mean, with an explicit call, such as B.meth1(self), you want
some sort of warning if class B changes the signature of meth1?
(Presumably at compile-time; an incompatible change will already be
caught at run-time.)

-jJ

From greg.ewing at canterbury.ac.nz  Fri Apr 20 00:43:17 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 20 Apr 2007 10:43:17 +1200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <ca471dc20704181554j44567889v50b5a0f3f5418d13@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<46269EA4.3070500@canterbury.ac.nz>
	<ca471dc20704181554j44567889v50b5a0f3f5418d13@mail.gmail.com>
Message-ID: <4627F085.7000301@canterbury.ac.nz>

Guido van Rossum wrote:
> On 4/18/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> I'd rather see syntactic support for making a
>> normal inherited method call without danger of
>> using the wrong class, e.g. if you change a
>> base class but forget to update inherited calls
>> to match.
> 
> You will get that too with the proposal.

Is there an actual proposal yet? I'd like to
see how this would work.

--
Greg


From guido at python.org  Fri Apr 20 00:55:40 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Apr 2007 15:55:40 -0700
Subject: [Python-3000] Need help completing ABC pep
Message-ID: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>

I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:

    http://www.python.org/dev/peps/pep-3119/

While I'm not ready yet to answer tough questions about this compared
to alternative proposals, I *am* ready for feedback on the various
open issues sprinkled throughout the current text, especially
concerning decisions regarding exactly which operations to include in
the various ABCs, and also regarding the level of detail required in
the PEP.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Fri Apr 20 03:41:39 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Apr 2007 18:41:39 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>

On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
>
>     http://www.python.org/dev/peps/pep-3119/
>
> While I'm not ready yet to answer tough questions about this compared
> to alternative proposals, I *am* ready for feedback on the various
> open issues sprinkled throughout the current text, especially
> concerning decisions regarding exactly which operations to include in
> the various ABCs, and also regarding the level of detail required in
> the PEP.

Since there is no embedded copy of the PEP I am just quoting the
various open issues and such.  That way a simple search of the PEP wil
find what  am talking about.  At the end I have some random comments
that do not directly mention an open issue.


* "perhaps abstractmethod and AbstractInstantiationError should become
built-ins, Abstract's functionality should be subsumed by object, and
AbstractClass's functionality should be merged into type"

Maybe, but what is the performance cost of doing the checking?  If
it's too high I don't want to have to pay for it if I am not using
anything that uses ABCs.  Then again, if the decision is made to run
with this then a whole lot of stdlib code is going to end up with ABCs
and thus the cost will be unavoidable.


* "Should we also implement the issubset and issuperset methods found
on the set type in Python 2? As these are just aliases for __le__ and
__ge__, I'm tempted to leave these out."

Leave them out.  Not terribly needed plus it is better to start out
small.  They can easily be added later if called for.


* Discussing sets: "Open issue: Should this class also implement the
operators and/or methods that compute union, intersection, symmetric
and asymmetric difference?"

If you are willing to let them just return the built-in set type
(since one could argue that 'set' is the base type that almost any set
will inherit from), then go ahead and add them as concrete methods.
Otherwise leave them out.


* "Should we support all the operations implemented by the Python 2 set type?"

No, only core methods should be directly supported.  Everything else
is just convenience functions.  If these can be implemented simply as
concrete methods then fine, but otherwise leave them out.


* "Do we need BasicMapping and IterableMapping? Perhaps we should just
start with Mapping"

Just Mapping.  Doing two checks for Mapping and Iterable is not
exactly complex.  =)


* "Also pop, popitem, setdefault?"

Eh, my gut reaction is "no".  But you do have 'pop' on MutableSequence.


* Discussing sequences: "Other candidate methods, which can all have
default concrete implementations that only depend on __len__ and
__getitem__ with an integer argument: __reversed__, index, count,
__add__, __mul__, __eq__, __lt__, __le__."

I say go ahead and include the concrete implementation.


* Discussing sequences: "What about += and *=?"

No.  Only worth it when a special gain is available by the implementation.


* "Do we have a use case for Cardinal (Integer >= 0)?"

No.


* "Do we need Indexable (converts to Integer using __index__)? Or is
that just subsumed into Integer and should we use __index__ only at
the C level?"

I actually immediately thought of this as a use-case for ABCs when I
started reading this PEP.  So my vote is for Indexable, although the
name doesn't fit as the object doesn't get indexed but can be used as
an index.  So I would rather have Index or IndexItem or something.


* "Should we strive to use ABCs for all areas of Python?"

No.  As you said, Orderable is probably the only thing really missing.
 But if we want some inspiration we can look at what Haskell has.


* "Perhaps the numeric classes could be moved to a separate PEP"

Well, one could argue then that we should have a PEP on how ABCs
should work and then individual PEPs on each group of defined ABCs.  I
suspect that if we go with ABCs we will not take their definitions
lightly and end up haggling over definitions a lot.


Non-open issue comments:

About Iterator, does it subclass Iterable?  It seems to meet the definition.

I really don't like the name of Finite.  When I read the name before
knowing what it represented I really had no clue what it represented.
In hindsight it makes sense, but it not immediately obvious.

What about 'append' on MutableSequence?

-Brett

From guido at python.org  Fri Apr 20 04:35:59 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Apr 2007 19:35:59 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
Message-ID: <ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>

On 4/19/07, Brett Cannon <brett at python.org> wrote:
> On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
> >
> >     http://www.python.org/dev/peps/pep-3119/
> >
> > While I'm not ready yet to answer tough questions about this compared
> > to alternative proposals, I *am* ready for feedback on the various
> > open issues sprinkled throughout the current text, especially
> > concerning decisions regarding exactly which operations to include in
> > the various ABCs, and also regarding the level of detail required in
> > the PEP.
>
> Since there is no embedded copy of the PEP I am just quoting the
> various open issues and such.  That way a simple search of the PEP wil
> find what  am talking about.  At the end I have some random comments
> that do not directly mention an open issue.

Thanks!

> * "perhaps abstractmethod and AbstractInstantiationError should become
> built-ins, Abstract's functionality should be subsumed by object, and
> AbstractClass's functionality should be merged into type"
>
> Maybe, but what is the performance cost of doing the checking?  If
> it's too high I don't want to have to pay for it if I am not using
> anything that uses ABCs.  Then again, if the decision is made to run
> with this then a whole lot of stdlib code is going to end up with ABCs
> and thus the cost will be unavoidable.

Good point. Now that you mention this, and I don't understand why I
didn't already do it this way in sandbox/abc, the check could be made
at *class creation time*, and at instantiation time it should just
have to make one tiny check.

> * "Should we also implement the issubset and issuperset methods found
> on the set type in Python 2? As these are just aliases for __le__ and
> __ge__, I'm tempted to leave these out."
>
> Leave them out.  Not terribly needed plus it is better to start out
> small.  They can easily be added later if called for.

Already gone. I think you read the penultimate version. :-)

> * Discussing sets: "Open issue: Should this class also implement the
> operators and/or methods that compute union, intersection, symmetric
> and asymmetric difference?"
>
> If you are willing to let them just return the built-in set type
> (since one could argue that 'set' is the base type that almost any set
> will inherit from), then go ahead and add them as concrete methods.
> Otherwise leave them out.

I doubt that 'set' will be inherited from much (except for trivial
stuff) but returning a 'set' does make some sense. Although for
trivial subclasses (e.g. adding a different repr or a brand new
method) you'd like the results also to be an instance of the subclass.
If we could find a way to do that it would be best.

> * "Should we support all the operations implemented by the Python 2 set type?"
>
> No, only core methods should be directly supported.  Everything else
> is just convenience functions.  If these can be implemented simply as
> concrete methods then fine, but otherwise leave them out.

Sure. But one can differ about whether union() is the convenience and
__or__() the core, or vice versa. :-)

> * "Do we need BasicMapping and IterableMapping? Perhaps we should just
> start with Mapping"
>
> Just Mapping.  Doing two checks for Mapping and Iterable is not
> exactly complex.  =)

OK, I'll think about this.

> * "Also pop, popitem, setdefault?"
>
> Eh, my gut reaction is "no".  But you do have 'pop' on MutableSequence.

But for sequences it trivially maps on core methods. pop() for
mappings is a little more convoluted (how to pick a random key? Just
pick the first one I guess?)

> * Discussing sequences: "Other candidate methods, which can all have
> default concrete implementations that only depend on __len__ and
> __getitem__ with an integer argument: __reversed__, index, count,
> __add__, __mul__, __eq__, __lt__, __le__."
>
> I say go ahead and include the concrete implementation.

OK.  I was just being lazy.

> * Discussing sequences: "What about += and *=?"
>
> No.  Only worth it when a special gain is available by the implementation.

OTOH not requiring these gives them fuzzy semantics -- some mutable
sequences will return self, others a new object. That's not good. I'd
like to standardize self-modifying +=, *=.

> * "Do we have a use case for Cardinal (Integer >= 0)?"
>
> No.
>
>
> * "Do we need Indexable (converts to Integer using __index__)? Or is
> that just subsumed into Integer and should we use __index__ only at
> the C level?"
>
> I actually immediately thought of this as a use-case for ABCs when I
> started reading this PEP.  So my vote is for Indexable, although the
> name doesn't fit as the object doesn't get indexed but can be used as
> an index.  So I would rather have Index or IndexItem or something.

Index is a good one.

> * "Should we strive to use ABCs for all areas of Python?"
>
> No.  As you said, Orderable is probably the only thing really missing.
>  But if we want some inspiration we can look at what Haskell has.

Pointer? (I'm lazy. :-)

> * "Perhaps the numeric classes could be moved to a separate PEP"
>
> Well, one could argue then that we should have a PEP on how ABCs
> should work and then individual PEPs on each group of defined ABCs.  I
> suspect that if we go with ABCs we will not take their definitions
> lightly and end up haggling over definitions a lot.

Yeah, but IMO the collections are the most contentious area, and doing
these right is a motivation for the whole idea of ABCs -- it makes
more concrete "what ABCs can do for you."

> Non-open issue comments:
>
> About Iterator, does it subclass Iterable?  It seems to meet the definition.

It does, and it says so.

> I really don't like the name of Finite.  When I read the name before
> knowing what it represented I really had no clue what it represented.
> In hindsight it makes sense, but it not immediately obvious.

I have been playing with Sizeable and Lengthy, but rejected both as
being too cute. Other suggestions?

> What about 'append' on MutableSequence?

Oh, duh. It's listed as add. Will fix.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Fri Apr 20 05:53:26 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 19 Apr 2007 20:53:26 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
Message-ID: <bbaeab100704192053ud28fdc1s11deaafc8061b4a5@mail.gmail.com>

On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> On 4/19/07, Brett Cannon <brett at python.org> wrote:
> > On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> > > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
> > >
> > >     http://www.python.org/dev/peps/pep-3119/
> > >
> > > While I'm not ready yet to answer tough questions about this compared
> > > to alternative proposals, I *am* ready for feedback on the various
> > > open issues sprinkled throughout the current text, especially
> > > concerning decisions regarding exactly which operations to include in
> > > the various ABCs, and also regarding the level of detail required in
> > > the PEP.
> >
> > Since there is no embedded copy of the PEP I am just quoting the
> > various open issues and such.  That way a simple search of the PEP wil
> > find what  am talking about.  At the end I have some random comments
> > that do not directly mention an open issue.
>
> Thanks!
>

No problem.  Gave me something to do while I waited for my girlfriend
to get off of work.  =)

> > * "perhaps abstractmethod and AbstractInstantiationError should become
> > built-ins, Abstract's functionality should be subsumed by object, and
> > AbstractClass's functionality should be merged into type"
> >
> > Maybe, but what is the performance cost of doing the checking?  If
> > it's too high I don't want to have to pay for it if I am not using
> > anything that uses ABCs.  Then again, if the decision is made to run
> > with this then a whole lot of stdlib code is going to end up with ABCs
> > and thus the cost will be unavoidable.
>
> Good point. Now that you mention this, and I don't understand why I
> didn't already do it this way in sandbox/abc, the check could be made
> at *class creation time*, and at instantiation time it should just
> have to make one tiny check.
>

I thought you were already doing that.  =)  I was just worried that it
would be prohibitively expensive at class creation time for some short
script that used mostly stdlib code that happened to use ABCs.

> > * "Should we also implement the issubset and issuperset methods found
> > on the set type in Python 2? As these are just aliases for __le__ and
> > __ge__, I'm tempted to leave these out."
> >
> > Leave them out.  Not terribly needed plus it is better to start out
> > small.  They can easily be added later if called for.
>
> Already gone. I think you read the penultimate version. :-)
>

Quite possibly.  Did this all offline.

> > * Discussing sets: "Open issue: Should this class also implement the
> > operators and/or methods that compute union, intersection, symmetric
> > and asymmetric difference?"
> >
> > If you are willing to let them just return the built-in set type
> > (since one could argue that 'set' is the base type that almost any set
> > will inherit from), then go ahead and add them as concrete methods.
> > Otherwise leave them out.
>
> I doubt that 'set' will be inherited from much (except for trivial
> stuff) but returning a 'set' does make some sense. Although for
> trivial subclasses (e.g. adding a different repr or a brand new
> method) you'd like the results also to be an instance of the subclass.
> If we could find a way to do that it would be best.
>

Well, self.__class__() would work, right?  That should resolve
properly.  Or maybe we need that super keyword.  =)

> > * "Should we support all the operations implemented by the Python 2 set type?"
> >
> > No, only core methods should be directly supported.  Everything else
> > is just convenience functions.  If these can be implemented simply as
> > concrete methods then fine, but otherwise leave them out.
>
> Sure. But one can differ about whether union() is the convenience and
> __or__() the core, or vice versa. :-)
>

So I say base it on ease of implementation.  If it can be done
trivially in five lines of Python code then include it, otherwise let
the user do it.

> > * "Do we need BasicMapping and IterableMapping? Perhaps we should just
> > start with Mapping"
> >
> > Just Mapping.  Doing two checks for Mapping and Iterable is not
> > exactly complex.  =)
>
> OK, I'll think about this.
>
> > * "Also pop, popitem, setdefault?"
> >
> > Eh, my gut reaction is "no".  But you do have 'pop' on MutableSequence.
>
> But for sequences it trivially maps on core methods. pop() for
> mappings is a little more convoluted (how to pick a random key? Just
> pick the first one I guess?)
>

Right, which is why my gut reaction was "no".  I don't know how you
choose.  But if you are popping off of a mapping object you should
know the order will be undefined.  I would take your latter
suggestion: get a list of keys, take the first one and use that as
what gets popped off.

> > * Discussing sequences: "Other candidate methods, which can all have
> > default concrete implementations that only depend on __len__ and
> > __getitem__ with an integer argument: __reversed__, index, count,
> > __add__, __mul__, __eq__, __lt__, __le__."
> >
> > I say go ahead and include the concrete implementation.
>
> OK.  I was just being lazy.
>

=)

> > * Discussing sequences: "What about += and *=?"
> >
> > No.  Only worth it when a special gain is available by the implementation.
>
> OTOH not requiring these gives them fuzzy semantics -- some mutable
> sequences will return self, others a new object. That's not good. I'd
> like to standardize self-modifying +=, *=.
>

Then I guess you answered your own question.  If you were asking about
semantics I would just follow what lists do.

> > * "Do we have a use case for Cardinal (Integer >= 0)?"
> >
> > No.
> >
> >
> > * "Do we need Indexable (converts to Integer using __index__)? Or is
> > that just subsumed into Integer and should we use __index__ only at
> > the C level?"
> >
> > I actually immediately thought of this as a use-case for ABCs when I
> > started reading this PEP.  So my vote is for Indexable, although the
> > name doesn't fit as the object doesn't get indexed but can be used as
> > an index.  So I would rather have Index or IndexItem or something.
>
> Index is a good one.
>
> > * "Should we strive to use ABCs for all areas of Python?"
> >
> > No.  As you said, Orderable is probably the only thing really missing.
> >  But if we want some inspiration we can look at what Haskell has.
>
> Pointer? (I'm lazy. :-)
>

Damn, I guess I have to do a little extra work now....

See http://haskell.org/onlinereport/prelude-index.html for the Haskell
98 report and the prelude (Haskell's version of built-ins).  You care
about the type classes (which they just call "classes" on the page).
If you follow the links for any of the type classes they will tell you
exactly what functions must be defined for something to meet that type
class.  Some of them are Haskell-specific, but not all.

> > * "Perhaps the numeric classes could be moved to a separate PEP"
> >
> > Well, one could argue then that we should have a PEP on how ABCs
> > should work and then individual PEPs on each group of defined ABCs.  I
> > suspect that if we go with ABCs we will not take their definitions
> > lightly and end up haggling over definitions a lot.
>
> Yeah, but IMO the collections are the most contentious area, and doing
> these right is a motivation for the whole idea of ABCs -- it makes
> more concrete "what ABCs can do for you."
>

Gotcha.

> > Non-open issue comments:
> >
> > About Iterator, does it subclass Iterable?  It seems to meet the definition.
>
> It does, and it says so.
>

Ah, see it now.

> > I really don't like the name of Finite.  When I read the name before
> > knowing what it represented I really had no clue what it represented.
> > In hindsight it makes sense, but it not immediately obvious.
>
> I have been playing with Sizeable and Lengthy, but rejected both as
> being too cute. Other suggestions?
>

Umm, Sized?  =)  Basically, no, I don't have a better suggestion.

> > What about 'append' on MutableSequence?
>
> Oh, duh. It's listed as add. Will fix.

Ah.  =)  I just figured 'add' was some method to specify where to
insert a value or something.

-Brett

From talin at acm.org  Fri Apr 20 06:54:13 2007
From: talin at acm.org (Talin)
Date: Thu, 19 Apr 2007 21:54:13 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <bbaeab100704192053ud28fdc1s11deaafc8061b4a5@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
	<bbaeab100704192053ud28fdc1s11deaafc8061b4a5@mail.gmail.com>
Message-ID: <46284775.8020806@acm.org>

Brett Cannon wrote:
> On 4/19/07, Guido van Rossum <guido at python.org> wrote:
>> On 4/19/07, Brett Cannon <brett at python.org> wrote:
>>> I really don't like the name of Finite.  When I read the name before
>>> knowing what it represented I really had no clue what it represented.
>>> In hindsight it makes sense, but it not immediately obvious.
>> I have been playing with Sizeable and Lengthy, but rejected both as
>> being too cute. Other suggestions?
>>
> 
> Umm, Sized?  =)  Basically, no, I don't have a better suggestion.

Hmmm.

"Countable" or "Enumerable" come to mind. "Enumerable" might be 
confusing, however, since some languages use that word to mean "Iterable".

Other words that come to mind are "Dimension" and "Extent", although I'm 
not sure how to turn them into adjectives.

-- Talin


From jyasskin at gmail.com  Fri Apr 20 09:48:28 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Fri, 20 Apr 2007 00:48:28 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com>

On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
>
>     http://www.python.org/dev/peps/pep-3119/
>
> While I'm not ready yet to answer tough questions about this compared
> to alternative proposals, I *am* ready for feedback on the various
> open issues sprinkled throughout the current text, especially
> concerning decisions regarding exactly which operations to include in
> the various ABCs, and also regarding the level of detail required in
> the PEP.

I have a bunch of comments that aren't all related to the open issues.
I'm more familiar with statically typed languages, so sorry if some of
these comments don't make sense in a Python context.

Hashable might want to specify the exception that should be thrown for
mutable objects.

I wouldn't have guessed what Finite was from the name, but I don't
mind it now that I know. I do prefer 'Sized'.

A possible name for the __contains__ ABC for sequences and strings is
'Searchable'. On the other hand, an annotation of some sort for the
running time of the method might do as well, but I don't know of any
prior art for that.


On Sets:
Do you have an example of a data structure that is a Set but not a
ComposableSet?

Set should inherit from a PartiallyOrdered ABC that defines the
comparison operators.

The final specification of Set should spell out the invariants, but
they're clear enough that the PEP probably doesn't need to. It's odd,
though, that you mention optimizations to __eq__ and __le__ but not
their semantics.


ComposableSet's open issues:
> Should we define an API for creating new instances (e.g. a class method or a fixed constructor signature)?

There are some set types that can't be constructed from arbitrary
sequences. For example, Patricia Tries
(http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-IntSet.html)
can be used to make very efficient sets of integers, but only
integers.

> Should we just pick a concrete return type (e.g. set)?

For set compositions, you should say whether the arguments need to be
uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do,
then you don't need a concrete return type; if they don't, then you
do, and the default implementations should use it. Even this may be a
problem if someone writes a TreeSet which requires
TotallyOrdered-but-not-Hashable elements. In any case, the ABC should
encourage implementors to return an instance of the same type as the
arguments when the arguments are of the same type.

> Should we add the copy method?
Not on ComposableSet, but maybe in a Copyable ABC, which it might inherit from.


MutableSet:
> Should we support more operations implemented by the Python 2 set type?

You have groups of equivalent operations (add, update, __ior__),
(discard, difference_update, __isub__), (??, intersection_update,
__iand__), and (??, symmetric_difference_update, __ixor__)

It would be nice to make these all uniform. Intersection is hard to
support one element at a time, but symmetric difference would work
with a toggle() method. The operations that work a group at a time are
likely be more efficient on Sized arguments, and much more efficient
on arguments of the same type. How much are you trying to support
these kinds of optimizations to implementations?

> Should we unify remove and discard, a la Java (which has a single method returning a boolean indicating whether it was removed or not)?

Yes. :)


Mappings:
Set-like unions, intersections, and differences are useful on maps
too, but you usually have to provide a function to say what to do with
colliding values. This is probably a different PEP though.


Sequence:
I guess you've already dealt with this, but allowing __getitem__ with
integer indices on str objects forces you to use UTF-32 or play
interesting games with caching. Most string algorithms are quite happy
with (bidirectional? multipass?) iterators, but programmers aren't
used to using them.


Numbers:
Haskell might be good to use for inspiration on the numeric classes:
http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#7.
Note that Haskell's "Real" doesn't imply that the number is
fractional, just that it's not complex. Some differences I see from
Bill Janssen's proposal:
 * A Haskell Num is constructible from any Integer (python's long).
 * A Haskell Fractional (including Complex) is constructible from any
Rational (which can be constructed from a float)
 * The operations are spread into more classes. I think that primarily
helps Complex and Fixed-point types fit into the hierarchy better. The
Haskell folks actually complain that the operations aren't spread into
_enough_ classes. They mostly want more group-theory related classes
below Num.
 * The bitwise operators are segregated into a Bits class, but I don't
see any benefit to that.
 * In a possible use for Cardinal, Integral**Cardinal=>Integral,
Fractional**Integral=>Fractional, and Floating**Floating=>Floating.
 * Complex isn't Orderable

Fortress (http://research.sun.com/projects/plrg/fortress.pdf#page=270)
has an extremely detailed hierarchy of algebraic traits, and it looks
like they plan a similarly detailed hierarchy of numbers, but the
numbers aren't yet specified beyond integers and rationals.

Cardinal is also useful as the argument to (Sequence * Cardinal).

Index (Ix) is used in Haskell primarily in the form of tuples of Ints
for multi-dimensional arrays.

-- 
Namast?,
Jeffrey Yasskin

From jan.grant at bristol.ac.uk  Fri Apr 20 11:21:39 2007
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Fri, 20 Apr 2007 10:21:39 +0100 (BST)
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <46284775.8020806@acm.org>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
	<bbaeab100704192053ud28fdc1s11deaafc8061b4a5@mail.gmail.com>
	<46284775.8020806@acm.org>
Message-ID: <20070420101924.B1402@tribble.ilrt.bris.ac.uk>

On Thu, 19 Apr 2007, Talin wrote:

> Brett Cannon wrote:
> > On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> >> On 4/19/07, Brett Cannon <brett at python.org> wrote:
> >>> I really don't like the name of Finite.  When I read the name before
> >>> knowing what it represented I really had no clue what it represented.
> >>> In hindsight it makes sense, but it not immediately obvious.
> >> I have been playing with Sizeable and Lengthy, but rejected both as
> >> being too cute. Other suggestions?
> >>
> > 
> > Umm, Sized?  =)  Basically, no, I don't have a better suggestion.
> 
> Hmmm.
> 
> "Countable" or "Enumerable" come to mind. "Enumerable" might be 
> confusing, however, since some languages use that word to mean "Iterable".
> 
> Other words that come to mind are "Dimension" and "Extent", although I'm 
> not sure how to turn them into adjectives.

"Bounded?" I was going to suggest that all reifications of 
"infinite sequences" in a running program are, in practice, finite but 
unbounded.

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Whose kung-fu is the best?

From ncoghlan at gmail.com  Fri Apr 20 11:49:54 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 20 Apr 2007 19:49:54 +1000
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
Message-ID: <46288CC2.30304@gmail.com>

Guido van Rossum wrote:
> On 4/19/07, Brett Cannon <brett at python.org> wrote:
>> On 4/19/07, Guido van Rossum <guido at python.org> wrote:
>> * "Also pop, popitem, setdefault?"
>>
>> Eh, my gut reaction is "no".  But you do have 'pop' on MutableSequence.
> 
> But for sequences it trivially maps on core methods. pop() for
> mappings is a little more convoluted (how to pick a random key? Just
> pick the first one I guess?)

pop() on dict requires you to say which key to pop, so a concrete 
Mapping.pop might look something like:

_sentinel=object()
def pop(self, k, d=_sentinel):
     try:
         v = self[k]
     except KeyError:
         if d is _sentinel:
             raise
         v = d
     else:
         del self[k]
     return v

>> I really don't like the name of Finite.  When I read the name before
>> knowing what it represented I really had no clue what it represented.
>> In hindsight it makes sense, but it not immediately obvious.
> 
> I have been playing with Sizeable and Lengthy, but rejected both as
> being too cute. Other suggestions?

Sized? Still not wonderful, but probably easier to figure out and 
remember than Finite.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Fri Apr 20 11:59:45 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 20 Apr 2007 19:59:45 +1000
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <46288F11.3030505@gmail.com>

Guido van Rossum wrote:
> I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
> 
>     http://www.python.org/dev/peps/pep-3119/
> 
> While I'm not ready yet to answer tough questions about this compared
> to alternative proposals, I *am* ready for feedback on the various
> open issues sprinkled throughout the current text, especially
> concerning decisions regarding exactly which operations to include in
> the various ABCs, and also regarding the level of detail required in
> the PEP.
> 

It's not listed as an open issue, but when looking at the list of ABCs, 
I noticed a potential solution to a different problem we have 
occasionally wrestled with: the unfortunate side-effects of having a 
default __hash__ method defined on object.

If the details of collections.Hashable.__hash__ are declared to be 
implementation dependent rather than having it always return 0, then we 
can get rid of object.__hash__.

For CPython, collections.Hashable.__hash__ would then just invoke id() 
on the object, and other implementations can also do whatever they 
currently do for object.__hash__

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From michele.simionato at gmail.com  Fri Apr 20 12:07:55 2007
From: michele.simionato at gmail.com (Michele Simionato)
Date: Fri, 20 Apr 2007 10:07:55 +0000 (UTC)
Subject: [Python-3000] Fixing super anyone?
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<ca471dc20704181515l30273010xccab92b3e6caa876@mail.gmail.com>
Message-ID: <loom.20070420T120626-51@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:
> On 4/18/07, Jim Jewett <jimjjewett <at> gmail.com> wrote:
> > Does this mean it should find the super of the current class (which
> > isn't yet defined when the method is being defined)?

> Right. It should definitely use the current class and not the name of
> the current class. I hope you'll be able to find a way to implement
> this! Perhaps the class could be gotten from a hidden cell reference,
> where the cell is filled in by the metaclass whe the class is created.
> (The cell can't represent the result of the super call, because that
> also incorporates the current value of 'self', for which you can use
> the first argument.)
> 
> So the syntax "super" could be translated into "super(__thisclass__,
> self)" where __thisclass__ is the cell reference, self refers to the
> (current or original?) value of the first argument, and super is
> exactly the same as the 2.x super function. (Though it could be
> referenced via an opcode.)

Here is an implentation based on bytecode hacks that works in current Python:
http://groups.google.com/group/comp.lang.python/browse_frm/thread/a6010c7494871bb1/62a2da68961caeb6?lnk=gst&q=simionato+challenge&rnum=1&hl=en#62a2da68961caeb6


 Michele Simionato


From thomas at python.org  Fri Apr 20 13:56:36 2007
From: thomas at python.org (Thomas Wouters)
Date: Fri, 20 Apr 2007 13:56:36 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
Message-ID: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>

On 4/19/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > Is anyone available to write up a PEP on how to turn super into a
> > keyword? Inside regular and class methods, super.foo(args) should be
> > equivalent to super(ThisClass, self).foo(args). I think there are ways
> > to make the old syntax work too, but maybe that's only necessary for
> > 2.6.
>
> Does this mean it should find the super of the current class (which
> isn't yet defined when the method is being defined)?
>
> That would be a slight change (though probably an improvement) against
> today's lookup-by-name.
>
> If that change is OK, and no one else volunteers, I'll try to take a
> go at it this weekend.


I've had this near the top of my (Python) TODO list for a while, but I
haven't been able to find the time. I've considered it while doing the
dishes and such, though. I can think of two ways of doing the underlying
magic "properly", and one way that's too ugly to think about:

 1) Add a new type of descriptor-hook, for associating an object with the
class it is an attribute of. After class creation (in the metaclass
__init__), any object with this __associate__ (or whatever) hook gets it
called. It's only called for objects that are attributes of *that* class,
not of any superclasses (since they will already be associated with the
superclass.) I'm sure there are other uses for this hook, just not methods
that want to use super() -- like zope interfaces ;)

 2) Add a new argument to __get__ (or a new method that acts like __get__
but with the extra argument, and is called instead of __get__ if it is
defined) where the extra argument is the class that the retrieved descriptor
is actually an attribute of. It currently (correctly) gets the class the
attribute was retrieved through, which could be a subclass.

In both these cases, the (un)bound method object would end up with knowledge
of the class it is defined in, and create a local variable with a
super-object for the 'super' keyword to use. There still are some grey areas
to solve -- what if classes share functions, what if functions share
code-objects, etc. #2 is probably a better way to go about it than #1, in
that regard.

(and 3) the fugly method: don't do anything special with methods, but have
super() search through __mro__ for the first class to have the current
function as an attribute, then pick the next class from that. Yecch, and
more grey areas than the other two methods.)

There's also the question of what the super keyword itself should look like,
e.g.

  # declaration-style, no dot
  res = super currentmethod(arg, arg)
  # treat super like self
  res = super.currentmethod(arg, arg)
  # treat super like self.currentmethod
  res = super(arg, arg)

I think super.currentmethod(arg, arg) makes the most sense, but that may be
because it most closely resembles the current practice. However, it may call
the "wrong" supermethod when the class does, for instance, '__repr__ =
__str__'.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070420/f3dd83c8/attachment.html 

From jason.orendorff at gmail.com  Fri Apr 20 17:10:54 2007
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Fri, 20 Apr 2007 11:10:54 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704191234h5b6a8214p1f17b8588ba730e1@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<bb8868b90704191151u1ea9e28al93a00fa711d01378@mail.gmail.com>
	<fb6fbf560704191234h5b6a8214p1f17b8588ba730e1@mail.gmail.com>
Message-ID: <bb8868b90704200810l56302b4y66925dcd855bfb7@mail.gmail.com>

On 4/19/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/19/07, Jason Orendorff <jason.orendorff at gmail.com> wrote:
> > Collation can be done right: provide a function text.sort_key()
> > that converts a str into an opaque thing that has the desired
> > ordering.
>
> If this function is context-free (depending on only the input string),
> it will violate the unicode standard (and, apparently, do the wrong
> thing for some languages -- usually including French).

I meant this to be a function of the string and the locale,[*]
implemented as a thin wrapper around wcsxfrm() on posix, LCMapString()
on Win32, Collator.getCollationKey() in Java, CompareInfo.GetSortKey()
in .NET.  Whether these are Unicode-compliant is out of our hands.

We're not Sun or IBM.  I don't think we're going to implement and
maintain this ourselves.  So I see two options: (1) swallow a hard
dependency on a particular implementation, maybe ICU; (2) use whatever
the system happens to provide.  Either one is fine with me.

> I'm not saying relying strictly on unicode properties *can't* be done
> right -- I'm just saying that it would be very difficult and very
> inefficient, so it probably won't happen soon -- which is an argument
> for keeping the half-measures around meanwhile.

This would be true if the only option were to implement it all
ourselves.

-j

[*] I would prefer a context-free function really takes both the
string and the locale as arguments... but the posix API doesn't
support that.  :-P

From jyasskin at gmail.com  Fri Apr 20 17:19:08 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Fri, 20 Apr 2007 08:19:08 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704191152u4d1d0494rd6ca2730fad86694@mail.gmail.com>
References: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<5d44f72f0704182319j124b4743he67b669630d303ad@mail.gmail.com>
	<20070418233622.6344.JCARLSON@uci.edu>
	<fb6fbf560704191152u4d1d0494rd6ca2730fad86694@mail.gmail.com>
Message-ID: <5d44f72f0704200819m111c6ff2i94e1fa78d0fc318e@mail.gmail.com>

On 4/19/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > "Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> > > ... Are you saying that in your app, just because I've set
> > > the en_US locale, I won't be able to type "????"?  Or that those
> > > characters won't be recognized as letters?
>
> The latter.  Some applications may reject them for that reason; for
> example some domain registrars have policies to prevent domain name
> spoofing with similar-looking characters.  One way to do that is to
> say that a character used in a domain name (under that registrar) is
> limited to those letters used by the appropriate national language.

Filtering to a certain language's alphabet is neither necessary nor
sufficient for a domain registrar. They're going to implement their
own system paying attention to http://www.unicode.org/reports/tr36/
(see particularly section 2.3 for why even a restriction to one script
doesn't keep them safe), so they don't justify keeping an attractive
nuisance in the standard library.

-- 
Namast?,
Jeffrey Yasskin

From guido at python.org  Fri Apr 20 19:53:23 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 10:53:23 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
Message-ID: <ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>

On 4/20/07, Thomas Wouters <thomas at python.org> wrote:
> On 4/19/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > > Is anyone available to write up a PEP on how to turn super into a
> > > keyword? Inside regular and class methods, super.foo(args) should be
> > > equivalent to super(ThisClass, self).foo(args). I think there are ways
> > > to make the old syntax work too, but maybe that's only necessary for
> > > 2.6.
> >
> > Does this mean it should find the super of the current class (which
> > isn't yet defined when the method is being defined)?
> >
> > That would be a slight change (though probably an improvement) against
> > today's lookup-by-name.
> >
> > If that change is OK, and no one else volunteers, I'll try to take a
> > go at it this weekend.
>
> I've had this near the top of my (Python) TODO list for a while, but I
> haven't been able to find the time. I've considered it while doing the
> dishes and such, though.

(Not in the shower? :-)

> I can think of two ways of doing the underlying
> magic "properly", and one way that's too ugly to think about:
>
>  1) Add a new type of descriptor-hook, for associating an object with the
> class it is an attribute of. After class creation (in the metaclass
> __init__), any object with this __associate__ (or whatever) hook gets it
> called. It's only called for objects that are attributes of *that* class,
> not of any superclasses (since they will already be associated with the
> superclass.) I'm sure there are other uses for this hook, just not methods
> that want to use super() -- like zope interfaces ;)

Nice and elegant. Though what would the hook *do*? I.e. where does it
store the class reference for super to find? Perhaps this is where my
idea of a designated cell comes in handy (although you'd end up with a
cell per method instead of all methods sharing one cell).

Also, what arguments are passed to the hook? For this purpose the
class object is the only thing needed -- but is that always
sufficient?

>  2) Add a new argument to __get__ (or a new method that acts like __get__
> but with the extra argument, and is called instead of __get__ if it is
> defined) where the extra argument is the class that the retrieved descriptor
> is actually an attribute of. It currently (correctly) gets the class the
> attribute was retrieved through, which could be a subclass.

I'm not so keen on this; calling an existing API with an extra
argument is fraught with transitional problems (2to3 notwithstanding).
And checking for two different special methods slows things down. I'd
propose to pass the class where it's found as the 3rd arg always,
since (almost) nobody uses this argument, but I'm not sure of the
consequences for class methods.

Anyway, this also still requires us to answer how that extra argument
gets passed on to the executing code -- but now a cell won't do, as
the argument may be different (theoretically anyway) on each call to
__get__. So that also makes me wary because this adds overhead per
call (technically: per lookup that binds a method) while the thing we
need to pass on typically only changes if the class changes -- so
doing some magic at class creation time makes more sense to me.

(Of course, there's the issue what to do if we dynamically define a
new function, and poke it into a class as a method, and that function
needs to use super. IMO it's fine to require such a function to use
the underlying super() machinery -- this must be an exceedignly rare
case!)

> In both these cases, the (un)bound method object would end up with knowledge
> of the class it is defined in, and create a local variable with a
> super-object for the 'super' keyword to use.

Ah, so you propose to store it in a local variable. But where is it
stored before the function is called? I guess on the "bound method"
object. But how is the value passed on as part of the call? I like a
cell a lot better because we have an established way to pass cells
into calls.

> There still are some grey areas
> to solve -- what if classes share functions, what if functions share
> code-objects, etc. #2 is probably a better way to go about it than #1, in
> that regard.

Sharing functions is very rare and IMO it's okay if the mechanism
doesn't work in that case -- as long as it fails loudly. Sharing code
objects shouldn't be a problem -- code objects are immutable anyway,
cells are associated with the function object, not with the code
object.

> (and 3) the fugly method: don't do anything special with methods, but have
> super() search through __mro__ for the first class to have the current
> function as an attribute, then pick the next class from that. Yecch, and
> more grey areas than the other two methods.)

This does not get my vote.

> There's also the question of what the super keyword itself should look like,
> e.g.
>
>   # declaration-style, no dot
>   res = super currentmethod(arg, arg)
>   # treat super like self
>   res = super.currentmethod (arg, arg)
>   # treat super like self.currentmethod
>   res = super(arg, arg)
>
> I think super.currentmethod(arg, arg) makes the most sense, but that may be
> because it most closely resembles the current practice. However, it may call
> the "wrong" supermethod when the class does, for instance, '__repr__ =
> __str__'.

Depends on how you define "wrong". :-)

I am strongly in favor of super.currentmethod(...) if only because
that's closest to how we've always done it, and there may even be a
use case for callong some *other* super method.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

My signature virus killed your signature virus!

From pje at telecommunity.com  Fri Apr 20 20:26:53 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 20 Apr 2007 14:26:53 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.co
 m>
References: <9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070420141539.02bdf0b0@sparrow.telecommunity.com>

At 10:53 AM 4/20/2007 -0700, Guido van Rossum wrote:
>Also, what arguments are passed to the hook? For this purpose the
>class object is the only thing needed -- but is that always
>sufficient?

I'd want to have the attribute name as an argument, too, at least for other 
uses of the hook.  PEAK and Chandler both have special metaclasses to do 
this sort of attribute-class association, and I believe Paste does too (it 
might be some other project(s) by Ian Bicking, e.g. SQLObject or 
FormEncode).  It comes up pretty much anywhere that people have some sort 
of object-attribute system.  Having a built-in hook for this would be nice.

Ideally, the return value of the hook should then be used as the 
descriptor, rather than the original one; PEAK uses an approach like this 
to do things like:

     class Whatever:
         foo = bar = DelegateTo('otherattribute')

where the foo and bar attributes are being set up to delegate to the foo 
and bar attributes of self.someattribute.  (i.e., foo and bar end up with 
separate descriptors, because the DelegateTo descriptor's association hook 
is called twice, once with 'foo' and once with 'bar', and the return values 
become the actual descriptors used in the class.)

One question, however, would be whether it should refer to the class 
*after* decoration, or before.  For the super() use case, it would 
presumably need to be the *decorated* class, which might make things a bit 
more complicated.

I also wonder if this hook mightn't be usable to implement abstractmethod 
as well?


From guido at python.org  Fri Apr 20 20:43:20 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 11:43:20 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <bbaeab100704192053ud28fdc1s11deaafc8061b4a5@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
	<bbaeab100704192053ud28fdc1s11deaafc8061b4a5@mail.gmail.com>
Message-ID: <ca471dc20704201143l4fdaaa37ve5a4c02250070131@mail.gmail.com>

On 4/19/07, Brett Cannon <brett at python.org> wrote:
> > Good point. Now that you mention this, and I don't understand why I
> > didn't already do it this way in sandbox/abc, the check could be made
> > at *class creation time*, and at instantiation time it should just
> > have to make one tiny check.
>
> I thought you were already doing that.  =)

I am now.

> I was just worried that it
> would be prohibitively expensive at class creation time for some short
> script that used mostly stdlib code that happened to use ABCs.

The built-in types can and will use certain shortcuts. :)

> > I doubt that 'set' will be inherited from much (except for trivial
> > stuff) but returning a 'set' does make some sense. Although for
> > trivial subclasses (e.g. adding a different repr or a brand new
> > method) you'd like the results also to be an instance of the subclass.
> > If we could find a way to do that it would be best.
>
> Well, self.__class__() would work, right?  That should resolve
> properly.  Or maybe we need that super keyword.  =)

No, self.__class__() emphatically does *not* work. The PEP
specifically claims not to define the constructor signatures -- it's
quite possible to implement an object that conforms to the set ABC but
whose constructor does not take a sequence of values. Assuming an
empty constructor always works is unwarranted too. This is why I don't
quite know what to do, except perhaps always returning a (built-in)
set instance.

> > > * "Should we support all the operations implemented by the Python 2 set type?"
> > >
> > > No, only core methods should be directly supported.  Everything else
> > > is just convenience functions.  If these can be implemented simply as
> > > concrete methods then fine, but otherwise leave them out.
> >
> > Sure. But one can differ about whether union() is the convenience and
> > __or__() the core, or vice versa. :-)
>
> So I say base it on ease of implementation.  If it can be done
> trivially in five lines of Python code then include it, otherwise let
> the user do it.

OK, for now I'm including |= and the other operators and excluding
union and the other methods. I'm thinking the argument to |= and
friends should be any iterable though. (But not the argument to the
non-inline | operator!) Concrete implementations are trivial.

> > > * "Also pop, popitem, setdefault?"
> > >
> > > Eh, my gut reaction is "no".  But you do have 'pop' on MutableSequence.
> >
> > But for sequences it trivially maps on core methods. pop() for
> > mappings is a little more convoluted (how to pick a random key? Just
> > pick the first one I guess?)
>
> Right, which is why my gut reaction was "no".  I don't know how you
> choose.  But if you are popping off of a mapping object you should
> know the order will be undefined.  I would take your latter
> suggestion: get a list of keys, take the first one and use that as
> what gets popped off.

OK, pop and popitem are back in, as concrete methods.

> > > * Discussing sequences: "What about += and *=?"
> > >
> > > No.  Only worth it when a special gain is available by the implementation.
> >
> > OTOH not requiring these gives them fuzzy semantics -- some mutable
> > sequences will return self, others a new object. That's not good. I'd
> > like to standardize self-modifying +=, *=.
>
> Then I guess you answered your own question.  If you were asking about
> semantics I would just follow what lists do.

OK, they're in, as concrete operators.

> See http://haskell.org/onlinereport/prelude-index.html for the Haskell
> 98 report and the prelude (Haskell's version of built-ins).  You care
> about the type classes (which they just call "classes" on the page).
> If you follow the links for any of the type classes they will tell you
> exactly what functions must be defined for something to meet that type
> class.  Some of them are Haskell-specific, but not all.

One for my copious spare time...

> Umm, Sized?  =)  Basically, no, I don't have a better suggestion.

I like it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Apr 20 20:52:55 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 11:52:55 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <46288F11.3030505@gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<46288F11.3030505@gmail.com>
Message-ID: <ca471dc20704201152u7eb9f595l3c4dd444fe2cf86@mail.gmail.com>

On 4/20/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It's not listed as an open issue, but when looking at the list of ABCs,
> I noticed a potential solution to a different problem we have
> occasionally wrestled with: the unfortunate side-effects of having a
> default __hash__ method defined on object.
>
> If the details of collections.Hashable.__hash__ are declared to be
> implementation dependent rather than having it always return 0, then we
> can get rid of object.__hash__.
>
> For CPython, collections.Hashable.__hash__ would then just invoke id()
> on the object, and other implementations can also do whatever they
> currently do for object.__hash__

I don't like that at all. I want to force folks to implement a
__hash__ that matches their __eq__. As soon as you implement an __eq__
you need to redefine __hash__. I'd much rather have the default
__hash__ be correct but insanely slow than fast but incorrect -- it
being incorrect is much harder to debug, and it smells of premature
optimization. As Hashable.__hash__ is abstract, you are *forced* to
define your own hash.

I do see your point though: object.__hash__ makes it possible to use
all sorts of objects as dict keys by default, as long as they don't
implement __eq__. But perhaps this is not such a good idea and we
should just get rid of it and add __hash__ back to specific objects
that are useful to use as keys? (E.g. classes but not iterators, to
take some extreme examples.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Apr 20 20:56:46 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 11:56:46 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <5.1.1.6.0.20070420141539.02bdf0b0@sparrow.telecommunity.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<5.1.1.6.0.20070420141539.02bdf0b0@sparrow.telecommunity.com>
Message-ID: <ca471dc20704201156sf90013ej555a0d726e673b89@mail.gmail.com>

If you are serious about this, it'd need a PEP, clarifying the use cases etc.

On 4/20/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:53 AM 4/20/2007 -0700, Guido van Rossum wrote:
> >Also, what arguments are passed to the hook? For this purpose the
> >class object is the only thing needed -- but is that always
> >sufficient?
>
> I'd want to have the attribute name as an argument, too, at least for other
> uses of the hook.  PEAK and Chandler both have special metaclasses to do
> this sort of attribute-class association, and I believe Paste does too (it
> might be some other project(s) by Ian Bicking, e.g. SQLObject or
> FormEncode).  It comes up pretty much anywhere that people have some sort
> of object-attribute system.  Having a built-in hook for this would be nice.
>
> Ideally, the return value of the hook should then be used as the
> descriptor, rather than the original one; PEAK uses an approach like this
> to do things like:
>
>      class Whatever:
>          foo = bar = DelegateTo('otherattribute')
>
> where the foo and bar attributes are being set up to delegate to the foo
> and bar attributes of self.someattribute.  (i.e., foo and bar end up with
> separate descriptors, because the DelegateTo descriptor's association hook
> is called twice, once with 'foo' and once with 'bar', and the return values
> become the actual descriptors used in the class.)
>
> One question, however, would be whether it should refer to the class
> *after* decoration, or before.  For the super() use case, it would
> presumably need to be the *decorated* class, which might make things a bit
> more complicated.
>
> I also wonder if this hook mightn't be usable to implement abstractmethod
> as well?
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Fri Apr 20 21:07:04 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 20 Apr 2007 12:07:04 PDT
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com> 
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <07Apr20.120713pdt."57996"@synergy1.parc.xerox.com>

Guido,

A good start.  A few comments:

Where do strings and bytes fit in?  They seem to be sequences.  Are
they sub-typable?  If so, shouldn't their base interface be defined?

I'd either add numeric types to this, or split it in two pieces: one
about ABCs in general (perhaps defining Comparable and Object), and
another for collection types.  Then a third for numeric types.

I don't like the name of the "pop()" method on Sets.  It's too much
associated with stacks.  People reading code that uses "pop" on a value
will naturally assume that the value is a stack.  I'd prefer
"remove_and_return_random_item()".

> Python will not require that a class derives from BasicMapping or
> Sequence when it defines a __getitem__ method,

Good.

> nor will the x[y] syntax require that x is an instance of either ABC.

Good.

> You will still be able to assign any "file-like" object to
> sys.stdout, as long as it has a write method.

Well, yes, but...  I think there should be an ABC for "file-like",
which the standard "file" and "StringIO" implementations inherit from.
This could even be a fourth PEP.

Bill

From talin at acm.org  Fri Apr 20 21:13:22 2007
From: talin at acm.org (Talin)
Date: Fri, 20 Apr 2007 12:13:22 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <07Apr20.120713pdt."57996"@synergy1.parc.xerox.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<07Apr20.120713pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <462910D2.7060301@acm.org>

Bill Janssen wrote:
>> You will still be able to assign any "file-like" object to
>> sys.stdout, as long as it has a write method.
> 
> Well, yes, but...  I think there should be an ABC for "file-like",
> which the standard "file" and "StringIO" implementations inherit from.
> This could even be a fourth PEP.

I suggest the ABCs for stream objects should be incorporated into the 
PEP for the new IO library, using this PEP as a prerequisite. The reason 
for this is that the stream ABCs will naturally follow the same class 
decomposition and methods as are defined by the new library (i.e. input 
stream, output stream, and so on.)

-- Talin

From brett at python.org  Fri Apr 20 21:21:18 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 20 Apr 2007 12:21:18 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <bbaeab100704201221p15c2e371kccf8d3162b23042@mail.gmail.com>

On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
>
>     http://www.python.org/dev/peps/pep-3119/
>
> While I'm not ready yet to answer tough questions about this compared
> to alternative proposals, I *am* ready for feedback on the various
> open issues sprinkled throughout the current text, especially
> concerning decisions regarding exactly which operations to include in
> the various ABCs, and also regarding the level of detail required in
> the PEP.
>

Just noticed your latest checkin asks whether to unify 'remove' and
'discard' for sets.  I say yes.  If people care about whether the item
pre-exists or not they can check if the item is contained in the set
since it implements Container.  So either have it throw an exception
if the item does not exist or return a boolean based on whether
something was remove ala Java.  But I don't see a need for both when
the check is as simple as::

  set.remove(item) if item in set else None

or something along those lines.

-Brett

From talin at acm.org  Fri Apr 20 21:33:42 2007
From: talin at acm.org (Talin)
Date: Fri, 20 Apr 2007 12:33:42 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <46291596.1020105@acm.org>

Guido van Rossum wrote:
> I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
> 
>     http://www.python.org/dev/peps/pep-3119/
> 
> While I'm not ready yet to answer tough questions about this compared
> to alternative proposals, I *am* ready for feedback on the various
> open issues sprinkled throughout the current text, especially
> concerning decisions regarding exactly which operations to include in
> the various ABCs, and also regarding the level of detail required in
> the PEP.

One thing that I noticed is that you say "ABCs are not intrinsically 
incompatible with Interfaces", but you don't define what interfaces are.

I'm not sure my understanding of the interface proposals is correct, but 
here is what I would write based on my current understanding:

---

"Interfaces" in this context refers to a set of proposals for additional 
metadata elements attached to a class which are not part of the regular 
class hierarchy, but do allow for certain types of inheritance testing.

Such metadata would be designed, at least in some proposals, so as to be 
easily mutable by an application, allowing application writers to 
override the normal classification of an object.

The drawback to this idea of attaching mutable metadata to a class is 
that classes are shared state, and mutating them may lead to conflicts 
of intent. Additionally, the need to override the classification of an 
object can be done more cleanly using generic functions: In the simplest 
case, one can define a "category membership" generic function that 
simply returns False in the base implementation, and then provide 
overrides that return True for any classes of interest.

-- Talin

From thomas at python.org  Fri Apr 20 22:06:56 2007
From: thomas at python.org (Thomas Wouters)
Date: Fri, 20 Apr 2007 22:06:56 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
Message-ID: <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com>

On 4/20/07, Guido van Rossum <guido at python.org> wrote:
>
> On 4/20/07, Thomas Wouters <thomas at python.org> wrote:
> > I've had this near the top of my (Python) TODO list for a while, but I
> > haven't been able to find the time. I've considered it while doing the
> > dishes and such, though.
>
> (Not in the shower? :-)


Yes, there too, when I shower alone; I was going to spare everyone the
visual though :)

> I can think of two ways of doing the underlying
> > magic "properly", and one way that's too ugly to think about:
> >
> >  1) Add a new type of descriptor-hook, for associating an object with
> the
> > class it is an attribute of. After class creation (in the metaclass
> > __init__), any object with this __associate__ (or whatever) hook gets it
> > called. It's only called for objects that are attributes of *that*
> class,
> > not of any superclasses (since they will already be associated with the
> > superclass.) I'm sure there are other uses for this hook, just not
> methods
> > that want to use super() -- like zope interfaces ;)
>
> Nice and elegant. Though what would the hook *do*? I.e. where does it
> store the class reference for super to find? Perhaps this is where my
> idea of a designated cell comes in handy (although you'd end up with a
> cell per method instead of all methods sharing one cell).
>
> Also, what arguments are passed to the hook? For this purpose the
> class object is the only thing needed -- but is that always
> sufficient?


The hook would 'bind' the function to the class -- in some unspecified way.
Unspecified because I haven't thought that far ahead yet, and I don't forsee
any particular difficulties. Either the function gets wrapped in an
'associated function' type that knows which class it was defined in (so
somewhat different from the current 'unbound method' type, but not much) or
we just store the associated-class in the existing function object (probably
the better option.). In either case we need some new magic to pass this
class-object to the actual code object, or give the code object a way to
refer to the function object (sometimes also useful for other reasons; it's
not an uncommon request, in newbie areas.)

As for the arguments to this hook -- I guess passing the attribute name as
well as the class object makes sense. It's not much more overhead, and there
isn't much more we could sensibly pass that isn't easily available given the
class object and attribute name.

Phillip mentioned class decorators -- I'd forgotten about those. I don't see
much use for them myself, as metaclasses make much more sense to me, in
particular because they are naturally inherited, whereas class decorators
are (I assume) not inherited. I'm not sure how they should -- or could --
work together with super. It would only be a problem if the class decorator
returns a wrapper class, rather than mutate the class. I guess we'd need a
way to tell associated-functions "instead of ever visiting this class, visit
this wrapper class instead. But don't mess with MRO"... I think. I'm not
entirely sure how class decorators would affect MRO, really.

>  2) Add a new argument to __get__ (or a new method that acts like __get__
> > but with the extra argument, and is called instead of __get__ if it is
> > defined) where the extra argument is the class that the retrieved
> descriptor
> > is actually an attribute of. It currently (correctly) gets the class the
> > attribute was retrieved through, which could be a subclass.
>
> I'm not so keen on this; calling an existing API with an extra
> argument is fraught with transitional problems (2to3 notwithstanding).
> And checking for two different special methods slows things down. I'd
> propose to pass the class where it's found as the 3rd arg always,
> since (almost) nobody uses this argument, but I'm not sure of the
> consequences for class methods.


The consequence for classmethods would be bad. They would always get the
class they were defined on as first argument, making them completely
pointless. So, let's not do that :)

(Of course, there's the issue what to do if we dynamically define a
> new function, and poke it into a class as a method, and that function
> needs to use super. IMO it's fine to require such a function to use
> the underlying super() machinery -- this must be an exceedignly rare
> case!)


It's easy to handle such cases if we flag super-syntax-using-functions
specially (which is also easy.) type's __setattr__ can complain if such a
function gets assigned to an attribute, or it could do the binding magic
right then. Functions that don't use the super syntax will continue to work
like usual. And super-syntax-using-functions can scream and shout when they
get 'associated' with more than one class, of course.

> In both these cases, the (un)bound method object would end up with
> knowledge
> > of the class it is defined in, and create a local variable with a
> > super-object for the 'super' keyword to use.
>
> Ah, so you propose to store it in a local variable. But where is it
> stored before the function is called? I guess on the "bound method"
> object. But how is the value passed on as part of the call? I like a
> cell a lot better because we have an established way to pass cells
> into calls.


As I mentioned above, I didn't really think about this part. By 'local
variable' I just mean that we don't use a global registry or some such ugly
hack -- it's a reference, stored wherever, specific to that call.

> There still are some grey areas
> > to solve -- what if classes share functions, what if functions share
> > code-objects, etc. #2 is probably a better way to go about it than #1,
> in
> > that regard.
>
> Sharing functions is very rare and IMO it's okay if the mechanism
> doesn't work in that case -- as long as it fails loudly. Sharing code
> objects shouldn't be a problem -- code objects are immutable anyway,
> cells are associated with the function object, not with the code
> object.


Agreed.

> There's also the question of what the super keyword itself should look
> like,
> > e.g.
> >
> >   # declaration-style, no dot
> >   res = super currentmethod(arg, arg)
> >   # treat super like self
> >   res = super.currentmethod (arg, arg)
> >   # treat super like self.currentmethod
> >   res = super(arg, arg)
> >
> > I think super.currentmethod(arg, arg) makes the most sense, but that may
> be
> > because it most closely resembles the current practice. However, it may
> call
> > the "wrong" supermethod when the class does, for instance, '__repr__ =
> > __str__'.
>
> Depends on how you define "wrong". :-)


That's why it's in "quotes" :) No matter what we do, it will always be wrong
for someone somewhere, I'm sure.

I am strongly in favor of super.currentmethod(...) if only because
> that's closest to how we've always done it, and there may even be a
> use case for callong some *other* super method.


I agree. The big downside of thinking of these things off-line (like in the
shower) is that it's hard to visualize the actual code. The moment I wrote
down the three alternatives I'd imagined, I realized 'super.currentmethod()'
is the only sane spelling ;)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a signature-virus-eating-signature-virus-resistant .signature virus!
copy me into your .signature file to help me spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070420/cb041827/attachment-0001.html 

From pje at telecommunity.com  Fri Apr 20 23:15:19 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 20 Apr 2007 17:15:19 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com
 >
References: <ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070420170331.04af8cf8@sparrow.telecommunity.com>

At 10:06 PM 4/20/2007 +0200, Thomas Wouters wrote:
>It would only be a problem if the class decorator returns a wrapper class, 
>rather than mutate the class. I guess we'd need a way to tell 
>associated-functions "instead of ever visiting this class, visit this 
>wrapper class instead. But don't mess with MRO"... I think. I'm not 
>entirely sure how class decorators would affect MRO, really.

This depends somewhat, I think, on whether the conceptual translation of 
super.foo is "super(ClassNameHere, self)" or "super(__thisclass__, 
self)".  If it is ClassNameHere, then the decorated class should be used; 
if it's __thisclass__, there's a miniscule amount more argument for it 
being the original class -- but even then, I think practicality probably 
beats purity.

Meanwhile, I still think that cell (closure) variables are the One Obvious 
Way to implement this.  We can implement the super.foo syntax as an AST 
transform to "super(..., firstarg)" where ... is either __thisclass__ or 
ClassName, and then say that __thisclass__ or ClassName is then treated as 
a cell variable defined in the surrounding scope.

In either case, we simply have the language definition say that the class 
name or __thisclass__ (when used in a function inside a class) always 
refers to the (decorated) class.

(Notice that this means there really isn't a need for the descriptor 
association protocol, at least for Python's own use; I missed that in my 
previous post, due to my enthusiasm for the independent merits of the 
__associate__ idea.)


From guido at python.org  Fri Apr 20 23:33:01 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 14:33:01 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <6558863970457745851@unknownmsgid>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<6558863970457745851@unknownmsgid>
Message-ID: <ca471dc20704201433l6d564454rc9c0ed977ea5cc63@mail.gmail.com>

On 4/20/07, Bill Janssen <janssen at parc.com> wrote:
> Where do strings and bytes fit in?  They seem to be sequences.  Are
> they sub-typable?  If so, shouldn't their base interface be defined?

Yes, yes and yes. I haven't gotten to them yet. :-)

> I'd either add numeric types to this, or split it in two pieces: one
> about ABCs in general (perhaps defining Comparable and Object), and
> another for collection types.  Then a third for numeric types.

Yeah, I'm thinking of at least separating the numeric types. I'd like
to keep the collections in the same PEP as the general topics just so
that it shows "what can ABCs do for you".

> I don't like the name of the "pop()" method on Sets.  It's too much
> associated with stacks.  People reading code that uses "pop" on a value
> will naturally assume that the value is a stack.  I'd prefer
> "remove_and_return_random_item()".

Yeah, but it has a long history in Python. Python 2's sets have a
pop() method, and dicts have an equivalent popitem() method (as well
as a pop() that requires you to specify a key).

> > You will still be able to assign any "file-like" object to
> > sys.stdout, as long as it has a write method.
>
> Well, yes, but...  I think there should be an ABC for "file-like",
> which the standard "file" and "StringIO" implementations inherit from.
> This could even be a fourth PEP.

PEP 3116 already documents ABCs like RawIOBase, BufferedIOBase,
TextIOBase. (It needs to be updated; the implementation adds a fourth,
the base of them all, IOBase.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr 21 00:24:26 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 15:24:26 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com>
Message-ID: <ca471dc20704201524r3116d116ob34e84043798e818@mail.gmail.com>

On 4/20/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
> >
> >     http://www.python.org/dev/peps/pep-3119/
> >
> > While I'm not ready yet to answer tough questions about this compared
> > to alternative proposals, I *am* ready for feedback on the various
> > open issues sprinkled throughout the current text, especially
> > concerning decisions regarding exactly which operations to include in
> > the various ABCs, and also regarding the level of detail required in
> > the PEP.
>
> I have a bunch of comments that aren't all related to the open issues.
> I'm more familiar with statically typed languages, so sorry if some of
> these comments don't make sense in a Python context.

Thanks for the feedback! You're doing fine. :-)

> Hashable might want to specify the exception that should be thrown for
> mutable objects.

Thanks -- done.

> I wouldn't have guessed what Finite was from the name, but I don't
> mind it now that I know. I do prefer 'Sized'.

Sized it is.

> A possible name for the __contains__ ABC for sequences and strings is
> 'Searchable'. On the other hand, an annotation of some sort for the
> running time of the method might do as well, but I don't know of any
> prior art for that.

I've added the name suggestion to the open issues; I like it. I'm not
sure I want to break new ground specifying running time; it's a tricky
business.

> On Sets:
> Do you have an example of a data structure that is a Set but not a
> ComposableSet?

Take for example a set that is a contiguous range of integers,
represented by a (lo, hi) tuple. In many cases the return type of
union and symmetric difference cannot be represented as an object of
the same type. I don't want to force the implementer of such a set to
provide implementations for operations they don't need.

A different POV could be to move the concrete composition operations
back into Set and let the default implementations return built-in
``set`` instances. I'm not sure how useful that is.

> Set should inherit from a PartiallyOrdered ABC that defines the
> comparison operators.

I guess then there would also be a TotallyOrdered ABC deriving from
PartiallyOrdered which adds no operations but does add a constraint.
I've added both, and made Set derive from PartiallyOrdered.

> The final specification of Set should spell out the invariants, but
> they're clear enough that the PEP probably doesn't need to. It's odd,
> though, that you mention optimizations to __eq__ and __le__ but not
> their semantics.

The semantics are kind of obvious. :-)

Text spelling out the invariants in more detail that I can just paste
into the PEP is most welcome!

> ComposableSet's open issues:
> > Should we define an API for creating new instances (e.g. a class method or a fixed constructor signature)?
>
> There are some set types that can't be constructed from arbitrary
> sequences. For example, Patricia Tries
> (http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-IntSet.html)
> can be used to make very efficient sets of integers, but only
> integers.

Right. That's why I'm separating ComposableSet from Set.

> > Should we just pick a concrete return type (e.g. set)?
>
> For set compositions, you should say whether the arguments need to be
> uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do,
> then you don't need a concrete return type; if they don't, then you
> do, and the default implementations should use it.

You may be misunderstanding what I meant. I certainly don't want to
constrain the potential return type of the composition operations
(except that they should themselves be composable sets). The question
about picking a specific return type was meant as a way out to
providing a concrete default implementation. I'm still on the fence of
this, as it could be a big waste of time (I imagine that composing
Patricia trees can be done more efficiently than iterating over the
elements).

> Even this may be a
> problem if someone writes a TreeSet which requires
> TotallyOrdered-but-not-Hashable elements. In any case, the ABC should
> encourage implementors to return an instance of the same type as the
> arguments when the arguments are of the same type.

Another good argument against having a default concrete implementation
that always returns a built-in set.

> > Should we add the copy method?
> Not on ComposableSet, but maybe in a Copyable ABC, which it might inherit from.

I'd rather leave copying APIs out of this completely. There's a
separate copying protocol in Python already, you can define __copy__()
for a shallow copy and __deepcopy__() for a deep copy (copies the
elements recursively). This is also tied into pickling
(serialization).

> MutableSet:
> > Should we support more operations implemented by the Python 2 set type?
>
> You have groups of equivalent operations (add, update, __ior__),
> (discard, difference_update, __isub__), (??, intersection_update,
> __iand__), and (??, symmetric_difference_update, __ixor__)
>
> It would be nice to make these all uniform. Intersection is hard to
> support one element at a time, but symmetric difference would work
> with a toggle() method.

Cool, I've added toggle(). FWIW, I've remoevd the named methods
(update etc.) in favor of stretching the accepted args for the
operators (__ior__ etc.).

> The operations that work a group at a time are
> likely be more efficient on Sized arguments, and much more efficient
> on arguments of the same type. How much are you trying to support
> these kinds of optimizations to implementations?

Haven't thought much about it. Implementations can just override the
concrete methods if they have a faster way, and fall back on the
concrete base class methods if their faster way doesn't apply for a
particular argument.

> > Should we unify remove and discard, a la Java (which has a single method returning a boolean indicating whether it was removed or not)?
>
> Yes. :)

Done.

> Mappings:
> Set-like unions, intersections, and differences are useful on maps
> too, but you usually have to provide a function to say what to do with
> colliding values. This is probably a different PEP though.

Right. My dream is to unify the set and dict types, solving the
problem of how to specify an empty set ({} would do it), but this was
rejected by Raymond Hettinger for valid pragmatic reasons.

> Sequence:
> I guess you've already dealt with this, but allowing __getitem__ with
> integer indices on str objects forces you to use UTF-32 or play
> interesting games with caching. Most string algorithms are quite happy
> with (bidirectional? multipass?) iterators, but programmers aren't
> used to using them.

Yeah, we've had our fill of discussions about alternative
implementations. I want the *semantics* to be those of UTF-32 or
UTF-16, and the initial implementation will use that, since these
semantics are most familiar to Python users. At some point we may have
competing implementations.

> Numbers:

Would you mind helping to write a PEP for numbers? I think I'm running
out of round tuits just for the collections and the ABC
infrastructure.

> Haskell might be good to use for inspiration on the numeric classes:
> http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#7.
> Note that Haskell's "Real" doesn't imply that the number is
> fractional, just that it's not complex. Some differences I see from
> Bill Janssen's proposal:
>  * A Haskell Num is constructible from any Integer (python's long).
>  * A Haskell Fractional (including Complex) is constructible from any
> Rational (which can be constructed from a float)
>  * The operations are spread into more classes. I think that primarily
> helps Complex and Fixed-point types fit into the hierarchy better. The
> Haskell folks actually complain that the operations aren't spread into
> _enough_ classes. They mostly want more group-theory related classes
> below Num.
>  * The bitwise operators are segregated into a Bits class, but I don't
> see any benefit to that.

Right. Python has a long tradition of using ints (and longs) for that.

>  * In a possible use for Cardinal, Integral**Cardinal=>Integral,
> Fractional**Integral=>Fractional, and Floating**Floating=>Floating.

That can be (and is currently) done dynamically (see below).

>  * Complex isn't Orderable
>
> Fortress (http://research.sun.com/projects/plrg/fortress.pdf#page=270)
> has an extremely detailed hierarchy of algebraic traits, and it looks
> like they plan a similarly detailed hierarchy of numbers, but the
> numbers aren't yet specified beyond integers and rationals.
>
> Cardinal is also useful as the argument to (Sequence * Cardinal).

I think there's little value for this in a dynamically typed language;
the implementation can raise ValueError or treat negative numbers as
zero (which is what Python currently does).

> Index (Ix) is used in Haskell primarily in the form of tuples of Ints
> for multi-dimensional arrays.

Right, that's where we use it too (especially the NumPy community).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr 21 00:40:18 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Apr 2007 15:40:18 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<9e804ac0704201306q38eaf531k9ff0a83b633b3c8b@mail.gmail.com>
Message-ID: <ca471dc20704201540o2038c727h90ec96afd089fa34@mail.gmail.com>

No time to respond in kind but feeling general agreement. I hope you &
Jim Jewett (who first volunteered) can work together on the details
for a PEP?

On 4/20/07, Thomas Wouters <thomas at python.org> wrote:
>
>
> On 4/20/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/20/07, Thomas Wouters <thomas at python.org> wrote:
> > > I've had this near the top of my (Python) TODO list for a while, but I
> > > haven't been able to find the time. I've considered it while doing the
> > > dishes and such, though.
> >
> > (Not in the shower? :-)
>
> Yes, there too, when I shower alone; I was going to spare everyone the
> visual though :)
>
> > > I can think of two ways of doing the underlying
> > > magic "properly", and one way that's too ugly to think about:
> > >
> > >  1) Add a new type of descriptor-hook, for associating an object with
> the
> > > class it is an attribute of. After class creation (in the metaclass
> > > __init__), any object with this __associate__ (or whatever) hook gets it
> > > called. It's only called for objects that are attributes of *that*
> class,
> > > not of any superclasses (since they will already be associated with the
> > > superclass.) I'm sure there are other uses for this hook, just not
> methods
> > > that want to use super() -- like zope interfaces ;)
> >
> > Nice and elegant. Though what would the hook *do*? I.e. where does it
> > store the class reference for super to find? Perhaps this is where my
> > idea of a designated cell comes in handy (although you'd end up with a
> > cell per method instead of all methods sharing one cell).
> >
> > Also, what arguments are passed to the hook? For this purpose the
> > class object is the only thing needed -- but is that always
> > sufficient?
>
>
> The hook would 'bind' the function to the class -- in some unspecified way.
> Unspecified because I haven't thought that far ahead yet, and I don't forsee
> any particular difficulties. Either the function gets wrapped in an
> 'associated function' type that knows which class it was defined in (so
> somewhat different from the current 'unbound method' type, but not much) or
> we just store the associated-class in the existing function object (probably
> the better option.). In either case we need some new magic to pass this
> class-object to the actual code object, or give the code object a way to
> refer to the function object (sometimes also useful for other reasons; it's
> not an uncommon request, in newbie areas.)
>
> As for the arguments to this hook -- I guess passing the attribute name as
> well as the class object makes sense. It's not much more overhead, and there
> isn't much more we could sensibly pass that isn't easily available given the
> class object and attribute name.
>
> Phillip mentioned class decorators -- I'd forgotten about those. I don't see
> much use for them myself, as metaclasses make much more sense to me, in
> particular because they are naturally inherited, whereas class decorators
> are (I assume) not inherited. I'm not sure how they should -- or could --
> work together with super. It would only be a problem if the class decorator
> returns a wrapper class, rather than mutate the class. I guess we'd need a
> way to tell associated-functions "instead of ever visiting this class, visit
> this wrapper class instead. But don't mess with MRO"... I think. I'm not
> entirely sure how class decorators would affect MRO, really.
>
> > >  2) Add a new argument to __get__ (or a new method that acts like
> __get__
> > > but with the extra argument, and is called instead of __get__ if it is
> > > defined) where the extra argument is the class that the retrieved
> descriptor
> > > is actually an attribute of. It currently (correctly) gets the class the
> > > attribute was retrieved through, which could be a subclass.
> >
> > I'm not so keen on this; calling an existing API with an extra
> > argument is fraught with transitional problems (2to3 notwithstanding).
> > And checking for two different special methods slows things down. I'd
> > propose to pass the class where it's found as the 3rd arg always,
> > since (almost) nobody uses this argument, but I'm not sure of the
> > consequences for class methods.
>
> The consequence for classmethods would be bad. They would always get the
> class they were defined on as first argument, making them completely
> pointless. So, let's not do that :)
>
> > (Of course, there's the issue what to do if we dynamically define a
> > new function, and poke it into a class as a method, and that function
> > needs to use super. IMO it's fine to require such a function to use
> > the underlying super() machinery -- this must be an exceedignly rare
> > case!)
>
> It's easy to handle such cases if we flag super-syntax-using-functions
> specially (which is also easy.) type's __setattr__ can complain if such a
> function gets assigned to an attribute, or it could do the binding magic
> right then. Functions that don't use the super syntax will continue to work
> like usual. And super-syntax-using-functions can scream and shout when they
> get 'associated' with more than one class, of course.
>
> > > In both these cases, the (un)bound method object would end up with
> knowledge
> > > of the class it is defined in, and create a local variable with a
> > > super-object for the 'super' keyword to use.
> >
> > Ah, so you propose to store it in a local variable. But where is it
> > stored before the function is called? I guess on the "bound method"
> > object. But how is the value passed on as part of the call? I like a
> > cell a lot better because we have an established way to pass cells
> > into calls.
>
> As I mentioned above, I didn't really think about this part. By 'local
> variable' I just mean that we don't use a global registry or some such ugly
> hack -- it's a reference, stored wherever, specific to that call.
>
> > > There still are some grey areas
> > > to solve -- what if classes share functions, what if functions share
> > > code-objects, etc. #2 is probably a better way to go about it than #1,
> in
> > > that regard.
> >
> > Sharing functions is very rare and IMO it's okay if the mechanism
> > doesn't work in that case -- as long as it fails loudly. Sharing code
> > objects shouldn't be a problem -- code objects are immutable anyway,
> > cells are associated with the function object, not with the code
> > object.
>
> Agreed.
>
> > > There's also the question of what the super keyword itself should look
> like,
> > > e.g.
> > >
> > >   # declaration-style, no dot
> > >   res = super currentmethod(arg, arg)
> > >   # treat super like self
> > >   res = super.currentmethod (arg, arg)
> > >   # treat super like self.currentmethod
> > >   res = super(arg, arg)
> > >
> > > I think super.currentmethod(arg, arg) makes the most sense, but that may
> be
> > > because it most closely resembles the current practice. However, it may
> call
> > > the "wrong" supermethod when the class does, for instance, '__repr__ =
> > > __str__'.
> >
> > Depends on how you define "wrong". :-)
>
> That's why it's in "quotes" :) No matter what we do, it will always be wrong
> for someone somewhere, I'm sure.
>
> > I am strongly in favor of super.currentmethod(...) if only because
> > that's closest to how we've always done it, and there may even be a
> > use case for callong some *other* super method.
>
> I agree. The big downside of thinking of these things off-line (like in the
> shower) is that it's hard to visualize the actual code. The moment I wrote
> down the three alternatives I'd imagined, I realized '
> super.currentmethod()' is the only sane spelling ;)
>
> --
> Thomas Wouters <thomas at python.org>
>
> Hi! I'm a signature-virus-eating-signature-virus-resistant
> .signature virus! copy me into your .signature file to help me spread!


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jyasskin at gmail.com  Sat Apr 21 09:30:16 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sat, 21 Apr 2007 00:30:16 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704201524r3116d116ob34e84043798e818@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com>
	<ca471dc20704201524r3116d116ob34e84043798e818@mail.gmail.com>
Message-ID: <5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com>

On 4/20/07, Guido van Rossum <guido at python.org> wrote:
> On 4/20/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> > On Sets:
> > Do you have an example of a data structure that is a Set but not a
> > ComposableSet?
>
> Take for example a set that is a contiguous range of integers,
> represented by a (lo, hi) tuple. In many cases the return type of
> union and symmetric difference cannot be represented as an object of
> the same type. I don't want to force the implementer of such a set to
> provide implementations for operations they don't need.

Ah, good example. I think that PEPs that propose ABCs should be
encouraged to describe a few possible implementations, and should be
particularly encouraged to give examples of the implementations that
made them split two classes, or omit a method from a particular class.

> > Set should inherit from a PartiallyOrdered ABC that defines the
> > comparison operators.
>
> I guess then there would also be a TotallyOrdered ABC deriving from
> PartiallyOrdered which adds no operations but does add a constraint.
> I've added both, and made Set derive from PartiallyOrdered.

Cool. Here's some language for their invariants (checking myself with
the Fortress spec, and stealing its properties nearly verbatim):

PartiallyOrdered:
This ABC defines the 5 inequality operations <, <=, >=, >, and cmp().
(Note that == and != are defined by object.) Classes deriving from
this ABC should ensure that '<' and '>' form strict partial orders,
and '<=' and '>=' form weak partial orders, in the sense of
http://en.wikipedia.org/wiki/Partially_ordered_set. These should be
compatible with each other and with ==:

Where PO is a subtype of PartiallyOrdered, and a and b are instances of PO,
cmp(a, b) < 0 ? cmp(b, a) > 0
cmp(a, b) == 0 ? cmp(b, a) == 0
cmp(a, b) == None ? cmp(b, a) == None
a < b ? cmp(a, b) < 0
a <= b ? cmp(a, b) <= 0
a == b ? cmp(a, b) == 0
a >=b ? cmp(a, b) >= 0
a > b ? cmp(a, b) > 0

Subclasses of PartiallyOrdered must define one of cmp() or <=. The
other operators are defaulted as follows:
  def <=(a,b): return cmp(a,b) <= 0
  def cmp(a,b):
    if a<=b:
      if b<=a: return 0
      else: return -1
    else:
      if b<=a: return 1
      else: return None
  def ==(a,b): return a<=b and b<=a
  def <(a,b): return a<=b and not b<=a
  def >=(a,b): return b<=a
  def >(a,b): return b<a

Open Issues: 1) Is it reasonable to let cmp() return None for
unordered pairs, or should we instead define a 4-value enumeration for
it to return? 2) The @abstractmethod decorator isn't enough to specify
that subclasses must override at least one of a group of methods, and
there are classes with even more complex requirements like "subclasses
must override a or (b and (c or d))". 3)
http://docs.python.org/ref/customization.html#l2h-187 still implies
that < defines a total order over all python objects. Has that changed
for py3k? 4) How should we handle the reflected forms of these
operators?

TotallyOrdered:
This ABC derives from PartiallyOrdered. It adds no new operations but
implies a promise that <, <=, >, and >= are total orders in the sense
of http://en.wikipedia.org/wiki/Totally_ordered_set, and that cmp(a,b)
never returns None. Given this additional restriction, subclasses only
need to define one of <, <=, or cmp(), and the rest of the methods can
be defaulted.

Open Issue: How does the TotallyOrdered property inherit? If Real (an
abstract class) inherits from TotallyOrdered, do different subclasses
need to be totally comparable with each other? If long (a concrete
class) inherits from TotallyOrdered, and I subclass long, do I have to
ensure that my instances are totally comparable with other longs? I
think 'yes' to the second, to obey the Liskov substitution principle.
I don't know about the first: for Real it seems fine, but it might
inhibit refactoring the ABC hierarchy.

> > The final specification of Set should spell out the invariants, but
> > they're clear enough that the PEP probably doesn't need to. It's odd,
> > though, that you mention optimizations to __eq__ and __le__ but not
> > their semantics.
>
> The semantics are kind of obvious. :-)
>
> Text spelling out the invariants in more detail that I can just paste
> into the PEP is most welcome!

Bah! ;)

Set:
Sets should implement an efficient (no worse than O(log N))
__contains__ operation. Sets are defined by their extensions.
Subclasses may override the following default definitions of __eq__
and __le__ with more efficient versions, but they must respect the
same semantics:
  def __eq__(a, b): return len(a) == len(b) and all(x in a for x in b)
  def __le__(a, b): return len(a) <= len(b) and all(x in a for x in b)

ComposableSet:
... mathematical meaning of set composition:
Where a and b are instances of ComposableSet and __op__(a,b) is defined:
  x in (a | b) ? x in a or x in b
  x in (a & b) ? x in a and x in b
  x in (a ^ b) ? x in a != x in b
  x in (a - b) ? x in a and not x in b

> > ComposableSet's open issues:
> > > Should we just pick a concrete return type (e.g. set)?
> >
> > For set compositions, you should say whether the arguments need to be
> > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do,
> > then you don't need a concrete return type; if they don't, then you
> > do, and the default implementations should use it.
>
> You may be misunderstanding what I meant. I certainly don't want to
> constrain the potential return type of the composition operations
> (except that they should themselves be composable sets). The question
> about picking a specific return type was meant as a way out to
> providing a concrete default implementation. I'm still on the fence of
> this, as it could be a big waste of time (I imagine that composing
> Patricia trees can be done more efficiently than iterating over the
> elements).

Borrowing some syntax from Java generics, which of:
  ComposableSet __or__(ComposableSet a, ComposableSet b);
  <CS extends ComposableSet> ComposableSet __or__(CS a, CS b);
  <CS extends ComposableSet> CS __or__(CS a, CS b);
do you mean to define? I think they have different implications about
what defaults you need to provide. The second is probably out because
it would mean (a|b|c) would likely break. This gets even more complex
if we consider the types of the contents of the sets... Here's a
possible overloaded signature for __or__. (Do you already have a
better notation for this?)

  <C super A, C super B, C extends Hashable> ComposableSet<C>
__or__(ComposableSet<A> a, ComposableSet<B> b);
  <C super A, C super B, (for x in A,B,C: CS<x> extends
ComposableSet<x>)> CS<C> __or__(CS<A> a, CS<B> b);

The first would probably be optimized when its arguments are of the
same type, but could fall back on the abstract methods from
ComposableSet, which would return a built-in (frozen?)set.

> > Numbers:
>
> Would you mind helping to write a PEP for numbers? I think I'm running
> out of round tuits just for the collections and the ABC
> infrastructure.

Bah! again. Yes, I'll try to throw something together with some help
from Neal. Who else has been thinking about the Numbers PEP?

> > Cardinal is also useful as the argument to (Sequence * Cardinal).
>
> I think there's little value for this in a dynamically typed language;
> the implementation can raise ValueError or treat negative numbers as
> zero (which is what Python currently does).

True, and any hypothetical type checkers that want to use such
information can define their own subclasses of Integer.

-- 
Namast?,
Jeffrey Yasskin

From ncoghlan at gmail.com  Sat Apr 21 11:17:15 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Apr 2007 19:17:15 +1000
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704201152u7eb9f595l3c4dd444fe2cf86@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>	
	<46288F11.3030505@gmail.com>
	<ca471dc20704201152u7eb9f595l3c4dd444fe2cf86@mail.gmail.com>
Message-ID: <4629D69B.9010508@gmail.com>

Guido van Rossum wrote:
> I do see your point though: object.__hash__ makes it possible to use
> all sorts of objects as dict keys by default, as long as they don't
> implement __eq__. But perhaps this is not such a good idea and we
> should just get rid of it and add __hash__ back to specific objects
> that are useful to use as keys? (E.g. classes but not iterators, to
> take some extreme examples.)

We've tried to do this before - the point where we got stuck was the 
fact that Jython needed a separate location to store the default hash 
method (unlike CPython, Jython's id() and object.__hash__() do different 
things), and we couldn't come up with a reasonable location for it.

I agree that moving the default method from object to Hashable doesn't 
eliminate the errors that arise from redefining __eq__ without 
redefining __hash__, but at least it restricts them to those classes 
which have Hashable somewhere in their MRO.

Cheers,
Nick.



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From michele.cella at gmail.com  Sat Apr 21 16:07:13 2007
From: michele.cella at gmail.com (Michele Cella)
Date: Sat, 21 Apr 2007 16:07:13 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
Message-ID: <f0d5qi$2fu$1@sea.gmane.org>

Guido van Rossum wrote:
> 
>> There's also the question of what the super keyword itself should look like,
>> e.g.
>>
>>   # declaration-style, no dot
>>   res = super currentmethod(arg, arg)
>>   # treat super like self
>>   res = super.currentmethod (arg, arg)
>>   # treat super like self.currentmethod
>>   res = super(arg, arg)
>>
>> I think super.currentmethod(arg, arg) makes the most sense, but that may be
>> because it most closely resembles the current practice. However, it may call
>> the "wrong" supermethod when the class does, for instance, '__repr__ =
>> __str__'.
> 
> Depends on how you define "wrong". :-)
> 
> I am strongly in favor of super.currentmethod(...) if only because
> that's closest to how we've always done it, and there may even be a
> use case for callong some *other* super method.
> 

If super is going to be a keyword (+1) shouldn't it behave like any 
other keyword [1]?

I mean, it's there any keyword (ATM) that uses an attribute access syntax?

Personally, I really like this syntax:

res = super currentmethod(arg, arg)

Why? because it's very consistent (pythonic?) with the way I'm used to 
use any other keyword.

Anyway, thanks for fixing super in python 3000, it's probably the only 
python thing I really hate.

PS
Excuse me if I sound like a noob, that's the first time I post to a 
python dev list.

[1] http://docs.python.org/ref/keywords.html


From guido at python.org  Sat Apr 21 16:37:37 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Apr 2007 07:37:37 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <4629D69B.9010508@gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<46288F11.3030505@gmail.com>
	<ca471dc20704201152u7eb9f595l3c4dd444fe2cf86@mail.gmail.com>
	<4629D69B.9010508@gmail.com>
Message-ID: <ca471dc20704210737u17344d1by3c2e2f4141a6373e@mail.gmail.com>

On 4/21/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Guido van Rossum wrote:
> > I do see your point though: object.__hash__ makes it possible to use
> > all sorts of objects as dict keys by default, as long as they don't
> > implement __eq__. But perhaps this is not such a good idea and we
> > should just get rid of it and add __hash__ back to specific objects
> > that are useful to use as keys? (E.g. classes but not iterators, to
> > take some extreme examples.)
>
> We've tried to do this before - the point where we got stuck was the
> fact that Jython needed a separate location to store the default hash
> method (unlike CPython, Jython's id() and object.__hash__() do different
> things), and we couldn't come up with a reasonable location for it.

I don't understand. Why does the default hash method need to be stored
in a location? Why couldn't it be a method on PyObj or whatever their
base object it? Or are you talking about a place to store the hash
*value*? But why would that need to be stored?

> I agree that moving the default method from object to Hashable doesn't
> eliminate the errors that arise from redefining __eq__ without
> redefining __hash__, but at least it restricts them to those classes
> which have Hashable somewhere in their MRO.

Actually it would significantly reduce the errors because
Hashable.__hash__ is abstract, *forcing* you to redefine it. My
concern is more with the amount of broken code if we remove
object.__hash__.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From michele.cella at gmail.com  Sat Apr 21 16:56:26 2007
From: michele.cella at gmail.com (Michele Cella)
Date: Sat, 21 Apr 2007 16:56:26 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <f0d5qi$2fu$1@sea.gmane.org>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org>
Message-ID: <f0d8n1$bfp$1@sea.gmane.org>

Michele Cella wrote:
> 
> Personally, I really like this syntax:
> 
> res = super currentmethod(arg, arg)
> 

Sorry for replying to myself, what about:

	def mymethod(self, arg):
		super self.mymethod(arg)

this syntax resembles very closely the way you actually invoke a *self* 
method, the only difference is that the super keyword clearly denotes 
that you want to call the super implementation of that method (very much 
like the global keyword).

So you get something like:

	class Example():

		def example(self):
			# do it

		def mymethod(self, arg):
			super self.mymethod(arg)
			self.example()
		
that's all very consistent and explicit IMHO.

Ciao
Michele


From pje at telecommunity.com  Sat Apr 21 17:01:55 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 21 Apr 2007 11:01:55 -0400
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <46291596.1020105@acm.org>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>

At 12:33 PM 4/20/2007 -0700, Talin wrote:
>Additionally, the need to override the classification of an
>object can be done more cleanly using generic functions: In the simplest
>case, one can define a "category membership" generic function that
>simply returns False in the base implementation, and then provide
>overrides that return True for any classes of interest.

Yes, but that's sort of like saying it's better to point a pistol at your 
foot than a shotgun.  :)  Explicitly testing category membership is a bug, 
not a feature; you're far better off defining an overload to do whatever it 
was you were going to make the if-then test do.


From guido at python.org  Sat Apr 21 17:01:41 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Apr 2007 08:01:41 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com>
	<ca471dc20704201524r3116d116ob34e84043798e818@mail.gmail.com>
	<5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com>
Message-ID: <ca471dc20704210801q7d53601fr83c23b8c6579dc3c@mail.gmail.com>

On 4/21/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On 4/20/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/20/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> > > On Sets:
> > > Do you have an example of a data structure that is a Set but not a
> > > ComposableSet?
> >
> > Take for example a set that is a contiguous range of integers,
> > represented by a (lo, hi) tuple. In many cases the return type of
> > union and symmetric difference cannot be represented as an object of
> > the same type. I don't want to force the implementer of such a set to
> > provide implementations for operations they don't need.
>
> Ah, good example. I think that PEPs that propose ABCs should be
> encouraged to describe a few possible implementations, and should be
> particularly encouraged to give examples of the implementations that
> made them split two classes, or omit a method from a particular class.

Yeah, the PEP is short on examples.

> > > Set should inherit from a PartiallyOrdered ABC that defines the
> > > comparison operators.
> >
> > I guess then there would also be a TotallyOrdered ABC deriving from
> > PartiallyOrdered which adds no operations but does add a constraint.
> > I've added both, and made Set derive from PartiallyOrdered.
>
> Cool. Here's some language for their invariants (checking myself with
> the Fortress spec, and stealing its properties nearly verbatim):
>
> PartiallyOrdered:
> This ABC defines the 5 inequality operations <, <=, >=, >, and cmp().

Actually, I'm hoping to eliminate cmp() completely. Also, even if it
doesn't get eliminated, the existence of cmp() suggests a total
ordering.

> (Note that == and != are defined by object.) Classes deriving from
> this ABC should ensure that '<' and '>' form strict partial orders,
> and '<=' and '>=' form weak partial orders, in the sense of
> http://en.wikipedia.org/wiki/Partially_ordered_set. These should be
> compatible with each other and with ==:
>
> Where PO is a subtype of PartiallyOrdered, and a and b are instances of PO,
> cmp(a, b) < 0 ? cmp(b, a) > 0
> cmp(a, b) == 0 ? cmp(b, a) == 0
> cmp(a, b) == None ? cmp(b, a) == None
> a < b ? cmp(a, b) < 0
> a <= b ? cmp(a, b) <= 0
> a == b ? cmp(a, b) == 0
> a >=b ? cmp(a, b) >= 0
> a > b ? cmp(a, b) > 0

> Subclasses of PartiallyOrdered must define one of cmp() or <=.

Actually, Python typically uses < as the single comparison operator
that must be defined (e.g. list.sort() uses < exclusively).

> The other operators are defaulted as follows:
>   def <=(a,b): return cmp(a,b) <= 0
>   def cmp(a,b):
>     if a<=b:
>       if b<=a: return 0
>       else: return -1
>     else:
>       if b<=a: return 1
>       else: return None
>   def ==(a,b): return a<=b and b<=a
>   def <(a,b): return a<=b and not b<=a
>   def >=(a,b): return b<=a
>   def >(a,b): return b<a
>
> Open Issues: 1) Is it reasonable to let cmp() return None for
> unordered pairs, or should we instead define a 4-value enumeration for
> it to return?

As I said, I'd rather kill it or at least ignore it for this part.

> 2) The @abstractmethod decorator isn't enough to specify
> that subclasses must override at least one of a group of methods, and
> there are classes with even more complex requirements like "subclasses
> must override a or (b and (c or d))".

I guess those requirements will have to be expressed in English. I
don't see the point to invent a decorator mini-language or other
mechanism that's powerful enough to express such possibilities. If
someone *really* wants they can implement a new metaclass that checks
these, but IMO it's a rather silly waste of time since the metaclass
can't check that any of the methods are implemented correcttly, so the
exercise is kind of pointless (super-strict in one area while 100%
lenient in another).

> 3) http://docs.python.org/ref/customization.html#l2h-187 still implies
> that < defines a total order over all python objects. Has that changed
> for py3k? 4) How should we handle the reflected forms of these
> operators?

Yes, in Py3k < <= > >= are not defined by default.  We may not have
updated the docs though.

> TotallyOrdered:
> This ABC derives from PartiallyOrdered. It adds no new operations but
> implies a promise that <, <=, >, and >= are total orders in the sense
> of http://en.wikipedia.org/wiki/Totally_ordered_set, and that cmp(a,b)
> never returns None. Given this additional restriction, subclasses only
> need to define one of <, <=, or cmp(), and the rest of the methods can
> be defaulted.
>
> Open Issue: How does the TotallyOrdered property inherit? If Real (an
> abstract class) inherits from TotallyOrdered, do different subclasses
> need to be totally comparable with each other? If long (a concrete
> class) inherits from TotallyOrdered, and I subclass long, do I have to
> ensure that my instances are totally comparable with other longs? I
> think 'yes' to the second, to obey the Liskov substitution principle.
> I don't know about the first: for Real it seems fine, but it might
> inhibit refactoring the ABC hierarchy.

Perhaps this is more appropriate for the Numeric ABCs PEP. IMO the
only concern is the realities of floating point arithmetic (whether
binary or decimal); I would be fine with some language that explains
that numeric types computed/represented with limited precision may
violate the rules. The only alternative I see is to force float and
decimal (and hence Real, Complex and Number) to inherit from
PartiallyOrdered instead of TotallyOrdered, which sounds like a shame
-- or is it?

> > > The final specification of Set should spell out the invariants, but
> > > they're clear enough that the PEP probably doesn't need to. It's odd,
> > > though, that you mention optimizations to __eq__ and __le__ but not
> > > their semantics.
> >
> > The semantics are kind of obvious. :-)
> >
> > Text spelling out the invariants in more detail that I can just paste
> > into the PEP is most welcome!
>
> Bah! ;)
>
> Set:
> Sets should implement an efficient (no worse than O(log N))
> __contains__ operation. Sets are defined by their extensions.

What's an extension?

> Subclasses may override the following default definitions of __eq__
> and __le__ with more efficient versions, but they must respect the
> same semantics:
>   def __eq__(a, b): return len(a) == len(b) and all(x in a for x in b)
>   def __le__(a, b): return len(a) <= len(b) and all(x in a for x in b)
>
> ComposableSet:
> ... mathematical meaning of set composition:
> Where a and b are instances of ComposableSet and __op__(a,b) is defined:
>   x in (a | b) ? x in a or x in b
>   x in (a & b) ? x in a and x in b
>   x in (a ^ b) ? x in a != x in b
>   x in (a - b) ? x in a and not x in b
>
> > > ComposableSet's open issues:
> > > > Should we just pick a concrete return type (e.g. set)?
> > >
> > > For set compositions, you should say whether the arguments need to be
> > > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do,
> > > then you don't need a concrete return type; if they don't, then you
> > > do, and the default implementations should use it.
> >
> > You may be misunderstanding what I meant. I certainly don't want to
> > constrain the potential return type of the composition operations
> > (except that they should themselves be composable sets). The question
> > about picking a specific return type was meant as a way out to
> > providing a concrete default implementation. I'm still on the fence of
> > this, as it could be a big waste of time (I imagine that composing
> > Patricia trees can be done more efficiently than iterating over the
> > elements).
>
> Borrowing some syntax from Java generics, which of:
>   ComposableSet __or__(ComposableSet a, ComposableSet b);
>   <CS extends ComposableSet> ComposableSet __or__(CS a, CS b);
>   <CS extends ComposableSet> CS __or__(CS a, CS b);
> do you mean to define?

Can't say I understand that. The minimum requirement is that the
return type is a subclass of ComposableSet. *Some* subclasses CS may
themselves guarantee that the result is always a subclass of CS, but I
don't want to force that. Returning a 'set' in the default
implementation satisfies the former but not the latter. This is in
itself fine (CS can override __or__ to return a CS). But it could be
expensive -- even if the author of CS doesn't care about the return
type being a CS itself, they might still care about th efficiency.

> I think they have different implications about
> what defaults you need to provide. The second is probably out because
> it would mean (a|b|c) would likely break. This gets even more complex
> if we consider the types of the contents of the sets... Here's a
> possible overloaded signature for __or__. (Do you already have a
> better notation for this?)

I think this is more appropriate for other languages.

>   <C super A, C super B, C extends Hashable> ComposableSet<C>
> __or__(ComposableSet<A> a, ComposableSet<B> b);
>   <C super A, C super B, (for x in A,B,C: CS<x> extends
> ComposableSet<x>)> CS<C> __or__(CS<A> a, CS<B> b);

It's Saturday AM, I'm not even going to try to parse that. :-)

> The first would probably be optimized when its arguments are of the
> same type, but could fall back on the abstract methods from
> ComposableSet, which would return a built-in (frozen?)set.
>
> > > Numbers:
> >
> > Would you mind helping to write a PEP for numbers? I think I'm running
> > out of round tuits just for the collections and the ABC
> > infrastructure.
>
> Bah! again. Yes, I'll try to throw something together with some help
> from Neal. Who else has been thinking about the Numbers PEP?
>
> > > Cardinal is also useful as the argument to (Sequence * Cardinal).
> >
> > I think there's little value for this in a dynamically typed language;
> > the implementation can raise ValueError or treat negative numbers as
> > zero (which is what Python currently does).
>
> True, and any hypothetical type checkers that want to use such
> information can define their own subclasses of Integer.
>
> --
> Namast?,
> Jeffrey Yasskin
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Apr 21 17:06:37 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Apr 2007 08:06:37 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<46291596.1020105@acm.org>
	<5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>
Message-ID: <ca471dc20704210806u1c92be42y53e8949716dc7982@mail.gmail.com>

On 4/21/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 12:33 PM 4/20/2007 -0700, Talin wrote:
> >Additionally, the need to override the classification of an
> >object can be done more cleanly using generic functions: In the simplest
> >case, one can define a "category membership" generic function that
> >simply returns False in the base implementation, and then provide
> >overrides that return True for any classes of interest.
>
> Yes, but that's sort of like saying it's better to point a pistol at your
> foot than a shotgun.  :)  Explicitly testing category membership is a bug,
> not a feature; you're far better off defining an overload to do whatever it
> was you were going to make the if-then test do.

I don't believe that's always true. I can imagine use cases where, if
I were writing it in an if-then-else fashion, I'd be writing the
*same* if-test several times as part of a particular algorithm,
without having an easy way to refactor the algorithm so that there's
only one place testing it. Turning each place where the if-test is
repeated into a different GF might be a wasted of time; turning just
the test into a GF seems just right. (IN THAT PARTICULAR CASE.)

Note that I don't object against the idea that usually a GF is better;
I just don't think it always is so.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Sat Apr 21 17:21:02 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Apr 2007 01:21:02 +1000
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704210737u17344d1by3c2e2f4141a6373e@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>	
	<46288F11.3030505@gmail.com>	
	<ca471dc20704201152u7eb9f595l3c4dd444fe2cf86@mail.gmail.com>	
	<4629D69B.9010508@gmail.com>
	<ca471dc20704210737u17344d1by3c2e2f4141a6373e@mail.gmail.com>
Message-ID: <462A2BDE.5030101@gmail.com>

Guido van Rossum wrote:
> On 4/21/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> We've tried to do this before - the point where we got stuck was the
>> fact that Jython needed a separate location to store the default hash
>> method (unlike CPython, Jython's id() and object.__hash__() do different
>> things), and we couldn't come up with a reasonable location for it.
> 
> I don't understand. Why does the default hash method need to be stored
> in a location? Why couldn't it be a method on PyObj or whatever their
> base object it? Or are you talking about a place to store the hash
> *value*? But why would that need to be stored?

I believe the point was that currently object.__hash__ combines with 
object.__eq__ to give identity based comparison by default. If 
object.__hash__ is removed, then an appropriate hash function for 
identity-based comparison needs to be available somewhere else in the 
standard library - it was pointed out (by Samuele?) that id() is too 
expensive in Jython to be used directly as such a hash function.

I think the thread then devolved into a bikeshed discussion about 
possible different names (object.default_hash, sys.id_hash, etc, etc, 
...), which is why nothing was ever actually done about it.

Cheers,
Nick.

P.S. This discussion happened quite some time ago, and I haven't trawled 
through the list archive to check I'm remembering the details correctly. 
But I'm pretty sure I have the gist correct.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Sat Apr 21 18:43:17 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 21 Apr 2007 12:43:17 -0400
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704210806u1c92be42y53e8949716dc7982@mail.gmail.co
 m>
References: <5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>
	<ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<46291596.1020105@acm.org>
	<5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070421122647.02b67e70@sparrow.telecommunity.com>

At 08:06 AM 4/21/2007 -0700, Guido van Rossum wrote:
>On 4/21/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 12:33 PM 4/20/2007 -0700, Talin wrote:
> > >Additionally, the need to override the classification of an
> > >object can be done more cleanly using generic functions: In the simplest
> > >case, one can define a "category membership" generic function that
> > >simply returns False in the base implementation, and then provide
> > >overrides that return True for any classes of interest.
> >
> > Yes, but that's sort of like saying it's better to point a pistol at your
> > foot than a shotgun.  :)  Explicitly testing category membership is a bug,
> > not a feature; you're far better off defining an overload to do whatever it
> > was you were going to make the if-then test do.
>
>I don't believe that's always true. I can imagine use cases where, if
>I were writing it in an if-then-else fashion, I'd be writing the
>*same* if-test several times as part of a particular algorithm,
>without having an easy way to refactor the algorithm so that there's
>only one place testing it. Turning each place where the if-test is
>repeated into a different GF might be a wasted of time; turning just
>the test into a GF seems just right. (IN THAT PARTICULAR CASE.)

I understand the motivation; it's just been my personal experience that 
when I encounter such a case, there are always other parts of the library 
that will need those "different" GF's, because they end up being expressive 
and relevant to the problem domain of the library as a whole.  Those 
"different" GF's then become a clean specification of the interface(s) 
required by the library.

For example, pydoc frequently asks whether something 'isroutine()' or 
'ismethod()', but if it were factored to use elementary GF's instead, we 
would have things like 'getsignature()' and 'getmembers()' instead, and it 
would then be straightforward to document how to make a new kind of object 
work with it.  (i.e. if you have members, implement a getmembers() 
overload, if you're callable, implement a getsignature() overload, etc.)

So, if you're writing any sort of library, you will usually get a solid 
payoff from doing factoring these primitives, given the high probability 
that your own library will use them more than once, and the ease of 
documenting the idea of having a call signature vs. trying to explain (or 
making your users figure out) the nine jillion consequences of having 
'isroutine()' return True.  I suppose you could say it's a tradeoff of 
"wasting" your time, vs. wasting your users' time.


>Note that I don't object against the idea that usually a GF is better;
>I just don't think it always is so.

I can certainly conceive that there might be an exception, I just have yet 
to encounter it.  But then, my focus is nearly always on library 
development, where my "customer" is the library's user.  Within that niche, 
I think there's a much bigger case to be made that it is at least *nearly* 
always so.


From talin at acm.org  Sat Apr 21 22:51:57 2007
From: talin at acm.org (Talin)
Date: Sat, 21 Apr 2007 13:51:57 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <bc1fe4bd0704211321o446cfd76u57bb1ff9e37c9b84@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bc1fe4bd0704211321o446cfd76u57bb1ff9e37c9b84@mail.gmail.com>
Message-ID: <462A796D.50700@acm.org>

John Reese wrote:
> "This PEP proposes a particular strategy for organizing these tests
> known as Abstract Base Classes, or ABC. ABCs are simply Python classes
> that are added into an object's inheritance tree to signal certain
> features of that object to an external inspector. Tests are done using
> isinstance(), and the presence of a particular ABC means that the test
> has passed."
> 
> I'd like to see another few lines in the rationale section explaining
> the intent of the concrete methods.  For example, inheriting from
> abc.Mapping would have a similar effect to inheriting from
> UserDict.DictMixin -- by overriding it's few abstract methods you get
> close to the full complement of dict's convenience methods.  But from
> the open issues and other discussion in this thread, I don't think the
> intention is to provide full coverage of the convenience methods in
> all of the builtin classes in their corresponding ABCs, but merely to
> expand the number of methods you can assume given an ABC.
> 
> I'd like an explanation of exactly where the line is being drawn in
> terms of what convenience methods ABCs will provide through concrete
> methods.

OK how about this. Replace the paragraph immediately following the one 
you quote with these two paragraphs:

In addition, the ABCs define a minimal set of methods that establish the 
characteristic behavior of the type. Code that discriminates objects 
based on their ABC type can trust that those methods will always be 
present. Each of these methods are accompanied by an generalized 
abstract semantic definition that is described in the documentation for 
the ABC. These standard semantic definitions are not enforced, but are 
strongly recommended.

Like all other things in Python, these promises are in the nature of a 
gentlemen's agreement, which in this case means that while the language 
does enforce some of the promises made in the ABC, it is up to the 
implementer of the concrete class to insure that the remaining ones are 
kept.

-- Talin


From jimjjewett at gmail.com  Sun Apr 22 06:20:43 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 22 Apr 2007 00:20:43 -0400
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
Message-ID: <fb6fbf560704212120u16061536wc25f269a1dd87407@mail.gmail.com>

On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> On 4/19/07, Brett Cannon <brett at python.org> wrote:
> > On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> > > I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:

> > * "Should we also implement the issubset and issuperset methods found
> > on the set type in Python 2? As these are just aliases for __le__ and
> > __ge__, I'm tempted to leave these out."

> > Leave them out.  Not terribly needed plus it is better to start out
> > small.  They can easily be added later if called for.

I think the names are more sensible than repurposing the numeric
operators.  The "concrete" implementation can just forward to other
name, so it doesn't cost much.

> I doubt that 'set' will be inherited from much (except for trivial
> stuff) but returning a 'set' does make some sense. Although for
> trivial subclasses (e.g. adding a different repr or a brand new
> method) you'd like the results also to be an instance of the subclass.
> If we could find a way to do that it would be best.

You could use a variant of the numeric operator resolution.

The left class wins, unless the right class is a subclass of the left.
If this class does not have a constructor that accepts an iterable,
(maybe fall back to the other class, then) fall back to a builtin set.

Going back to the PEP itself, in the ABC support framework, I would change:

"We define the following four new built-in objects that help defining ABCs:"
->
"We define the following four new objects (in module abc) that help
defining ABCs:"

and in the @abstractmethod description,


"Such a method may be called"
->
"These abstract methods may still be called"

In the Hashable section:

> Another constraint is that hashable objects, once created,
> should never change their value (as compared by ==) or their hash
> value. If a class cannot guarantee this, it should not derive from
> Hashable; if it cannot guarantee this for certain instances only,
> __hash__ for those instances should raise a TypeError exception.

Why not just return -1 (unless/until the value is stable)?  Is the -1
special case being phased out too?

...
I would say that Searchable might be worth separating into a Container
subclass, but don't worry about the speed differences.  Many
containers are small enough that constant effects are more important
than big-O.  When big-O does matter, then the user may well want to
distinguish between the O(1) and O(log N) case.  I think this gets
detailed enough that it isn't worth putting in the common library.

Why must a HashableSet or MutableSet be composable?  If this is
because you figure any useful set has those properties (and I don't
think so, when doing uniquification), then Set and ComposableSet
should become BasicSet and Set?

...

Note that in the current sandbox  sandbox (which doesn't have
*Ordered), the name BasicSet is used for a Set which isn't Sized.
(Except that Set doesn't actually inherit from BasicSet, as I assume
it should do.)

Continuing with the current sandbox

Why must @abstractmethod come last, when mixed with other decorators?

IterableMapping needs to override __iter__; as written, it returns the
empty iterator.

Should Sequence.index take optional start and stop arguments?

    def index(self, value, start=0, stop=None):
        if stop is None:
            stop = len(self)
        for i, elem in enumerate(self):
            if i < start:
                continue
            if stop < i:
                break
            if elem == value:
                return i
        raise ValueError

-jJ

From jtr at miskatonic.nu  Sat Apr 21 22:21:26 2007
From: jtr at miskatonic.nu (John Reese)
Date: Sat, 21 Apr 2007 13:21:26 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
Message-ID: <bc1fe4bd0704211321o446cfd76u57bb1ff9e37c9b84@mail.gmail.com>

On 4/19/07, Guido van Rossum <guido at python.org> wrote:
> I've started a PEP on Abstract Base Classes (ABCs), PEP 3119:
>
>     http://www.python.org/dev/peps/pep-3119/
>
> While I'm not ready yet to answer tough questions about this compared
> to alternative proposals, I *am* ready for feedback on the various
> open issues sprinkled throughout the current text, especially
> concerning decisions regarding exactly which operations to include in
> the various ABCs, and also regarding the level of detail required in
> the PEP.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/john.reese%40gmail.com
>


"This PEP proposes a particular strategy for organizing these tests
known as Abstract Base Classes, or ABC. ABCs are simply Python classes
that are added into an object's inheritance tree to signal certain
features of that object to an external inspector. Tests are done using
isinstance(), and the presence of a particular ABC means that the test
has passed."

I'd like to see another few lines in the rationale section explaining
the intent of the concrete methods.  For example, inheriting from
abc.Mapping would have a similar effect to inheriting from
UserDict.DictMixin -- by overriding it's few abstract methods you get
close to the full complement of dict's convenience methods.  But from
the open issues and other discussion in this thread, I don't think the
intention is to provide full coverage of the convenience methods in
all of the builtin classes in their corresponding ABCs, but merely to
expand the number of methods you can assume given an ABC.

I'd like an explanation of exactly where the line is being drawn in
terms of what convenience methods ABCs will provide through concrete
methods.

From jyasskin at gmail.com  Sun Apr 22 10:12:33 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sun, 22 Apr 2007 01:12:33 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704210801q7d53601fr83c23b8c6579dc3c@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<5d44f72f0704200048p2d410de9x516e78a888747278@mail.gmail.com>
	<ca471dc20704201524r3116d116ob34e84043798e818@mail.gmail.com>
	<5d44f72f0704210030r4fc572f0ta07b6de6d4ed08cd@mail.gmail.com>
	<ca471dc20704210801q7d53601fr83c23b8c6579dc3c@mail.gmail.com>
Message-ID: <5d44f72f0704220112j5cc5bc53u82171e86779c6368@mail.gmail.com>

On 4/21/07, Guido van Rossum <guido at python.org> wrote:
> On 4/21/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> > PartiallyOrdered:
> > This ABC defines the 5 inequality operations <, <=, >=, >, and cmp().
>
> Actually, I'm hoping to eliminate cmp() completely. Also, even if it
> doesn't get eliminated, the existence of cmp() suggests a total
> ordering.

Ok. It's more efficient when comparing large structures, but it's not
absolutely needed. In some languages it can be more convenient too,
but that's because you take advantage of their case statements.

> > Subclasses of PartiallyOrdered must define one of cmp() or <=.
>
> Actually, Python typically uses < as the single comparison operator
> that must be defined (e.g. list.sort() uses < exclusively).

With only a partial order, you need both < and == to define the other
operators, or you can do it with just <=. A total order allows you to
do it with either < or <= alone. sort(), at least in C++, needs at
least a "strict weak ordering"
(http://www.sgi.com/tech/stl/StrictWeakOrdering.html), which lets you
define equivalence but not equality using just <.

> > > > The final specification of Set should spell out the invariants, but
> > > > they're clear enough that the PEP probably doesn't need to. It's odd,
> > > > though, that you mention optimizations to __eq__ and __le__ but not
> > > > their semantics.
> > >
> > > The semantics are kind of obvious. :-)
> > >
> > > Text spelling out the invariants in more detail that I can just paste
> > > into the PEP is most welcome!
> >
> > Bah! ;)
> >
> > Set:
> > Sets should implement an efficient (no worse than O(log N))
> > __contains__ operation. Sets are defined by their extensions.
>
> What's an extension?

Oops. It's the collection of objects 'x' for which 'x in set' is true.
We can probably just skip that sentence.

> > Subclasses may override the following default definitions of __eq__
> > and __le__ with more efficient versions, but they must respect the
> > same semantics:
> >   def __eq__(a, b): return len(a) == len(b) and all(x in a for x in b)
> >   def __le__(a, b): return len(a) <= len(b) and all(x in a for x in b)
> >
> > ComposableSet:
> > ... mathematical meaning of set composition:
> > Where a and b are instances of ComposableSet and __op__(a,b) is defined:
> >   x in (a | b) ? x in a or x in b
> >   x in (a & b) ? x in a and x in b
> >   x in (a ^ b) ? x in a != x in b
> >   x in (a - b) ? x in a and not x in b
> >
> > > > ComposableSet's open issues:
> > > > > Should we just pick a concrete return type (e.g. set)?
> > > >
> > > > For set compositions, you should say whether the arguments need to be
> > > > uniform. (Is (IntSet(1,2,3) | set('a','b','c')) allowed?) If they do,
> > > > then you don't need a concrete return type; if they don't, then you
> > > > do, and the default implementations should use it.
> > >
> > > You may be misunderstanding what I meant. I certainly don't want to
> > > constrain the potential return type of the composition operations
> > > (except that they should themselves be composable sets). The question
> > > about picking a specific return type was meant as a way out to
> > > providing a concrete default implementation. I'm still on the fence of
> > > this, as it could be a big waste of time (I imagine that composing
> > > Patricia trees can be done more efficiently than iterating over the
> > > elements).
> >
> > Borrowing some syntax from Java generics, which of:
> >   ComposableSet __or__(ComposableSet a, ComposableSet b);
> >   <CS extends ComposableSet> ComposableSet __or__(CS a, CS b);
> >   <CS extends ComposableSet> CS __or__(CS a, CS b);
> > do you mean to define?
>
> Can't say I understand that. The minimum requirement is that the
> return type is a subclass of ComposableSet. *Some* subclasses CS may
> themselves guarantee that the result is always a subclass of CS, but I
> don't want to force that. Returning a 'set' in the default
> implementation satisfies the former but not the latter. This is in
> itself fine (CS can override __or__ to return a CS). But it could be
> expensive -- even if the author of CS doesn't care about the return
> type being a CS itself, they might still care about th efficiency.

I think you want my first option then. How about:

"The default implementation ensures that __or__, __and__, __sub__, and
__xor__ do the right thing for any ComposableSet with Hashable
elements, but because it iterates over both arguments and returns a
built-in set, it may be slow. Subclasses should optimize these
operations for types they know about but continue to support
heterogenous operations by delegating to ComposableSet.

"This should be rare: If you define a ComposableSet that can hold
non-Hashable elements, your users should expect that they can compose
sets with identical types, but not necessarily sets with different
types, unless, of course, you give them extra guarantees."

> > I think they have different implications about
> > what defaults you need to provide. The second is probably out because
> > it would mean (a|b|c) would likely break. This gets even more complex
> > if we consider the types of the contents of the sets... Here's a
> > possible overloaded signature for __or__. (Do you already have a
> > better notation for this?)
>
> I think this is more appropriate for other languages.

Having the compiler check the signature is definitely more appropriate
for other languages, or at least a third-party module. Knowing what
types we handle as arguments and what that type the result can be is
important for any language. My notation is, of course, probably not
the best we could come up with. :)

-- 
Namast?,
Jeffrey Yasskin

From pedronis at openendsystems.com  Sun Apr 22 12:35:47 2007
From: pedronis at openendsystems.com (Samuele Pedroni)
Date: Sun, 22 Apr 2007 12:35:47 +0200
Subject: [Python-3000] question Re:  Need help completing ABC pep
In-Reply-To: <ca471dc20704210806u1c92be42y53e8949716dc7982@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>	<46291596.1020105@acm.org>	<5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>
	<ca471dc20704210806u1c92be42y53e8949716dc7982@mail.gmail.com>
Message-ID: <462B3A83.1060204@openendsystems.com>

Guido van Rossum wrote:
>
>
> I don't believe that's always true. I can imagine use cases where, if
> I were writing it in an if-then-else fashion, I'd be writing the
> *same* if-test several times as part of a particular algorithm,
> without having an easy way to refactor the algorithm so that there's
> only one place testing it. Turning each place where the if-test is
> repeated into a different GF might be a wasted of time; turning just
> the test into a GF seems just right. (IN THAT PARTICULAR CASE.)
>
> Note that I don't object against the idea that usually a GF is better;
> I just don't think it always is so.
>
>   

If we keep s % d (do we? reading the list archives and the peps didn't 
give an answer, I suppose it depends if PEP 3101
is accepted):

will it check if d is a Mapping or continue to use the current heuristics?

I suppose this is a legitimate question for other corners of the 
language, std lib.






From greg.ewing at canterbury.ac.nz  Mon Apr 23 00:07:38 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 23 Apr 2007 10:07:38 +1200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <f0d5qi$2fu$1@sea.gmane.org>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org>
Message-ID: <462BDCAA.7020501@canterbury.ac.nz>

Michele Cella wrote:

> Personally, I really like this syntax:
> 
> res = super currentmethod(arg, arg)
> 
> Why? because it's very consistent (pythonic?) with the way I'm used to 
> use any other keyword.

But it's inconsistent with the way every other
method is called.

--
Greg

From aahz at pythoncraft.com  Mon Apr 23 00:47:08 2007
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 22 Apr 2007 15:47:08 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <462BDCAA.7020501@canterbury.ac.nz>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
Message-ID: <20070422224708.GA7208@panix.com>

On Mon, Apr 23, 2007, Greg Ewing wrote:
> Michele Cella wrote:
>> 
>> Personally, I really like this syntax:
>> 
>> res = super currentmethod(arg, arg)
>> 
>> Why? because it's very consistent (pythonic?) with the way I'm used to 
>> use any other keyword.
> 
> But it's inconsistent with the way every other method is called.

But we also don't have any keywords with methods "attached".  I don't
strongly favor Michele's approach, but it doesn't seem to me that there
are any strong arguments in favor of super.method().
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"...string iteration isn't about treating strings as sequences of strings, 
it's about treating strings as sequences of characters.  The fact that
characters are also strings is the reason we have problems, but characters 
are strings for other good reasons."  --Aahz

From brett at python.org  Mon Apr 23 03:45:18 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Apr 2007 18:45:18 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
Message-ID: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>

This PEP is to change the ``if __name__ == "__main__": ...`` idiom to
``if __name__ == sys.main: ...``  so that you at least have a chance
to execute module in a package that use relative imports.

Ran this PEP past python-ideas.  Stopped the discussion there when too
many new ideas were being proposed.  =)  I have listed all of them in
the Rejected Ideas section, although if overwhelming support for one
comes forward the PEP can shift to one of them.


---------------------------------------------

PEP: XXX
Title: Delineation of the main module
Version: $Revision: 52916 $
Last-Modified: $Date: 2006-12-04 11:59:42 -0800 (Mon, 04 Dec 2006) $
Author: Brett Cannon
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: XXX-Apr-2007

Abstract
========

Because of how name resolution works for relative imports in a world
where PEP 328 is implemented, the ability to execute modules within a
package ceases being possible.  This failing stems from the fact that
the module being executed as the "main" module replaces its
``__name__`` attribute with ``"__main__"`` instead of leaving it as
the absolute name of the module.  This breaks import's ability
to resolve relative imports from the main module into absolute names.

In order to resolve this issue, this PEP proposes to change how the
main module is delineated.  By leaving the ``__name__`` attribute in
a module alone and setting ``sys.main`` to the name of the main
module this will allow at least some instances of executing a module
within a package that uses relative imports.

This PEP does not address the idea of introducing a module-level
function that is automatically executed like PEP 299 proposes.


The Problem
===========

With the introduction of PEP 328, relative imports became dependent on
the ``__name__`` attribute of the module performing the import.  This
is because the use of dots in a relative import are used to strip away
parts of the calling module's name to calculate where in the package
hierarchy an import should fall (prior to PEP 328 relative
imports could fail and would fall back on absolute imports which had a
chance of succeeding).

For instance, consider the import ``from .. import spam`` made from the
``bacon.ham.beans`` module (``bacon.ham.beans`` is not a package
itself, i.e., does not define ``__path__``).  Name resolution of the
relative import takes the caller's name (``bacon.ham.beans``), splits
on dots, and then slices off the last n parts based on the level
(which is 2).  In this example both ``ham`` and ``beans`` are dropped
and ``spam`` is joined with what is left (``bacon``).  This leads to
the proper import of the module ``bacon.spam``.

This reliance on the ``__name__`` attribute of a module when handling
relative imports becomes an issue when executing a script within a
package.  Because the executing script has its name set to
``'__main__'``, import cannot resolve any relative imports, leading to
an ``ImportError``.

For example, assume we have a package named ``bacon`` with an
``__init__.py`` file containing::

  from . import spam

Also create a module named ``spam`` within the ``bacon`` package (it
can be an empty file).  Now if you try to execute the ``bacon``
package (either through ``python bacon/__init__.py`` or
``python -m bacon``) you will get an ``ImportError`` about trying to
do a relative import from within a non-package.  Obviously the import
is valid, but because of the setting of ``__name__`` to ``'__main__'``
import thinks that ``bacon/__init__.py`` is not in a package since no
dots exist in ``__name__``.  To see how the algorithm works in more
detail, see ``importlib.Import._resolve_name()`` in the sandbox
[#importlib]_.

Currently a work-around is to remove all relative imports in the
module being executed and make them absolute.  This is unfortunate,
though, as one should not be required to use a specific type of
resource in order to make a module in a package be able to be
executed.


The Solution
============

The solution to the problem is to not change the value of ``__name__``
in modules.  But there still needs to be a way to let executing code
know it is being executed as a script.  This is handled with a new
attribute in the ``sys`` module named ``main``.

When a module is being executed as a script, ``sys.main`` will be set
to the name of the module.  This changes the current idiom of::

  if __name__ == '__main__':
      ...

to::

  import sys
  if __name__ == sys.main:
      ...

The newly proposed solution does introduce an added line of
boilerplate which is a module import.  But as the solution does not
introduce a new built-in or module attribute (as discussed in
`Rejected Ideas`_) it has been deemed worth the extra line.

Another issue with the proposed solution (which also applies to all
rejected ideas as well) is that it does not directly solve the problem
of discovering the name of a file.  Consider ``python bacon/spam.py``.
By the file name alone it is not obvious whether ``bacon`` is a
package.  In order to properly find this out both the current
direction must exist on ``sys.path`` as well as ``bacon/__init__.py``
existing.

But this is the simple example.  Consider ``python ../spam.py``.  From
the file name alone it is not at all clear if ``spam.py`` is in a
package or not.  One possible solution is to find out what the
absolute name of ``..``, check if a file named ``__init__.py`` exists,
and then look if the directory is on ``sys.path``.  If it is not, then
continue to walk up the directory until no more ``__init__.py`` files
are found or the directory is found on ``sys.path``.

This could potentially be an expensive process.  If the package depth
happens to be deep then it could require a large amount of disk access
to discover where the package is anchored on ``sys.path``, if at all.
The stat calls alone can be expensive if the file system the executed
script is on is something like NFS.

Because of these issues, only when the ``-m`` command-line argument
(introduced by PEP 338) is used will ``__name__`` be set.  Otherwise
the fallback semantics of setting ``__name__`` to ``"__main__"`` will
occur.  ``sys.main`` will still be set to the proper value,
regardless of what ``__name__`` is set to.


Implementation
==============

When the ``-m`` option is used, ``sys.main`` will be set to the
argument passed in.  ``sys.argv`` will be adjusted as it is currently.
Then the equivalent of ``__import__(self.main)`` will occur.  This
differs from current semantics as the ``runpy`` module fetches the
code object for the file specified by the module name in order to
explicitly set ``__name__`` and other attributes.  This is no longer
needed as import can perform its normal operation in this situation.

If a file name is specified, then ``sys.main`` will be set to
``"__main__"``.  The specified file will then be read and have a code
object created and then be executed with ``__name__`` set to
``"__main__"``.  This mirrors current semantics.


Transition Plan
===============

In order for Python 2.6 to be able to support both the current
semantics and the proposed semantics, ``sys.main`` will always be set
to ``"__main__"``.  Otherwise no change will occur for Python 2.6.
This unfortunately means that no benefit from this change will occur
in Python 2.6, but it maximizes compatibility for code that is to
work as much as possible with 2.6 and 3.0.

To help transition to the new idiom, 2to3 [#2to3]_ will gain a rule to
transform the current ``if __name__ == '__main__': ...`` idiom to the
new one.  This will not help with code that checks ``__name__``
outside of the idiom, though.


Rejected Ideas
==============

``__main__`` built-in
---------------------

A counter-proposal to introduce a built-in named ``__main__``.
The value of the built-in would be the name of the module being
executed (just like the proposed ``sys.main``).  This would lead to a
new idiom of::

  if __name__ == __main__:
      ...

A drawback is that the syntactic difference is subtle; the dropping
of quotes around "__main__".  Some believe that for existing Python
programmers bugs will be introduced where the quotation marks will be
put on by accident.  But one could argue that the bug would be
discovered quickly through testing as it is a very shallow bug.

While the name of built-in could obviously be different (e.g.,
``main``) the other drawback is that it introduces a new built-in.
With a simple solution such as ``sys.main`` being possible without
adding another built-in to Python, this proposal was rejected.

``__main__`` module attribute
-----------------------------

Another proposal was to add a ``__main__`` attribute to every module.
For the one that was executing as the main module, the attribute would
have a true value while all other modules had a false value.  This has
a nice consequence of simplify the main module idiom to::

  if __main__:
      ...

The drawback was the introduction of a new module attribute.  It also
required more integration with the import machinery than the proposed
solution.


Use ``__file__`` instead of ``__name__``
----------------------------------------

Any of the proposals could be changed to use the ``__file__``
attribute on modules instead of ``__name__``, including the current
semantics.  The problem with this is that with the proposed solutions
there is the issue of modules having no ``__file__`` attribute defined
or having the same value as other modules.

The problem that comes up with the current semantics is you still have
to try to resolve the file path to a module name for the import to
work.


Special string subclass for ``__name__`` that overrides ``__eq__``
------------------------------------------------------------------

One proposal was to define a subclass of ``str`` that overrode the
``__eq__`` method so that it would compare equal to ``"__main__"`` as
well as the actual name of the module.  In all other respects the
subclass would be the same as ``str``.

This was rejected as it seemed like too much of a hack.


References
==========

.. [#2to3]  2to3 tool
    (http://svn.python.org/view/sandbox/trunk/2to3/) [ViewVC]

.. [#importlib] importlib
    (http://svn.python.org/view/sandbox/trunk/import_in_py/importlib.py?view=markup)
    [ViewVC]



Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From lists at cheimes.de  Mon Apr 23 04:20:08 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 23 Apr 2007 04:20:08 +0200
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
Message-ID: <f0h54s$8s1$1@sea.gmane.org>

Brett Cannon wrote:
> Implementation
> ==============
> 
> When the ``-m`` option is used, ``sys.main`` will be set to the
> argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> Then the equivalent of ``__import__(self.main)`` will occur.  This
> differs from current semantics as the ``runpy`` module fetches the
> code object for the file specified by the module name in order to
> explicitly set ``__name__`` and other attributes.  This is no longer
> needed as import can perform its normal operation in this situation.

__import__(self.main)? I guess you wanted to write __import__(sys.main) :)

 > The newly proposed solution does introduce an added line of
 > boilerplate which is a module import.  But as the solution does not
 > introduce a new built-in or module attribute (as discussed in
 > `Rejected Ideas`_) it has been deemed worth the extra line.

Someone on python-idea brought up the idea to add the sys module to 
builtins. The feature would remove the requirement for an extra line and 
the costs should be minimal or even zero since sys is a special builtin 
module.

Some people including me like to resurrect the PEP 299 __main__() 
function idea. http://www.python.org/dev/peps/pep-0299/

Christian


From jimjjewett at gmail.com  Mon Apr 23 05:05:14 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 22 Apr 2007 23:05:14 -0400
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function Currently
	Being Defined (this)
Message-ID: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>

(Please note that several groups were Cc'd.  For now, please limit
followups to python-3000.  This would *probably* be backported to 2.6,
but that wouldn't be decided until the implementation strategy was
settled.)

PEP: 30XX
Title: Access to Module/Class/Function Currently Being Defined (this)
Version: $Revision$
Last-Modified: $Date$
Author: Jim J. Jewett <jimjjewett at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 22-Apr-2007
Python-Version: 3.0
Post-History: 22-Apr-2007


Abstract

    It is common to need a reference to the current module, class,
    or function, but there is currently no entirely correct way to
    do this.  This PEP proposes adding the keywords __module__,
    __class__, and __function__.


Rationale

    Many modules export various functions, classes, and other objects,
    but will perform additional activities (such as running unit tests)
    when run as a script.  The current idiom is to test whether the
    module's name has been set to magic value.

        if __name__ == "__main__": ...

    More complicated introspection requires a module to (attempt to)
    import itself.  If importing the expected name actually produces
    a different module, there is no good workaround.

    Proposal:  Add a __module__ keyword which refers to the module
    currently being defined (executed).  (But see open issues.)

        if __module__ is sys.main: ...   # assuming PEP 3020, Cannon


    Class methods are passed the current instance; from this they can
    determine self.__class__ (or cls, for classmethods).  Unfortunately,
    this reference is to the object's actual class, which may be a
    subclass of the defining class.  The current workaround is to repeat
    the name of the class, and assume that the name will not be rebound.

        class C(B):
            def meth(self):
                super(C, self).meth() # Hope C is never rebound.

        class D(C):
            def meth(self):
                super(C, self).meth() # ?!? issubclass(D,C), so it "works"

    Proposal:  Add a __class__ keyword which refers to the class currently
    being defined (executed).  (But see open issues.)

        class C(B):
            def meth(self):
                super(__class__, self).meth()

    Note that super calls may be further simplified by PEP 30XX, Jewett.
    The __class__ (or __this_class__) attribute came up in attempts to
    simplify the explanation and/or implementation of that PEP, but was
    separated out as an independent decision.

    Note that __class__ (or __this_class__) is not quite the same as the
    __thisclass__ property on bound super objects.  The existing
    super.__thisclass__ property refers to the class from which the Method
    Resolution Order search begins.  In the above class D, it would refer to
    (the current reference of name) C.


    Functions (including methods) often want access to themselves,
    usually for a private storage location.  While there are several
    workarounds, all have their drawbacks.

        def counter(_total=[0]):   # _total shouldn't really appear in the
            _total[0] += 1         # signature at all; the list wrapping and
            return _total[0]       # [0] unwrapping obscure the code

        @annotate(total=0)
        def counter():
            counter.total += 1    # Assume name counter is never rebound
            return counter.total

        class _wrap(object):  # class exists only to provide storage
            __total=0
            def f(self):
                self.__total += 1
                return self.__total
        accum=_wrap().f       # set module attribute to a bound method

    Proposal:  Add a __function__ keyword which refers to the function
    (or method) currently being defined (executed).  (But see open issues.)

        @annotate(total=0)
        def counter():
            __function__.total += 1    # Always refers to this function obj
            return __function__.total


Backwards Compatibility

    While a user could be using these names already, __anything__ names
    are explicitly reserved to the interpreter.  It is therefore acceptable
    to introduce special meaning to these names within a single feature
    release.


Implementation

    Ideally, these names would be keywords treated specially by the bytecode
    compiler.

    Guido has suggested [1] using a cell variable filled in by the metaclass.

    Michele Simionato has provided a prototype using bytecode hacks [2].


Open Issues

    - Are __module__, __class__, and __function__ the right names?
      In particular, should the names include the word "this", either as
      __this_module__, __this_class__, and __this_function__, (format
      discussed on the python-3000 and python-ideas lists) or as
      __thismodule__, __thisclass__, and __thisfunction__ (inspired by,
      but conflicting with, current usage of super.__thisclass__).

    - Are all three keywords needed, or should this enhancement be limited
      to a subset of the objects?  Should methods be treated separately from
      other functions?


References

    [1] Fixing super anyone?  Guido van Rossum
        http://mail.python.org/pipermail/python-3000/2007-April/006671.html

    [2] Descriptor/Decorator challenge,  Michele Simionato
        http://groups.google.com/group/comp.lang.python/browse_frm/thread/a6010c7494871bb1/62a2da68961caeb6?lnk=gst&q=simionato+challenge&rnum=1&hl=en#62a2da68961caeb6


Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:

From nnorwitz at gmail.com  Mon Apr 23 05:12:40 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 22 Apr 2007 20:12:40 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
Message-ID: <ee2a432c0704222012w4464e52epee8c09f876430726@mail.gmail.com>

Can you add refs to all the PEPs?

Will this go into 2.6 also?  If so, you should send to python-dev too.
If this is accepted, can you update the 2.6 PEP 361.

On 4/22/07, Brett Cannon <brett at python.org> wrote:
>
> Implementation
> ==============
>
> When the ``-m`` option is used, ``sys.main`` will be set to the
> argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> Then the equivalent of ``__import__(self.main)`` will occur.  This
> differs from current semantics as the ``runpy`` module fetches the
> code object for the file specified by the module name in order to
> explicitly set ``__name__`` and other attributes.  This is no longer
> needed as import can perform its normal operation in this situation.
>
> If a file name is specified, then ``sys.main`` will be set to
> ``"__main__"``.  The specified file will then be read and have a code
> object created and then be executed with ``__name__`` set to
> ``"__main__"``.  This mirrors current semantics.

What happens when no file is passed in (ie, interactive) or when -c is used?
Will sys.main be set?  If so, to what?

n

From steven.bethard at gmail.com  Mon Apr 23 05:34:54 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 22 Apr 2007 21:34:54 -0600
Subject: [Python-3000] [Python-Dev] PEP 30xx: Access to
	Module/Class/Function Currently Being Defined (this)
In-Reply-To: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
Message-ID: <d11dcfba0704222034t284e260byb2e48222c2622158@mail.gmail.com>

On 4/22/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> PEP: 30XX
> Title: Access to Module/Class/Function Currently Being Defined (this)
[snip]
>
> Abstract
>
>     It is common to need a reference to the current module, class,
>     or function, but there is currently no entirely correct way to
>     do this.  This PEP proposes adding the keywords __module__,
>     __class__, and __function__.

+1 on the general idea, if it can be implemented. ;-)

One thing that isn't clear...  You're saying "keywords". So that means
"__module__" would not be a normal attribute looked up on in the
globals or builtins, but instead would be associated with its own
op-code?

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Mon Apr 23 05:36:04 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Apr 2007 20:36:04 -0700
Subject: [Python-3000] question Re: Need help completing ABC pep
In-Reply-To: <462B3A83.1060204@openendsystems.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<46291596.1020105@acm.org>
	<5.1.1.6.0.20070421105729.04c77840@sparrow.telecommunity.com>
	<ca471dc20704210806u1c92be42y53e8949716dc7982@mail.gmail.com>
	<462B3A83.1060204@openendsystems.com>
Message-ID: <ca471dc20704222036i44237fb5rd7da211bb7be6f46@mail.gmail.com>

On 4/22/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
> If we keep s % d (do we? reading the list archives and the peps didn't
> give an answer, I suppose it depends if PEP 3101 is accepted):
>
> will it check if d is a Mapping or continue to use the current heuristics?
>
> I suppose this is a legitimate question for other corners of the
> language, std lib.

Any place that currently uses a heuristic should be changed to check
for Mapping instead (actually, in this particular case BasicMapping,
as the % operator doesn't need to iterate over the keys or items). And
most places that currently check for a dict should probably check for
a Mapping or BasicMapping as well. BUT BUT BUT -- this is only for
places that already check (like s%d). Places that just plunge in and
call __getitem__ should continue to do so and should not be modified
to require a flavor Mapping. Similarly, assignment to sys.stdout
shouldn't be constrained to subclasses of io.IOBase; we can and should
continue to use duck typing where it makes sense.

PS. I fully intend to be dropping s%d in favor of the PEP 3101 solution.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Mon Apr 23 05:52:03 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Apr 2007 20:52:03 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <f0h54s$8s1$1@sea.gmane.org>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<f0h54s$8s1$1@sea.gmane.org>
Message-ID: <bbaeab100704222052y1111376fwa333e371877a504b@mail.gmail.com>

On 4/22/07, Christian Heimes <lists at cheimes.de> wrote:
> Brett Cannon wrote:
> > Implementation
> > ==============
> >
> > When the ``-m`` option is used, ``sys.main`` will be set to the
> > argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> > Then the equivalent of ``__import__(self.main)`` will occur.  This
> > differs from current semantics as the ``runpy`` module fetches the
> > code object for the file specified by the module name in order to
> > explicitly set ``__name__`` and other attributes.  This is no longer
> > needed as import can perform its normal operation in this situation.
>
> __import__(self.main)? I guess you wanted to write __import__(sys.main) :)
>

Yep, that's what I meant.

>  > The newly proposed solution does introduce an added line of
>  > boilerplate which is a module import.  But as the solution does not
>  > introduce a new built-in or module attribute (as discussed in
>  > `Rejected Ideas`_) it has been deemed worth the extra line.
>
> Someone on python-idea brought up the idea to add the sys module to
> builtins. The feature would remove the requirement for an extra line and
> the costs should be minimal or even zero since sys is a special builtin
> module.
>

That's a separate PEP.

> Some people including me like to resurrect the PEP 299 __main__()
> function idea. http://www.python.org/dev/peps/pep-0299/

As I have already said multiple times on python-ideas, I don't
personally like that idea.  If someone wants to do the work to try to
push that view then they can.

-Brett

From brett at python.org  Mon Apr 23 05:56:54 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Apr 2007 20:56:54 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <ee2a432c0704222012w4464e52epee8c09f876430726@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<ee2a432c0704222012w4464e52epee8c09f876430726@mail.gmail.com>
Message-ID: <bbaeab100704222056t7380dd8dw5364678fa38a5b8f@mail.gmail.com>

On 4/22/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> Can you add refs to all the PEPs?
>

Sure, although considering my laptop died within minutes of my last
commit to 25-maint it might be a little while (using my girlfriend's
laptop to check email).

> Will this go into 2.6 also?

It's covered in the transition plan.  Basically sys.main just becomes
equal to "__main__" and nothing else changes.  Figured didn't have to
involve python-dev with such a minor change.  Plus Guido has to go for
this first.  =)

>  If so, you should send to python-dev too.
> If this is accepted, can you update the 2.6 PEP 361.
>

If I remember, sure.  =)

> On 4/22/07, Brett Cannon <brett at python.org> wrote:
> >
> > Implementation
> > ==============
> >
> > When the ``-m`` option is used, ``sys.main`` will be set to the
> > argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> > Then the equivalent of ``__import__(self.main)`` will occur.  This
> > differs from current semantics as the ``runpy`` module fetches the
> > code object for the file specified by the module name in order to
> > explicitly set ``__name__`` and other attributes.  This is no longer
> > needed as import can perform its normal operation in this situation.
> >
> > If a file name is specified, then ``sys.main`` will be set to
> > ``"__main__"``.  The specified file will then be read and have a code
> > object created and then be executed with ``__name__`` set to
> > ``"__main__"``.  This mirrors current semantics.
>
> What happens when no file is passed in (ie, interactive) or when -c is used?
> Will sys.main be set?  If so, to what?

Good question.  I would think None would be a good value.  Otherwise
the empty string if people don't want to have a possibly different
type other than str in the attribute.

-Brett

From pje at telecommunity.com  Mon Apr 23 06:09:57 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 00:09:57 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
Message-ID: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>

I got off to a good start in terms of figuring out what needs to be in the 
PEP...  good enough to realize that a complete specification of what I have 
in mind is probably going to dwarf the WSGI spec.  There are just too many 
details that have to be explained; the roughly 1000 lines of Python code I 
have in mind is translating to quite a *lot* more words of English, to 
explain intentions, ramifications, etc.

In particular, when you implement generic functions using generic 
functions, it reads a lot like trying to explain the relationships between 
"type" and "object" (i.e. type being its own type and an instance of 
object, and object being a subclass of type).  You can say what needs to be 
said in a fairly short space, but saying it in a way that lets people 
actually wrap their heads around it takes a bit more text.  :)

So, there is no way I could get it done in time for anybody to *read* it 
before the deadline, let alone discuss it (and there are at least a few 
things about it I think would really *need* some discussion before 
finalizing the proposal).

On the plus side, it was interesting to think through how I'd take the 
current peak.rules.core and adapt it to Python 3.0, with argument 
annotations and no classic classes to worry about.  It was also interesting 
to look at how certain things might overlap with other Py3K PEP's; for 
example, the peak.rules.core API has an '@abstract' decorator that's used 
in basically the same way as '@abstractmethod'...  except that you can add 
overloads to an @abstract function.  (Functions not declared @abstract are 
considered to be a 1-method generic function whose default rule is the 
function's original body code.)

Anyway, I'm not abandoning the idea, but it's beginning to seem to me that 
it might be a lot easier to close the open issues by actually working on an 
implementation instead of first trying to write an adequate explanation of 
what the issues are.

However, I was *definitely* being delusional about how much time it would 
take to write this up -- probably because how to make systems like this is 
now so deeply embedded in my subconscious that I don't think about it much 
any more.  Like "type" and "object", it's all very simple...  until you try 
to explain it.  :)

(It occurs to me that I should mention that, just like the relationship 
between 'type' and 'object' rarely matters unless you delve into 
metaclasses or other wizardry, so too am I referring here only to the 
wizard-level aspects of a generic function module, where my design allows 
for people to write their own dispatchers or interface implementations and 
plug them into the base module, using the same decorators and basic 
machinery.  Merely *using* generic functions and interfaces would of course 
be at least as simple as interfaces and static overloads in Java or C++ -- 
that's *not* the part of the PEP that's hard to write.)


From tjreedy at udel.edu  Mon Apr 23 06:17:23 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 23 Apr 2007 00:17:23 -0400
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	CurrentlyBeing Defined (this)
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
Message-ID: <f0hc0j$2pg$1@sea.gmane.org>


"Jim Jewett" <jimjjewett at gmail.com> wrote in 
message news:fb6fbf560704222005le4798a4j5daa5e71e644f069 at mail.gmail.com...
|    Functions (including methods) often want access to themselves,
|    usually for a private storage location.

Or for true recursion.  At present, a function has to 'hope' that its 
definition name is not rebound.

tjr




From steven.bethard at gmail.com  Mon Apr 23 06:43:21 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 22 Apr 2007 22:43:21 -0600
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
Message-ID: <d11dcfba0704222143k66f1b7bye9a52e452098dc8e@mail.gmail.com>

On 4/22/07, Brett Cannon <brett at python.org> wrote:
> Implementation
> ==============
>
> When the ``-m`` option is used, ``sys.main`` will be set to the
> argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> Then the equivalent of ``__import__(self.main)`` will occur.  This
> differs from current semantics as the ``runpy`` module fetches the
> code object for the file specified by the module name in order to
> explicitly set ``__name__`` and other attributes.  This is no longer
> needed as import can perform its normal operation in this situation.
>
> If a file name is specified, then ``sys.main`` will be set to
> ``"__main__"``.  The specified file will then be read and have a code
> object created and then be executed with ``__name__`` set to
> ``"__main__"``.  This mirrors current semantics.

So __name__ will still sometimes be "__main__". That's disappointing.

To clarify, assuming that the foo.bar module contains something like
"from . import baz", this PEP only addresses the following situation::

    python -mfoo.bar

and all of the following will still raise ImportErrors::

    ~> python foo/bar
    ~/grok> python ../foo/bar
    ~/foo> python bar

Right?

If that's right, I'm -1 on the proposal. It's complicating the
standard "am I the main module?" idiom to solve a tiny little problem
with "-m". The "am I the main module?" idiom has been around much
longer than the "-m" flag, and I'd prefer to see a more compelling
reason to change it.

Two reasons that would be compelling for me:

* Simplifying the "am I the main module?" idiom, e.g. with the
rejected ``if __main__`` proposal.

* Getting rid of the "__main__" value for __name__ entirely. This
would require code like the following to determine the name of a
module given at the command line::

    def get_name(path):
        sys_path_set = set(sys.path)
        path, _ = os.path.splitext(path)
        path = os.path.abspath(path)
        parts = path.split(os.path.sep)
        for i in range(1, len(parts)):
            sub_path = os.path.sep.join(parts[:i])
            if sub_path in sys_path_set:
                return '.'.join(parts[i:])
        return parts[-1]

  Even if we can't do the name resolution perfectly, if the flaws are
documented (e.g. when '..' and symbolic links are combined) I think
that would be enough better than the current situation to merit
changing the standard "am I the current module?" idiom.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From michele.cella at gmail.com  Mon Apr 23 08:48:28 2007
From: michele.cella at gmail.com (Michele Cella)
Date: Mon, 23 Apr 2007 08:48:28 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <462BDCAA.7020501@canterbury.ac.nz>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>	<f0d5qi$2fu$1@sea.gmane.org>
	<462BDCAA.7020501@canterbury.ac.nz>
Message-ID: <f0hkrs$aro$1@sea.gmane.org>

Greg Ewing wrote:
> Michele Cella wrote:
> 
>> Personally, I really like this syntax:
>>
>> res = super currentmethod(arg, arg)
>>
>> Why? because it's very consistent (pythonic?) with the way I'm used to 
>> use any other keyword.
> 
> But it's inconsistent with the way every other
> method is called.
> 

Yeah, you're absolutely right. :-)

Have you seen my second proposal?

Calling a method:

	self.method(arg)

Calling a super method:

	super self.method(arg)

That's consistent with:
	* the way you call any other method
	* the way you use any other keyword

Thanks for the attention.

Ciao
Michele


From ncoghlan at gmail.com  Mon Apr 23 11:31:39 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 23 Apr 2007 19:31:39 +1000
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
Message-ID: <462C7CFB.5060607@gmail.com>

Brett Cannon wrote:
> This PEP is to change the ``if __name__ == "__main__": ...`` idiom to
> ``if __name__ == sys.main: ...``  so that you at least have a chance
> to execute module in a package that use relative imports.
> 
> Ran this PEP past python-ideas.  Stopped the discussion there when too
> many new ideas were being proposed.  =)  I have listed all of them in
> the Rejected Ideas section, although if overwhelming support for one
> comes forward the PEP can shift to one of them.

-1

I still prefer the "__module_name__" fallback that was actually briefly 
in the tree for 2.5 (the change was made really late in the 2.5 cycle, 
so it was backed out with the intention of revisiting the idea for 2.6 - 
check the svn log for lib/runpy.py).

The trick with that approach was that runpy.run_module() would *always* 
set "__module_name__" to the real name of the module being executed, 
regardless of what it was told to set "__name__" to.

Then the import machinery was modified such that if it came across an 
explicit relative import being made from a module called "__main__", it 
would check to see if "__module_name__" was set, and if so, use that 
instead.

The issue discussed in PEP 338 (and this new PEP) gets fixed, with zero 
impact on anything else (even normal import speed is unaffected, as the 
check for "__module_name__" would only occur on an error handling path 
that currently raises an exception).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From thomas at python.org  Mon Apr 23 15:07:05 2007
From: thomas at python.org (Thomas Wouters)
Date: Mon, 23 Apr 2007 15:07:05 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <f0hkrs$aro$1@sea.gmane.org>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
Message-ID: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>

On 4/23/07, Michele Cella <michele.cella at gmail.com> wrote:

> Calling a method:
>
>         self.method(arg)
>
> Calling a super method:
>
>         super self.method(arg)
>
> That's consistent with:
>         * the way you call any other method
>         * the way you use any other keyword


But how would it *work*? I assume you want 'super' to take a single
expression, and be an expression itself. How does it know which instance you
want to 'super'? Does it specialcase the variable named 'self'? Threat
'everything up to the first dot' as 'the instance'? Specialcase the first
argument to the method? (That's what 'super.foo()' would do.) How would you
do things like:

  # call the supermethod 'register' with the nonsuper 'callback':
  super.register(self.callback)
  # call the nonsuper 'register' with the supermethod 'callback'
  self.register(super.callback)
  # call the 'frominstance' method of the 'innerclass' attribute on the
superclass
  super.innerclass.frominstance(self)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070423/e1da0542/attachment.html 

From lists at cheimes.de  Mon Apr 23 15:07:33 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 23 Apr 2007 15:07:33 +0200
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704222052y1111376fwa333e371877a504b@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>	<f0h54s$8s1$1@sea.gmane.org>
	<bbaeab100704222052y1111376fwa333e371877a504b@mail.gmail.com>
Message-ID: <f0ib2l$tpc$1@sea.gmane.org>

Brett Cannon wrote:
>> Someone on python-idea brought up the idea to add the sys module to
>> builtins. The feature would remove the requirement for an extra line and
>> the costs should be minimal or even zero since sys is a special builtin
>> module.
>>
> 
> That's a separate PEP.

Do you remember the PEP number? I can't find the PEP.

>> Some people including me like to resurrect the PEP 299 __main__()
>> function idea. http://www.python.org/dev/peps/pep-0299/
> 
> As I have already said multiple times on python-ideas, I don't
> personally like that idea.  If someone wants to do the work to try to
> push that view then they can.

I know. I mentioned it because I "wanted to push that view". O:-)

Christian


From lists at cheimes.de  Mon Apr 23 15:21:01 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 23 Apr 2007 15:21:01 +0200
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
 Currently Being Defined (this)
In-Reply-To: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
Message-ID: <f0ibru$35c$1@sea.gmane.org>

>     Proposal:  Add a __module__ keyword which refers to the module
>     currently being defined (executed).  (But see open issues.)
> 
>         if __module__ is sys.main: ...   # assuming PEP 3020, Cannon
>

-1 on __module__

I understand PEP 3020 correctly then sys.main will contain the dotted 
name of the module as *string*.

Also __module__ is already used in objects like classes.

 >>> class C: pass
...
 >>> C.__module__
'__main__'
 >>> type(C.__module__)
<type 'str'>

I don't like the fact that __module__ is going to be a string in some 
cases and a module object in other cases. It's too confusing.

class C:
     def egg(self):
         # string from class C or an object from module?
         # confused ...
         nonlocal __module__

It might work with __this_module__ but I don't see the point in getting 
the local module. I'v used it about 5 times in my many years as Python 
developer when I had to deal with a legacy product. The module and 
package names were badly chosen (Foo/Foo.py: import Foo).

Christian


From barry at python.org  Mon Apr 23 15:36:20 2007
From: barry at python.org (Barry Warsaw)
Date: Mon, 23 Apr 2007 09:36:20 -0400
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	Currently Being Defined (this)
In-Reply-To: <f0ibru$35c$1@sea.gmane.org>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<f0ibru$35c$1@sea.gmane.org>
Message-ID: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote:

> -1 on __module__
>
> I understand PEP 3020 correctly then sys.main will contain the dotted
> name of the module as *string*.
>
> Also __module__ is already used in objects like classes.

I'm not sure whether a general mechanism is necessary or not, but  
I've always wished I could just do something like "import __me__" to  
get the current module.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRiy2VHEjvBPtnXfVAQKgdwQAmuAVdKvz+rZdOPHHSyk03bOM6wkUQ9FU
+qhhThUPl9yZIexJP5DvE/fEo6b3hsU87kkSBmTVynCCdVzzhwYDqILdGvCaJ8wP
L+K3oD9mq69BJ8bVi1mGu8st9XmK/jwWuq6d6Fo3pMKGehuk38EtW3N6dzHHFG0g
oECLki6Fc1I=
=yw83
-----END PGP SIGNATURE-----

From steven.bethard at gmail.com  Mon Apr 23 15:53:44 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 23 Apr 2007 07:53:44 -0600
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	Currently Being Defined (this)
In-Reply-To: <88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<f0ibru$35c$1@sea.gmane.org>
	<88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org>
Message-ID: <d11dcfba0704230653o2bfc7f2fs8698d9d44dfb5927@mail.gmail.com>

On 4/23/07, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote:
>
> > -1 on __module__
> >
> > I understand PEP 3020 correctly then sys.main will contain the dotted
> > name of the module as *string*.
> >
> > Also __module__ is already used in objects like classes.
>
> I'm not sure whether a general mechanism is necessary or not, but
> I've always wished I could just do something like "import __me__" to
> get the current module.

Well, you can always do::

    __import__(__name__)

That's not much more verbose.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From steven.bethard at gmail.com  Mon Apr 23 16:19:12 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 23 Apr 2007 08:19:12 -0600
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	Currently Being Defined (this)
In-Reply-To: <d11dcfba0704230653o2bfc7f2fs8698d9d44dfb5927@mail.gmail.com>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<f0ibru$35c$1@sea.gmane.org>
	<88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org>
	<d11dcfba0704230653o2bfc7f2fs8698d9d44dfb5927@mail.gmail.com>
Message-ID: <d11dcfba0704230719i157f2582lb032780093483a37@mail.gmail.com>

On 4/23/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/23/07, Barry Warsaw <barry at python.org> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote:
> >
> > > -1 on __module__
> > >
> > > I understand PEP 3020 correctly then sys.main will contain the dotted
> > > name of the module as *string*.
> > >
> > > Also __module__ is already used in objects like classes.
> >
> > I'm not sure whether a general mechanism is necessary or not, but
> > I've always wished I could just do something like "import __me__" to
> > get the current module.
>
> Well, you can always do::
>
>     __import__(__name__)
>
> That's not much more verbose.

Of course, to make it work with packages, you have to write something like::

    mod = __import__(__name__)
    for sub_mod_name in __name__.split('.')[1:]:
        mod = getattr(mod, sub_mod_name)

I guess that's not so simple after all.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From lists at cheimes.de  Mon Apr 23 16:41:42 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 23 Apr 2007 16:41:42 +0200
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
 Currently Being Defined (this)
In-Reply-To: <d11dcfba0704230719i157f2582lb032780093483a37@mail.gmail.com>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>	<f0ibru$35c$1@sea.gmane.org>	<88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org>	<d11dcfba0704230653o2bfc7f2fs8698d9d44dfb5927@mail.gmail.com>
	<d11dcfba0704230719i157f2582lb032780093483a37@mail.gmail.com>
Message-ID: <f0igj7$tpc$2@sea.gmane.org>

Steven Bethard wrote:
> Of course, to make it work with packages, you have to write something like::
> 
>     mod = __import__(__name__)
>     for sub_mod_name in __name__.split('.')[1:]:
>         mod = getattr(mod, sub_mod_name)
> 
> I guess that's not so simple after all.
> 
> STeVe

Do you want to do something like:

import sys
mod = sys.modules[__name__]
mod = sys.modules[self.__class__.__module__]

?

Christian


From exarkun at divmod.com  Mon Apr 23 16:53:32 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 23 Apr 2007 10:53:32 -0400
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
 Currently Being Defined (this)
In-Reply-To: <d11dcfba0704230719i157f2582lb032780093483a37@mail.gmail.com>
Message-ID: <20070423145332.19381.2022886003.divmod.quotient.4334@ohm>

On Mon, 23 Apr 2007 08:19:12 -0600, Steven Bethard <steven.bethard at gmail.com> wrote:
>On 4/23/07, Steven Bethard <steven.bethard at gmail.com> wrote:
>> On 4/23/07, Barry Warsaw <barry at python.org> wrote:
>> > -----BEGIN PGP SIGNED MESSAGE-----
>> > Hash: SHA1
>> >
>> > On Apr 23, 2007, at 9:21 AM, Christian Heimes wrote:
>> >
>> > > -1 on __module__
>> > >
>> > > I understand PEP 3020 correctly then sys.main will contain the dotted
>> > > name of the module as *string*.
>> > >
>> > > Also __module__ is already used in objects like classes.
>> >
>> > I'm not sure whether a general mechanism is necessary or not, but
>> > I've always wished I could just do something like "import __me__" to
>> > get the current module.
>>
>> Well, you can always do::
>>
>>     __import__(__name__)
>>
>> That's not much more verbose.
>
>Of course, to make it work with packages, you have to write something like::
>
>    mod = __import__(__name__)
>    for sub_mod_name in __name__.split('.')[1:]:
>        mod = getattr(mod, sub_mod_name)
>
>I guess that's not so simple after all.

Maybe this indicates a problem with __import__.  Twisted has something
called namedAny, and I always use that, because it just works, unlike
__import__, which seems to take too many arguments and has weird behavior
for dotted names.

   me = namedAny(__name__)

The inverse is also available (qual(me) == __name__), and is another
function I've always thought it strange was missing from the stdlib.

Jean-Paul

From guido at python.org  Mon Apr 23 16:57:04 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 07:57:04 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <ca471dc20704230757s3aff408fu8018d762e18d3345@mail.gmail.com>

That's okay. We won't be able to consider GFs for 3.0a1, but the whole
library reorg thing will be done later, so I expect adding GFs later
(to the library) will also be okay. I do hope that you'll stay with us
while I barrel forward with the ABC PEP. I see no serious
incompatibilities between GFs and ABCs, so I don't believe I should
hold it up just because your proposal is late; but there is time to
tweak the details to avoid gratuitous friction between the two
mechanisms.

--Guido

On 4/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> I got off to a good start in terms of figuring out what needs to be in the
> PEP...  good enough to realize that a complete specification of what I have
> in mind is probably going to dwarf the WSGI spec.  There are just too many
> details that have to be explained; the roughly 1000 lines of Python code I
> have in mind is translating to quite a *lot* more words of English, to
> explain intentions, ramifications, etc.
>
> In particular, when you implement generic functions using generic
> functions, it reads a lot like trying to explain the relationships between
> "type" and "object" (i.e. type being its own type and an instance of
> object, and object being a subclass of type).  You can say what needs to be
> said in a fairly short space, but saying it in a way that lets people
> actually wrap their heads around it takes a bit more text.  :)
>
> So, there is no way I could get it done in time for anybody to *read* it
> before the deadline, let alone discuss it (and there are at least a few
> things about it I think would really *need* some discussion before
> finalizing the proposal).
>
> On the plus side, it was interesting to think through how I'd take the
> current peak.rules.core and adapt it to Python 3.0, with argument
> annotations and no classic classes to worry about.  It was also interesting
> to look at how certain things might overlap with other Py3K PEP's; for
> example, the peak.rules.core API has an '@abstract' decorator that's used
> in basically the same way as '@abstractmethod'...  except that you can add
> overloads to an @abstract function.  (Functions not declared @abstract are
> considered to be a 1-method generic function whose default rule is the
> function's original body code.)
>
> Anyway, I'm not abandoning the idea, but it's beginning to seem to me that
> it might be a lot easier to close the open issues by actually working on an
> implementation instead of first trying to write an adequate explanation of
> what the issues are.
>
> However, I was *definitely* being delusional about how much time it would
> take to write this up -- probably because how to make systems like this is
> now so deeply embedded in my subconscious that I don't think about it much
> any more.  Like "type" and "object", it's all very simple...  until you try
> to explain it.  :)
>
> (It occurs to me that I should mention that, just like the relationship
> between 'type' and 'object' rarely matters unless you delve into
> metaclasses or other wizardry, so too am I referring here only to the
> wizard-level aspects of a generic function module, where my design allows
> for people to write their own dispatchers or interface implementations and
> plug them into the base module, using the same decorators and basic
> machinery.  Merely *using* generic functions and interfaces would of course
> be at least as simple as interfaces and static overloads in Java or C++ --
> that's *not* the part of the PEP that's hard to write.)
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Mon Apr 23 17:26:23 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 11:26:23 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704230757s3aff408fu8018d762e18d3345@mail.gmail.co
 m>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423111608.00a04748@sparrow.telecommunity.com>

At 07:57 AM 4/23/2007 -0700, Guido van Rossum wrote:
>That's okay. We won't be able to consider GFs for 3.0a1, but the whole
>library reorg thing will be done later, so I expect adding GFs later
>(to the library) will also be okay. I do hope that you'll stay with us
>while I barrel forward with the ABC PEP. I see no serious
>incompatibilities between GFs and ABCs, so I don't believe I should
>hold it up just because your proposal is late; but there is time to
>tweak the details to avoid gratuitous friction between the two
>mechanisms.

Yeah, I don't see any issues there at all; the only places I can think of 
right now where there's even any overlap with non-library stuff is GF 
@abstract with ABC @abstractmethod, and the possibility of having the 
default metaclass (i.e., 'type') support an "implements" keyword 
argument.  And all of the stuff I'm proposing (except for argument 
annotation support) has been implemented before in either PEAK-Rules or 
PyProtocols, so it's mostly a matter of things like getting rid of bytecode 
hacks, and making sure the spec can be implemented in other Python 
interpreters.  (PyPy should have no problems, but I don't know enough about 
Jython or IronPython to be able to say the same for them.)


From p.f.moore at gmail.com  Mon Apr 23 17:35:22 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 23 Apr 2007 16:35:22 +0100
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <79990c6b0704230835h715f799au32ee4ceb73637784@mail.gmail.com>

On 23/04/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> (It occurs to me that I should mention that, just like the relationship
> between 'type' and 'object' rarely matters unless you delve into
> metaclasses or other wizardry, so too am I referring here only to the
> wizard-level aspects of a generic function module, where my design allows
> for people to write their own dispatchers or interface implementations and
> plug them into the base module, using the same decorators and basic
> machinery.  Merely *using* generic functions and interfaces would of course
> be at least as simple as interfaces and static overloads in Java or C++ --
> that's *not* the part of the PEP that's hard to write.)

I wonder whether starting off with a PEP that omits or skims over the
wizard-level stuff would not actually be more use. If you already have
most of that PEP, why not publish it, with a section called something
like "Advanced Use" which for now is simply a "to be completed"
placeholder.

If the basic stuff really is that simple, I'd love to see just that
for now. It would put the advanced stuff into context when it finally
arrived.

Paul

From jimjjewett at gmail.com  Mon Apr 23 17:51:37 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 11:51:37 -0400
Subject: [Python-3000] [Python-Dev] PEP 30xx: Access to
	Module/Class/Function Currently Being Defined (this)
In-Reply-To: <d11dcfba0704222034t284e260byb2e48222c2622158@mail.gmail.com>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<d11dcfba0704222034t284e260byb2e48222c2622158@mail.gmail.com>
Message-ID: <fb6fbf560704230851s74733be3p6a4e36982de5c45@mail.gmail.com>

On 4/22/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/22/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > PEP: 30XX
> > Title: Access to Module/Class/Function Currently Being Defined (this)

> One thing that isn't clear...  You're saying "keywords". So that means
> "__module__" would not be a normal attribute looked up on in the
> globals or builtins, but instead would be associated with its own
> op-code?

My assumption would be something closer to Michele's solution -- the
parser would create a new type of AST node, and the compiler would
emit slightly different bytecode, but no new op-codes would be needed.

-jJ

From daniel at stutzbachenterprises.com  Mon Apr 23 17:56:54 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 10:56:54 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one modest,
	one radical
Message-ID: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>

I have two proposals for Python 3000 that I'll write PEP(s) for if
there's interest.  The first proposal is, I think, a modest one.  The
second proposal is related, but more radical.  I fully expect the
second proposal to be rejected for that alone, especially since I am a
relatively an outsider to the Python developer community (though it
was great to meet some of you at PyCon this year).  I bring up the
more radical proposal primarily for completeness.

The Modest Proposal
-------------------------------

As a pet project, I've written a container type that looks, acts, and
quacks like a Python list(), but has better asymptotic performance.
Specifically, O(log n) time for inserts and deletes in the middle of
the list.  I'd like to offer it for inclusion in the collections
module.

Probably, you have encountered some other proposals for types that
meet this description.  Here are some of the unique features of mine,
called a BList, which is based on B+Trees.  I'm open to suggestions
for a better name.

- just as fast as a Python list() when the list is small
- getslice runs in O(log n) time
- making a shallow copy runs in O(1) time
- setslice runs in O(log n + log k) time if the inserted slice is a
BList of length k
- multipling a BList by k takes O(log k) time and O(log k) memory

Example:

>>> from blist import *
>>> x = BList([0])
>>> x *= 2**29
>>> x.append(5)
>>> y = x[4:-234234]
>>> del x[3:1024]

None of the above operations have a noticeable delay, even though the
lists have over 500 million elements due to line 3.

The BList has two key features that allow it to pull this off this performance:

1) The BList type features transparent copy-on-write.  If a non-root
node needs to be copied (as part of a getslice, copy, setslice, etc.),
the node is shared between multiple parents instead of being copied.
If it needs to be modified later, it will be copied at that time.
This is completely behind-the-scenes; from the user's point of view,
the BList works just like a regular Python list.

2) A B+Tree is a wide, squat tree.  The maximum number of children per
node is defined by a compile-time parameter (LIMIT).  If the list is
shorter than LIMIT, then there is only one node, which simply contains
an array of the objects in the list.  In other words, for short lists,
a BList works just like Python's array-based list() type.  Thus, it
has the same good performance on small lists.

So you can see the performance of the BList in more detail, I've made
several performance graphs available at the following link:
   http://stutzbachenterprises.com/blist/

Each webpage shows two graphs:
 - The raw execution time for the operation as a function of the list size
 - The execution time for the operation, normalized by the execution
time of Python's list()

These graphs were made using LIMIT=128.

The only operation where list() is much faster for small lists is
getitem through the x[5] syntax.  There's a special case for that
operation deep inside the Python bytecode interpreter.  My extension
module can't compete with that. :-)  BList is just as fast as list()
for small lists via .__getitem__().

The source code for the type is available for compilation as an external module:
    http://stutzbachenterprises.com/blist/blist.tar.gz

I wrote it against Python 2.5.

The Radical Proposal
-------------------------------

Replace list() with the BList.

There are only a few use-cases (that I can think of) where Python's
list() regularly outperforms the BList.  These are:

1. A large LIFO stack, where there are many .append() and .pop(-1)
operations.  These are O(1) for a Python list, but O(log n) for the
BList().  (BList's are just as fast for small lists)

2. An large, unchanging list, where there are many getitem() calls,
and none of the inserts or deletes where the BList() shines.  Again,
these are O(1) for a Python list, but O(log n) for the BList(), and
the BList() is just as fast if the list is small.

Use case #1 is covered by collections.deque.  Use case #2 is covered by tuples.

When I first started learning Python, I often ended up inadvertently
writing O(n**2) algorithms by putting a O(n) list operation inside a
loop (which is a problem when n=100,000).  Since nothing in the
signature of list() informs the user when an operation is going to be
slow, it took me a while to learn how to write my code to work around
this constraint.  In fact, I initially assumed "list" meant "linked
list" so most of my assumptions about which operations would be fast
vs. slow were backwards.  I think it would be more Pythonic if the
user can trust that the implementation will always be fast (at least
O(log n)).

I recognize that the Python developer community is understandably very
attached to the current array-based list, so I expect this to get shot
down.  I hope this doesn't reflect badly on the more modest proposal
of including a new type in the collections module.  Also, please don't
kill the messenger. :-)

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From pje at telecommunity.com  Mon Apr 23 18:09:37 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 12:09:37 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <79990c6b0704230835h715f799au32ee4ceb73637784@mail.gmail.co
 m>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>

At 04:35 PM 4/23/2007 +0100, Paul Moore wrote:
>If the basic stuff really is that simple, I'd love to see just that
>for now. It would put the advanced stuff into context when it finally
>arrived.

Here you go:


   from overloading import overload, abstract

   def foo(bar, baz):
       """Default implementation here"""


   @overload
   def foo(bar:int, baz:str):
       """Overloaded implementation for int/str here"""


   @abstract
   def spam(fizz):
       """This function has no default implementation, and raises
          a "no applicable methods" error if called..."""

   @overload
   def spam(fizz:list):
       """...unless an appropriate overload is added first"""


Any questions?  :)

Miscellaneous notes/points for discussion:

* Names/locations of errors to be raised when methods aren't found or are 
ambiguous (e.g. should they be (or subclass) NotImplementedError?  TypeError?)

* Notice that it's not necessary to specially declare a function as being 
"generic"; you can therefore overload an existing function, classmethod, 
staticmethod, etc., and the existing function object is just modified in-place.

* CLOS/AspectJ-style method combination is supported, using @before, 
@after, and @around decorators.

That's pretty much it, for the generic function part.  The interface part 
looks like the "recombinable interfaces" one I previously posted, where you 
simply subclass Interface, and you don't have to write any adapter classes, 
because the interface is its own adapter class.  You just register methods 
for stuff.

I would also propose an 'implements' class keyword argument that asks the 
interface objects its been given in order to register the class.  In the 
case of overloading.Interface, that would be done by registering the class' 
methods of matching names with the generic functions embedded in the 
Interface, but other interface implementations might work differently.

There are perhaps a few more details or features visible at this user 
level, but all the
"interesting" stuff (i.e. wizardry and defense against the dark arts) takes 
place under the hood.  Specifically, how to make all this such that 
advanced or customized dispatch systems and interfaces can play in the same 
playground, being able to use the *same decorators* and "implements" 
keyword, so that systems like PEAK-Rules and zope.interface won't appear to 
be second-class citizens.


From pje at telecommunity.com  Mon Apr 23 18:11:30 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 12:11:30 -0400
Subject: [Python-3000] [Python-Dev] PEP 30xx: Access to
 Module/Class/Function Currently Being Defined (this)
In-Reply-To: <fb6fbf560704230851s74733be3p6a4e36982de5c45@mail.gmail.com
 >
References: <d11dcfba0704222034t284e260byb2e48222c2622158@mail.gmail.com>
	<fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<d11dcfba0704222034t284e260byb2e48222c2622158@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070423121019.04426ca8@sparrow.telecommunity.com>

At 11:51 AM 4/23/2007 -0400, Jim Jewett wrote:
>On 4/22/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > On 4/22/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > PEP: 30XX
> > > Title: Access to Module/Class/Function Currently Being Defined (this)
>
> > One thing that isn't clear...  You're saying "keywords". So that means
> > "__module__" would not be a normal attribute looked up on in the
> > globals or builtins, but instead would be associated with its own
> > op-code?
>
>My assumption would be something closer to Michele's solution -- the
>parser would create a new type of AST node, and the compiler would
>emit slightly different bytecode, but no new op-codes would be needed.

Why would a new type of AST node be required?  Wouldn't these just be a 
special symbol, like __debug__?  Or does __debug__ have its own AST node 
type now?


From janssen at parc.com  Mon Apr 23 18:31:36 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 23 Apr 2007 09:31:36 PDT
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com> 
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <07Apr23.093138pdt."57996"@synergy1.parc.xerox.com>

> Replace list() with the BList.

Generally, I like the idea.  But how about extension modules
which make use of the current implementation details?

Bill

From jarausch at igpm.rwth-aachen.de  Mon Apr 23 18:30:57 2007
From: jarausch at igpm.rwth-aachen.de (Helmut Jarausch)
Date: Mon, 23 Apr 2007 18:30:57 +0200 (CEST)
Subject: [Python-3000] a plea : invalid comparisons should raise an exception
Message-ID: <tkrat.4dd8c654819e14b7@igpm.rwth-aachen.de>

Some years ago, I was allured to convert from Perl to Python.
The main argument was that Python is much more secure.
That turned out to be true with one big exception.
The following lines would run just fine (as equivalent
Perl code)

from cStringIO import StringIO
Inp= StringIO('Guido 10')
for line in Inp:
    (Name,Age)= line.split()
    if Age > 64 :
        print Name,"has retired"

But with Python it returns a very unexpected result
without any evidence of a problem.

So, please make comparisons raise an exception (at least
optionally) if "uncompatible" types are compared.

-- 
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany


From jimjjewett at gmail.com  Mon Apr 23 18:46:59 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 12:46:59 -0400
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	Currently Being Defined (this)
In-Reply-To: <f0ibru$35c$1@sea.gmane.org>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<f0ibru$35c$1@sea.gmane.org>
Message-ID: <fb6fbf560704230946x1361ee0nc9299a37f263bace@mail.gmail.com>

On 4/23/07, Christian Heimes <lists at cheimes.de> wrote:
> >     Proposal:  Add a __module__ keyword which refers to the module
> >     currently being defined (executed).  (But see open issues.)

> -1 on __module__

> Also __module__ is already used in objects like classes.

Yes,  I worried that it might cause some confusion, but I also worry
that it might cause confusion *not* to reuse the name.

X.__module__ is the __module__ attribute of X
X.__class__ is the __class__ attribute of X

An (unqualified) __module__ would represent the current module.
An (unqualified) __class__ would represent the current class.

That said, the choice between reused short names (__module__) and
longer names (__this_module__, __current_module__, __thismodule__) is
indeed one of the open issues.  I'll count this as one vote against
the short forms.

-jJ

From p.f.moore at gmail.com  Mon Apr 23 18:50:06 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 23 Apr 2007 17:50:06 +0100
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
Message-ID: <79990c6b0704230950secc75aflb8fc92c7ea950877@mail.gmail.com>

On 23/04/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> Here you go:
[...]
> Any questions?  :)

No :-)

I really do think that putting this in a PEP as it is, would be a good start.

> That's pretty much it, for the generic function part.  The interface part
> looks like the "recombinable interfaces" one I previously posted, where you
> simply subclass Interface, and you don't have to write any adapter classes,
> because the interface is its own adapter class.  You just register methods
> for stuff.

Again, I'd argue for getting it in a PEP.

> There are perhaps a few more details or features visible at this user
> level, but all the "interesting" stuff (i.e. wizardry and defense against the dark
> arts) takes place under the hood.

I don't see the harm in putting that in later - particularly if it
means that the basic end user story is documented formally from the
start in a PEP. If nothing else, it makes it easier for you to remind
people that GFs are easy, without having to repeatedly post that
example... :-)

Paul.

From jimjjewett at gmail.com  Mon Apr 23 18:52:59 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 12:52:59 -0400
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	Currently Being Defined (this)
In-Reply-To: <f0igj7$tpc$2@sea.gmane.org>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<f0ibru$35c$1@sea.gmane.org>
	<88B9B3C2-D226-48C1-9C03-8028BC8390C6@python.org>
	<d11dcfba0704230653o2bfc7f2fs8698d9d44dfb5927@mail.gmail.com>
	<d11dcfba0704230719i157f2582lb032780093483a37@mail.gmail.com>
	<f0igj7$tpc$2@sea.gmane.org>
Message-ID: <fb6fbf560704230952n44e6445l9855754ad81435c8@mail.gmail.com>

On 4/23/07, Christian Heimes <lists at cheimes.de> wrote:
> Steven Bethard wrote:
> > Of course, to make it work with packages, you have to write something like::

> >     mod = __import__(__name__)
> >     for sub_mod_name in __name__.split('.')[1:]:
> >         mod = getattr(mod, sub_mod_name)

> > I guess that's not so simple after all.
> >
> > STeVe

> Do you want to do something like:

> import sys
> mod = sys.modules[__name__]
> mod = sys.modules[self.__class__.__module__]

Sure.  But I also want it to work, even if module.__name__ has been
changed, or module has been removed from sys.modules (perhaps for
security reasons), or some other module is already using that name.

That said, I won't be shocked if Guido eventually decides to support
only one or two of the magic words, just as he initially limited
decorators to functions.

-jJ

From nevillegrech at gmail.com  Mon Apr 23 18:58:25 2007
From: nevillegrech at gmail.com (Neville Grech Neville Grech)
Date: Mon, 23 Apr 2007 18:58:25 +0200
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <de9ae4950704230958u1cacd22bqfe231eaaee23dff2@mail.gmail.com>

I like the implementation of the BList, however I'm not so sure whether
replacing a list with BList is a good idea since there are many instances
where python scripts will iterate through endless lists of items. In such
case time complexity (to iterate a whole list) as you have mentioned is O
(nlog(n)) instead of O(n). You will probably find that such cases (iterating
through long lists) are very common and at the same time the developer still
wants his lists to be mutable. If a developer wants to optimise his lists
for insertion and deletion speed he can then use an optimised implementation
like the one you're proposing anyway.

What's the performance of concatenation in these BLists? O(n) ?

Are you suggesting BList as part of the standard library?

-Neville

On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
>
> I have two proposals for Python 3000 that I'll write PEP(s) for if
> there's interest.  The first proposal is, I think, a modest one.  The
> second proposal is related, but more radical.  I fully expect the
> second proposal to be rejected for that alone, especially since I am a
> relatively an outsider to the Python developer community (though it
> was great to meet some of you at PyCon this year).  I bring up the
> more radical proposal primarily for completeness.
>
> The Modest Proposal
> -------------------------------
>
> As a pet project, I've written a container type that looks, acts, and
> quacks like a Python list(), but has better asymptotic performance.
> Specifically, O(log n) time for inserts and deletes in the middle of
> the list.  I'd like to offer it for inclusion in the collections
> module.
>
> Probably, you have encountered some other proposals for types that
> meet this description.  Here are some of the unique features of mine,
> called a BList, which is based on B+Trees.  I'm open to suggestions
> for a better name.
>
> - just as fast as a Python list() when the list is small
> - getslice runs in O(log n) time
> - making a shallow copy runs in O(1) time
> - setslice runs in O(log n + log k) time if the inserted slice is a
> BList of length k
> - multipling a BList by k takes O(log k) time and O(log k) memory
>
> Example:
>
> >>> from blist import *
> >>> x = BList([0])
> >>> x *= 2**29
> >>> x.append(5)
> >>> y = x[4:-234234]
> >>> del x[3:1024]
>
> None of the above operations have a noticeable delay, even though the
> lists have over 500 million elements due to line 3.
>
> The BList has two key features that allow it to pull this off this
> performance:
>
> 1) The BList type features transparent copy-on-write.  If a non-root
> node needs to be copied (as part of a getslice, copy, setslice, etc.),
> the node is shared between multiple parents instead of being copied.
> If it needs to be modified later, it will be copied at that time.
> This is completely behind-the-scenes; from the user's point of view,
> the BList works just like a regular Python list.
>
> 2) A B+Tree is a wide, squat tree.  The maximum number of children per
> node is defined by a compile-time parameter (LIMIT).  If the list is
> shorter than LIMIT, then there is only one node, which simply contains
> an array of the objects in the list.  In other words, for short lists,
> a BList works just like Python's array-based list() type.  Thus, it
> has the same good performance on small lists.
>
> So you can see the performance of the BList in more detail, I've made
> several performance graphs available at the following link:
>    http://stutzbachenterprises.com/blist/
>
> Each webpage shows two graphs:
> - The raw execution time for the operation as a function of the list size
> - The execution time for the operation, normalized by the execution
> time of Python's list()
>
> These graphs were made using LIMIT=128.
>
> The only operation where list() is much faster for small lists is
> getitem through the x[5] syntax.  There's a special case for that
> operation deep inside the Python bytecode interpreter.  My extension
> module can't compete with that. :-)  BList is just as fast as list()
> for small lists via .__getitem__().
>
> The source code for the type is available for compilation as an external
> module:
>     http://stutzbachenterprises.com/blist/blist.tar.gz
>
> I wrote it against Python 2.5.
>
> The Radical Proposal
> -------------------------------
>
> Replace list() with the BList.
>
> There are only a few use-cases (that I can think of) where Python's
> list() regularly outperforms the BList.  These are:
>
> 1. A large LIFO stack, where there are many .append() and .pop(-1)
> operations.  These are O(1) for a Python list, but O(log n) for the
> BList().  (BList's are just as fast for small lists)
>
> 2. An large, unchanging list, where there are many getitem() calls,
> and none of the inserts or deletes where the BList() shines.  Again,
> these are O(1) for a Python list, but O(log n) for the BList(), and
> the BList() is just as fast if the list is small.
>
> Use case #1 is covered by collections.deque.  Use case #2 is covered by
> tuples.
>
> When I first started learning Python, I often ended up inadvertently
> writing O(n**2) algorithms by putting a O(n) list operation inside a
> loop (which is a problem when n=100,000).  Since nothing in the
> signature of list() informs the user when an operation is going to be
> slow, it took me a while to learn how to write my code to work around
> this constraint.  In fact, I initially assumed "list" meant "linked
> list" so most of my assumptions about which operations would be fast
> vs. slow were backwards.  I think it would be more Pythonic if the
> user can trust that the implementation will always be fast (at least
> O(log n)).
>
> I recognize that the Python developer community is understandably very
> attached to the current array-based list, so I expect this to get shot
> down.  I hope this doesn't reflect badly on the more modest proposal
> of including a new type in the collections module.  Also, please don't
> kill the messenger. :-)
>
> --
> Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/nevillegrech%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070423/f0bd8136/attachment.html 

From daniel at stutzbachenterprises.com  Mon Apr 23 18:57:18 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 11:57:18 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <2410590390749666520@unknownmsgid>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<2410590390749666520@unknownmsgid>
Message-ID: <eae285400704230957t3d9bdec4s1d26ca9a6b72eded@mail.gmail.com>

On 4/23/07, Bill Janssen <janssen at parc.com> wrote:
> > Replace list() with the BList.
>
> Generally, I like the idea.  But how about extension modules
> which make use of the current implementation details?

PyListObject could be moved to the collections module so that C
extension modules can continue to use it.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From collinw at gmail.com  Mon Apr 23 19:12:03 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 23 Apr 2007 10:12:03 -0700
Subject: [Python-3000] a plea : invalid comparisons should raise an
	exception
In-Reply-To: <tkrat.4dd8c654819e14b7@igpm.rwth-aachen.de>
References: <tkrat.4dd8c654819e14b7@igpm.rwth-aachen.de>
Message-ID: <43aa6ff70704231012q4f6f4cb7i75997d8850f52b2f@mail.gmail.com>

On 4/23/07, Helmut Jarausch <jarausch at igpm.rwth-aachen.de> wrote:
> Some years ago, I was allured to convert from Perl to Python.
> The main argument was that Python is much more secure.
> That turned out to be true with one big exception.
> The following lines would run just fine (as equivalent
> Perl code)
>
> from cStringIO import StringIO
> Inp= StringIO('Guido 10')
> for line in Inp:
>     (Name,Age)= line.split()
>     if Age > 64 :
>         print Name,"has retired"
>
> But with Python it returns a very unexpected result
> without any evidence of a problem.
>
> So, please make comparisons raise an exception (at least
> optionally) if "uncompatible" types are compared.

This is already going to happen; see
http://www.python.org/dev/peps/pep-3100/#core-language, 12th bullet
point.

Collin Winter

From jcarlson at uci.edu  Mon Apr 23 19:16:29 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 10:16:29 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230957t3d9bdec4s1d26ca9a6b72eded@mail.gmail.com>
References: <2410590390749666520@unknownmsgid>
	<eae285400704230957t3d9bdec4s1d26ca9a6b72eded@mail.gmail.com>
Message-ID: <20070423101315.63C8.JCARLSON@uci.edu>


"Daniel Stutzbach" <daniel at stutzbachenterprises.com> wrote:
> On 4/23/07, Bill Janssen <janssen at parc.com> wrote:
> > > Replace list() with the BList.
> >
> > Generally, I like the idea.  But how about extension modules
> > which make use of the current implementation details?
> 
> PyListObject could be moved to the collections module so that C
> extension modules can continue to use it.

Unless the implementation of a module is #included from Python.h, it is
not going to be available to extension modules; either its
implementation or its interface.


 - Josiah


From jcarlson at uci.edu  Mon Apr 23 19:23:23 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 10:23:23 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <de9ae4950704230958u1cacd22bqfe231eaaee23dff2@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<de9ae4950704230958u1cacd22bqfe231eaaee23dff2@mail.gmail.com>
Message-ID: <20070423101749.63CB.JCARLSON@uci.edu>


"Neville Grech Neville Grech" <nevillegrech at gmail.com> wrote:
> I like the implementation of the BList, however I'm not so sure whether
> replacing a list with BList is a good idea since there are many instances
> where python scripts will iterate through endless lists of items. In such
> case time complexity (to iterate a whole list) as you have mentioned is O
> (nlog(n)) instead of O(n). You will probably find that such cases (iterating
> through long lists) are very common and at the same time the developer still
> wants his lists to be mutable. If a developer wants to optimise his lists
> for insertion and deletion speed he can then use an optimised implementation
> like the one you're proposing anyway.

A naive implementation of iteration would be O(nlogn), with a minimal
amount of work, it could be optimized to O(nlogn) worst case (mutations
all the time everywhere) but O(n) when no mutations occur (the majority
of iteration cases seen in the wild).


> What's the performance of concatenation in these BLists? O(n) ?

It would seem to be O(1), because one could trivially add a single new
node that points to the copy-on-write root nodes of each input tree.  At
worst, it would take O(log n + log m) to combine the rightmost branch of
the left addend and the leftmost branch of the right addend.


> Are you suggesting BList as part of the standard library?

He is, in fact, suggesting it as a replacement for list().

 - Josiah


From jcarlson at uci.edu  Mon Apr 23 19:45:26 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 10:45:26 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <20070423102411.63CE.JCARLSON@uci.edu>


"Daniel Stutzbach" <daniel at stutzbachenterprises.com> wrote:
> I have two proposals for Python 3000 that I'll write PEP(s) for if
> there's interest.  The first proposal is, I think, a modest one.  The
> second proposal is related, but more radical.  I fully expect the
> second proposal to be rejected for that alone, especially since I am a
> relatively an outsider to the Python developer community (though it
> was great to meet some of you at PyCon this year).  I bring up the
> more radical proposal primarily for completeness.
[snip]
> I recognize that the Python developer community is understandably very
> attached to the current array-based list, so I expect this to get shot
> down.  I hope this doesn't reflect badly on the more modest proposal
> of including a new type in the collections module.  Also, please don't
> kill the messenger. :-)

I would be +1 on including the object in the collections module in 2.6
and 3.0 .  I've implemented variants of this particular structure using
binary treaps a few different times.

In terms of 3rd party extensions that rely on the interface of a list,
one could probably use the extended buffer protocol being discussed, but
it really isn't nearly as simple as just traversing the array of
PyObject* that a list exposes.  -1 on replacing lists (generally) with
your tree implementation.  Base types should be as simple as possible.


 - Josiah


From pje at telecommunity.com  Mon Apr 23 19:54:53 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 13:54:53 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <2f2e5f950704230937x4b8116cbmd3cd95ebfd96c924@mail.gmail.co
 m>
References: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423134904.045c1e10@sparrow.telecommunity.com>

At 01:37 PM 4/23/2007 -0300, Leonardo Santagada wrote:
>i've thought that the overloading in GF was more for rules like (bar > 
>100) and not for diferent types. will both be suported or only the example 
>that you gave?

The idea is that it should be possible for third-party implementations to 
add support for more sophisticated rules.  The base system would support 
type annotations on arguments only -- and *maybe* interfaces.

This would be done by implementing the generic function implementation 
using generic functions -- something I've already done in 
PEAK-Rules.  Basically, the dispatch machinery is decoupled from the 
mechanism for specifying methods, and if you implement a new dispatcher 
(e.g. one supporting rules like "bar>100"), you simply also register 
methods with the overloading module's generic functions to support your new 
dispatcher.  That way, people can use your "rule" objects with @when, 
@around, @before, @after, etc., without having to learn anything but how to 
create and use other kinds of rules.


>ps: I didn't respond to python-3000 because I was afraid of disturbing 
>serious discussions over there.

This is as serious a discussion as any; if you thought of the question, 
chances are good that somebody else had this question also.  Thus, I'm 
cc:ing this back to the list, before any of them decide to ask me, too.  :)


From agthorr at barsoom.org  Mon Apr 23 19:53:34 2007
From: agthorr at barsoom.org (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 12:53:34 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <de9ae4950704230958u1cacd22bqfe231eaaee23dff2@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<de9ae4950704230958u1cacd22bqfe231eaaee23dff2@mail.gmail.com>
Message-ID: <eae285400704231053w57a63056se4c44cee11808031@mail.gmail.com>

On 4/23/07, Neville Grech Neville Grech <nevillegrech at gmail.com> wrote:
> I like the implementation of the BList, however I'm not so sure whether
> replacing a list with BList is a good idea since there are many instances
> where python scripts will iterate through endless lists of items. In such
> case time complexity (to iterate a whole list) as you have mentioned is O
> (nlog(n)) instead of O(n).

Iteration using an iterator object is O(n).  For example, see the
left-hand graph at:
    http://stutzbachenterprises.com/blist/forloop.html

Of course, the following idiom is still O(n log n):

for i in range(len(x)):
    do_something(x[i])

(but the base of the logarithm is large, so for n < 100 or so, ceil(log n) = 1)

> You will probably find that such cases (iterating
> through long lists) are very common and at the same time the developer still
> wants his lists to be mutable. If a developer wants to optimise his lists
> for insertion and deletion speed he can then use an optimised implementation
> like the one you're proposing anyway.
>
> What's the performance of concatenation in these BLists? O(n) ?

Concatenating a BList to a BList is O(log n + log m) where n and m are
the size of the two BLists.

Concatenating any other iterable to a BList is (log n + m) where n is
the size of the BList and m is the length of the iterable.

> Are you suggesting BList as part of the standard library?

I'm making two mutually exclusive but related suggestions:

1) Add BList to the standard library as part of the collections module
(which I hope will be accepted), or
2) Replace list() with BList (which I expect to be rejected)

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From daniel at stutzbachenterprises.com  Mon Apr 23 19:58:20 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 12:58:20 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <20070423101315.63C8.JCARLSON@uci.edu>
References: <2410590390749666520@unknownmsgid>
	<eae285400704230957t3d9bdec4s1d26ca9a6b72eded@mail.gmail.com>
	<20070423101315.63C8.JCARLSON@uci.edu>
Message-ID: <eae285400704231058x527269d3g57f10a5b3e6fb532@mail.gmail.com>

On 4/23/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> "Daniel Stutzbach" <daniel at stutzbachenterprises.com> wrote:
> > On 4/23/07, Bill Janssen <janssen at parc.com> wrote:
> > > > Replace list() with the BList.
> > >
> > > Generally, I like the idea.  But how about extension modules
> > > which make use of the current implementation details?
> >
> > PyListObject could be moved to the collections module so that C
> > extension modules can continue to use it.
>
> Unless the implementation of a module is #included from Python.h, it is
> not going to be available to extension modules; either its
> implementation or its interface.

So, I was unclear.  I was picturing PyListObject still being built-in
to the interpreter and exposed via Python.h (in C).  That way,
extension modules could still use it.  Within the Python language, it
could be made available via collections.list.  Or it could continue to
be __buitins__.list.

By "replacing list", I'm mostly concerned about the return type of
things like "[1,2,3,4]" and list comprehensions.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From guido at python.org  Mon Apr 23 20:38:07 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 11:38:07 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <ca471dc20704231138q6bdfa35cj31ddd14a8df8f3ac@mail.gmail.com>

On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> I have two proposals for Python 3000 that I'll write PEP(s) for if
> there's interest.  The first proposal is, I think, a modest one.  The
> second proposal is related, but more radical.  I fully expect the
> second proposal to be rejected for that alone, especially since I am a
> relatively an outsider to the Python developer community (though it
> was great to meet some of you at PyCon this year).  I bring up the
> more radical proposal primarily for completeness.
>
> The Modest Proposal
> -------------------------------
>
> As a pet project, I've written a container type that looks, acts, and
> quacks like a Python list(), but has better asymptotic performance.
> Specifically, O(log n) time for inserts and deletes in the middle of
> the list.  I'd like to offer it for inclusion in the collections
> module.
[...]

> The Radical Proposal
> -------------------------------
>
> Replace list() with the BList.

Some meta-comments:

(1) You really need Raymond Hettinger's review. Nothing else matters much.

(2) I'm against proliferation of data types, because it makes the
choices for developers harder. For that reason, I'd be against the
modest proposal without the radical proposal. But I'd be in favor of
the radical proposal if the pragmatics (mostly of existing core code
and 3rd party extensions using the PyList_* APIs) can be solved, and
if the performance is uniformly better.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mike.klaas at gmail.com  Mon Apr 23 20:42:02 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Mon, 23 Apr 2007 11:42:02 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>

On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:

> So you can see the performance of the BList in more detail, I've made
> several performance graphs available at the following link:
>    http://stutzbachenterprises.com/blist/

Very cool.  These detailed timings are very useful.

Might it be possible to include variance bars?  A few of the graphs
show your data structure faster than the list even for small N, and it
would be great to see if it is meaningfully faster in these cases (if
so, perhaps there are optimization possibilities for the current list
data structure).

A quite glance at your results shows a few more cases where the blist
faces problems, such no longer using Timsort, and so suffering in
performance for sorted/reversed cases.  That would be painful to give
up in the built-in list implementation.

-Mike

From guido at python.org  Mon Apr 23 20:43:12 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 11:43:12 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
Message-ID: <ca471dc20704231143n5851c591l6333a4d1af72c9aa@mail.gmail.com>

On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>    from overloading import overload, abstract
>
>    def foo(bar, baz):
>        """Default implementation here"""
>
>
>    @overload
>    def foo(bar:int, baz:str):
>        """Overloaded implementation for int/str here"""
>
>
>    @abstract
>    def spam(fizz):
>        """This function has no default implementation, and raises
>           a "no applicable methods" error if called..."""

(a) What's the point of having a separate keyword for this, as opposed
to just raising an exception from the body?

(b) I'm proposing an @abstractmethod, which sets its argument's
__isabstractmethod__ attribute and then returns it, otherwise
unchanged. Would this be good enough for you, or do you need @abstract
to do more? My point is, I could rename mine to @abstract so that it
would serve your purpose as well -- but only if it would serve.

>    @overload
>    def spam(fizz:list):
>        """...unless an appropriate overload is added first"""
>
> Any questions?  :)

I'd like the examples even better if they used ABCs from PEP 3119. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nevillegrech at gmail.com  Mon Apr 23 20:58:45 2007
From: nevillegrech at gmail.com (Neville Grech Neville Grech)
Date: Mon, 23 Apr 2007 20:58:45 +0200
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704231053w57a63056se4c44cee11808031@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<de9ae4950704230958u1cacd22bqfe231eaaee23dff2@mail.gmail.com>
	<eae285400704231053w57a63056se4c44cee11808031@mail.gmail.com>
Message-ID: <de9ae4950704231158t1f807333u43e45b558b07f2ea@mail.gmail.com>

You're right (I was seeing the statistics). Seems that it will outperform
the built in list in most cases then. I would be +1 for both proposals if
there are no underlying integration issues. Great work.

On 4/23/07, Daniel Stutzbach <agthorr at barsoom.org> wrote:
>
> On 4/23/07, Neville Grech Neville Grech <nevillegrech at gmail.com> wrote:
> > I like the implementation of the BList, however I'm not so sure whether
> > replacing a list with BList is a good idea since there are many
> instances
> > where python scripts will iterate through endless lists of items. In
> such
> > case time complexity (to iterate a whole list) as you have mentioned is
> O
> > (nlog(n)) instead of O(n).
>
> Iteration using an iterator object is O(n).  For example, see the
> left-hand graph at:
>     http://stutzbachenterprises.com/blist/forloop.html
>
> Of course, the following idiom is still O(n log n):
>
> for i in range(len(x)):
>     do_something(x[i])
>
> (but the base of the logarithm is large, so for n < 100 or so, ceil(log n)
> = 1)
>
> > You will probably find that such cases (iterating
> > through long lists) are very common and at the same time the developer
> still
> > wants his lists to be mutable. If a developer wants to optimise his
> lists
> > for insertion and deletion speed he can then use an optimised
> implementation
> > like the one you're proposing anyway.
> >
> > What's the performance of concatenation in these BLists? O(n) ?
>
> Concatenating a BList to a BList is O(log n + log m) where n and m are
> the size of the two BLists.
>
> Concatenating any other iterable to a BList is (log n + m) where n is
> the size of the BList and m is the length of the iterable.
>
> > Are you suggesting BList as part of the standard library?
>
> I'm making two mutually exclusive but related suggestions:
>
> 1) Add BList to the standard library as part of the collections module
> (which I hope will be accepted), or
> 2) Replace list() with BList (which I expect to be rejected)
>
> --
> Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070423/2a4efde8/attachment.html 

From brett at python.org  Mon Apr 23 21:05:24 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Apr 2007 12:05:24 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <d11dcfba0704222143k66f1b7bye9a52e452098dc8e@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<d11dcfba0704222143k66f1b7bye9a52e452098dc8e@mail.gmail.com>
Message-ID: <bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>

On 4/22/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/22/07, Brett Cannon <brett at python.org> wrote:
> > Implementation
> > ==============
> >
> > When the ``-m`` option is used, ``sys.main`` will be set to the
> > argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> > Then the equivalent of ``__import__(self.main)`` will occur.  This
> > differs from current semantics as the ``runpy`` module fetches the
> > code object for the file specified by the module name in order to
> > explicitly set ``__name__`` and other attributes.  This is no longer
> > needed as import can perform its normal operation in this situation.
> >
> > If a file name is specified, then ``sys.main`` will be set to
> > ``"__main__"``.  The specified file will then be read and have a code
> > object created and then be executed with ``__name__`` set to
> > ``"__main__"``.  This mirrors current semantics.
>
> So __name__ will still sometimes be "__main__". That's disappointing.
>

Yeah, I know.  But I can't force people to only run Python scripts
that happen to be on sys.path.

> To clarify, assuming that the foo.bar module contains something like
> "from . import baz", this PEP only addresses the following situation::
>
>     python -mfoo.bar
>
> and all of the following will still raise ImportErrors::
>
>     ~> python foo/bar
>     ~/grok> python ../foo/bar
>     ~/foo> python bar
>
> Right?

It could handle the last one no problem.  The foo/bar it could if it
was deemed worth it to infer the module name.  The middle one *might*
be solvable with some checking on sys.path and walking the directory.

In other words they are possible if we want to pay the startup cost of
trying to infer the module's name.

>
> If that's right, I'm -1 on the proposal. It's complicating the
> standard "am I the main module?" idiom to solve a tiny little problem
> with "-m".

It's not a '-m' problem, it's a relative import problem.  This issue
occurs no matter what solution you go with, even with the specially
named main() function idea some people are tossing around.

> The "am I the main module?" idiom has been around much
> longer than the "-m" flag, and I'd prefer to see a more compelling
> reason to change it.
>

When you start using relative imports exclusively and find you have to
tweak your imports to execute certain modules you might change your
mind.  =)

> Two reasons that would be compelling for me:
>
> * Simplifying the "am I the main module?" idiom, e.g. with the
> rejected ``if __main__`` proposal.
>
> * Getting rid of the "__main__" value for __name__ entirely. This
> would require code like the following to determine the name of a
> module given at the command line::
>
>     def get_name(path):
>         sys_path_set = set(sys.path)
>         path, _ = os.path.splitext(path)
>         path = os.path.abspath(path)
>         parts = path.split(os.path.sep)
>         for i in range(1, len(parts)):
>             sub_path = os.path.sep.join(parts[:i])
>             if sub_path in sys_path_set:
>                 return '.'.join(parts[i:])
>         return parts[-1]
>
>   Even if we can't do the name resolution perfectly, if the flaws are
> documented (e.g. when '..' and symbolic links are combined) I think
> that would be enough better than the current situation to merit
> changing the standard "am I the current module?" idiom.
>

As I said in the PEP, I rejected this because of startup cost.  If
people are willing to pay for it then we can go with it and just
assume that if the current directory is not a prefix for the module's
path that we just use the filename to set the module's name and
sys.main, that's fine as well.  It just is not as accurate as it might
be.

So the questions for people to weigh in on are:

* For spam/bacon.py w/ with spam/__init__.py existing:

  - Should we infer it as spam.bacon?
    Startup cost, but accurate name allows for relative imports.
  - Should we infer as bacon?
    Cheap but breaks relative imports.
  - Should we infer the name as __main__?
    Status quo and breaks relative imports.

* For spam/bacon.py with *no* spam/__init__.py:

  - Should we infer the name as bacon?
    No relative imports possible since not in a package.
  - Should we infer the name as __main__?
    Status quo and no relative import issue as not in a package.

* For ../bacon.py

   - With ../__init__.py defined and ../../ on sys.path:

      + Should we infer the name as bacon?
         Simple, but breaks relative imports.
      + Should we walk up the directories until __init__.py is not
found and check for that directory on sys.path?
          Costly startup, but allows for relative imports to work.
       + Should we infer the name as __main__?
          Status quo and relative imports don't work.

   - Without ../__init__.py

       + Infer as bacon?
          No relative imports possible so fine.
       + Infer as __main__?
          Status quo, and no relative imports even possible.


If people want a simple inference rule we can ditch __main__ entirely
and just go with a simple rule that would be more costly for startup.
Otherwise we can stick with __main__ as the PEP has proposed and have
a partial solution.  Or we can ditch the PEP.

-Brett

From janssen at parc.com  Mon Apr 23 21:19:23 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 23 Apr 2007 12:19:23 PDT
Subject: [Python-3000] Principles (was: Two proposals for a new list-like
	type: one modest, one radical)
In-Reply-To: <20070423102411.63CE.JCARLSON@uci.edu> 
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<20070423102411.63CE.JCARLSON@uci.edu>
Message-ID: <07Apr23.121924pdt."57996"@synergy1.parc.xerox.com>

> Base types should be as simple as possible.

Is that really a good idea?  Wouldn't we wind up with BLISS (unsigned
integers)?  I think it's always desirable to provide more useful base
types, but that it has to be balanced against conversion and
maintenance costs.

Bill

From pje at telecommunity.com  Mon Apr 23 21:28:14 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 15:28:14 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704231143n5851c591l6333a4d1af72c9aa@mail.gmail.co
 m>
References: <5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>

At 11:43 AM 4/23/2007 -0700, Guido van Rossum wrote:
>On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >    from overloading import overload, abstract
> >
> >    def foo(bar, baz):
> >        """Default implementation here"""
> >
> >
> >    @overload
> >    def foo(bar:int, baz:str):
> >        """Overloaded implementation for int/str here"""
> >
> >
> >    @abstract
> >    def spam(fizz):
> >        """This function has no default implementation, and raises
> >           a "no applicable methods" error if called..."""
>
>(a) What's the point of having a separate keyword for this, as opposed
>to just raising an exception from the body?

EIBTI, mainly.  Specifically, it's clear from the beginning that you're 
looking at an empty function -- i.e., one that *can't* be called 
successfully except for explicitly registered patterns -- versus one that 
has a truly "generic" base implementation.

For example, a "pprint" generic function whose implementation falls back to 
repr() is a bit different than say, a "get_stream()" function that can only 
be called on path objects, stringIOs, files, and sockets.  The latter has 
no "generic" implementation.

Using the decorator also makes it easier to be consistent about what error 
is raised -- @abstract is easier for me to remember than which exception to 
raise.  :)

I believe there was also another, somewhat more esoteric reason that gets 
involved when you build more sophisticated rulesystems on top of the base 
machinery, but the exact reason is escaping me at this moment.  I remember 
there being something that bothered me about your original Py3K overloading 
prototype having to always use the original function body as the default 
condition, but I don't remember specifically what, except that it was sort 
of like doing math without being able to write "zero": you can work around 
it, but it's kind of weird.  :)


>(b) I'm proposing an @abstractmethod, which sets its argument's
>__isabstractmethod__ attribute and then returns it, otherwise
>unchanged. Would this be good enough for you, or do you need @abstract
>to do more? My point is, I could rename mine to @abstract so that it
>would serve your purpose as well -- but only if it would serve.

Assuming that:

1.  If you call such a function, it will raise some error, like 
NotImplementedError
2.  It's a normal function object (i.e., can have its func_code repointed 
later)
3.  The __isabstractmethod__ can be set back to False once there is other 
code registered

Then yes, it's perfect.


> >    @overload
> >    def spam(fizz:list):
> >        """...unless an appropriate overload is added first"""
> >
> > Any questions?  :)
>
>I'd like the examples even better if they used ABCs from PEP 3119. :-)

I'll keep that in mind for the actual PEP.  :)


From steven.bethard at gmail.com  Mon Apr 23 21:49:01 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 23 Apr 2007 13:49:01 -0600
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<d11dcfba0704222143k66f1b7bye9a52e452098dc8e@mail.gmail.com>
	<bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>
Message-ID: <d11dcfba0704231249o7e0dc825p8e6d74a6deeb55eb@mail.gmail.com>

On 4/23/07, Brett Cannon <brett at python.org> wrote:
> On 4/22/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > On 4/22/07, Brett Cannon <brett at python.org> wrote:
> > > Implementation
> > > ==============
> > >
> > > When the ``-m`` option is used, ``sys.main`` will be set to the
> > > argument passed in.  ``sys.argv`` will be adjusted as it is currently.
> > > Then the equivalent of ``__import__(self.main)`` will occur.  This
> > > differs from current semantics as the ``runpy`` module fetches the
> > > code object for the file specified by the module name in order to
> > > explicitly set ``__name__`` and other attributes.  This is no longer
> > > needed as import can perform its normal operation in this situation.
> > >
> > > If a file name is specified, then ``sys.main`` will be set to
> > > ``"__main__"``.  The specified file will then be read and have a code
> > > object created and then be executed with ``__name__`` set to
> > > ``"__main__"``.  This mirrors current semantics.
[snip]
> > If that's right, I'm -1 on the proposal. It's complicating the
> > standard "am I the main module?" idiom to solve a tiny little problem
> > with "-m".
>
> It's not a '-m' problem, it's a relative import problem.

Right, but the PEP was only solving the relative import problem for '-m'.

> > Two reasons that would be compelling for me:
> >
> > * Simplifying the "am I the main module?" idiom, e.g. with the
> > rejected ``if __main__`` proposal.
> >
> > * Getting rid of the "__main__" value for __name__ entirely. This
> > would require code like the following to determine the name of a
> > module given at the command line::
[snip]
> >   Even if we can't do the name resolution perfectly, if the flaws are
> > documented (e.g. when '..' and symbolic links are combined) I think
> > that would be enough better than the current situation to merit
> > changing the standard "am I the current module?" idiom.
>
> As I said in the PEP, I rejected this because of startup cost.  If
> people are willing to pay for it then we can go with it

I'm personally willing to pay for it because it's only going to apply
at most once each time ``python`` is started.

> So the questions for people to weigh in on are:

I'm basically +1 on all the versions that do the full lookup. That
means scanning up the parent directories until we encounter either a
directory with no __init__.py or a directory on sys.path.

> * For spam/bacon.py w/ with spam/__init__.py existing:

Infer 'spam.bacon':
* parent dir of 'bacon.py', 'spam', has an '__init__.py', so 'spam.bacon'
* parent dir of 'spam' is on sys.path (it's the current directory), so stop

> * For spam/bacon.py with *no* spam/__init__.py:

Infer 'bacon':
* parent dir of 'bacon.py' has no '__init__.py', so stop

> * For ../bacon.py
>
>    - With ../__init__.py defined and ../../ on sys.path:

Infer 'spam.bacon' (assuming parent directory is 'spam'):
* parent dir of 'bacon.py', 'spam', has an '__init__.py', so 'spam.bacon'
* parent dir of 'spam' is on sys.path, so stop

>    - Without ../__init__.py

Infer 'bacon'
* parent dir of 'bacon.py' has no '__init__.py', so stop.

The above were pretty easy. Here's a harder one:

    foo/
      __init__.py
      bar/
        __init__.py
        baz/
          __init__.py
          frobble.py

where the 'foo' directory is on sys.path. Should this infer
'bar.baz.frobble' or 'foo.bar.baz.frobble'?  The former assumes that
sys.path takes precedence over the presence of an '__init__.py'.  The
latter assumes the reverse.  (I lean towards the fomer FWIW, though it
might be worth giving the person who organized their code this way
some sort of bodily threat.) ;-)

> If people want a simple inference rule we can ditch __main__ entirely
> and just go with a simple rule that would be more costly for startup.

This is my preference, but I guess I'd like to know how big the
startup costs would be. Clearly if Python takes twice as long to
start, it's not worth it. ;-)

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From brett at python.org  Mon Apr 23 22:00:18 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Apr 2007 13:00:18 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <462C7CFB.5060607@gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<462C7CFB.5060607@gmail.com>
Message-ID: <bbaeab100704231300ofe82254y503c42814c5f9a04@mail.gmail.com>

On 4/23/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Brett Cannon wrote:
> > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to
> > ``if __name__ == sys.main: ...``  so that you at least have a chance
> > to execute module in a package that use relative imports.
> >
> > Ran this PEP past python-ideas.  Stopped the discussion there when too
> > many new ideas were being proposed.  =)  I have listed all of them in
> > the Rejected Ideas section, although if overwhelming support for one
> > comes forward the PEP can shift to one of them.
>
> -1
>
> I still prefer the "__module_name__" fallback that was actually briefly
> in the tree for 2.5 (the change was made really late in the 2.5 cycle,
> so it was backed out with the intention of revisiting the idea for 2.6 -
> check the svn log for lib/runpy.py).
>
> The trick with that approach was that runpy.run_module() would *always*
> set "__module_name__" to the real name of the module being executed,
> regardless of what it was told to set "__name__" to.
>
> Then the import machinery was modified such that if it came across an
> explicit relative import being made from a module called "__main__", it
> would check to see if "__module_name__" was set, and if so, use that
> instead.
>

I don't like having to tweak the import machinery for this.  It feels
like working around a problem instead of trying to solve it.  Plus it
doesn't address Steve's concerns of when modules are still named
'__main__' because there absolute name cannot be figured out.

-Brett

From brett at python.org  Mon Apr 23 22:01:55 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Apr 2007 13:01:55 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <f0ib2l$tpc$1@sea.gmane.org>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<f0h54s$8s1$1@sea.gmane.org>
	<bbaeab100704222052y1111376fwa333e371877a504b@mail.gmail.com>
	<f0ib2l$tpc$1@sea.gmane.org>
Message-ID: <bbaeab100704231301u4dda6925tb0a8ea3ff09c9d49@mail.gmail.com>

On 4/23/07, Christian Heimes <lists at cheimes.de> wrote:
> Brett Cannon wrote:
> >> Someone on python-idea brought up the idea to add the sys module to
> >> builtins. The feature would remove the requirement for an extra line and
> >> the costs should be minimal or even zero since sys is a special builtin
> >> module.
> >>
> >
> > That's a separate PEP.
>
> Do you remember the PEP number? I can't find the PEP.
>

299.

> >> Some people including me like to resurrect the PEP 299 __main__()
> >> function idea. http://www.python.org/dev/peps/pep-0299/
> >
> > As I have already said multiple times on python-ideas, I don't
> > personally like that idea.  If someone wants to do the work to try to
> > push that view then they can.
>
> I know. I mentioned it because I "wanted to push that view". O:-)

That's fine, but this view in no way addresses the problem I am
tackling; how to allow a module to be executed that has a relative
import.  With the change you want you don't even address the issue I
am trying to tackle.

-Brett

From jcarlson at uci.edu  Mon Apr 23 22:13:23 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 13:13:23 -0700
Subject: [Python-3000] Principles (was: Two proposals for a new
	list-like type: one modest, one radical)
In-Reply-To: <07Apr23.121924pdt."57996"@synergy1.parc.xerox.com>
References: <20070423102411.63CE.JCARLSON@uci.edu>
	<07Apr23.121924pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <20070423123456.63D7.JCARLSON@uci.edu>


Bill Janssen <janssen at parc.com> wrote:
> > Base types should be as simple as possible.
> Is that really a good idea?  Wouldn't we wind up with BLISS (unsigned
> integers)?

With everything, there is a balance.  I do not advocate *removing*
functionality to make an object simpler; that would be absurd.  On the
other hand, when we already have types that are necessarily as simple as
possible to implement a reasonable set of functionality (lists, tuples,
integers, floats, dictionaries, str, etc.), I think that's a good
starting point.

Now, do we replace a structure which is very easy to understand and
maintain with a structure that necessarily isn't as easy to understand
and maintain? What if it breaks 3rd party code that expects the old
implementation? At what point do we say that certain performance
improvements are or are not significant enough to balance API breakage,
performance assumptions, etc.?

I believe that no performance improvement warrants such large scale
breakage of 3rd party extensions.  This is the same reason why I didn't
support the string view+concatenation mechanism that was offered by
Larry Hastings.


> I think it's always desirable to provide more useful base
> types, but that it has to be balanced against conversion and
> maintenance costs.

Indeed.  As a base type, I don't believe a BTree implementation
replacing the array implementation of a list is a good thing.

Guido seems to be against the addition of yet another type into
collections (which has defaultdict and deque in 2.5).  Me, I'd be happy
to have this object in collections (as I said, I've implemented this
kind of thing before).

Depending on my mood, I may even support replacing the deque
implementation with the Daniel's BTree, as the timings that Daniel has
provided show that inserting and deleting is quite fast for moderately
sized sequences (10k elements).  When people ask, "I want a list-like
thing that is able to do X", we can tell them, "use a deque, it's really
fast for just about everything".  Later when people invariably ask, "if
it's fast for most everything, why doesn't it replace list?" we can tell
them, "it breaks too many 3rd party extensions that rely on the
array-based implementation of list."


 - Josiah


From jcarlson at uci.edu  Mon Apr 23 22:20:46 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 13:20:46 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>
References: <d11dcfba0704222143k66f1b7bye9a52e452098dc8e@mail.gmail.com>
	<bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>
Message-ID: <20070423122833.63D4.JCARLSON@uci.edu>


"Brett Cannon" <brett at python.org> wrote:
[snip]
> So the questions for people to weigh in on are:
[snip]
> * For ../bacon.py
[snip]
>       + Should we walk up the directories until __init__.py is not
> found and check for that directory on sys.path?
>           Costly startup, but allows for relative imports to work.

+.99, but should be applied regardless of ../bacon.py, bacon.py, or
spam/bacon.py .

Generally, I believe that we should advance up the file system/whatever
paths until we run out of __init__.py .  That would give every package,
module, etc., a unique name that is the same regardless of whether it is
run as bacon.py, ../bacon.py, or spam/bacon.py .  I also realize that
this idea may not be popular.

> If people want a simple inference rule we can ditch __main__ entirely
> and just go with a simple rule that would be more costly for startup.

Ditch __main__, at least check for path/__init__.py when path/bacon.py
is run.  Whether to advance up the parent paths, and/or how deep to
advance up the parent paths I don't know.  I know what I want, but I
understand that it may not be what others want (especially when
confronted with NFS, database imports, etc.).


 - Josiah


From jimjjewett at gmail.com  Mon Apr 23 22:17:35 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 16:17:35 -0400
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<d11dcfba0704222143k66f1b7bye9a52e452098dc8e@mail.gmail.com>
	<bbaeab100704231205i4e2d36ecg2d098b4f81ed53f1@mail.gmail.com>
Message-ID: <fb6fbf560704231317h720bbf34g5cd0624f15065933@mail.gmail.com>

On 4/23/07, Brett Cannon <brett at python.org> wrote:
> On 4/22/07, Steven Bethard <steven.bethard at gmail.com> wrote:

> In other words they are possible if we want to pay the startup cost of
> trying to infer the module's name.

This doesn't need to be a startup cost for python; it can be delayed
until an actual relative import is requested.  (Though I suppose that
might mean making __name__ a lazy attribute, which itself seems like
overkill.)

Call me cynical, but the problem seems to crop up only when *all* of
the following are true:

(a)  Your helper modules are not on sys.path.   (and you won't add
them -- or at the very least, you want to ensure using a local copy
instead of one that might be earlier on sys.path.)

(b)  You helper modules are not in the same directory as your main
code, or even a subdirectory of it.  (Maybe someone else is supplying
a black-box API, but you can't tell them where to install/symlink it.
Maybe your code is so complex that nothing is at the top level.)

(c)  You insist on running as __main__, instead of getting called as a
library by a better-placed wrapper script.  (Because doing it that way
is more convenient, for a quick-and-dirty version?  This conflicts
with (b).)

(d)  You have either a *lot* of imports or a very slow filesystem, and
can't afford much slop.  This also conflicts with (c).

I won't say this use case doesn't matter, but I'm not sure it is worth
any extra complexity in the import rules.

I *am* convinced that people needing that much complexity should be
the only ones to pay the extra startup cost -- but extending it to all
uses of -m might be OK.

The choice for those people who actually use it is really (1) Do it
right, despite the cost or (2) Accept that you don't really need such
a comprehensive install/config framework, and simplify your code.

> * For spam/bacon.py w/ with spam/__init__.py existing:

>   - Should we infer it as spam.bacon?
>     Startup cost, but accurate name allows for relative imports.

uh ... how much cost?  To me it looks like one extra file check --
less than the cost of having more than one directory in sys.path.

>   - Should we infer as bacon?
>     Cheap but breaks relative imports.
>   - Should we infer the name as __main__?
>     Status quo and breaks relative imports.

But still probably good enough most of the time ... just put "." on
sys.path.  If you can't do that, you're in the special-case zone.

> * For spam/bacon.py with *no* spam/__init__.py:

>   - Should we infer the name as bacon?
>     No relative imports possible since not in a package.
>   - Should we infer the name as __main__?
>     Status quo and no relative import issue as not in a package.

So does it matter which you choose?
Would it really be all that awful even if you did allow relative
imports based strictly on the directory structure?

> * For ../bacon.py
>
>    - With ../__init__.py defined and ../../ on sys.path:

So you can't change directory before launching, *and* you are
launching with a relative filename instead of an absolute ... which
suggests quick-hack.  You may not need the fancy inferences, nor
should you object to the price.

>       + Should we infer the name as bacon?
>          Simple, but breaks relative imports.

>       + Should we walk up the directories until __init__.py is not
> found and check for that directory on sys.path?
>           Costly startup, but allows for relative imports to work.

So if you care about startup costs, launch from the right directory...

-jJ

From jimjjewett at gmail.com  Mon Apr 23 22:52:57 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 16:52:57 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
Message-ID: <fb6fbf560704231352j3f8b0a27kd068861c4fc48075@mail.gmail.com>

On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:43 AM 4/23/2007 -0700, Guido van Rossum wrote:
> >On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:

> > >    @abstract
> > >    def spam(fizz):
> > >        """This function has no default implementation, and raises
> > >           a "no applicable methods" error if called..."""

> >(a) What's the point of having a separate keyword for this, as
> >opposed to just raising an exception from the body?

> EIBTI, mainly.  Specifically, it's clear from the beginning that you're
> looking at an empty function -- i.e., one that *can't* be called
> successfully except for explicitly registered patterns -- versus one that
> has a truly "generic" base implementation.

erm... not to me.  To me it looks like any short function until I look
carefully.  Once I do notice that the function body is only a string,
I get a bad reaction, until I remember that is is really just a
strange abbreviation for "pass".

> I believe there was also another, somewhat more esoteric
> reason that gets involved when you build more sophisticated
> rulesystems on top of the base machinery, but the exact reason
> is escaping me at this moment. ... sort
> of like doing math without being able to write "zero":

I think it had to do with next_method, and was the same problem super
has.  If you aren't sure the job is done -- no matter what you got
mixed with -- then you have to call the next method.  99% of the time,
it won't do anything useful, but you have to call it just in case.

My favorite example is __del__(), or at least close().  I can be sure
that I've cleaned up *my* scarce resources, but I don't know what
subclasses might have mixed me in with.  So I call super  __del__, and
catch the Exception when it turns out I wasn't mixed in at all.

A "pass" default implementation is much nicer to subclasses than an
Exception that they have to try...except.

> > My point is, I could rename mine to @abstract so that it
> >would serve your purpose as well -- but only if it would serve.

> Assuming that:

> 1.  If you call such a function, it will raise some error, like
> NotImplementedError

Note that your default methods don't do this either, except as part of
the wrapper.  Guido's current implementation is that

    (a)  An error is raised as soon as you try to instantiate a class
that *could* call the method directly, even if you don't plan to ever
*actually* call it.

    (b)  The method itself is annotated, so that the rules engine
could raise an error if it so chose.

> 2.  It's a normal function object (i.e., can have its func_code
> repointed later)
> 3.  The __isabstractmethod__ can be set back to False once there
> is other code registered

I'm not sure that actually *doing* this would play well with ABC
assumptions, but, yes, it is possible.

> Then yes, it's perfect.

-jJ

From michele.cella at gmail.com  Mon Apr 23 23:08:03 2007
From: michele.cella at gmail.com (Michele Cella)
Date: Mon, 23 Apr 2007 23:08:03 +0200
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>	<f0d5qi$2fu$1@sea.gmane.org>
	<462BDCAA.7020501@canterbury.ac.nz>	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
Message-ID: <f0j77k$nj5$1@sea.gmane.org>

Thomas Wouters wrote:
> 
> But how would it *work*? I assume you want 'super' to take a single 
> expression, and be an expression itself. How does it know which instance 
> you want to 'super'? Does it specialcase the variable named 'self'? 
> Threat 'everything up to the first dot' as 'the instance'? Specialcase 
> the first argument to the method? (That's what ' super.foo()' would do.) 
> How would you do things like:
> 
>   # call the supermethod 'register' with the nonsuper 'callback':
>   super.register(self.callback)
>   # call the nonsuper 'register' with the supermethod 'callback'
>   self.register(super.callback)
>   # call the 'frominstance' method of the 'innerclass' attribute on the 
> superclass
>   super.innerclass.frominstance(self)
> 

Hi Thomas,

Yes, you raise good points.

My solution is nice (IMHO) for the simplest use case (and, probably, the 
most frequent) but it's behavior is really unclear for these complex use 
cases. Thanks for pointing that out.

As far as I'm concerned, I can only confirm that having a super keyword 
with methods attached (like Aahz said) when no other keyword has (and 
there are many) doesn't feel right (to me) and can be confusing at 
first. That's unfortunate as I rarely found python confusing up to now 
(contrary to other languages).

The only other option that comes to my mind is using a special attribute 
in the instance itself (like __class__, __dict__, ...):

	self.__super__.mymethod(arg)

I'm sure this (again) introduces other problems I simply overlooked, in 
this case, forgive me please. ;-)

Thanks again for your attention.

Ciao
Michele


From ncoghlan at gmail.com  Mon Apr 23 23:22:00 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 24 Apr 2007 07:22:00 +1000
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704231300ofe82254y503c42814c5f9a04@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>	
	<462C7CFB.5060607@gmail.com>
	<bbaeab100704231300ofe82254y503c42814c5f9a04@mail.gmail.com>
Message-ID: <462D2378.3020006@gmail.com>

Brett Cannon wrote:
> On 4/23/07, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Then the import machinery was modified such that if it came across an
>> explicit relative import being made from a module called "__main__", it
>> would check to see if "__module_name__" was set, and if so, use that
>> instead.
>>
> 
> I don't like having to tweak the import machinery for this.  It feels
> like working around a problem instead of trying to solve it.

As you yourself said, it's a relative import problem, more so than a 
main module execution problem. I think addressing it on the import side 
(with some cooperation from the main module execution code) reflects that.

The other nice thing about this solution, is that you can explicitly 
enable main relative imports without using "-m", simply by setting 
__module_name__ appropriately (an alternative would be to call the 
additional attribute __package_name__, to save a small amount of string 
fiddling in the relative import code)

>  Plus it
> doesn't address Steve's concerns of when modules are still named
> '__main__' because there absolute name cannot be figured out.

That problem is never going to go away, IMO - if you run a file 
directly, Python has no clue where in the package hierarchy you intend 
it to live. We provide a mechanism (the -m switch) that allows people to 
be explicit if they want to - other than that, I say we resist the 
temptation to guess (take a look at PEP 302 to get some idea of just how 
wildly wrong attempts to guess a package name from a file name may be).

Now, when it comes to use cases for executing packages inside a module:

   ./python -m test.regrtest  (I use this one all the time)
   python -m pychecker.checker (I should use this more than I do...)
   Test modules embedded inside a package [1]

I forget who first described the use case in [1] to me, but it was 
instrumental in convincing me the current behaviour was a bug that 
should be fixed. SF Bug #1510172 is currently sitting open, pending a 
resolution of this question (i.e. am I correct in thinking that this is 
a bug that should be fixed, or are we better off keeping main module 
behaviour consistent between direct execution and the -m switch?).

Cheers,
Nick.

[1]
http://mail.python.org/pipermail/python-dev/2006-June/066680.html




-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Mon Apr 23 23:25:04 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 17:25:04 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <fb6fbf560704231352j3f8b0a27kd068861c4fc48075@mail.gmail.co
 m>
References: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423170905.02ebd3d0@sparrow.telecommunity.com>

At 04:52 PM 4/23/2007 -0400, Jim Jewett wrote:
>On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 11:43 AM 4/23/2007 -0700, Guido van Rossum wrote:
>> >On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>
>> > >    @abstract
>> > >    def spam(fizz):
>> > >        """This function has no default implementation, and raises
>> > >           a "no applicable methods" error if called..."""
>
>> >(a) What's the point of having a separate keyword for this, as
>> >opposed to just raising an exception from the body?
>
>>EIBTI, mainly.  Specifically, it's clear from the beginning that you're
>>looking at an empty function -- i.e., one that *can't* be called
>>successfully except for explicitly registered patterns -- versus one that
>>has a truly "generic" base implementation.
>
>erm... not to me.  To me it looks like any short function until I look
>carefully.

Which is precisely why @abstract is useful.  Are you saying you *don't* 
want @abstract explicitly indicating that the function has no behavior?



>>I believe there was also another, somewhat more esoteric
>>reason that gets involved when you build more sophisticated
>>rulesystems on top of the base machinery, but the exact reason
>>is escaping me at this moment. ... sort
>>of like doing math without being able to write "zero":
>
>I think it had to do with next_method, and was the same problem super
>has.  If you aren't sure the job is done -- no matter what you got
>mixed with -- then you have to call the next method.  99% of the time,
>it won't do anything useful, but you have to call it just in case.
>
>My favorite example is __del__(), or at least close().  I can be sure
>that I've cleaned up *my* scarce resources, but I don't know what
>subclasses might have mixed me in with.  So I call super  __del__, and
>catch the Exception when it turns out I wasn't mixed in at all.
>
>A "pass" default implementation is much nicer to subclasses than an
>Exception that they have to try...except.

Right.  If you're defining a generic function as a hook, you would just 
define the function as a no-op -- i.e. NOT an @abstract function.  (e.g., 
the "foo()" function in my previous post.)


>> > My point is, I could rename mine to @abstract so that it
>> >would serve your purpose as well -- but only if it would serve.
>
>>Assuming that:
>
>>1.  If you call such a function, it will raise some error, like
>>NotImplementedError
>
>Note that your default methods don't do this either, except as part of
>the wrapper.

I don't understand what the sentence above means.  I don't even understand 
it enough to misunderstand it.  :)


>   Guido's current implementation is that
>
>    (a)  An error is raised as soon as you try to instantiate a class
>that *could* call the method directly, even if you don't plan to ever
>*actually* call it.
>
>    (b)  The method itself is annotated, so that the rules engine
>could raise an error if it so chose.

Ah.  I'm suggesting that a function object marked as abstract should also 
raise an exception (perhaps NotImplementedError) when it's actually 
called.  @abstract could either change the function object's type, or set 
its func_code, or whatever else makes sense.


>>2.  It's a normal function object (i.e., can have its func_code
>>repointed later)
>>3.  The __isabstractmethod__ can be set back to False once there
>>is other code registered
>
>I'm not sure that actually *doing* this would play well with ABC
>assumptions, but, yes, it is possible.

Since @overload requires a function of the same name to alread exist in the 
current namespace, you'll normally be doing this:

    class Concrete(Abstract):

        def my_implementation_of_something_abstract(self, ...):
            """Default implementation"""

        @overload
        def my_implementation_of_something_abstract(self, ...):
            """This overloads *this* class' definition, not the ABC's"""

In any case, I don't really see any reason to add overloads to an ABC's 
abstract methods.  The GF usage of @abstract is really only useful for 
standalone functions, at least as far as I can see.


From pje at telecommunity.com  Mon Apr 23 23:31:31 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 17:31:31 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <f0j77k$nj5$1@sea.gmane.org>
References: <9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com>

At 11:08 PM 4/23/2007 +0200, Michele Cella wrote:
>My solution is nice (IMHO) for the simplest use case (and, probably, the
>most frequent) but it's behavior is really unclear for these complex use
>cases. Thanks for pointing that out.
>
>As far as I'm concerned, I can only confirm that having a super keyword
>with methods attached (like Aahz said) when no other keyword has (and
>there are many) doesn't feel right (to me) and can be confusing at
>first. That's unfortunate as I rarely found python confusing up to now
>(contrary to other languages).
>
>The only other option that comes to my mind is using a special attribute
>in the instance itself (like __class__, __dict__, ...):
>
>         self.__super__.mymethod(arg)
>
>I'm sure this (again) introduces other problems I simply overlooked, in
>this case, forgive me please. ;-)

One other option that's been used more and more lately are parentheses, as in:

    (super self).foo()

But I can't say I particularly like this idea, compared to "super.foo" or 
even "super(self).foo".  In fact, the latter invocation doesn't even 
require a keyword -- it just means the compiler needs to include a cell 
variable for the current class whenever it thinks you might be using super().


From daniel at stutzbachenterprises.com  Tue Apr 24 00:05:12 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 17:05:12 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
Message-ID: <eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>

On 4/23/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> > So you can see the performance of the BList in more detail, I've made
> > several performance graphs available at the following link:
> >    http://stutzbachenterprises.com/blist/
>
> Very cool.  These detailed timings are very useful.
>
> Might it be possible to include variance bars?

I'm not really sure what variance bars would reveal.  Performing
exactly the same operations follows exactly the same code path, so
there's not a lot of variance to be found.  Let me explain my
methodology for generating these figures.

Right now I'm generating the timings using timeit.py, where the setup
step is run once, then the actual operation is run enough times to
exceed 10 milliseconds.  I repeat the process 3 times and take the
lowest value (to try to factor out noise introduced by other processes
and other OS issues).  The machine I run the tests on is generally
unloaded.

Once I had an initial working implementation, I've been using theses
experiments and graphs to figure out where I needed to do optimization
work.  I can testify that when I rerun the experiments with the same
code, the results would look *very* similar.

> A few of the graphs
> show your data structure faster than the list even for small N, and it
> would be great to see if it is meaningfully faster in these cases (if
> so, perhaps there are optimization possibilities for the current list
> data structure).

Yes, in some cases the optimizations I've made could be ported back to
the current list data structure.  I will try to hunt these down and
send patches via SourceForge.

> A quite glance at your results shows a few more cases where the blist
> faces problems, such no longer using Timsort, and so suffering in
> performance for sorted/reversed cases.  That would be painful to give
> up in the built-in list implementation.

I plan on implementing Timsort for BLists; I just haven't had time yet.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From guido at python.org  Tue Apr 24 00:05:38 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 15:05:38 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
Message-ID: <ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>

On 4/22/07, Brett Cannon <brett at python.org> wrote:
> This PEP is to change the ``if __name__ == "__main__": ...`` idiom to
> ``if __name__ == sys.main: ...``  so that you at least have a chance
> to execute module in a package that use relative imports.
>
> Ran this PEP past python-ideas.  Stopped the discussion there when too
> many new ideas were being proposed.  =)  I have listed all of them in
> the Rejected Ideas section, although if overwhelming support for one
> comes forward the PEP can shift to one of them.

I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Tue Apr 24 00:08:03 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 23 Apr 2007 15:08:03 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
Message-ID: <bbaeab100704231508r26f59aefx2e580cfe24e20baa@mail.gmail.com>

On 4/23/07, Guido van Rossum <guido at python.org> wrote:
> On 4/22/07, Brett Cannon <brett at python.org> wrote:
> > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to
> > ``if __name__ == sys.main: ...``  so that you at least have a chance
> > to execute module in a package that use relative imports.
> >
> > Ran this PEP past python-ideas.  Stopped the discussion there when too
> > many new ideas were being proposed.  =)  I have listed all of them in
> > the Rejected Ideas section, although if overwhelming support for one
> > comes forward the PEP can shift to one of them.
>
> I'm -1 on this and on any other proposed twiddlings of the __main__
> machinery. The only use case seems to be running scripts that happen
> to be living inside a module's directory, which I've always seen as an
> antipattern. To make me change my mind you'd have to convince me that
> it isn't.

With Guido not liking it and people not agreeing on a solution with me
not feeling like fighting for this one I consider the PEP rejected.
When I get my laptop back (end of the week hopefully) I will check the
PEP in as rejected so that we don't have to go through this again.

Thanks to everyone to contributed to the conversation.

-Brett

From mike.klaas at gmail.com  Tue Apr 24 00:15:59 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Mon, 23 Apr 2007 15:15:59 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
	<eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
Message-ID: <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com>

On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> On 4/23/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> > Might it be possible to include variance bars?
>
> I'm not really sure what variance bars would reveal.  Performing
> exactly the same operations follows exactly the same code path, so
> there's not a lot of variance to be found.  Let me explain my
> methodology for generating these figures.

You;re right.  I was interested in whether some of the jagged graphs
were due to noise or underlying behaviour.  On closer inspection it is
clear that the jumps tend to happen near the same place on most ops
(esp the first at ~128), which makes it clear that it is underlying
behaviour.

> > A few of the graphs
> > show your data structure faster than the list even for small N, and it
> > would be great to see if it is meaningfully faster in these cases (if
> > so, perhaps there are optimization possibilities for the current list
> > data structure).
>
> Yes, in some cases the optimizations I've made could be ported back to
> the current list data structure.  I will try to hunt these down and
> send patches via SourceForge.

The init-from iterable is especially interesting: a 20% speed increase
for list comps is not to be scoffed at.

Have you analyzed the memory consumption of the two data structures?

-Mike

From guido at python.org  Tue Apr 24 00:16:02 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 15:16:02 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
Message-ID: <ca471dc20704231516t3e71ae8eod38cb59ea9751b8@mail.gmail.com>

[Guido]
> >(b) I'm proposing an @abstractmethod, which sets its argument's
> >__isabstractmethod__ attribute and then returns it, otherwise
> >unchanged. Would this be good enough for you, or do you need @abstract
> >to do more? My point is, I could rename mine to @abstract so that it
> >would serve your purpose as well -- but only if it would serve.

On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> Assuming that:
>
> 1.  If you call such a function, it will raise some error, like NotImplementedError

That would be up to the author of the function; they would have to
explicitly raise NotImplementedError in the body. The ABC proposal
allows meaningful abstract methods that can be called (only) via
"super"; the use case for this is that the abstract method might be
the one that decides which exception should be thrown (e.g.
__getitem__ and __next__ do this), or perhaps it could provide a
default implementation for certain types. (Potential example of the
latter: Sequence.__getitem__() could raise IndexError when the
argument is an Integer but handle the case where the argument is a
slice instance, assuming there's a suitable factory which could be a
designated class method.)

> 2.  It's a normal function object (i.e., can have its func_code repointed later)

Yes.

> 3.  The __isabstractmethod__ can be set back to False once there is other
> code registered

Yes.

> Then yes, it's perfect.

I guess it falls short. I'll stick with @abstractmethod, which rhymes
with classmethod and staticmethod. You can have @abstract.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Tue Apr 24 00:21:43 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 23 Apr 2007 16:21:43 -0600
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com>
Message-ID: <aac2c7cb0704231521o5ffc21ebta5cef8e6a9fa2146@mail.gmail.com>

On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> But I can't say I particularly like this idea, compared to "super.foo" or
> even "super(self).foo".  In fact, the latter invocation doesn't even
> require a keyword -- it just means the compiler needs to include a cell
> variable for the current class whenever it thinks you might be using super().

+1 on super(self).foo.  It's SomeLongClassName we want to get rid of, not self.
As a bonus, super() and super(cls) have obvious semantics.

-- 
Adam Olsen, aka Rhamphoryncus

From krstic at solarsail.hcs.harvard.edu  Tue Apr 24 00:21:06 2007
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Mon, 23 Apr 2007 18:21:06 -0400
Subject: [Python-3000] Principles
In-Reply-To: <20070423123456.63D7.JCARLSON@uci.edu>
References: <20070423102411.63CE.JCARLSON@uci.edu>	<07Apr23.121924pdt."57996"@synergy1.parc.xerox.com>
	<20070423123456.63D7.JCARLSON@uci.edu>
Message-ID: <462D3152.8050702@solarsail.hcs.harvard.edu>

Josiah Carlson wrote:
> I believe that no performance improvement warrants such large scale
> breakage of 3rd party extensions.

If there's an implementation that's deemed superior to what's there now,
then 3.0 is exactly the right time to break 3rd party extensions. Of
course 3.0 isn't a blank check for gratuitous breakage, but being
conservative *solely* for the purpose of not breaking compatibility
defeats the purpose of having a 3.0 separate from the 2.x series, IMO.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From guido at python.org  Tue Apr 24 00:38:21 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 15:38:21 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <aac2c7cb0704231521o5ffc21ebta5cef8e6a9fa2146@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com>
	<aac2c7cb0704231521o5ffc21ebta5cef8e6a9fa2146@mail.gmail.com>
Message-ID: <ca471dc20704231538q6fe19931p577cbccd750f9d64@mail.gmail.com>

On 4/23/07, Adam Olsen <rhamph at gmail.com> wrote:
> On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > But I can't say I particularly like this idea, compared to "super.foo" or
> > even "super(self).foo".  In fact, the latter invocation doesn't even
> > require a keyword -- it just means the compiler needs to include a cell
> > variable for the current class whenever it thinks you might be using super().
>
> +1 on super(self).foo.  It's SomeLongClassName we want to get rid of, not self.
> As a bonus, super() and super(cls) have obvious semantics.

At least +0 from me too.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Tue Apr 24 00:41:37 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 23 Apr 2007 16:41:37 -0600
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
Message-ID: <aac2c7cb0704231541u4860cb4bx875642b7d8ee1e67@mail.gmail.com>

On 4/23/07, Guido van Rossum <guido at python.org> wrote:
> I'm -1 on this and on any other proposed twiddlings of the __main__
> machinery. The only use case seems to be running scripts that happen
> to be living inside a module's directory, which I've always seen as an
> antipattern. To make me change my mind you'd have to convince me that
> it isn't.

So ./python Lib/test/regrtest.py and ./python
Lib/test/test_something.py are both wrong?  I note there's 26 files
marked executable and 50 with #!/usr/bin/env python, so there's quite
a bit of prior art in the "old way".

Besides correcting the stdlib, adding a warning when running a script
from a dir with __init__.py would help change the momentum.

-- 
Adam Olsen, aka Rhamphoryncus

From daniel at stutzbachenterprises.com  Tue Apr 24 00:54:00 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 17:54:00 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
	<eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
	<3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com>
Message-ID: <eae285400704231554m2e60fe31y9aaac63077d8d88a@mail.gmail.com>

On 4/23/07, Mike Klaas <mike.klaas at gmail.com> wrote:
> The init-from iterable is especially interesting: a 20% speed increase
> for list comps is not to be scoffed at.

That one is because list()'s tp_new method is PyType_GenericNew
instead of a wrapper around the more efficient PyList_New.  Probably
list comprehensions and the [] syntax already call PyList_New, but I
haven't checked.

> Have you analyzed the memory consumption of the two data structures?

I haven't done any empirical studies, but I can give you a pretty good picture.

For small lists, a BList uses a fixed amount of memory rather than
aiming for the "just large enough" for Python lists.  So, for very
small lists, BLists are currently rather wasteful.  The BList code
could be changed so that for small lists they also do the "just large
enough" thing, at the expense of a bit more code complexity.

For large lists, the *worst* case for BLists is around twice the
memory requirements of a regular Python list (each BList node is
required to be at least half-full).  I know that sounds horrible, but
hear me out. :-)  The *best* case for BLists is when each node is
completely full, in which case they use negligibly more memory than a
regular Python list.

If a user creates a BList from whole-cloth (e.g., BList(iterable)),
they will get a best-case BList.  This would probably be the common
case for user's who don't plan to modify their list much.

If a user is doing lots of inserts and deletes, the BList's node's
will typically be three-quarters full, translating into 33% extra
memory above a regular Python list.  However (and this is a big
however!), these users will greatly appreciate the BList's additional
speed for these operations.

I hasten to add that these memory estimates are only about the space
required to store the list structure as well as pointers to the
objects in the list.  The objects themselves (e.g., integers, class
instances, whatever) will take up substantially more memory in any
case.  For example, for the regular 32-bit integer type, a single
object takes up 12 bytes.  A regular python list will spend an extra 4
bytes per integer (16 bytes total).  A worst-case BList spends an
extra 8 bytes per integer (20 bytes total).  So, worst case is only a
25% increase.

If the user makes use of operations that leverage the copy-on-write
feature, then BLists are *much* more memory efficient that regular
lists.  __getslice__ is probably the most commonly used function in
this category.  A slice of a large BList uses only O(log n) memory in
the worst case.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From guido at python.org  Tue Apr 24 00:56:49 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 15:56:49 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <aac2c7cb0704231541u4860cb4bx875642b7d8ee1e67@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
	<aac2c7cb0704231541u4860cb4bx875642b7d8ee1e67@mail.gmail.com>
Message-ID: <ca471dc20704231556i1013dfdcm9b90c7734a5bbe6d@mail.gmail.com>

On 4/23/07, Adam Olsen <rhamph at gmail.com> wrote:
> On 4/23/07, Guido van Rossum <guido at python.org> wrote:
> > I'm -1 on this and on any other proposed twiddlings of the __main__
> > machinery. The only use case seems to be running scripts that happen
> > to be living inside a module's directory, which I've always seen as an
> > antipattern. To make me change my mind you'd have to convince me that
> > it isn't.
>
> So ./python Lib/test/regrtest.py and ./python
> Lib/test/test_something.py are both wrong?

I'll happily admit they are bad examples, though as long as they work
I don't care (and since they do work, I don't see them as motivating
the PEP :-).

This occasionally causes problems when someone adds a new test module
that imports stuff using implicit relative imports. Fortunately those
will be gone in py3k.

> I note there's 26 files
> marked executable and 50 with #!/usr/bin/env python, so there's quite
> a bit of prior art in the "old way".

Where are these files? If they are directly in Lib, that's not a
problem. The PEP only addresses what happens if these are in a
subpackage (like test).

> Besides correcting the stdlib, adding a warning when running a script
> from a dir with __init__.py would help change the momentum.

Patch anyone? We could move regrtest out of the test package into Lib.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Tue Apr 24 01:13:12 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 19:13:12 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704231516t3e71ae8eod38cb59ea9751b8@mail.gmail.com
 >
References: <5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>

At 03:16 PM 4/23/2007 -0700, Guido van Rossum wrote:
>On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>Assuming that:
>>
>>1.  If you call such a function, it will raise some error, like 
>>NotImplementedError
>
>That would be up to the author of the function; they would have to
>explicitly raise NotImplementedError in the body. The ABC proposal
>allows meaningful abstract methods that can be called (only) via
>"super"; the use case for this is that the abstract method might be
>the one that decides which exception should be thrown (e.g.
>__getitem__ and __next__ do this), or perhaps it could provide a
>default implementation for certain types. (Potential example of the
>latter: Sequence.__getitem__() could raise IndexError when the
>argument is an Integer but handle the case where the argument is a
>slice instance, assuming there's a suitable factory which could be a
>designated class method.)

Ah...  interesting.  This is different from what I understood to be 
"abstract" methods in other languages where an abstract method is always 
one that does not have an actual implementation.  I guess I skimmed PEP 
3119 a little too quickly.

It sounds like your proposal is to mark methods as "abstract" even if they 
have a useful "null" implementation.  I guess I don't see what this adds, 
at least for the examples in the sandbox, except for making the class 
non-instantiable as a side-effect.

Of course, if @abstract were a class decorator as well as a function 
decorator, then it could have a single meaning in both contexts: "this 
thing shouldn't be callable".

That is to say, this:

    @abstract
    class Iterator(Iterable):
        ...

would simply mean calling "Iterator()" would result in a 
NotImplementedError, just like marking a function @abstract means that 
calling it would result in a NotImplementedError.

And, as far as I can see, the only ABC method I'd mark @abstract 
individually would be Hashable.__hash__: everything else here looks to me 
like a perfectly valid "empty" implementation of the method(s) in question.

I suppose there is some value in requiring people to override @abstract 
methods to make a subclass instantiable, versus merely using the lack of an 
@abstract class decorator to indicate that a subclass is concrete.  But I 
wonder if being able to have just one @abstract decorator (that always 
means "you can't call this by default") mightn't be worth giving up that 
tiny bit of extra type checking?


From rhamph at gmail.com  Tue Apr 24 01:32:45 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 23 Apr 2007 17:32:45 -0600
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704231554m2e60fe31y9aaac63077d8d88a@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
	<eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
	<3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com>
	<eae285400704231554m2e60fe31y9aaac63077d8d88a@mail.gmail.com>
Message-ID: <aac2c7cb0704231632w28bf636fs82109341baf40692@mail.gmail.com>

On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> If a user creates a BList from whole-cloth (e.g., BList(iterable)),
> they will get a best-case BList.  This would probably be the common
> case for user's who don't plan to modify their list much.
>
> If a user is doing lots of inserts and deletes, the BList's node's
> will typically be three-quarters full, translating into 33% extra
> memory above a regular Python list.  However (and this is a big
> however!), these users will greatly appreciate the BList's additional
> speed for these operations.

For contrast, the existing list seems to over-allocate by 12.5%, for
an average of 6.25% wasted.  However, when deleting it only resizes
when it drops below half full, leaving it with a worst case of 100%,
the same as BList.

The usage patterns can make a big difference there though.  A list's
worst-case is only reached if you have a spike, then drop to just
above half-full.  A BList's worst-case is reached through random
slicing.

Daniel, does repeated getitem/setitem (as in random.shuffle())
increase memory usage, or does it have no effect?

A random.shuffle() benchmark might be a better a better demonstration
of the overall costs.

-- 
Adam Olsen, aka Rhamphoryncus

From jimjjewett at gmail.com  Tue Apr 24 01:42:21 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 19:42:21 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <f0j77k$nj5$1@sea.gmane.org>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
Message-ID: <fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>

On 4/23/07, Michele Cella <michele.cella at gmail.com> wrote:

> ... having a super keyword with methods attached ... doesn't feel right

Agreed.  The goal is to *fix* super, not to make it a keyword.  Making
it a keyword might -- or might not -- be the way to do that.  A
keyword with attributes is probably not the right answer.

For What Its Worth, the longer I think about this, the more convinced
I am that the right answer is to keep it an object.  The problem is
that it has to grow even more magical than it already is, unless we
break a lot of backwards compatibility.

If we're willing to put up with the magic, then it would work to make
super syntactic sugar for

    super(__this_class__, self)

At the moment, I can't see anything wrong with this, but I have a
feeling I'm missing something about how the super object should behave
on its own.  Is there a good way (other than "being inside a method"?)
to distinguish between:

    super.__str__()     # I want to call the super-class' str method

and

    super.__str__()     # I want to call str(super)



> The only other option that comes to my mind is using a special attribute
> in the instance itself (like __class__, __dict__, ...):

>         self.__super__.mymethod(arg)

__class__ and __dict__ are (sort of) ordinary attributes -- they point
to a specific object.  super would probably need to be an active
property.

    class A: ...
    class B1(A): ...
    class C1(B1): ...
    class B2(A): ...
    class C2(B1): ...

    class D(C1, C2): ...
    class E(D): ...

When an instance of E is created, the super call may well be made in
D, so you don't know the instance's class when compiling the code.

When methods in B1 make a super call for "normal" instances of C1,
they go straight to class A.  But if the instance is really an
instance of E, then they first have to loop back through C2 and C1.
So when compiling the class, you don't know which class will be next
in the mro.

So you can't do it from the class -- but you also can't do it only
from the instance.  For an instance of E, super(C1, self) and
super(B2, self) should go to different next-classes.

So you need both the instance's own true class, and the class where
the current method was defined -- but even that isn't quite enough.
You can't quite say "next in the instance's mro", because if the
method happens to be something that C2 doesn't override, then super(D,
self) should go straight to C1 -- but it still needs to look at C2 for
other methods.

-jJ

From guido at python.org  Tue Apr 24 01:49:52 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 16:49:52 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
Message-ID: <ca471dc20704231649r7ecb7116k8c411ddb7699185a@mail.gmail.com>

On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 03:16 PM 4/23/2007 -0700, Guido van Rossum wrote:
> >On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>Assuming that:
> >>
> >>1.  If you call such a function, it will raise some error, like
> >>NotImplementedError
> >
> >That would be up to the author of the function; they would have to
> >explicitly raise NotImplementedError in the body. The ABC proposal
> >allows meaningful abstract methods that can be called (only) via
> >"super"; the use case for this is that the abstract method might be
> >the one that decides which exception should be thrown (e.g.
> >__getitem__ and __next__ do this), or perhaps it could provide a
> >default implementation for certain types. (Potential example of the
> >latter: Sequence.__getitem__() could raise IndexError when the
> >argument is an Integer but handle the case where the argument is a
> >slice instance, assuming there's a suitable factory which could be a
> >designated class method.)
>
> Ah...  interesting.  This is different from what I understood to be
> "abstract" methods in other languages where an abstract method is always
> one that does not have an actual implementation.  I guess I skimmed PEP
> 3119 a little too quickly.
>
> It sounds like your proposal is to mark methods as "abstract" even if they
> have a useful "null" implementation.

Not exactly. Some of the proposed ABCs have concrete methods. These
are the kinds you'd find in mixin classes for Python 2. (Example:
Iterator.__iter__() is a concrete method returning self. You almost
never have a need to override it.) The abstract ones have null
implementations that aren't all that useful for two reasons: (a) they
implement an utter edge case (e.g. an empty iterator) (b) they must be
overridden. Their main purpose is to provide an example -- either they
show which exception to raise (__next__, __getitem__) or they show the
type of value to return (__hash__).

> I guess I don't see what this adds,
> at least for the examples in the sandbox, except for making the class
> non-instantiable as a side-effect.

I guess I exaggerated the usefulness of the null implementations. It's
perfectly fine not to call the null implementation but to inline it
(e.g. raise StopIteration instead of return super(self).__next__()).

> Of course, if @abstract were a class decorator as well as a function
> decorator, then it could have a single meaning in both contexts: "this
> thing shouldn't be callable".
>
> That is to say, this:
>
>     @abstract
>     class Iterator(Iterable):
>         ...
>
> would simply mean calling "Iterator()" would result in a
> NotImplementedError, just like marking a function @abstract means that
> calling it would result in a NotImplementedError.

Yeah, but it would be less specific because (in the case of ABCs that
define multiple methods) it wouldn't tell you which mwethods you have
to override.

> And, as far as I can see, the only ABC method I'd mark @abstract
> individually would be Hashable.__hash__: everything else here looks to me
> like a perfectly valid "empty" implementation of the method(s) in question.

Actually, Hashable.__hash__ is also perfectly valid. :-)

> I suppose there is some value in requiring people to override @abstract
> methods to make a subclass instantiable, versus merely using the lack of an
> @abstract class decorator to indicate that a subclass is concrete.  But I
> wonder if being able to have just one @abstract decorator (that always
> means "you can't call this by default") mightn't be worth giving up that
> tiny bit of extra type checking?

I prefer to follow the lead of C++ here -- an abstract class is
abstract by virtue of having at least one abstract method.

That the abstract methods are still somewhat useful implementations is
mostly to provide a valid (if not necessarily useful) end point for
super-calling in cooperative MI schemes.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Tue Apr 24 01:50:39 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 23 Apr 2007 19:50:39 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <aac2c7cb0704231521o5ffc21ebta5cef8e6a9fa2146@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com>
	<aac2c7cb0704231521o5ffc21ebta5cef8e6a9fa2146@mail.gmail.com>
Message-ID: <fb6fbf560704231650l753b058v5751716d31161c05@mail.gmail.com>

On 4/23/07, Adam Olsen <rhamph at gmail.com> wrote:
> On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > But I can't say I particularly like this idea, compared to "super.foo" or
> > even "super(self).foo".  In fact, the latter invocation doesn't even
> > require a keyword -- it just means the compiler needs to include a cell
> > variable for the current class whenever it thinks you might be using super().

> +1 on super(self).foo.  It's SomeLongClassName we want to get rid of, not self.

It won't normally be for an attribute.  Typically, it would look like:

    super(self).foo(arg1, arg2)

or even

    super(self).foo()

I'm not sure that is much improvement over explicitly showing the
class, though it is certainly an improvement over using the
(rebindable) class name.

> As a bonus, super() and super(cls) have obvious semantics.

What would they be?  Are you assuming that the two-argument forms
would go away?  Or that the *first* positional argument could be
defaulted to __this_class__, but the second would be mandatory?

-jJ

From rhamph at gmail.com  Tue Apr 24 02:00:42 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 23 Apr 2007 18:00:42 -0600
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <ca471dc20704231556i1013dfdcm9b90c7734a5bbe6d@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
	<aac2c7cb0704231541u4860cb4bx875642b7d8ee1e67@mail.gmail.com>
	<ca471dc20704231556i1013dfdcm9b90c7734a5bbe6d@mail.gmail.com>
Message-ID: <aac2c7cb0704231700g1393c333m13e27c65c5250802@mail.gmail.com>

On 4/23/07, Guido van Rossum <guido at python.org> wrote:
> On 4/23/07, Adam Olsen <rhamph at gmail.com> wrote:
> > I note there's 26 files
> > marked executable and 50 with #!/usr/bin/env python, so there's quite
> > a bit of prior art in the "old way".
>
> Where are these files? If they are directly in Lib, that's not a
> problem. The PEP only addresses what happens if these are in a
> subpackage (like test).

$ ls -l Lib/test/*.py | grep -c '^...x'
26
$ head -qn 1 Lib/test/*.py | grep -c /usr/bin/env
50


> > Besides correcting the stdlib, adding a warning when running a script
> > from a dir with __init__.py would help change the momentum.
>
> Patch anyone? We could move regrtest out of the test package into Lib.

-- 
Adam Olsen, aka Rhamphoryncus

From daniel at stutzbachenterprises.com  Tue Apr 24 02:37:12 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Mon, 23 Apr 2007 19:37:12 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <aac2c7cb0704231632w28bf636fs82109341baf40692@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
	<eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
	<3d2ce8cb0704231515k3c75246eud35c5c5a3a2fb8f4@mail.gmail.com>
	<eae285400704231554m2e60fe31y9aaac63077d8d88a@mail.gmail.com>
	<aac2c7cb0704231632w28bf636fs82109341baf40692@mail.gmail.com>
Message-ID: <eae285400704231737q3426c305m9fc9ee3caaa1e3f8@mail.gmail.com>

On 4/23/07, Adam Olsen <rhamph at gmail.com> wrote:
> The usage patterns can make a big difference there though.  A list's
> worst-case is only reached if you have a spike, then drop to just
> above half-full.  A BList's worst-case is reached through random
> slicing.
>
> Daniel, does repeated getitem/setitem (as in random.shuffle())
> increase memory usage, or does it have no effect?

getitem and setitem have no effect on the BList's internal structure
(and thus no effect on memory usage).  Only operations that add or
remove elements do.

> A random.shuffle() benchmark might be a better a better demonstration
> of the overall costs.

Here you go: http://stutzbachenterprises.com/fig/shuffle.html

The regular list's has an advantage due to the special casing of x[k]
within the Python bytecode interpreter.  After that, the O(log n) cost
of BList's item is visible in the right-hand graph via the step at
n=128.  The next step would be at n=16384.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From rhamph at gmail.com  Tue Apr 24 02:41:44 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 23 Apr 2007 18:41:44 -0600
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <fb6fbf560704231650l753b058v5751716d31161c05@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<5.1.1.6.0.20070423172551.04c4e5f0@sparrow.telecommunity.com>
	<aac2c7cb0704231521o5ffc21ebta5cef8e6a9fa2146@mail.gmail.com>
	<fb6fbf560704231650l753b058v5751716d31161c05@mail.gmail.com>
Message-ID: <aac2c7cb0704231741j10ddde63n7dc44ac6bc5f3cea@mail.gmail.com>

On 4/23/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/23/07, Adam Olsen <rhamph at gmail.com> wrote:
> > On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > > But I can't say I particularly like this idea, compared to "super.foo" or
> > > even "super(self).foo".  In fact, the latter invocation doesn't even
> > > require a keyword -- it just means the compiler needs to include a cell
> > > variable for the current class whenever it thinks you might be using super().
>
> > +1 on super(self).foo.  It's SomeLongClassName we want to get rid of, not self.
>
> It won't normally be for an attribute.  Typically, it would look like:
>
>     super(self).foo(arg1, arg2)
>
> or even
>
>     super(self).foo()
>
> I'm not sure that is much improvement over explicitly showing the
> class, though it is certainly an improvement over using the
> (rebindable) class name.
>
> > As a bonus, super() and super(cls) have obvious semantics.
>
> What would they be?  Are you assuming that the two-argument forms
> would go away?  Or that the *first* positional argument could be
> defaulted to __this_class__, but the second would be mandatory?

Err.. yes?  Actually I assumed that there'd be both the old
two-argument forms and the new implicit __this_class__ form, but in
hindsight that's obvious ambiguous.

Although I don't understand the purpose of the old super(type) option,
so maybe that can be deleted to remove the ambiguity.

-- 
Adam Olsen, aka Rhamphoryncus

From collinw at gmail.com  Tue Apr 24 03:03:31 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 23 Apr 2007 18:03:31 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
Message-ID: <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com>

On 4/23/07, Jim Jewett <jimjjewett at gmail.com> wrote:
[snip]
> If we're willing to put up with the magic, then it would work to make
> super syntactic sugar for
>
>     super(__this_class__, self)
>
> At the moment, I can't see anything wrong with this, but I have a
> feeling I'm missing something about how the super object should behave
> on its own.

You mean, have "super.method_name(*args)" expanded by the AST compiler
into super(__this_class__, self).method_name(*args)"? The object
reference -> function call change is too much magic; -1.

However, having the AST compiler expand

class A:
  def m(self):
    blah()
    super(self).m()
    blah()

to

class A:
  def m(self):
    super = super_factory(A)
    blah()
    super(self).m()
    blah()

strikes me as an "appropriate" amount of magic. Restricting the magic
to a source transformation should enable other Python implementations
to implement this relatively easily.

Collin Winter

From pje at telecommunity.com  Tue Apr 24 03:09:53 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 23 Apr 2007 21:09:53 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704231649r7ecb7116k8c411ddb7699185a@mail.gmail.co
 m>
References: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>

At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote:
>>I suppose there is some value in requiring people to override @abstract
>>methods to make a subclass instantiable, versus merely using the lack of an
>>@abstract class decorator to indicate that a subclass is concrete.  But I
>>wonder if being able to have just one @abstract decorator (that always
>>means "you can't call this by default") mightn't be worth giving up that
>>tiny bit of extra type checking?
>
>I prefer to follow the lead of C++ here -- an abstract class is
>abstract by virtue of having at least one abstract method.

It's been way too long since I did any C++, but isn't it the case that an 
abstract (aka "pure virtual"?) method in C++ is one that *can't be 
invoked*?  [pause to Google it...]  Well I'll be darned.  I didn't know you 
could actually provide an implementation for a pure virtual 
function.  Hm.  Learn something new every day...  except while I was 
writing C++ evidently.  :)

But are there any other languages besides C++ that have this idiom?  I'm 
not familiar with any that do (at least, not that I know of!), and the fact 
that C++ allows it seems a bit non-obvious to me.  Java and C# pretty much 
do @abstract the way I proposed it, for example:

Java: http://java.sun.com/docs/books/tutorial/java/IandI/abstract.html
C#: http://www.codeproject.com/useritems/Abstract_CLS_MTHD.asp

That is, they both separate class-abstraction from method-abstraction, even 
though the latter implies the former.  Thus, you can have an abstract class 
'Iterator' even if all its methods are non-abstract.  (However, if any of 
its methods are abstract, the class itself is required to be abstract.)

As someone with more recent background in Java than C++, I find the idea of 
abstract methods having an executable implementation to be quite confusing, 
and suspect that other people with that Java or C# background will do the 
same thing.  That is, skim the explanation and miss the significant 
difference between the C++ way and what they're used to.


>That the abstract methods are still somewhat useful implementations is
>mostly to provide a valid (if not necessarily useful) end point for
>super-calling in cooperative MI schemes.

Right; I guess my point is that if those "somewhat" useful implementations 
are useful, they're useful, and there's no need to treat the corresponding 
class as "abstract" (in the "non-instantiable" sense) in that case.  For 
example, I could pass an Iterable() to something that expected an iterable.


From jcarlson at uci.edu  Tue Apr 24 03:16:47 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 18:16:47 -0700
Subject: [Python-3000] Principles
In-Reply-To: <462D3152.8050702@solarsail.hcs.harvard.edu>
References: <20070423123456.63D7.JCARLSON@uci.edu>
	<462D3152.8050702@solarsail.hcs.harvard.edu>
Message-ID: <20070423174934.63E0.JCARLSON@uci.edu>


Ivan Krstic <krstic at solarsail.hcs.harvard.edu> wrote:
> Josiah Carlson wrote:
> > I believe that no performance improvement warrants such large scale
> > breakage of 3rd party extensions.
> 
> If there's an implementation that's deemed superior to what's there now,
> then 3.0 is exactly the right time to break 3rd party extensions. Of
> course 3.0 isn't a blank check for gratuitous breakage, but being
> conservative *solely* for the purpose of not breaking compatibility
> defeats the purpose of having a 3.0 separate from the 2.x series, IMO.

The question is whether or not the breakage would be "gratuitous".

The only use of lists that wouldn't be broken is if people relied solely
on the use of PyList_* functions and not lst->ob_item[] references.

A quick google code search offers about 5-10 3rd party examples against
40 varaints of the Python C source.  The lack of results seems a bit
fishy, but maybe my search for  \-\>ob_item\[ was too restrictive.
Searching for \-\>ob_item gets more results, some of which are things
like PyLucene, guppy, etc.  This also seems a bit slim (Google doesn't
seem to index all projects on sourceforge).


 - Josiah


From jcarlson at uci.edu  Tue Apr 24 05:19:40 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 23 Apr 2007 20:19:40 -0700
Subject: [Python-3000] PEP to change how the main module is delineated
In-Reply-To: <ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
References: <bbaeab100704221845i18417626o8dc1289fa7f9b685@mail.gmail.com>
	<ca471dc20704231505w2ae34a0dwb6a2227f1df52831@mail.gmail.com>
Message-ID: <20070423195834.63E9.JCARLSON@uci.edu>


"Guido van Rossum" <guido at python.org> wrote:
> 
> On 4/22/07, Brett Cannon <brett at python.org> wrote:
> > This PEP is to change the ``if __name__ == "__main__": ...`` idiom to
> > ``if __name__ == sys.main: ...``  so that you at least have a chance
> > to execute module in a package that use relative imports.
> >
> > Ran this PEP past python-ideas.  Stopped the discussion there when too
> > many new ideas were being proposed.  =)  I have listed all of them in
> > the Rejected Ideas section, although if overwhelming support for one
> > comes forward the PEP can shift to one of them.
> 
> I'm -1 on this and on any other proposed twiddlings of the __main__
> machinery. The only use case seems to be running scripts that happen
> to be living inside a module's directory, which I've always seen as an
> antipattern. To make me change my mind you'd have to convince me that
> it isn't.

"Guido's Decision" on PEP 328 offers a module/package hierarchy where
'from ..subpackage2.moduleZ import eggs' is offered as a valid use-case. 
It does not specify the name of subpackage1/moduleX.py (that is
performing the import), but there is the somewhat hidden implication
that if it were executed directly from the command line, it would not be
allowed (but this is only revealed in understanding the implications of
__name__ == '__main__' with regards to the PEP).

Personally, I would much prefer a change in __name__ == sys.main than
having to hack relative imports to offer __name__s like
__main__..subpackage.moduleZ, or even 'import
sys;sys.path.insert(1, os.path.basename(os.path.basename(__file__)))',
the latter of which I already do in every script in a codebase laid out
like...

    /.../packageX/
        __init__.py
        application1/
            __init__.py
            script1.py
            ...
            tests/
                ...
        application2/
            __init__.py
            script5.py
            ...
            tests/
                ...
        ...
        shared/
            __init__.py
            shared1.py
            ...

The preamble to properly deal with error conditions in the current path
became tiresome to place in every application path after the first few. 
I can't believe that the above is an antipattern; seems to be the
simplest way to update and distribute a group of related applications
with a simple 'svn co packageX' or 'svn up packageX'.


 - Josiah


From ironfroggy at gmail.com  Tue Apr 24 06:11:01 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Tue, 24 Apr 2007 00:11:01 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
Message-ID: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>

On 4/23/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/23/07, Michele Cella <michele.cella at gmail.com> wrote:
>
> > ... having a super keyword with methods attached ... doesn't feel right
>
> Agreed.  The goal is to *fix* super, not to make it a keyword.  Making
> it a keyword might -- or might not -- be the way to do that.  A
> keyword with attributes is probably not the right answer.
>
> For What Its Worth, the longer I think about this, the more convinced
> I am that the right answer is to keep it an object.  The problem is
> that it has to grow even more magical than it already is, unless we
> break a lot of backwards compatibility.
>
> If we're willing to put up with the magic, then it would work to make
> super syntactic sugar for
>
>     super(__this_class__, self)
>
> At the moment, I can't see anything wrong with this, but I have a
> feeling I'm missing something about how the super object should behave
> on its own.  Is there a good way (other than "being inside a method"?)
> to distinguish between:
>
>     super.__str__()     # I want to call the super-class' str method
>
> and
>
>     super.__str__()     # I want to call str(super)
>
>
>
> > The only other option that comes to my mind is using a special attribute
> > in the instance itself (like __class__, __dict__, ...):
>
> >         self.__super__.mymethod(arg)
>
> __class__ and __dict__ are (sort of) ordinary attributes -- they point
> to a specific object.  super would probably need to be an active
> property.
>
>     class A: ...
>     class B1(A): ...
>     class C1(B1): ...
>     class B2(A): ...
>     class C2(B1): ...
>
>     class D(C1, C2): ...
>     class E(D): ...
>
> When an instance of E is created, the super call may well be made in
> D, so you don't know the instance's class when compiling the code.
>
> When methods in B1 make a super call for "normal" instances of C1,
> they go straight to class A.  But if the instance is really an
> instance of E, then they first have to loop back through C2 and C1.
> So when compiling the class, you don't know which class will be next
> in the mro.
>
> So you can't do it from the class -- but you also can't do it only
> from the instance.  For an instance of E, super(C1, self) and
> super(B2, self) should go to different next-classes.
>
> So you need both the instance's own true class, and the class where
> the current method was defined -- but even that isn't quite enough.
> You can't quite say "next in the instance's mro", because if the
> method happens to be something that C2 doesn't override, then super(D,
> self) should go straight to C1 -- but it still needs to look at C2 for
> other methods.
>
> -jJ
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com
>

I will +1 on the self.__super__ suggestion. Hey, its very doable and I
even whipped it up with a simple metaclass, so it would be a tiny
change to 'type' in order to actually implement it as a standard
feature. The demonstration is as follows:

class _superdesc(object):
    def __get__(self, obj, objcls):
        return super(cls, obj)

class autosuper(type):
    def __init__(cls, name, bases, clsdict):
        cls.__super__ = _superdesc()

class A(object):
    __metaclass__ = autosuper
    x = 1

class B(A):
    x = 2

assert B().__super__.x == 1

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From steven.bethard at gmail.com  Tue Apr 24 06:22:44 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 23 Apr 2007 22:22:44 -0600
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
Message-ID: <d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>

On 4/23/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> I will +1 on the self.__super__ suggestion. Hey, its very doable and I
> even whipped it up with a simple metaclass, so it would be a tiny
> change to 'type' in order to actually implement it as a standard
> feature. The demonstration is as follows:
>
> class _superdesc(object):
>     def __get__(self, obj, objcls):
>         return super(cls, obj)
>
> class autosuper(type):
>     def __init__(cls, name, bases, clsdict):
>         cls.__super__ = _superdesc()
>
> class A(object):
>     __metaclass__ = autosuper
>     x = 1
>
> class B(A):
>     x = 2
>
> assert B().__super__.x == 1

Does that really work?  There's a typo in _superdesc (I don't know
where 'cls' comes from) but if you meant 'objcls', here's what I get::

    >>> class _superdesc(object):
    ...     def __get__(self, obj, cls):
    ...         return super(cls, obj)
    ...
    >>> class autosuper(type):
    ...     def __init__(cls, name, bases, clsdict):
    ...         cls.__super__ = _superdesc()
    ...
    >>> class A:
    ...     __metaclass__ = autosuper
    ...     def f(self):
    ...         print 'A'
    ...
    >>> class B(A):
    ...     def f(self):
    ...         print 'B'
    ...         self.__super__.f()
    ...
    >>> class C(A):
    ...     def f(self):
    ...         print 'C'
    ...         self.__super__.f()
    ...
    >>> class D(B, C):
    ...     def f(self):
    ...         print 'D'
    ...         self.__super__.f()
    ...
    >>> D().f()
    D
    B
    B
    B
    ...
    Traceback (most recent call last):
      File "<interactive input>", line 1, in <module>
    ...
    RuntimeError: maximum recursion depth exceeded

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Tue Apr 24 06:25:01 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 23 Apr 2007 21:25:01 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
Message-ID: <ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>

On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote:
> >>I suppose there is some value in requiring people to override @abstract
> >>methods to make a subclass instantiable, versus merely using the lack of an
> >>@abstract class decorator to indicate that a subclass is concrete.  But I
> >>wonder if being able to have just one @abstract decorator (that always
> >>means "you can't call this by default") mightn't be worth giving up that
> >>tiny bit of extra type checking?
> >
> >I prefer to follow the lead of C++ here -- an abstract class is
> >abstract by virtue of having at least one abstract method.
>
> It's been way too long since I did any C++, but isn't it the case that an
> abstract (aka "pure virtual"?) method in C++ is one that *can't be
> invoked*?  [pause to Google it...]  Well I'll be darned.  I didn't know you
> could actually provide an implementation for a pure virtual
> function.  Hm.  Learn something new every day...  except while I was
> writing C++ evidently.  :)

Actually I just meant to follow its lead in defining that a class is
abstract as soon as one or more of its methods are abstract. In Java
you need to repeat this by making the class abstract; that seems
unpythonic.

> But are there any other languages besides C++ that have this idiom?  I'm
> not familiar with any that do (at least, not that I know of!), and the fact
> that C++ allows it seems a bit non-obvious to me.  Java and C# pretty much
> do @abstract the way I proposed it, for example:
>
> Java: http://java.sun.com/docs/books/tutorial/java/IandI/abstract.html
> C#: http://www.codeproject.com/useritems/Abstract_CLS_MTHD.asp
>
> That is, they both separate class-abstraction from method-abstraction, even
> though the latter implies the former.  Thus, you can have an abstract class
> 'Iterator' even if all its methods are non-abstract.  (However, if any of
> its methods are abstract, the class itself is required to be abstract.)

I don't see the point of having an abstract class without abstract methods.

The main reasons I came up (without knowing C++ allows this too!) with
the idea of giving abstract methods a valid implementation wer (a)
there's got to be *something* in the body; (b) to provide an end point
for cooperative-MI-style code.

> As someone with more recent background in Java than C++, I find the idea of
> abstract methods having an executable implementation to be quite confusing,
> and suspect that other people with that Java or C# background will do the
> same thing.  That is, skim the explanation and miss the significant
> difference between the C++ way and what they're used to.

Well, too bad. After a day of coding in Java I start typing curly
braces and semicolons too. But that doesn't mean Python should adopt
these.

> >That the abstract methods are still somewhat useful implementations is
> >mostly to provide a valid (if not necessarily useful) end point for
> >super-calling in cooperative MI schemes.
>
> Right; I guess my point is that if those "somewhat" useful implementations
> are useful, they're useful, and there's no need to treat the corresponding
> class as "abstract" (in the "non-instantiable" sense) in that case.  For
> example, I could pass an Iterable() to something that expected an iterable.

Perhaps. Though I think it would defeat the purpose of being required
to provide an implementation. If forced to choose, I'd rather remove
the meaningful bodies (and replace them with "raise
NotImplementedError") than remove the @abstractmethod decorators. But
I prefer the current proposal.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Tue Apr 24 06:25:09 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 23 Apr 2007 21:25:09 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <ee2a432c0704232125qc0f4230m8eff32b9219826b5@mail.gmail.com>

On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
>
> The Radical Proposal
> -------------------------------
>
> Replace list() with the BList.

I looked over this patch and have various questions/comments.  In
rough priority order:

I'm concerned about the moderately heavy use of macros wrt to debugging.

I noticed the use of alloca().  This isn't used in the core anywhere
(except a few uses on Windows).  That might be an issue.  I didn't
check how it was used.

I noticed that calculating the height was recursive.  Not sure where
this function was called from, but does it handle blists which contain
themselves?

Have you tried running this code under regrtest.py with the -R flag to
check for memory leaks?

There were C++ (//) style comments and C99 declarations of variables.
These would have to be changed if included in the core.  Also, size_t
and Py_ssize_t should be used throughout.  (ssize_t should be
Py_ssize_t).  I noticed a bunch of ints and unsigneds.  Although I
also noticed you used PyIndex_Check().

In general I thought the code was pretty good.  Had a decent amount of
error checking/handling.  There were some cases I wasn't sure, but in
the paths that I traced, I didn't find any obvious problems.

n

From tom at vector-seven.com  Tue Apr 24 03:34:32 2007
From: tom at vector-seven.com (Thomas Lee)
Date: Tue, 24 Apr 2007 11:34:32 +1000
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>	<f0d5qi$2fu$1@sea.gmane.org>
	<462BDCAA.7020501@canterbury.ac.nz>	<f0hkrs$aro$1@sea.gmane.org>	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>	<f0j77k$nj5$1@sea.gmane.org>	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com>
Message-ID: <462D5EA8.4070902@vector-seven.com>

Oops, here's the (broken) patch.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: super_python-3000.patch
Type: text/x-patch
Size: 1401 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070424/4120a433/attachment.bin 

From tom at vector-seven.com  Tue Apr 24 03:32:59 2007
From: tom at vector-seven.com (Thomas Lee)
Date: Tue, 24 Apr 2007 11:32:59 +1000
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<fb6fbf560704181503t73cef73bm212981f5b3703e75@mail.gmail.com>	<9e804ac0704200456x77496e60ta6f3fdc465259503@mail.gmail.com>	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>	<f0d5qi$2fu$1@sea.gmane.org>
	<462BDCAA.7020501@canterbury.ac.nz>	<f0hkrs$aro$1@sea.gmane.org>	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>	<f0j77k$nj5$1@sea.gmane.org>	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<43aa6ff70704231803r7eff65f0j823814422b5a1ab9@mail.gmail.com>
Message-ID: <462D5E4B.2000604@vector-seven.com>

Collin Winter wrote:
> However, having the AST compiler expand
>
> class A:
>   def m(self):
>     blah()
>     super(self).m()
>     blah()
>
> to
>
> class A:
>   def m(self):
>     super = super_factory(A)
>     blah()
>     super(self).m()
>     blah()
>   

I'm sold on the super(self) syntax and whacked together a quick & dirty 
patch. I made my changes in Objects/super.c, rather than the AST as you 
suggest. Determining __this_class__ ('A' in your example) without 
touching the AST is obviously going to be problematic, but even *with* 
the AST how would you propose we determine the "current" class?

I've attached my patch irrespective. All relevant changes are within 
super_init.

Forgive the ignorance, I'm relatively new to this stuff :)

Cheers,
Tom

From talin at acm.org  Tue Apr 24 08:17:44 2007
From: talin at acm.org (Talin)
Date: Mon, 23 Apr 2007 23:17:44 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
Message-ID: <462DA108.8090307@acm.org>

Guido van Rossum wrote:
> On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>> At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote:
>>>> I suppose there is some value in requiring people to override @abstract
>>>> methods to make a subclass instantiable, versus merely using the lack of an
>>>> @abstract class decorator to indicate that a subclass is concrete.  But I
>>>> wonder if being able to have just one @abstract decorator (that always
>>>> means "you can't call this by default") mightn't be worth giving up that
>>>> tiny bit of extra type checking?
>>> I prefer to follow the lead of C++ here -- an abstract class is
>>> abstract by virtue of having at least one abstract method.
>> It's been way too long since I did any C++, but isn't it the case that an
>> abstract (aka "pure virtual"?) method in C++ is one that *can't be
>> invoked*?  [pause to Google it...]  Well I'll be darned.  I didn't know you
>> could actually provide an implementation for a pure virtual
>> function.  Hm.  Learn something new every day...  except while I was
>> writing C++ evidently.  :)

I'm not sure what Phillip is talking about here - C++ does not allow you 
to instantiate a class that has non-overridden pure virtual methods, and 
no pure virtual method can have a body. You cannot provide an 
implementation for a pure virtual function.

Now, there are occasions in C++ where it is possible for buggy code to 
"call" a pure virtual function - but the result is always an exception.

Recall that in C++, when an instance is destructed, as each base class's 
destructor is called the vtable pointer is modified to point to the 
vtable for that base class, not the vtable for the actual class of the 
instance. (I've always had problems with this rule, but there it is.)

If you attempt to call a subclass method after that subclass's 
destructor has been called (which is, of course, an error), you may end 
up invoking the little stub function which a pure virtual vtable entry 
points to. This is about as "legal" as dereferencing memory after it has 
been freed.

> Actually I just meant to follow its lead in defining that a class is
> abstract as soon as one or more of its methods are abstract. In Java
> you need to repeat this by making the class abstract; that seems
> unpythonic.
> 
>> But are there any other languages besides C++ that have this idiom?  I'm
>> not familiar with any that do (at least, not that I know of!), and the fact
>> that C++ allows it seems a bit non-obvious to me.  Java and C# pretty much
>> do @abstract the way I proposed it, for example:
>>
>> Java: http://java.sun.com/docs/books/tutorial/java/IandI/abstract.html
>> C#: http://www.codeproject.com/useritems/Abstract_CLS_MTHD.asp
>>
>> That is, they both separate class-abstraction from method-abstraction, even
>> though the latter implies the former.  Thus, you can have an abstract class
>> 'Iterator' even if all its methods are non-abstract.  (However, if any of
>> its methods are abstract, the class itself is required to be abstract.)
> 
> I don't see the point of having an abstract class without abstract methods.
> 
> The main reasons I came up (without knowing C++ allows this too!) with
> the idea of giving abstract methods a valid implementation wer (a)
> there's got to be *something* in the body; (b) to provide an end point
> for cooperative-MI-style code.
> 
>> As someone with more recent background in Java than C++, I find the idea of
>> abstract methods having an executable implementation to be quite confusing,
>> and suspect that other people with that Java or C# background will do the
>> same thing.  That is, skim the explanation and miss the significant
>> difference between the C++ way and what they're used to.
> 
> Well, too bad. After a day of coding in Java I start typing curly
> braces and semicolons too. But that doesn't mean Python should adopt
> these.

I think I am in (weak) agreement with Phillip here. In every language 
that I am familiar with, the adjective "abstract" when applied to a 
method implies two things: (1) that the method does not have a body, and 
(2) the method must be implemented by a subclass if that subclass is 
able to be instantiated. In fact, (2) is merely a consequence of (1), 
when combined with the additional constraint that no instantiated class 
can have methods with no body.

However, I think what is going on is that you're using the word 
"abstract" in a slightly difference sense. Normally, an abstract method 
provides merely an interface, in other words it defines the method name, 
argument types, and return type of a method - and nothing more.

What you're doing, I think, is saying that an abstract method provides 
all those things, and in addition it provides a kind of sketch or 
outline of what the functionality ought to be. I don't have a problem 
with that part so much.

But in addition there is a sense that you also want these methods to be 
useful helper functions for base classes to call, but you don't want 
them to be called directly. In languages such as Java and C++, this is 
what "protected" is for - methods that aren't available to the outside 
world, but which can be called by subclasses.

So it seems to me -- at least from the standpoint of someone steeped in 
the mindset of Java / C++ / C# et al -- that you are conflating 
"abstract" and "protected".

In what is considered "good style" for such languages, protected helper 
methods ought to have different names than the methods which invoke 
them. So for example, you might have the following:

    /// Abstract base class for hashable object
    class Hashable {
    public:
       /// Declare abstract hash method
       virtual int GetHash() const = 0;

    protected:
       /// Protected helper method
       int DefaultHashFunc() { return 0; }
    };

    // A concrete class derived from hashable
    class MyConcreteClass : public Hashable {
    public:
       /// Declare abstract hash method
       virtual int GetHash() const { return DefaultHashFunc(); }
    };

Granted, in this particular case the decision to provide a helper 
function is rather odd. Why go through the trouble of defining 
DefaultHash() in the base class that simply returns 0, when any 
non-trivial subclass will provide a real implementation of GetHash() 
instead of merely forwarding the call to DefaultHash()?

But the solution isn't to give Hashable::GetHash() a body. From a C++ 
programmer's view, there's only two choices: Either Hashable::GetHash() 
is abstract, or it's not. If it's abstract, then every subclass is 
required to re-implement it, and giving it a body has no purpose 
whatsoever; If it's not, then subclasses are allowed to default to its 
implementation.

>>> That the abstract methods are still somewhat useful implementations is
>>> mostly to provide a valid (if not necessarily useful) end point for
>>> super-calling in cooperative MI schemes.
>> Right; I guess my point is that if those "somewhat" useful implementations
>> are useful, they're useful, and there's no need to treat the corresponding
>> class as "abstract" (in the "non-instantiable" sense) in that case.  For
>> example, I could pass an Iterable() to something that expected an iterable.
> 
> Perhaps. Though I think it would defeat the purpose of being required
> to provide an implementation. If forced to choose, I'd rather remove
> the meaningful bodies (and replace them with "raise
> NotImplementedError") than remove the @abstractmethod decorators. But
> I prefer the current proposal.

-- Talin

From daniel at stutzbachenterprises.com  Tue Apr 24 13:10:03 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 24 Apr 2007 06:10:03 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <ee2a432c0704232125qc0f4230m8eff32b9219826b5@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<ee2a432c0704232125qc0f4230m8eff32b9219826b5@mail.gmail.com>
Message-ID: <eae285400704240410u17b2b924j3e0c925c5d94bf43@mail.gmail.com>

On 4/23/07, Neal Norwitz <nnorwitz at gmail.com> wrote:
> On 4/23/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> > Replace list() with the BList.
>
> I looked over this patch and have various questions/comments.  In
> rough priority order:

Thanks for taking the time to look over the code.  I greatly appreciate it.

> I'm concerned about the moderately heavy use of macros wrt to debugging.

Yes, some of the debugging code is a little hairy and needs a clean-up.

I originally had written a prototype for BLists in Python and used
function decorators to indicate which invariants the function must
maintain.  This was immensely helpful during development.  C doesn't
have function decorators, so I went with some very hairy macros.
Perhaps it would be better to go with something more like this:

PyObject *some_function(PyObject *self, some_other_args)
{
    PyObject *ret;

    check_invariants(self);

    // do some stuff
    if (some_condition) {
        ret = case1;
        goto done;
    }

    // do some more stuff

 done:
    check_return_invariants(self, VALID_RW, VALID_PARENT)
    return ret;
}

> I noticed the use of alloca().  This isn't used in the core anywhere
> (except a few uses on Windows).  That might be an issue.  I didn't
> check how it was used.

These were the result of converting code that was doing malloc/free's
to be more efficient.  They could all be converted to ordinary
variable declarations now.

> I noticed that calculating the height was recursive.  Not sure where
> this function was called from, but does it handle blists which contain
> themselves?

Yes.  If the ->leaf flag is true, the BList node contains user objects
which will not be recursed on (ever if they are also BList nodes).  If
the ->flag is false, the BList node contains other BList nodes that
are part of the same data structure.

Also, FWIW, the maximum height (and therefore recursion depth) of a
BList on a 32-bit machine is 16 (assuming the maximum children per
node is 128, as I have used).

> Have you tried running this code under regrtest.py with the -R flag to
> check for memory leaks?

I hadn't heard of regrtest.py, so I re-invented the wheel.

I ran the code through the list and sequence unit tests, as well as
several additional unit tests that I wrote.  For each test, I setup
things up so that when compiled in debug mode it runs the test, counts
the total number of references in the interpreter, runs the test
again, and confirms that the total number of references have not
changed.

> There were C++ (//) style comments and C99 declarations of variables.
> These would have to be changed if included in the core.  Also, size_t
> and Py_ssize_t should be used throughout.  (ssize_t should be
> Py_ssize_t).  I noticed a bunch of ints and unsigneds.  Although I
> also noticed you used PyIndex_Check().

Is there a document that describes the right times to use int vs
size_t vs Py_ssize_t?

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From pje at telecommunity.com  Tue Apr 24 13:17:22 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 24 Apr 2007 07:17:22 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <462DA108.8090307@acm.org>
References: <ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070424071405.04effb30@sparrow.telecommunity.com>

At 11:17 PM 4/23/2007 -0700, Talin wrote:
>I'm not sure what Phillip is talking about here - C++ does not allow you 
>to instantiate a class that has non-overridden pure virtual methods, and 
>no pure virtual method can have a body. You cannot provide an 
>implementation for a pure virtual function.

I've never done it myself, but at least these guys say you can:

http://www.parashift.com/c++-faq-lite/abcs.html#faq-22.4
http://en.wikipedia.org/wiki/Virtual_function#Abstract_classes_and_pure_virtual_functions

But clearly, it's pretty obscure.


From pje at telecommunity.com  Tue Apr 24 13:51:47 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 24 Apr 2007 07:51:47 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.co
 m>
References: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>

At 09:25 PM 4/23/2007 -0700, Guido van Rossum wrote:
>On 4/23/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 04:49 PM 4/23/2007 -0700, Guido van Rossum wrote:
> > >>I suppose there is some value in requiring people to override @abstract
> > >>methods to make a subclass instantiable, versus merely using the lack 
> of an
> > >>@abstract class decorator to indicate that a subclass is concrete.  But I
> > >>wonder if being able to have just one @abstract decorator (that always
> > >>means "you can't call this by default") mightn't be worth giving up that
> > >>tiny bit of extra type checking?
> > >
> > >I prefer to follow the lead of C++ here -- an abstract class is
> > >abstract by virtue of having at least one abstract method.
> >
> > It's been way too long since I did any C++, but isn't it the case that an
> > abstract (aka "pure virtual"?) method in C++ is one that *can't be
> > invoked*?  [pause to Google it...]  Well I'll be darned.  I didn't know you
> > could actually provide an implementation for a pure virtual
> > function.  Hm.  Learn something new every day...  except while I was
> > writing C++ evidently.  :)
>
>Actually I just meant to follow its lead in defining that a class is
>abstract as soon as one or more of its methods are abstract. In Java
>you need to repeat this by making the class abstract; that seems
>unpythonic.

Right - but as I understand it, Java and C# allow a class to be abstract 
without having *any* of its methods being abstract.  I wasn't proposing 
that a redudnant class level @abstract be required; just noting that it 
seems useful to declare a class @abstract regardless of whether it has any 
abstract methods.


>I don't see the point of having an abstract class without abstract methods.

Well, by my definition of "abstract", none of the ABCs in the sandbox 
*have* any abstract methods (i.e. methods without a usable implementation), 
so apparently you *do* see the point.  We just have different definitions 
of "abstract".  :)

Mine (courtesy of other languages) is, "a method without an 
implementation".  Yours appears to be "a method that must be 
overridden".  An abstract class, to me, is simply one that is not 
instantiable.  Is that your definition as well?

(By the way, I'm not continuing this thread to push for my definitions -- 
I'm seeking clarity so that your PEP can better explain just what it is 
that you mean to do.  E.g. "While in other languages, abstract methods are 
X, Python abstract methods are Y".  I'm clear now about what the *behavior* 
of Y is, but am trying to understand the *reasons* of Y.)


>The main reasons I came up (without knowing C++ allows this too!) with
>the idea of giving abstract methods a valid implementation wer (a)
>there's got to be *something* in the body; (b) to provide an end point
>for cooperative-MI-style code.

But "b", as far as I can tell, doesn't have anything to do with 
abstractness as I understand it.  If there's a useful end-point, why does 
it need to be abstract?  What do we gain by making someone override it?  It 
seems like making somebody jump through a Java-style typechecking hoop for 
no purpose.  ISTM that "consenting adults" means I should be able to just 
use that code without overriding it, as long as there is something there to 
use.


>Well, too bad. After a day of coding in Java I start typing curly
>braces and semicolons too. But that doesn't mean Python should adopt
>these.

Of course not -- but those differences are something you're not likely to 
miss by skimming the docs!  ;)  My point here is to make sure that this 
gets called out clearly and boldly, since it is a Python-only concept that 
nonetheless is using a term that means something completely different in 
other languages.  We might, perhaps, consider an alternative term for it 
than 'abstractmethod', since it is quite different from what anybody else 
means by "abstract method", as far as I can tell.


> > >That the abstract methods are still somewhat useful implementations is
> > >mostly to provide a valid (if not necessarily useful) end point for
> > >super-calling in cooperative MI schemes.
> >
> > Right; I guess my point is that if those "somewhat" useful implementations
> > are useful, they're useful, and there's no need to treat the corresponding
> > class as "abstract" (in the "non-instantiable" sense) in that case.  For
> > example, I could pass an Iterable() to something that expected an iterable.
>
>Perhaps. Though I think it would defeat the purpose of being required
>to provide an implementation.

And what *is* the purpose of being required to provide an implementation, 
if a useful one exists?  This is the bit I guess I'm still missing.


>If forced to choose, I'd rather remove
>the meaningful bodies (and replace them with "raise
>NotImplementedError") than remove the @abstractmethod decorators. But
>I prefer the current proposal.

Right; so let's work on making the rationale for it a bit clearer.  (i.e., 
I've already stopped arguing to change the behavior -- I just want to 
*understand* it better.)


From ironfroggy at gmail.com  Tue Apr 24 16:11:21 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Tue, 24 Apr 2007 10:11:21 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<ca471dc20704201053k251fe818ged13404c233f4b81@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
Message-ID: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>

I must have miscopied then because it worked perfectly here. Yes, I
meant to have the _superdesc defined inside the metaclass __init__,
but thought I could pull it out to make it cleaner. I forgot it
actually had to be there! Here is the metaclass that works.

class autosuper(type):
	def __init__(cls, name, bases, clsdict):
		class _superdesc(object):
			def __get__(self, obj, objcls):
				return super(cls, obj)
		cls.__super__ = _superdesc()

On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/23/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > I will +1 on the self.__super__ suggestion. Hey, its very doable and I
> > even whipped it up with a simple metaclass, so it would be a tiny
> > change to 'type' in order to actually implement it as a standard
> > feature. The demonstration is as follows:
> >
> > class _superdesc(object):
> >     def __get__(self, obj, objcls):
> >         return super(cls, obj)
> >
> > class autosuper(type):
> >     def __init__(cls, name, bases, clsdict):
> >         cls.__super__ = _superdesc()
> >
> > class A(object):
> >     __metaclass__ = autosuper
> >     x = 1
> >
> > class B(A):
> >     x = 2
> >
> > assert B().__super__.x == 1
>
> Does that really work?  There's a typo in _superdesc (I don't know
> where 'cls' comes from) but if you meant 'objcls', here's what I get::
>
>     >>> class _superdesc(object):
>     ...     def __get__(self, obj, cls):
>     ...         return super(cls, obj)
>     ...
>     >>> class autosuper(type):
>     ...     def __init__(cls, name, bases, clsdict):
>     ...         cls.__super__ = _superdesc()
>     ...
>     >>> class A:
>     ...     __metaclass__ = autosuper
>     ...     def f(self):
>     ...         print 'A'
>     ...
>     >>> class B(A):
>     ...     def f(self):
>     ...         print 'B'
>     ...         self.__super__.f()
>     ...
>     >>> class C(A):
>     ...     def f(self):
>     ...         print 'C'
>     ...         self.__super__.f()
>     ...
>     >>> class D(B, C):
>     ...     def f(self):
>     ...         print 'D'
>     ...         self.__super__.f()
>     ...
>     >>> D().f()
>     D
>     B
>     B
>     B
>     ...
>     Traceback (most recent call last):
>       File "<interactive input>", line 1, in <module>
>     ...
>     RuntimeError: maximum recursion depth exceeded
>
> STeVe
> --
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>         --- Bucky Katt, Get Fuzzy
>


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From emin.shopper at gmail.com  Tue Apr 24 16:39:45 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Tue, 24 Apr 2007 10:39:45 -0400
Subject: [Python-3000] An introduction to ABC's
Message-ID: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com>

Dear Guido, Talin, and other Python Experts,

I would like to register my strong support/encouragement for some type of
abstract base class (ABC) mechanism in python. While I think dynamism,
reflection, duck-typing, unit tests, etc. are wonderful, I often come across
situations where ABCs are very useful.

For example, when writing programs which take a long time to run (e.g.,
numerical work or large simulations), it is very painful to run a unit test
that takes many minutes or even hours to run only to discover that a newly
defined class is missing a crucial method. By using ABCs appropriately, I
can make sure that all classes have the required methods as soon as a class
is imported instead of part-way through the simulation.

Below are links to the ABC implementation based on meta-classes and
decorators which I have been using for quite a while.

http://mail.python.org/pipermail/python-list/2006-December/419941.html
http://alum.mit.edu/www/emin/source_code/py_abc/abc.py

Some features of this module include:

  * Allows you to declare an ABC by inheriting from AbstractBaseClass.
  * Allows you to declare abstract methods using the @Abstract decorator.
  * Works with multiple inheritance (i.e., can inherit from multiple ABCs).
  * Allows you to turn off ABC checks on a per-class basis by setting
klass.__allow_abstract__=True.
  * Allows partial ABCs (i.e., an ABC which inherits from another ABC but is
also abstract itself)
  * Relatively simple implementation (only a few hundred lines of pure
python including comments and doctests).
  * Documentation and doctests to illustrate/test features.

Sincerely,
-Emin Martinian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070424/d538d5be/attachment.html 

From guido at python.org  Tue Apr 24 18:52:26 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Apr 2007 09:52:26 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <462DA108.8090307@acm.org>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462DA108.8090307@acm.org>
Message-ID: <ca471dc20704240952v84fbdddwd907630ec7b12fe5@mail.gmail.com>

On 4/23/07, Talin <talin at acm.org> wrote:
> I think I am in (weak) agreement with Phillip here. In every language
> that I am familiar with, the adjective "abstract" when applied to a
> method implies two things: (1) that the method does not have a body, and
> (2) the method must be implemented by a subclass if that subclass is
> able to be instantiated. In fact, (2) is merely a consequence of (1),
> when combined with the additional constraint that no instantiated class
> can have methods with no body.
>
> However, I think what is going on is that you're using the word
> "abstract" in a slightly difference sense. Normally, an abstract method
> provides merely an interface, in other words it defines the method name,
> argument types, and return type of a method - and nothing more.
>
> What you're doing, I think, is saying that an abstract method provides
> all those things, and in addition it provides a kind of sketch or
> outline of what the functionality ought to be. I don't have a problem
> with that part so much.

Right. That's was the original goal.

> But in addition there is a sense that you also want these methods to be
> useful helper functions for base classes to call, but you don't want
> them to be called directly. In languages such as Java and C++, this is
> what "protected" is for - methods that aren't available to the outside
> world, but which can be called by subclasses.

But Python doesn't have protected, and isn't about to get it.

> So it seems to me -- at least from the standpoint of someone steeped in
> the mindset of Java / C++ / C# et al -- that you are conflating
> "abstract" and "protected".
>
> In what is considered "good style" for such languages, protected helper
> methods ought to have different names than the methods which invoke
> them.

Unfortunately then the ABC would need to invent names for those helpers.

> So for example, you might have the following:
>
>     /// Abstract base class for hashable object
>     class Hashable {
>     public:
>        /// Declare abstract hash method
>        virtual int GetHash() const = 0;
>
>     protected:
>        /// Protected helper method
>        int DefaultHashFunc() { return 0; }
>     };
>
>     // A concrete class derived from hashable
>     class MyConcreteClass : public Hashable {
>     public:
>        /// Declare abstract hash method
>        virtual int GetHash() const { return DefaultHashFunc(); }
>     };
>
> Granted, in this particular case the decision to provide a helper
> function is rather odd. Why go through the trouble of defining
> DefaultHash() in the base class that simply returns 0, when any
> non-trivial subclass will provide a real implementation of GetHash()
> instead of merely forwarding the call to DefaultHash()?

Hashable.__hash__ is probably not the best example of the idea; its
implementation is valid but useless, so it really serves more as a
"sketch or outline of the functionality".

> But the solution isn't to give Hashable::GetHash() a body. From a C++
> programmer's view, there's only two choices: Either Hashable::GetHash()
> is abstract, or it's not. If it's abstract, then every subclass is
> required to re-implement it, and giving it a body has no purpose
> whatsoever; If it's not, then subclasses are allowed to default to its
> implementation.

I'm not afraid of breaking new ground. Since the fact of the matter is
that we can't prevent abstract methods from having a body (that would
require syntactic support for abstract methods which is out of scope
here) we might as well make good use of the body.

There's also the use case of providing an end-point for cooperative
super-calls, which I've brought up repeatedly but which nobody seems
to pick up. Perhaps because cooperative super-calls are not supported
by either Java or C++. But they do exist in other languages (Lisp, and
the C++ extension described in "Putting Metaclasses to Work: A New
Dimension in Object-Oriented Programming", by Ira R. Forman and Scott
H. Danforth (which formed the basis of much of my thinking on
metaclasses, and which I reference in "Unifying types and classes in
Python 2.2": http://www.python.org/download/releases/2.2.3/descrintro/).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From daniel at stutzbachenterprises.com  Tue Apr 24 19:18:15 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Tue, 24 Apr 2007 12:18:15 -0500
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704240952v84fbdddwd907630ec7b12fe5@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462DA108.8090307@acm.org>
	<ca471dc20704240952v84fbdddwd907630ec7b12fe5@mail.gmail.com>
Message-ID: <eae285400704241018k1ca32e0btb7f639e1a51078f6@mail.gmail.com>

On 4/24/07, Guido van Rossum <guido at python.org> wrote:
> I'm not afraid of breaking new ground. Since the fact of the matter is
> that we can't prevent abstract methods from having a body (that would
> require syntactic support for abstract methods which is out of scope
> here) we might as well make good use of the body.

Sure, we could.

class SomeABC:
    some_abstract_method = AbstractMethod

where "AbstractMethod" is a singleton.  This is similar to C++'s "= 0"
syntax for pure virtual functions.

Unfortunately, that would leave no place for docstrings, so I like the
existing proposal better (@abstract).

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From jimjjewett at gmail.com  Tue Apr 24 19:35:08 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 24 Apr 2007 13:35:08 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <462DA108.8090307@acm.org>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462DA108.8090307@acm.org>
Message-ID: <fb6fbf560704241035n6913c5cdn14ed816582af11cd@mail.gmail.com>

On 4/24/07, Talin <talin at acm.org> wrote:
> In what is considered "good style" for such languages, protected helper
> methods ought to have different names than the methods which invoke
> them. So for example, you might have the following:

>     /// Abstract base class for hashable object
>     class Hashable {
>     public:
>        /// Declare abstract hash method
>        virtual int GetHash() const = 0;
>
>     protected:
>        /// Protected helper method
>        int DefaultHashFunc() { return 0; }
>     };

This is exactly why diamond inheritance is tricky.  Try this with a
cooperative method, such as save_state.

    def save_state(self):
        # Save my own state, then...

        # Do call the helper method if no one else will...
        # Do not call the helper method in case someone else does...

The solution is to avoid making the "top" of the hierarchy a
singleton.  Calling it (through super) is OK.  It might even be
helpful, or it might be a no-op -- but it is safe to call through
inheritance.

The reason to mark it @abstract is to say that the implementation
isn't complete yet.  For hashable, you may never call super at all;
for save_state, you should always call it.

-jJ

From steven.bethard at gmail.com  Tue Apr 24 19:35:51 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 24 Apr 2007 11:35:51 -0600
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<f0d5qi$2fu$1@sea.gmane.org> <462BDCAA.7020501@canterbury.ac.nz>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
Message-ID: <d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>

On 4/24/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> I must have miscopied then because it worked perfectly here. Yes, I
> meant to have the _superdesc defined inside the metaclass __init__,
> but thought I could pull it out to make it cleaner. I forgot it
> actually had to be there! Here is the metaclass that works.
>
> class autosuper(type):
>         def __init__(cls, name, bases, clsdict):
>                 class _superdesc(object):
>                         def __get__(self, obj, objcls):
>                                 return super(cls, obj)
>                 cls.__super__ = _superdesc()

I still get the same error. Take a look at what the 'self' and
'__super__' objects are at each level::

>>> class A:
...     __metaclass__ = autosuper
...     def f(self):
...         print 'A', self
...
>>> class B(A):
...     def f(self):
...         print 'B', self, self.__super__
...         self.__super__.f()
...
>>> class C(A):
...     def f(self):
...         print 'C', self, self.__super__
...         self.__super__.f()
...
>>> class D(B, C):
...     def f(self):
...         print 'D', self, self.__super__
...         self.__super__.f()
...
>>> D().f()
D <__main__.D object at 0x00E88530> <super: <class 'D'>, <D object>>
B <__main__.D object at 0x00E88530> <super: <class 'D'>, <D object>>
B <__main__.D object at 0x00E88530> <super: <class 'D'>, <D object>>
...
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
  File "<interactive input>", line 4, in f
  File "<interactive input>", line 4, in f
  ...
RuntimeError: maximum recursion depth exceeded
>>>

Notice that the 'self' object is always the 'D()' object, and
therefore the '__super__' object is always the 'B' wrapper.  If the
'self' object were first 'D()', then 'D().__super__', then
'D().__super__.__super__', etc. your code might work::

>>> D().__super__
<super: <class 'D'>, <D object>>
>>> D().__super__.__super__
<super: <class 'B'>, <D object>>
>>> D().__super__.__super__.__super__
<super: <class 'C'>, <D object>>
>>> D().__super__.__super__.__super__.__super__
<super: <class 'A'>, <D object>>


STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From brett at python.org  Tue Apr 24 19:40:11 2007
From: brett at python.org (Brett Cannon)
Date: Tue, 24 Apr 2007 10:40:11 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704240410u17b2b924j3e0c925c5d94bf43@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<ee2a432c0704232125qc0f4230m8eff32b9219826b5@mail.gmail.com>
	<eae285400704240410u17b2b924j3e0c925c5d94bf43@mail.gmail.com>
Message-ID: <bbaeab100704241040h18bc7f9ds71b1ee960eae2006@mail.gmail.com>

On 4/24/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
[SNIP]
> Is there a document that describes the right times to use int vs
> size_t vs Py_ssize_t?

Try PEP 353: http://www.python.org/dev/peps/pep-0353/

-Brett

From guido at python.org  Tue Apr 24 19:50:16 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Apr 2007 10:50:16 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
Message-ID: <ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com>

On 4/24/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> Right - but as I understand it, Java and C# allow a class to be abstract
> without having *any* of its methods being abstract.  I wasn't proposing
> that a redudnant class level @abstract be required; just noting that it
> seems useful to declare a class @abstract regardless of whether it has any
> abstract methods.

OK. It's a minor feature, but I have a use case: I've seen code that says

  raise Exception("some error message")

and that's really a mistake -- if they are to lazy to define their own
exception subclass they should raise RuntimeError() or another
appropriate standard error. So if we had this feature I'd make
BaseException, Exception, StandardError, EnvironmentError and Warning
abstract (or at least the first 2-3 of those).

> >I don't see the point of having an abstract class without abstract methods.
>
> Well, by my definition of "abstract", none of the ABCs in the sandbox
> *have* any abstract methods (i.e. methods without a usable implementation),
> so apparently you *do* see the point.  We just have different definitions
> of "abstract".  :)

In my definition they are abstract if they have an @abstractmethod decorator. :)

> Mine (courtesy of other languages) is, "a method without an
> implementation".  Yours appears to be "a method that must be
> overridden".

That's a very subtle distinction, since the *goal* in other languages
is certainly to force it to be overridden.

> An abstract class, to me, is simply one that is not
> instantiable.  Is that your definition as well?

As the PEP stands today it's a class that has at least one abstract
method (that isn't overridden).

> (By the way, I'm not continuing this thread to push for my definitions --
> I'm seeking clarity so that your PEP can better explain just what it is
> that you mean to do.  E.g. "While in other languages, abstract methods are
> X, Python abstract methods are Y".  I'm clear now about what the *behavior*
> of Y is, but am trying to understand the *reasons* of Y.)

Frankly, I don't care all that much about the reasons -- you can read
into it what you want. I am just trying to define mechanisms that I
think are useful.

> >The main reasons I came up (without knowing C++ allows this too!) with
> >the idea of giving abstract methods a valid implementation wer (a)
> >there's got to be *something* in the body; (b) to provide an end point
> >for cooperative-MI-style code.
>
> But "b", as far as I can tell, doesn't have anything to do with
> abstractness as I understand it.  If there's a useful end-point, why does
> it need to be abstract?

It doesn't have to be.  Being a useful end-point and being abstract
are separate concepts.  They are intertwined only because IMO
abstract-ness shouldn't prevent something from being a useful
end-point.  The cooperative-MI theory makes a big distinction between
defining and overriding of methods; abstract methods (if not by
definition then by convention) are defining new methods, not
overriding them, so that pretty much requires them to be useful
end-points -- but being an end-point doesn't mean they can't be
abstract.

> What do we gain by making someone override it?  It
> seems like making somebody jump through a Java-style typechecking hoop for
> no purpose.  ISTM that "consenting adults" means I should be able to just
> use that code without overriding it, as long as there is something there to
> use.

The only practical concrete use I see of these abstract classes would
be to have a handy "empty sequence/set/mapping/iterator/etc."
implementation. Since an empty list/set/dict/etc. serves just as well
for that purpose I don't think this is a strong use case.

What I'm *trying* to do with the @abstractmethod decorator is telling
potential implementors of these classes: the minimum set of operations
you need to provide in order to be an X is this one (or these two or
three) methods (the ones that have the @abstractmethod decorator).

> >Well, too bad. After a day of coding in Java I start typing curly
> >braces and semicolons too. But that doesn't mean Python should adopt
> >these.
>
> Of course not -- but those differences are something you're not likely to
> miss by skimming the docs!  ;)  My point here is to make sure that this
> gets called out clearly and boldly, since it is a Python-only concept that
> nonetheless is using a term that means something completely different in
> other languages.  We might, perhaps, consider an alternative term for it
> than 'abstractmethod', since it is quite different from what anybody else
> means by "abstract method", as far as I can tell.

I don't expect confusion (except from folks who tend to over-analyze
things), since the practical consequences are pretty much the same.
You could happily ignore the fact that some abstract methods are
callable (via super). Remember Python makes a business out of taking
concepts and terms from other languages and giving them new meaning.
E.g. assignment -> name binding, classes and imports as run-time
features, etc. Python's MI is another big one.

> And what *is* the purpose of being required to provide an implementation,
> if a useful one exists?  This is the bit I guess I'm still missing.

What is the point of claiming to be a Sequence *without* overriding __getitem__?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Tue Apr 24 20:00:18 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 24 Apr 2007 14:00:18 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
Message-ID: <fb6fbf560704241100o47805b88iae5ff49a4c1ffdc9@mail.gmail.com>

On 4/24/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:25 PM 4/23/2007 -0700, Guido van Rossum wrote:

> >I don't see the point of having an abstract class without abstract methods.

Phillip is used to writing classes with (unbased) mutally recursive
functions.  Concrete example:

    class Eq(object):
        def __eq__(self, other): return not self.__ne__(other)
        def __ne__(self, other): return not self.__eq__(other)

There isn't a single method that needs to be overridden, so he doesn't
want to mark either method as abstract.  You need to override one of
them, so he would like to mark the class abstract.

> Mine (courtesy of other languages) is, "a method without an
> implementation".  Yours appears to be "a method that must be
> overridden".

+ " or extended"

I still call it overriding, but Guido sometimes distinguishes the two
-- and methods which must be extended are important cooperative super
calls.


> >...  (b) to provide an end point for cooperative-MI-style code.

> But "b", as far as I can tell, doesn't have anything to do with
> abstractness as I understand it.  If there's a useful end-point, why does
> it need to be abstract?

Usefor for cooperative-MI-style might mean "useful to the programmer,
who doesn't need to add boilerplate catching the exception just in
case his class was the last *concrete* class"

Though in this case, I think he is also referring to partial implementations.

Look at Sequence.__getitem__ in

http://svn.python.org/view/sandbox/trunk/abc/abc.py?rev=54894&view=markup

An implementation does need to override the method, but the override
could just be an extension that handles only integer indices, and
defers slice indices.

-jJ

From jimjjewett at gmail.com  Tue Apr 24 20:22:27 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 24 Apr 2007 14:22:27 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<462BDCAA.7020501@canterbury.ac.nz> <f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
Message-ID: <fb6fbf560704241122g449ac316j34e92ee1c0293469@mail.gmail.com>

On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:

> > class autosuper(type):

> I still get the same error.

No doubt it is fiddly, but he did remind me to look at some of the
older recipes.

In the stdlib test\test_descr.py (inside function metaclass), there is
an autosuper class that comes pretty close to what we want.

I'm not sure we can just add it to type, though, because it relies on
name mangling.
    self.__super.meth()

I think for default behavior, we want either either no underscores, or
matched underscores.

    # too magical?
    self.super.meth()
    self.super().meth()

    # Should a double_underscore method ever "normally" need to be called
    # by name, outside of another double_underscore method?
    self.__super__ .meth()

    # __attribute__ rather than method
    __this_class__.super.meth()

-jJ

From pje at telecommunity.com  Tue Apr 24 20:35:10 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 24 Apr 2007 14:35:10 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com
 >
References: <5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com>

At 10:50 AM 4/24/2007 -0700, Guido van Rossum wrote:
>On 4/24/07, Phillip J. Eby <pje at telecommunity.com> wrote:
>>Mine (courtesy of other languages) is, "a method without an
>>implementation".  Yours appears to be "a method that must be
>>overridden".
>
>That's a very subtle distinction, since the *goal* in other languages
>is certainly to force it to be overridden.

But this just goes right back around to the original point...  if you 
*have* an implementation, you don't really *need* to force it to be 
overridden.  Or do you?

As far as I can tell, you still haven't said what the point of *forcing* 
somebody to override perfectly usable methods is.  You've merely said you 
don't see the point of inheriting or using them directly, but that's not 
the same thing as a rationale for *forcing* them not to.


>Being a useful end-point and being abstract
>are separate concepts.  They are intertwined only because IMO
>abstract-ness shouldn't prevent something from being a useful
>end-point.

But that's the very definition of "abstract" -- i.e., that it's *not* useful.


>The only practical concrete use I see of these abstract classes would
>be to have a handy "empty sequence/set/mapping/iterator/etc."
>implementation. Since an empty list/set/dict/etc. serves just as well
>for that purpose I don't think this is a strong use case.

That's certainly true for the Python-supplied ABCs, but it's not so for 
many other use cases.  I frequently write base classes that have various 
default implementations, combined with a few methods that have to be 
overridden.  On the other hand, specifying which methods


>What I'm *trying* to do with the @abstractmethod decorator is telling
>potential implementors of these classes: the minimum set of operations
>you need to provide in order to be an X is this one (or these two or
>three) methods (the ones that have the @abstractmethod decorator).

Then let's call it @required or @requiredmethod, and all is well.  :)  I've 
just been trying to get clarity on your intent, so I could suggest a name 
that reflects that intent.  At this point, it seems clear your intent is to 
say, "this method is required in a subclass, even if there's an 
implementation here".  So let's have a name that reflects that intention.

(I still don't see the point of marking a method as required if it has an 
actual implementation, since to my way of thinking there is no harm in 
someone *not* overriding it.  They simply don't get as rich of a behavior 
in that case.  But that's a separate issue.)


>>And what *is* the purpose of being required to provide an implementation,
>>if a useful one exists?  This is the bit I guess I'm still missing.
>
>What is the point of claiming to be a Sequence *without* overriding 
>__getitem__?

I didn't say there was a point to that.  However, as Jim Jewett pointed 
out, you can have mutually recursive default implementations of a protocol, 
and probably at least *one* of the methods needs to be overridden.

Really, the patterns of what needs to be overridden in subclasses is 
complex enough that we don't really want to get into a bunch of ways to 
specify how those things should be overridden.  That's why I don't see the 
"make sure you override everything" testing as being all that useful; it 
seems too much like adding empty methods and extra "try" blocks in Java 
just to get your code past the compiler.


From lcaamano at gmail.com  Tue Apr 24 20:56:21 2007
From: lcaamano at gmail.com (Luis P Caamano)
Date: Tue, 24 Apr 2007 14:56:21 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
Message-ID: <c56e219d0704241156n57077d22k9c815b4ae25ff86@mail.gmail.com>

On 4/24/07, "Guido van Rossum" <guido at python.org> wrote:
>
>
>
> But Python doesn't have protected, and isn't about to get it.
>

We use methods names with one underscore as a convention for protected
methods and I think we learned that from somebody else:

class Eggs:

  def thePublicMethod(self):
      things = self._myProtected()
      return [ "thing_%s" % str(thing) for thing in things]

  def _myProtected(self):
      raise RuntimeError("missing base method")

class RealEggs(AbstractEggs):
  def _myProtected(self):
      return ["one", "two"]


-- 
Luis P Caamano
Atlanta, GA USA

From guido at python.org  Tue Apr 24 21:00:59 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Apr 2007 12:00:59 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com>
Message-ID: <ca471dc20704241200n74a5209ex5bd96dba3b8e72a7@mail.gmail.com>

Let's agree to disagree. I am getting tired from going around in
circles about this.

On 4/24/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:50 AM 4/24/2007 -0700, Guido van Rossum wrote:
> >On 4/24/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> >>Mine (courtesy of other languages) is, "a method without an
> >>implementation".  Yours appears to be "a method that must be
> >>overridden".
> >
> >That's a very subtle distinction, since the *goal* in other languages
> >is certainly to force it to be overridden.
>
> But this just goes right back around to the original point...  if you
> *have* an implementation, you don't really *need* to force it to be
> overridden.  Or do you?
>
> As far as I can tell, you still haven't said what the point of *forcing*
> somebody to override perfectly usable methods is.  You've merely said you
> don't see the point of inheriting or using them directly, but that's not
> the same thing as a rationale for *forcing* them not to.
>
>
> >Being a useful end-point and being abstract
> >are separate concepts.  They are intertwined only because IMO
> >abstract-ness shouldn't prevent something from being a useful
> >end-point.
>
> But that's the very definition of "abstract" -- i.e., that it's *not* useful.
>
>
> >The only practical concrete use I see of these abstract classes would
> >be to have a handy "empty sequence/set/mapping/iterator/etc."
> >implementation. Since an empty list/set/dict/etc. serves just as well
> >for that purpose I don't think this is a strong use case.
>
> That's certainly true for the Python-supplied ABCs, but it's not so for
> many other use cases.  I frequently write base classes that have various
> default implementations, combined with a few methods that have to be
> overridden.  On the other hand, specifying which methods
>
>
> >What I'm *trying* to do with the @abstractmethod decorator is telling
> >potential implementors of these classes: the minimum set of operations
> >you need to provide in order to be an X is this one (or these two or
> >three) methods (the ones that have the @abstractmethod decorator).
>
> Then let's call it @required or @requiredmethod, and all is well.  :)  I've
> just been trying to get clarity on your intent, so I could suggest a name
> that reflects that intent.  At this point, it seems clear your intent is to
> say, "this method is required in a subclass, even if there's an
> implementation here".  So let's have a name that reflects that intention.
>
> (I still don't see the point of marking a method as required if it has an
> actual implementation, since to my way of thinking there is no harm in
> someone *not* overriding it.  They simply don't get as rich of a behavior
> in that case.  But that's a separate issue.)
>
>
> >>And what *is* the purpose of being required to provide an implementation,
> >>if a useful one exists?  This is the bit I guess I'm still missing.
> >
> >What is the point of claiming to be a Sequence *without* overriding
> >__getitem__?
>
> I didn't say there was a point to that.  However, as Jim Jewett pointed
> out, you can have mutually recursive default implementations of a protocol,
> and probably at least *one* of the methods needs to be overridden.
>
> Really, the patterns of what needs to be overridden in subclasses is
> complex enough that we don't really want to get into a bunch of ways to
> specify how those things should be overridden.  That's why I don't see the
> "make sure you override everything" testing as being all that useful; it
> seems too much like adding empty methods and extra "try" blocks in Java
> just to get your code past the compiler.
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Tue Apr 24 21:54:22 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 24 Apr 2007 15:54:22 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704241200n74a5209ex5bd96dba3b8e72a7@mail.gmail.co
 m>
References: <5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424141334.02ed6d00@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070424154919.02d9b5d0@sparrow.telecommunity.com>

At 12:00 PM 4/24/2007 -0700, Guido van Rossum wrote:
>Let's agree to disagree. I am getting tired from going around in
>circles about this.

Does that mean you're -1 on renaming @abstractmethod to @required or 
@requiredmethod, or did you not see that proposal?

Other than that question, I'm done on this also.  For that matter, I'll 
assume a non-reply means -1, so there's no need to reply if you did in fact 
see it.


From janssen at parc.com  Wed Apr 25 01:30:37 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 24 Apr 2007 16:30:37 PDT
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com> 
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com>
Message-ID: <07Apr24.163038pdt."57996"@synergy1.parc.xerox.com>

Just to chime in...

My preference would be to have "abstract" just mark classes that
define an API, that define the "abstraction" of that particular
interface.  Not carry over meanings from Java and C++ that "abstract"
somehow always means "partial" or "incomplete".  I'd envision having
"abstract" classes that are in fact also useful and complete
implementations.  "Abstract" methods don't really make sense (to me)
in Python.  This sounds very close to what Guido has already.

Maybe "abstraction" is a better keyword than "abstract".

Maybe it makes sense to have "abstraction" classes, and an
"@ignorethismethod" decorator to mark methods that are *not* part of
the abstraction, instead of "@abstractmethod" decorators to mark the
methods that *are* part of the abstraction.

Bill

From guido at python.org  Wed Apr 25 02:01:10 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Apr 2007 17:01:10 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <3003523447303727199@unknownmsgid>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com>
	<3003523447303727199@unknownmsgid>
Message-ID: <ca471dc20704241701x612da573hdeffe08beb0762ef@mail.gmail.com>

On 4/24/07, Bill Janssen <janssen at parc.com> wrote:
> Just to chime in...
>
> My preference would be to have "abstract" just mark classes that
> define an API, that define the "abstraction" of that particular
> interface.  Not carry over meanings from Java and C++ that "abstract"
> somehow always means "partial" or "incomplete".  I'd envision having
> "abstract" classes that are in fact also useful and complete
> implementations.  "Abstract" methods don't really make sense (to me)
> in Python.  This sounds very close to what Guido has already.
>
> Maybe "abstraction" is a better keyword than "abstract".
>
> Maybe it makes sense to have "abstraction" classes, and an
> "@ignorethismethod" decorator to mark methods that are *not* part of
> the abstraction, instead of "@abstractmethod" decorators to mark the
> methods that *are* part of the abstraction.

Hm, but sometimes characteristic methods are abstract, and in other
cases they are concrete. For exampe:

class Iterator(Iterable):
  @abstract
  def __next__(self):
    raise StopIteration
  def __iter__(self):  # This is abstract in Iterable, but concrete here
    return self

Both methods are part of the ABC. But only __next__ is abstract. Once
could define a perfectly valid empty iterator like this:

class EmptyIterator(Iterator):
  def __next__(self):
    return super(EmptyIterator, self).__next__()

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Apr 25 02:23:51 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 24 Apr 2007 17:23:51 PDT
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704241701x612da573hdeffe08beb0762ef@mail.gmail.com> 
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com>
	<3003523447303727199@unknownmsgid>
	<ca471dc20704241701x612da573hdeffe08beb0762ef@mail.gmail.com>
Message-ID: <07Apr24.172353pdt."57996"@synergy1.parc.xerox.com>

> > Maybe "abstraction" is a better keyword than "abstract".
> >
> > Maybe it makes sense to have "abstraction" classes, and an
> > "@ignorethismethod" decorator to mark methods that are *not* part of
> > the abstraction, instead of "@abstractmethod" decorators to mark the
> > methods that *are* part of the abstraction.
> 
> class Iterator(Iterable):
>   @abstract
>   def __next__(self):
>     raise StopIteration
>   def __iter__(self):  # This is abstract in Iterable, but concrete here
>     return self
> 
> Both methods are part of the ABC. But only __next__ is abstract. Once
> could define a perfectly valid empty iterator like this:

If I'm following you correctly, you are saying that "Iterator" is an
abstraction class, but the only method which is part of the
abstraction is "__next__".

So it could be marked up as

@abstraction
class Iterator(Iterable):
  @abstractionmethod
  def __next__(self):
    raise StopIteration
  def __iter__(self):  # This is abstract in Iterable, but concrete here
    return self

or

@abstraction
class Iterator(Iterable):
  def __next__(self):
    raise StopIteration
  @ignorethismethod
  def __iter__(self):  # This is abstract in Iterable, but concrete here
    return self

"Iterator" inherits "__iter__" from "Iterable", so I think I prefer
the idea of explicitly marking the methods which form part of the
abstraction (the first form), because it gives you a chance to define
new methods with the same name:

@abstraction
class Iterator(Iterable):
  @abstractionmethod
  def __next__(self):
    raise StopIteration
  @abstractionmethod
  def __iter__(self, n):  # This is *not* the __iter__ from Iterable, but a new method
    return self * n

Bill

From greg.ewing at canterbury.ac.nz  Wed Apr 25 03:05:07 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Apr 2007 13:05:07 +1200
Subject: [Python-3000] Two proposals for a new list-like type:
 one	modest, one radical
In-Reply-To: <eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<3d2ce8cb0704231142n4448fd0apb3eb7f3dd350b5a3@mail.gmail.com>
	<eae285400704231505j4904850ci678064daac17eeb6@mail.gmail.com>
Message-ID: <462EA943.1020403@canterbury.ac.nz>

Daniel Stutzbach wrote:

> I'm not really sure what variance bars would reveal.

>  I can testify that when I rerun the experiments with the same
> code, the results would look *very* similar.

If you're running real-world timing tests, then there
will be some variance in the results, so you need to
plot error bars.

If the variance is indeed small, as you claim, then
the error bars will show that.

--
Greg

From tcdelaney at optusnet.com.au  Wed Apr 25 03:13:05 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Wed, 25 Apr 2007 11:13:05 +1000
Subject: [Python-3000] Fixing super anyone?
Message-ID: <003501c786d6$e222e710$0201a8c0@ryoko>

I've been off sick from work for over a week - I've been following this 
discussion, but now I think I've got something which matches all the 
criteria we've been discussing, so I've changed my subscription address 
temporarily to my home one.

It involves bytecode hacking for now, as well as a metaclass. The code is 
attached. It's based partially off my 'self.super' recipe, but doesn't need 
to use sys._getframe().

Recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286195

Basically, the metaclass modifies the bytecode to include some setup code to 
any function that uses 'super'. The setup code generates the correct 'super' 
object, and stores it as a local variable. All uses of LOAD_GLOBAL(super) 
are changed to LOAD_FAST(super).

The one thing I would like to add is something I've got in my 'self.super' 
recipe - I've got my own 'super' object that is callable. Calling it invokes 
the super method of the current method i.e. if you've got the following 
code:

    class A(autosuper):
        def f(self):
            pass

    class B(A):
        def f(self):
            super()

it would be functionally identical to the following:

    class B(A):
        def f(self):
            super.f()

This could be easily implemented by having the 'super' constructor take a 
'name' parameter (which would be passed the name of current method in the 
setup code). Then when super() is called, it would perform the name lookup.

Tim Delaney 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: autosuper.py
Type: application/octet-stream
Size: 5330 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070425/6b9124d7/attachment-0001.obj 

From tcdelaney at optusnet.com.au  Wed Apr 25 03:20:54 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Wed, 25 Apr 2007 11:20:54 +1000
Subject: [Python-3000] Fixing super anyone?
References: <003501c786d6$e222e710$0201a8c0@ryoko>
Message-ID: <004701c786d7$f8f22d60$0201a8c0@ryoko>

From: "Tim Delaney" <tcdelaney at optusnet.com.au>

> I've been off sick from work for over a week - I've been following this
> discussion, but now I think I've got something which matches all the
> criteria we've been discussing, so I've changed my subscription address
> temporarily to my home one.
>
> It involves bytecode hacking for now, as well as a metaclass. The code is
> attached. It's based partially off my 'self.super' recipe, but doesn't 
> need
> to use sys._getframe().

Oh - some output ...

>>> class autosuper(object):
...     __metaclass__ = _autosuper
...
>>> class A(autosuper):
...     def f(self):
...         print 'A:', super
...
>>> class B(A):
...     def f(self):
...         print 'B:', super
...         super.f()
...
>>> class C(A):
...     def f(self):
...         print 'C:', super
...         super.f()
...
>>> class D(B, C):
...     def f(self):
...         print 'D:', super
...         super.f()
...
>>> D().f()
D: <super: <class 'D'>, <D object>>
B: <super: <class 'B'>, <D object>>
C: <super: <class 'C'>, <D object>>
A: <super: <class 'A'>, <D object>>

Tim Delaney 


From greg.ewing at canterbury.ac.nz  Wed Apr 25 04:06:02 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Apr 2007 14:06:02 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
Message-ID: <462EB78A.207@canterbury.ac.nz>

Do we really need to have B&D-ish enforcement of abstract
method implementation? It doesn't seem pythonic to me.
I might want to leave some methods of an ABC unimplemented
because I'm not intending to use them. I'd be annoyed if
I were prevented from doing that because of some theoretical
notion that they "should" be implemented.

--
Greg

From steven.bethard at gmail.com  Wed Apr 25 04:29:20 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 24 Apr 2007 20:29:20 -0600
Subject: [Python-3000] adding @abstractmethods after class creation
Message-ID: <d11dcfba0704241929n4006dcbbwbaf860160c21cb15@mail.gmail.com>

On 4/24/07, guido.van.rossum <python-checkins at python.org> wrote:
> +We define a new built-in decorator, ``@abstractmethod``
[snip]
> +**Implementation:** The ``@abstractmethod`` decorator sets the
> +function attribute ``__isabstractmethod__`` to the value ``True``.
> +The ``type.__new__`` method computes the type attribute
> +``__abstractmethods__`` as the set of all method names that have an
> +``__isabstractmethod__`` attribute whose value is true.  It does this
> +by combining the ``__abstractmethods__` attributes of the base
> +classes, adding the names of all methods in the new class dict that
> +have a true ``__isabstractmethod__`` attribute, and removing the names
> +of all methods in the new class dict that don't have a true
> +``__isabstractmethod__`` attribute.  If the resulting
> +``__abstractmethods__`` set is non-empty, the class is considered
> +abstract, and attempts to instantiate it will raise ``TypeError``.

(Hope this wasn't covered in that really long thread. I couldn't see
an answer in the PEP.)

So what happens in a situation like::

    class C:
        pass
    C.foo = abstractmethod(foo)

It seems from the description above like ``C.__abstractmethods__``
would be empty, and therefore C could be instantiated even though it
has an abstract method.

I don't mind at all if we just say "don't do that", but then
@abstractmethod should probably be documented explicitly as only being
usable within a class body.  (Note that this is different from
@classmethod and @staticmethod which can be used after the fact.)

I guess the other option would be to have __setattr__ on classes
append to the __abstractmethods__ list when passed a value with
__isabstractmethod__ == True. But this would probably make setting
attributes on classes (though not on other objects) slower.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From ironfroggy at gmail.com  Wed Apr 25 04:42:26 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Tue, 24 Apr 2007 22:42:26 -0400
Subject: [Python-3000] adding @abstractmethods after class creation
In-Reply-To: <d11dcfba0704241929n4006dcbbwbaf860160c21cb15@mail.gmail.com>
References: <d11dcfba0704241929n4006dcbbwbaf860160c21cb15@mail.gmail.com>
Message-ID: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com>

On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/24/07, guido.van.rossum <python-checkins at python.org> wrote:
> > +We define a new built-in decorator, ``@abstractmethod``
> [snip]
> > +**Implementation:** The ``@abstractmethod`` decorator sets the
> > +function attribute ``__isabstractmethod__`` to the value ``True``.
> > +The ``type.__new__`` method computes the type attribute
> > +``__abstractmethods__`` as the set of all method names that have an
> > +``__isabstractmethod__`` attribute whose value is true.  It does this
> > +by combining the ``__abstractmethods__` attributes of the base
> > +classes, adding the names of all methods in the new class dict that
> > +have a true ``__isabstractmethod__`` attribute, and removing the names
> > +of all methods in the new class dict that don't have a true
> > +``__isabstractmethod__`` attribute.  If the resulting
> > +``__abstractmethods__`` set is non-empty, the class is considered
> > +abstract, and attempts to instantiate it will raise ``TypeError``.
>
> (Hope this wasn't covered in that really long thread. I couldn't see
> an answer in the PEP.)
>
> So what happens in a situation like::
>
>     class C:
>         pass
>     C.foo = abstractmethod(foo)
>
> It seems from the description above like ``C.__abstractmethods__``
> would be empty, and therefore C could be instantiated even though it
> has an abstract method.
>
> I don't mind at all if we just say "don't do that", but then
> @abstractmethod should probably be documented explicitly as only being
> usable within a class body.  (Note that this is different from
> @classmethod and @staticmethod which can be used after the fact.)
>
> I guess the other option would be to have __setattr__ on classes
> append to the __abstractmethods__ list when passed a value with
> __isabstractmethod__ == True. But this would probably make setting
> attributes on classes (though not on other objects) slower.
>
> STeVe
> --
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>         --- Bucky Katt, Get Fuzzy
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com
>

Or could __abstractmethods__ be created dynamically? Which would be
used less, __abstractmethods__ or setting class attributes, that it
would be OK to be a bit slower?

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From steven.bethard at gmail.com  Wed Apr 25 04:56:10 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 24 Apr 2007 20:56:10 -0600
Subject: [Python-3000] adding @abstractmethods after class creation
In-Reply-To: <76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com>
References: <d11dcfba0704241929n4006dcbbwbaf860160c21cb15@mail.gmail.com>
	<76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com>
Message-ID: <d11dcfba0704241956l1f0fa550hf04206ecf8c1c090@mail.gmail.com>

On 4/24/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> Or could __abstractmethods__ be created dynamically? Which would be
> used less, __abstractmethods__ or setting class attributes, that it
> would be OK to be a bit slower?

I considered that, but __abstractmethods__ gets checked at every class
instantiation (as I understand it).  That means that
__abstractmethods__ has to be fast. I assumed that was why Guido
introduced it in the first place...

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From greg.ewing at canterbury.ac.nz  Wed Apr 25 04:58:38 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 25 Apr 2007 14:58:38 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<5.1.1.6.0.20070424071728.04f03c70@sparrow.telecommunity.com>
	<ca471dc20704241050i5de46a46g133e39f8d8adb5e@mail.gmail.com>
Message-ID: <462EC3DE.4010407@canterbury.ac.nz>

Guido van Rossum wrote:
> So if we had this feature I'd make
> BaseException, Exception, StandardError, EnvironmentError and Warning
> abstract (or at least the first 2-3 of those).

I'm not sure I follow the reasoning here.

I've just written some code which raises EnvironmentError
in numerous places. This works perfectly well for the
application at hand, because I don't need to distinguish
between them any more finely. If I need to do so later,
I'll have to make some changes -- but that would happen
anyway even if I had created a subclass of EnvironmentError
such as MyEnvironmentError at the outset.

For library code, it's probably advisable not to use the
standard exception classes directly -- but I don't think
that guideline applies to *all* code.

--
Greg

From guido at python.org  Wed Apr 25 06:29:22 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Apr 2007 21:29:22 -0700
Subject: [Python-3000] adding @abstractmethods after class creation
In-Reply-To: <d11dcfba0704241956l1f0fa550hf04206ecf8c1c090@mail.gmail.com>
References: <d11dcfba0704241929n4006dcbbwbaf860160c21cb15@mail.gmail.com>
	<76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com>
	<d11dcfba0704241956l1f0fa550hf04206ecf8c1c090@mail.gmail.com>
Message-ID: <ca471dc20704242129g454d28c6mf3189a955778a4c@mail.gmail.com>

On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/24/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > Or could __abstractmethods__ be created dynamically? Which would be
> > used less, __abstractmethods__ or setting class attributes, that it
> > would be OK to be a bit slower?
>
> I considered that, but __abstractmethods__ gets checked at every class
> instantiation (as I understand it).  That means that
> __abstractmethods__ has to be fast. I assumed that was why Guido
> introduced it in the first place...

Check the patch on SF (referenced by the PEP now). The instantiation
code only checks a bit in tp_flags, which is set at class creation
time iff __abstractmethods__ is non-empty.

I'm not sure what the point would be of supporting dynamically
modifying ABCs -- it looks like you're just exploring the corners of
the semantics rather than providing actual use cases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Wed Apr 25 06:50:54 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 24 Apr 2007 22:50:54 -0600
Subject: [Python-3000] adding @abstractmethods after class creation
In-Reply-To: <ca471dc20704242129g454d28c6mf3189a955778a4c@mail.gmail.com>
References: <d11dcfba0704241929n4006dcbbwbaf860160c21cb15@mail.gmail.com>
	<76fd5acf0704241942o19d24029q7472713fcf7b05d3@mail.gmail.com>
	<d11dcfba0704241956l1f0fa550hf04206ecf8c1c090@mail.gmail.com>
	<ca471dc20704242129g454d28c6mf3189a955778a4c@mail.gmail.com>
Message-ID: <d11dcfba0704242150x6a3a238fg933a6277786402c4@mail.gmail.com>

On 4/24/07, Guido van Rossum <guido at python.org> wrote:
> I'm not sure what the point would be of supporting dynamically
> modifying ABCs -- it looks like you're just exploring the corners of
> the semantics rather than providing actual use cases.

Yep, I'm just checking the corner case semantics.  Like I said::

    I don't mind at all if we just say "don't do that", but then
    @abstractmethod should probably be documented explicitly as only being
    usable within a class body.  (Note that this is different from
    @classmethod and @staticmethod which can be used after the fact.)

I just want to make sure it's intended and documented.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From steven.bethard at gmail.com  Wed Apr 25 07:26:37 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 24 Apr 2007 23:26:37 -0600
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<462BDCAA.7020501@canterbury.ac.nz> <f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
Message-ID: <d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>

On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/24/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > I must have miscopied then because it worked perfectly here. Yes, I
> > meant to have the _superdesc defined inside the metaclass __init__,
> > but thought I could pull it out to make it cleaner. I forgot it
> > actually had to be there! Here is the metaclass that works.
> >
> > class autosuper(type):
> >         def __init__(cls, name, bases, clsdict):
> >                 class _superdesc(object):
> >                         def __get__(self, obj, objcls):
> >                                 return super(cls, obj)
> >                 cls.__super__ = _superdesc()
>
> I still get the same error.

I played around with this for a while. Below is some code that does
work. The trick is to redefine super() so that when you're calling
__get__ to bind a method to an instance, you bind it to the Super()
instance, not the original object instance.

class Super(object):
    def __init__(self, type, obj=None):
        if isinstance(obj, Super):
            obj = obj.__obj__
        self.__type__ = type
        self.__obj__ = obj
    def __get__(self, obj, cls=None):
        if obj is None:
            raise Exception('only supports instances')
        else:
            return Super(self.__type__, obj)
    def __getattr__(self, attr):
        mro = iter(self.__obj__.__class__.__mro__)
        for cls in mro:
            if cls is self.__type__:
                break
        for cls in mro:
            if attr in cls.__dict__:
                x = cls.__dict__[attr]
                if hasattr(x, '__get__'):
                    x = x.__get__(self, cls)
                return x
        raise AttributeError, attr

class autosuper(type):
    def __init__(cls, name, bases, clsdict):
        cls.__super__ = Super(cls)

class A:
    __metaclass__ = autosuper
    def f(self):
        return 'A'

class B(A):
    def f(self):
        return 'B' + self.__super__.f()

class C(A):
    def f(self):
        return 'C' + self.__super__.f()

class D(B, C):
    def f(self):
        return 'D' + self.__super__.f()

assert D().f() == 'DBCA'

See ma, no bytecode hacks! ;-)

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From nnorwitz at gmail.com  Wed Apr 25 07:48:37 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 24 Apr 2007 22:48:37 -0700
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com>
References: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com>
Message-ID: <ee2a432c0704242248p7167d6c4t1d40d9489402a5c2@mail.gmail.com>

On 4/24/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
>
> Below are links to the ABC implementation based on meta-classes and
> decorators which I have been using for quite a while.

Emin,

You can get similar functionality by using tools such as pychecker.
PyLint and pyflakes may also have this functionality.

If a method's body is:  raise NotImplemenedError and a class with such
a method is instantiated, a warning will be generated.

Cheers,
n

From python at rcn.com  Wed Apr 25 08:04:27 2007
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 24 Apr 2007 23:04:27 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
Message-ID: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1>

[Daniel Stutzbach]
> As a pet project, I've written a container type that looks, acts, and
> quacks like a Python list(), but has better asymptotic performance.
> Specifically, O(log n) time for inserts and deletes in the middle of
> the list.  I'd like to offer it for inclusion in the collections
> module.

In principle, I think this new container would be a fine addition to the collections module.  That is a perfect place for alternate 
implementations of standard types. By matching the list-api, you've reduced the learning curve to near zero while still offering 
intermediate-level users an alternative data structure with different performance trade-offs.

In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven. 
Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let the 
hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be hard 
to say no to this going into Py2.6.


> I'm open to suggestions for a better name.

I'm hoping for a better name too.   Hope you enjoy the impending naming debate --  very few people will read your code, more than a 
few will read the proposal, but *everyone* will have an opinion on the name ;-)   Perhaps start-off with something romantic and 
enigmatic like "treelist" ;-)


> - just as fast as a Python list() when the list is small
> - getslice runs in O(log n) time
> - making a shallow copy runs in O(1) time
> - setslice runs in O(log n + log k) time if the inserted slice is a
> BList of length k
> - multipling a BList by k takes O(log k) time and O(log k) memory

How does BList's memory consumption compare with the current list over-allocation scheme?

Does it take longer to instantiate an emtpy list?  What are the malloc/free patterns?



> In other words, for short lists,
> a BList works just like Python's array-based list() type.  Thus, it
> has the same good performance on small lists.

While this is a good implementation idea (similar ideas are used to speed-up quicksort for instance), it makes the first listed 
advantage seem like a straw-man designed to mislead proposal reviewers into thinking that the blist structure is more efficient than 
is actually is.  Perhaps rewrite it as: "To keep the performance acceptable for short lists, btreelists fall back to the existing 
implementation."


> The Radical Proposal
> Replace list() with the BList.

FWIW, I prefer that the current list implementation remain in-place. Because what we have is simple, it is not hard to maintain, its 
performance characteristics are easily understood, and we have a straight-forward C-API that allows module writers to have fast, 
intuitive direct access to the structure.

OTOH, if your btreelist goes into Py2.6's collections module and users flock to it instead of regular lists, then there may be some 
basis for further discussion.

BTW, please offer-up a pure python version (like we do for the heapq module).  That will make it easier for the curious to find-out 
how the magic is done.  And, it will make life easier for Jython, IronPython, and PyPy implementer so they won't have to do any 
extra work to support Py2.6.




> There are only a few use-cases (that I can think of) where Python's
> list() regularly outperforms the BList.  These are:
>
> 1. A large LIFO stack, where there are many .append() and .pop(-1)
> operations.  These are O(1) for a Python list, but O(log n) for the
> BList().

This is a somewhat important use-case (we devote two methods to it).

> 2. An large, unchanging list, where there are many getitem() calls,
> and none of the inserts or deletes where the BList() shines.  Again,
> these are O(1) for a Python list, but O(log n) for the BList(), and
> the BList() is just as fast if the list is small.

This is also a critical use-case.


> When I first started learning Python, I often ended up inadvertently
> writing O(n**2) algorithms by putting a O(n) list operation inside a
> loop (which is a problem when n=100,000).  Since nothing in the
> signature of list() informs the user when an operation is going to be
> slow, it took me a while to learn how to write my code to work around
> this constraint.  In fact, I initially assumed "list" meant "linked
> list" so most of my assumptions about which operations would be fast
> vs. slow were backwards.

You've come a long way since then.  Congrats.

Thanks for your contribution and thoughtful discussion.  This weekend, I look forward to reading you source code in detail.


Raymond 

From p.f.moore at gmail.com  Wed Apr 25 10:05:42 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Apr 2007 09:05:42 +0100
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <462EB78A.207@canterbury.ac.nz>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
Message-ID: <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>

On 25/04/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Do we really need to have B&D-ish enforcement of abstract
> method implementation? It doesn't seem pythonic to me.
> I might want to leave some methods of an ABC unimplemented
> because I'm not intending to use them. I'd be annoyed if
> I were prevented from doing that because of some theoretical
> notion that they "should" be implemented.

I agree - this whole discussion still sounds like things are getting
overly strict.

OTOH, in Greg's case maybe he should just not bother inheriting from
the ABC. From what I hear, that shouldn't cause him any problems
("duck typing isn't going away" is the message I've heard). Of course,
it's difficult to be sure without specific examples (on either side of
the debate).

Paul.

From jyasskin at gmail.com  Wed Apr 25 10:36:06 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 25 Apr 2007 01:36:06 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
Message-ID: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>

Here's a draft of the numbers ABCs PEP. The most up to date version
will live in the darcs repository at
http://jeffrey.yasskin.info/darcs/PEPs/pep-3141.txt (unless the number
changes) for now. Naming a PEP about numbers 3.141 seems cute, but of
course, I don't get to pick the number. :) This is my first PEP, so
apologies for any obvious mistakes.

I'd particularly like the numpy people's input on whether I've gotten
floating-point support right.

Thanks,
Jeffrey Yasskin

-------------------------------
PEP: 3141
Title: A Type Hierarchy for Numbers (and other algebraic entities)
Version: $Revision: 54928 $
Last-Modified: $Date: 2007-04-23 16:37:29 -0700 (Mon, 23 Apr 2007) $
Author: Jeffrey Yasskin <jyasskin at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 23-Apr-2007
Post-History: Not yet posted


Abstract
========

This proposal defines a hierarchy of Abstract Base Classes (ABCs)
[#pep3119] to represent numbers and other algebraic entities similar
to numbers.  It proposes:

* A hierarchy of algebraic concepts, including monoids, groups, rings,
  and fields with successively more operators and constraints on their
  operators. This will be added as a new library module named
  "algebra".

* A hierarchy of specifically numeric types, which can be converted to
  and from the native Python types. This will be added as a new
  library module named "numbers".

Rationale
=========

Functions that take numbers as arguments should be able to determine
the properties of those numbers, and if and when overloading based on
types is added to the language, should be overloadable based on the
types of the arguments. This PEP defines some abstract base classes
that are useful in numerical calculations. A function can check that
variable is an instance of one of these classes and then rely on the
properties specified for them. Of course, the language cannot check
these properties, so where I say something is "guaranteed", I really
just mean that it's one of those properties a user should be able to
rely on.

This PEP tries to find a balance between providing fine-grained
distinctions and specifying types that few people will ever use.

Specification
=============

Although this PEP uses terminology from PEP3119, the hierarchy is
meaningful for any systematic method of defining sets of
classes. **Todo:** link to the Interfaces PEP when it's ready. I'm
also using the extra notation from [#pep3107] (annotations) to specify
some types.

Object oriented systems have a general problem in constraining
functions that take two arguments. To take addition as an example,
``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)``
is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense. So
``a + b`` is not guaranteed to be defined for any two instances of
``AdditiveGroup``, but it is guaranteed to be defined when ``type(a)
== type(b)``. On the other hand, ``+`` does make sense for any sorts
of numbers, so the ``Complex`` ABC refines the properties for plus so
that ``a + b`` is defined whenever ``isinstance(a,Complex) and
isinstance(b,Complex)``, even if ``type(a) != type(b)``.


Monoids (http://en.wikipedia.org/wiki/Monoid) consist of a set with an
associative operation, and an identity element under that
operation. **Open issue**: Is a @classmethod the best way to define
constants that depend only on the type?::

    class MonoidUnderPlus(Abstract):
	"""+ is associative but not necessarily commutative and has an
	identity given by plus_identity().

	Subclasses follow the laws:

	  a + (b + c) === (a + b) + c
	  a.plus_identity() + a === a === a + a.plus_identity()

	Sequences are monoids under plus (in Python) but are not
	AdditiveGroups.
	"""
	@abstractmethod
	def __add__(self, other):
	    raise NotImplementedError

	@classmethod
	@abstractmethod
	def plus_identity(cls):
	    raise NotImplementedError

I skip ordinary non-commutative groups here because I don't have any
common examples of groups that use ``+`` as their operator but aren't
commutative. If we find some, the class can be added later.::

    class AdditiveGroup(MonoidUnderPlus):
	"""Defines a commutative group whose operator is +, and whose inverses
	are produced by -x.
	See http://en.wikipedia.org/wiki/Abelian_group.

	Where a, b, and c are instances of the same subclass of
	AdditiveGroup, the operations should follow these laws, where
	'zero' is a.__class__.zero().

	      a + b === b + a
	(a + b) + c === a + (b + c)
	   zero + a === a
	   a + (-a) === zero
	      a - b === a + -b

	Some abstract subclasses, such as Complex, may extend the
	definition of + to heterogenous subclasses, but AdditiveGroup only
	guarantees it's defined on arguments of exactly the same types.

	Vectors are AdditiveGroups but are not Rings.
	"""
	@abstractmethod
	def __add__(self, other):
	    """Associative commutative operation, whose inverse is negation."""
	    raise NotImplementedError

**Open issue:** Do we want to give people a choice of which of the
following to define, or should we pick one arbitrarily?::

	def __neg__(self):
	    """Must define this or __sub__()."""
	    return self.zero() - self

	def __sub__(self, other):
	    """Must define this or __neg__()."""
	    return self + -other

	@classmethod
	@abstractmethod
	def zero(cls):
	    """A better name for +'s identity as we move into more mathematical
	    domains."""
	    raise NotImplementedError

	@classmethod
	def plus_identity(cls):
	    return cls.zero()

Including Semiring (http://en.wikipedia.org/wiki/Semiring) would help
a little with defining a type for the natural numbers. That can be
split out once someone needs it (see ``IntegralDomain`` for how).::

    class Ring(AdditiveGroup):
	"""A mathematical ring over the operations + and *.
	See http://en.wikipedia.org/wiki/Ring_%28mathematics%29.

	In addition to the requirements of the AdditiveGroup superclass, a
	Ring has an associative but not necessarily commutative
	multiplication operation with identity (one) that distributes over
	addition. A Ring can be constructed from any integer 'i' by adding
	'one' to itself 'i' times. When R is a subclass of Ring, the
	additive identity is R(0), and the multiplicative identity is
	R(1).

	Matrices are Rings but not Commutative Rings or Division
	Rings. The quaternions are a Division Ring but not a
	Field. The integers are a Commutative Ring but not a Field.
	"""
	@abstractmethod
	def __init__(self, i:int):
	    """An instance of a Ring may be constructed from an integer.

	    This may be a lossy conversion, as in the case of the integers
	    modulo N."""
	    pass

	@abstractmethod
	def __mul__(self, other):
	    """Satisfies:
	    a * (b * c) === (a * b) * c
		one * a === a
		a * one === a
	    a * (b + c) === a * b + a * c

	    where one == a.__class__(1)
	    """
	    raise NotImplementedError

	@classmethod
	def zero(cls):
	    return cls(0)

	@classmethod
	def one(cls):
	    return cls(1)

I'm skipping both CommutativeRing and DivisionRing here.

    class Field(Ring):
	"""The class Field adds to Ring the requirement that * be a
	commutative group operation except that zero does not have an
	inverse.
	See http://en.wikipedia.org/wiki/Field_%28mathematics%29.

	Practically, that means we can define division on a Field. The
	additional laws are:

	a * b === b * a
	a / a === a.__class_(1)  # when a != a.__class__(0)

	Division lets us construct a Field from any Python float,
	although the conversion is likely to be lossy. Some Fields
	include the real numbers, rationals, and integers mod a
	prime. Python's ``float`` resembles a Field closely.
	"""
	def __init__(self, f:float):
	    """A Field should be constructible from any rational number, which
	    includes Python floats."""
	    pass

	@abstractmethod
	def __div__(self, divisor):
	    raise NotImplementedError

Division is somewhat complicated in Python. You have both __floordiv__
and __div__, and ints produce floats when they're divided. For the
purposes of this hierarchy, ``__floordiv__(a, b)`` is defined by
``floor(__div__(a, b))``, and, since int is not a subclass of Field,
it's allowed to do whatever it wants with __div__.

There are four more reasonable classes that I'm skipping here in the
interest of keeping the initial library simple. They are:

``Algebraic``
    Rational powers of its elements are defined (and maybe a few other
    operations)
    (http://en.wikipedia.org/wiki/Algebraic_number). Complex numbers
    are the most well-known algebraic set. Real numbers are _not_
    algebraic, but Python does define these operations on floats,
    which makes defining this class somewhat difficult.

``Trancendental``
    The elementary functions
    (http://en.wikipedia.org/wiki/Elementary_function) are
    defined. These are basically arbitrary powers, trig functions, and
    logs, the contents of ``cmath``.

The following two classes can be reasonably combined with ``Integral``
for now.
``IntegralDomain``
    Defines __divmod__.
    (http://darcs.haskell.org/numericprelude/docs/html/Algebra-IntegralDomain.html#t%3AC)

``PrincipalIdealDomain``
    Defines gcd and lcm.
    (http://darcs.haskell.org/numericprelude/docs/html/Algebra-PrincipalIdealDomain.html#t%3AC)

If someone needs to split them later, they can use code like::
    import numbers
    class IntegralDomain(Ring): ...
    numbers.Integral.__bases__ = (IntegralDomain,) + numbers.Integral.__bases__


Finally, we get to numbers. This is where we switch from the "algebra"
module to the "numbers" module.::

    class Complex(Ring, Hashable):
        """The ``Complex`` ABC indicates that the value lies somewhere
        on the complex plane, not that it in fact has a complex
        component: ``int`` is a subclass of ``Complex``. Because these
        actually represent complex numbers, they can be converted to
        the ``complex`` type.

        ``Complex`` finally gets around to requiring its subtypes to
        be immutable so they can be hashed in a standard way.

        ``Complex`` also requires its operations to accept
        heterogenous arguments. Subclasses should override the
        operators to be more accurate when they can, but should fall
        back on the default definitions to handle arguments of
        different (Complex) types.

        **Open issue:** __abs__ doesn't fit here because it doesn't
        exist for the Gaussian integers
        (http://en.wikipedia.org/wiki/Gaussian_integer). In fact, it
        only exists for algebraic complex numbers and real numbers. We
        could define it in both places, or leave it out of the
        ``Complex`` classes entirely and let it be a custom extention
        of the ``complex`` type.

        The Gaussian integers are ``Complex`` but not a ``Field``.
	"""
	@abstractmethod
	def __complex__(self):
	    """Any Complex can be converted to a native complex object."""
	    raise NotImplementedError

	def __hash__(self):
	    return hash(complex(self))

	@abstractmethod
	def real(self) => Real:
	    raise NotImplementedError

	@abstractmethod
	def imag(self) => Real:
	    raise NotImplementedError

	@abstractmethod
	def __add__(self, other):
	    """The other Ring operations should be implemented similarly."""
	    if isinstance(other, Complex):
		return complex(self) + complex(other)
	    else:
		return NotImplemented

``FractionalComplex(Complex, Field)`` might fit here, except that it
wouldn't give us any new operations.

    class Real(Complex, TotallyOrdered):
	"""Numbers along the real line. Some subclasses of this class
	may contain NaNs that are not ordered with the rest of the
	instances of that type. Oh well. **Open issue:** what problems
	will that cause? Is it worth it in order to get a
	straightforward type hierarchy?
	"""
	@abstractmethod
	def __float__(self):
	    raise NotImplementedError
	def __complex__(self):
	    return complex(float(self))
	def real(self) => self.__class__:
	    return self
	def imag(self) => self.__class__:
	    return self.__class__(0)
        def __abs__(self) => self.__class__:
            if self < 0: return -self
            else: return self


    class FractionalReal(Real, Field):
	"""Rationals and floats. This class provides concrete
	definitions of the other four methods from properfraction and
	allows you to convert fractional reals to integers in a
	disciplined way.
	"""
	@abstractmethod
	def properfraction(self) => (int, self.__class__):
	    """Returns a pair (n,f) such that self == n+f, and:
	      * n is an integral number with the same sign as self; and
	      * f is a fraction with the same type and sign as self, and with
		absolute value less than 1.
	      """
	    raise NotImplementedError
	def floor(self) => int:
            n, r = self.properfraction()
            if r < 0 then n - 1 else n
	def ceiling(self) => int: ...
	def __trunc__(self) => int: ...
	def round(self) => int: ...


**Open issue:** What's the best name for this class? RealIntegral? Integer?::

    class Integral(Real):
	"""Integers!"""
	@abstractmethod
	def __int__(self):
	    raise NotImplementedError
	def __float__(self):
	    return float(int(self))

	@abstractmethod
	def __or__(self, other):
	    raise NotImplementedError
	@abstractmethod
	def __xor__(self, other):
	    raise NotImplementedError
	@abstractmethod
	def __and__(self, other):
	    raise NotImplementedError
	@abstractmethod
	def __lshift__(self, other):
	    raise NotImplementedError
	@abstractmethod
	def __rshift__(self, other):
	    raise NotImplementedError
	@abstractmethod
	def __invert__(self):
	    raise NotImplementedError


Floating point values may not exactly obey several of the properties
you would expect from their superclasses. For example, it is possible
for ``(large_val + -large_val) + 3 == 3``, but ``large_val +
(-large_val + 3) == 0``. On the values most functions deal with this
isn't a problem, but it is something to be aware of. Types like this
inherit from ``FloatingReal`` so that functions that care can know to
use a numerically stable algorithm on them. **Open issue:** Is this
the proper way to handle floating types?::

    class FloatingReal:
	"""A "floating" number is one that is represented as
	``mantissa * radix**exponent`` where mantissa, radix, and
	exponent are all integers. Subclasses of FloatingReal don't
	follow all the rules you'd expect numbers to follow. If you
	really care about the answer, you have to use numerically
	stable algorithms, whatever those are.

        **Open issue:** What other operations would be useful here?

	These include floats and Decimals.
	"""
	@classmethod
	@abstractmethod
	def radix(cls) => int:
	    raise NotImplementedError

	@classmethod
	@abstractmethod
	def digits(cls) => int:
	    """The number of significant digits of base cls.radix()."""
	    raise NotImplementedError

	@classmethod
	@abstractmethod
	def exponentRange(cls) => (int, int):
	    """A pair of the (lowest,highest) values possible in the exponent."""
	    raise NotImplementedError

	@abstractmethod
	def decode(self) => (int, int):
	    """Returns a pair (mantissa, exponent) such that
	    mantissa*self.radix()**exponent == self."""
	    raise NotImplementedError



Inspiration
===========
http://hackage.haskell.org/trac/haskell-prime/wiki/StandardClasses
http://repetae.net/john/recent/out/classalias.html

References
==========

.. [#pep3119] Introducing Abstract Base Classes
   (http://www.python.org/dev/peps/pep-3119/)

.. [#pep3107] Function Annotations
   (http://www.python.org/dev/peps/pep-3107/)

.. [3] Possible Python 3K Class Tree?, wiki page created by Bill Janssen
   (http://wiki.python.org/moin/AbstractBaseClasses)

.. [#numericprelude] NumericPrelude: An experimental alternative
hierarchy of numeric type classes
   (http://darcs.haskell.org/numericprelude/docs/html/index.html)


Acknowledgements
----------------

Thanks to Neil Norwitz for helping me through the PEP process.

The Haskell Numeric Prelude [#numericprelude] nicely condensed a lot
of experience with the Haskell numeric hierarchy into a form that was
relatively easily adaptable to Python.

Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From ironfroggy at gmail.com  Wed Apr 25 14:52:50 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 25 Apr 2007 08:52:50 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
Message-ID: <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com>

On 4/25/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > On 4/24/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > > I must have miscopied then because it worked perfectly here. Yes, I
> > > meant to have the _superdesc defined inside the metaclass __init__,
> > > but thought I could pull it out to make it cleaner. I forgot it
> > > actually had to be there! Here is the metaclass that works.
> > >
> > > class autosuper(type):
> > >         def __init__(cls, name, bases, clsdict):
> > >                 class _superdesc(object):
> > >                         def __get__(self, obj, objcls):
> > >                                 return super(cls, obj)
> > >                 cls.__super__ = _superdesc()
> >
> > I still get the same error.
>
> I played around with this for a while. Below is some code that does
> work. The trick is to redefine super() so that when you're calling
> __get__ to bind a method to an instance, you bind it to the Super()
> instance, not the original object instance.
>
> class Super(object):
>     def __init__(self, type, obj=None):
>         if isinstance(obj, Super):
>             obj = obj.__obj__
>         self.__type__ = type
>         self.__obj__ = obj
>     def __get__(self, obj, cls=None):
>         if obj is None:
>             raise Exception('only supports instances')
>         else:
>             return Super(self.__type__, obj)
>     def __getattr__(self, attr):
>         mro = iter(self.__obj__.__class__.__mro__)
>         for cls in mro:
>             if cls is self.__type__:
>                 break
>         for cls in mro:
>             if attr in cls.__dict__:
>                 x = cls.__dict__[attr]
>                 if hasattr(x, '__get__'):
>                     x = x.__get__(self, cls)
>                 return x
>         raise AttributeError, attr
>
> class autosuper(type):
>     def __init__(cls, name, bases, clsdict):
>         cls.__super__ = Super(cls)
>
> class A:
>     __metaclass__ = autosuper
>     def f(self):
>         return 'A'
>
> class B(A):
>     def f(self):
>         return 'B' + self.__super__.f()
>
> class C(A):
>     def f(self):
>         return 'C' + self.__super__.f()
>
> class D(B, C):
>     def f(self):
>         return 'D' + self.__super__.f()
>
> assert D().f() == 'DBCA'
>
> See ma, no bytecode hacks! ;-)
>
> STeVe
> --
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>         --- Bucky Katt, Get Fuzzy

Awesomeness. The 'super keyword' brought up in the original post would
also be possible via the same route and some stack pokin' to get at
self from the super object in uses like super.foo(args) and such.

Jim, you said you'd take a crack "if no one else volunteers", so
perhaps I will, if you didn't have your heart set on it. I would like
more opportunities to contribute and this wouldn't be a terribly long
PEP for my first foray into writing one.

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From emin.shopper at gmail.com  Wed Apr 25 15:16:24 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 09:16:24 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
Message-ID: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>

I disagree. I think that B&D-ish enforcement of ABCs is the most important
feature of an ABC (with the option of being able to turn off enforcement on
a per-class basis).

Imagine you have a program that takes a long time to run. If you use
duck-typing or similar ideas you may watch the program run for hours or days
and then raise an AttributeError or NotImplemented error because a derived
class didn't implement a required method. With B&D-ish enforcement of ABCs,
you can automatically catch such problems as soon as the program starts.
This is a problem that I see no other good solution to. For long running
programs, unit-tests will also take a long time to run so the argument that
your unit-tests should catch such errors is no longer valid. Also, while
pylint and pychecker are great tools that could be used to catch such
errors, they must be run manually by the programmer and generally take a
long time to run (at least longer than the ABC implementation I have).

To reiterate, while I think the documentation and reflection benefits of
ABCs are nice, I don't think they are nearly as essential as enforcement.
For most of the use cases I've seen, you could just as easily read the
documentation or the source code or try a simple example to determine how to
implement an iterable or hashable that will play nice with other classes. In
contrast, ABC enforcement provides the real benefit of catching errors
before many hours of program runtime are wasted. Consequently, I'm +1 on B&D
in this case.

-Emin Martinian

On 4/25/07, Paul Moore <p.f.moore at gmail.com> wrote:
>
> On 25/04/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > Do we really need to have B&D-ish enforcement of abstract
> > method implementation? It doesn't seem pythonic to me.
> > I might want to leave some methods of an ABC unimplemented
> > because I'm not intending to use them. I'd be annoyed if
> > I were prevented from doing that because of some theoretical
> > notion that they "should" be implemented.
>
> I agree - this whole discussion still sounds like things are getting
> overly strict.
>
> OTOH, in Greg's case maybe he should just not bother inheriting from
> the ABC. From what I hear, that shouldn't cause him any problems
> ("duck typing isn't going away" is the message I've heard). Of course,
> it's difficult to be sure without specific examples (on either side of
> the debate).
>
> Paul.
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/emin.shopper%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/860a297b/attachment.html 

From emin.shopper at gmail.com  Wed Apr 25 15:19:07 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 09:19:07 -0400
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <ee2a432c0704242248p7167d6c4t1d40d9489402a5c2@mail.gmail.com>
References: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com>
	<ee2a432c0704242248p7167d6c4t1d40d9489402a5c2@mail.gmail.com>
Message-ID: <32e43bb70704250619o7cda8942ka9a76457ea9c52ea@mail.gmail.com>

On 4/25/07, Neal Norwitz <nnorwitz at gmail.com> wrote:

You can get similar functionality by using tools such as pychecker.
> PyLint and pyflakes may also have this functionality.


I like PyLint and pychecker a lot, but the funtionality is not similar.
Both take a lot longer to run (at least longer than my ABC implementation)
and you have to remember to run them each time you change your program.
Using an ABC adds very little overhead and gets checked automatically.

Thanks,
-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/9733d63c/attachment.html 

From jimjjewett at gmail.com  Wed Apr 25 15:52:35 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 09:52:35 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com>
Message-ID: <fb6fbf560704250652w1c34fe85n42d051e0ec242b5f@mail.gmail.com>

On 4/25/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> Jim, you said you'd take a crack "if no one else volunteers", so
> perhaps I will, if you didn't have your heart set on it. I would like
> more opportunities to contribute and this wouldn't be a terribly long
> PEP for my first foray into writing one.


I would be thrilled if you took it; I posted the __this_class__ PEP as
a separate piece, but I still haven't figured out the full super --
and if you have time, that would be great.

-jJ

From jjb5 at cornell.edu  Wed Apr 25 15:21:28 2007
From: jjb5 at cornell.edu (Joel Bender)
Date: Wed, 25 Apr 2007 09:21:28 -0400
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<f0hkrs$aro$1@sea.gmane.org>	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>	<f0j77k$nj5$1@sea.gmane.org>	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com>
Message-ID: <462F55D8.5000100@cornell.edu>

I mucked around with this, and here is my version:

     class _super(property):
         def __init__(self):
             property.__init__(self, self.get_super, None, None)

         def get_super(self, klass):
             def dong(obj):
                 class wrapper:
                     def __getattr__(self, fn):
                         save = obj.__class__
                         obj.__class__ = klass.__mro__[1]
                         f = getattr(obj, fn)
                         obj.__class__ = save
                         return f

                 return wrapper()

             return dong

     class _superable(type):
         __super__ = _super()

     class W(object):
         __metaclass__ = _superable

         def f(self):
             print "W"

     class X(W):
         def f(self):
             print "X"
             X.__super__(self).f()

     class Y(W):
         def f(self):
             print "Y"
             Y.__super__(self).f()

     class Z(X, Y):
         def f(self):
             print "Z"
             Z.__super__(self).f()

I would rather the end result look more like this:

     class Z(X, Y):
         def f(self):
             print "Z"
             Z.__super__.f(self)

But I haven't figured out how to do that yet, and the self.__super__.f() 
pattern that Steven Bethard posted trumps it anyway!


Joel


From p.f.moore at gmail.com  Wed Apr 25 16:04:22 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 25 Apr 2007 15:04:22 +0100
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
Message-ID: <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>

On 25/04/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
> I disagree. I think that B&D-ish enforcement of ABCs is the most important
> feature of an ABC (with the option of being able to turn off enforcement on
> a per-class basis).

OK. We can agree to disagree. My point was that as far as I am aware,
Guido's current position on ABCs is that they are, and will remain,
optional - people like myself (and Greg, from the sound of it) who
don't want to derive from ABCs will not be penalised by being excluded
from anything in the Python core or the stdlib.

Example - I recall Guido saying explicitly that there will be no
requirement for objects assigned to sys.stdout to derive from any of
the IO ABCs.

Paul.

From jjb5 at cornell.edu  Wed Apr 25 16:44:15 2007
From: jjb5 at cornell.edu (Joel Bender)
Date: Wed, 25 Apr 2007 10:44:15 -0400
Subject: [Python-3000] Fixing super anyone? (redux)
In-Reply-To: <462F55D8.5000100@cornell.edu>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<f0hkrs$aro$1@sea.gmane.org>	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>	<f0j77k$nj5$1@sea.gmane.org>	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>	<76fd5acf0704250552p19bb3c51sb66cf569beebfda6@mail.gmail.com>
	<462F55D8.5000100@cornell.edu>
Message-ID: <462F693F.90306@cornell.edu>

> But I haven't figured out how to do that yet...

It turns out to be easier than I thought, and it avoids changing the 
object __class__, which is ugly.

     class _super(property):
         def __init__(self):
             property.__init__(self, self.get_super, None, None)

         def get_super(self, klass):
             class wrapper:
                 def __getattr__(self, fn):
                     for cls in klass.__mro__[1:]:
                         f = getattr(cls, fn, None)
                         if f:
                             return f

                     raise AttributeError, fn

             return wrapper()

     class _superable(type):
         __super__ = _super()

     class W(object):
         __metaclass__ = _superable

         def f(self):
             print "W"

     class X(W):
         def f(self):
             print "X"
             X.__super__.f(self)

     class Y(W):
         def f(self):
             print "Y"
             Y.__super__.f(self)

     class Z(X, Y):
         def f(self):
             print "Z"
             Z.__super__.f(self)

I'll go back to lurking now...


Joel

From bingham at cenix-bioscience.com  Wed Apr 25 16:58:04 2007
From: bingham at cenix-bioscience.com (Aaron Bingham)
Date: Wed, 25 Apr 2007 16:58:04 +0200
Subject: [Python-3000] An introduction to ABC's
In-Reply-To: <32e43bb70704250619o7cda8942ka9a76457ea9c52ea@mail.gmail.com>
References: <32e43bb70704240739x53d80ff3j863e64d9a11b5f@mail.gmail.com>	<ee2a432c0704242248p7167d6c4t1d40d9489402a5c2@mail.gmail.com>
	<32e43bb70704250619o7cda8942ka9a76457ea9c52ea@mail.gmail.com>
Message-ID: <462F6C7C.5090608@cenix-bioscience.com>

Emin.shopper Martinian.shopper wrote:
> On 4/25/07, *Neal Norwitz* <nnorwitz at gmail.com 
> <mailto:nnorwitz at gmail.com>> wrote:
>
>     You can get similar functionality by using tools such as pychecker.
>     PyLint and pyflakes may also have this functionality.
>
>
> I like PyLint and pychecker a lot, but the funtionality is not 
> similar.  Both take a lot longer to run (at least longer than my ABC 
> implementation) and you have to remember to run them each time you 
> change your program. Using an ABC adds very little overhead and gets 
> checked automatically.
You could use an editor that automatically runs PyLint and/or 
pychecker.  The PyDev plugin for Eclipse runs PyLint in the background 
and highlights any lines that produced errors or warnings.  Now that 
I've worked with PyDev for several months, I wouldn't want to work 
without the immediate feedback it provides.

Regards,

-- 
Aaron Bingham
Senior Software Engineer
bingham at cenix-bioscience.com
Tel. +49 (0)351 4173-146
Fax +49 (0)351 4173-109

Cenix BioScience GmbH
Tatzberg 47
01307 Dresden, Germany
www.cenix-bioscience.com

---------------------------------------------------------
Sitz der Gesellschaft (Place of Business): Dresden
Gesch?ftsf?hrer (CEO): Dr. Christophe J. Echeverri
Amtsgericht (Local Court): Dresden, HRB 19964
Ust-ID (VAT-No.): DE205824437
---------------------------------------------------------


From jcarlson at uci.edu  Wed Apr 25 18:03:14 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 25 Apr 2007 09:03:14 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
Message-ID: <20070425085350.6408.JCARLSON@uci.edu>


"Jeffrey Yasskin" <jyasskin at gmail.com> wrote:
> Here's a draft of the numbers ABCs PEP. The most up to date version
> will live in the darcs repository at
> http://jeffrey.yasskin.info/darcs/PEPs/pep-3141.txt (unless the number
> changes) for now. Naming a PEP about numbers 3.141 seems cute, but of
> course, I don't get to pick the number. :) This is my first PEP, so
> apologies for any obvious mistakes.
> 
> I'd particularly like the numpy people's input on whether I've gotten
> floating-point support right.
> 
> Thanks,
> Jeffrey Yasskin
> 
> -------------------------------
> PEP: 3141
> Title: A Type Hierarchy for Numbers (and other algebraic entities)
> Version: $Revision: 54928 $
> Last-Modified: $Date: 2007-04-23 16:37:29 -0700 (Mon, 23 Apr 2007) $
> Author: Jeffrey Yasskin <jyasskin at gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 23-Apr-2007
> Post-History: Not yet posted
> 
> 
> Abstract
> ========
> 
> This proposal defines a hierarchy of Abstract Base Classes (ABCs)
> [#pep3119] to represent numbers and other algebraic entities similar
> to numbers.  It proposes:
[snip]

Are the base number operations in Python all that difficult to
understand?  Do we really need to add mathematical formalism into
Python's type system before people understand the meaning of X * Y? 
Because all I really see this doing is confusing the hell out of people
who aren't mathematicians; I'm a theoretical computer scientist and I
had to try to think for a moment to verify that all of those were really
necessary to separate the cases.

I really do understand wanting to be able to ask "is operation X
supported for values Y and Z", but is this really necessary for numbers?


 - Josiah


From jyasskin at gmail.com  Wed Apr 25 18:50:20 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 25 Apr 2007 09:50:20 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<20070425085350.6408.JCARLSON@uci.edu>
Message-ID: <5d44f72f0704250950j2c2f6c77i61560a0d60ae7da4@mail.gmail.com>

[bcc'ing numpy-discussion. Comments should probably try to stay on the
python-3000 list.]

On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> Are the base number operations in Python all that difficult to
> understand?  Do we really need to add mathematical formalism into
> Python's type system before people understand the meaning of X * Y?
> Because all I really see this doing is confusing the hell out of people
> who aren't mathematicians; I'm a theoretical computer scientist and I
> had to try to think for a moment to verify that all of those were really
> necessary to separate the cases.
>
> I really do understand wanting to be able to ask "is operation X
> supported for values Y and Z", but is this really necessary for numbers?

If you don't want to use the full power of the system, nothing forces
you to. If someone doesn't know what a Ring or a complex number is,
they can just check for Real and rely on the same operations, at the
small cost of overconstraining their inputs. One reason I'm suggesting
splitting the classes into the "algebra" and "numbers" modules is so
that people who don't know or care about the algebra can rely on just
"numbers" and still get nearly all of the benefits.

However, I think it's important to provide a framework for people who
want to do more powerful things. If you write a function that can take
an arbitrary ring, it should Just Work on a python int, or any other
Complex subclass, without you having to check for a lot of operations
separately. The Haskell98 numerical hierarchy actually does start at
Num, ignoring the algebraic structures underneath, and it draws a lot
of criticism for it.

That does mean that the names and documentation in the numbers module
need to be as clear as we can make them for non-mathematical
audiences. I'm certainly open to suggestions on that front.

-- 
Namast?,
Jeffrey Yasskin

From collinw at gmail.com  Wed Apr 25 18:54:20 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 25 Apr 2007 09:54:20 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
Message-ID: <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>

The following things mean absolutely nothing to me:

- Monoid
- MonoidUnderPlus
- Group
- Ring
- Semiring
- Field

So, most of the terminology in the PEP.

I can see use-cases for this level of formalism, but I'm a strong -1
on making any part of the stdlib effectively off-limits for people
without advanced math degrees. Why can't this be shipped as a
third-party module?

Collin Winter

From daniel at stutzbachenterprises.com  Wed Apr 25 18:54:33 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Wed, 25 Apr 2007 11:54:33 -0500
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <00c601c786ff$95839700$f101a8c0@RaymondLaptop1>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<00c601c786ff$95839700$f101a8c0@RaymondLaptop1>
Message-ID: <eae285400704250954w4a493f55k9218bdc89802850c@mail.gmail.com>

On 4/25/07, Raymond Hettinger <python at rcn.com> wrote:
> In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven.
> Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let the
> hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be hard
> to say no to this going into Py2.6.

Will do.  What's the best way to release a module to make it easy for
people to find, download, and install?  (a pointer to a relevant
document would be great)

> How does BList's memory consumption compare with the current list over-allocation scheme?
>
> Does it take longer to instantiate an emtpy list?  What are the malloc/free patterns?

See these earlier message in the thread:

http://mail.python.org/pipermail/python-3000/2007-April/006801.html
http://mail.python.org/pipermail/python-3000/2007-April/006804.html
http://mail.python.org/pipermail/python-3000/2007-April/006809.html

The short summary is that BList and the current list have very similar
best-case and worst-case memory consumption for a single list.  (If
copy-on-write comes into play, BList is much more memory efficient
than a regular list)

Malloc/free operations occur when the list is large and growing or
shrinking (approximately when 64 elements have been added or removed).
 They occur more often than for list(), but if you're doing a lot of
growing and shrinking the O(log n) vs O(n) performance boost easily
pays for the extra memory operations.  All the blocks are the same
size.  The BList code keeps a small list of recently freed nodes to
recycle, which speeds things up in certain common cases where
additions and removals are interchanged.

> > In other words, for short lists,
> > a BList works just like Python's array-based list() type.  Thus, it
> > has the same good performance on small lists.
>
> While this is a good implementation idea (similar ideas are used to speed-up quicksort for instance), it makes the first listed
> advantage seem like a straw-man designed to mislead proposal reviewers into thinking that the blist structure is more efficient than
> is actually is.  Perhaps rewrite it as: "To keep the performance acceptable for short lists, btreelists fall back to the existing
> implementation."

I fear that you have misunderstood me, as the BList does not fall back
on the existing implementation.  I'm sorry that you may have gotten
the impression that I was attempting to mislead.  I admit openly that
the performance for small lists is not due to any fancy data
structure.  However, it's not array-like as a special case just for
small lists, either.

Each BList node contains an array of up to 128 children.  If the BList
node is a leaf, these children are pointers to items stored by the
user, otherwise they are pointers to other BList nodes.  If the list
contains 128 or fewer children, the BList can have one node which is
both the root and a leaf.  This node just has an array of the user's
items, so it ends up having the same performance as Python's
array-based list.

It is true that you end up with array-operations when you have a small
list and as the base case for many operations implemented via
recursion.  In some sense that is similar to a QuickSort that uses
InsertionSort for n < threshold.  Unlike QuickSort, I couldn't rewrite
it *without* array operations as the base case, and array operations
also occur in the middle of the tree.  They're fundamental, not a
performance tweak.  So I don't think the QuickSort with InsertionSort
analogy holds.

How about if I include a brief description up-front of how it works
like: "The BList internally uses a tree structure where each node
contains an array of up to 128 children."  Then the first performance
bullet can be something like "just as fast as a Python list() when the
list is small, since BList's are also based on arrays"?

> FWIW, I prefer that the current list implementation remain in-place. Because what we have is simple, it is not hard to maintain, its
> performance characteristics are easily understood, and we have a straight-forward C-API that allows module writers to have fast,
> intuitive direct access to the structure.
>
> OTOH, if your btreelist goes into Py2.6's collections module and users flock to it instead of regular lists, then there may be some
> basis for further discussion.

This sounds like a good plan to me, though how does that square with
Guido's desire to avoid a proliferation of data types?
http://mail.python.org/pipermail/python-3000/2007-April/006774.html

> BTW, please offer-up a pure python version (like we do for the heapq module).  That will make it easier for the curious to find-out
> how the magic is done.  And, it will make life easier for Jython, IronPython, and PyPy implementer so they won't have to do any
> extra work to support Py2.6.

Will do.

> > There are only a few use-cases (that I can think of) where Python's
> > list() regularly outperforms the BList.  These are:
> >
> > 1. A large LIFO stack, where there are many .append() and .pop(-1)
> > operations.  These are O(1) for a Python list, but O(log n) for the
> > BList().
>
> This is a somewhat important use-case (we devote two methods to it).
>
> > 2. An large, unchanging list, where there are many getitem() calls,
> > and none of the inserts or deletes where the BList() shines.  Again,
> > these are O(1) for a Python list, but O(log n) for the BList(), and
> > the BList() is just as fast if the list is small.
>
> This is also a critical use-case.

I don't disagree.

I think the key question is this:  Given that there's no performance
difference for small lists, would you rather have a type that for
large lists:

1) Gives you O(1) speed for some critical use-cases, but O(n) for others, or
2) Gives you O(log n) across the board?

Put another way, for a 10,000 element list how would you feel about
taking a small performance hit (50% increase in execution time for
LIFO; 5% increase in execution time for getitem) in exchange for large
performance improvements elsewhere (< 1% of the old execution time for
getslice, setslice, FIFO, etc.)?

I obviously favor #2, but I acknowledge that this is a subjective
question that depends on which use-cases you care most about.  Your
incremental deployment plan (extension module, possibly in collections
for 2.6, go from there) may be a good way to evaluate how the
community as a whole feels.

> You've come a long way since then.  Congrats.
>
> Thanks for your contribution and thoughtful discussion.  This weekend, I look forward to reading you source code in detail.

Thanks!

I've just adding my Python prototype implementation to the .tar.gz.
It has a lengthy text section at the top that describes the structure
of the BList and the invariants that it must maintain.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From collinw at gmail.com  Wed Apr 25 19:05:25 2007
From: collinw at gmail.com (Collin Winter)
Date: Wed, 25 Apr 2007 10:05:25 -0700
Subject: [Python-3000] Suggested PEP 3108 addition
Message-ID: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com>

I'd like to propose that the bsddb185 module be added to PEP 3108's
list of modules slated for removal. Reasons to kill it:

- The module isn't built by default; manual tweaking of setup.py is required.

- The following scare-paragraph is included at the bottom of
http://docs.python.org/lib/module-bsddb.html:

"""
Beginning in 2.3 some Unix versions of Python may have a bsddb185
module. This is present only to allow backwards compatibility with
systems which ship with the old Berkeley DB 1.85 database library. The
bsddb185 module should never be used directly in new code.
"""

Collin Winter

From jyasskin at gmail.com  Wed Apr 25 19:09:45 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 25 Apr 2007 10:09:45 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<20070425085350.6408.JCARLSON@uci.edu>
Message-ID: <5d44f72f0704251009w12eddd82l971f5d06b35b22fb@mail.gmail.com>

[bcc'ing numpy-discussion. Comments should probably try to stay on the
python-3000 list.]

On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> Are the base number operations in Python all that difficult to
> understand?  Do we really need to add mathematical formalism into
> Python's type system before people understand the meaning of X * Y?
> Because all I really see this doing is confusing the hell out of people
> who aren't mathematicians; I'm a theoretical computer scientist and I
> had to try to think for a moment to verify that all of those were really
> necessary to separate the cases.
>
> I really do understand wanting to be able to ask "is operation X
> supported for values Y and Z", but is this really necessary for numbers?

If you don't want to use the full power of the system, nothing forces
you to. If someone doesn't know what a Ring or a complex number is,
they can just check for Real and rely on the same operations, at the
small cost of overconstraining their inputs. One reason I'm suggesting
splitting the classes into the "algebra" and "numbers" modules is so
that people who don't know or care about the algebra can rely on just
"numbers" and still get nearly all of the benefits.

However, I think it's important to provide a framework for people who
want to do more powerful things. If you write a function that can take
an arbitrary ring, it should Just Work on a python int, or any other
Complex subclass, without you having to check for a lot of operations
separately. The Haskell98 numerical hierarchy actually does start at
Num, ignoring the algebraic structures underneath, and it draws a lot
of criticism for it.

That does mean that the names and documentation in the numbers module
need to be as clear as we can make them for non-mathematical
audiences. I'm certainly open to suggestions on that front.

-- 
Namast?,
Jeffrey Yasskin

From jason.orendorff at gmail.com  Wed Apr 25 19:27:31 2007
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 25 Apr 2007 13:27:31 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
Message-ID: <bb8868b90704251027j3875e6cbma9206e2cf8a4091d@mail.gmail.com>

On 4/25/07, Collin Winter <collinw at gmail.com> wrote:
> I can see use-cases for this level of formalism, but I'm a strong -1
> on making any part of the stdlib effectively off-limits for people
> without advanced math degrees. Why can't this be shipped as a
> third-party module?

I agree.  That last question says something interesting
about ABCs, though.  You and I don't care about Rings, but
some people do, and they would probably like this as a
third-party library... if it were possible.

In Haskell, you can define a typeclass (which is a lot like an ABC):

    class Ring r where
      zero :: r
      add :: r -> r -> r
      negate :: r -> r
      mul :: r -> r -> r

Then you can declare that certain existing types, even builtin
types, conform to it:

    -- tell Haskell that "the Integer type is a Ring"
    instance Ring Integer where
      zero = 0
      add x y = x + y
      negate x = -x
      mul x y = x * y

This feature makes stuff like NumericPrelude possible.

My understanding is that Guido's ABCs won't support this,
so in fact you *couldn't* ship Jeffrey's numeric classes
as a third-party library.

(My perspective is a little different from Jeffrey's.  Maybe Haskell
does draw fire for not having a ton of numeric typeclasses; I don't
know.  But it seems to me Haskell gets it Just Right: the standard
typeclasses are simple, practical, and loose enough to accomodate Int
and Float; and the language lets you roll your own crazy mathematical
mojo if that's what you like.)

-j

From ironfroggy at gmail.com  Wed Apr 25 19:37:47 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 25 Apr 2007 13:37:47 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
Message-ID: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>

On 4/25/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
> I disagree. I think that B&D-ish enforcement of ABCs is the most important
> feature of an ABC (with the option of being able to turn off enforcement on
> a per-class basis).
>
> Imagine you have a program that takes a long time to run. If you use
> duck-typing or similar ideas you may watch the program run for hours or days
> and then raise an AttributeError or NotImplemented error because a derived
> class didn't implement a required method. With B&D-ish enforcement of ABCs,
> you can automatically catch such problems as soon as the program starts.
> This is a problem that I see no other good solution to. For long running
> programs, unit-tests will also take a long time to run so the argument that
> your unit-tests should catch such errors is no longer valid. Also, while
> pylint and pychecker are great tools that could be used to catch such
> errors, they must be run manually by the programmer and generally take a
> long time to run (at least longer than the ABC implementation I! have).
>
> To reiterate, while I think the documentation and reflection benefits of
> ABCs are nice, I don't think they are nearly as essential as enforcement.
> For most of the use cases I've seen, you could just as easily read the
> documentation or the source code or try a simple example to determine how to
> implement an iterable or hashable that will play nice with other classes. In
> contrast, ABC enforcement provides the real benefit of catching errors
> before many hours of program runtime are wasted. Consequently, I'm +1 on B&D
> in this case.
>
> -Emin Martinian

It sounds like you just need better unit tests.

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From lists at cheimes.de  Wed Apr 25 20:07:03 2007
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 25 Apr 2007 20:07:03 +0200
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<20070425085350.6408.JCARLSON@uci.edu>
Message-ID: <f0o5cc$mip$1@sea.gmane.org>

Josiah Carlson wrote:
> I really do understand wanting to be able to ask "is operation X
> supported for values Y and Z", but is this really necessary for numbers?

It's not required for ordinary people who deal with ints and floats. 
Python is pushing into the scientific market. The math guys need it when 
they have to deal with vectors, matrix operation and other advanced 
stuff. I'm seeing the proposal as an important step to make Python a 
professional tool for scientific research.


From guido at python.org  Wed Apr 25 20:19:56 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 11:19:56 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
Message-ID: <ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>

On 4/25/07, Collin Winter <collinw at gmail.com> wrote:
> The following things mean absolutely nothing to me:
>
> - Monoid
> - MonoidUnderPlus
> - Group
> - Ring
> - Semiring
> - Field
>
> So, most of the terminology in the PEP.
>
> I can see use-cases for this level of formalism, but I'm a strong -1
> on making any part of the stdlib effectively off-limits for people
> without advanced math degrees. Why can't this be shipped as a
> third-party module?

As a math major I have no excuse, but I have to confess I'm also
really rusty on these things. (Nothing a quick look at wikipedia can't
refresh though.)

Jeffrey, is there any way you can drop the top of the tree and going
straight from Number to Complex -> Real -> Rational -> Integer? These
are the things that everyone with high school math will know.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Apr 25 20:33:59 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 11:33:59 -0700
Subject: [Python-3000] Fixing super anyone?
In-Reply-To: <d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<f0hkrs$aro$1@sea.gmane.org>
	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
Message-ID: <ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>

On 4/24/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> class Super(object):
>     def __init__(self, type, obj=None):
>         if isinstance(obj, Super):
>             obj = obj.__obj__
>         self.__type__ = type
>         self.__obj__ = obj
>     def __get__(self, obj, cls=None):
>         if obj is None:
>             raise Exception('only supports instances')
>         else:
>             return Super(self.__type__, obj)
>     def __getattr__(self, attr):
>         mro = iter(self.__obj__.__class__.__mro__)
>         for cls in mro:
>             if cls is self.__type__:
>                 break
>         for cls in mro:
>             if attr in cls.__dict__:
>                 x = cls.__dict__[attr]
>                 if hasattr(x, '__get__'):
>                     x = x.__get__(self, cls)
>                 return x
>         raise AttributeError, attr
>
> class autosuper(type):
>     def __init__(cls, name, bases, clsdict):
>         cls.__super__ = Super(cls)
>
> class A:
>     __metaclass__ = autosuper
>     def f(self):
>         return 'A'
>
> class B(A):
>     def f(self):
>         return 'B' + self.__super__.f()
>
> class C(A):
>     def f(self):
>         return 'C' + self.__super__.f()
>
> class D(B, C):
>     def f(self):
>         return 'D' + self.__super__.f()
>
> assert D().f() == 'DBCA'

But:

class E(D): pass

print E().f()

This prints DDBCA which surely isn't right.

Sounds like the classic bug in such attempts.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Wed Apr 25 20:41:34 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 25 Apr 2007 11:41:34 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
In-Reply-To: <eae285400704250954w4a493f55k9218bdc89802850c@mail.gmail.com>
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<00c601c786ff$95839700$f101a8c0@RaymondLaptop1>
	<eae285400704250954w4a493f55k9218bdc89802850c@mail.gmail.com>
Message-ID: <bbaeab100704251141g1cd1f720oa4c94b722beab82@mail.gmail.com>

On 4/25/07, Daniel Stutzbach <daniel at stutzbachenterprises.com> wrote:
> On 4/25/07, Raymond Hettinger <python at rcn.com> wrote:
> > In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven.
> > Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let the
> > hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be hard
> > to say no to this going into Py2.6.
>
> Will do.  What's the best way to release a module to make it easy for
> people to find, download, and install?  (a pointer to a relevant
> document would be great)
>

Create a Distutils package or egg, get it up on the Cheeseshop (PyPI),
and then send an announcement to comp.lang.python.announce.

-Brett

From brett at python.org  Wed Apr 25 20:44:44 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 25 Apr 2007 11:44:44 -0700
Subject: [Python-3000] Suggested PEP 3108 addition
In-Reply-To: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com>
References: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com>
Message-ID: <bbaeab100704251144m6a99ca5bh9339643ea52779a5@mail.gmail.com>

On 4/25/07, Collin Winter <collinw at gmail.com> wrote:
> I'd like to propose that the bsddb185 module be added to PEP 3108's
> list of modules slated for removal. Reasons to kill it:
>
> - The module isn't built by default; manual tweaking of setup.py is required.
>
> - The following scare-paragraph is included at the bottom of
> http://docs.python.org/lib/module-bsddb.html:
>
> """
> Beginning in 2.3 some Unix versions of Python may have a bsddb185
> module. This is present only to allow backwards compatibility with
> systems which ship with the old Berkeley DB 1.85 database library. The
> bsddb185 module should never be used directly in new code.
> """

That's enough by me to kill it and fits the primary desire to kill old
modules that have been replaced (in this case bsddb3).  Unless someone
speaks up I will add the module to the PEP when I get my laptop back
or if some kind soul beats me to it.

-Brett

From emin.shopper at gmail.com  Wed Apr 25 20:49:29 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 14:49:29 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
Message-ID: <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>

On 4/25/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
>
> It sounds like you just need better unit tests.


Only if you believe you can unit test everything within a few seconds. If
you believe that there exist data structures or algorithms where a
meaningful unit test takes many minutes or hours, then there is value in an
ABC enforcement mechanism.

-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/46d39207/attachment.htm 

From jcarlson at uci.edu  Wed Apr 25 21:22:07 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 25 Apr 2007 12:22:07 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
References: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
	<32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
Message-ID: <20070425121618.6417.JCARLSON@uci.edu>


"Emin.shopper Martinian.shopper" <emin.shopper at gmail.com> wrote:
> On 4/25/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> >
> > It sounds like you just need better unit tests.
> 
> Only if you believe you can unit test everything within a few seconds. If
> you believe that there exist data structures or algorithms where a
> meaningful unit test takes many minutes or hours, then there is value in an
> ABC enforcement mechanism.

If your unit tests take hours to run, then you aren't going to get
anything useful from the ABC requirements for hours either (unless you
are pre-instantiating everything you are going to use during program
execution, in which case this could become your unittest).

Please remember that this isn't about Java style type checking and
verification during compilation.  This is, strictly speaking, ABC
checking on object instantiation, method invocation, and possibly even
argument verification.  This isn't the magic bullet you are looking for
(you'll have to go to one of those static typed languages for that).


 - Josiah


From jimjjewett at gmail.com  Wed Apr 25 22:03:32 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 16:03:32 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <20070425121618.6417.JCARLSON@uci.edu>
References: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
	<32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
	<20070425121618.6417.JCARLSON@uci.edu>
Message-ID: <fb6fbf560704251303v1d08f41coa959f899a3ae3eba@mail.gmail.com>

On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:

> Please remember that this isn't about Java style type checking and
> verification during compilation.  This is, strictly speaking, ABC
> checking on object instantiation, method invocation, and possibly even
> argument verification.  This isn't the magic bullet you are looking for
> (you'll have to go to one of those static typed languages for that).

Yes and no.

With duck typing, you call the method when you're ready to use it.  At
best, he could pre-check his objects to ensure that they have the
methods he'll need later.  There will inevitably be times when the
"wrong" object slips by.

With an ABC, he gets a quick check that the object at least *claims*
to do the right thing (as opposed to having unrelated methods of the
same name), and that they have taken *some* effort to fulfill this
pledge.  The upside is that there won't be as many wrong objects
getting by.  The downside is that there will be some objects which get
wrongly rejected.  (gmpy.mpz is probably a good enough integer, even
before it gets updated to say so.)

For Ermin's use case, a few false rejections is cheaper than a few false hits.

-jJ

From jyasskin at gmail.com  Wed Apr 25 22:04:18 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 25 Apr 2007 13:04:18 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
Message-ID: <5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com>

On 4/25/07, Guido van Rossum <guido at python.org> wrote:
> Jeffrey, is there any way you can drop the top of the tree and going
> straight from Number to Complex -> Real -> Rational -> Integer? These
> are the things that everyone with high school math will know.

I think yes, if you can confirm that the
    import numbers
    class Ring(AdditiveGroup): ...
    numbers.Complex.__bases__ = (Ring,) + numbers.Complex.__bases__
hack I mention under IntegralDomain will make isinstance(int, Ring) return True.

Do you want "Number" to be equivalent to Ring+Hashable (or a
Haskell-like Ring+Hashable+__abs__+__str__)? I don't think a "Number"
class is strictly necessary since people can check for Complex or Real
directly, and one of those two is probably what they'll expect after
knowing that something's a number.

Also, I don't see much point in putting Rational between Real and
Integer. The current hierarchy is Real (int, float, Decimal, rational)
:> Integer (int) and Real :> FractionalReal (float, Decimal,
rational), but Integer and FractionalReal are just siblings.


I'm wondering how many people are writing new numeric types who don't
know the abstract algebra. I think we should be able to insulate
people who are just using the types from the complexities underneath,
and the people writing new types will benefit. Or will seeing "Ring"
in a list of superclasses be enough to confuse people?

-- 
Namast?,
Jeffrey Yasskin

From guido at python.org  Wed Apr 25 22:14:17 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 13:14:17 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
Message-ID: <ca471dc20704251314o1ad32995tbce975c8579b4211@mail.gmail.com>

On 4/25/07, Paul Moore <p.f.moore at gmail.com> wrote:
> On 25/04/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
> > I disagree. I think that B&D-ish enforcement of ABCs is the most important
> > feature of an ABC (with the option of being able to turn off enforcement on
> > a per-class basis).
>
> OK. We can agree to disagree. My point was that as far as I am aware,
> Guido's current position on ABCs is that they are, and will remain,
> optional - people like myself (and Greg, from the sound of it) who
> don't want to derive from ABCs will not be penalised by being excluded
> from anything in the Python core or the stdlib.
>
> Example - I recall Guido saying explicitly that there will be no
> requirement for objects assigned to sys.stdout to derive from any of
> the IO ABCs.

Yes, that's actually in the PEP.

But while I don't like Emin Shopper's argument (it sounds like too
much like the classic static type checking argument), I am not about
to remove the @abstractmethod decorator or its semantics (that it must
be overridden in order to be allowed to instantiate the class). You
don't need to use it (and I recommend thinking twice before you use
it).

But it makes sense in some cases to insist that someone who goes
through the trouble of inheriting from an ABC gets an error if they
forget to implement one of the methods that are deemed 'essential' to
that ABC.

Concerning the use case where you'd like to claim to implement an ABC
but happen to know that a particular method will never be called, I
suspect that this is the exception of ABC usage rather than the common
case, so I'd rather cater to the common case. ESNPS.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tcdelaney at optusnet.com.au  Wed Apr 25 22:29:43 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Thu, 26 Apr 2007 06:29:43 +1000
Subject: [Python-3000] Fixing super anyone?
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com><f0hkrs$aro$1@sea.gmane.org><9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com><f0j77k$nj5$1@sea.gmane.org><fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com><76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com><d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com><76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com><d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com><d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
Message-ID: <002401c78778$75fb7eb0$0201a8c0@ryoko>

From: "Guido van Rossum" <guido at python.org>

> class E(D): pass
>
> print E().f()
>
> This prints DDBCA which surely isn't right.
>
> Sounds like the classic bug in such attempts.

Yep - missing overridden methods tend to either do the above, or end up in 
infinite recursion. My bytecode hacking version doesn't suffer from these 
problems ...

>>> class autosuper(object):
...     __metaclass__ = _autosuper
...
>>> class A(autosuper):
...     def f(self):
...         print 'A:', super
...
>>> class B(A):
...     def f(self):
...         print 'B:', super
...         super.f()
...
>>> class C(A):
...     def f(self):
...         print 'C:', super
...         super.f()
...
>>> class D(B, C):
...     pass
...
>>> class E(D):
...     def f(self):
...         print 'E:', super
...         super.f()
...
>>> class F(E, A):
...     pass
...
>>> F().f()
E: <super: <class 'E'>, <F object>>
B: <super: <class 'B'>, <F object>>
C: <super: <class 'C'>, <F object>>
A: <super: <class 'A'>, <F object>>

What I haven't worked out yet is if you should be able to do the following:

    class A(autosuper):
        def f(self):
            print 'A:', super

    class B(A):
        def f(self):
            def inner():
                print 'B:', super
                super.f()
            inner()

Should the call to inner() result in a call to A.f? Currently my bytecode 
version doesn't do this. I think this should be addressed in the PEP, as 
well as my proposal to have super(args) work as super.func(args) when called 
inside func.

Tim Delaney 


From emin.shopper at gmail.com  Wed Apr 25 22:44:49 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 16:44:49 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <20070425121618.6417.JCARLSON@uci.edu>
References: <76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
	<32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
	<20070425121618.6417.JCARLSON@uci.edu>
Message-ID: <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>

On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:
>
>
> If your unit tests take hours to run, then you aren't going to get
> anything useful from the ABC requirements for hours either (unless you
> are pre-instantiating everything you are going to use during program
> execution, in which case this could become your unittest).


No, the ABC requirements are checked as soon as a module is imported (i.e.,
classes are checked at definition time not when an instance is created). I
have been using my own ABC implementation for quite a while and it has
repeatedly proven it's value when I create a new derived class and forget to
implement a necessary method.

Please remember that this isn't about Java style type checking and
> verification during compilation.  This is, strictly speaking, ABC
> checking on object instantiation, method invocation, and possibly even
> argument verification.  This isn't the magic bullet you are looking for
> (you'll have to go to one of those static typed languages for that).


Again, I disagree. With metaclasses it is easy and useful to have checking
at class definition (see
http://web.mit.edu/~emin/www/source_code/py_abc/abc.py for an example of
what I mean). Generally I don't want static checking, but occasionally it is
nice to have something like that and ABCs provide a good way to do that.

I'm not claiming that is the only benefit of ABCs. I'm just pointing out
what I find to be a very useful feature of ABCs that I hope makes into
python.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/43c2b864/attachment.htm 

From guido at python.org  Wed Apr 25 22:58:45 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 13:58:45 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com>
Message-ID: <ca471dc20704251358m6c06dddfga9f300b2aa1b4550@mail.gmail.com>

On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On 4/25/07, Guido van Rossum <guido at python.org> wrote:
> > Jeffrey, is there any way you can drop the top of the tree and going
> > straight from Number to Complex -> Real -> Rational -> Integer? These
> > are the things that everyone with high school math will know.
>
> I think yes, if you can confirm that the
>     import numbers
>     class Ring(AdditiveGroup): ...
>     numbers.Complex.__bases__ = (Ring,) + numbers.Complex.__bases__
> hack I mention under IntegralDomain will make isinstance(int, Ring) return True.

Hm... That would require built-in types to be mutable (by mere
mortals) which would kill the idea of multiple interpreters. AFAIK the
only user of multiple interpreters is mod_python, which is a pretty
popular Apache module... (Google claims 1.8M hits, vs. 5.66M for
mod_perl, the market leader.)

I guess the alternative would be to keep the mathematical ones in but
make it so that mere mortals (among which I count myself :) never need
to know about them. Sort of like metaclasses, for most people.

But this still presents a bit of a problem. If we insist that built-in
types are immutable, whenever a mathematician needs a new kind of
algebraic category (you mention a few in the PEP), they have no way to
declare that int or float are members of that category. They could
write things like

class MyFloat(float, DivisionRing): pass

but this would be fairly painful. I would like to let users modify the
built-ins, but such changes ought to be isolated from other
interpreters running in the same address space. I don't want to be the
one to tell the mod_python community they won't be able to upgrade to
3.0...

> Do you want "Number" to be equivalent to Ring+Hashable (or a
> Haskell-like Ring+Hashable+__abs__+__str__)? I don't think a "Number"
> class is strictly necessary since people can check for Complex or Real
> directly, and one of those two is probably what they'll expect after
> knowing that something's a number.

Fair enough, you could start with Complex.

> Also, I don't see much point in putting Rational between Real and
> Integer. The current hierarchy is Real (int, float, Decimal, rational)
> :> Integer (int) and Real :> FractionalReal (float, Decimal,
> rational), but Integer and FractionalReal are just siblings.

Why? Why not have a properfraction() on Integer that always returns 0
for the fraction? The current math.modf() works just fine on ints
(returning a tuple of floats).

And why distinguish between Real and FractionalReal? What's the use
case for reals that don't know how to compute their fractional part?

> I'm wondering how many people are writing new numeric types who don't
> know the abstract algebra.

It's easy to know the algebra without recalling the terminologies.
Lots of programmers (myself included!) are largely self-taught.

> I think we should be able to insulate
> people who are just using the types from the complexities underneath,
> and the people writing new types will benefit. Or will seeing "Ring"
> in a list of superclasses be enough to confuse people?

Shouldn't only mathematicians be tempted to write classes deriving
from Ring? Everyone else is likely to subclass one of the more
familiar types, from Complex onwards.

PS. I've checked your PEP in, as PEP3141. We might as well keep a
fairly complete record in svn, even if we decide to cut out the
algebraic foundational classes.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Wed Apr 25 23:11:47 2007
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 25 Apr 2007 14:11:47 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
Message-ID: <20070425140134.641D.JCARLSON@uci.edu>


"Emin.shopper Martinian.shopper" <emin.shopper at gmail.com> wrote:
> On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > If your unit tests take hours to run, then you aren't going to get
> > anything useful from the ABC requirements for hours either (unless you
> > are pre-instantiating everything you are going to use during program
> > execution, in which case this could become your unittest).
> 
> No, the ABC requirements are checked as soon as a module is imported (i.e.,
> classes are checked at definition time not when an instance is created). I
> have been using my own ABC implementation for quite a while and it has
> repeatedly proven it's value when I create a new derived class and forget to
> implement a necessary method.

But that is not part of the ABC PEP (according to my most recent reading).
See the "ABC Support Framework" portion.  Only instantiation triggers
the "you didn't implement method X". Certainly you can add metaclasses
to handle checking for "does class X implement method Y", but it isn't
going to be included out of the box.

> I'm not claiming that is the only benefit of ABCs. I'm just pointing out
> what I find to be a very useful feature of ABCs that I hope makes into
> python.

I doubt this particular feature will make it in.  It seems too
"B&D"-static typing to me (and others).


 - Josiah


From guido at python.org  Wed Apr 25 23:19:27 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 14:19:27 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <20070425140134.641D.JCARLSON@uci.edu>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
Message-ID: <ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>

On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> "Emin.shopper Martinian.shopper" <emin.shopper at gmail.com> wrote:
> > On 4/25/07, Josiah Carlson <jcarlson at uci.edu> wrote:
> > > If your unit tests take hours to run, then you aren't going to get
> > > anything useful from the ABC requirements for hours either (unless you
> > > are pre-instantiating everything you are going to use during program
> > > execution, in which case this could become your unittest).
> >
> > No, the ABC requirements are checked as soon as a module is imported (i.e.,
> > classes are checked at definition time not when an instance is created). I
> > have been using my own ABC implementation for quite a while and it has
> > repeatedly proven it's value when I create a new derived class and forget to
> > implement a necessary method.
>
> But that is not part of the ABC PEP (according to my most recent reading).
> See the "ABC Support Framework" portion.  Only instantiation triggers
> the "you didn't implement method X". Certainly you can add metaclasses
> to handle checking for "does class X implement method Y", but it isn't
> going to be included out of the box.
>
> > I'm not claiming that is the only benefit of ABCs. I'm just pointing out
> > what I find to be a very useful feature of ABCs that I hope makes into
> > python.
>
> I doubt this particular feature will make it in.  It seems too
> "B&D"-static typing to me (and others).

IIUC, Emin's framework requires you to explicitly declare any class as
abstract that has abstract methods. IMO this is a requirement to
redundancy which I find unpythonic. So indeed I am not about to add
this to the PEP.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Wed Apr 25 23:23:25 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 17:23:25 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
Message-ID: <fb6fbf560704251423qaac984cv6d81821f660b8940@mail.gmail.com>

On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
>     class MonoidUnderPlus(Abstract):

Is this useful?  Just because two things are both Monoid instances
doesn't mean I can add them -- they have to be part of the same
Monoid.  By the time you do

    assert isinstance(a, MonoidUnderPlus)
    assert isinstance(b, MonoidUnderPlus)
    assert isinstance(a, b.__class__) or isinstance(b, a.__class__)

I'm not sure how much you've really saved.

> **Open issue:** Do we want to give people a choice of which of the
> following to define, or should we pick one arbitrarily?::

>         def __neg__(self):
>             """Must define this or __sub__()."""
>             return self.zero() - self

>         def __sub__(self, other):
>             """Must define this or __neg__()."""
>             return self + -other

Probably better to pick one;  then it is clear that they have to
override something, and there won't be as many accidental infinite
loops.  If they really want to define the other, they can just do

    def __neg__(self):
        super(_this_class__, self).__neg__()

The decorator doesn't know it isn't a real override.

-jJ

From emin.shopper at gmail.com  Wed Apr 25 23:56:13 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 17:56:13 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
Message-ID: <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>

On 4/25/07, Guido van Rossum <guido at python.org> wrote:

> IIUC, Emin's framework requires you to explicitly declare any class as
> abstract that has abstract methods.


I believe that is an easily fixable implementation issue, which I would be
happy to remedy.

I think a bigger issue is whether to check at definition time or check at
instantiation time. It seems to me the former has the benefit of catching
errors earlier and potentially saving time. I don't immediately see the
advantage of the latter (probably that is due to my lack of imagination).

I would like to see the PEP include the ability to (optionally) enforce ABC
requirements at definition time. For example, you could set an attribute (
e.g., __check_abstract__) or inherit from something special if you want to
turn enforcement on (if not checking at def time is the default) or turn it
off (if checking at def time is the default). I am not recommending that
anyone be forced to conform to this to use anything in the standard library.


I think this would be useful, not conflict with the other goals of the PEP,
and be relatively easy to implement in pure python (as illustrated by my
example implementation). If others find this useful (or at least not
objectionable), I would be happy to help with the implementation or writing
additional material for the PEP or even propose it separately if people
think this distracts from the point of the original ABC PEP.

Thanks,
-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/ebfe0ef7/attachment.html 

From jimjjewett at gmail.com  Thu Apr 26 00:10:23 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 18:10:23 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
Message-ID: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>

The current ABC proposal is to use isinstance as the test; Jeffrey
Yaskin's numbers PEP highlighted the weakness there with a concrete
example.

If you need to an abstraction less powerful than an existing ABC,
you're out of luck; you can't just assert that the existing class is
already sufficient, nor can you expect everyone else to use multiple
annotations.

Short of allowing more __bases__ surgery, we need a function parallel
to isinstance, which at least makes 3rd-party registration possible.
I suspect Phillip will say that we really need to make the ABCs
generic functions... but I disagree; I'm not sure that 3rd-party
adapters should even be allowed by default, but it *should* be simple
to create an ABC that does take them.

Perhaps

def isexample(obj, ABC):
    for cls in obj.__class__.__mro__:
        result = ABC.meets(obj, cls)
        if result:
            return result
    return False

class Abstract...

    # override this with a dictionary to allow 3rd-party registration
    _good_enough = ()

    @classmethod
    def meets(cls, obj=None, objclass=None):
        if objclass is cls:   # covers isinstance
            return obj
        if objclass in cls._good_enough:   # Nothing is, by default
            return cls._good_enough.[objclass](obj)

    @classmethod
    def assert_sufficient(cls, objclass, adapter):
        cls[objclass]=adapter


On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:

> If someone needs to split them later, they can use code like::
>     import numbers
>     class IntegralDomain(Ring): ...
>     numbers.Integral.__bases__ = (IntegralDomain,) + numbers.Integral.__bases__

This only works with old-style classes, which are going away.

>>> class Abstract1(object): pass
>>> class Abstract2(object): pass
>>> Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__

Traceback (most recent call last):
  File "<pyshell#33>", line 1, in <module>
    Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__
TypeError: __bases__ assignment: 'Abstract2' deallocator differs from 'object'

From jimjjewett at gmail.com  Thu Apr 26 00:20:18 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 18:20:18 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
Message-ID: <fb6fbf560704251520w6647a4d5ub6c858db07d26a24@mail.gmail.com>

On 4/25/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:

> I think a bigger issue is whether to check at definition time or check at
> instantiation time. It seems to me the former has the benefit of catching
> errors earlier and potentially saving time. I don't immediately see the
> advantage of the latter (probably that is due to my lack of imagination).

The advantage of waiting is that you can *have* (partially) abstract
classes.  So long as you never instantiate them directly (super calls
are OK), everything is fine.

If you want an earlier check, then just create an instance of your
class at import time (or, alternatively, in your unit tests).  If it
can be created, it passed the test.

-jJ

From guido at python.org  Thu Apr 26 00:27:48 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 15:27:48 -0700
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <fb6fbf560704212120u16061536wc25f269a1dd87407@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
	<fb6fbf560704212120u16061536wc25f269a1dd87407@mail.gmail.com>
Message-ID: <ca471dc20704251527h74fa558bmd2fd2db15e87a61@mail.gmail.com>

(Forgot to answer this promptly.)

On 4/21/07, Jim Jewett <jimjjewett at gmail.com> wrote:
[PEP 3119]
> > > * "Should we also implement the issubset and issuperset methods found
> > > on the set type in Python 2? As these are just aliases for __le__ and
> > > __ge__, I'm tempted to leave these out."
[Brett]
> > > Leave them out.  Not terribly needed plus it is better to start out
> > > small.  They can easily be added later if called for.
>
> I think the names are more sensible than repurposing the numeric
> operators.  The "concrete" implementation can just forward to other
> name, so it doesn't cost much.

But the numeric operators are what everybody uses. So it makes more
sense to define these, and only these. It probably isn't even
necessary to implement the 'r' variants (__ror__ etc.) since these
only work if the other argument is also a Set.

[Guido]
> > I doubt that 'set' will be inherited from much (except for trivial
> > stuff) but returning a 'set' does make some sense. Although for
> > trivial subclasses (e.g. adding a different repr or a brand new
> > method) you'd like the results also to be an instance of the subclass.
> > If we could find a way to do that it would be best.
>
> You could use a variant of the numeric operator resolution.
>
> The left class wins, unless the right class is a subclass of the left.
> If this class does not have a constructor that accepts an iterable,
> (maybe fall back to the other class, then) fall back to a builtin set.

The problem isn't which side wins. The problem is what API to use to
create a new set. Since this is an *abstract* class, it's wrong to
just calling self.__class__() and assume its constructor takes an
iterable of values.

> In the Hashable section:
>
> > Another constraint is that hashable objects, once created,
> > should never change their value (as compared by ==) or their hash
> > value. If a class cannot guarantee this, it should not derive from
> > Hashable; if it cannot guarantee this for certain instances only,
> > __hash__ for those instances should raise a TypeError exception.
>
> Why not just return -1 (unless/until the value is stable)?  Is the -1
> special case being phased out too?

I don't recall that -1 was ever a special case at the Python level;
that was only done at the C level to make error checking easier.
Returning any hash value at all implies a guarantee that the object
value won't ever change (not just the hash value!); if you can't
guarantee that (e.g. for hash(([], []))) then you shouldn't return a
hash value. I don't think that hashing all objects with an unstable
value together on -1 supports a valid use case.

> ...
> I would say that Searchable might be worth separating into a Container
> subclass, but don't worry about the speed differences.  Many
> containers are small enough that constant effects are more important
> than big-O.  When big-O does matter, then the user may well want to
> distinguish between the O(1) and O(log N) case.  I think this gets
> detailed enough that it isn't worth putting in the common library.

Right. What is your specific proposal here? Adding
Searchable(Container) with the implication that __contains__ takes a
sequence as well as a single value? How should the type of the
argument be described? Should it be a Sequence? a Searchable?
something that is an instance of self.__class__?

> Why must a HashableSet or MutableSet be composable?  If this is
> because you figure any useful set has those properties (and I don't
> think so, when doing uniquification), then Set and ComposableSet
> should become BasicSet and Set?

Hmm, good point! This is in fact listed as an open issue for
HashableSet. If we can find a good use case for non-composable sets
that should nevertheless be hashable, we may end up with four classes:
set, composable set, hashable set, composable hashable set. (I think
you're hinting at an example; can you work it out a bit more?) For
mutable sets I think the case is weaker; these already support
in-place operations (|= etc.) which can easily and (usually
efficiently) be implemented using the primitives add() and discard(),
and it would be somewhat weird to have |= without |.

> ...
>
> Note that in the current sandbox  sandbox (which doesn't have
> *Ordered), the name BasicSet is used for a Set which isn't Sized.
> (Except that Set doesn't actually inherit from BasicSet, as I assume
> it should do.)

The sandbox/abc/abc.py code is way out of date. I'll try to update it.

> Continuing with the current sandbox
>
> Why must @abstractmethod come last, when mixed with other decorators?

Because it sets a function attribute. It is conceivable that other
decorators would make funtion attributes readonly. However the example
given is currently wrong -- @classmethod doesn't pass on function
attributes at all!

> IterableMapping needs to override __iter__; as written, it returns the
> empty iterator.

No, it is still an abstract method.

> Should Sequence.index take optional start and stop arguments?
>
>     def index(self, value, start=0, stop=None):
>         if stop is None:
>             stop = len(self)
>         for i, elem in enumerate(self):
>             if i < start:
>                 continue
>             if stop < i:
>                 break
>             if elem == value:
>                 return i
>         raise ValueError

Good question. I'm tempted to say no, and consider that a
list-specific extension.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From emin.shopper at gmail.com  Thu Apr 26 00:32:35 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 18:32:35 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <fb6fbf560704251520w6647a4d5ub6c858db07d26a24@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
	<fb6fbf560704251520w6647a4d5ub6c858db07d26a24@mail.gmail.com>
Message-ID: <32e43bb70704251532o6370254cyf574d0d3c2b19465@mail.gmail.com>

On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> On 4/25/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
>
> > I think a bigger issue is whether to check at definition time or check
> at
> > instantiation time. It seems to me the former has the benefit of
> catching
> > errors earlier and potentially saving time. I don't immediately see the
> > advantage of the latter (probably that is due to my lack of
> imagination).
>
> The advantage of waiting is that you can *have* (partially) abstract
> classes.  So long as you never instantiate them directly (super calls
> are OK), everything is fine.


Good point. However, you can do this without precluding def time checks. For
example, you can have any class which has abstract methods not be forced to
satisfy parent ABC rules since it is still abstract, or you can provide
another way to mark the class as partially abstract (my initial
implemenation does this by having the partially abstract class inherit from
both the parent abstract class and the class that all abstract classes
inherit from).

You could reasonably argue that the benefit of def time enforcement is
outweighed by the need to do something special for a class that inherits
from an abstract base class but has no abstract methods itself.

If you want an earlier check, then just create an instance of your
> class at import time (or, alternatively, in your unit tests).  If it
> can be created, it passed the test.


Yes, I could do that assuming that the inputs to the class __init__ method
can be constructed quickly. Personally, I would find it preferable to have
to do something slightly different for partially abstract classes with no
abstract methods, but I guess it's a matter of taste.

-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/f6271f55/attachment.html 

From exarkun at divmod.com  Thu Apr 26 00:40:25 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 25 Apr 2007 18:40:25 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
Message-ID: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm>

On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <jimjjewett at gmail.com> wrote:
>The current ABC proposal is to use isinstance as the test; Jeffrey
>Yaskin's numbers PEP highlighted the weakness there with a concrete
>example.
>
>If you need to an abstraction less powerful than an existing ABC,
>you're out of luck; you can't just assert that the existing class is
>already sufficient, nor can you expect everyone else to use multiple
>annotations.

I'm sure everyone is already aware of the behavior of the classImplements
and directlyProvides functions available in zope.interface, which exactly
satisfy this use-case in the interface world.

Jean-Paul

From jimjjewett at gmail.com  Thu Apr 26 00:48:13 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 18:48:13 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704251358m6c06dddfga9f300b2aa1b4550@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<5d44f72f0704251304o15f2e080wcb9ade83be56403@mail.gmail.com>
	<ca471dc20704251358m6c06dddfga9f300b2aa1b4550@mail.gmail.com>
Message-ID: <fb6fbf560704251548peaa4234h76a99c6637aaeb36@mail.gmail.com>

On 4/25/07, Guido van Rossum <guido at python.org> wrote:

> ... If we insist that built-in types are immutable ... They
> could write things like

> class MyFloat(float, DivisionRing): pass

> but this would be fairly painful.

Worse than that once you get 3rd parties involved.

try:
   from gmpy import mpf
except ImportError:
   class mpf(object): pass

class MyFloat(float, DivisionRing, gmpy): pass
    # err ... will this even work at all, with two concrete types?

> On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> > will seeing "Ring"
> > in a list of superclasses be enough to confuse people?

Yes.  That is part of the problem with twisted -- there are so many
layers of abstraction that it takes a while to find something.  If
users ever see "Ring" outside of a footnote or the advanced math
section, then something went wrong.

-jJ

From guido at python.org  Thu Apr 26 01:32:03 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 16:32:03 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
Message-ID: <ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>

On 4/25/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
> On 4/25/07, Guido van Rossum <guido at python.org> wrote:
>
> > IIUC, Emin's framework requires you to explicitly declare any class as
> > abstract that has abstract methods.
>
> I believe that is an easily fixable implementation issue, which I would be
> happy to remedy.
>
> I think a bigger issue is whether to check at definition time or check at
> instantiation time. It seems to me the former has the benefit of catching
> errors earlier and potentially saving time. I don't immediately see the
> advantage of the latter (probably that is due to my lack of imagination).

I'm confused. Can you show a simple example of something that can be
checked at definition time? Using the machinery currently in the PEP,
any class that inherits an abstract method and doesn't override it is
automatically an abstract class itself. There's nothing to complain
about at definition time.

> I would like to see the PEP include the ability to (optionally) enforce ABC
> requirements at definition time. For example, you could set an attribute
> (e.g., __check_abstract__) or inherit from something special if you want to
> turn enforcement on (if not checking at def time is the default) or turn it
> off (if checking at def time is the default). I am not recommending that
> anyone be forced to conform to this to use anything in the standard library.

So I'm confused about what you can enforce. (And I do hope you will
read the PEP before replying.)

> I think this would be useful, not conflict with the other goals of the PEP,
> and be relatively easy to implement in pure python (as illustrated by my
> example implementation). If others find this useful (or at least not
> objectionable), I would be happy to help with the implementation or writing
> additional material for the PEP or even propose it separately if people
> think this distracts from the point of the original ABC PEP.

Your example implementation was a bit too much code for me to read.
:-) Please show some usage examples instead.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 26 01:41:40 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 16:41:40 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<20070425224025.19381.1937422068.divmod.quotient.5365@ohm>
Message-ID: <ca471dc20704251641ra605566j7e28c5b16d5a942c@mail.gmail.com>

On 4/25/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> I'm sure everyone is already aware of the behavior of the classImplements
> and directlyProvides functions available in zope.interface, which exactly
> satisfy this use-case in the interface world.

Don't be so sure; I wasn't (aware, that is). At this point it doesn't
look like there will be an Interfaces PEP. If you want one, or if you
want to borrow specific ideas from zope.interface, now would be the
time to bring them up and enlighten the rest of us about them.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 26 01:43:45 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 16:43:45 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
Message-ID: <ca471dc20704251643x8d966b5w4ddeaad265aefe9a@mail.gmail.com>

This is a very good point. Perhaps we can come up with a way to make
isinstance and issubclass into something like GFs (without requiring
the whole GF machinery).

I'll think about it some more.

--Guido

On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> The current ABC proposal is to use isinstance as the test; Jeffrey
> Yaskin's numbers PEP highlighted the weakness there with a concrete
> example.
>
> If you need to an abstraction less powerful than an existing ABC,
> you're out of luck; you can't just assert that the existing class is
> already sufficient, nor can you expect everyone else to use multiple
> annotations.
>
> Short of allowing more __bases__ surgery, we need a function parallel
> to isinstance, which at least makes 3rd-party registration possible.
> I suspect Phillip will say that we really need to make the ABCs
> generic functions... but I disagree; I'm not sure that 3rd-party
> adapters should even be allowed by default, but it *should* be simple
> to create an ABC that does take them.
>
> Perhaps
>
> def isexample(obj, ABC):
>     for cls in obj.__class__.__mro__:
>         result = ABC.meets(obj, cls)
>         if result:
>             return result
>     return False
>
> class Abstract...
>
>     # override this with a dictionary to allow 3rd-party registration
>     _good_enough = ()
>
>     @classmethod
>     def meets(cls, obj=None, objclass=None):
>         if objclass is cls:   # covers isinstance
>             return obj
>         if objclass in cls._good_enough:   # Nothing is, by default
>             return cls._good_enough.[objclass](obj)
>
>     @classmethod
>     def assert_sufficient(cls, objclass, adapter):
>         cls[objclass]=adapter
>
>
> On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
>
> > If someone needs to split them later, they can use code like::
> >     import numbers
> >     class IntegralDomain(Ring): ...
> >     numbers.Integral.__bases__ = (IntegralDomain,) + numbers.Integral.__bases__
>
> This only works with old-style classes, which are going away.
>
> >>> class Abstract1(object): pass
> >>> class Abstract2(object): pass
> >>> Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__
>
> Traceback (most recent call last):
>   File "<pyshell#33>", line 1, in <module>
>     Abstract1.__bases__ = (Abstract2,) + Abstract1.__bases__
> TypeError: __bases__ assignment: 'Abstract2' deallocator differs from 'object'
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Thu Apr 26 02:19:03 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 20:19:03 -0400
Subject: [Python-3000] Need help completing ABC pep
In-Reply-To: <ca471dc20704251527h74fa558bmd2fd2db15e87a61@mail.gmail.com>
References: <ca471dc20704191555t3a1140f0jd92e3f146148317f@mail.gmail.com>
	<bbaeab100704191841k64be8fd2h9cfed219e0e73e23@mail.gmail.com>
	<ca471dc20704191935r7bd19fd1w8dc5e51f6a97df39@mail.gmail.com>
	<fb6fbf560704212120u16061536wc25f269a1dd87407@mail.gmail.com>
	<ca471dc20704251527h74fa558bmd2fd2db15e87a61@mail.gmail.com>
Message-ID: <fb6fbf560704251719h2f489068y51219d746953054f@mail.gmail.com>

On 4/25/07, Guido van Rossum <guido at python.org> wrote:
> On 4/21/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> [PEP 3119]
> > > > * "Should we also implement the issubset and issuperset methods found
> > > > on the set type in Python 2? As these are just aliases for __le__ and
> > > > __ge__, I'm tempted to leave these out."
> [Brett]
> > > > Leave them out.  Not terribly needed plus it is better to start out
> > > > small.  They can easily be added later if called for.

> > I think the names are more sensible than repurposing the numeric
> > operators.  The "concrete" implementation can just forward to other
> > name, so it doesn't cost much.

> But the numeric operators are what everybody uses. So it makes more
> sense to define these, and only these.

ahh... because you want to ensure that implementors override the correct one.
If you defined

    def union(self, other):  return self.__or__(other)

and I overrode union but not __or__, things would be inconsistent.
But that's still true even if you don't define union in the ABC.
Maybe add a sanity check in the actually-doing-something default
implementation?

    def __or__(self, other):
        # If they overrode union but not __or__, honor the override.
        if self.union is not __this_class__.union:
            return self.union(other)
        new = set(self)
        new.update(other)
        return frozenset(new)


> > In the Hashable section:

> > > Another constraint is that hashable objects, once created,
> > > should never change their value (as compared by ==) or their hash
> > > value. If a class cannot guarantee this, it should not derive from
> > > Hashable; if it cannot guarantee this for certain instances only,
> > > __hash__ for those instances should raise a TypeError exception.

> > Why not just return -1 (unless/until the value is stable)?  Is the -1
> > special case being phased out too?

> I don't recall that -1 was ever a special case at the Python level;
> that was only done at the C level to make error checking easier.
> Returning any hash value at all implies a guarantee that the object
> value won't ever change (not just the hash value!); if you can't
> guarantee that (e.g. for hash(([], []))) then you shouldn't return a
> hash value. I don't think that hashing all objects with an unstable
> value together on -1 supports a valid use case.

If you want to enforce that, it is worth documenting.  As nearly as I
can tell, hash is used only by dict and set, and they always treat a
hash of -1 as unhashable, so it won't get into the dictionary (or
set).

To me, this suggested it was OK to change value (if both values have
the same hash), and it was OK to (once per object) change hash from -1
to a real value.

I have used this for objects where:
    (a)  The most significant attributes are immutable and known at creation
    (b)  Additional attributes are not known at creation, so they may
change from "None" to a value.
    (c)  Equality depends on all attributes, and therefore could --
theoretically -- change.
    (d)  In practice, several objects might pass through the same
(partially completed) value, but only one would be in that state at a
time.  (In other words, the result of a==b was stable, even though the
result of a<b wasn't.)

I based the hash on only the known-at-creation time attributes; if
they were unequal, than the objects could never be equal, which was
all hash needed to tell me.  If the differences were in the later
attributes -- oh well; I got some extra hash collisions.

> What is your specific proposal here? Adding
> Searchable(Container) with the implication that __contains__ takes a
> sequence as well as a single value?

Yes.

> How should the type of the argument be described?

Sequence would be fine if you want to leave yourself room to change
your mind later.  I wouldn't bother to describe it at all.

    def __contains__(self, item):
        atomic = super(Searchable, self).__contains__(item)  # or __this_class__
        if atomic:
            return atomic
        try:
            it = iter(item)
        except TypeError:
            return False # It isn't iterable, so give up
        # Note that this doesn't assume Sequence.  It would say "bbbbaa" in "ab"
        return all(super(Searchable, self).__contains__(elt) for elt in it)


> > Why must a HashableSet or MutableSet be composable?  If this is
> > because you figure any useful set has those properties (and I don't
> > think so, when doing uniquification), then Set and ComposableSet
> > should become BasicSet and Set?

> Hmm, good point! This is in fact listed as an open issue for
> HashableSet. If we can find a good use case for non-composable sets
> that should nevertheless be hashable, we may end up with four classes:
> set, composable set, hashable set, composable hashable set. (I think
> you're hinting at an example; can you work it out a bit more?)

Yes, but not immediately.  :D

> > Should Sequence.index take optional start and stop arguments?

> Good question. I'm tempted to say no, and consider that a
> list-specific extension.

It is also a pretty important string method, unless it has been
deprecated for find.

-jJ

From pje at telecommunity.com  Thu Apr 26 02:31:10 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 25 Apr 2007 20:31:10 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20070425202858.04ee1ee8@sparrow.telecommunity.com>

At 06:10 PM 4/25/2007 -0400, Jim Jewett wrote:
>I suspect Phillip will say that we really need to make the ABCs
>generic functions...

Nope; they're *already* generic functions: hash(), len(), iter(), various 
operator.* functions, and I believe we're adding next().  I've got nothing 
to add to that list.  :)


From jimjjewett at gmail.com  Thu Apr 26 02:31:39 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 25 Apr 2007 20:31:39 -0400
Subject: [Python-3000] ABC PEP -- dropping set.clear
Message-ID: <fb6fbf560704251731p57b26c5eoe60c606009dfc501@mail.gmail.com>

On 4/25/07, guido.van.rossum <python-checkins at python.org> wrote:
> +        implementation.)  **Open issues:** Forcing every mutable set
> +        to implement this may be a pain for such a fairly
> +        non-essential method.  Perhaps just drop it?

There are also comments worrying that an abstract implementation would
be horribly slow -- but so what?

Alex Martelli posted some stats (for dicts, I think) showing that
(even today) clear was slower than just creating a new object.   It
still makes sense if you care about the "is" relation

Since you are keeping "pop", the abstract implementation can at least
approximate iteration.

    def clear(self):
        while self.pop(): pass

-jJ

From pje at telecommunity.com  Thu Apr 26 02:35:01 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 25 Apr 2007 20:35:01 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704251643x8d966b5w4ddeaad265aefe9a@mail.gmail.com
 >
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070425203455.05062ae8@sparrow.telecommunity.com>

At 04:43 PM 4/25/2007 -0700, Guido van Rossum wrote:
>This is a very good point. Perhaps we can come up with a way to make
>isinstance and issubclass into something like GFs (without requiring
>the whole GF machinery).
>
>I'll think about it some more.

__special__ methods are how Python currently defines built-in GFs, so 
perhaps __isinstance__ and __issubclass__ slots (and their __r*__ 
counterparts) should suffice.

OTOH, the old PEP 246 (implemented by both PyProtocols and zope.interface) 
just uses __conform__ and __adapt__ and basically works the same way as an 
__isinstance__/__risinstance__ pair...


From pje at telecommunity.com  Thu Apr 26 02:40:14 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 25 Apr 2007 20:40:14 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <20070425224025.19381.1937422068.divmod.quotient.5365@ohm>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070425203654.04ee17d8@sparrow.telecommunity.com>

At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote:
>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <jimjjewett at gmail.com> wrote:
> >The current ABC proposal is to use isinstance as the test; Jeffrey
> >Yaskin's numbers PEP highlighted the weakness there with a concrete
> >example.
> >
> >If you need to an abstraction less powerful than an existing ABC,
> >you're out of luck; you can't just assert that the existing class is
> >already sufficient, nor can you expect everyone else to use multiple
> >annotations.
>
>I'm sure everyone is already aware of the behavior of the classImplements
>and directlyProvides functions available in zope.interface, which exactly
>satisfy this use-case in the interface world.

I'm either misunderstanding Jim or you, because I don't see the 
relationship here.  If I understand Jim correctly, he's actually asking for 
something like PyProtocols' "protocolIsSubsetOf" declaration -- something 
that would be like dynamically adding a superclass to an existing interface 
in zope.interface.  Whereas the features you're talking about sound like 
declaring that a class object itself implements an interface -- something 
apparently unrelated to the question at hand.


From guido at python.org  Thu Apr 26 02:39:38 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 17:39:38 -0700
Subject: [Python-3000] ABC PEP -- dropping set.clear
In-Reply-To: <fb6fbf560704251731p57b26c5eoe60c606009dfc501@mail.gmail.com>
References: <fb6fbf560704251731p57b26c5eoe60c606009dfc501@mail.gmail.com>
Message-ID: <ca471dc20704251739p372b2f8ck4a7ca0568f246ee7@mail.gmail.com>

On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/25/07, guido.van.rossum <python-checkins at python.org> wrote:
> > +        implementation.)  **Open issues:** Forcing every mutable set
> > +        to implement this may be a pain for such a fairly
> > +        non-essential method.  Perhaps just drop it?
>
> There are also comments worrying that an abstract implementation would
> be horribly slow -- but so what?

True. In the discussion of __hash__ returning 0 by default I even
mention that I'd rather be slow and correct than fast and wrong. :-)

> Alex Martelli posted some stats (for dicts, I think) showing that
> (even today) clear was slower than just creating a new object.

:)

> It still makes sense if you care about the "is" relation
>
> Since you are keeping "pop", the abstract implementation can at least
> approximate iteration.
>
>     def clear(self):
>         while self.pop(): pass

That would have to be

  def clear(self):
    while True:
      try:
        self.pop()
      except KeyError:
        break

But yeah, I'll do this. Thanks for the reality check!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Apr 26 02:38:52 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Apr 2007 12:38:52 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
Message-ID: <462FF49C.3060503@canterbury.ac.nz>

Paul Moore wrote:
> OTOH, in Greg's case maybe he should just not bother inheriting from
> the ABC. From what I hear, that shouldn't cause him any problems
> ("duck typing isn't going away" is the message I've heard). Of course,
> it's difficult to be sure without specific examples (on either side of
> the debate).

I gathered that some ABCs would be providing useful
default implementations of some methods. That would
be a reason for me to use one of the supplied ABCs
instead of doing it all myself.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Apr 26 02:47:19 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Apr 2007 12:47:19 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
Message-ID: <462FF697.2060801@canterbury.ac.nz>

Emin.shopper Martinian.shopper wrote:

> Imagine you have a program that takes a long time to run. If you use 
> duck-typing or similar ideas you may watch the program run for hours or 
> days and then raise an AttributeError or NotImplemented error because a 
> derived class didn't implement a required method.

You should have a way of exercising the program with a set of
small test cases before using it for big runs. You need that
anyway, because there are many other ways the program could
fail at run-time that wouldn't be caught by this feature.

This is just another version of the argument for static type
checking, and all the same counter-arguments apply to it.

--
Greg

From guido at python.org  Thu Apr 26 02:50:54 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 17:50:54 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <5.1.1.6.0.20070425203455.05062ae8@sparrow.telecommunity.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<5.1.1.6.0.20070425203455.05062ae8@sparrow.telecommunity.com>
Message-ID: <ca471dc20704251750s6b6debe9oca4b2b006d1d9ac4@mail.gmail.com>

On 4/25/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:43 PM 4/25/2007 -0700, Guido van Rossum wrote:
> >This is a very good point. Perhaps we can come up with a way to make
> >isinstance and issubclass into something like GFs (without requiring
> >the whole GF machinery).
> >
> >I'll think about it some more.
>
> __special__ methods are how Python currently defines built-in GFs, so
> perhaps __isinstance__ and __issubclass__ slots (and their __r*__
> counterparts) should suffice.

Yeah. The pragmatic issues are (a) how easy is it to add this to the
metaclass; (b) whether to keep the existing "abstract
isinstance/issubclass" code which looks for __class__ and __bases__
attributes and believes whatever they return.

> OTOH, the old PEP 246 (implemented by both PyProtocols and zope.interface)
> just uses __conform__ and __adapt__ and basically works the same way as an
> __isinstance__/__risinstance__ pair...

Are pairs needed? I was hoping that isinstance(x, C) would only need
to be overridden on the class C, and issubclass(D, C) would only need
to be overridden on C as well. In particular, the use case brought up
here is isinstance(42, Ring) and issubclass(int, Ring). So we'd want
to add something to Ring.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Apr 26 03:07:13 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Apr 2007 13:07:13 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
Message-ID: <462FFB41.8060507@canterbury.ac.nz>

Paul Moore wrote:
> My point was that as far as I am aware,
> Guido's current position on ABCs is that they are, and will remain,
> optional - people like myself (and Greg, from the sound of it) who
> don't want to derive from ABCs will not be penalised by being excluded
> from anything in the Python core or the stdlib.

Maybe not in the core or stdlib, but it seems like
there will be cases where one *is* penalised for
not using the ABCs, if they get used the way the
PEP proposes.

 From the PEP, it seems there are two purposes for
these proposed ABCs:

1) To document what is meant when we say that something
is a "sequence", "mapping", etc.

This could be done simply by writing documentation. There
is no need for anything embodied in the language.

2) To mark a class for the purposes of introspection.

I thought we had generally agreed that this is an
anti-pattern in Python. If I want my object to
interoperate with someone else's code that tests for
the presence of some ABC, then I *am* forced to use
that ABC, even if it doesn't entirely suit my
purposes.

I'm -1 on including anything in the language, stdlib
or docs that appears to officially sanction type
testing as a normal style of programming.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Apr 26 03:14:55 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Apr 2007 13:14:55 +1200
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <5d44f72f0704251009w12eddd82l971f5d06b35b22fb@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<20070425085350.6408.JCARLSON@uci.edu>
	<5d44f72f0704251009w12eddd82l971f5d06b35b22fb@mail.gmail.com>
Message-ID: <462FFD0F.4040305@canterbury.ac.nz>

Jeffrey Yasskin wrote:
> If you write a function that can take
> an arbitrary ring, it should Just Work on a python int, or any other
> Complex subclass,

There's no reason it shouldn't "just work", unless
it's being anal and trying to make sure it's been
given things of what it considers to be the proper
type.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Apr 26 03:27:13 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 26 Apr 2007 13:27:13 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
	<32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
Message-ID: <462FFFF1.3040307@canterbury.ac.nz>

Emin.shopper Martinian.shopper wrote:

> If you believe that there exist data structures or algorithms where a 
> meaningful unit test takes many minutes or hours,

I wouldn't be worried about spending minutes to save hours
later.

As for unit tests taking hours -- I'm skeptical. Do you
know of algorithms that it's provably impossible to test
without it taking hours?

Thirdly -- your program can *still* fail after many hours
due to one of the many things your static checks don't and
can't cover.

--
Greg

From exarkun at divmod.com  Thu Apr 26 04:06:05 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Wed, 25 Apr 2007 22:06:05 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <5.1.1.6.0.20070425203654.04ee17d8@sparrow.telecommunity.com>
Message-ID: <20070426020605.19381.476607338.divmod.quotient.5402@ohm>

On Wed, 25 Apr 2007 20:40:14 -0400, "Phillip J. Eby" <pje at telecommunity.com> wrote:
>At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote:
>>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <jimjjewett at gmail.com> 
>>wrote:
>> >The current ABC proposal is to use isinstance as the test; Jeffrey
>> >Yaskin's numbers PEP highlighted the weakness there with a concrete
>> >example.
>> >
>> >If you need to an abstraction less powerful than an existing ABC,
>> >you're out of luck; you can't just assert that the existing class is
>> >already sufficient, nor can you expect everyone else to use multiple
>> >annotations.
>>
>>I'm sure everyone is already aware of the behavior of the classImplements
>>and directlyProvides functions available in zope.interface, which exactly
>>satisfy this use-case in the interface world.
>
>I'm either misunderstanding Jim or you, because I don't see the relationship 
>here.  If I understand Jim correctly, he's actually asking for something 
>like PyProtocols' "protocolIsSubsetOf" declaration -- something that would 
>be like dynamically adding a superclass to an existing interface in 
>zope.interface.  Whereas the features you're talking about sound like 
>declaring that a class object itself implements an interface -- something 
>apparently unrelated to the question at hand.
>

Ugh, you're at least half right.  I didn't read the implementation of
ABC.meets carefully enough to notice that it's actually an adapter
registry, not just a predicate (meets seems like a funny name for that
method).

classImplements would have satisfied the adapterless use-case by letting
Jim declare that someone else's class definition actually implements the
feature set that some unit of code he wants to use that class with is
expecting.

With the adapter registry, I don't see why this isn't just PEP 246 adapt()
(if I am reading one of your later mails correctly though, you think it is
too).

I'm not particularly familiar with protocolIsSubsetOf, but it wouldn't
surprise me at all if it could also satisfy this use-case.  There are
lots of ways to do this, many of them fairly simple.  Using inheritance
is the part that makes it hard, since inheritance is controlled by the
wrong party in most cases and comes with unrelated features that are,
at best, irrelevant to the particular use case and at worst actively
detrimental.

I'm sure a way around this can be invented, I just don't see why it
should be.

Jean-Paul

From emin.shopper at gmail.com  Thu Apr 26 04:21:43 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 22:21:43 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
	<ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>
Message-ID: <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>

On 4/25/07, Guido van Rossum <guido at python.org> wrote:

> I'm confused. Can you show a simple example of something that can be
> checked at definition time?


Sure. Below is a slightly modified example from the doctest:

>>> class AbstractCar(AbstractBaseClass): # inherit from AbstractBaseClass
to make something abstract
...     @Abstract
...     def Drive(self,x): pass

To make a class abstract, you inherit from AbstractBaseClass in order to get
the metaclass set properly and then use a function decorator to declare
methods abstract. This is similar to the proposed implementation in your
abc.py and equally pythonic or unpythonic. The following shows what happens
when you define a class that doesn't override the abstract methods:

>>> try: # illustrate what happens when you don't implement @Abstract
methods
...     class fails(AbstractCar): # an erroneous implementation of
AbstractCar
...             pass
... except AssertionError, e:  # doctest: +ELLIPSIS
...     print e
Class <class '...fails'> must override Drive to implement:
[<class '...AbstractCar'>].

The exception gets raised at definition time because the __call__ method
of the metaclass inherited from AbstractBaseClass checks whether abstract
methods have been implemented. In your implementation, you use the __new__
method of the metaclass to do the check at instantiation. If desired you
could enable/disable checking at definition and instantiation separately.

A slightly tricky point is that if something inherits DIRECTLY from
AbstractBaseClass (e.g., AbstractCar) then it is allowed to have abstract
methods that have not been overridden. This allows you to define a partial
abstraction as shown below:

>>> class AbstractFastCar(AbstractCar,AbstractBaseClass):# inherit from ABC
...     @Abstract                                        # so that
interpreter
...     def DriveFast(self): pass                        # doesn't worry
about
...                                                      # Drive being
abstract

My implementation will consider this a valid definition because
AbstractFastCar inherits DIRECTLY from AbstractBaseClass and is therefore
allowed to have abstract methods that have not been overridden. In an
earlier post you claimed that my implementation forces you to declare a
class abstract redundantly (and therefore unpythonic-ly) when this should be
inferred from the class having abstract methods. I believe you were
referring to the partially abstract class example above. I agree that this
is redundant and unpythonic, but I believe this could be fixed by having the
metaclass __call__ method check whether the class itself defines any
abstract methods so you wouldn't need partial abstractions which define
abstract methods to still inherit from AbstractBaseClass directly.

So far I believe that the main reasonable objection raised to definition
time checks is in how the following example would be handled:

>>> # Define a class that provides a helper function but is still
>>> # abstract in the sense that children must implement Drive.
>>> class AbstractCarHelper(AbstractCar,AbstractBaseClass):
...      def DriveHelper(self):
...            print 'Vroom!'

The example above has no abstract methods, but if you want to make this kind
of partial abstraction a legal definition you need to tell the interpreter
that this is still an abstract class by inheriting directly from
AbstractBaseClass. If we want def time checking, I don't see a way to make
the above definition legal without somehow telling the interpreter to allow
it to be abstract. I don't think this is unpythonic since inheriting from
AbstractBaseClass is non-redundant. In fact, one could argue that it is good
since it lets the programmer that AbstractCarHelper is still an abstract
class even though it does not directly define any abstract methods.

If you ignore the last example, I don't think having def time checks for ABC
enforcement has any disadvantages. You can easily turn off the def time
checking or have it off by default and turn it on if desired. Since def time
checks are done by __call__ and instantiation checks are done by __new__,
this shouldn't interfere with anything in the original ABC PEP. So in my
mind, the only reason to object to def time checks is if it bothers you that
the last example must inherit directly from AbstractBaseClass.



> Using the machinery currently in the PEP,
> any class that inherits an abstract method and doesn't override it is
> automatically an abstract class itself. There's nothing to complain
> about at definition time.


Yes, I see that. What I'm proposing is to have __call__ in the metaclass
complain if a class does not inherit directly from AbstractBaseClass and
does not have any abstract methods.

>From your point of view, if you have def time checks turned off by default
then the difference in using my proposal vs. the original ABC proposal all
comes from the last example. For example, your Iterator class in
abc.pyinherits from Iterable (which is abstract) and declares its own
abstract
method. Under my current implementation this would raise an error unless
Iterator also inherits from AbstractBaseClass. This is easily fixable by
having my implementation allow classes which declare their own abstract
methods to be abstract. The incompatibly I pointed out in my last example is
illustrated by the Set class in your abc.py. I believe that you want Set to
be an ABC which requires children to support the ABC requirements of its
parents while providing useful implementations of __le__, __lt__, and
__eq__. In my implementation you would need to have Set either inherit
directly from AbstractBaseClass or define at least one abstract method to
prevent an error at def time (if def time checks are on at all). Note that I
don't think it would be unpythonic to have Set inherit from
AbstractBaseClass since it provides someone reading the source code with the
useful information that this is an ABC and not a concrete implementation.

In summary, let me try to clarify what I see as the only incompatibility
between the current PEP and an ABC implementation: with def time checks
turned on you would need partial abstractions like Set to somehow indicate
that they are abstract by either inheriting directly from AbstractBaseClass
or defining at least one abstract method. If people don't like that, then I
don't have any response. But if people are worried about B&D and other
things, I think those are non-issues. B&D at runtime is no better than B&D
at def time (I think B&D at runtime is worse).



>
> Your example implementation was a bit too much code for me to read.
> :-)


No problem. Unfortunately, I think I might have made the example usage in
this post a little too long to read (but hopefully not). :)

-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/761c38c7/attachment.html 

From emin.shopper at gmail.com  Thu Apr 26 04:29:13 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 22:29:13 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <462FFFF1.3040307@canterbury.ac.nz>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<76fd5acf0704251037s17761b66t8fcbe7a62a3b31ff@mail.gmail.com>
	<32e43bb70704251149l3acca60ak70d30da2b4dd4395@mail.gmail.com>
	<462FFFF1.3040307@canterbury.ac.nz>
Message-ID: <32e43bb70704251929s27714009r6ed6c9487559d9c3@mail.gmail.com>

On 4/25/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> Emin.shopper Martinian.shopper wrote:
>
> > If you believe that there exist data structures or algorithms where a
> > meaningful unit test takes many minutes or hours,
>
> I wouldn't be worried about spending minutes to save hours
> later.


OK, but I would rather not have to spend 10 minutes waiting for a unit test
each time I edit a file when I can easily have the test done in seconds at
import time.

As for unit tests taking hours -- I'm skeptical. Do you
> know of algorithms that it's provably impossible to test
> without it taking hours?


Imagine you have to query a database and do something with the resulting
data. Or imagine some complicated network interaction among p2p clients. Or
imagine numerical simulation over a very fine mesh. I've worked on all of
these and the unit-tests can easily take long enough that you don't want to
run them every time you make any changes.

Thirdly -- your program can *still* fail after many hours
> due to one of the many things your static checks don't and
> can't cover.


Of course. I'm not saying ABCs will solve all bugs. What I am saying is that
ABCs provide an easy way to catch simple bugs that can sometimes save you a
lot of time. Sure you could produce fake data for input to your unit-tests
or otherwise contrive things so that you have very quick unit-tests that
really only check if things have the right methods. But if you want a quick
check that derived classes implement the right methods, I think ABCs are a
good solution. At least, they're a solution I find useful. I'm not trying to
advocate that you should use them only that they should be available.

-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/9cca2a15/attachment.htm 

From emin.shopper at gmail.com  Thu Apr 26 04:37:41 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Wed, 25 Apr 2007 22:37:41 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
	<ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>
	<32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>
Message-ID: <32e43bb70704251937k6d81bb0ao3ee752806d8a453a@mail.gmail.com>

References to __call__ in my post should have been to __init__. Apologies
for the error.

-Emin


On 4/25/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
>
> On 4/25/07, Guido van Rossum <guido at python.org> wrote:
>
> > I'm confused. Can you show a simple example of something that can be
> > checked at definition time?
>
>
> Sure. Below is a slightly modified example from the doctest:
>
> >>> class AbstractCar(AbstractBaseClass): # inherit from AbstractBaseClass
> to make something abstract
> ...     @Abstract
> ...     def Drive(self,x): pass
>
> To make a class abstract, you inherit from AbstractBaseClass in order to
> get the metaclass set properly and then use a function decorator to declare
> methods abstract. This is similar to the proposed implementation in your
> abc.py and equally pythonic or unpythonic. The following shows what
> happens when you define a class that doesn't override the abstract methods:
>
> >>> try: # illustrate what happens when you don't implement @Abstract
> methods
> ...     class fails(AbstractCar): # an erroneous implementation of
> AbstractCar
> ...             pass
> ... except AssertionError, e:  # doctest: +ELLIPSIS
> ...     print e
> Class <class '...fails'> must override Drive to implement:
> [<class '...AbstractCar'>].
>
> The exception gets raised at definition time because the __call__ method
> of the metaclass inherited from AbstractBaseClass checks whether abstract
> methods have been implemented. In your implementation, you use the __new__
> method of the metaclass to do the check at instantiation. If desired you
> could enable/disable checking at definition and instantiation separately.
>
> A slightly tricky point is that if something inherits DIRECTLY from
> AbstractBaseClass (e.g., AbstractCar) then it is allowed to have abstract
> methods that have not been overridden. This allows you to define a partial
> abstraction as shown below:
>
> >>> class AbstractFastCar(AbstractCar,AbstractBaseClass):# inherit from
> ABC
> ...     @Abstract                                        # so that
> interpreter
> ...     def DriveFast(self): pass                        # doesn't worry
> about
> ...                                                      # Drive being
> abstract
>
> My implementation will consider this a valid definition because
> AbstractFastCar inherits DIRECTLY from AbstractBaseClass and is therefore
> allowed to have abstract methods that have not been overridden. In an
> earlier post you claimed that my implementation forces you to declare a
> class abstract redundantly (and therefore unpythonic-ly) when this should be
> inferred from the class having abstract methods. I believe you were
> referring to the partially abstract class example above. I agree that this
> is redundant and unpythonic, but I believe this could be fixed by having the
> metaclass __call__ method check whether the class itself defines any
> abstract methods so you wouldn't need partial abstractions which define
> abstract methods to still inherit from AbstractBaseClass directly.
>
> So far I believe that the main reasonable objection raised to definition
> time checks is in how the following example would be handled:
>
> >>> # Define a class that provides a helper function but is still
> >>> # abstract in the sense that children must implement Drive.
> >>> class AbstractCarHelper(AbstractCar,AbstractBaseClass):
> ...      def DriveHelper(self):
> ...            print 'Vroom!'
>
> The example above has no abstract methods, but if you want to make this
> kind of partial abstraction a legal definition you need to tell the
> interpreter that this is still an abstract class by inheriting directly from
> AbstractBaseClass. If we want def time checking, I don't see a way to make
> the above definition legal without somehow telling the interpreter to allow
> it to be abstract. I don't think this is unpythonic since inheriting from
> AbstractBaseClass is non-redundant. In fact, one could argue that it is good
> since it lets the programmer that AbstractCarHelper is still an abstract
> class even though it does not directly define any abstract methods.
>
> If you ignore the last example, I don't think having def time checks for
> ABC enforcement has any disadvantages. You can easily turn off the def time
> checking or have it off by default and turn it on if desired. Since def time
> checks are done by __call__ and instantiation checks are done by __new__,
> this shouldn't interfere with anything in the original ABC PEP. So in my
> mind, the only reason to object to def time checks is if it bothers you that
> the last example must inherit directly from AbstractBaseClass.
>
>
>
> > Using the machinery currently in the PEP,
> > any class that inherits an abstract method and doesn't override it is
> > automatically an abstract class itself. There's nothing to complain
> > about at definition time.
>
>
> Yes, I see that. What I'm proposing is to have __call__ in the metaclass
> complain if a class does not inherit directly from AbstractBaseClass and
> does not have any abstract methods.
>
> From your point of view, if you have def time checks turned off by default
> then the difference in using my proposal vs. the original ABC proposal all
> comes from the last example. For example, your Iterator class in abc.pyinherits from Iterable (which is abstract) and declares its own abstract
> method. Under my current implementation this would raise an error unless
> Iterator also inherits from AbstractBaseClass. This is easily fixable by
> having my implementation allow classes which declare their own abstract
> methods to be abstract. The incompatibly I pointed out in my last example is
> illustrated by the Set class in your abc.py. I believe that you want Set
> to be an ABC which requires children to support the ABC requirements of its
> parents while providing useful implementations of __le__, __lt__, and
> __eq__. In my implementation you would need to have Set either inherit
> directly from AbstractBaseClass or define at least one abstract method to
> prevent an error at def time (if def time checks are on at all). Note that I
> don't think it would be unpythonic to have Set inherit from
> AbstractBaseClass since it provides someone reading the source code with the
> useful information that this is an ABC and not a concrete implementation.
>
> In summary, let me try to clarify what I see as the only incompatibility
> between the current PEP and an ABC implementation: with def time checks
> turned on you would need partial abstractions like Set to somehow indicate
> that they are abstract by either inheriting directly from AbstractBaseClass
> or defining at least one abstract method. If people don't like that, then I
> don't have any response. But if people are worried about B&D and other
> things, I think those are non-issues. B&D at runtime is no better than B&D
> at def time (I think B&D at runtime is worse).
>
>
>
> >
> > Your example implementation was a bit too much code for me to read.
> > :-)
>
>
> No problem. Unfortunately, I think I might have made the example usage in
> this post a little too long to read (but hopefully not). :)
>
> -Emin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070425/01dfd8a3/attachment-0001.html 

From guido at python.org  Thu Apr 26 05:49:15 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 20:49:15 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <462FFB41.8060507@canterbury.ac.nz>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<462FFB41.8060507@canterbury.ac.nz>
Message-ID: <ca471dc20704252049n1388e9e9qc83153efa3be7335@mail.gmail.com>

On 4/25/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>  From the PEP, it seems there are two purposes for
> these proposed ABCs:
>
> 1) To document what is meant when we say that something
> is a "sequence", "mapping", etc.
>
> This could be done simply by writing documentation. There
> is no need for anything embodied in the language.
>
> 2) To mark a class for the purposes of introspection.
>
> I thought we had generally agreed that this is an
> anti-pattern in Python.

Then how do you explain the popularity of zope.interfaces in both the
Zope and the Twisted world? While it often is an anti-pattern, it
isn't always, and I don't see why we should forever have to tell
people who need this functionality "go download this obfuscated piece
of 3rd party code."

> If I want my object to
> interoperate with someone else's code that tests for
> the presence of some ABC, then I *am* forced to use
> that ABC, even if it doesn't entirely suit my
> purposes.

No different than if you want to interoperate with someone who uses
zope.interfaces, except you won't have to use 3rd party code, so the
cost to you of conforming is many times lower.

> I'm -1 on including anything in the language, stdlib
> or docs that appears to officially sanction type
> testing as a normal style of programming.

Read PEP 3119's Rationale.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 26 06:07:58 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 21:07:58 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
	<ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>
	<32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>
Message-ID: <ca471dc20704252107r402e6cf8y9dd79f18d3c8d63@mail.gmail.com>

On 4/25/07, Emin.shopper Martinian.shopper <emin.shopper at gmail.com> wrote:
> Sure. Below is a slightly modified example from the doctest:
>
> >>> class AbstractCar(AbstractBaseClass): # inherit from AbstractBaseClass
> to make something abstract
> ...     @Abstract
> ...     def Drive(self,x): pass
>
> To make a class abstract, you inherit from AbstractBaseClass in order to get
> the metaclass set properly and then use a function decorator to declare
> methods abstract. This is similar to the proposed implementation in your
> abc.py and equally pythonic or unpythonic. The following shows what happens
> when you define a class that doesn't override the abstract methods:
>
> >>> try: # illustrate what happens when you don't implement @Abstract
> methods
> ...     class fails(AbstractCar): # an erroneous implementation of
> AbstractCar
> ...             pass
> ... except AssertionError, e:  # doctest: +ELLIPSIS
> ...     print e
> Class <class '...fails'> must override Drive to implement:
> [<class '...AbstractCar'>].
>
> The exception gets raised at definition time because the __call__ method of
> the metaclass inherited from AbstractBaseClass checks whether abstract
> methods have been implemented. In your implementation, you use the __new__
> method of the metaclass to do the check at instantiation. If desired you
> could enable/disable checking at definition and instantiation separately.

Well I like the PEP's approach better (and AFAIK it matches what C++
does). In the PEP, it is not an error to inherit from an abstract
class -- the resulting class itself is abstract.

In your version one has to repeat "AbstractBaseClass" each time a
class contains (or inherits) abstract methods. I find that an
unpythonic requirement, bordering on the repetition of the same
information that is so common to statically typed languages.

(Note that the PEP now proposes @abstractmethod as a builtin and the
Abstract[Class] functionality merged into object and type; I already
have an implementation, referenced in the PEP's References section.)

[...]
> In summary, let me try to clarify what I see as the only incompatibility
> between the current PEP and an ABC implementation: with def time checks
> turned on you would need partial abstractions like Set to somehow indicate
> that they are abstract by either inheriting directly from AbstractBaseClass
> or defining at least one abstract method. If people don't like that, then I
> don't have any response. But if people are worried about B&D and other
> things, I think those are non-issues. B&D at runtime is no better than B&D
> at def time (I think B&D at runtime is worse).

You summarize the differences clearly. Let's agree to disagree. I
think that having to have an abstraction marker on the class *and* on
the abstract methods is asking the user to repeat (nearly) the same
information twice, and I really don't think that a (partially)
abstract class needs to be re-marked as abstract.

I find it hard to believe that the definition-time error saves you
much development time at all compared to a instantiation-time error.
Others have already responded to your attitude towards unit tests --
if your unit tests take 10 minutes to run, you're doing something else
wrong. May I suggest you read this:
http://googletesting.blogspot.com/2007/01/introducing-testing-on-toilet.html

PS. My implementation costs one flag bit check and jump on each object
allocation, which surely vanishes compared to the rest of the
allocation cost (and if not, it could be shrunk even further by
substituting a different function for tp_alloc).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 26 06:10:54 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 25 Apr 2007 21:10:54 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <20070426020605.19381.476607338.divmod.quotient.5402@ohm>
References: <5.1.1.6.0.20070425203654.04ee17d8@sparrow.telecommunity.com>
	<20070426020605.19381.476607338.divmod.quotient.5402@ohm>
Message-ID: <ca471dc20704252110t6c27543egdf626ce7ded51b4e@mail.gmail.com>

On 4/25/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> On Wed, 25 Apr 2007 20:40:14 -0400, "Phillip J. Eby" <pje at telecommunity.com> wrote:
> >At 06:40 PM 4/25/2007 -0400, Jean-Paul Calderone wrote:
> >>On Wed, 25 Apr 2007 18:10:23 -0400, Jim Jewett <jimjjewett at gmail.com>
> >>wrote:
> >> >The current ABC proposal is to use isinstance as the test; Jeffrey
> >> >Yaskin's numbers PEP highlighted the weakness there with a concrete
> >> >example.
> >> >
> >> >If you need to an abstraction less powerful than an existing ABC,
> >> >you're out of luck; you can't just assert that the existing class is
> >> >already sufficient, nor can you expect everyone else to use multiple
> >> >annotations.
> >>
> >>I'm sure everyone is already aware of the behavior of the classImplements
> >>and directlyProvides functions available in zope.interface, which exactly
> >>satisfy this use-case in the interface world.
> >
> >I'm either misunderstanding Jim or you, because I don't see the relationship
> >here.  If I understand Jim correctly, he's actually asking for something
> >like PyProtocols' "protocolIsSubsetOf" declaration -- something that would
> >be like dynamically adding a superclass to an existing interface in
> >zope.interface.  Whereas the features you're talking about sound like
> >declaring that a class object itself implements an interface -- something
> >apparently unrelated to the question at hand.
> >
>
> Ugh, you're at least half right.  I didn't read the implementation of
> ABC.meets carefully enough to notice that it's actually an adapter
> registry, not just a predicate (meets seems like a funny name for that
> method).
>
> classImplements would have satisfied the adapterless use-case by letting
> Jim declare that someone else's class definition actually implements the
> feature set that some unit of code he wants to use that class with is
> expecting.
>
> With the adapter registry, I don't see why this isn't just PEP 246 adapt()
> (if I am reading one of your later mails correctly though, you think it is
> too).
>
> I'm not particularly familiar with protocolIsSubsetOf, but it wouldn't
> surprise me at all if it could also satisfy this use-case.  There are
> lots of ways to do this, many of them fairly simple.  Using inheritance
> is the part that makes it hard, since inheritance is controlled by the
> wrong party in most cases and comes with unrelated features that are,
> at best, irrelevant to the particular use case and at worst actively
> detrimental.
>
> I'm sure a way around this can be invented, I just don't see why it
> should be.

JP, would you mind dropping this and instead considering how we could
turn isinstance and issubclass into (poor-man's) GFs by allowing
classes to define class methods __isinstance__ and __issubclass__?
(Though the arguments are reversed so to avoid confusion perhaps the
names should be different, similar as with the 'in' operator and the
__contains__ magic method.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Apr 26 07:16:22 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 26 Apr 2007 01:16:22 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704252049n1388e9e9qc83153efa3be7335@mail.gmail.co
 m>
References: <462FFB41.8060507@canterbury.ac.nz>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<462FFB41.8060507@canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20070426010727.02d2b898@sparrow.telecommunity.com>

At 08:49 PM 4/25/2007 -0700, Guido van Rossum wrote:
>Then how do you explain the popularity of zope.interfaces in both the
>Zope and the Twisted world?

In a word: adaptation.

(Caveat: it's been a while since I looked at very much of Twisted, and even 
longer for Zope; but ISTR that Twisted hardly ever checked to see what 
things implemented, compared to how often it just adapted.  Zope 3, 
meanwhile, has major infrastructure built on adaptation, even though it 
does do some introspection and typechecking.)


From jyasskin at gmail.com  Thu Apr 26 08:13:55 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Wed, 25 Apr 2007 23:13:55 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
Message-ID: <5d44f72f0704252313n15c86257n3a480b2d534e04a@mail.gmail.com>

On 4/25/07, Guido van Rossum <guido at python.org> wrote:
> Jeffrey, is there any way you can drop the top of the tree and going
> straight from Number to Complex -> Real -> Rational -> Integer? These
> are the things that everyone with high school math will know.

Backing up a bit, what kinds of types do you want this hierarchy to
support out of the box? Here are all of the concrete numeric types I
can think of. (Adding to this list is where the numpy people would be
really helpful.)

  complex(Real) (that is, pairs of any type of real number, treated as
complex. So complex(float) is another spelling for the built-in
complex, and complex(int) is the gaussian integers.)
  float, Rational, and Decimal,
  int,
  IntsMod(n) (for any n. prime n's are special. n==2^8 is a C unsigned
char. Also written as "Z/nZ")
  Interval(Real) ([3,5] * [1,3] == [1,15])
  Range (I'm not sure how this should work, but the idea is that
range(2,5) <: range(0,10).)

I'm inclined to exclude Interval because it can't be converted into an
int, and Range because operators will return results with types
different from their arguments, so it breaks my static typing and
mathematical prejudices.

Given the rest of the types, I think the following hierarchy is minimal:
class Complex(Abstract)  # complex({int, rational}) + AlgebraicComplex + Real
class AlgebraicComplex(Complex)  # complex({float,Decimal})
class Real(Complex)  # int + FractionalReal
class FractionalReal(Real)  # float, rational, Decimal

We don't want an Integer abstract class because there is only one type
in it, ``int``, so we'd have no guidance on picking the right methods.
Why not include IntsMod(n), you ask? Their arithmetic, particularly
division, is so different from true integers that I think they'd be
likely to violate the Liskov substitution principle. (IntsMod(11)(7) /
IntsMod(11)(3) == IntsMod(11)(6), whee!)

I'm not completely satisfied that there's no way to distinguish the
gaussian integers from Reals with a single check, but since none of
you are probably using them, the simpler hierarchy is probably worth
it. Maybe we should even combine Real with FractionalReal. Are there
any functions that work on floats but not ints or do different things
depending on the type?

The numpy people do need support for the mathematical ABCs, but if
they can be implemented in a separate module that most people don't
need to know about, I think they should be (I'll move the classes I
currently have into algebra.py to serve as a jumping-off point). And I
think that for ABCs to be viable, it must be possible to add
"superclasses" to an existing abstract class from a separate module.
Hopefully Jim's thread will be fruitful.

I haven't updated the PEP to correspond to this yet, in case it's even
worse than what I have. :)

-- 
Namast?,
Jeffrey Yasskin

From python at rcn.com  Thu Apr 26 08:29:41 2007
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 25 Apr 2007 23:29:41 -0700
Subject: [Python-3000] Two proposals for a new list-like type: one
	modest, one radical
References: <eae285400704230856n633aff15j9a16ed0811ea4adc@mail.gmail.com>
	<00c601c786ff$95839700$f101a8c0@RaymondLaptop1>
	<eae285400704250954w4a493f55k9218bdc89802850c@mail.gmail.com>
Message-ID: <014501c787cc$463d2940$f101a8c0@RaymondLaptop1>

[Raymond Hettinger on treelist being added to the collections module]
>> In practice, the code for BList is somewhat complex, and its desirability and performance in actual applications is unproven.
>> Fortunately, Py2.6 is still a long way off. My recommendation is that you release it right away as a third-party module and let 
>> the
>> hordes of Pythonistas test it in battle. With the code base thoroughly exercised and some positive user feedback, it would be 
>> hard
>> to say no to this going into Py2.6.

[Daniel Stutzbach]
> Will do.

I look forward to the announcement.


Raymond 

From oliphant.travis at ieee.org  Thu Apr 26 12:28:14 2007
From: oliphant.travis at ieee.org (Travis E. Oliphant)
Date: Thu, 26 Apr 2007 04:28:14 -0600
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
Message-ID: <f0pucb$86m$1@sea.gmane.org>


Forgive my ignorance, but I'm not really sure what this PEP is trying to 
do.  I don't want to sound negative, I really just don't understand the 
purpose.  I've just never encountered a problem this that I can see how 
this PEP will help me solve.

That doesn't mean it's not worthwhile, I just don't understand it.

> 
> Abstract
> ========
> 
> This proposal defines a hierarchy of Abstract Base Classes (ABCs)
> [#pep3119] to represent numbers and other algebraic entities similar
> to numbers.  It proposes:
> 
> * A hierarchy of algebraic concepts, including monoids, groups, rings,
>   and fields with successively more operators and constraints on their
>   operators. This will be added as a new library module named
>   "algebra".
> 

The SAGE people may be interested in this, but I doubt there will more 
than a handful of users of these algebraic base classes.


> * A hierarchy of specifically numeric types, which can be converted to
>   and from the native Python types. This will be added as a new
>   library module named "numbers".
> 

I could see having a hierarchy of "numbers"  we have one in NumPy.  All 
the NumPy array scalars fall under a hierarchy of types so that it is 
easy to check whether or not you have signed or unsigned integers, or 
inexact numbers.


> 
> Object oriented systems have a general problem in constraining
> functions that take two arguments. To take addition as an example,
> ``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)``
> is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense.

In NumPy, this kind of operation makes sense we just broadcast int(3) to
the equivalent vector(3,3,3) and do the element-by-element operation.

This is how most array packages outside of Python do it as well.

I think adding the abstract base-classes for "algebras" is more 
complicated than has been discussed if you are going to start treating 
arrays as elements in their own algebra.  There is a lot of flexibility 
in how arrays can be viewed in an algebraic context.

The author gives a valiant go at defining base classes for generic 
algebras, but I didn't see everything I've encountered in my travels 
through algebra, and I don't understand the motivation for trying to 
include these things in Python specifically.

Therefore, I would stick with defining a hierarchy of numbers and leave 
it at that for now (if I even went that far).

For numbers, several suggestions have been offered.  In NumPy we use 
abstract base classes that look like this

Number
    Integer
       SignedInteger
       UnsignedInteger
    Inexact
       Floating
       ComplexFloating

But, this is because the array scalars in NumPy are based on 
C-data-types and not on general-purpose integers or floats.  In NumPy it 
is useful to check if you have an integer-typed array and so the 
base-classes become useful.

For general purpose Python, I would do something like

Complex
   Rational_Complex
     Integer_Complex
   Floating_Complex  # uses hardware float
   Decimal_Complex
   Real
     Rational
       Integer
     Floating        # uses hardware float
     Decimal


I don't like the "copied" structure which occurs because complex numbers 
can are isomorphic to a pair of reals but don't know else how to specify 
  a rational-complex number.


I also don't know if the decimal base-class includes un-limited 
precision floats (the analog of Python long integers) or if they must be 
fixed-precision.


-Travis











From emin.shopper at gmail.com  Thu Apr 26 14:20:04 2007
From: emin.shopper at gmail.com (Emin.shopper Martinian.shopper)
Date: Thu, 26 Apr 2007 08:20:04 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704252107r402e6cf8y9dd79f18d3c8d63@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
	<ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>
	<32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>
	<ca471dc20704252107r402e6cf8y9dd79f18d3c8d63@mail.gmail.com>
Message-ID: <32e43bb70704260520v1095d2f6o3be0e0924beac4c4@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:
>
>
> You summarize the differences clearly. Let's agree to disagree. I
> think that having to have an abstraction marker on the class *and* on
> the abstract methods is asking the user to repeat (nearly) the same
> information twice, and I really don't think that a (partially)
> abstract class needs to be re-marked as abstract.


OK. I still believe that this slight flaw is worth the benefit of def time
checks, but accept your judgement that the use cases I have in mind are not
common enough to justify it. Consider my proposal withdrawn.

I find it hard to believe that the definition-time error saves you
> much development time at all compared to a instantiation-time error.
> Others have already responded to your attitude towards unit tests --
> if your unit tests take 10 minutes to run, you're doing something else
> wrong. May I suggest you read this:
>
> http://googletesting.blogspot.com/2007/01/introducing-testing-on-toilet.html


Sigh. I think unit tests are great. I write unit tests for all of my code. I
have a script that runs all the unit tests on our development branch
overnight and emails me about the things that fails. I'm not motivated by a
dislike of unit-tests. If anything, I am too much a believer in testing
which is why I advocated the option to have more testing facilities (i.e.,
def time tests). I think having more options available for testing is a good
thing and I don't understand the vehement objections people have to anything
that vaguely resembles static checks.

Thanks,
-Emin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20070426/aaec9bbf/attachment.htm 

From bjourne at gmail.com  Thu Apr 26 15:33:08 2007
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Thu, 26 Apr 2007 06:33:08 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <740c3aec0704260633u64ab70ffy2e68cbe5429d0566@mail.gmail.com>

On 4/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> I got off to a good start in terms of figuring out what needs to be in the
> PEP...  good enough to realize that a complete specification of what I have
> in mind is probably going to dwarf the WSGI spec.  There are just too many
> details that have to be explained; the roughly 1000 lines of Python code I
> have in mind is translating to quite a *lot* more words of English, to
> explain intentions, ramifications, etc.

"If the implementation is hard to explain, it's a bad idea."

-- 
mvh Bj?rn

From goodger at python.org  Thu Apr 26 16:39:47 2007
From: goodger at python.org (David Goodger)
Date: Thu, 26 Apr 2007 10:39:47 -0400
Subject: [Python-3000] call for PEPs
Message-ID: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com>

I'm one of the PEP editors at the end of peps-at-python-dot-org; Barry
Warsaw is the other. Since before PyCon, I haven't had time for PEP
number assignments, editing, and checkins. I don't follow python-dev
or python-3000 closely, so I only recently learned of the April 30
deadline for PEPs. I'd like to get all the outstanding PEPs checked
in, but I need help. There are a bunch of draft PEPs in my inbox, but
I can't trust it because some have already been checked in, and others
have since been updated or become obsolete.

Please re-send the latest drafts (as attachments!) of any outstanding
PEPs to peps-at-python-dot-org. Include the dates of any postings of
the PEP to python-dev, python-3000, or python-list (in the
Post-History header). Be sure to read and follow PEP 1 (PEP Purpose
and Guidelines; http://www.python.org/dev/peps/pep-0001/) for details
on what to include.

Posting a draft PEP to a python-* mailing list IS NOT SUFFICIENT. It
must be sent to peps-at-python-dot-org.

Committers (those with commit access to Python's SVN) are welcome to
add PEPs themselves. Just take the next available number from PEP 0
(which must be updated).

BTW, with PSF and PyCon duties (I'm chairing PyCon 2008), I don't see
my availability for PEP editing increasing any time soon. A new PEP
Editor would be welcome!

-- 
David Goodger <http://python.net/~goodger>

From pje at telecommunity.com  Thu Apr 26 17:30:23 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 26 Apr 2007 11:30:23 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <740c3aec0704260633u64ab70ffy2e68cbe5429d0566@mail.gmail.co
 m>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070426112300.0515f820@sparrow.telecommunity.com>

At 06:33 AM 4/26/2007 -0700, BJ?rn Lindqvist wrote:
>On 4/22/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> > I got off to a good start in terms of figuring out what needs to be in the
> > PEP...  good enough to realize that a complete specification of what I have
> > in mind is probably going to dwarf the WSGI spec.  There are just too many
> > details that have to be explained; the roughly 1000 lines of Python code I
> > have in mind is translating to quite a *lot* more words of English, to
> > explain intentions, ramifications, etc.
>
>"If the implementation is hard to explain, it's a bad idea."

The implementation is *easy* to explain.  Here's an equally easy 
explanation of the new-style type hierarchy:

'type' is a subclass of 'object'.  'object' is an instance of 
'type'.  'type' is an instance of 'type'.  The method resolution order 
(mro) of a type is a sequence such that the types bases appear in the same 
relative order in the mro as they do in the bases list, and such that the 
types in the mros of the bases maintain the same relative order in the 
subclass' mro as they do in that base's mro.  If no such sequence can be 
constructed, the type is invalid.

Short, sweet, and beautiful in its simplicity.  It's the *rationale* and 
*consequences* that take a lot of explaining, not the implementation.

In the case of the explanation above for new-style types, the rationale and 
consequences fill most of a book: the "Putting Metaclasses to Work" book 
that Guido used as inspiration.


From talin at acm.org  Thu Apr 26 18:00:45 2007
From: talin at acm.org (Talin)
Date: Thu, 26 Apr 2007 09:00:45 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <4630CCAD.1040400@acm.org>

Phillip J. Eby wrote:
> In particular, when you implement generic functions using generic 
> functions, it reads a lot like trying to explain the relationships between 
> "type" and "object" (i.e. type being its own type and an instance of 
> object, and object being a subclass of type).  You can say what needs to be 
> said in a fairly short space, but saying it in a way that lets people 
> actually wrap their heads around it takes a bit more text.  :)

I meant to comment on this earlier, and forgot.

I kind of wonder about the idea of implementing generic functions using 
generic functions. It's one of those ideas that is so obviously elegant 
and clever, that it's very easy for a programmer to fall in love with. 
As someone who frequently finds himself in a similar situation, I often 
discover later that the clever idea I loved wasn't the most practical.

I've occasionally been accused of being an "architecture astronaut", 
someone whose designs have a theoretical purity that is far above the 
heads of most of its users, to the point where it is hard for them to 
wrap their head around the design:

   http://www.joelonsoftware.com/articles/fog0000000018.html

Related to this is my tendency towards "dual-duty design", in which 
every line of code is serving multiple agendas (like the way that all of 
the major components on an airplane serve more than one functions - the 
wings are not only the primary lift surfaces, they are also the fuel 
tanks and the mounting points for landing gear, engines, running lights, 
and so on.)

The advantage of this is that it makes the code very compact and dense, 
with lots of ideas being embodied in a relatively few lines of code.

The problem with this is that someone who comes along and reads my code 
may not be able to make any sense of it, especially if they are only 
aware of the primary agenda that is informing the code.

Now, I may be talking out of my butt here - I don't know much about your 
situation, or what rationales you have for designing things the way that 
you have. However, I would suggest that you might want to consider what 
your actual, practical needs are, and if there is a way to get them that 
is more straightforward and easy to explain.

-- Talin

From jdc at uwo.ca  Thu Apr 26 18:07:20 2007
From: jdc at uwo.ca (Dan Christensen)
Date: Thu, 26 Apr 2007 12:07:20 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<f0pucb$86m$1@sea.gmane.org>
Message-ID: <873b2n5dc7.fsf@uwo.ca>

[sage list added]

"Travis E. Oliphant" <oliphant.travis at ieee.org> writes:

> The SAGE people may be interested in this, but I doubt there will more 
> than a handful of users of these algebraic base classes.

SAGE has quite a sophisticated type hierarchy, and a sophisticated set
of coercion methods.  What is done in SAGE should definitely be
consulted, since it is probably the most complex set of mathematical
types yet written in python.  The SAGE tutorial is at

  http://www.sagemath.org/doc/html/tut/tut.html

and Section 2.2 gives a brief introduction to numbers:

  http://www.sagemath.org/doc/html/tut/node9.html

The SAGE reference manual is at

  http://www.sagemath.org/doc/html/ref/index.html

Chapter 20:

  http://www.sagemath.org/doc/html/ref/node198.html

and nearby chapters are quite relevant for this discussion.

> For general purpose Python, I would do something like
>
> Complex
>    Rational_Complex
>      Integer_Complex
>    Floating_Complex  # uses hardware float
>    Decimal_Complex
>    Real
>      Rational
>        Integer
>      Floating        # uses hardware float
>      Decimal

Note also that double-precision reals are a subset of the rationals,
since each double precision real is exactly representable as a
rational number, but many rational numbers are not exactly
representable as double precision reals.  Not sure if this means
that reals should be a subclass of the rationals.

I believe that in SAGE these relationships aren't expressed using
subclassing.

Dan


From jimjjewett at gmail.com  Thu Apr 26 18:59:30 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 26 Apr 2007 12:59:30 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704252107r402e6cf8y9dd79f18d3c8d63@mail.gmail.com>
References: <20070425121618.6417.JCARLSON@uci.edu>
	<32e43bb70704251344p6540ba91g19a287949d2bbdb0@mail.gmail.com>
	<20070425140134.641D.JCARLSON@uci.edu>
	<ca471dc20704251419i548df397ucd3226df66906cce@mail.gmail.com>
	<32e43bb70704251456r31fa9598q3eda426a4d6be183@mail.gmail.com>
	<ca471dc20704251632g5bfdf904p87f3bca8389a1859@mail.gmail.com>
	<32e43bb70704251921n4fb470c2uf9154180a75c593c@mail.gmail.com>
	<ca471dc20704252107r402e6cf8y9dd79f18d3c8d63@mail.gmail.com>
Message-ID: <fb6fbf560704260959o2f772340t6ca8e592e93552f1@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:

> In [Emin's] version one has to repeat "AbstractBaseClass" each
> time a class contains (or inherits) abstract methods.

His modified proposal is better, though perhaps still too complicated.

You must declare a class abstract *either* by explicitly marking it,
*or* by including an abstract method.  (Maybe it should be any
attribute, not just methods.)

This would make it more obvious that, say, an IterableMapping still
has some abstract methods that you need to override before you're
done.

On the other hand, what I would really want to see is the list of
missing methods (next and getitem; possibly iter); maybe this sort of
de-abstraction should be left to an IDE for now.

-jJ

From janssen at parc.com  Thu Apr 26 19:11:27 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 26 Apr 2007 10:11:27 PDT
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <20070425085350.6408.JCARLSON@uci.edu> 
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<20070425085350.6408.JCARLSON@uci.edu>
Message-ID: <07Apr26.101132pdt."57996"@synergy1.parc.xerox.com>

> Are the base number operations in Python all that difficult to
> understand?

Well, they're a little tricky.

But the basic problem with "number" support, in almost every
programming language, is that they are too low-level an abstraction.
A number in a program is never *just* an integer or a float.  It's
always an integer or float (or complex, OK) deployed in support of a
very specific purpose.  It's a loop index, or a weight in kilos, or
the number of calendar entries for a particular day.  It often has
units, and a fixed range of values or fixed set of values, neither of
which are taken into account in most programming languages.

Strings really suffer from the same lack of focus.  A string is never
just a string.  It's a book title, or a filename, or a person's
surname, or an email address.  It's usually got a particular human
language (or computer OS) associated with it.  Each of these usages
has particular limitations and patterns which limit both its range of
values, and the operations which can be successfully applied to it.

Bill

From exarkun at divmod.com  Thu Apr 26 19:26:02 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 26 Apr 2007 13:26:02 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <07Apr26.101132pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <20070426172602.19381.679137565.divmod.quotient.5553@ohm>

On Thu, 26 Apr 2007 10:11:27 PDT, Bill Janssen <janssen at parc.com> wrote:
>> Are the base number operations in Python all that difficult to
>> understand?
>
>Well, they're a little tricky.
>
>But the basic problem with "number" support, in almost every
>programming language, is that they are too low-level an abstraction.
>A number in a program is never *just* an integer or a float.  It's
>always an integer or float (or complex, OK) deployed in support of a
>very specific purpose.  It's a loop index, or a weight in kilos, or
>the number of calendar entries for a particular day.  It often has
>units, and a fixed range of values or fixed set of values, neither of
>which are taken into account in most programming languages.
>
>Strings really suffer from the same lack of focus.  A string is never
>just a string.  It's a book title, or a filename, or a person's
>surname, or an email address.  It's usually got a particular human
>language (or computer OS) associated with it.  Each of these usages
>has particular limitations and patterns which limit both its range of
>values, and the operations which can be successfully applied to it.

I definitely agree with this, and resolving it would probably address
a large class of mistakes made when doing any programming involving
numbers and strings (anyone done much of that lately? :).

Does the introduction of a lot of base classes for `int' and `float'
do anything to address this, though?  What you really want is a bunch
of new subclasses  of them, the opposite.

Jean-Paul

From janssen at parc.com  Thu Apr 26 19:26:18 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 26 Apr 2007 10:26:18 PDT
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com> 
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
Message-ID: <07Apr26.102627pdt."57996"@synergy1.parc.xerox.com>

> Jeffrey, is there any way you can drop the top of the tree and going
> straight from Number to Complex -> Real -> Rational -> Integer? These
> are the things that everyone with high school math will know.

I think knowledge of the concepts of group, ring, and field is
supposed to be standard knowledge for any high-school senior -- isn't
this what the "new math" was all about?.  But they're pretty
fundamental to computer science; anyone trying to do serious work in
the field (that is, anyone with a reason to read the PEP) should have
a nodding acquaintance with them.

Bill


From janssen at parc.com  Thu Apr 26 19:31:14 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 26 Apr 2007 10:31:14 PDT
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704251314o1ad32995tbce975c8579b4211@mail.gmail.com> 
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<ca471dc20704251314o1ad32995tbce975c8579b4211@mail.gmail.com>
Message-ID: <07Apr26.103122pdt."57996"@synergy1.parc.xerox.com>

> But it makes sense in some cases to insist that someone who goes
> through the trouble of inheriting from an ABC gets an error if they
> forget to implement one of the methods that are deemed 'essential' to
> that ABC.

This is the problem with abstract methods.  What should happen is that
the default method (the one defined in the ABC class) gets called.
Otherwise you're back in the morass of Java types, without effective
mixin capability.  In Python, we need to be able to define an abstract
class with real implementation methods -- a class that both defines an
interface, and provides an implementation.  That means that anyone
providing a subclass need only implement the methods that they
override.

Now, we can still have a way of marking the methods of an ABC as being
"not implemented", which would in fact force the subclass designer to
implement them.  But this concept should be carefully separated from
the concept of "this method is part of the interface defined by this
class".

Bill

From guido at python.org  Thu Apr 26 20:32:33 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Apr 2007 11:32:33 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <1251171308604668819@unknownmsgid>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<ca471dc20704251314o1ad32995tbce975c8579b4211@mail.gmail.com>
	<1251171308604668819@unknownmsgid>
Message-ID: <ca471dc20704261132l4dd7b768tafc84b1ce5bf6ccf@mail.gmail.com>

On 4/26/07, Bill Janssen <janssen at parc.com> wrote:
[Guido]
> > But it makes sense in some cases to insist that someone who goes
> > through the trouble of inheriting from an ABC gets an error if they
> > forget to implement one of the methods that are deemed 'essential' to
> > that ABC.
>
> This is the problem with abstract methods.  What should happen is that
> the default method (the one defined in the ABC class) gets called.
> Otherwise you're back in the morass of Java types, without effective
> mixin capability.  In Python, we need to be able to define an abstract
> class with real implementation methods -- a class that both defines an
> interface, and provides an implementation.  That means that anyone
> providing a subclass need only implement the methods that they
> override.
>
> Now, we can still have a way of marking the methods of an ABC as being
> "not implemented", which would in fact force the subclass designer to
> implement them.  But this concept should be carefully separated from
> the concept of "this method is part of the interface defined by this
> class".

Yeah. I like @abstractmethod to mean "not implemented" (except perhaps
as an end point for cooperative super calls). For the latter concept,
I propose that all methods present in an apparent ABC are considered
part of the interface, unless their name starts with a single
underscore (the latter should be considered implementation details, or
perhaps definitional details).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jjb5 at cornell.edu  Thu Apr 26 20:55:14 2007
From: jjb5 at cornell.edu (Joel Bender)
Date: Thu, 26 Apr 2007 14:55:14 -0400
Subject: [Python-3000] Fixing super anyone? (reflux)
In-Reply-To: <ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>	<f0hkrs$aro$1@sea.gmane.org>	<9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com>	<f0j77k$nj5$1@sea.gmane.org>	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
Message-ID: <4630F592.9090806@cornell.edu>

I've come late to this thread, and misunderstood what was wrong with 
super(), so a thousand pardons please.  But since that's never stopped 
me before...

Guido van Rossum wrote:

> But:
> 
> class E(D): pass
> 
> print E().f()
> 
> This prints DDBCA which surely isn't right.
> 
> Sounds like the classic bug in such attempts.

About about this?

class _super(property):
     def __init__(self):
         property.__init__(self, self.get_super, None, None)
     def get_super(self, klass):
         class wrapper:
             def __getattr__(self, fn):
                 self.fn = fn
                 return self
             def __call__(self, obj, *args, **kwargs):
                 mro = iter(obj.__class__.__mro__)
                 for cls in mro:
                     if cls is klass:
                         break
                 for cls in mro:
                     f = getattr(cls, self.fn)
                     if f:
                         return f(obj, *args, **kwargs)
                 raise AttributeError, self.fn
         return wrapper()

class _superable(type):
     __super__ = _super()

class A(object):
     __metaclass__ = _superable
     def f(self):
         return "A"

class B(A):
     def f(self):
         return "B" + B.__super__.f(self)

class C(A):
     def f(self):
         return "C" + C.__super__.f(self)

class D(B, C):
     def f(self):
         return "D" + D.__super__.f(self)

class E(D):
     pass

assert E().f() == "DBCA"


Tim Delaney wrote:

 > What I haven't worked out yet is if you should be able to do
 > the following:
 >
 >     class A(autosuper):
 >         def f(self):
 >             print 'A:', super
 >
 >     class B(A):
 >         def f(self):
 >             def inner():
 >                 print 'B:', super
 >                 super.f()
 >             inner()

And here's my version:

class A(object):
     __metaclass__ = _superable
     def f(self):
         return "A"

class B(A):
     def f(self):
         def inner():
             return "B" + B.__super__.f(self)
         inner()

assert B().f() == "BA"


Now, back to the original request, I came up with this:


def super(klass, obj=None):
     class wrapper:
         def __init__(self):
             self.klass = klass
             self.obj = obj
         def __getattr__(self, fn):
             self.fn = fn
             return self
         def __call__(self, *args, **kwargs):
             if not self.obj:
                 self.obj = args[0]
                 args = args[1:]
             mro = iter(self.obj.__class__.__mro__)
             for cls in mro:
                 if cls is self.klass:
                     break
             for cls in mro:
                 f = getattr(cls, self.fn)
                 if f:
                     return f(self.obj, *args, **kwargs)
             raise AttributeError, self.fn
     return wrapper()

class A(object):
     def f(self):
         return "A"

class B(A):
     def f(self):
         return "B" + super(B).f(self)

class C(A):
     def f(self):
         return "C" + super(C, self).f()

class D(B, C):
     def f(self):
         return "D" + super(D, self).f()

class E(D):
     pass

assert E().f() == "DBCA"


I prefer the version in C, but B works as well.  Comments?  Is this 
ground that has already been covered?


Joel


From guido at python.org  Thu Apr 26 21:06:45 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Apr 2007 12:06:45 -0700
Subject: [Python-3000] Fixing super anyone? (reflux)
In-Reply-To: <4630F592.9090806@cornell.edu>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<f0j77k$nj5$1@sea.gmane.org>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<4630F592.9090806@cornell.edu>
Message-ID: <ca471dc20704261206g27aab2b4w742e1b6efb96f388@mail.gmail.com>

Writing B.__super__.f(self) instead of super(B, self).f() is not an improvement.

On 4/26/07, Joel Bender <jjb5 at cornell.edu> wrote:
> I've come late to this thread, and misunderstood what was wrong with
> super(), so a thousand pardons please.  But since that's never stopped
> me before...
>
> Guido van Rossum wrote:
>
> > But:
> >
> > class E(D): pass
> >
> > print E().f()
> >
> > This prints DDBCA which surely isn't right.
> >
> > Sounds like the classic bug in such attempts.
>
> About about this?
>
> class _super(property):
>      def __init__(self):
>          property.__init__(self, self.get_super, None, None)
>      def get_super(self, klass):
>          class wrapper:
>              def __getattr__(self, fn):
>                  self.fn = fn
>                  return self
>              def __call__(self, obj, *args, **kwargs):
>                  mro = iter(obj.__class__.__mro__)
>                  for cls in mro:
>                      if cls is klass:
>                          break
>                  for cls in mro:
>                      f = getattr(cls, self.fn)
>                      if f:
>                          return f(obj, *args, **kwargs)
>                  raise AttributeError, self.fn
>          return wrapper()
>
> class _superable(type):
>      __super__ = _super()
>
> class A(object):
>      __metaclass__ = _superable
>      def f(self):
>          return "A"
>
> class B(A):
>      def f(self):
>          return "B" + B.__super__.f(self)
>
> class C(A):
>      def f(self):
>          return "C" + C.__super__.f(self)
>
> class D(B, C):
>      def f(self):
>          return "D" + D.__super__.f(self)
>
> class E(D):
>      pass
>
> assert E().f() == "DBCA"
>
>
> Tim Delaney wrote:
>
>  > What I haven't worked out yet is if you should be able to do
>  > the following:
>  >
>  >     class A(autosuper):
>  >         def f(self):
>  >             print 'A:', super
>  >
>  >     class B(A):
>  >         def f(self):
>  >             def inner():
>  >                 print 'B:', super
>  >                 super.f()
>  >             inner()
>
> And here's my version:
>
> class A(object):
>      __metaclass__ = _superable
>      def f(self):
>          return "A"
>
> class B(A):
>      def f(self):
>          def inner():
>              return "B" + B.__super__.f(self)
>          inner()
>
> assert B().f() == "BA"
>
>
> Now, back to the original request, I came up with this:
>
>
> def super(klass, obj=None):
>      class wrapper:
>          def __init__(self):
>              self.klass = klass
>              self.obj = obj
>          def __getattr__(self, fn):
>              self.fn = fn
>              return self
>          def __call__(self, *args, **kwargs):
>              if not self.obj:
>                  self.obj = args[0]
>                  args = args[1:]
>              mro = iter(self.obj.__class__.__mro__)
>              for cls in mro:
>                  if cls is self.klass:
>                      break
>              for cls in mro:
>                  f = getattr(cls, self.fn)
>                  if f:
>                      return f(self.obj, *args, **kwargs)
>              raise AttributeError, self.fn
>      return wrapper()
>
> class A(object):
>      def f(self):
>          return "A"
>
> class B(A):
>      def f(self):
>          return "B" + super(B).f(self)
>
> class C(A):
>      def f(self):
>          return "C" + super(C, self).f()
>
> class D(B, C):
>      def f(self):
>          return "D" + super(D, self).f()
>
> class E(D):
>      pass
>
> assert E().f() == "DBCA"
>
>
> I prefer the version in C, but B works as well.  Comments?  Is this
> ground that has already been covered?
>
>
> Joel
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Apr 26 22:08:25 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 26 Apr 2007 16:08:25 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <4630CCAD.1040400@acm.org>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070426153850.02ce0038@sparrow.telecommunity.com>

At 09:00 AM 4/26/2007 -0700, Talin wrote:
>I kind of wonder about the idea of implementing generic functions using
>generic functions. It's one of those ideas that is so obviously elegant
>and clever, that it's very easy for a programmer to fall in love with.
>As someone who frequently finds himself in a similar situation, I often
>discover later that the clever idea I loved wasn't the most practical.

In this case, it's the reverse; I previously implemented generic functions 
the other way around, in RuleDispatch.  PEAK-Rules is considerably simpler; 
in fact I've been realizing lately that the more "object orientation" I 
*remove* from it, the easier it gets to work with.

For example, it's recently become clear that my "TreeBuilder" class (used 
to assemble efficient dispatch trees by figuring out what tests come next) 
is nothing more than a memoized generic function.  Having the class exist 
just adds to the number of things you have to subclass when you change 
anything, and makes it more complex because you also have to change any 
code that *creates* a TreeBuilder to create an instance of your subclass 
instead.  Making it a generic function makes  a bunch of extra bookkeeping 
just disappear.


>Now, I may be talking out of my butt here - I don't know much about your
>situation, or what rationales you have for designing things the way that
>you have.

Pretty simple, really -- RuleDispatch's heavily object-based internal 
structure made it brittle and complex to refactor or extend, compared to 
the outer layers where it used generic functions.  The obvious solution was 
to make the inner layers generic also.

The real key, though, was looking at Guido's "overloading" prototype for 
Py3K and realizing I didn't need to settle for single-dispatch generics for 
the core, or have to try and bootstrap the engine on itself; I could use a 
simple implementation like his that did multi-dispatch instead.  And so I did.

After some experimentation, I found that you can make any generic function 
self-bootstrapping (i.e., usable in its own implementation) as long as your 
implementation is re-entrant.  That is, if a generic function has to call 
itself while a method is being added to it, it must simply call the *last 
stable version* of itself, rather than the version it's modifying.  This is 
actually quite simple to arrange, even in the presence of threads.


>However, I would suggest that you might want to consider what
>your actual, practical needs are, and if there is a way to get them that
>is more straightforward and easy to explain.

Well, I could use some other language of course, but I prefer Python.  :)


From guido at python.org  Thu Apr 26 22:25:28 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Apr 2007 13:25:28 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
Message-ID: <ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>

On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> The current ABC proposal is to use isinstance as the test; Jeffrey
> Yaskin's numbers PEP highlighted the weakness there with a concrete
> example.
>
> If you need to an abstraction less powerful than an existing ABC,
> you're out of luck; you can't just assert that the existing class is
> already sufficient, nor can you expect everyone else to use multiple
> annotations.

I now have a proposal to allow overloading isinstance() and
issubclass(), by defining special (class) methods on the second
argument. See http://python.org/sf/1708353. Does this need a PEP? The
unit test shows that it can be used to support the use case described
above:


class ABC(type):

    def __instancecheck__(cls, inst):
        """Implement isinstance(inst, cls)."""
        return any(cls.__subclasscheck__(c)
                   for c in {type(inst), inst.__class__})

    def __subclasscheck__(cls, sub):
        """Implement issubclass(sub, cls)."""
        candidates = cls.__dict__.get("__subclass__", set())
        return any(c in candidates for c in sub.mro())


class Integer(metaclass=ABC):

    __subclass__ = {int}


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Apr 26 22:36:46 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Apr 2007 13:36:46 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
Message-ID: <ca471dc20704261336i3569fb24r435667bad276c50a@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:
> I now have a proposal to allow overloading isinstance() and
> issubclass(), by defining special (class) methods on the second
> argument. See http://python.org/sf/1708353. Does this need a PEP? The
> unit test shows that it can be used to support the use case described
> above:
>
>
> class ABC(type):
>
>     def __instancecheck__(cls, inst):
>         """Implement isinstance(inst, cls)."""
>         return any(cls.__subclasscheck__(c)
>                    for c in {type(inst), inst.__class__})
>
>     def __subclasscheck__(cls, sub):
>         """Implement issubclass(sub, cls)."""
>         candidates = cls.__dict__.get("__subclass__", set())
>         return any(c in candidates for c in sub.mro())
>
>
> class Integer(metaclass=ABC):
>
>     __subclass__ = {int}

That example has a bug. Try this instead:


class ABC(type):

    def __instancecheck__(cls, inst):
        """Implement isinstance(inst, cls)."""
        return any(cls.__subclasscheck__(c)
                   for c in {type(inst), inst.__class__})

    def __subclasscheck__(cls, sub):
        """Implement issubclass(sub, cls)."""
        candidates = cls.__dict__.get("__subclass__", set()) | {cls}
        return any(c in candidates for c in sub.mro())


class Integer(metaclass=ABC):

    __subclass__ = {int}


To complete the example:

isinstance(42, Integer) and issubclass(int, Integer) now both return
True, while isinstance(3.14, Integer) and issubclass(float, Integer)
return False. Of course, isinstance(Integer(), Integer) and
issubclass(Integer, Integer) are both True. If you subclass Integer,
int won't be considered a subclass of it!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Thu Apr 26 23:13:59 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 26 Apr 2007 17:13:59 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
Message-ID: <fb6fbf560704261413k25c0c8aewbc3ed88b323abcc@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:
> I now have a proposal to allow overloading isinstance() and
> issubclass(), by defining special (class) methods on the second
> argument. See http://python.org/sf/1708353. Does this need a PEP?

It needs to be mentioned in the ABC PEP; whether it needs its own PEP
is probably up to you.

-jJ

From skip at pobox.com  Thu Apr 26 19:18:44 2007
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 26 Apr 2007 12:18:44 -0500
Subject: [Python-3000] [Python-Dev] call for PEPs
In-Reply-To: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com>
References: <4335d2c40704260739k1b9fc46fs1701ecabd9eb7d4e@mail.gmail.com>
Message-ID: <17968.57076.311577.775834@montanaro.dyndns.org>

    David> ... I only recently learned of the April 30 deadline for
    David> PEPs.

This is the first I heard of any sort of deadline...

Skip

From talin at acm.org  Fri Apr 27 04:27:57 2007
From: talin at acm.org (Talin)
Date: Thu, 26 Apr 2007 19:27:57 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704261132l4dd7b768tafc84b1ce5bf6ccf@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>	<462EB78A.207@canterbury.ac.nz>	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>	<ca471dc20704251314o1ad32995tbce975c8579b4211@mail.gmail.com>	<1251171308604668819@unknownmsgid>
	<ca471dc20704261132l4dd7b768tafc84b1ce5bf6ccf@mail.gmail.com>
Message-ID: <46315FAD.8050702@acm.org>

Guido van Rossum wrote:
> Yeah. I like @abstractmethod to mean "not implemented" (except perhaps
> as an end point for cooperative super calls). For the latter concept,
> I propose that all methods present in an apparent ABC are considered
> part of the interface, unless their name starts with a single
> underscore (the latter should be considered implementation details, or
> perhaps definitional details).

I kind of wish that it was called "incomplete" rather than abstract, for 
reasons that we've already discussed.

-- Talin

From guido at python.org  Fri Apr 27 06:12:51 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Apr 2007 21:12:51 -0700
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <46315FAD.8050702@acm.org>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<ca471dc20704251314o1ad32995tbce975c8579b4211@mail.gmail.com>
	<1251171308604668819@unknownmsgid>
	<ca471dc20704261132l4dd7b768tafc84b1ce5bf6ccf@mail.gmail.com>
	<46315FAD.8050702@acm.org>
Message-ID: <ca471dc20704262112j43a85415jdc98bacfa22a56c5@mail.gmail.com>

On 4/26/07, Talin <talin at acm.org> wrote:
> I kind of wish that it was called "incomplete" rather than abstract, for
> reasons that we've already discussed.

Can't remember the details though I vaguely recall some grumblings about that.

But why invent new terminology when we can leverage existing
terminology, even if the semantics aren't quite the same? Python has a
long history of doing that -- e.g. list, int, assignment, class,
method, multiple inheritance, all come from other languages but Python
gives them a new twist -- sometimes intentional (assignment, list,
int), sometimes because the simplistic, dynamic implementation just
can't provide what a typical compiler can do. I find that the current
meanings of "abstract method" and "abstract class" are still close
enough to C++ or Java that reusing the name is helpful rather than
harmful.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Fri Apr 27 09:25:50 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Apr 2007 19:25:50 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <4630CCAD.1040400@acm.org>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<4630CCAD.1040400@acm.org>
Message-ID: <4631A57E.1000301@canterbury.ac.nz>

Talin wrote:

> I kind of wonder about the idea of implementing generic functions using 
> generic functions.

I'd be worried about efficiency. Generic functions on
top of generic functions sounds like you would have two
layers of dynamic lookup to go through instead of one.

--
Greg

From guido at python.org  Thu Apr 26 20:50:59 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 26 Apr 2007 11:50:59 -0700
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
Message-ID: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>

After a fair amount of pre-discussion, I'm ready for the first
official review of this PEP. The PEP is online at
http://www.python.org/dev/peps/pep-3119/ . Here's a summary of open
issues on which I could use more help (more details in the full text
of the PEP below):

- Where should PartiallyOrdered and TotallyOrdered live?

- Should we support comparison of different concrete set types?
- Ditto for mapping types?
- Ditto for sequence types?
- Should Sequence derive from TotallyOrdered?

- Should ComposableSet.__or__ and friends be abstract or concrete?
- If concrete, how should they create the result?

- Do we need a non-composable hashable set type?
- Ditto for a non-composable mutable set type?

- Should we require that the iteration order for keys, values and
items of a mapping are always consistent?

- Which standard methods should sequences have?

Of course, feel free to discuss any issues not marked as "open" as well.

Full text of the PEP:

PEP: 3119
Title: Introducing Abstract Base Classes
Version: $Revision: 54986 $
Last-Modified: $Date: 2007-04-26 11:24:07 -0700 (Thu, 26 Apr 2007) $
Author: Guido van Rossum <guido at python.org>, Talin <talin at acm.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 18-Apr-2007
Post-History: 26-Apr-2007


Abstract
========

This is a proposal to add Abstract Base Class (ABC) support to Python
3000.  It proposes:

* An "ABC support framework" which defines a built-in decorator that
  can be used to define abstract methods.  A class containing an
  abstract method that isn't overridden cannot be instantiated.

* Specific ABCs for containers and iterators, to be added to the
  collections module.

Much of the thinking that went into the proposal is not about the
specific mechanism of ABCs, as contrasted with Interfaces or Generic
Functions (GFs), but about clarifying philosophical issues like "what
makes a set", "what makes a mapping" and "what makes a sequence".


Acknowledgements
----------------

Talin wrote the Rationale below [1]_ as well as most of the section on
ABCs vs. Interfaces.  For that alone he deserves co-authorship.  The
rest of the PEP uses "I" referring to the first author.


Rationale
=========

In the domain of object-oriented programming, the usage patterns for
interacting with an object can be divided into two basic categories,
which are 'invocation' and 'inspection'.

Invocation means interacting with an object by invoking its methods.
Usually this is combined with polymorphism, so that invoking a given
method may run different code depending on the type of an object.

Inspection means the ability for external code (outside of the
object's methods) to examine the type or properties of that object,
and make decisions on how to treat that object based on that
information.

Both usage patterns serve the same general end, which is to be able to
support the processing of diverse and potentially novel objects in a
uniform way, but at the same time allowing processing decisions to be
customized for each different type of object.

In classical OOP theory, invocation is the preferred usage pattern,
and inspection is actively discouraged, being considered a relic of an
earlier, procedural programming style.  However, in practice this view
is simply too dogmatic and inflexible, and leads to a kind of design
rigidity that is very much at odds with the dynamic nature of a
language like Python.

In particular, there is often a need to process objects in a way that
wasn't anticipated by the creator of the object class.  It is not
always the best solution to build in to every object methods that
satisfy the needs of every possible user of that object.  Moreover,
there are many powerful dispatch philosophies that are in direct
contrast to the classic OOP requirement of behavior being strictly
encapsulated within an object, examples being rule or pattern-match
driven logic.

On the the other hand, one of the criticisms of inspection by classic
OOP theorists is the lack of formalisms and the ad hoc nature of what
is being inspected.  In a language such as Python, in which almost any
aspect of an object can be reflected and directly accessed by external
code, there are many different ways to test whether an object conforms
to a particular protocol or not.  For example, if asking 'is this
object a mutable sequence container?', one can look for a base class
of 'list', or one can look for a method named '__getitem__'.  But note
that although these tests may seem obvious, neither of them are
correct, as one generates false negatives, and the other false
positives.

The generally agreed-upon remedy is to standardize the tests, and
group them into a formal arrangement.  This is most easily done by
associating with each class a set of standard testable properties,
either via the inheritance mechanism or some other means.  Each test
carries with it a set of promises: it contains a promise about the
general behavior of the class, and a promise as to what other class
methods will be available.

This PEP proposes a particular strategy for organizing these tests
known as Abstract Base Classes, or ABC.  ABCs are simply Python
classes that are added into an object's inheritance tree to signal
certain features of that object to an external inspector.  Tests are
done using isinstance(), and the presence of a particular ABC means
that the test has passed.

In addition, the ABCs define a minimal set of methods that establish
the characteristic behavior of the type.  Code that discriminates
objects based on their ABC type can trust that those methods will
always be present.  Each of these methods are accompanied by an
generalized abstract semantic definition that is described in the
documentation for the ABC.  These standard semantic definitions are
not enforced, but are strongly recommended.

Like all other things in Python, these promises are in the nature of a
gentlemen's agreement, which in this case means that while the
language does enforce some of the promises made in the ABC, it is up
to the implementer of the concrete class to insure that the remaining
ones are kept.


Specification
=============

The specification follows the categories listed in the abstract:

* An "ABC support framework" which defines a built-in decorator that
  make it easy to define ABCs, and mechanisms to support it.

* Specific ABCs for containers and iterators, to be added to the
  collections module.


ABC Support Framework
---------------------

We define a new built-in decorator, ``@abstractmethod``, to be used to
declare abstract methods.  A class containing at least one method
declared with this decorator that hasn't been overridden yet cannot be
instantiated.  Such a methods may be called from the overriding method
in the subclass (using ``super`` or direct invocation).  For example::

    class A:
        @abstractmethod
        def foo(self): pass

    A()  # raises TypeError

    class B(A):
        pass

    B()  # raises TypeError

    class C(A):
        def foo(self): print(42)

    C()  # works

**Note:** The ``@abstractmethod`` decorator should only be used inside
a class body.  Dynamically adding abstract methods to a class, or
attempting to modify the abstraction status of a method or class once
it is created, are not supported.

**Implementation:** The ``@abstractmethod`` decorator sets the
function attribute ``__isabstractmethod__`` to the value ``True``.
The ``type.__new__`` method computes the type attribute
``__abstractmethods__`` as the set of all method names that have an
``__isabstractmethod__`` attribute whose value is true.  It does this
by combining the ``__abstractmethods__`` attributes of the base
classes, adding the names of all methods in the new class dict that
have a true ``__isabstractmethod__`` attribute, and removing the names
of all methods in the new class dict that don't have a true
``__isabstractmethod__`` attribute.  If the resulting
``__abstractmethods__`` set is non-empty, the class is considered
abstract, and attempts to instantiate it will raise ``TypeError``.
(CPython can uses an internal flag ``Py_TPFLAGS_ABSTRACT`` to speed up
this check [6]_.)

**Discussion:** Unlike C++ or Java, abstract methods as defined here
may have an implementation.  This implementation can be called via the
``super`` mechanism from the class that overrides it.  This could be
useful as an end-point for a super-call in framework using a
cooperative multiple-inheritance [7]_, [8]_.


ABCs for Containers and Iterators
---------------------------------

The ``collections`` module will define ABCs necessary and sufficient
to work with sets, mappings, sequences, and some helper types such as
iterators and dictionary views.

The ABCs provide implementations of their abstract methods that are
technically valid but fairly useless; e.g. ``__hash__`` returns 0, and
``__iter__`` returns an empty iterator.  In general, the abstract
methods represent the behavior of an empty container of the indicated
type.

Some ABCs also provide concrete (i.e. non-abstract) methods; for
example, the ``Iterator`` class has an ``__iter__`` method returning
itself, fulfilling an important invariant of iterators (which in
Python 2 has to be implemented anew by each iterator class).

No ABCs override ``__init__``, ``__new__``, ``__str__`` or
``__repr__``.  Defining a standard constructor signature would
unnecessarily constrain custom container types, for example Patricia
trees or gdbm files.  Defining a specific string representation for a
collection is similarly left up to individual implementations.


Ordering ABCs
'''''''''''''

These ABCs are closer to ``object`` in the ABC hierarchy.

``PartiallyOrdered``
    This ABC defines the 4 inequality operations ``<``, ``<=``, ``>=``,
    ``>``.  (Note that ``==`` and ``!=`` are defined by ``object``.)
    Classes deriving from this ABC should implement a partial order
    as defined in mathematics.  [9]_

``TotallyOrdered``
    This ABC derives from ``PartiallyOrdered``.  It adds no new
    operations but implies a promise of stronger invariants.
    Classes deriving from this ABC should implement a total order
    as defined in mathematics.  [10]_

**Open issues:** Where should these live?  The ``collections`` module
doesn't seem right, but making them built-ins seems a slippery slope
too.


One Trick Ponies
''''''''''''''''

These abstract classes represent single methods like ``__iter__`` or
``__len__``.

``Hashable``
    The base class for classes defining ``__hash__``.  The
    ``__hash__`` method should return an ``Integer`` (see "Numbers"
    below).  The abstract ``__hash__`` method always returns 0, which
    is a valid (albeit inefficient) implementation.  **Invariant:** If
    classes ``C1`` and ``C2`` both derive from ``Hashable``, the
    condition ``o1 == o2`` must imply ``hash(o1) == hash(o2)`` for all
    instances ``o1`` of ``C1`` and all instances ``o2`` of ``C2``.
    IOW, two objects shouldn't compare equal but have different hash
    values.

    Another constraint is that hashable objects, once created, should
    never change their value (as compared by ``==``) or their hash
    value.  If a class cannot guarantee this, it should not derive
    from ``Hashable``; if it cannot guarantee this for certain
    instances only, ``__hash__`` for those instances should raise a
    ``TypeError`` exception.

    **Note:** being an instance of this class does not imply that an
    object is immutable; e.g. a tuple containing a list as a member is
    not immutable; its ``__hash__`` method raises ``TypeError``.

``Iterable``
    The base class for classes defining ``__iter__``.  The
    ``__iter__`` method should always return an instance of
    ``Iterator`` (see below).  The abstract ``__iter__`` method
    returns an empty iterator.

``Iterator``
    The base class for classes defining ``__next__``.  This derives
    from ``Iterable``.  The abstract ``__next__`` method raises
    ``StopIteration``.  The concrete ``__iter__`` method returns
    ``self``.

``Sized``
    The base class for classes defining ``__len__``.  The ``__len__``
    method should return an ``Integer`` (see "Numbers" below) >= 0.
    The abstract ``__len__`` method returns 0.  **Invariant:** If a
    class ``C`` derives from ``Sized`` as well as from ``Iterable``,
    the invariant ``sum(1 for x in o) == len(o)`` should hold for any
    instance ``o`` of ``C``.

``Container``
    The base class for classes defining ``__contains__``.  The
    ``__contains__`` method should return a ``bool``.  The abstract
    ``__contains__`` method returns ``False``.  **Invariant:** If a
    class ``C`` derives from ``Container`` as well as from
    ``Iterable``, then ``(x in o for x in o)`` should be a generator
    yielding only True values for any instance ``o`` of ``C``.

    **Note:** strictly speaking, there are three variants of this method's
    semantics.  The first one is for sets and mappings, which is fast:
    O(1) or O(log N).  The second one is for membership checking on
    sequences, which is slow: O(N).  The third one is for subsequence
    checking on (character or byte) strings, which is also slow: O(N).
    Would it make sense to distinguish these?  The signature of the
    third variant is different, since it takes a sequence (typically
    of the same type as the method's target) intead of an element.
    For now, I'm using the same type for all three.  This means that
    is is possible for ``x in o`` to be True even though ``x`` is
    never yielded by ``iter(o)``.  A suggested name for the third form
    is ``Searchable``.


Sets
''''

These abstract classes represent various stages of "set-ness".  The
most fundamental set operation is the membership test, written as ``x
in s`` and implemented by ``s.__contains__(x)``.  This is already
taken care of by the `Container`` class defined above.  Therefore, we
define a set as a sized, iterable container for which certain
invariants from mathematical set theory hold.

The built-in type ``set`` derives from ``MutableSet``.  The built-in
type ``frozenset`` derives from ``HashableSet``.

You might wonder why we require a set to be sized -- surely certain
infinite sets can be represented just fine in Python.  For example,
the set of even integers could be defined like this::

    class EvenIntegers(Container):
        def __contains__(self, x):
            return x % 2 == 0

However, such sets have rather limited practical value, and deciding
whether one such set is a subset of another would be difficult in
general without using a symbolic algebra package.  So I consider this
out of the scope of a pragmatic proposal like this.

``Set``
    This is a sized, iterable, partially ordered container, i.e. a
    subclass of ``Sized``, ``Iterable``, ``Container`` and
    ``PartiallyOrdered``.  Not every subset of those three classes is
    a set though!  Sets have the additional invariant that each
    element occurs only once (as can be determined by iteration), and
    in addition sets define concrete operators that implement the
    inequality operations as subclass/superclass tests.  In general,
    the invariants for finite sets in mathematics hold. [11]_

    Sets with different implementations can be compared safely,
    (usually) efficiently and correctly using the mathematical
    definitions of the subclass/superclass operations for finite sets.
    The ordering operations have concrete implementations; subclasses
    may override these for speed but should maintain the semantics.
    Because ``Set`` derives from ``Sized``, ``__eq__`` may take a
    shortcut and returns ``False`` immediately if two sets of unequal
    length are compared.  Similarly, ``__le__`` may return ``False``
    immediately if the first set has more members than the second set.
    Note that set inclusion implements only a partial ordering;
    e.g. ``{1, 2}`` and ``{1, 3}`` are not ordered (all three of
    ``<``, ``==`` and ``>`` return ``False`` for these arguments).
    Sets cannot be ordered relative to mappings or sequences, but they
    can be compared to those for equality (and then they always
    compare unequal).

    **Note:** the ``issubset`` and ``issuperset`` methods found on the
    set type in Python 2 are not supported, as these are mostly just
    aliases for ``__le__`` and ``__ge__``.

    **Open issues:** should we define comparison of instances of
    different concrete set types this way?

``ComposableSet``
    This is a subclass of ``Set`` that defines abstract operators to
    compute union, intersection, symmetric and asymmetric difference,
    respectively ``__or__``, ``__and__``, ``__xor__`` and ``__sub__``.
    These operators should return instances of ``ComposableSet``.  The
    abstract implementations return no meaningful values but raise
    ``NotImplementedError``; this is because any generic
    implementation would have to create new instances but the ABCs
    don't (and shouldn't, IMO) provide an API for creating new
    instances.  The implementations of these operators should ensure
    that the results match the mathematical definition of set
    composition. [11]_

    **Open issues:** Should ``__or__`` and friends be abstract or
    concrete methods?  Making them abstract means that every
    ComposableSet implementation must reimplement all of them.  But
    making them concrete begs the question of the actual return type:
    since the ABC doesn't (and IMO shouldn't) define the constructor
    signature for subclasses, the concrete implementations in the ABC
    don't have an API to construct a new instance given an iterable.
    Perhaps the right choice is to have a static concrete factory
    function ``fromiterable`` which takes an iterable and returns
    a ``ComposableSet`` instance.  Subclasses can override this and
    benefit from the default implementations of ``__or__`` etc.; or
    they can override ``__or__`` if they want to.

``HashableSet``
    This is a subclass of both ``ComposableSet`` and ``Hashable``.  It
    implements a concrete ``__hash__`` method that subclasses should
    not override; or if they do, the subclass should compute the same
    hash value.  This is so that sets with different implementations
    still hash to the same value, so they can be used interchangeably
    as dictionary keys.  (A similar constraint exists on the hash
    values for different types of numbers and strings.)

    **Open issues:** Spell out the hash algorithm.  Should there be
    another ABC that derives from Set and Hashable, but not from
    Composable?

``MutableSet``
    This is a subclass of ``ComposableSet`` implementing additional
    operations to add and remove elements.  The supported methods have
    the semantics known from the ``set`` type in Python 2 (except
    for ``discard``, which is modeled after Java):

    ``.add(x)``
        Abstract method returning a ``bool`` that adds the element
        ``x`` if it isn't already in the set.  It should return
        ``True`` if ``x`` was added, ``False`` if it was already
        there. The abstract implementation raises
        ``NotImplementedError``.

    ``.discard(x)``
        Abstract method returning a ``bool`` that removes the element
        ``x`` if present.  It should return ``True`` if the element
        was present and ``False`` if it wasn't.  The abstract
        implementation raises ``NotImplementedError``.

    ``.pop()``
        Concrete method that removes an arbitrary item.  If the set is
        empty, it raises ``KeyError``.  The default implementation
        removes the first item returned by the set's iterator.

    ``.toggle(x)``
        Concrete method returning a ``bool`` that adds x to the set if
        it wasn't there, but removes it if it was there.  It should
        return ``True`` if ``x`` was added, ``False`` if it was
        removed.

    ``.clear()``
        Concrete method that empties the set.  The default
        implementation repeatedly calls ``self.pop()`` until
        ``KeyError`` is caught.  (**Note:** this is likely much slower
        than simply creating a new set, even if an implementation
        overrides it with a faster approach; but in some cases object
        identity is important.)

    This also supports the in-place mutating operations ``|=``,
    ``&=``, ``^=``, ``-=``.  These are concrete methods whose right
    operand can be an arbitrary ``Iterable``, except for ``&=``, whose
    right operand must be a ``Container``.  This ABC does not support
    the named methods present on the built-in concrete ``set`` type
    that perform (almost) the same operations.


Mappings
''''''''

These abstract classes represent various stages of mapping-ness.  The
``Mapping`` class represents the most common read-only mapping API.
However, code *accepting* a mapping is encouraged to check for the
``BasicMapping`` ABC when iteration is not used.  This allows for
certain "black-box" implementations that can look up values by key but
don't provide a convenient iteration API.  A hypothetical example
would be an interface to a hierarchical filesystem, where keys are
pathnames relative to some root directory.  Iterating over all
pathnames would presumably take forever, as would counting the number
of valid pathnames.

The built-in type ``dict`` derives from ``MutableMapping``.

``BasicMapping``
    A subclass of ``Container`` defining the following methods:

    ``.__getitem__(key)``
        Abstract method that returns the value corresponding to
        ``key``, or raises ``KeyError``.  The implementation always
        raises ``KeyError``.

    ``.get(key, default=None)``
        Concrete method returning ``self[key]`` if this does not raise
        ``KeyError``, and the ``default`` value if it does.

    ``.__contains__()``
        Concrete method returning ``True`` if ``self[key]`` does not
        raise ``KeyError``, and ``False`` if it does.

``Mapping``
    A subclass of ``BasicMapping``, ``Iterable`` and ``Sized``.  The
    keys of a mapping naturally form a set.  The (key, value) pairs
    are also referred to as items.  The items also form a set.
    Methods:

    ``__len__``
        Abstract method returning the length of the key set.

    ``__iter__``
        Abstract method returning each key in the key set exactly once.

    ``__eq__``
        Concrete method for comparing mappings.  Two mappings, even
        with different implementations, can be compared for equality,
        and are considered equal if and only iff their item sets are
        equal.  **Open issues:** should we define comparison of
        instances of different concrete mapping types this way?

    ``keys``
        Concrete method returning the key set as a ``Set``.  The
        default concrete implementation returns a "view" on the key
        set (meaning if the underlying mapping is modified, the view's
        value changes correspondingly); subclasses are not required to
        return a view but they should return a ``Set``.

    ``items``
        Concrete method returning the items as a ``Set``.  The default
        concrete implementation returns a "view" on the item set;
        subclasses are not required to return a view but they should
        return a ``Set``.

    ``values``
        Concrete method returning the values as a sized, iterable
        container (not a set!).  The default concrete implementation
        returns a "view" on the values of the mapping; subclasses are
        not required to return a view but they should return a sized,
        iterable container.

    The following invariant should hold for any mapping ``m``::

        set(m.items()) == set(zip(m.keys(), m.values()))

    i.e. iterating over the keys and the values in parallel should
    return *corresponding* keys and values.  **Open issues:** Should
    this always be required?  How about the stronger invariant using
    ``list()`` instead of ``set()``?

``HashableMapping``
    A subclass of ``Mapping`` and ``Hashable``.  The values should be
    instances of ``Hashable``.  The concrete ``__hash__`` method
    should be equal to ``hash(m.items())``.

``MutableMapping``
    A subclass of ``Mapping`` that also implements some standard
    mutating methods.  Abstract methods include ``__setitem__``,
    ``__delitem__``.  Concrete methods include ``pop``, ``popitem``,
    ``clear``, ``update``.  **Note:** ``setdefault`` is *not* included.
    **Open issues:** Write out the specs for the methods.


Sequences
'''''''''

These abstract classes represent various stages of sequence-ness.

The built-in ``list`` and ``bytes`` types derive from
``MutableSequence``.  The built-in ``tuple`` and ``str`` types derive
from ``HashableSequence``.

``Sequence``
    A subclass of ``Iterable``, ``Sized``, ``Container``.  It
    defines a new abstract method ``__getitem__`` that has a somewhat
    complicated signature: when called with an integer, it returns an
    element of the sequence or raises ``IndexError``; when called with
    a ``slice`` object, it returns another ``Sequence``.  The concrete
    ``__iter__`` method iterates over the elements using
    ``__getitem__`` with integer arguments 0, 1, and so on, until
    ``IndexError`` is raised.  The length should be equal to the
    number of values returned by the iterator.

    **Open issues:** Other candidate methods, which can all have
    default concrete implementations that only depend on ``__len__``
    and ``__getitem__`` with an integer argument: __reversed__, index,
    count, __add__, __mul__, __eq__, __lt__, __le__.

``HashableSequence``
    A subclass of ``Sequence`` and ``Hashable``.  The concrete
    ``__hash__`` method should implements the hashing algorithms used
    by tuples in Python 2.

``MutableSequence``
    A subclass of ``Sequence`` adding some standard mutating methods.
    Abstract mutating methods: ``__setitem__`` (for integer indices as
    well as slices), ``__delitem__`` (ditto), ``insert``, ``append``,
    ``reverse``.  Concrete mutating methods: ``extend``, ``pop``,
    ``remove``.  Concrete mutating operators: ``+=``, ``*=`` (these
    mutate the object in place).  **Note:** this does not define
    ``sort()`` -- that is only required to exist on genuine ``list``
    instances.

**Open issues:** If all the elements of a sequence are totally
ordered, the sequence itself can be totally ordered with respect to
other sequences containing corresponding items of the same type.
Should we reflect this by making ``Sequence`` derive from
``TotallyOrdered``?  Or ``Partiallyordered``?  Also, we could easily
define comparison of sequences of different types, so that e.g.
``(1, 2, 3) == [1, 2, 3]`` and ``(1, 2) < [1, 2, 3]``.  Should we?
(It might imply ``["a", "b"] == "ab"`` and ``[1, 2] == b"\1\2"``.)


Strings
-------

Python 3000 has two built-in string types: byte strings (``bytes``),
deriving from ``MutableSequence``, and (Unicode) character strings
(``str``), deriving from ``HashableSequence``.  They also derive from
``TotallyOrdered``.  If we were to introduce ``Searchable``, they
would also derive from that.

**Open issues:** define the base interfaces for these so alternative
implementations and subclasses know what they are in for.  This may be
the subject of a new PEP or PEPs (PEP 358 should be co-opted for the
``bytes`` type).


Numbers
-------

ABCs for numerical types are defined in PEP 3141.


Guidelines for Writing ABCs
---------------------------

Some suggestions for writing ABCs:

* Use the ``@abstractmethod`` decorator.

* Define abstract methods that could be useful as an end point when
  called via a super chain.

* Define concrete methods that are very simple permutations of
  abstract methods (e.g. ``Mapping.get``).

* Keep abstract classes small, one per use case instead of one per
  concept.


ABCs vs. Alternatives
=====================

In this section I will attempt to compare and contrast ABCs to other
approaches that have been proposed.


ABCs vs. Duck Typing
--------------------

Does the introduction of ABCs mean the end of Duck Typing?  I don't
think so.  Python will not require that a class derives from
``BasicMapping`` or ``Sequence`` when it defines a ``__getitem__``
method, nor will the ``x[y]`` syntax require that ``x`` is an instance
of either ABC.  You will still be able to assign any "file-like"
object to ``sys.stdout``, as long as it has a ``write`` method.

Of course, there will be some carrots to encourage users to derive
from the appropriate base classes; these vary from default
implementations for certain functionality to an improved ability to
distinguish between mappings and sequences.  But there are no sticks.
If ``hasattr(x, __len__)`` works for you, great!  ABCs are intended to
solve problems that don't have a good solution at all in Python 2,
such as distinguishing between mappings and sequences.


ABCs vs. Generic Functions
--------------------------

ABCs are compatible with Generic Functions (GFs).  For example, my own
Generic Functions implementation [4]_ uses the classes (types) of the
arguments as the dispatch key, allowing derived classes to override
base classes.  Since (from Python's perspective) ABCs are quite
ordinary classes, using an ABC in the default implementation for a GF
can be quite appropriate.  For example, if I have an overloaded
``prettyprint`` function, it would make total sense to define
pretty-printing of sets like this::

    @prettyprint.register(Set)
    def pp_set(s):
        return "{" + ... + "}"  # Details left as an exercise

and implementations for specific subclasses of Set could be added
easily.

I believe ABCs also won't present any problems for RuleDispatch,
Phillip Eby's GF implementation in PEAK [5]_.

Of course, GF proponents might claim that GFs (and concrete, or
implementation, classes) are all you need.  But even they will not
deny the usefulness of inheritance; and one can easily consider the
ABCs proposed in this PEP as optional implementation base classes;
there is no requirement that all user-defined mappings derive from
``BasicMapping``.


ABCs vs. Interfaces
-------------------

ABCs are not intrinsically incompatible with Interfaces, but there is
considerable overlap.  For now, I'll leave it to proponents of
Interfaces to explain why Interfaces are better.  I expect that much
of the work that went into e.g. defining the various shades of
"mapping-ness" and the nomenclature could easily be adapted for a
proposal to use Interfaces instead of ABCs.

"Interfaces" in this context refers to a set of proposals for
additional metadata elements attached to a class which are not part of
the regular class hierarchy, but do allow for certain types of
inheritance testing.

Such metadata would be designed, at least in some proposals, so as to
be easily mutable by an application, allowing application writers to
override the normal classification of an object.

The drawback to this idea of attaching mutable metadata to a class is
that classes are shared state, and mutating them may lead to conflicts
of intent.  Additionally, the need to override the classification of
an object can be done more cleanly using generic functions: In the
simplest case, one can define a "category membership" generic function
that simply returns False in the base implementation, and then provide
overrides that return True for any classes of interest.


References
==========

.. [1] An Introduction to ABC's, by Talin
   (http://mail.python.org/pipermail/python-3000/2007-April/006614.html)

.. [2] Incomplete implementation prototype, by GvR
   (http://svn.python.org/view/sandbox/trunk/abc/)

.. [3] Possible Python 3K Class Tree?, wiki page created by Bill Janssen
   (http://wiki.python.org/moin/AbstractBaseClasses)

.. [4] Generic Functions implementation, by GvR
   (http://svn.python.org/view/sandbox/trunk/overload/)

.. [5] Charming Python: Scaling a new PEAK, by David Mertz
   (http://www-128.ibm.com/developerworks/library/l-cppeak2/)

.. [6] Implementation of @abstractmethod
   (http://python.org/sf/1706989)

.. [7] Unifying types and classes in Python 2.2, by GvR
   (http://www.python.org/download/releases/2.2.3/descrintro/)

.. [8] Putting Metaclasses to Work: A New Dimension in Object-Oriented
   Programming, by Ira R. Forman and Scott H. Danforth
   (http://www.amazon.com/gp/product/0201433052)

.. [9] Partial order, in Wikipedia
   (http://en.wikipedia.org/wiki/Partial_order)

.. [10] Total order, in Wikipedia
   (http://en.wikipedia.org/wiki/Total_order)

.. [11] Finite set, in Wikipedia
   (http://en.wikipedia.org/wiki/Finite_set)


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Fri Apr 27 10:21:04 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 27 Apr 2007 20:21:04 +1200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <ca471dc20704252049n1388e9e9qc83153efa3be7335@mail.gmail.com>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<462FFB41.8060507@canterbury.ac.nz>
	<ca471dc20704252049n1388e9e9qc83153efa3be7335@mail.gmail.com>
Message-ID: <4631B270.7070900@canterbury.ac.nz>

Guido van Rossum wrote:

> Then how do you explain the popularity of zope.interfaces in both the
> Zope and the Twisted world?

I don't know -- from my point of view, the fact that
Zope and Twisted rely on them so much seems like a
strangely unpythonic aberration. They don't seem to
have caught on in a big way outside those two packages.

> Read PEP 3119's Rationale.

I did read it, and it seemed to be nothing more than
a proposal to provide a way of doing something that
shouldn't be done in the first place, i.e. large-scale
use of LYBL.

I see that you're now talking about providing some
way of declaring a type as conforming to an ABC after
the fact, which does mitigate things somewhat. But
I still have large misgivings about the comefromish
nature of such a thing, and its close cousins,
adaptation and generic functions.

 From my point of view, it still seems like a rather
elaborate solution in search of a deservingly pressing
problem.

--
Greg

From martin at v.loewis.de  Fri Apr 27 10:30:05 2007
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Apr 2007 10:30:05 +0200
Subject: [Python-3000] PEP 3120 added
Message-ID: <4631B48D.3080806@v.loewis.de>

I added PEP 3120 (Using UTF-8 as the default source encoding,
http://www.python.org/dev/peps/pep-3120/) to the repository.
Should I set it to the accepted state?

Martin


From martin at v.loewis.de  Fri Apr 27 10:33:52 2007
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 27 Apr 2007 10:33:52 +0200
Subject: [Python-3000] PEP 3121: Module Initialization and finalization
Message-ID: <4631B570.7030307@v.loewis.de>

Continuing a discussion from last April, I added
PEP 3121, included below for convenience. Please
comment.

Regards,
Martin

PEP: 3121
Title: Module Initialization and finalization
Version: $Revision: 54998 $
Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $
Author: Martin v. L?wis <martin at v.loewis.de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 27-Apr-2007
Python-Version: 3.0
Post-History:

Abstract
========

Module initialization currently has a few deficiencies.  There is no
cleanup for modules, the entry point name might give naming conflicts,
the entry functions don't follow the usual calling convention, and
multiple interpreters are not supported well. This PEP addresses these
issues.

Module Finalization
===================

Currently, C modules are initialized usually once and then "live"
forever. The only exception is when Py_Finalize() is called: then
the initialization routine is invoked a second time. This is bad
from a resource management point of view: memory and other resources
might get allocated each time initialization is called, but there
is no way to reclaim them. As a result, there is currently no
way to completely release all resources Python has allocated.

Entry point name conflicts
==========================

The entry point is currently called init<module>. This might conflict
with other symbols also called init<something>. In particular,
initsocket is known to have conflicted in the past (this specific
problem got resolved as a side effect of renaming the module to
_socket).

Entry point signature
=====================

The entry point is currently a procedure (returning void).  This
deviates from the usual calling conventions; callers can find out
whether there was an error during initialization only by checking
PyErr_Occurred. The entry point should return a PyObject*, which will
be the module created, or NULL in case of an exception.

Multiple Interpreters
=====================

Currently, extension modules share their state across all
interpreters. This allows for undesirable information leakage across
interpreters: one script could permanently corrupt objects in an
extension module, possibly breaking all scripts in other interpreters.

Specification
=============

The module initialization routines change their signature
to::

  PyObject *PyInit_<modulename>()

The initialization routine will be invoked once per
interpreter, when the module is imported. It should
return a new module object each time.

In order to store per-module state in C variables,
each module object will contain a block of memory
that is interpreted only by the module. The amount
of memory used for the module is specified at
the point of creation of the module.

In addition to the initialization function, a module
may implement a number of additional callback
function, which are invoked when the module's
tp_traverse, tp_clear, and tp_free functions are
invoked, and when the module is reloaded.

The entire module definition is combined in a struct
PyModuleDef::

  struct PyModuleDef{
    PyModuleDef_Base m_base;  /* To be filled out by the interpreter */
    Py_ssize_t m_size; /* Size of per-module data */
    PyMethodDef *m_methods;
    inquiry m_reload;
    traverseproc m_traverse;
    inquiry m_clear;
    freefunc m_free;
  };

Creation of a module is changed to expect an optional
PyModuleDef*. The module state will be
null-initialized.

Each module method with be passed the module object
as the first parameter. To access the module data,
a function::

  void* PyModule_GetData(PyObject*);

will be provided. In addition, to lookup a module
more efficiently than going through sys.modules,
a function::

  PyObject* PyState_FindModule(struct PyModuleDef*);

will be provided. This lookup function will use an
index located in the m_base field, to find the
module by index, not by name.

As all Python objects should be controlled through
the Python memory management, usage of "static"
type objects is discouraged, unless the type object
itself has no memory-managed state. To simplify
definition of heap types, a new method::

  PyTypeObject* PyType_Copy(PyTypeObject*);

is added.

Example
=======

xxmodule.c would be changed to remove the initxx
function, and add the following code instead::

  struct xxstate{
    PyObject *ErrorObject;
    PyObject *Xxo_Type;
  };

  #define xxstate(o) ((struct xxstate*)PyModule_GetState(o))

  static int xx_traverse(PyObject *m, visitproc v,
                         void *arg)
  {
    Py_VISIT(xxstate(m)->ErrorObject);
    Py_VISIT(xxstate(m)->Xxo_Type);
    return 0;
  }

  static int xx_clear(PyObject *m)
  {
    Py_CLEAR(xxstate(m)->ErrorObject);
    Py_CLEAR(xxstate(m)->Xxo_Type);
    return 0;
  }

  static struct PyModuleDef xxmodule = {
    {}, /* m_base */
    sizeof(struct xxstate),
    &xx_methods,
    0,  /* m_reload */
    xx_traverse,
    xx_clear,
    0,  /* m_free - not needed, since all is done in m_clear */
  }

  PyObject*
  PyInit_xx()
  {
    PyObject *res = PyModule_New("xx", &xxmodule);
    if (!res) return NULL;
    xxstate(res)->ErrorObject = PyErr_NewException("xx.error, NULL, NULL);
    if (!xxstate(res)->ErrorObject) {
      Py_DECREF(res);
      return NULL;
    }
    xxstate(res)->XxoType = PyType_Copy(&Xxo_Type);
    if (!xxstate(res)->Xxo_Type) {
      Py_DECREF(res);
      return NULL;
    }
    return res;
  }


Discussion
==========

Tim Peters reports in [1]_ that PythonLabs considered such a feature
at one point, and lists the following additional hooks which aren't
currently supported in this PEP:

 * when the module object is deleted from sys.modules

 * when Py_Finalize is called

 * when Python exits

 * when the Python DLL is unloaded (Windows only)


References
==========

.. [1] Tim Peters, reporting earlier conversation about such a feature
   http://mail.python.org/pipermail/python-3000/2006-April/000726.html


Copyright
=========

This document has been placed in the public domain.


From jan.grant at bristol.ac.uk  Fri Apr 27 15:47:59 2007
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Fri, 27 Apr 2007 14:47:59 +0100 (BST)
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <873b2n5dc7.fsf@uwo.ca>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<f0pucb$86m$1@sea.gmane.org> <873b2n5dc7.fsf@uwo.ca>
Message-ID: <20070427144537.T1402@tribble.ilrt.bris.ac.uk>

On Thu, 26 Apr 2007, Dan Christensen wrote:

> Note also that double-precision reals are a subset of the rationals,
> since each double precision real is exactly representable as a
> rational number, but many rational numbers are not exactly
> representable as double precision reals.  Not sure if this means
> that reals should be a subclass of the rationals.

Not quite all: the space of doubles include a small number of things 
that aren't representable by a rational (+/- inf, for instance).




-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Spreadsheet through network. Oh yeah.

From guido at python.org  Fri Apr 27 16:14:16 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Apr 2007 07:14:16 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <20070427144537.T1402@tribble.ilrt.bris.ac.uk>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<f0pucb$86m$1@sea.gmane.org> <873b2n5dc7.fsf@uwo.ca>
	<20070427144537.T1402@tribble.ilrt.bris.ac.uk>
Message-ID: <ca471dc20704270714t42e6bb67pffe935854fcaa889@mail.gmail.com>

On 4/27/07, Jan Grant <jan.grant at bristol.ac.uk> wrote:
> On Thu, 26 Apr 2007, Dan Christensen wrote:
>
> > Note also that double-precision reals are a subset of the rationals,
> > since each double precision real is exactly representable as a
> > rational number, but many rational numbers are not exactly
> > representable as double precision reals.  Not sure if this means
> > that reals should be a subclass of the rationals.
>
> Not quite all: the space of doubles include a small number of things
> that aren't representable by a rational (+/- inf, for instance).

This suddenly makes me think of a new idea -- perhaps we could changes
the type of Inf and NaNs to some *other* numeric type? We could then
reserve a place in the numeric hierarchy for its abstract base class.
Though I don't know if this extends to complex numbers with one or
both parts NaN/Inf or not.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Apr 27 16:16:55 2007
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Apr 2007 10:16:55 -0400
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
Message-ID: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 26, 2007, at 2:50 PM, Guido van Rossum wrote:

> After a fair amount of pre-discussion, I'm ready for the first
> official review of this PEP. The PEP is online at
> http://www.python.org/dev/peps/pep-3119/

Thanks for posting the PEP Guido.  I haven't had time to follow the  
mailing list discussions, so this PEP was a great way to understand  
where you're heading with this proposal.  Here are some comments.

I think there's a typo in the paragraph that introduces  
__instancecheck__:

"The call isinstance(x, C) first checks whether C.__instancecheck__  
exists, and if so, calls C.__subclasscheck__(x) instead of its normal  
implementation."

I think the C.__subclasscheck__ there ought to be  
C.__instancecheck__, right?

After reading the PEP, it's pretty clear to me that ABCs are  
orthogonal to what /I've/ started using interfaces for, and that's  
fine.  If they can live together, as I think is the case, then cool.   
There are a few things that I think are part of interfaces that  
aren't part of ABCs:

- - Attributes.  Interfaces allow you to make assertions about  
attributes, not just methods, while ABCs necessarily cover only methods.

- - Documentation.  Sure, you can docstring the abstract methods in an  
ABC, but it's not the emphasis of the ABC.  For me, the main strength  
of interfaces is the emphasis on documenting the semantics of the  
interfaces, methods, and attributes that contribute to the contract  
you're asserting.  Not just documenting them in off-line web pages,  
but right there in the code.

There are of course plenty of other differences that people with more  
experience can hash out.  Some that draw me to using interfaces include:

- - The separation of inheritance and interface.  ABCs conflate these  
concepts while interfaces keep them separate.  I like the separation  
because it's clearer when reading the class definition which is  
which.  If I care about what the class promises, I look at its  
interfaces.  If I care about how the class keeps those promises, I  
look at its base classes.  With ABCs I think I'd have to look at  
everything and suss out which is which.

- - With interfaces, you can make assertions about individual objects  
which may be different than what their classes assert.  Interface  
proponents seem to care a lot about this and it seems there are valid  
uses cases for it.  For example, I've seen classes which map to a  
database table, where certain instances might assert additional  
properties (by adding interfaces) based on the value of a column.

Another example of separating inheritance and interface comes up when  
you want to derive a subclass to share implementation details, but  
you want to subtly change the semantics, which would invalidate an  
ABC claim by the base class.  Something like a GrowOnlyDictionary  
that derived from dict for implementation purposes, but didn't want  
to implement __delitem__ as required by the MutableMapping ABC.

Finally, I'm concerned with the "weight" of adding ABCs to all the  
built-in types.  There are two weights I'm thinking about: cognitive  
weight for people (newbies, casual programmers) trying to understand  
Python's built-in type system, and the weightiness of the additional  
implementation details (new modules, lots of new classes and objects)  
on applications that don't care about things like ABCs and  
interfaces.  Not everything is a framework! :)

I wonder if it would make sense or be possible to leave the built-in  
types lightweight and ABC-free.  The Python implementation itself  
shouldn't care about ABCs, but any application, framework, or library  
that does would be able to define their own MutableMappingDict which  
derived from dict and MutableMapping.  Or to promote consistency,  
there could be an abclib module which provides these subclasses for  
those applications and frameworks that care about them.  Or we could  
have a standard set of GFs, adaptors, whatever, which hide the  
lightweightedness of the built-in types.

Cheers,
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRjIF3XEjvBPtnXfVAQLySQP+MRhTdbZzYqZVfUQu1Pu++QmLu5+Tfiu0
DHoF6EBxeaghPpLT6XElYXG7AkFOWk18XajtJWUikaaSlgckujVpEJ7nEUh6dfue
rko3hiyy8XblHN4FjbGPkIja/+vgAdpFVpDIGBcNQouK1TPJ1pj7qfGPqoYpYdU2
7Woq2y9kIPU=
=sfLK
-----END PGP SIGNATURE-----

From barry at python.org  Fri Apr 27 16:26:05 2007
From: barry at python.org (Barry Warsaw)
Date: Fri, 27 Apr 2007 10:26:05 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <4631B270.7070900@canterbury.ac.nz>
References: <5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
	<ca471dc20704232125r5fbff102mec5fdb1ae1496eb2@mail.gmail.com>
	<462EB78A.207@canterbury.ac.nz>
	<79990c6b0704250105j3d8dca25mba7cf5180b80f3fc@mail.gmail.com>
	<32e43bb70704250616x44f904fam42327c41762947e3@mail.gmail.com>
	<79990c6b0704250704x7bde694bp38213e8724faa71f@mail.gmail.com>
	<462FFB41.8060507@canterbury.ac.nz>
	<ca471dc20704252049n1388e9e9qc83153efa3be7335@mail.gmail.com>
	<4631B270.7070900@canterbury.ac.nz>
Message-ID: <EE363294-0F38-464F-8D3B-6146C2863384@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Apr 27, 2007, at 4:21 AM, Greg Ewing wrote:

> Guido van Rossum wrote:
>
>> Then how do you explain the popularity of zope.interfaces in both the
>> Zope and the Twisted world?
>
> I don't know -- from my point of view, the fact that
> Zope and Twisted rely on them so much seems like a
> strangely unpythonic aberration. They don't seem to
> have caught on in a big way outside those two packages.

Part of the reason may be that interfaces, ABCs, or whatever become  
more important as applications turn into frameworks.  I sometimes  
think of my code along this axis of progression:

quick hack < script < library < application < framework

You can quibble about the order <wink>.  I've almost never cared  
about interfaces for anything smaller than an application, and even  
for most of those I don't care.  But as you approach frameworkiness,  
you start to care more because other people have to know where -- and  
how -- they can hook in.

I think there are just few Python applications complex enough to  
warrant the additional weight of interfaces.  For most Python code,  
keeping things light and malleable far outweighs any benefit that  
could be attained through interfaces.  But get far enough to the  
right of that scale and the weight of dealing with complexity through  
interfaces becomes more cost effective.

Cheers,
- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRjIH/XEjvBPtnXfVAQJ0CAP/Qbk6dXnNqpML2ujemL3uT7jT34veNb2g
kvxsfeDq4ibaJ/wDLR/ToMUaltBxmO7Y8GoCd455uZJ7JvCNlx7UanjjICp5MA/S
SZYV97W2fxZKTnlcATd1YEDCA7l+Um+ysP/aDQNle9QmyrdlW81DzG8GFU9lsn6W
GaR1k+/s/4s=
=HXQ1
-----END PGP SIGNATURE-----

From jyasskin at gmail.com  Fri Apr 27 16:58:43 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Fri, 27 Apr 2007 07:58:43 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704270714t42e6bb67pffe935854fcaa889@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<f0pucb$86m$1@sea.gmane.org> <873b2n5dc7.fsf@uwo.ca>
	<20070427144537.T1402@tribble.ilrt.bris.ac.uk>
	<ca471dc20704270714t42e6bb67pffe935854fcaa889@mail.gmail.com>
Message-ID: <5d44f72f0704270758n27f3c20y8e2a33c24d8128dd@mail.gmail.com>

On 4/27/07, Guido van Rossum <guido at python.org> wrote:
> On 4/27/07, Jan Grant <jan.grant at bristol.ac.uk> wrote:
> > On Thu, 26 Apr 2007, Dan Christensen wrote:
> >
> > > Note also that double-precision reals are a subset of the rationals,
> > > since each double precision real is exactly representable as a
> > > rational number, but many rational numbers are not exactly
> > > representable as double precision reals.  Not sure if this means
> > > that reals should be a subclass of the rationals.
> >
> > Not quite all: the space of doubles include a small number of things
> > that aren't representable by a rational (+/- inf, for instance).
>
> This suddenly makes me think of a new idea -- perhaps we could changes
> the type of Inf and NaNs to some *other* numeric type? We could then
> reserve a place in the numeric hierarchy for its abstract base class.
> Though I don't know if this extends to complex numbers with one or
> both parts NaN/Inf or not.

From the Fortress spec: "The trait Q ( QQ ) encompasses all finite
rational numbers, the results of dividing any integer by any nonzero
integer. The trait Q? ( QQ_star ) is Q with two extra elements, + ?
and ?? . The trait Q# ( QQ_splat ) is Q? with one additional element,
the indefinite rational (written 0/0 ), which is used as the result of
dividing zero by zero or of adding ?? to +?."

So separating Inf and NaN into other types has some precedent.

I'd also point out that A being a subset of B doesn't make it a
subtype also. If operations on A behave differently than the
specification of the operations on B, then As aren't substitutable for
Bs and A isn't a subtype. Because doubles have finite precision and
rationals don't, I don't think doubles are a subtype of the rationals,
even if you juggle Nan/Inf to make them a subset.

Then again, doubles aren't a group either because of this imprecision,
and I'm suggesting claiming they're a subclass of that, so maybe
there's room in a practical language to make them a subclass of the
rationals too.

-- 
Namast?,
Jeffrey Yasskin

From pje at telecommunity.com  Fri Apr 27 17:22:07 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 27 Apr 2007 11:22:07 -0400
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <4631A57E.1000301@canterbury.ac.nz>
References: <4630CCAD.1040400@acm.org>
	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>
	<4630CCAD.1040400@acm.org>
Message-ID: <5.1.1.6.0.20070427112016.02bf4710@sparrow.telecommunity.com>

At 07:25 PM 4/27/2007 +1200, Greg Ewing wrote:
>Talin wrote:
>
> > I kind of wonder about the idea of implementing generic functions using
> > generic functions.
>
>I'd be worried about efficiency. Generic functions on
>top of generic functions sounds like you would have two
>layers of dynamic lookup to go through instead of one.

No, it only affects method addition and index updating (which only occurs 
when new types are encountered or new methods are added).  You don't need 
the generic functions in order to dispatch generic functions, only to build 
the dispatch tables.


From jimjjewett at gmail.com  Fri Apr 27 19:05:29 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 27 Apr 2007 13:05:29 -0400
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
Message-ID: <fb6fbf560704271005p5e9e4b30nff9655f53f82283e@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:

> - Where should PartiallyOrdered and TotallyOrdered live?

The abc module seems reasonable, even though the rest are containers.

> - Should we support comparison of different concrete set types?
> - Ditto for mapping types?
> - Ditto for sequence types?
> - Should Sequence derive from TotallyOrdered?

I think either choice would have been reasonable once upon a time, but
it is too late for "abc" = ("a", "b", "c"), because they can appear as
keys in the same dict.  I don't know if this constraint applies to
mappings or sets.

> - Do we need a non-composable hashable set type?
> - Ditto for a non-composable mutable set type?

Given iterable, non-composable is at best rare.  So long as it is
possible to override the isinstance and issubclass checks, I think
they can be left out of the stdlib.  (Though I think you use one as a
test case, which is a good idea.)

> No ABCs override ``__init__``, ``__new__``, ``__str__`` or
> ``__repr__``.  Defining a standard constructor signature would
> unnecessarily constrain custom container types, for example Patricia
> trees or gdbm files.  Defining a specific string representation for a
> collection is similarly left up to individual implementations.

Is this a style guide, or an actual constraint that should apply to
user-created ABCs?

> ``Hashable``
>     The base class for classes defining ``__hash__``.  The
>     ``__hash__`` method should return an ``Integer`` (see "Numbers"
>     below).  The abstract ``__hash__`` method always returns 0, which
>     is a valid (albeit inefficient) implementation.  **Invariant:** If
>     classes ``C1`` and ``C2`` both derive from ``Hashable``, the
>     condition ``o1 == o2`` must imply ``hash(o1) == hash(o2)`` for all
>     instances ``o1`` of ``C1`` and all instances ``o2`` of ``C2``.
>     IOW, two objects shouldn't compare equal but have different hash
>     values.

Is this really restricted to classes implementing Hashable?  I thought
it was an existing  general python rule that for all non-buggy objects
a and b

    (a == b)  ==> (hash(a) == hash(b))

unless at least one of  hash(a), hash(b) returned -1 or raised an error.

>     Another constraint is that hashable objects, once created, should
>     never change their value (as compared by ``==``) or their hash
>     value.

I've had a use case where objects were placed in a dict before they
were finalized.  Their value would change only in that certain
attributes could go (once) from undefined to defined.  Several
different objects might pass through the same indetermininate stage,
but only one would be in that state at a time.  (Yes, there was
probably a better architecture if I weren't dealing with legacy
formats.)

Basing hash on only the attributes always known from the start meant
that the hash would never change, and there weren't any equality tests
whose results could change, though the value as a whole (for complete
ordering) could change.

>     **Note:** the ``issubset`` and ``issuperset`` methods found on the
>     set type in Python 2 are not supported, as these are mostly just
>     aliases for ``__le__`` and ``__ge__``.

And I'll again lobby to include them, if only as concrete
implementations that say forward to self.__le__(other).

> Strings
> -------
>
> Python 3000 has two built-in string types: byte strings (``bytes``),
> deriving from ``MutableSequence``, and (Unicode) character strings
> (``str``), deriving from ``HashableSequence``.  They also derive from
> ``TotallyOrdered``.  If we were to introduce ``Searchable``, they
> would also derive from that.
>
> **Open issues:** define the base interfaces for these so alternative
> implementations and subclasses know what they are in for.  This may be
> the subject of a new PEP or PEPs (PEP 358 should be co-opted for the
> ``bytes`` type).

By Monday...?

-jJ

From jimjjewett at gmail.com  Fri Apr 27 19:10:24 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 27 Apr 2007 13:10:24 -0400
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <04A4F15C-38C3-4727-875D-82803F4FB974@python.org>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<04A4F15C-38C3-4727-875D-82803F4FB974@python.org>
Message-ID: <fb6fbf560704271010j50a6319et27d519fc468afeb8@mail.gmail.com>

On 4/27/07, Barry Warsaw <barry at python.org> wrote:

> - - Attributes.  Interfaces allow you to make assertions about
> attributes, not just methods, while ABCs necessarily cover only methods.

Why can't they have data attributes as well?

> - - With interfaces, you can make assertions about individual objects
> which may be different than what their classes assert.  Interface
> proponents seem to care a lot about this and it seems there are valid
> uses cases for it.

Isn't this something that could be handled by overriding isinstance?

> Another example of separating inheritance and interface comes up when
> you want to derive a subclass to share implementation details, but
> you want to subtly change the semantics, which would invalidate an
> ABC claim by the base class.  Something like a GrowOnlyDictionary
> that derived from dict for implementation purposes, but didn't want
> to implement __delitem__ as required by the MutableMapping ABC.

OK, that makes the isubclass override trickier, so there should be an
example, but I think it can still be done.

> Finally, I'm concerned with the "weight" of adding ABCs to all the
> built-in types.

What if the builtin types did not initially derive from any ABC, but
were added (through an issubclass override) when the abc module was
imported?

-jJ

From janssen at parc.com  Fri Apr 27 19:40:02 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 27 Apr 2007 10:40:02 PDT
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <fb6fbf560704271010j50a6319et27d519fc468afeb8@mail.gmail.com> 
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<04A4F15C-38C3-4727-875D-82803F4FB974@python.org>
	<fb6fbf560704271010j50a6319et27d519fc468afeb8@mail.gmail.com>
Message-ID: <07Apr27.104005pdt."57996"@synergy1.parc.xerox.com>

Jim Jewett wrote:
> On 4/27/07, Barry Warsaw <barry at python.org> wrote:
> 
> > - - Attributes.  Interfaces allow you to make assertions about
> > attributes, not just methods, while ABCs necessarily cover only methods.
> 
> Why can't they have data attributes as well?

I agree with Jim.  It seems to me that documenting standard attributes
of a type is one thing that ABC's should be able to do.  Perhaps a
"@abstractattribute" decorator of some sort?  Perhaps a class decorator
"@attributes(name1, name2, ...)"?  Attributes with those names would be
automatically added to an instance, initialized to None, before the
first __init__ method would be called.

Bill

From lists at cheimes.de  Fri Apr 27 19:28:00 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 27 Apr 2007 19:28:00 +0200
Subject: [Python-3000] Generic function PEP won't make it in time
In-Reply-To: <5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>	<5.1.1.6.0.20070422234428.04388208@sparrow.telecommunity.com>	<5.1.1.6.0.20070423115250.044ffec0@sparrow.telecommunity.com>	<5.1.1.6.0.20070423150735.045ee4c0@sparrow.telecommunity.com>	<5.1.1.6.0.20070423184759.045d92d8@sparrow.telecommunity.com>
	<ca471dc20704231649r7ecb7116k8c411ddb7699185a@mail.gmail.co m>
	<5.1.1.6.0.20070423203951.04c29650@sparrow.telecommunity.com>
Message-ID: <f0tbr4$97e$1@sea.gmane.org>

Phillip J. Eby wrote:
> As someone with more recent background in Java than C++, I find the idea of 
> abstract methods having an executable implementation to be quite confusing, 
> and suspect that other people with that Java or C# background will do the 
> same thing.  That is, skim the explanation and miss the significant 
> difference between the C++ way and what they're used to.

 From the naive point of view from somebody with less background 
knowledge about ABCs, abstract methods and generics I expect that

* an abstract class can't be instantiated. An abstract class must
   be sub-classed before it can be instantiated.

* an abstract method has no implementation and simply raises a
   NonImplementedError based exception.

* a class with abstract methods is automatically an abstract class
   unless generics are entering the game of class creation. When a
   generic overwrites an abstract method the class is a concrete
   class but when the abstract method is called an exception is
   raised.

class AbstractEgg(abstract=True):
     """Abstract class, cannot be instantiated
     """
     def egg(self, foo):
         pass

or:

@abstract
class AbstractEgg:
     pass

class Egg(AbstractEgg):
     """Subclass can be instantiated
     """

class AbstractSpam:
     """Abstract class because it contains an abstract method
     """
     @abstract
     def egg(self, foo):
         pass

class Spam(AbstractSpam):
     """Still an abstract method because it doesn't overwrite egg()
     """

class GenericSpam(AbstractSpam):
     """Subclass can be instantiated

     I'm not sure about the @generic syntax but it's just an example
     """
     @generic(int)
     def egg(self, foo):
         pass

 >>> gs = GenericSpam()

This works:
 >>> gs.egg(1)

This raises an AbstractMethodException(NotImplementedException)
 >>> gs.egg('foo')

Just my 5 cents from an ordinary Python user who is trying to learn ABCs 
and generics.

Christian


From brett at python.org  Fri Apr 27 20:23:49 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 27 Apr 2007 11:23:49 -0700
Subject: [Python-3000] PEP 3121: Module Initialization and finalization
In-Reply-To: <4631B570.7030307@v.loewis.de>
References: <4631B570.7030307@v.loewis.de>
Message-ID: <bbaeab100704271123jd423cd5q4f377515ea91be06@mail.gmail.com>

On 4/27/07, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Continuing a discussion from last April, I added
> PEP 3121, included below for convenience. Please
> comment.
>
> Regards,
> Martin
>
> PEP: 3121
> Title: Module Initialization and finalization
> Version: $Revision: 54998 $
> Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 27-Apr-2007
> Python-Version: 3.0
> Post-History:
>
> Abstract
> ========
>
> Module initialization currently has a few deficiencies.  There is no
> cleanup for modules, the entry point name might give naming conflicts,
> the entry functions don't follow the usual calling convention, and
> multiple interpreters are not supported well. This PEP addresses these
> issues.
>

Thanks for trying to solve this, Martin!

> Module Finalization
> ===================
>
> Currently, C modules are initialized usually once and then "live"
> forever. The only exception is when Py_Finalize() is called: then
> the initialization routine is invoked a second time. This is bad
> from a resource management point of view: memory and other resources
> might get allocated each time initialization is called, but there
> is no way to reclaim them. As a result, there is currently no
> way to completely release all resources Python has allocated.
>
> Entry point name conflicts
> ==========================
>
> The entry point is currently called init<module>. This might conflict
> with other symbols also called init<something>. In particular,
> initsocket is known to have conflicted in the past (this specific
> problem got resolved as a side effect of renaming the module to
> _socket).
>
> Entry point signature
> =====================
>
> The entry point is currently a procedure (returning void).  This
> deviates from the usual calling conventions; callers can find out
> whether there was an error during initialization only by checking
> PyErr_Occurred. The entry point should return a PyObject*, which will
> be the module created, or NULL in case of an exception.
>
> Multiple Interpreters
> =====================
>
> Currently, extension modules share their state across all
> interpreters. This allows for undesirable information leakage across
> interpreters: one script could permanently corrupt objects in an
> extension module, possibly breaking all scripts in other interpreters.
>

After the intro and up to here, it would seem like changing each
section into a sub-section of a "Problems" section of some sort would
make the document more organized and easier to read.

> Specification
> =============
>
> The module initialization routines change their signature
> to::
>
>   PyObject *PyInit_<modulename>()
>
> The initialization routine will be invoked once per
> interpreter, when the module is imported. It should
> return a new module object each time.
>
> In order to store per-module state in C variables,
> each module object will contain a block of memory
> that is interpreted only by the module. The amount
> of memory used for the module is specified at
> the point of creation of the module.
>
> In addition to the initialization function, a module
> may implement a number of additional callback
> function, which are invoked when the module's
> tp_traverse, tp_clear, and tp_free functions are
> invoked, and when the module is reloaded.
>
> The entire module definition is combined in a struct
> PyModuleDef::
>
>   struct PyModuleDef{
>     PyModuleDef_Base m_base;  /* To be filled out by the interpreter */
>     Py_ssize_t m_size; /* Size of per-module data */
>     PyMethodDef *m_methods;
>     inquiry m_reload;
>     traverseproc m_traverse;
>     inquiry m_clear;
>     freefunc m_free;
>   };
>
> Creation of a module is changed to expect an optional
> PyModuleDef*. The module state will be
> null-initialized.
>
> Each module method with be passed the module object

I think you meant "will", not "with".

> as the first parameter. To access the module data,
> a function::
>
>   void* PyModule_GetData(PyObject*);
>
> will be provided. In addition, to lookup a module
> more efficiently than going through sys.modules,
> a function::
>
>   PyObject* PyState_FindModule(struct PyModuleDef*);
>
> will be provided. This lookup function will use an
> index located in the m_base field, to find the
> module by index, not by name.
>
> As all Python objects should be controlled through
> the Python memory management, usage of "static"
> type objects is discouraged, unless the type object
> itself has no memory-managed state.

Ooh, I like this side-effect of the proposal!

> To simplify
> definition of heap types, a new method::
>
>   PyTypeObject* PyType_Copy(PyTypeObject*);
>
> is added.
>
> Example
> =======
>
> xxmodule.c would be changed to remove the initxx
> function, and add the following code instead::
>
>   struct xxstate{
>     PyObject *ErrorObject;
>     PyObject *Xxo_Type;
>   };
>
>   #define xxstate(o) ((struct xxstate*)PyModule_GetState(o))
>
>   static int xx_traverse(PyObject *m, visitproc v,
>                          void *arg)
>   {
>     Py_VISIT(xxstate(m)->ErrorObject);
>     Py_VISIT(xxstate(m)->Xxo_Type);
>     return 0;
>   }
>
>   static int xx_clear(PyObject *m)
>   {
>     Py_CLEAR(xxstate(m)->ErrorObject);
>     Py_CLEAR(xxstate(m)->Xxo_Type);
>     return 0;
>   }
>
>   static struct PyModuleDef xxmodule = {
>     {}, /* m_base */
>     sizeof(struct xxstate),
>     &xx_methods,
>     0,  /* m_reload */
>     xx_traverse,
>     xx_clear,
>     0,  /* m_free - not needed, since all is done in m_clear */
>   }
>
>   PyObject*
>   PyInit_xx()
>   {
>     PyObject *res = PyModule_New("xx", &xxmodule);
>     if (!res) return NULL;
>     xxstate(res)->ErrorObject = PyErr_NewException("xx.error, NULL, NULL);
>     if (!xxstate(res)->ErrorObject) {
>       Py_DECREF(res);
>       return NULL;
>     }
>     xxstate(res)->XxoType = PyType_Copy(&Xxo_Type);
>     if (!xxstate(res)->Xxo_Type) {
>       Py_DECREF(res);
>       return NULL;
>     }
>     return res;
>   }
>

How would I raise xx.error in C code now?  I am guessing like this::

  PyObject* module = PyState_FindModule(&xxmodule);
  if (!module)
    return NULL;
  PyObject* xx_error = xxstate(module)->ErrorObject;
  if (!xx_error) {
    PyErr_SetString(PyExc_SystemError, "xx.error missing");
    return NULL;
  }
  PyErr_SetString(xx_error, "oops");
  return NULL;

Since most objects will move to being memory-managed, one needs to
worry about checking that the object still exists.  I assume I didn't
go overboard with the error checking here, right?  I guess people are
going to end up writing helper functions to access the various data
fields as the above would get rather tedious if you had to write it
more than twice.

>
> Discussion
> ==========
>
> Tim Peters reports in [1]_ that PythonLabs considered such a feature
> at one point, and lists the following additional hooks which aren't
> currently supported in this PEP:
>
>  * when the module object is deleted from sys.modules
>
>  * when Py_Finalize is called
>
>  * when Python exits
>

Wouldn't the above be covered by the deallocation of the module?

Overall I like the idea.  I think people will need to get used to the
idea of writing more accessor functions for the data field, though, if
using static variables to hold things like exceptions becomes
discouraged.

-Brett

From tony at PageDNA.com  Fri Apr 27 20:17:41 2007
From: tony at PageDNA.com (Tony Lownds)
Date: Fri, 27 Apr 2007 11:17:41 -0700
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <07Apr27.104005pdt."57996"@synergy1.parc.xerox.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<04A4F15C-38C3-4727-875D-82803F4FB974@python.org>
	<fb6fbf560704271010j50a6319et27d519fc468afeb8@mail.gmail.com>
	<07Apr27.104005pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <217F4CF1-3CC1-48CE-A635-877C22562C78@PageDNA.com>


On Apr 27, 2007, at 10:40 AM, Bill Janssen wrote:

> Jim Jewett wrote:
>> On 4/27/07, Barry Warsaw <barry at python.org> wrote:
>>
>>> - - Attributes.  Interfaces allow you to make assertions about
>>> attributes, not just methods, while ABCs necessarily cover only  
>>> methods.
>>
>> Why can't they have data attributes as well?
>
> I agree with Jim.  It seems to me that documenting standard attributes
> of a type is one thing that ABC's should be able to do.  Perhaps a
> "@abstractattribute" decorator of some sort?  Perhaps a class  
> decorator
> "@attributes(name1, name2, ...)"?  Attributes with those names  
> would be
> automatically added to an instance, initialized to None, before the
> first __init__ method would be called.

+0 on abstract attributes. Methods seem to dominate most APIs that make
use of interfaces, but there are always a few exceptions.

Attribute annotations could provide an elegant solution here.

class ABC:
   concrete_attribute : "any expression" = None
   abstract_attribute : "any expression"

I posted about attribute annotations before on python-ideas and there  
was
lukewarm response. If anyone thinks a PEP on attribute annotations is  
worth
submitting, let me know.

Thanks
-Tony


From fdrudol5 at rockwellcollins.com  Fri Apr 27 20:25:07 2007
From: fdrudol5 at rockwellcollins.com (fdrudol5 at rockwellcollins.com)
Date: Fri, 27 Apr 2007 13:25:07 -0500
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
Message-ID: <OFE9A193F1.98B252EA-ON862572CA.00557D8D-862572CA.00652D44@rockwellcollins.com>


The PEP looks good. A few nits and a couple of related questions:

MutableSet.pop() "...removes an arbitrary..." # should be "...removes and
returns an arbitrary..."

BasicMapping .__contains__() # should be .__contains__(key)

In Mapping.__eq__, "if and only iff" is redundant. And yes, I think the
stated semantics are useful.

The invariant   set(m.items()) == set(zip(m.keys(), m.values()))
is not strong enough and using list wouldn't fix it. A really goofy
implementation that iterates
values in a different order than keys, then defines items as returning the
zip would satisfy
the invariant. How about:
      [value for value in m.values()] == [m[key] for key in m.keys()]
      [item for item in m.items()] == [(key, m[key]) for key in m.keys()]
Do we need to explicitly state the invariant   len(m.values()) ==
len(m.keys())?
Is this an invariant?  all(isinstance(item,tuple) for item in m.items())

Really minor nit: inconsistent use of "." and "()": ".pop()", ".clear()",
but "items", "values", etc.

In Sequence, the other method candidates all make sense, should have
standard semantics,
and have easy default implementations. Might as well do it.

Can the standard ABCs be usefully superclassed? For example, in a class for
potentially infinite
sets that can be unioned and intersected with builtin sets, would
__subclass__ = {ComposableSet}
be useful?

In the context of parameter annotations, having homogeneous containers
(sets of strings, sequences
of integers) were discussed. Can wrappers be created so that a content
invariant can be enforced
on a container and easily checked against a constraint on a formal
parameter?

#Rudy


From thomas at thomas-lotze.de  Fri Apr 27 20:53:35 2007
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Fri, 27 Apr 2007 20:53:35 +0200
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
Message-ID: <pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>

Guido van Rossum wrote:

>     Another constraint is that hashable objects, once created, should
>     never change their value (as compared by ``==``) or their hash value. 
>     If a class cannot guarantee this, it should not derive from
>     ``Hashable``; if it cannot guarantee this for certain instances only,
>     ``__hash__`` for those instances should raise a ``TypeError``
>     exception.

Shouldn't this rather be a ValueError since it occurs not because of the
type of the object in question, but its value (while in general, there are
instances of the same type representing other values which are hashable)?

>     The third one is for subsequence
>     checking on (character or byte) strings, which is also slow: O(N).
>     Would it make sense to distinguish these?  The signature of the third
>     variant is different, since it takes a sequence (typically of the same
>     type as the method's target) intead of an element. For now, I'm using
>     the same type for all three.  This means that is is possible for ``x
>     in o`` to be True even though ``x`` is never yielded by ``iter(o)``. 
>     A suggested name for the third form is ``Searchable``.

To me, 'searchable' isn't associated with subsequences in any way. I'd
think of a container that is able to answer the question "where is this
element?" A sequence might return an index or a set of indexes, a mapping
might return a key or a set of keys in reply to this. If what you're after
is really a subsequence containment test, it should just live on sequences
instead of getting an ABC of its own, IMO.

> **Open issues:** If all the elements of a sequence are totally ordered,
> the sequence itself can be totally ordered with respect to other sequences
> containing corresponding items of the same type. Should we reflect this by
> making ``Sequence`` derive from ``TotallyOrdered``?  Or
> ``Partiallyordered``?

Unless I'm mistaken, since a sequence type in general can't know whether
all the elements in a given instance will be totally ordered, deriving
Sequence from TotallyOrdered would make promises it can't keep. But I'm +1
on deriving Sequence from PartiallyOrdered since sequences do have an
ordering relationship between some of their values.

> Also, we could easily define comparison of
> sequences of different types, so that e.g. ``(1, 2, 3) == [1, 2, 3]`` and
> ``(1, 2) < [1, 2, 3]``.  Should we? (It might imply ``["a", "b"] == "ab"``
> and ``[1, 2] == b"\1\2"``.)

+1

-- 
Thomas




From jimjjewett at gmail.com  Fri Apr 27 21:06:31 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 27 Apr 2007 15:06:31 -0400
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>
Message-ID: <fb6fbf560704271206q4a4ab59ax37d9af9924988fa2@mail.gmail.com>

On 4/27/07, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> Guido van Rossum wrote:

> >     ``__hash__`` for those instances should raise a ``TypeError``
> >     exception.

> Shouldn't this rather be a ValueError since it occurs not because of the
> type of the object in question, but its value (while in general, there are
> instances of the same type representing other values which are hashable)?

Yes, but it is a TypeError today.  Is it worth the backwards compatibility?

> >     [It] is possible for ``x
> >     in o`` to be True even though ``x`` is never yielded by ``iter(o)``.

> To me, 'searchable' isn't associated with subsequences in any way. I'd
> think of a container that is able to answer the question "where is this
> element?" A sequence might return an index or a set of indexes, a mapping
> might return a key or a set of keys in reply to this. If what you're after
> is really a subsequence containment test, it should just live on sequences
> instead of getting an ABC of its own, IMO.

That makes sense.  Looking at unordered containers like sets, should

{"a", "b"} in {"b", "a", "d"} be able to return true, by deriving from
Searchable?

If not, then I agree that it is really about sequences.

-jJ

From thomas at thomas-lotze.de  Fri Apr 27 21:47:30 2007
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Fri, 27 Apr 2007 21:47:30 +0200
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>
	<fb6fbf560704271206q4a4ab59ax37d9af9924988fa2@mail.gmail.com>
Message-ID: <pan.2007.04.27.19.47.29.672306@ID-174572.user.uni-berlin.de>

Jim Jewett wrote:

> Yes, but it is a TypeError today.  Is it worth the backwards
> compatibility?

I'd say yes for the sake of doing things right, in particular as we are
talking about Py3k, but then I don't think I have a good picture of all
the things that would be broken by this change.

> That makes sense.  Looking at unordered containers like sets, should
> 
> {"a", "b"} in {"b", "a", "d"} be able to return true, by deriving from
> Searchable?

No. {"a", "b"} clearly is a subset of {"b", "a", "d"}, not an element. So
{"a", "b"} < {"b", "a", "d"}, but {"a", "b"} not in {"b", "a", "d"}.

-- 
Thomas




From jason.orendorff at gmail.com  Fri Apr 27 22:24:14 2007
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Fri, 27 Apr 2007 16:24:14 -0400
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <fb6fbf560704271005p5e9e4b30nff9655f53f82283e@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<fb6fbf560704271005p5e9e4b30nff9655f53f82283e@mail.gmail.com>
Message-ID: <bb8868b90704271324t318859ccy59c33183120e552a@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:
> - Where should PartiallyOrdered and TotallyOrdered live?

Could someone please post some sample code using either of
these?  I don't see what they're good for.

Here's my failed attempt:

  def my_max(a : TotallyOrdered, b : TotallyOrdered):
      return a if a >= b else b

But this is both too loose (consider my_max(7, "spam"))
and too strict (consider my_max([1, 2], [2, 1])).

-j

From martin at v.loewis.de  Sat Apr 28 00:44:25 2007
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Apr 2007 00:44:25 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
Message-ID: <46327CC9.4040901@v.loewis.de>

I propose the PEP below for Py3k.

Regards,
Martin

PEP: 3122
Title: Dropping PyObject_HEAD
Version: $Revision: 54998 $
Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $
Author: Martin v. L?wis <martin at v.loewis.de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 27-Apr-2007
Python-Version: 3.0
Post-History:

Abstract
========

Python currently relies on undefined C behavior, with its
usage of PyObject_HEAD. This PEP proposes to change that
into standard C.

Rationale
=========

Standard C defines that an object must be accessed only through a
pointer of its type, and that all other accesses are undefined
behavior, with a few exceptions. In particular, the following
code has undefined behavior::

  struct FooObject{
    PyObject_HEAD
    int data;
  };

  PyObject *foo(struct FooObject*f){
   return (PyObject*)f;
  }

  int bar(){
   struct FooObject *f = malloc(sizeof(struct FooObject));
   struct PyObject *o = foo(f);
   f->ob_refcnt = 0;
   o->ob_refcnt = 1;
   return f->ob_refcnt;
  }

The problem here is that the storage is both accessed as
if it where struct PyObject, and as struct FooObject.

Historically, compilers did not cause any problems with this
code. However, modern compiler use that clause as an
optimization opportunity, finding that f->ob_refcnt and
o->ob_refcnt cannot possibly refer to the same memory, and
that therefore the function should return 0, without having
to fetch the value of ob_refcnt at all in the return
statement. For GCC, Python now uses -fno-strict-aliasing
to work around that problem; with other compilers, it
may just see undefined behavior. Even with GCC, using
-fno-strict-aliasing may pessimize the generated code
unnecessarily.

Specification
=============

Standard C has one specific exception to its aliasing rules precisely
designed to support the case of Python: a value of a struct type may
also be accessed through a pointer to the first field. E.g. if a
struct starts with an int, the struct\* may also be cast to an int\*,
allowing to write int values into the first field.

For Python, PyObject_HEAD and PyObject_VAR_HEAD will be dropped, and
PyObject gets defined to contain all fields explicitly::

  typedef struct _object{
    _PyObject_HEAD_EXTRA
    Py_ssize_t ob_refcnt;
    struct _typeobject *ob_type;
  }PyObject;

  typedef struct {
    PyObject ob_base;
    Py_ssize_t ob_size;
  } PyVarObject;

Types defined as fixed-size structure will then include PyObject
as its first field; variable-sized objects PyVarObject. E.g.::

  typedef struct{
    PyObject ob_base;
    PyObject *start, *stop, *step;
  } PySliceObject;

  typedef struct{
    PyVarObject ob_base;
    PyObject **ob_item;
    Py_ssize_t allocated;
  } PyListObject;

As a convention, the base field SHOULD be called ob_base. However,
all accesses to ob_refcnt and ob_type MUST cast the object pointer
to PyObject* (unless the pointer is already known to have that
type), and SHOULD use the respective accessor macros. To simplify
access to ob_type, a macro::

  #define Py_Type(o) (((PyObject*)o)->ob_type)

is added.

Copyright
=========

This document has been placed in the public domain.


From brett at python.org  Sat Apr 28 01:30:50 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 27 Apr 2007 16:30:50 -0700
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <46327CC9.4040901@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de>
Message-ID: <bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>

Second PEP today.  Martin is on a roll!  =)

On 4/27/07, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I propose the PEP below for Py3k.
>
> Regards,
> Martin
>
> PEP: 3122
> Title: Dropping PyObject_HEAD
> Version: $Revision: 54998 $
> Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 27-Apr-2007
> Python-Version: 3.0
> Post-History:
>
> Abstract
> ========
>
> Python currently relies on undefined C behavior, with its
> usage of PyObject_HEAD. This PEP proposes to change that
> into standard C.
>
> Rationale
> =========
>
> Standard C defines that an object must be accessed only through a
> pointer of its type, and that all other accesses are undefined
> behavior, with a few exceptions. In particular, the following
> code has undefined behavior::
>
>   struct FooObject{
>     PyObject_HEAD
>     int data;
>   };
>
>   PyObject *foo(struct FooObject*f){
>    return (PyObject*)f;
>   }
>
>   int bar(){
>    struct FooObject *f = malloc(sizeof(struct FooObject));
>    struct PyObject *o = foo(f);
>    f->ob_refcnt = 0;
>    o->ob_refcnt = 1;
>    return f->ob_refcnt;
>   }
>
> The problem here is that the storage is both accessed as
> if it where struct PyObject, and as struct FooObject.
>
> Historically, compilers did not cause any problems with this

Reads easier if you replace "cause" with "have".

> code. However, modern compiler use that clause as an

Probably want to pluralize "compiler".

Your use of "clause" really confused me until I realized what you were
talking about.

> optimization opportunity, finding that f->ob_refcnt and
> o->ob_refcnt cannot possibly refer to the same memory, and
> that therefore the function should return 0, without having
> to fetch the value of ob_refcnt at all in the return
> statement. For GCC, Python now uses -fno-strict-aliasing
> to work around that problem; with other compilers, it
> may just see undefined behavior. Even with GCC, using
> -fno-strict-aliasing may pessimize the generated code
> unnecessarily.
>
> Specification
> =============
>
> Standard C has one specific exception to its aliasing rules precisely
> designed to support the case of Python: a value of a struct type may
> also be accessed through a pointer to the first field. E.g. if a
> struct starts with an int, the struct\* may also be cast to an int\*,
> allowing to write int values into the first field.
>
> For Python, PyObject_HEAD and PyObject_VAR_HEAD will be dropped, and
> PyObject gets defined to contain all fields explicitly::
>
>   typedef struct _object{
>     _PyObject_HEAD_EXTRA
>     Py_ssize_t ob_refcnt;
>     struct _typeobject *ob_type;
>   }PyObject;
>
>   typedef struct {
>     PyObject ob_base;
>     Py_ssize_t ob_size;
>   } PyVarObject;
>
> Types defined as fixed-size structure will then include PyObject
> as its first field; variable-sized objects PyVarObject. E.g.::
>
>   typedef struct{
>     PyObject ob_base;
>     PyObject *start, *stop, *step;
>   } PySliceObject;
>
>   typedef struct{
>     PyVarObject ob_base;
>     PyObject **ob_item;
>     Py_ssize_t allocated;
>   } PyListObject;
>
> As a convention, the base field SHOULD be called ob_base. However,
> all accesses to ob_refcnt and ob_type MUST cast the object pointer
> to PyObject* (unless the pointer is already known to have that
> type), and SHOULD use the respective accessor macros. To simplify
> access to ob_type, a macro::
>
>   #define Py_Type(o) (((PyObject*)o)->ob_type)
>
> is added.
>

An example of how this will change current code would be good.  E.g.,
``o->ob_type->tp_name`` becomes ``PyType(o)->typ_name`` or
``o->ob_base->ob_type->tp_name``.

Otherwise I am all for cleaning up the codebase and thus support this PEP.

-Brett

From jyasskin at gmail.com  Fri Apr 27 21:27:16 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Fri, 27 Apr 2007 12:27:16 -0700
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
Message-ID: <5d44f72f0704271227j7e94091m3b2e196e19d59e6a@mail.gmail.com>

This PEP says that no ABC defines __init__ or a couple other methods.
Is that a prohibition on ABCs in general, or just a comment about the
ones defined here?

Can properties be abstract? How would one specify it? (Is that
@abstractattribute?)

On the orderings: If I have a function foo(a, b) and I want to check
whether they're totally ordered (perhaps sort() will use a topological
sort when its argument's contents are only partially ordered.), how do
I do it?
  "isinstance(a, TotallyOrdered) and isinstance(b, TotallyOrdered)"
doesn't work since they may be in two different orders.
  "isinstance(a, TotallyOrdered) and type(a)==type(b)" doesn't work
when you tend to inherit from concrete types.
  " isinstance(a, TotallyOrdered) and isinstance(b, TotallyOrdered)
and (isinstance(a, type(b)) or isinstance(b, type(a))" fails when a
and b are sibling subclasses of int. (I didn't think of this option; I
think it was Jim Jewett's?)
  Searching up the hierarchy from a and b for the first concrete type,
and checking if those are equal would be pretty reliable if types were
singly-inherited, but with multiple inheritance, it would take more
thought to get it right.
  Java's solution is to parametrize Comparable with the type that the
subclass is comparable with. TotallyOrdered could specify an attribute
on its subclasses for users to check against.


On 4/26/07, Guido van Rossum <guido at python.org> wrote:
> After a fair amount of pre-discussion, I'm ready for the first
> official review of this PEP. The PEP is online at
> http://www.python.org/dev/peps/pep-3119/ . Here's a summary of open
> issues on which I could use more help (more details in the full text
> of the PEP below):
>
> - Where should PartiallyOrdered and TotallyOrdered live?
>
> - Should we support comparison of different concrete set types?
> - Ditto for mapping types?
> - Ditto for sequence types?
> - Should Sequence derive from TotallyOrdered?
>
> - Should ComposableSet.__or__ and friends be abstract or concrete?
> - If concrete, how should they create the result?
>
> - Do we need a non-composable hashable set type?
> - Ditto for a non-composable mutable set type?
>
> - Should we require that the iteration order for keys, values and
> items of a mapping are always consistent?
>
> - Which standard methods should sequences have?
>
> Of course, feel free to discuss any issues not marked as "open" as well.
>
> Full text of the PEP:
>
> PEP: 3119
> Title: Introducing Abstract Base Classes
> Version: $Revision: 54986 $
> Last-Modified: $Date: 2007-04-26 11:24:07 -0700 (Thu, 26 Apr 2007) $
> Author: Guido van Rossum <guido at python.org>, Talin <talin at acm.org>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 18-Apr-2007
> Post-History: 26-Apr-2007
>
>
> Abstract
> ========
>
> This is a proposal to add Abstract Base Class (ABC) support to Python
> 3000.  It proposes:
>
> * An "ABC support framework" which defines a built-in decorator that
>   can be used to define abstract methods.  A class containing an
>   abstract method that isn't overridden cannot be instantiated.
>
> * Specific ABCs for containers and iterators, to be added to the
>   collections module.
>
> Much of the thinking that went into the proposal is not about the
> specific mechanism of ABCs, as contrasted with Interfaces or Generic
> Functions (GFs), but about clarifying philosophical issues like "what
> makes a set", "what makes a mapping" and "what makes a sequence".
>
>
> Acknowledgements
> ----------------
>
> Talin wrote the Rationale below [1]_ as well as most of the section on
> ABCs vs. Interfaces.  For that alone he deserves co-authorship.  The
> rest of the PEP uses "I" referring to the first author.
>
>
> Rationale
> =========
>
> In the domain of object-oriented programming, the usage patterns for
> interacting with an object can be divided into two basic categories,
> which are 'invocation' and 'inspection'.
>
> Invocation means interacting with an object by invoking its methods.
> Usually this is combined with polymorphism, so that invoking a given
> method may run different code depending on the type of an object.
>
> Inspection means the ability for external code (outside of the
> object's methods) to examine the type or properties of that object,
> and make decisions on how to treat that object based on that
> information.
>
> Both usage patterns serve the same general end, which is to be able to
> support the processing of diverse and potentially novel objects in a
> uniform way, but at the same time allowing processing decisions to be
> customized for each different type of object.
>
> In classical OOP theory, invocation is the preferred usage pattern,
> and inspection is actively discouraged, being considered a relic of an
> earlier, procedural programming style.  However, in practice this view
> is simply too dogmatic and inflexible, and leads to a kind of design
> rigidity that is very much at odds with the dynamic nature of a
> language like Python.
>
> In particular, there is often a need to process objects in a way that
> wasn't anticipated by the creator of the object class.  It is not
> always the best solution to build in to every object methods that
> satisfy the needs of every possible user of that object.  Moreover,
> there are many powerful dispatch philosophies that are in direct
> contrast to the classic OOP requirement of behavior being strictly
> encapsulated within an object, examples being rule or pattern-match
> driven logic.
>
> On the the other hand, one of the criticisms of inspection by classic
> OOP theorists is the lack of formalisms and the ad hoc nature of what
> is being inspected.  In a language such as Python, in which almost any
> aspect of an object can be reflected and directly accessed by external
> code, there are many different ways to test whether an object conforms
> to a particular protocol or not.  For example, if asking 'is this
> object a mutable sequence container?', one can look for a base class
> of 'list', or one can look for a method named '__getitem__'.  But note
> that although these tests may seem obvious, neither of them are
> correct, as one generates false negatives, and the other false
> positives.
>
> The generally agreed-upon remedy is to standardize the tests, and
> group them into a formal arrangement.  This is most easily done by
> associating with each class a set of standard testable properties,
> either via the inheritance mechanism or some other means.  Each test
> carries with it a set of promises: it contains a promise about the
> general behavior of the class, and a promise as to what other class
> methods will be available.
>
> This PEP proposes a particular strategy for organizing these tests
> known as Abstract Base Classes, or ABC.  ABCs are simply Python
> classes that are added into an object's inheritance tree to signal
> certain features of that object to an external inspector.  Tests are
> done using isinstance(), and the presence of a particular ABC means
> that the test has passed.
>
> In addition, the ABCs define a minimal set of methods that establish
> the characteristic behavior of the type.  Code that discriminates
> objects based on their ABC type can trust that those methods will
> always be present.  Each of these methods are accompanied by an
> generalized abstract semantic definition that is described in the
> documentation for the ABC.  These standard semantic definitions are
> not enforced, but are strongly recommended.
>
> Like all other things in Python, these promises are in the nature of a
> gentlemen's agreement, which in this case means that while the
> language does enforce some of the promises made in the ABC, it is up
> to the implementer of the concrete class to insure that the remaining
> ones are kept.
>
>
> Specification
> =============
>
> The specification follows the categories listed in the abstract:
>
> * An "ABC support framework" which defines a built-in decorator that
>   make it easy to define ABCs, and mechanisms to support it.
>
> * Specific ABCs for containers and iterators, to be added to the
>   collections module.
>
>
> ABC Support Framework
> ---------------------
>
> We define a new built-in decorator, ``@abstractmethod``, to be used to
> declare abstract methods.  A class containing at least one method
> declared with this decorator that hasn't been overridden yet cannot be
> instantiated.  Such a methods may be called from the overriding method
> in the subclass (using ``super`` or direct invocation).  For example::
>
>     class A:
>         @abstractmethod
>         def foo(self): pass
>
>     A()  # raises TypeError
>
>     class B(A):
>         pass
>
>     B()  # raises TypeError
>
>     class C(A):
>         def foo(self): print(42)
>
>     C()  # works
>
> **Note:** The ``@abstractmethod`` decorator should only be used inside
> a class body.  Dynamically adding abstract methods to a class, or
> attempting to modify the abstraction status of a method or class once
> it is created, are not supported.
>
> **Implementation:** The ``@abstractmethod`` decorator sets the
> function attribute ``__isabstractmethod__`` to the value ``True``.
> The ``type.__new__`` method computes the type attribute
> ``__abstractmethods__`` as the set of all method names that have an
> ``__isabstractmethod__`` attribute whose value is true.  It does this
> by combining the ``__abstractmethods__`` attributes of the base
> classes, adding the names of all methods in the new class dict that
> have a true ``__isabstractmethod__`` attribute, and removing the names
> of all methods in the new class dict that don't have a true
> ``__isabstractmethod__`` attribute.  If the resulting
> ``__abstractmethods__`` set is non-empty, the class is considered
> abstract, and attempts to instantiate it will raise ``TypeError``.
> (CPython can uses an internal flag ``Py_TPFLAGS_ABSTRACT`` to speed up
> this check [6]_.)
>
> **Discussion:** Unlike C++ or Java, abstract methods as defined here
> may have an implementation.  This implementation can be called via the
> ``super`` mechanism from the class that overrides it.  This could be
> useful as an end-point for a super-call in framework using a
> cooperative multiple-inheritance [7]_, [8]_.
>
>
> ABCs for Containers and Iterators
> ---------------------------------
>
> The ``collections`` module will define ABCs necessary and sufficient
> to work with sets, mappings, sequences, and some helper types such as
> iterators and dictionary views.
>
> The ABCs provide implementations of their abstract methods that are
> technically valid but fairly useless; e.g. ``__hash__`` returns 0, and
> ``__iter__`` returns an empty iterator.  In general, the abstract
> methods represent the behavior of an empty container of the indicated
> type.
>
> Some ABCs also provide concrete (i.e. non-abstract) methods; for
> example, the ``Iterator`` class has an ``__iter__`` method returning
> itself, fulfilling an important invariant of iterators (which in
> Python 2 has to be implemented anew by each iterator class).
>
> No ABCs override ``__init__``, ``__new__``, ``__str__`` or
> ``__repr__``.  Defining a standard constructor signature would
> unnecessarily constrain custom container types, for example Patricia
> trees or gdbm files.  Defining a specific string representation for a
> collection is similarly left up to individual implementations.
>
>
> Ordering ABCs
> '''''''''''''
>
> These ABCs are closer to ``object`` in the ABC hierarchy.
>
> ``PartiallyOrdered``
>     This ABC defines the 4 inequality operations ``<``, ``<=``, ``>=``,
>     ``>``.  (Note that ``==`` and ``!=`` are defined by ``object``.)
>     Classes deriving from this ABC should implement a partial order
>     as defined in mathematics.  [9]_
>
> ``TotallyOrdered``
>     This ABC derives from ``PartiallyOrdered``.  It adds no new
>     operations but implies a promise of stronger invariants.
>     Classes deriving from this ABC should implement a total order
>     as defined in mathematics.  [10]_
>
> **Open issues:** Where should these live?  The ``collections`` module
> doesn't seem right, but making them built-ins seems a slippery slope
> too.
>
>
> One Trick Ponies
> ''''''''''''''''
>
> These abstract classes represent single methods like ``__iter__`` or
> ``__len__``.
>
> ``Hashable``
>     The base class for classes defining ``__hash__``.  The
>     ``__hash__`` method should return an ``Integer`` (see "Numbers"
>     below).  The abstract ``__hash__`` method always returns 0, which
>     is a valid (albeit inefficient) implementation.  **Invariant:** If
>     classes ``C1`` and ``C2`` both derive from ``Hashable``, the
>     condition ``o1 == o2`` must imply ``hash(o1) == hash(o2)`` for all
>     instances ``o1`` of ``C1`` and all instances ``o2`` of ``C2``.
>     IOW, two objects shouldn't compare equal but have different hash
>     values.
>
>     Another constraint is that hashable objects, once created, should
>     never change their value (as compared by ``==``) or their hash
>     value.  If a class cannot guarantee this, it should not derive
>     from ``Hashable``; if it cannot guarantee this for certain
>     instances only, ``__hash__`` for those instances should raise a
>     ``TypeError`` exception.
>
>     **Note:** being an instance of this class does not imply that an
>     object is immutable; e.g. a tuple containing a list as a member is
>     not immutable; its ``__hash__`` method raises ``TypeError``.
>
> ``Iterable``
>     The base class for classes defining ``__iter__``.  The
>     ``__iter__`` method should always return an instance of
>     ``Iterator`` (see below).  The abstract ``__iter__`` method
>     returns an empty iterator.
>
> ``Iterator``
>     The base class for classes defining ``__next__``.  This derives
>     from ``Iterable``.  The abstract ``__next__`` method raises
>     ``StopIteration``.  The concrete ``__iter__`` method returns
>     ``self``.
>
> ``Sized``
>     The base class for classes defining ``__len__``.  The ``__len__``
>     method should return an ``Integer`` (see "Numbers" below) >= 0.
>     The abstract ``__len__`` method returns 0.  **Invariant:** If a
>     class ``C`` derives from ``Sized`` as well as from ``Iterable``,
>     the invariant ``sum(1 for x in o) == len(o)`` should hold for any
>     instance ``o`` of ``C``.
>
> ``Container``
>     The base class for classes defining ``__contains__``.  The
>     ``__contains__`` method should return a ``bool``.  The abstract
>     ``__contains__`` method returns ``False``.  **Invariant:** If a
>     class ``C`` derives from ``Container`` as well as from
>     ``Iterable``, then ``(x in o for x in o)`` should be a generator
>     yielding only True values for any instance ``o`` of ``C``.
>
>     **Note:** strictly speaking, there are three variants of this method's
>     semantics.  The first one is for sets and mappings, which is fast:
>     O(1) or O(log N).  The second one is for membership checking on
>     sequences, which is slow: O(N).  The third one is for subsequence
>     checking on (character or byte) strings, which is also slow: O(N).
>     Would it make sense to distinguish these?  The signature of the
>     third variant is different, since it takes a sequence (typically
>     of the same type as the method's target) intead of an element.
>     For now, I'm using the same type for all three.  This means that
>     is is possible for ``x in o`` to be True even though ``x`` is
>     never yielded by ``iter(o)``.  A suggested name for the third form
>     is ``Searchable``.
>
>
> Sets
> ''''
>
> These abstract classes represent various stages of "set-ness".  The
> most fundamental set operation is the membership test, written as ``x
> in s`` and implemented by ``s.__contains__(x)``.  This is already
> taken care of by the `Container`` class defined above.  Therefore, we
> define a set as a sized, iterable container for which certain
> invariants from mathematical set theory hold.
>
> The built-in type ``set`` derives from ``MutableSet``.  The built-in
> type ``frozenset`` derives from ``HashableSet``.
>
> You might wonder why we require a set to be sized -- surely certain
> infinite sets can be represented just fine in Python.  For example,
> the set of even integers could be defined like this::
>
>     class EvenIntegers(Container):
>         def __contains__(self, x):
>             return x % 2 == 0
>
> However, such sets have rather limited practical value, and deciding
> whether one such set is a subset of another would be difficult in
> general without using a symbolic algebra package.  So I consider this
> out of the scope of a pragmatic proposal like this.
>
> ``Set``
>     This is a sized, iterable, partially ordered container, i.e. a
>     subclass of ``Sized``, ``Iterable``, ``Container`` and
>     ``PartiallyOrdered``.  Not every subset of those three classes is
>     a set though!  Sets have the additional invariant that each
>     element occurs only once (as can be determined by iteration), and
>     in addition sets define concrete operators that implement the
>     inequality operations as subclass/superclass tests.  In general,
>     the invariants for finite sets in mathematics hold. [11]_
>
>     Sets with different implementations can be compared safely,
>     (usually) efficiently and correctly using the mathematical
>     definitions of the subclass/superclass operations for finite sets.
>     The ordering operations have concrete implementations; subclasses
>     may override these for speed but should maintain the semantics.
>     Because ``Set`` derives from ``Sized``, ``__eq__`` may take a
>     shortcut and returns ``False`` immediately if two sets of unequal
>     length are compared.  Similarly, ``__le__`` may return ``False``
>     immediately if the first set has more members than the second set.
>     Note that set inclusion implements only a partial ordering;
>     e.g. ``{1, 2}`` and ``{1, 3}`` are not ordered (all three of
>     ``<``, ``==`` and ``>`` return ``False`` for these arguments).
>     Sets cannot be ordered relative to mappings or sequences, but they
>     can be compared to those for equality (and then they always
>     compare unequal).
>
>     **Note:** the ``issubset`` and ``issuperset`` methods found on the
>     set type in Python 2 are not supported, as these are mostly just
>     aliases for ``__le__`` and ``__ge__``.
>
>     **Open issues:** should we define comparison of instances of
>     different concrete set types this way?
>
> ``ComposableSet``
>     This is a subclass of ``Set`` that defines abstract operators to
>     compute union, intersection, symmetric and asymmetric difference,
>     respectively ``__or__``, ``__and__``, ``__xor__`` and ``__sub__``.
>     These operators should return instances of ``ComposableSet``.  The
>     abstract implementations return no meaningful values but raise
>     ``NotImplementedError``; this is because any generic
>     implementation would have to create new instances but the ABCs
>     don't (and shouldn't, IMO) provide an API for creating new
>     instances.  The implementations of these operators should ensure
>     that the results match the mathematical definition of set
>     composition. [11]_
>
>     **Open issues:** Should ``__or__`` and friends be abstract or
>     concrete methods?  Making them abstract means that every
>     ComposableSet implementation must reimplement all of them.  But
>     making them concrete begs the question of the actual return type:
>     since the ABC doesn't (and IMO shouldn't) define the constructor
>     signature for subclasses, the concrete implementations in the ABC
>     don't have an API to construct a new instance given an iterable.
>     Perhaps the right choice is to have a static concrete factory
>     function ``fromiterable`` which takes an iterable and returns
>     a ``ComposableSet`` instance.  Subclasses can override this and
>     benefit from the default implementations of ``__or__`` etc.; or
>     they can override ``__or__`` if they want to.
>
> ``HashableSet``
>     This is a subclass of both ``ComposableSet`` and ``Hashable``.  It
>     implements a concrete ``__hash__`` method that subclasses should
>     not override; or if they do, the subclass should compute the same
>     hash value.  This is so that sets with different implementations
>     still hash to the same value, so they can be used interchangeably
>     as dictionary keys.  (A similar constraint exists on the hash
>     values for different types of numbers and strings.)
>
>     **Open issues:** Spell out the hash algorithm.  Should there be
>     another ABC that derives from Set and Hashable, but not from
>     Composable?
>
> ``MutableSet``
>     This is a subclass of ``ComposableSet`` implementing additional
>     operations to add and remove elements.  The supported methods have
>     the semantics known from the ``set`` type in Python 2 (except
>     for ``discard``, which is modeled after Java):
>
>     ``.add(x)``
>         Abstract method returning a ``bool`` that adds the element
>         ``x`` if it isn't already in the set.  It should return
>         ``True`` if ``x`` was added, ``False`` if it was already
>         there. The abstract implementation raises
>         ``NotImplementedError``.
>
>     ``.discard(x)``
>         Abstract method returning a ``bool`` that removes the element
>         ``x`` if present.  It should return ``True`` if the element
>         was present and ``False`` if it wasn't.  The abstract
>         implementation raises ``NotImplementedError``.
>
>     ``.pop()``
>         Concrete method that removes an arbitrary item.  If the set is
>         empty, it raises ``KeyError``.  The default implementation
>         removes the first item returned by the set's iterator.
>
>     ``.toggle(x)``
>         Concrete method returning a ``bool`` that adds x to the set if
>         it wasn't there, but removes it if it was there.  It should
>         return ``True`` if ``x`` was added, ``False`` if it was
>         removed.
>
>     ``.clear()``
>         Concrete method that empties the set.  The default
>         implementation repeatedly calls ``self.pop()`` until
>         ``KeyError`` is caught.  (**Note:** this is likely much slower
>         than simply creating a new set, even if an implementation
>         overrides it with a faster approach; but in some cases object
>         identity is important.)
>
>     This also supports the in-place mutating operations ``|=``,
>     ``&=``, ``^=``, ``-=``.  These are concrete methods whose right
>     operand can be an arbitrary ``Iterable``, except for ``&=``, whose
>     right operand must be a ``Container``.  This ABC does not support
>     the named methods present on the built-in concrete ``set`` type
>     that perform (almost) the same operations.
>
>
> Mappings
> ''''''''
>
> These abstract classes represent various stages of mapping-ness.  The
> ``Mapping`` class represents the most common read-only mapping API.
> However, code *accepting* a mapping is encouraged to check for the
> ``BasicMapping`` ABC when iteration is not used.  This allows for
> certain "black-box" implementations that can look up values by key but
> don't provide a convenient iteration API.  A hypothetical example
> would be an interface to a hierarchical filesystem, where keys are
> pathnames relative to some root directory.  Iterating over all
> pathnames would presumably take forever, as would counting the number
> of valid pathnames.
>
> The built-in type ``dict`` derives from ``MutableMapping``.
>
> ``BasicMapping``
>     A subclass of ``Container`` defining the following methods:
>
>     ``.__getitem__(key)``
>         Abstract method that returns the value corresponding to
>         ``key``, or raises ``KeyError``.  The implementation always
>         raises ``KeyError``.
>
>     ``.get(key, default=None)``
>         Concrete method returning ``self[key]`` if this does not raise
>         ``KeyError``, and the ``default`` value if it does.
>
>     ``.__contains__()``
>         Concrete method returning ``True`` if ``self[key]`` does not
>         raise ``KeyError``, and ``False`` if it does.
>
> ``Mapping``
>     A subclass of ``BasicMapping``, ``Iterable`` and ``Sized``.  The
>     keys of a mapping naturally form a set.  The (key, value) pairs
>     are also referred to as items.  The items also form a set.
>     Methods:
>
>     ``__len__``
>         Abstract method returning the length of the key set.
>
>     ``__iter__``
>         Abstract method returning each key in the key set exactly once.
>
>     ``__eq__``
>         Concrete method for comparing mappings.  Two mappings, even
>         with different implementations, can be compared for equality,
>         and are considered equal if and only iff their item sets are
>         equal.  **Open issues:** should we define comparison of
>         instances of different concrete mapping types this way?
>
>     ``keys``
>         Concrete method returning the key set as a ``Set``.  The
>         default concrete implementation returns a "view" on the key
>         set (meaning if the underlying mapping is modified, the view's
>         value changes correspondingly); subclasses are not required to
>         return a view but they should return a ``Set``.
>
>     ``items``
>         Concrete method returning the items as a ``Set``.  The default
>         concrete implementation returns a "view" on the item set;
>         subclasses are not required to return a view but they should
>         return a ``Set``.
>
>     ``values``
>         Concrete method returning the values as a sized, iterable
>         container (not a set!).  The default concrete implementation
>         returns a "view" on the values of the mapping; subclasses are
>         not required to return a view but they should return a sized,
>         iterable container.
>
>     The following invariant should hold for any mapping ``m``::
>
>         set(m.items()) == set(zip(m.keys(), m.values()))
>
>     i.e. iterating over the keys and the values in parallel should
>     return *corresponding* keys and values.  **Open issues:** Should
>     this always be required?  How about the stronger invariant using
>     ``list()`` instead of ``set()``?
>
> ``HashableMapping``
>     A subclass of ``Mapping`` and ``Hashable``.  The values should be
>     instances of ``Hashable``.  The concrete ``__hash__`` method
>     should be equal to ``hash(m.items())``.
>
> ``MutableMapping``
>     A subclass of ``Mapping`` that also implements some standard
>     mutating methods.  Abstract methods include ``__setitem__``,
>     ``__delitem__``.  Concrete methods include ``pop``, ``popitem``,
>     ``clear``, ``update``.  **Note:** ``setdefault`` is *not* included.
>     **Open issues:** Write out the specs for the methods.
>
>
> Sequences
> '''''''''
>
> These abstract classes represent various stages of sequence-ness.
>
> The built-in ``list`` and ``bytes`` types derive from
> ``MutableSequence``.  The built-in ``tuple`` and ``str`` types derive
> from ``HashableSequence``.
>
> ``Sequence``
>     A subclass of ``Iterable``, ``Sized``, ``Container``.  It
>     defines a new abstract method ``__getitem__`` that has a somewhat
>     complicated signature: when called with an integer, it returns an
>     element of the sequence or raises ``IndexError``; when called with
>     a ``slice`` object, it returns another ``Sequence``.  The concrete
>     ``__iter__`` method iterates over the elements using
>     ``__getitem__`` with integer arguments 0, 1, and so on, until
>     ``IndexError`` is raised.  The length should be equal to the
>     number of values returned by the iterator.
>
>     **Open issues:** Other candidate methods, which can all have
>     default concrete implementations that only depend on ``__len__``
>     and ``__getitem__`` with an integer argument: __reversed__, index,
>     count, __add__, __mul__, __eq__, __lt__, __le__.
>
> ``HashableSequence``
>     A subclass of ``Sequence`` and ``Hashable``.  The concrete
>     ``__hash__`` method should implements the hashing algorithms used
>     by tuples in Python 2.
>
> ``MutableSequence``
>     A subclass of ``Sequence`` adding some standard mutating methods.
>     Abstract mutating methods: ``__setitem__`` (for integer indices as
>     well as slices), ``__delitem__`` (ditto), ``insert``, ``append``,
>     ``reverse``.  Concrete mutating methods: ``extend``, ``pop``,
>     ``remove``.  Concrete mutating operators: ``+=``, ``*=`` (these
>     mutate the object in place).  **Note:** this does not define
>     ``sort()`` -- that is only required to exist on genuine ``list``
>     instances.
>
> **Open issues:** If all the elements of a sequence are totally
> ordered, the sequence itself can be totally ordered with respect to
> other sequences containing corresponding items of the same type.
> Should we reflect this by making ``Sequence`` derive from
> ``TotallyOrdered``?  Or ``Partiallyordered``?  Also, we could easily
> define comparison of sequences of different types, so that e.g.
> ``(1, 2, 3) == [1, 2, 3]`` and ``(1, 2) < [1, 2, 3]``.  Should we?
> (It might imply ``["a", "b"] == "ab"`` and ``[1, 2] == b"\1\2"``.)
>
>
> Strings
> -------
>
> Python 3000 has two built-in string types: byte strings (``bytes``),
> deriving from ``MutableSequence``, and (Unicode) character strings
> (``str``), deriving from ``HashableSequence``.  They also derive from
> ``TotallyOrdered``.  If we were to introduce ``Searchable``, they
> would also derive from that.
>
> **Open issues:** define the base interfaces for these so alternative
> implementations and subclasses know what they are in for.  This may be
> the subject of a new PEP or PEPs (PEP 358 should be co-opted for the
> ``bytes`` type).
>
>
> Numbers
> -------
>
> ABCs for numerical types are defined in PEP 3141.
>
>
> Guidelines for Writing ABCs
> ---------------------------
>
> Some suggestions for writing ABCs:
>
> * Use the ``@abstractmethod`` decorator.
>
> * Define abstract methods that could be useful as an end point when
>   called via a super chain.
>
> * Define concrete methods that are very simple permutations of
>   abstract methods (e.g. ``Mapping.get``).
>
> * Keep abstract classes small, one per use case instead of one per
>   concept.
>
>
> ABCs vs. Alternatives
> =====================
>
> In this section I will attempt to compare and contrast ABCs to other
> approaches that have been proposed.
>
>
> ABCs vs. Duck Typing
> --------------------
>
> Does the introduction of ABCs mean the end of Duck Typing?  I don't
> think so.  Python will not require that a class derives from
> ``BasicMapping`` or ``Sequence`` when it defines a ``__getitem__``
> method, nor will the ``x[y]`` syntax require that ``x`` is an instance
> of either ABC.  You will still be able to assign any "file-like"
> object to ``sys.stdout``, as long as it has a ``write`` method.
>
> Of course, there will be some carrots to encourage users to derive
> from the appropriate base classes; these vary from default
> implementations for certain functionality to an improved ability to
> distinguish between mappings and sequences.  But there are no sticks.
> If ``hasattr(x, __len__)`` works for you, great!  ABCs are intended to
> solve problems that don't have a good solution at all in Python 2,
> such as distinguishing between mappings and sequences.
>
>
> ABCs vs. Generic Functions
> --------------------------
>
> ABCs are compatible with Generic Functions (GFs).  For example, my own
> Generic Functions implementation [4]_ uses the classes (types) of the
> arguments as the dispatch key, allowing derived classes to override
> base classes.  Since (from Python's perspective) ABCs are quite
> ordinary classes, using an ABC in the default implementation for a GF
> can be quite appropriate.  For example, if I have an overloaded
> ``prettyprint`` function, it would make total sense to define
> pretty-printing of sets like this::
>
>     @prettyprint.register(Set)
>     def pp_set(s):
>         return "{" + ... + "}"  # Details left as an exercise
>
> and implementations for specific subclasses of Set could be added
> easily.
>
> I believe ABCs also won't present any problems for RuleDispatch,
> Phillip Eby's GF implementation in PEAK [5]_.
>
> Of course, GF proponents might claim that GFs (and concrete, or
> implementation, classes) are all you need.  But even they will not
> deny the usefulness of inheritance; and one can easily consider the
> ABCs proposed in this PEP as optional implementation base classes;
> there is no requirement that all user-defined mappings derive from
> ``BasicMapping``.
>
>
> ABCs vs. Interfaces
> -------------------
>
> ABCs are not intrinsically incompatible with Interfaces, but there is
> considerable overlap.  For now, I'll leave it to proponents of
> Interfaces to explain why Interfaces are better.  I expect that much
> of the work that went into e.g. defining the various shades of
> "mapping-ness" and the nomenclature could easily be adapted for a
> proposal to use Interfaces instead of ABCs.
>
> "Interfaces" in this context refers to a set of proposals for
> additional metadata elements attached to a class which are not part of
> the regular class hierarchy, but do allow for certain types of
> inheritance testing.
>
> Such metadata would be designed, at least in some proposals, so as to
> be easily mutable by an application, allowing application writers to
> override the normal classification of an object.
>
> The drawback to this idea of attaching mutable metadata to a class is
> that classes are shared state, and mutating them may lead to conflicts
> of intent.  Additionally, the need to override the classification of
> an object can be done more cleanly using generic functions: In the
> simplest case, one can define a "category membership" generic function
> that simply returns False in the base implementation, and then provide
> overrides that return True for any classes of interest.
>
>
> References
> ==========
>
> .. [1] An Introduction to ABC's, by Talin
>    (http://mail.python.org/pipermail/python-3000/2007-April/006614.html)
>
> .. [2] Incomplete implementation prototype, by GvR
>    (http://svn.python.org/view/sandbox/trunk/abc/)
>
> .. [3] Possible Python 3K Class Tree?, wiki page created by Bill Janssen
>    (http://wiki.python.org/moin/AbstractBaseClasses)
>
> .. [4] Generic Functions implementation, by GvR
>    (http://svn.python.org/view/sandbox/trunk/overload/)
>
> .. [5] Charming Python: Scaling a new PEAK, by David Mertz
>    (http://www-128.ibm.com/developerworks/library/l-cppeak2/)
>
> .. [6] Implementation of @abstractmethod
>    (http://python.org/sf/1706989)
>
> .. [7] Unifying types and classes in Python 2.2, by GvR
>    (http://www.python.org/download/releases/2.2.3/descrintro/)
>
> .. [8] Putting Metaclasses to Work: A New Dimension in Object-Oriented
>    Programming, by Ira R. Forman and Scott H. Danforth
>    (http://www.amazon.com/gp/product/0201433052)
>
> .. [9] Partial order, in Wikipedia
>    (http://en.wikipedia.org/wiki/Partial_order)
>
> .. [10] Total order, in Wikipedia
>    (http://en.wikipedia.org/wiki/Total_order)
>
> .. [11] Finite set, in Wikipedia
>    (http://en.wikipedia.org/wiki/Finite_set)
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
>
> ..
>    Local Variables:
>    mode: indented-text
>    indent-tabs-mode: nil
>    sentence-end-double-space: t
>    fill-column: 70
>    coding: utf-8
>    End:
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/jyasskin%40gmail.com
>


-- 
Namast?,
Jeffrey Yasskin
http://jeffrey.yasskin.info/

"Religion is an improper response to the Divine." ? "Skinny Legs and
All", by Tom Robbins

From jyasskin at gmail.com  Sat Apr 28 01:59:26 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Fri, 27 Apr 2007 16:59:26 -0700
Subject: [Python-3000] string module trimming
In-Reply-To: <fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<f039of$ih$1@sea.gmane.org>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
Message-ID: <5d44f72f0704271659u2a991d7cy6bac14bfa5f61518@mail.gmail.com>

On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/18/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> > > Today, string.letters works most easily with ASCII supersets, and is
> > > effectively limited to 8-bit encodings.  Once everything is unicode, I
> > > don't think that 8-bit restriction should apply any more.
>
> > But we already went over this. There are over 40K letters in Unicode.
> > It simply makes no sense to have a string.letters approaching that
> > size.
>
> Agreed.  But there aren't 40K (alphabetic) letters in any particular
> locale.  Most individual languages will have less than 100.

Here's a relevant bunch of data from the CLDR:
http://www.unicode.org/cldr/data/charts/by_type/misc.exemplarCharacters.html

-- 
Namast?,
Jeffrey Yasskin

From ndbecker2 at gmail.com  Sat Apr 28 02:32:49 2007
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 27 Apr 2007 20:32:49 -0400
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
References: <46327CC9.4040901@v.loewis.de>
Message-ID: <f0u4nh$nqt$1@sea.gmane.org>

"Martin v. L?wis" wrote:

> I propose the PEP below for Py3k.
> 
> Regards,
> Martin
> 
> PEP: 3122
> Title: Dropping PyObject_HEAD
> Version: $Revision: 54998 $
> Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 27-Apr-2007
> Python-Version: 3.0
> Post-History:
> 
It sounds like a good thing, but what is the impact?  How much code will be
broken by this proposal?


From jimjjewett at gmail.com  Sat Apr 28 03:06:06 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 27 Apr 2007 21:06:06 -0400
Subject: [Python-3000] string module trimming
In-Reply-To: <5d44f72f0704271659u2a991d7cy6bac14bfa5f61518@mail.gmail.com>
References: <ee2a432c0704170203u7df8138en337a8631eba8912e@mail.gmail.com>
	<fb6fbf560704171555i3ca20a81g20f794f880bb95cf@mail.gmail.com>
	<ca471dc20704171607q33a7b0aftbf343dc7ff4001e2@mail.gmail.com>
	<fb6fbf560704181224g27aac443j980b0ee2fe8c8c11@mail.gmail.com>
	<ca471dc20704181418w11eeb1a1lbd5d5610c2f375d2@mail.gmail.com>
	<fb6fbf560704181458w387f1338t72f2840f761eae0@mail.gmail.com>
	<ca471dc20704181507r160a7d82y834735258d2b04c8@mail.gmail.com>
	<ca471dc20704181508x11e57a81udd91b5b531a12d97@mail.gmail.com>
	<fb6fbf560704181608p3f7408b3l9c25a114e98d6c55@mail.gmail.com>
	<5d44f72f0704271659u2a991d7cy6bac14bfa5f61518@mail.gmail.com>
Message-ID: <fb6fbf560704271806x646a76b9q331fec8e95195385@mail.gmail.com>

On 4/27/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On 4/18/07, Jim Jewett <jimjjewett at gmail.com> wrote:

> > Agreed.  But there aren't 40K (alphabetic) letters in any particular
> > locale.  Most individual languages will have less than 100.

> Here's a relevant bunch of data from the CLDR:
> http://www.unicode.org/cldr/data/charts/by_type/misc.exemplarCharacters.html

http://www.unicode.org/Public/UNIDATA/Scripts.txt is also relevant,
but I can't quite interpret it.

There are 5020 "Common" code points.  These are mostly non-letters,
but I suppose they could appear in some langauges.

Latin script has 1070 characters; most Latin-script languages use only
a small fraction of them.  The standard ASCII alphabet is still only
26 lower + 26 capital, but there are plenty of characters that get
used in some language or other. (The largest single block is 208
letters from LATIN CAPITAL LETTER DZ WITH CARON to LATIN SMALL LETTER
EZH WITH CURL)

-jJ

From rhamph at gmail.com  Sat Apr 28 04:02:18 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 27 Apr 2007 20:02:18 -0600
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>
Message-ID: <aac2c7cb0704271902k57c3627y8e463b8e9c37844c@mail.gmail.com>

On 4/27/07, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> Guido van Rossum wrote:
>
> >     Another constraint is that hashable objects, once created, should
> >     never change their value (as compared by ``==``) or their hash value.
> >     If a class cannot guarantee this, it should not derive from
> >     ``Hashable``; if it cannot guarantee this for certain instances only,
> >     ``__hash__`` for those instances should raise a ``TypeError``
> >     exception.
>
> Shouldn't this rather be a ValueError since it occurs not because of the
> type of the object in question, but its value (while in general, there are
> instances of the same type representing other values which are hashable)?

>>> hash([])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list objects are unhashable

Normally the exception is raised because the type is wrong.  Requiring
both TypeError and ValueError be caught just for this special case
would be confusing.  A subclass of both could be created, but that
seems pedantic with no practical value.

-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Sat Apr 28 05:31:09 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 27 Apr 2007 20:31:09 -0700
Subject: [Python-3000] PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <aac2c7cb0704271902k57c3627y8e463b8e9c37844c@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<pan.2007.04.27.16.21.57.837729@ID-174572.user.uni-berlin.de>
	<aac2c7cb0704271902k57c3627y8e463b8e9c37844c@mail.gmail.com>
Message-ID: <ca471dc20704272031i617938aeu76b0baef1c001120@mail.gmail.com>

On 4/27/07, Adam Olsen <rhamph at gmail.com> wrote:
> On 4/27/07, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> > Guido van Rossum wrote:
> >
> > >     Another constraint is that hashable objects, once created, should
> > >     never change their value (as compared by ``==``) or their hash value.
> > >     If a class cannot guarantee this, it should not derive from
> > >     ``Hashable``; if it cannot guarantee this for certain instances only,
> > >     ``__hash__`` for those instances should raise a ``TypeError``
> > >     exception.
> >
> > Shouldn't this rather be a ValueError since it occurs not because of the
> > type of the object in question, but its value (while in general, there are
> > instances of the same type representing other values which are hashable)?
>
> >>> hash([])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: list objects are unhashable
>
> Normally the exception is raised because the type is wrong.  Requiring
> both TypeError and ValueError be caught just for this special case
> would be confusing.  A subclass of both could be created, but that
> seems pedantic with no practical value.

Right. The hash of a tuple, or any other hashable container, is
computed from the hashes of its elements. If any of the elements
happens to be not hashable then it will raise a TypeError and that
will just be passed along by the toplevel hash() call. Turning this
into a ValueError would just be an error-masking accident waiting to
happen. If the sub-hash() call raises another exception, that gets
passed through too.

(At one point in the past I considered making __hash__ a dynamic
instance attribute that would be set to None when the value is
unhashable, and computed dynmically. But computing whether it should
be None is just about as expensive as computing the __hash__ function,
so I abandoned this idea. There's little merit to LBYL hashing
anyway.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Sat Apr 28 07:37:07 2007
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 28 Apr 2007 07:37:07 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <f0u4nh$nqt$1@sea.gmane.org>
References: <46327CC9.4040901@v.loewis.de> <f0u4nh$nqt$1@sea.gmane.org>
Message-ID: <4632DD83.2030301@v.loewis.de>

> It sounds like a good thing, but what is the impact?  How much code will be
> broken by this proposal?

Nearly all extension modules will be broken, as PyObject_HEAD goes away.

However, it is likely that Py3k breaks all extension modules for a lot
of other reasons, as well.

Fixing this specific breakage would be easy: Just replace PyObject_HEAD
with "PyObject ob_base;". How many modules then still break, I don't
know. I expect access to ob_type is frequent in Foo_Check() macros;
those need to be rewritten to use Py_Type(op).

Martin

From rasky at develer.com  Sat Apr 28 14:07:55 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 28 Apr 2007 14:07:55 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <46327CC9.4040901@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de>
Message-ID: <f0vder$5p6$1@sea.gmane.org>

On 28/04/2007 0.44, Martin v. L?wis wrote:

> PEP: 3122
> Title: Dropping PyObject_HEAD
> Version: $Revision: 54998 $
> Last-Modified: $Date: 2007-04-27 10:31:58 +0200 (Fr, 27 Apr 2007) $
> Author: Martin v. L?wis <martin at v.loewis.de>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 27-Apr-2007
> Python-Version: 3.0
> Post-History:

Strong +1 from me. This looks like the exact kind of cleanup Py3k has been 
thought for. It's a one-chance to modify Python's code to adhere to the C 
standard, we should not miss it.
-- 
Giovanni Bajo



From ironfroggy at gmail.com  Sat Apr 28 15:13:29 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sat, 28 Apr 2007 09:13:29 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
Message-ID: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com>

On 4/26/07, Guido van Rossum <guido at python.org> wrote:
> On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > The current ABC proposal is to use isinstance as the test; Jeffrey
> > Yaskin's numbers PEP highlighted the weakness there with a concrete
> > example.
> >
> > If you need to an abstraction less powerful than an existing ABC,
> > you're out of luck; you can't just assert that the existing class is
> > already sufficient, nor can you expect everyone else to use multiple
> > annotations.
>
> I now have a proposal to allow overloading isinstance() and
> issubclass(), by defining special (class) methods on the second
> argument. See http://python.org/sf/1708353. Does this need a PEP? The
> unit test shows that it can be used to support the use case described
> above:
>
>
> class ABC(type):
>
>     def __instancecheck__(cls, inst):
>         """Implement isinstance(inst, cls)."""
>         return any(cls.__subclasscheck__(c)
>                    for c in {type(inst), inst.__class__})
>
>     def __subclasscheck__(cls, sub):
>         """Implement issubclass(sub, cls)."""
>         candidates = cls.__dict__.get("__subclass__", set())
>         return any(c in candidates for c in sub.mro())
>
>
> class Integer(metaclass=ABC):
>
>     __subclass__ = {int}
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com

I'm just going to jump into this and voice a concern that allowing
overriding of isinstance and issubclass seems like a Bad Idea. They
should be trustworthy and predictable, and allowing classes to change
how objects are considered instances of them seems like the wrong
thing to do. I'm not saying that I don't think it should be done, but
that it feels quite wrong. When I do, rarely, use isinstance or such,
and I need to check isinstance(f, Foo), I know exactly what I am
saying. But if Foo can change the meaning of that, can I trust it
anymore?

Not everything should be dynamic. I am not sure which side of the
fence this falls on, but right now, in my mind, it teeters right on
the fence itself.

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From daniel at stutzbachenterprises.com  Sat Apr 28 15:57:45 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Sat, 28 Apr 2007 08:57:45 -0500
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <46327CC9.4040901@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de>
Message-ID: <eae285400704280657t5a0a56b7q199394c797f8c02a@mail.gmail.com>

To facilitate migration, I have a few suggestions:

- add the new access macros to 2.6 (defined appropriately)
- in 3.0, instead of removing PyObject_HEAD and company, redefine them
appropriately, .e.g, #define PyObject_HEAD PyObject obtype;

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From guido at python.org  Sat Apr 28 16:54:58 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Apr 2007 07:54:58 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
	<76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com>
Message-ID: <ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>

On 4/28/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On 4/26/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > The current ABC proposal is to use isinstance as the test; Jeffrey
> > > Yaskin's numbers PEP highlighted the weakness there with a concrete
> > > example.
> > >
> > > If you need to an abstraction less powerful than an existing ABC,
> > > you're out of luck; you can't just assert that the existing class is
> > > already sufficient, nor can you expect everyone else to use multiple
> > > annotations.
> >
> > I now have a proposal to allow overloading isinstance() and
> > issubclass(), by defining special (class) methods on the second
> > argument. See http://python.org/sf/1708353. Does this need a PEP? The
> > unit test shows that it can be used to support the use case described
> > above:
> >
> >
> > class ABC(type):
> >
> >     def __instancecheck__(cls, inst):
> >         """Implement isinstance(inst, cls)."""
> >         return any(cls.__subclasscheck__(c)
> >                    for c in {type(inst), inst.__class__})
> >
> >     def __subclasscheck__(cls, sub):
> >         """Implement issubclass(sub, cls)."""
> >         candidates = cls.__dict__.get("__subclass__", set())
> >         return any(c in candidates for c in sub.mro())
> >
> >
> > class Integer(metaclass=ABC):
> >
> >     __subclass__ = {int}
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com
>
> I'm just going to jump into this and voice a concern that allowing
> overriding of isinstance and issubclass seems like a Bad Idea. They
> should be trustworthy and predictable, and allowing classes to change
> how objects are considered instances of them seems like the wrong
> thing to do. I'm not saying that I don't think it should be done, but
> that it feels quite wrong. When I do, rarely, use isinstance or such,
> and I need to check isinstance(f, Foo), I know exactly what I am
> saying. But if Foo can change the meaning of that, can I trust it
> anymore?
>
> Not everything should be dynamic. I am not sure which side of the
> fence this falls on, but right now, in my mind, it teeters right on
> the fence itself.

Note though that only the second argument to either function can
overload the rules. IOW if you write isinstance(x, C), there is no way
that x could attempt to lie; but C could. Similar for issubclass(D, C)
-- only D can change the outcome. Whether or not a particular class
overload these functions ought to be part of its documentation. The
builtins won't be overloading them; you will be able to trust e.g.
isinstance(x, int). But it might be true that isinstance(42, Ring) if
Ring overloads isinstance -- however you will have been duly warned of
this possibility.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From exarkun at divmod.com  Sat Apr 28 18:12:04 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Sat, 28 Apr 2007 12:12:04 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>
Message-ID: <20070428161204.19381.458566240.divmod.quotient.6050@ohm>

On Sat, 28 Apr 2007 07:54:58 -0700, Guido van Rossum <guido at python.org> wrote:
>On 4/28/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
>> On 4/26/07, Guido van Rossum <guido at python.org> wrote:
>> > On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
>> > > The current ABC proposal is to use isinstance as the test; Jeffrey
>> > > Yaskin's numbers PEP highlighted the weakness there with a concrete
>> > > example.
>> > >
>> > > If you need to an abstraction less powerful than an existing ABC,
>> > > you're out of luck; you can't just assert that the existing class is
>> > > already sufficient, nor can you expect everyone else to use multiple
>> > > annotations.
>> >
>> > I now have a proposal to allow overloading isinstance() and
>> > issubclass(), by defining special (class) methods on the second
>> > argument. See http://python.org/sf/1708353. Does this need a PEP? The
>> > unit test shows that it can be used to support the use case described
>> > above:
>> >
>> >
>> > class ABC(type):
>> >
>> >     def __instancecheck__(cls, inst):
>> >         """Implement isinstance(inst, cls)."""
>> >         return any(cls.__subclasscheck__(c)
>> >                    for c in {type(inst), inst.__class__})
>> >
>> >     def __subclasscheck__(cls, sub):
>> >         """Implement issubclass(sub, cls)."""
>> >         candidates = cls.__dict__.get("__subclass__", set())
>> >         return any(c in candidates for c in sub.mro())
>> >
>> >
>> > class Integer(metaclass=ABC):
>> >
>> >     __subclass__ = {int}
>> >
>> >
>> > --
>> > --Guido van Rossum (home page: http://www.python.org/~guido/)
>> > _______________________________________________
>> > Python-3000 mailing list
>> > Python-3000 at python.org
>> > http://mail.python.org/mailman/listinfo/python-3000
>> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com
>>
>> I'm just going to jump into this and voice a concern that allowing
>> overriding of isinstance and issubclass seems like a Bad Idea. They
>> should be trustworthy and predictable, and allowing classes to change
>> how objects are considered instances of them seems like the wrong
>> thing to do. I'm not saying that I don't think it should be done, but
>> that it feels quite wrong. When I do, rarely, use isinstance or such,
>> and I need to check isinstance(f, Foo), I know exactly what I am
>> saying. But if Foo can change the meaning of that, can I trust it
>> anymore?
>>
>> Not everything should be dynamic. I am not sure which side of the
>> fence this falls on, but right now, in my mind, it teeters right on
>> the fence itself.
>
>Note though that only the second argument to either function can
>overload the rules. IOW if you write isinstance(x, C), there is no way
>that x could attempt to lie; but C could. Similar for issubclass(D, C)
>-- only D can change the outcome. Whether or not a particular class
>overload these functions ought to be part of its documentation. The
>builtins won't be overloading them; you will be able to trust e.g.
>isinstance(x, int). But it might be true that isinstance(42, Ring) if
>Ring overloads isinstance -- however you will have been duly warned of
>this possibility.
>

Aside from the way in which `x' can already lie:

    >>> class X(object):
    ...     __class__ = property(lambda self: int)
    ...
    >>> isinstance(X(), int)
    True
    >>>

Is this behavior changed/going to be changed in Py3k?

Jean-Paul

From guido at python.org  Sat Apr 28 18:17:38 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Apr 2007 09:17:38 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <20070428161204.19381.458566240.divmod.quotient.6050@ohm>
References: <ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>
	<20070428161204.19381.458566240.divmod.quotient.6050@ohm>
Message-ID: <ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.com>

On 4/28/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> Aside from the way in which `x' can already lie:
>
>     >>> class X(object):
>     ...     __class__ = property(lambda self: int)
>     ...
>     >>> isinstance(X(), int)
>     True
>     >>>
>
> Is this behavior changed/going to be changed in Py3k?

I'm not particularly enamored with it, but I believe it once served a
purpose for Zope. Does anyone know if it is still needed?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nicko at nicko.org  Sat Apr 28 18:14:41 2007
From: nicko at nicko.org (Nicko van Someren)
Date: Sat, 28 Apr 2007 17:14:41 +0100
Subject: [Python-3000] PEP 30xx: Access to Module/Class/Function
	CurrentlyBeing Defined (this)
In-Reply-To: <f0hc0j$2pg$1@sea.gmane.org>
References: <fb6fbf560704222005le4798a4j5daa5e71e644f069@mail.gmail.com>
	<f0hc0j$2pg$1@sea.gmane.org>
Message-ID: <5AE9FDD9-A78D-4C44-80A6-93515819BC84@nicko.org>

On 23 Apr 2007, at 05:17, Terry Reedy wrote:

> "Jim Jewett" <jimjjewett at gmail.com> wrote in
> message  
> news:fb6fbf560704222005le4798a4j5daa5e71e644f069 at mail.gmail.com...
> |    Functions (including methods) often want access to themselves,
> |    usually for a private storage location.
>
> Or for true recursion.  At present, a function has to 'hope' that its
> definition name is not rebound.

Indeed.  It's much clearing than having to write:
	Y = lambda f: partial(f,f)
	fac = Y(lambda f, n: n*f(f,n-1) if n>0 else 1)
:-)

Seriously though, while I have to confess never, ever having found  
rebinding of function names a limiting factor in writing recursive  
code I'm still in favour of the proposal.  Personally I think  
function-local storage is a more likely use.

+1
	Nicko


From nas at arctrix.com  Sat Apr 28 20:02:56 2007
From: nas at arctrix.com (Neil Schemenauer)
Date: Sat, 28 Apr 2007 18:02:56 +0000 (UTC)
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
References: <46327CC9.4040901@v.loewis.de>
Message-ID: <f1028g$3o0$1@sea.gmane.org>

Martin v. L?wis <martin at v.loewis.de> wrote:
> I propose the PEP below for Py3k.

I think it's a good idea.  It would be nice if the PEP included
something about forwards compatibility.  Ideally, there should be a
way to write a module that works with both 2.x and 3.x (e.g. using
preprocessor macros).  Obviously the 3.x design should not be
compromised for this sake though.

  Neil


From ironfroggy at gmail.com  Sat Apr 28 20:27:30 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sat, 28 Apr 2007 14:27:30 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.com>
References: <ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>
	<20070428161204.19381.458566240.divmod.quotient.6050@ohm>
	<ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.com>
Message-ID: <76fd5acf0704281127v74294545r6bf9573d3c83f21c@mail.gmail.com>

On 4/28/07, Guido van Rossum <guido at python.org> wrote:
> On 4/28/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> > Aside from the way in which `x' can already lie:
> >
> >     >>> class X(object):
> >     ...     __class__ = property(lambda self: int)
> >     ...
> >     >>> isinstance(X(), int)
> >     True
> >     >>>
> >
> > Is this behavior changed/going to be changed in Py3k?
>
> I'm not particularly enamored with it, but I believe it once served a
> purpose for Zope. Does anyone know if it is still needed?
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)

Consider this example:

>>> class X(object):
	pass
>>> class Y(object):
	a = 10
>>> x = X()
>>> x.__class__ = Y
>>> x.a
10
>>> isinstance(x, Y)
True

Now, I'm not saying we must keep this behavior, but it does have its
merits. In particular, the cases of type-changing objects, which I'm
not convinced always have a place, but thats not the point. The point
I want to make is that x is not lying about its type. It IS a Y,
because __class__ determines its type. The breakage may not be that x
could lie and say its an int, but that it actually can't its a
deception of the class, not the type, because you can't assign
x.__class__ = int, for example. Either the behavior should work for
any type, thus allowing heap types to be assigned to __class__
attributes, changing the type of objects (but, obviously immutables
would disallow __class__ assignment). Or, an objects type should not
be determined simply by the value of attributes, __class__, at all.

Does that make sense?
-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From guido at python.org  Sat Apr 28 20:57:37 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Apr 2007 11:57:37 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <76fd5acf0704281127v74294545r6bf9573d3c83f21c@mail.gmail.com>
References: <ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>
	<20070428161204.19381.458566240.divmod.quotient.6050@ohm>
	<ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.com>
	<76fd5acf0704281127v74294545r6bf9573d3c83f21c@mail.gmail.com>
Message-ID: <ca471dc20704281157h635a41acq1baef391c3181015@mail.gmail.com>

On 4/28/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On 4/28/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/28/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> > > Aside from the way in which `x' can already lie:
> > >
> > >     >>> class X(object):
> > >     ...     __class__ = property(lambda self: int)
> > >     ...
> > >     >>> isinstance(X(), int)
> > >     True
> > >     >>>
> > >
> > > Is this behavior changed/going to be changed in Py3k?
> >
> > I'm not particularly enamored with it, but I believe it once served a
> > purpose for Zope. Does anyone know if it is still needed?
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
>
> Consider this example:
>
> >>> class X(object):
>         pass
> >>> class Y(object):
>         a = 10
> >>> x = X()
> >>> x.__class__ = Y
> >>> x.a
> 10
> >>> isinstance(x, Y)
> True
>
> Now, I'm not saying we must keep this behavior, but it does have its
> merits. In particular, the cases of type-changing objects, which I'm
> not convinced always have a place, but thats not the point. The point
> I want to make is that x is not lying about its type. It IS a Y,
> because __class__ determines its type. The breakage may not be that x
> could lie and say its an int, but that it actually can't its a
> deception of the class, not the type, because you can't assign
> x.__class__ = int, for example. Either the behavior should work for
> any type, thus allowing heap types to be assigned to __class__
> attributes, changing the type of objects (but, obviously immutables
> would disallow __class__ assignment). Or, an objects type should not
> be determined simply by the value of attributes, __class__, at all.
>
> Does that make sense?

Alas not much. Your code example is about *actually* changing the type
of an object, which is a completely different issue, and is
constrained by the object lay-out (try changing the class where both
classes use different __slots__).

Overriding isinstance is about what classes *claim* that the object is
an instance of them, not for the purpose of changing the object's
behavior or representation, but purely for the sake of API
introspection.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at openendsystems.com  Sat Apr 28 21:00:18 2007
From: pedronis at openendsystems.com (Samuele Pedroni)
Date: Sat, 28 Apr 2007 21:00:18 +0200
Subject: [Python-3000] *View in abc.py not inheriting Iterable Re: PEP 3119
 - Introducing Abstract Base Classes
In-Reply-To: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
Message-ID: <463399C2.3090709@openendsystems.com>

Shouldn't the various *View in abc.py be Iterables (inherit from Iterable) ?



From guido at python.org  Sat Apr 28 21:17:29 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Apr 2007 12:17:29 -0700
Subject: [Python-3000] *View in abc.py not inheriting Iterable Re: PEP
	3119 - Introducing Abstract Base Classes
In-Reply-To: <463399C2.3090709@openendsystems.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<463399C2.3090709@openendsystems.com>
Message-ID: <ca471dc20704281217j4f3656dla2e833f5b6d879d9@mail.gmail.com>

On 4/28/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
> Shouldn't the various *View in abc.py be Iterables (inherit from Iterable) ?

Oops, it looks like this is a mess. There are two classes
_MappingView. Ignore them for the sake of the PEP. I'll clean them up
later. We probably need fewer mapping classes, nor more...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at openendsystems.com  Sat Apr 28 21:28:59 2007
From: pedronis at openendsystems.com (Samuele Pedroni)
Date: Sat, 28 Apr 2007 21:28:59 +0200
Subject: [Python-3000] isinstance(.,
 Iterable) vs lookup('__iter__') != None Re: *View in abc.py not
 inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <ca471dc20704281217j4f3656dla2e833f5b6d879d9@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>	
	<463399C2.3090709@openendsystems.com>
	<ca471dc20704281217j4f3656dla2e833f5b6d879d9@mail.gmail.com>
Message-ID: <4633A07B.8080706@openendsystems.com>

Guido van Rossum wrote:
> On 4/28/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
>> Shouldn't the various *View in abc.py be Iterables (inherit from 
>> Iterable) ?
>
> Oops, it looks like this is a mess. There are two classes
> _MappingView. Ignore them for the sake of the PEP. I'll clean them up
> later. We probably need fewer mapping classes, nor more...
>
It seems that something that defines __iter__ should also subclass Iterable,
in this kind of situation where the special method is unique enough it seems
quite repetitive to have to do both. It seems easy to forget to subclass.

Shouldn't now that there are isinstance hooks Iterable be defined again 
in terms
of just checking for the presence of '__iter__' lookup-wise?  I suppose 
there are some other similar
cases, of course in other situations the ABC carry an intention that 
goes beyond the presence
of methods.



From guido at python.org  Sat Apr 28 22:30:06 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 28 Apr 2007 13:30:06 -0700
Subject: [Python-3000] isinstance(.,
	Iterable) vs lookup('__iter__') != None Re: *View in abc.py not
	inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <4633A07B.8080706@openendsystems.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>
	<463399C2.3090709@openendsystems.com>
	<ca471dc20704281217j4f3656dla2e833f5b6d879d9@mail.gmail.com>
	<4633A07B.8080706@openendsystems.com>
Message-ID: <ca471dc20704281330o44dea445wcfbf196019827239@mail.gmail.com>

On 4/28/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
> Guido van Rossum wrote:
> > On 4/28/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
> >> Shouldn't the various *View in abc.py be Iterables (inherit from
> >> Iterable) ?
> >
> > Oops, it looks like this is a mess. There are two classes
> > _MappingView. Ignore them for the sake of the PEP. I'll clean them up
> > later. We probably need fewer mapping classes, nor more...
> >
> It seems that something that defines __iter__ should also subclass Iterable,
> in this kind of situation where the special method is unique enough it seems
> quite repetitive to have to do both. It seems easy to forget to subclass.
>
> Shouldn't now that there are isinstance hooks Iterable be defined again
> in terms
> of just checking for the presence of '__iter__' lookup-wise?  I suppose
> there are some other similar
> cases, of course in other situations the ABC carry an intention that
> goes beyond the presence
> of methods.

Hm, this certainly sounds like an interesting avenue. So Iterable
could override isinstance and issubclass that just check for the
presence of __iter__, and Sized could check for __len__, Hashable
could check for __hash__, Container could check for __contains__. I
think the others convey more meaning though; not everything iterable
sized container is a set (counter-examples are lists and bags). So it
would affect the "one trick ponies" section only. (Should Iterator
just check for __next__ and leave __iter__ up to the imagination?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at openendsystems.com  Sat Apr 28 23:01:06 2007
From: pedronis at openendsystems.com (Samuele Pedroni)
Date: Sat, 28 Apr 2007 23:01:06 +0200
Subject: [Python-3000] isinstance(.,
 Iterable) vs lookup('__iter__') != None Re: *View in abc.py not
 inheriting Iterable Re: PEP 3119 - Introducing Abstract Base Classes
In-Reply-To: <ca471dc20704281330o44dea445wcfbf196019827239@mail.gmail.com>
References: <ca471dc20704261150o45a3fa22u16a515b3d4aa14ba@mail.gmail.com>	
	<463399C2.3090709@openendsystems.com>	
	<ca471dc20704281217j4f3656dla2e833f5b6d879d9@mail.gmail.com>	
	<4633A07B.8080706@openendsystems.com>
	<ca471dc20704281330o44dea445wcfbf196019827239@mail.gmail.com>
Message-ID: <4633B612.1020405@openendsystems.com>

Guido van Rossum wrote:
> On 4/28/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
>> Guido van Rossum wrote:
>> > On 4/28/07, Samuele Pedroni <pedronis at openendsystems.com> wrote:
>> >> Shouldn't the various *View in abc.py be Iterables (inherit from
>> >> Iterable) ?
>> >
>> > Oops, it looks like this is a mess. There are two classes
>> > _MappingView. Ignore them for the sake of the PEP. I'll clean them up
>> > later. We probably need fewer mapping classes, nor more...
>> >
>> It seems that something that defines __iter__ should also subclass 
>> Iterable,
>> in this kind of situation where the special method is unique enough 
>> it seems
>> quite repetitive to have to do both. It seems easy to forget to 
>> subclass.
>>
>> Shouldn't now that there are isinstance hooks Iterable be defined again
>> in terms
>> of just checking for the presence of '__iter__' lookup-wise?  I suppose
>> there are some other similar
>> cases, of course in other situations the ABC carry an intention that
>> goes beyond the presence
>> of methods.
>
> Hm, this certainly sounds like an interesting avenue. So Iterable
> could override isinstance and issubclass that just check for the
> presence of __iter__, and Sized could check for __len__, Hashable
> could check for __hash__, Container could check for __contains__. I
> think the others convey more meaning though; not everything iterable
> sized container is a set (counter-examples are lists and bags). So it
> would affect the "one trick ponies" section only. 
yes, those are the one I had in mind. I think that designs that avoid to
have to repeat oneself unless the repetion really carries some meaning
are better. This would make the transation from current idioms
to 3.0 ABC idioms smoother and more sensible.

> (Should Iterator
> just check for __next__ and leave __iter__ up to the imagination?)
>
I think it would be a reasonable compromise, there's a trade off between
encouraging to subclass Iterator to get the default __iter__ for free,
vs. having to change current iterators to subclass the right thing to
pass future checks.

From pje at telecommunity.com  Sat Apr 28 23:17:38 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 28 Apr 2007 17:17:38 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.co
 m>
References: <20070428161204.19381.458566240.divmod.quotient.6050@ohm>
	<ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>
	<20070428161204.19381.458566240.divmod.quotient.6050@ohm>
Message-ID: <5.1.1.6.0.20070428171439.0420b008@sparrow.telecommunity.com>

At 09:17 AM 4/28/2007 -0700, Guido van Rossum wrote:
>On 4/28/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> > Aside from the way in which `x' can already lie:
> >
> >     >>> class X(object):
> >     ...     __class__ = property(lambda self: int)
> >     ...
> >     >>> isinstance(X(), int)
> >     True
> >     >>>
> >
> > Is this behavior changed/going to be changed in Py3k?
>
>I'm not particularly enamored with it, but I believe it once served a
>purpose for Zope. Does anyone know if it is still needed?

AFAIK, it's still used to support proxies in Zope 3, and my own ProxyTypes 
library on the Cheeseshop (and some other libraries I've written that use 
it) depend on it.  I believe Chandler uses this, too.

In all cases, it's to allow proxies to appear to be the object they're 
proxying.  I wouldn't be surprised if weakref.proxy does the same thing.


From jyasskin at gmail.com  Sat Apr 28 23:55:36 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sat, 28 Apr 2007 14:55:36 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <fb6fbf560704251423qaac984cv6d81821f660b8940@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<fb6fbf560704251423qaac984cv6d81821f660b8940@mail.gmail.com>
Message-ID: <5d44f72f0704281455o7c3c2217pa3d599993f3f8018@mail.gmail.com>

On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> >     class MonoidUnderPlus(Abstract):
>
> Is this useful?  Just because two things are both Monoid instances
> doesn't mean I can add them -- they have to be part of the same
> Monoid.  By the time you do
>
>     assert isinstance(a, MonoidUnderPlus)
>     assert isinstance(b, MonoidUnderPlus)
>     assert isinstance(a, b.__class__) or isinstance(b, a.__class__)
>
> I'm not sure how much you've really saved.

I brought this up on the PEP 3119 thread, as an issue for
TotallyOrdered too. The exact syntax for checking that two objects are
"in the same instance of ABC X" should be decided there.

As to the benefits of MonoidUnderPlus over just hasattr(a,
'__plus__'), by knowing that the operation is associative, a function
may be able to speed itself up. A somewhat academic example is a
SortedList type:
  class SortedList(MonoidUnderPlus):
    def __init__(self, iterable): self.contents = sorted(iterable)
    def __add__(lhs, rhs): return merge(lhs, rhs)
Then (ignoring that sum currently requires numbers) we can define mysorted() as:
  def mysorted(iterable): return sum(map(SortedList, iterable)).contents
With sum() implemented as a simple for loop, this is insertion sort,
taking O(n^2) time. However, by noticing that + is associative on its
argument, sum can be defined (assume the annotation implies the
appropriate checking) roughly as:
  def sum(lst : SequenceOfMonoidUnderPlus):
    if len(lst) == 1: return lst[0]
    else: return sum(lst[:len(lst)//2]) + sum(lst[len(lst)//2+1:]
and we've transformed the algorithm into merge sort, taking O(n log n)
time. (Hm, the zero element isn't as useful here as in other languages
since you can't get a type without an instance. Maybe it should go
leave the interface.)

This optimization is, of course, incorrect if + is not associative.

> > **Open issue:** Do we want to give people a choice of which of the
> > following to define, or should we pick one arbitrarily?::
>
> >         def __neg__(self):
> >         def __sub__(self, other):
> Probably better to pick one

Will do.

-- 
Namast?,
Jeffrey Yasskin

From baptiste13 at altern.org  Sun Apr 29 00:01:17 2007
From: baptiste13 at altern.org (Baptiste Carvello)
Date: Sun, 29 Apr 2007 00:01:17 +0200
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <07Apr26.102627pdt."57996"@synergy1.parc.xerox.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<07Apr26.102627pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <f10g7k$fag$1@sea.gmane.org>

Bill Janssen a ?crit :
>> Jeffrey, is there any way you can drop the top of the tree and going
>> straight from Number to Complex -> Real -> Rational -> Integer? These
>> are the things that everyone with high school math will know.
> 
> I think knowledge of the concepts of group, ring, and field is
> supposed to be standard knowledge for any high-school senior -- isn't
> this what the "new math" was all about?.  But they're pretty
> fundamental to computer science; anyone trying to do serious work in
> the field (that is, anyone with a reason to read the PEP) should have
> a nodding acquaintance with them.
> 
> Bill
> 
Even knowing the maths, you still have 2 problems:

1) the number types in Python are pretty much defined by the way CPUs work, not
by the math (Decimal beeing the exception), which causes some conceptual
impedance mismatch. Discussing to what extend floating point numbers can be
compared to theoritical real numbers is a can of worms we probably don't want to
open :-)

2) In the PEP, the concepts are used *inconsistently*. Complex derives from Ring
because the set of complex numbers *is a* ring. Int derives from Complex because
integer are complex numbers (or, alternatively, the set of integers *is included
in* the set of complex numbers). The consistent way could be to make the Complex
class an instance of Ring, not a subclass.

Cheers,
BC


From jyasskin at gmail.com  Sun Apr 29 00:19:47 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sat, 28 Apr 2007 15:19:47 -0700
Subject: [Python-3000] [Numpy-discussion] PEP 31XX: A Type Hierarchy for
	Numbers (and other algebraic entities)
In-Reply-To: <Mahogany-0.67.0-2680-20070427-131031.00@CASC64043305.american.edu>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<f0pucb$86m$1@sea.gmane.org> <873b2n5dc7.fsf@uwo.ca>
	<20070427144537.T1402@tribble.ilrt.bris.ac.uk>
	<ca471dc20704270714t42e6bb67pffe935854fcaa889@mail.gmail.com>
	<5d44f72f0704270758n27f3c20y8e2a33c24d8128dd@mail.gmail.com>
	<Mahogany-0.67.0-2680-20070427-131031.00@CASC64043305.american.edu>
Message-ID: <5d44f72f0704281519g316a12abl87e53d42b935deb6@mail.gmail.com>

On 4/27/07, Alan Isaac <aisaac at american.edu> wrote:
> On Fri, 27 Apr 2007, Jeffrey Yasskin wrote:
> > Then again, doubles aren't a group either because of this
> > imprecision, and I'm suggesting claiming they're
> > a subclass of that, so maybe there's room in a practical
> > language to make them a subclass of the rationals too.
>
> Would using language from the Scheme report be useful when
> discussing this?
> http://www-swiss.ai.mit.edu/projects/scheme/documentation/scheme_5.html

Very much so. Thanks for sending the link. I'm not going to get a
chance to update the document for the next several days, so if you
want to put together a patch using such language, I'd be happy to see
it go in. Or I'll integrate those ideas once I have a bit more spare
time.

It looks like the primary ideas there are that it was worthwhile for
Scheme to have the full Number:>Complex:>Real:>Rational:>Integer
tower, and that it wasn't worthwhile to duplicate it entirely for the
exact/inexact distinction.

Thanks,
Jeffrey

From rhamph at gmail.com  Sun Apr 29 02:06:42 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 28 Apr 2007 18:06:42 -0600
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <5d44f72f0704281455o7c3c2217pa3d599993f3f8018@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<fb6fbf560704251423qaac984cv6d81821f660b8940@mail.gmail.com>
	<5d44f72f0704281455o7c3c2217pa3d599993f3f8018@mail.gmail.com>
Message-ID: <aac2c7cb0704281706q586511e4y7066b7f96d403140@mail.gmail.com>

On 4/28/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On 4/25/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/25/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> > >     class MonoidUnderPlus(Abstract):
> >
> > Is this useful?  Just because two things are both Monoid instances
> > doesn't mean I can add them -- they have to be part of the same
> > Monoid.  By the time you do
> >
> >     assert isinstance(a, MonoidUnderPlus)
> >     assert isinstance(b, MonoidUnderPlus)
> >     assert isinstance(a, b.__class__) or isinstance(b, a.__class__)
> >
> > I'm not sure how much you've really saved.
>
> I brought this up on the PEP 3119 thread, as an issue for
> TotallyOrdered too. The exact syntax for checking that two objects are
> "in the same instance of ABC X" should be decided there.
>
> As to the benefits of MonoidUnderPlus over just hasattr(a,
> '__plus__'), by knowing that the operation is associative, a function
> may be able to speed itself up. A somewhat academic example is a
> SortedList type:
>   class SortedList(MonoidUnderPlus):
>     def __init__(self, iterable): self.contents = sorted(iterable)
>     def __add__(lhs, rhs): return merge(lhs, rhs)
> Then (ignoring that sum currently requires numbers) we can define mysorted() as:
>   def mysorted(iterable): return sum(map(SortedList, iterable)).contents
> With sum() implemented as a simple for loop, this is insertion sort,
> taking O(n^2) time. However, by noticing that + is associative on its
> argument, sum can be defined (assume the annotation implies the
> appropriate checking) roughly as:
>   def sum(lst : SequenceOfMonoidUnderPlus):
>     if len(lst) == 1: return lst[0]
>     else: return sum(lst[:len(lst)//2]) + sum(lst[len(lst)//2+1:]
> and we've transformed the algorithm into merge sort, taking O(n log n)
> time. (Hm, the zero element isn't as useful here as in other languages
> since you can't get a type without an instance. Maybe it should go
> leave the interface.)
>
> This optimization is, of course, incorrect if + is not associative.

Unfortunately, such a dramatic performance difference will mean people
will depend on it, then be surprised when it occasionally fails.
Since the original O(n**2) complexity is now perceived as a failure
mode, it's best to split it into an explicit method.

Python does have many optimizations, but they're well hidden, and
rarely if ever result in a noticeable complexity difference.

-- 
Adam Olsen, aka Rhamphoryncus

From jyasskin at gmail.com  Sun Apr 29 03:38:41 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sat, 28 Apr 2007 18:38:41 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <f0pucb$86m$1@sea.gmane.org>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<f0pucb$86m$1@sea.gmane.org>
Message-ID: <5d44f72f0704281838j4e8244c4g152df6e3762aa6d4@mail.gmail.com>

Thanks for the comments!

On 4/26/07, Travis E. Oliphant <oliphant.travis at ieee.org> wrote:
>
> Forgive my ignorance, but I'm not really sure what this PEP is trying to
> do.  I don't want to sound negative, I really just don't understand the
> purpose.  I've just never encountered a problem this that I can see how
> this PEP will help me solve.

It's trying to do the same thing for numbers as Guido is doing for
collections in PEP 3119: let people make the useful distinctions
between them for whatever purposes people need to make distinctions.
It looks like the community is a lot more comfortable with overloading
than static typing, so some of the distinctions I proposed are
probably too fine.

> That doesn't mean it's not worthwhile, I just don't understand it.
>
> >
> > Abstract
> > ========
> >
> > This proposal defines a hierarchy of Abstract Base Classes (ABCs)
> > [#pep3119] to represent numbers and other algebraic entities similar
> > to numbers.  It proposes:
> >
> > * A hierarchy of algebraic concepts, including monoids, groups, rings,
> >   and fields with successively more operators and constraints on their
> >   operators. This will be added as a new library module named
> >   "algebra".
> >
>
> The SAGE people may be interested in this, but I doubt there will more
> than a handful of users of these algebraic base classes.

I expected that if someone overloads a function between integers and
rationals, the numeric people would be happier if the integer overload
triggered for square matrices as well, in which case we'd want to
encourage people to overload on Ring vs Field instead of Integral vs
FractionalReal. You'd know better whether that's actually true.
If not, I guess I'd better stop claiming that they'll be useful for
the numpy people. :)

> > * A hierarchy of specifically numeric types, which can be converted to
> >   and from the native Python types. This will be added as a new
> >   library module named "numbers".
> >
>
> I could see having a hierarchy of "numbers"  we have one in NumPy.  All
> the NumPy array scalars fall under a hierarchy of types so that it is
> easy to check whether or not you have signed or unsigned integers, or
> inexact numbers.
>
> > Object oriented systems have a general problem in constraining
> > functions that take two arguments. To take addition as an example,
> > ``int(3) + int(4)`` is defined, and ``vector(1,2,3) + vector(3,4,5)``
> > is defined, but ``int(3) + vector(3,4,5)`` doesn't make much sense.
>
> In NumPy, this kind of operation makes sense we just broadcast int(3) to
> the equivalent vector(3,3,3) and do the element-by-element operation.
>
> This is how most array packages outside of Python do it as well.

Oops; I found a bad example. Does vector(3,3,3) + matrix([4,5],[6,7])
work? Are there other good examples of the addition having to be from
the "same" group?

> I think adding the abstract base-classes for "algebras" is more
> complicated than has been discussed if you are going to start treating
> arrays as elements in their own algebra.  There is a lot of flexibility
> in how arrays can be viewed in an algebraic context.

Really? Since the operators work element-wise, I would think that a
numpy array would have exactly the same algebraic structure as its
elements. One could wrap an array to put it into a particular
algebraic context, but the array itself seems pretty simple.

> The author gives a valiant go at defining base classes for generic
> algebras, but I didn't see everything I've encountered in my travels
> through algebra, and I don't understand the motivation for trying to
> include these things in Python specifically.
>
> Therefore, I would stick with defining a hierarchy of numbers and leave
> it at that for now (if I even went that far).
>
> For numbers, several suggestions have been offered.  In NumPy we use
> abstract base classes that look like this
>
> Number
>     Integer
>        SignedInteger
>        UnsignedInteger
>     Inexact
>        Floating
>        ComplexFloating
>
> But, this is because the array scalars in NumPy are based on
> C-data-types and not on general-purpose integers or floats.  In NumPy it
> is useful to check if you have an integer-typed array and so the
> base-classes become useful.

Do you have an example of when you'd want to check for an integer-typed array?

At one point, Guido was asking for reasons to include Cardinal (aka
Natural, aka Unsigned) in the hierarchy. Is UnsignedInteger only
present in NumPy because it describes some native C types, or does it
have some other purpose?

> For general purpose Python, I would do something like
>
> Complex
>    Rational_Complex
>      Integer_Complex
>    Floating_Complex  # uses hardware float
>    Decimal_Complex
>    Real
>      Rational
>        Integer
>      Floating        # uses hardware float
>      Decimal

Are you specifying an abstract Floating type to incorporate both
floats and doubles? Glancing over the Decimal spec, and trying a few
things with the implementation, it looks like Decimal objects have a
specific precision for each thread, which can be changed for future
operations at any time. Is there still a reason to separate Floating
and Decimal, or will a modification of FloatingReal suffice?

Thanks again,
Jeffrey

From greg.ewing at canterbury.ac.nz  Sun Apr 29 03:47:04 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 29 Apr 2007 13:47:04 +1200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <4632DD83.2030301@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de> <f0u4nh$nqt$1@sea.gmane.org>
	<4632DD83.2030301@v.loewis.de>
Message-ID: <4633F918.1010205@canterbury.ac.nz>

Martin v. L?wis wrote:
> I expect access to ob_type is frequent in Foo_Check() macros;
> those need to be rewritten to use Py_Type(op).

Maybe some of this breakage could be repaired by renaming
the real ob_type field and doing something like

   #define ob_type ob_base._ob_type

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Apr 29 03:57:49 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 29 Apr 2007 13:57:49 +1200
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
 Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.com>
References: <ca471dc20704280754m36454fb0ycbccc78b50b24ec@mail.gmail.com>
	<20070428161204.19381.458566240.divmod.quotient.6050@ohm>
	<ca471dc20704280917o5490f92di5b5272adff103fb8@mail.gmail.com>
Message-ID: <4633FB9D.9090705@canterbury.ac.nz>

Guido van Rossum wrote:
> On 4/28/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> 
>>    >>> class X(object):
>>    ...     __class__ = property(lambda self: int)
> 
> I'm not particularly enamored with it, but I believe it once served a
> purpose for Zope. Does anyone know if it is still needed?

I thought you wanted this so that proxy objects can
pretend to be instances of the class they're proxying.

--
Greg


From martin at v.loewis.de  Sun Apr 29 12:05:13 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Apr 2007 12:05:13 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <4633F918.1010205@canterbury.ac.nz>
References: <46327CC9.4040901@v.loewis.de>
	<f0u4nh$nqt$1@sea.gmane.org>	<4632DD83.2030301@v.loewis.de>
	<4633F918.1010205@canterbury.ac.nz>
Message-ID: <46346DD9.1020101@v.loewis.de>

Greg Ewing schrieb:
> Martin v. L?wis wrote:
>> I expect access to ob_type is frequent in Foo_Check() macros;
>> those need to be rewritten to use Py_Type(op).
> 
> Maybe some of this breakage could be repaired by renaming
> the real ob_type field and doing something like
> 
>    #define ob_type ob_base._ob_type

That doesn't work. For variable-sized objects, it would need
to be ob_base.ob_base.ob_type.

Regards,
Martin

From martin at v.loewis.de  Sun Apr 29 12:10:02 2007
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 29 Apr 2007 12:10:02 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <f1028g$3o0$1@sea.gmane.org>
References: <46327CC9.4040901@v.loewis.de> <f1028g$3o0$1@sea.gmane.org>
Message-ID: <46346EFA.1000408@v.loewis.de>

>> I propose the PEP below for Py3k.
> 
> I think it's a good idea.  It would be nice if the PEP included
> something about forwards compatibility.  Ideally, there should be a
> way to write a module that works with both 2.x and 3.x (e.g. using
> preprocessor macros).  Obviously the 3.x design should not be
> compromised for this sake though.

Daniel Stutzbach proposed (in private mail) to preserve PyObject_HEAD,
but define it as "PyObject ob_base;". Technically, that would work just
fine, and I'll revise the PEP in that direction.

I'd still prefer if the core would not use PyObject_HEAD on its own,
because the semicolon-missing macro breaks editors that do indentation
based on the C syntax, but providing this level of compatibility is
easy.

Regards,
Martin

From martin at v.loewis.de  Sun Apr 29 15:47:27 2007
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 29 Apr 2007 15:47:27 +0200
Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to
 standard C (Was: Dropping PyObject_HEAD)
In-Reply-To: <bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>
References: <46327CC9.4040901@v.loewis.de>
	<bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>
Message-ID: <4634A1EF.6080204@v.loewis.de>

> An example of how this will change current code would be good.  E.g.,
> ``o->ob_type->tp_name`` becomes ``PyType(o)->typ_name`` or
> ``o->ob_base->ob_type->tp_name``.
> 
> Otherwise I am all for cleaning up the codebase and thus support this PEP.

Thanks for the suggestions, I have add them all into PEP 3123.

I also changed the PEP title, to reflect that PyObject_HEAD is
maintained for compatibility.

Regards,
Martin

From martin at v.loewis.de  Sun Apr 29 15:48:33 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Apr 2007 15:48:33 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <eae285400704280657t5a0a56b7q199394c797f8c02a@mail.gmail.com>
References: <46327CC9.4040901@v.loewis.de>
	<eae285400704280657t5a0a56b7q199394c797f8c02a@mail.gmail.com>
Message-ID: <4634A231.8010700@v.loewis.de>

> - add the new access macros to 2.6 (defined appropriately)
> - in 3.0, instead of removing PyObject_HEAD and company, redefine them
> appropriately, .e.g, #define PyObject_HEAD PyObject obtype;

Thanks, I added both of them into PEP 3123 (now retitled to
indicate that PyObject_HEAD is not dropped).

Regards,
Martin


From martin at v.loewis.de  Sun Apr 29 16:17:01 2007
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 29 Apr 2007 16:17:01 +0200
Subject: [Python-3000] PEP 3121: Module Initialization and finalization
In-Reply-To: <bbaeab100704271123jd423cd5q4f377515ea91be06@mail.gmail.com>
References: <4631B570.7030307@v.loewis.de>
	<bbaeab100704271123jd423cd5q4f377515ea91be06@mail.gmail.com>
Message-ID: <4634A8DD.7030109@v.loewis.de>

Brett,

Thanks for the suggestions; I added them to the PEP.

> How would I raise xx.error in C code now?  I am guessing like this::

Interestingly enough, xxmodule doesn't raise error.

>  PyObject* module = PyState_FindModule(&xxmodule);

Either that, or just use the self parameter if you are in a
module-level function

>  if (!module)
>    return NULL;
>  PyObject* xx_error = xxstate(module)->ErrorObject;
>  if (!xx_error) {
>    PyErr_SetString(PyExc_SystemError, "xx.error missing");
>    return NULL;
>  }

That should not fail. If the module object is still present,
the state shouldn't be cleared normally. So this might
only occur if the code above is invoked in the "clear cycles"
phase of garbage collection.

> Since most objects will move to being memory-managed, one needs to
> worry about checking that the object still exists.  I assume I didn't
> go overboard with the error checking here, right? 

It would be good if the check for the NULL exception object could
be folded in PyErr_SetString (i.e. that raising an internal error
if the exception class is NULL).

> I guess people are
> going to end up writing helper functions to access the various data
> fields as the above would get rather tedious if you had to write it
> more than twice.

Over time, we can try to find convenience macro-functions, e.g.

  PyModule_ObjectField(xxmodule, ErrorObject)

which would return a borrowed reference of type PyObject*, or
NULL (with an exception set) if there is some kind of error.

>> Tim Peters reports in [1]_ that PythonLabs considered such a feature
>> at one point, and lists the following additional hooks which aren't
>> currently supported in this PEP:
>>
>>  * when the module object is deleted from sys.modules
>>
>>  * when Py_Finalize is called
>>
>>  * when Python exits
>>
> 
> Wouldn't the above be covered by the deallocation of the module?

No: deletion from sys.modules might not cause module deallocation,
if there are other references to the module (although Guido might
have meant this mean "module deallocation").

As for finalization/exiting: normally, all modules should be
released before Python exits. Again, if there is some garbage
reference to the module, it may not have been deallocated;
the question then is whether it should get a further opportunity
for cleanup.

Regards,
Martin


From jimjjewett at gmail.com  Sun Apr 29 17:40:52 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 11:40:52 -0400
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
	<76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com>
Message-ID: <fb6fbf560704290840i3b758489q6eb6a3fba3931d8d@mail.gmail.com>

On 4/28/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> I'm just going to jump into this and voice a concern that allowing
> overriding of isinstance and issubclass seems like a Bad Idea.

I'm also uncomfortable, which is why I originally suggested a parallel
predicate.  That said, I can't put my finger on why it is wrong, so
I'm not certain that a parallel predicate (isa?) would be better.

Guido wrote:
> Note though that only the second argument to either function can
> overload the rules. IOW if you write isinstance(x, C), there is no way
> that x could attempt to lie; but C could.

As Barry pointed out, this means a class can't easily say "my parent
may well implement ABC, but I don't."  That special case could be
added to the default overriding behavior, but ... it starts to get
fragile.

-jJ

From brett at python.org  Sun Apr 29 19:06:37 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 29 Apr 2007 10:06:37 -0700
Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to
	standard C (Was: Dropping PyObject_HEAD)
In-Reply-To: <4634A1EF.6080204@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de>
	<bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>
	<4634A1EF.6080204@v.loewis.de>
Message-ID: <bbaeab100704291006s245f154bv7e29f163076f9971@mail.gmail.com>

On 4/29/07, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > An example of how this will change current code would be good.  E.g.,
> > ``o->ob_type->tp_name`` becomes ``PyType(o)->typ_name`` or
> > ``o->ob_base->ob_type->tp_name``.
> >
> > Otherwise I am all for cleaning up the codebase and thus support this PEP.
>
> Thanks for the suggestions, I have add them all into PEP 3123.
>
> I also changed the PEP title, to reflect that PyObject_HEAD is
> maintained for compatibility.
>

Are we going to have a preference over PyObject_HEAD or defining the
struct field directly?

-Brett

From brett at python.org  Sun Apr 29 19:12:20 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 29 Apr 2007 10:12:20 -0700
Subject: [Python-3000] PEP 3121: Module Initialization and finalization
In-Reply-To: <4634A8DD.7030109@v.loewis.de>
References: <4631B570.7030307@v.loewis.de>
	<bbaeab100704271123jd423cd5q4f377515ea91be06@mail.gmail.com>
	<4634A8DD.7030109@v.loewis.de>
Message-ID: <bbaeab100704291012n3c5c7a3exb607e5bd0c72ae10@mail.gmail.com>

On 4/29/07, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Brett,
>
> Thanks for the suggestions; I added them to the PEP.
>
> > How would I raise xx.error in C code now?  I am guessing like this::
>
> Interestingly enough, xxmodule doesn't raise error.
>
> >  PyObject* module = PyState_FindModule(&xxmodule);
>
> Either that, or just use the self parameter if you are in a
> module-level function
>

OK.

> >  if (!module)
> >    return NULL;
> >  PyObject* xx_error = xxstate(module)->ErrorObject;
> >  if (!xx_error) {
> >    PyErr_SetString(PyExc_SystemError, "xx.error missing");
> >    return NULL;
> >  }
>
> That should not fail. If the module object is still present,
> the state shouldn't be cleared normally. So this might
> only occur if the code above is invoked in the "clear cycles"
> phase of garbage collection.
>

Maybe it shouldn't, but can it?  Is there something preventing the
exception object from being cleared beyond people not being stupid?

> > Since most objects will move to being memory-managed, one needs to
> > worry about checking that the object still exists.  I assume I didn't
> > go overboard with the error checking here, right?
>
> It would be good if the check for the NULL exception object could
> be folded in PyErr_SetString (i.e. that raising an internal error
> if the exception class is NULL).
>

Sounds reasonable.

> > I guess people are
> > going to end up writing helper functions to access the various data
> > fields as the above would get rather tedious if you had to write it
> > more than twice.
>
> Over time, we can try to find convenience macro-functions, e.g.
>
>   PyModule_ObjectField(xxmodule, ErrorObject)
>
> which would return a borrowed reference of type PyObject*, or
> NULL (with an exception set) if there is some kind of error.
>

I am sure some will pop up.

> >> Tim Peters reports in [1]_ that PythonLabs considered such a feature
> >> at one point, and lists the following additional hooks which aren't
> >> currently supported in this PEP:
> >>
> >>  * when the module object is deleted from sys.modules
> >>
> >>  * when Py_Finalize is called
> >>
> >>  * when Python exits
> >>
> >
> > Wouldn't the above be covered by the deallocation of the module?
>
> No: deletion from sys.modules might not cause module deallocation,
> if there are other references to the module (although Guido might
> have meant this mean "module deallocation").
>

You're right.  I was not thinking big enough to consider multiple
imports of the same module.

> As for finalization/exiting: normally, all modules should be
> released before Python exits. Again, if there is some garbage
> reference to the module, it may not have been deallocated;
> the question then is whether it should get a further opportunity
> for cleanup.

Eh.  I say no for now and if people really need it we can add the hook later.

-Brett

From jyasskin at gmail.com  Sun Apr 29 19:25:35 2007
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sun, 29 Apr 2007 10:25:35 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <f10g7k$fag$1@sea.gmane.org>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
Message-ID: <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>

On 4/28/07, Baptiste Carvello <baptiste13 at altern.org> wrote:
> 2) In the PEP, the concepts are used *inconsistently*. Complex derives from Ring
> because the set of complex numbers *is a* ring. Int derives from Complex because
> integer are complex numbers (or, alternatively, the set of integers *is included
> in* the set of complex numbers). The consistent way could be to make the Complex
> class an instance of Ring, not a subclass.

Good point. In this structure, isinstance(3, Ring) really means that 3
is a member of some (unspecified) ring, not that 3 isa Ring, but the
ambiguity there is probably the root cause of the problem with
mixed-mode operations. We should also say that isinstance(3, Complex)
means that 3 is a member of some subring of the complex numbers, which
preserves the claim that Complex is a subtype of Ring.

Up to here, things make sense, but because of how ABCs work, we need
issubclass(rational, Complex). I suppose that's true too, since
isinstance(3.4, rational) means "3.4 is a member of the rational
subring of the complex numbers", which implies that "3.4 is a member
of some subring of the complex numbers."

There may be better names for these concepts. Perhaps suffixing every
numeric ABC with "Element"? Do you have suggestions?


Jason Orendorff points out that Haskell typeclasses capture the fact
that complex is an instance of Ring. I think imitating them as much as
possible would indeed imply making the numeric ABCs into metaclasses
(in Haskell terminology, "kinds"). To tell if the arguments to a
function were in the same total order, you'd check if they had any
common superclasses that were themselves instances of TotallyOrdered.
I don't know enough about how metaclasses are typically used to know
how that would conflict.

-- 
Namast?,
Jeffrey Yasskin

From martin at v.loewis.de  Sun Apr 29 19:57:45 2007
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 29 Apr 2007 19:57:45 +0200
Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to
 standard C (Was: Dropping PyObject_HEAD)
In-Reply-To: <bbaeab100704291006s245f154bv7e29f163076f9971@mail.gmail.com>
References: <46327CC9.4040901@v.loewis.de>	
	<bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>	
	<4634A1EF.6080204@v.loewis.de>
	<bbaeab100704291006s245f154bv7e29f163076f9971@mail.gmail.com>
Message-ID: <4634DC99.2090307@v.loewis.de>

> Are we going to have a preference over PyObject_HEAD or defining the
> struct field directly?

That is an open issue. PyObject_HEAD clearly is more
backwards-compatible. However (besides its current non-C-conformance),
I never liked it, because of the missing semicolon (I thought it
confuses Emacs, but can't reproduce the problem anymore, though -
perhaps it did confuse Emacs in the past?)

So I personally would phase it out for style reasons, but others
may think differently; if I hear opinions in either direction,
I'm happy to code them into the PEP.

Regards,
Martin

From martin at v.loewis.de  Sun Apr 29 20:15:05 2007
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 29 Apr 2007 20:15:05 +0200
Subject: [Python-3000] PEP 3121: Module Initialization and finalization
In-Reply-To: <bbaeab100704291012n3c5c7a3exb607e5bd0c72ae10@mail.gmail.com>
References: <4631B570.7030307@v.loewis.de>	
	<bbaeab100704271123jd423cd5q4f377515ea91be06@mail.gmail.com>	
	<4634A8DD.7030109@v.loewis.de>
	<bbaeab100704291012n3c5c7a3exb607e5bd0c72ae10@mail.gmail.com>
Message-ID: <4634E0A9.2020900@v.loewis.de>

>> >  if (!module)
>> >    return NULL;
>> >  PyObject* xx_error = xxstate(module)->ErrorObject;
>> >  if (!xx_error) {
>> >    PyErr_SetString(PyExc_SystemError, "xx.error missing");
>> >    return NULL;
>> >  }
>>
>> That should not fail. If the module object is still present,
>> the state shouldn't be cleared normally. So this might
>> only occur if the code above is invoked in the "clear cycles"
>> phase of garbage collection.
>>
> 
> Maybe it shouldn't, but can it?  Is there something preventing the
> exception object from being cleared beyond people not being stupid?

I believe it would always be possible to trigger a problem, no matter
what we do: if you key invocation of the function to deletion of
the exception class (which is always possible through weak references),
then you can trigger a problem indeed.

I also now think there can't be a solution: E.g. you may associate
reference the exception from the method object itself, so that you
the exception "should" be alive while the function is still available.
However, breaking cycles may decide to tp_clear that reference, making
that approach futile.

In some cases, it may be possible to statically determine that clearing
certain reference will never be necessary to break all cycles (because
every such cycle would be broken at a different reference also),
however, that is difficult to generalize, and prone to errors that
result in uncollectable garbage.

Regards,
Martin

From rasky at develer.com  Sun Apr 29 21:16:52 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 29 Apr 2007 21:16:52 +0200
Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to
 standard C (Was: Dropping PyObject_HEAD)
In-Reply-To: <4634DC99.2090307@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de>		<bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>		<4634A1EF.6080204@v.loewis.de>	<bbaeab100704291006s245f154bv7e29f163076f9971@mail.gmail.com>
	<4634DC99.2090307@v.loewis.de>
Message-ID: <f12qv4$ou6$1@sea.gmane.org>

On 29/04/2007 19.57, Martin v. L?wis wrote:

> That is an open issue. PyObject_HEAD clearly is more
> backwards-compatible. However (besides its current non-C-conformance),
> I never liked it, because of the missing semicolon (I thought it
> confuses Emacs, but can't reproduce the problem anymore, though -
> perhaps it did confuse Emacs in the past?)
> 
> So I personally would phase it out for style reasons, but others
> may think differently; if I hear opinions in either direction,
> I'm happy to code them into the PEP.

The fact that it existed made it possible to do this transition though. 
Haven't we had this macro in the first place, now you would need to break all 
third party modules without an easy fallback.

So, I'm probably +0 on keeping it since history tends to repeat itself :)
-- 
Giovanni Bajo



From brett at python.org  Sun Apr 29 21:56:58 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 29 Apr 2007 12:56:58 -0700
Subject: [Python-3000] PEP 3123: Making PyObject_HEAD conform to
	standard C (Was: Dropping PyObject_HEAD)
In-Reply-To: <4634DC99.2090307@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de>
	<bbaeab100704271630y877ad3frf02c2db5abf075c2@mail.gmail.com>
	<4634A1EF.6080204@v.loewis.de>
	<bbaeab100704291006s245f154bv7e29f163076f9971@mail.gmail.com>
	<4634DC99.2090307@v.loewis.de>
Message-ID: <bbaeab100704291256u5211bd3eu56156e9d4a17c03d@mail.gmail.com>

On 4/29/07, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > Are we going to have a preference over PyObject_HEAD or defining the
> > struct field directly?
>
> That is an open issue. PyObject_HEAD clearly is more
> backwards-compatible. However (besides its current non-C-conformance),
> I never liked it, because of the missing semicolon (I thought it
> confuses Emacs, but can't reproduce the problem anymore, though -
> perhaps it did confuse Emacs in the past?)
>
> So I personally would phase it out for style reasons, but others
> may think differently; if I hear opinions in either direction,
> I'm happy to code them into the PEP.
>

I say phase out the macro.

-Brett

From brett at python.org  Sun Apr 29 22:04:09 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 29 Apr 2007 13:04:09 -0700
Subject: [Python-3000] Suggested PEP 3108 addition
In-Reply-To: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com>
References: <43aa6ff70704251005i4912e6d6s7efc6346483a142b@mail.gmail.com>
Message-ID: <bbaeab100704291304l4390a237j36dea3a3c9eda328@mail.gmail.com>

Done.

On 4/25/07, Collin Winter <collinw at gmail.com> wrote:
> I'd like to propose that the bsddb185 module be added to PEP 3108's
> list of modules slated for removal. Reasons to kill it:
>
> - The module isn't built by default; manual tweaking of setup.py is required.
>
> - The following scare-paragraph is included at the bottom of
> http://docs.python.org/lib/module-bsddb.html:
>
> """
> Beginning in 2.3 some Unix versions of Python may have a bsddb185
> module. This is present only to allow backwards compatibility with
> systems which ship with the old Berkeley DB 1.85 database library. The
> bsddb185 module should never be used directly in new code.
> """
>
> Collin Winter
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/brett%40python.org
>

From tcdelaney at optusnet.com.au  Sun Apr 29 22:44:45 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Mon, 30 Apr 2007 06:44:45 +1000
Subject: [Python-3000] Pre-pre PEP for 'super' keyword
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com><f0hkrs$aro$1@sea.gmane.org><9e804ac0704230607q7a50bb78hb7242ab6a09d0853@mail.gmail.com><f0j77k$nj5$1@sea.gmane.org><fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com><76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com><d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com><76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com><d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com><d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com><ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
Message-ID: <00b601c78a9f$38ec9390$0201a8c0@ryoko>

I've been intending to write up a PEP for fixing super, but I haven't had 
time to get to it. I've worked out the semantics I'm going to propose 
though. I've attached an implementation of those semantics as far as I've 
been able to get them so far.

I think this could be targeted at 2.6, with a __future__ import.

Basically, the proposals are:

1. 'super' becomes a keyword, that returns a super object for the instance 
method currently being executed.

2. 'super' cannot be rebound, but the object produced can be assigned to 
other names i.e.

    super = None # SyntaxError, or UnrebindableError (see #7 below)
    name = super # OK

This allows passing the super object to other functions.

3. 'super' is a cell variable i.e. inner functions can use it, and will get 
the super object of the outer method.

    class A(object):
        def f(self):
            def inner():
                print super
            inner()

4. super objects are callable, and calling them will execute the super 
method with the same name as the instance method currently being executed. 
Lookup of this method occurs when the instance method is entered.

    class A(object):
        def f(self):
            pass

    class B(A):
        def f(self):
            super() # Calls A.f(self)

If you want name lookup to occur at the time of the call, you can explicitly 
specify the method name (just like with any other super attribute):

    class A(object):
        def f(self):
            pass

    class B(A):
        def f(self):
            super.f() # Calls A.f(self)

5. The repr of a super object will include the current class, the instance 
type and the bound method that will be used when called (if present) or an 
AttributeError:

<super: <class 'E'>, <F object>, <bound method B.f of <__main__.F object at 
0x00B88790>>>
<super: <class 'A'>, <F object>, <AttributeError: 'super' object has no 
attribute 'f'>>

6. Methods will have an attribute holding the class where the unbound method 
was created. This attribute is available on both unbound and bound methods. 
In the attached code I've called it func_class, but it really should have an 
im_prefix (can't be im_class though). This is used to make the repr of a 
bound method actually reflect which class it comes from.

7. Code objects gain a tuple of unrebindable names, which can be local 
variables, cell variables, etc. For instance methods and class methods, this 
would initially contain two names - 'super' and the name of the first 
parameter to the method (normally 'self' or 'cls). Attempting to rebind 
these names would throw an UnrebindableError (maybe a subclass of 
SyntaxError). It would be good if these could actually be detected at 
compile time.

Differences in the attached code are:

1. You need to inherit from 'autosuper' (or use the _autosuper metaclass).

2. 'super' is not a keyword.

3. Rebinding 'super' will not throw an exception, but such rebinding will be 
ignored. Also see the next point.

4. It is *necessary* to rebind 'super' in the method in order to create a 
cell for use in inner classes. The recommended way in the attached code is:

    class A(autosuper):
        def f(self):
            super = super

            def a():
                print super

            a()

The 'super = super' line above would normally throw an UnboundLocalError, 
but the bytecode hacking I've done causes it to work.

Comments before I start writing up the PEP? Or if anyone has a PEP partially 
written, do you want to incorporate any of the above into it?

Tim Delaney 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: autosuper.py
Type: application/octet-stream
Size: 13032 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20070430/6e412e89/attachment-0001.obj 

From jimjjewett at gmail.com  Sun Apr 29 23:15:33 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 17:15:33 -0400
Subject: [Python-3000] New Super PEP - draft 2
Message-ID: <fb6fbf560704291415g7bb97970o47576e82cb822ddf@mail.gmail.com>

(Adding python-3000 to the Cc, since that is where  much of the recent
discussion occurred)

Calvin's full proposal is archived at:
http://mail.python.org/pipermail/python-dev/2007-April/072835.html

> Abstract
> ========

> The PEP defines the proposal to enhance the super builtin to work implicitly
> upon the class within which it is used and upon the instance the current
> function was called on. The premise of the new super usage suggested is as
> follows:

>     super.foo(1, 2)

> to replace the old:

>     super(Foo, self).foo(1, 2)

The alternatives section needs to explain why "super.foo(*args)" is
the right level of simplification.  Alternatives include

(1)    No changes

Con:  fragility in name binding

(2)    super(__this_class__, self).foo(*args)

Con:  The "(__this_class__, self)" is boilerplate.

Note that the __this_class__ PEP should be referenced, if only as a
possible explanation for what is happening under the covers.

(3)    self.__super__.foo(*args)   # or super.foo(*args)

Con:  Shouldn't need a __double_underscore_name__ in normal functions.
Con:  __super__ (even if renamed super) is not a simple attribute; it
is a property representing a partially applied function (that gets
further applied by the "self")

(4)  super(self, *args)    # ?  or __super__(self, *args)

Actually, I sort of like this one, as it allows the upcall signature
to exactly match the method definition signature.

Con:  super is still not a simple attribute.
Con:  changing the method name becomes a hassle.

(5)  super.foo(self, *args)    # ?  or __super__.foo(self, *args)

Con:  self isn't really an argument just to super.foo -- it is an
argument to super which is used to find/instantiate foo in the first
place.  (Plus the objections to partially applied function
attributes.)

(6)  super     # ? or super()

Pro:  matches java; doesn't require a repeat of *args
Con:  doesn't match anything else in python, needs to be a keyword
(and would still need object support).

In the first example:

        class A(object):
            def f(self):
                return 'A'

        class B(A):
            def f(self):
                return 'B' + super.f()

        class C(A):
            def f(self):
                return 'C' + super.f()

        class D(B, C):
            def f(self):
                return 'D' + super.f()

        assert D().f() == 'DBCA'

You should probably include tests that fail with some of the more
obvious (but wrong) solutions, such as

        class A(object):
            def f(self):
                return 'A'

        class Bempty(A): # Make sure it doesn't call the A or B method twice
            pass

        class B(Bempty):
            def f(self):
                return 'B' + super.f()

        class C(A):
            def f(self):
                return 'C' + super.f()

        class D(B, C):
            def f(self):
                return 'D' + super.f()

        assert D().f() == 'DBCA'

        class E(C, B):    # Show that B can point to C as next or vice versa
            def f(self):
                return 'E' + super.f()

        assert E().f() == 'ECBA'

        class F(D):    # show that the instance may not be a direct instance
            pass

        assert D().f() == 'DBCA'


> The enhancements to the super type will define a new __getattr__ classmethod
> of the super type,

Did you really mean, it gets the class of builtin.super, but no info
on which class is using super?  If not, it isn't a classmethod of the
super type.  It may be a classmethod of the
currently-being-defined-type (__this_class__).

> which must look backwards to the previous frame and locate
> the instance object.

frame manipulation is fragile.  Even if it didn't cause problems for
other implementations, it causes problems for nested functions and
callbacks.

    def f(self, button, *args):
        def callback(*args):
            super(__this_class__, self).f(*args)
        button.callback=callback

When this gets called, the appropriate self won't be in the frame,
except as a lexically scoped variable (which might have a different
name).

Also, note that this fails on most of Thomas Wouters' advanced usages.
 While you say that super shouldn't be called outside a method, you
can't really keep the super object itself from getting returned.

> "Every class will gain a new special attribute, __super__, which refers to an
> instance of the associated super object for that class" In this capacity, the
> new super also acts as its own descriptor, create an instance-specific super
> upon lookup.

If you didn't say instance-specific, this would be equivalent to a
class decorator, translating

    class A(object): ...

into

    class A(object): ...

    A.__super__=super(A)

As is, it gets a bit trickier, since you need to I think the
translation is closer to

    class A(object):...
        @property
        def __super__(self):
            return __this_class__.__supermaker()

    A.__supermaker=super(A)

I don't see a good non-magical way to pass both the instance and the
class-at-time-of-writing (as opposed to class of calling instance),
which is why the other solutions used bytecode hacks or name mangling.

> Much of this was discussed in the thread of the python-dev list, "Fixing super
> anyone?" [1]_.

I assume you meant the python-3000 list.

-jJ

From jimjjewett at gmail.com  Sun Apr 29 23:25:53 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 17:25:53 -0400
Subject: [Python-3000] Pre-pre PEP for 'super' keyword
In-Reply-To: <00b601c78a9f$38ec9390$0201a8c0@ryoko>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<fb6fbf560704231642s20aa2622nf0222e5925aaa3b0@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
Message-ID: <fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>

On 4/29/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
> I've been intending to write up a PEP for fixing super, but I haven't had
> time to get to it.

Calvin Spealman has the most recent draft. I hope he will incorporate
this into his draft.

> 1. 'super' becomes a keyword, that returns a super object for the instance
> method currently being executed.

So it is a "keyword" in the sense that None is a keyword; not in the
stronger sense that "if" is a keyword?

> 4. super objects are callable, and calling them will execute the super
> method with the same name as the instance method currently being executed.
> Lookup of this method occurs when the instance method is entered.
>
>     class A(object):
>         def f(self):
>             pass
>
>     class B(A):
>         def f(self):
>             super() # Calls A.f(self)

> If you want name lookup to occur at the time of the call, you can explicitly
> specify the method name (just like with any other super attribute):
>
>     class A(object):
>         def f(self):
>             pass
>
>     class B(A):
>         def f(self):
>             super.f() # Calls A.f(self)

As long as you can be explicit, should the shortcut be a full
shortcut?  That is,

    def f(self, a, b=c, *args, **kwargs):
        super()    # passes the exact arglist that f got

vs

    def __init__(self, myvar, passed_var):
        super.__init__(self, passed_var)    # flags that you are
changing the args

-jJ

From ironfroggy at gmail.com  Mon Apr 30 00:23:52 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sun, 29 Apr 2007 18:23:52 -0400
Subject: [Python-3000] Pre-pre PEP for 'super' keyword
In-Reply-To: <fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
	<fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
Message-ID: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com>

On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/29/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
> > I've been intending to write up a PEP for fixing super, but I haven't had
> > time to get to it.
>
> Calvin Spealman has the most recent draft. I hope he will incorporate
> this into his draft.

I will incorporate this into my draft, particularly taking care of the
issue with inner functions.

> > 1. 'super' becomes a keyword, that returns a super object for the instance
> > method currently being executed.
>
> So it is a "keyword" in the sense that None is a keyword; not in the
> stronger sense that "if" is a keyword?

I would like to say super becomes a constant in the way that None is a
constant, and if there is some reason the implementation today or
tomorrow can benefit from actually making it a keyword, that won't
break anything if its already constant. One problem with an actual
keyword, is there is no other part of Python where an actual keyword
evaluates to something.

> > 4. super objects are callable, and calling them will execute the super
> > method with the same name as the instance method currently being executed.
> > Lookup of this method occurs when the instance method is entered.
> >
> >     class A(object):
> >         def f(self):
> >             pass
> >
> >     class B(A):
> >         def f(self):
> >             super() # Calls A.f(self)

This might run into the same issue I had to cover, where you get an
ambiguous situation trying to distinguish between calling super and
calling the __call__ method of the next class in the MRO.

We should absolutely avoid a situation in python now where X() differs
from X.__call__()

> > If you want name lookup to occur at the time of the call, you can explicitly
> > specify the method name (just like with any other super attribute):
> >
> >     class A(object):
> >         def f(self):
> >             pass
> >
> >     class B(A):
> >         def f(self):
> >             super.f() # Calls A.f(self)
>
> As long as you can be explicit, should the shortcut be a full
> shortcut?  That is,
>
>     def f(self, a, b=c, *args, **kwargs):
>         super()    # passes the exact arglist that f got

I sure wish my previous complaints didn't hinder this, because I
really love the idea of being able to this, which would really
encourage more compatible method signatures, so you can use the
shortcut! I'm desperate for a solution that satisfies all the sides of
the equation.

> vs
>
>     def __init__(self, myvar, passed_var):
>         super.__init__(self, passed_var)    # flags that you are
> changing the args
>
> -jJ
>


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From guido at python.org  Mon Apr 30 00:46:58 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 15:46:58 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
Message-ID: <ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>

On 4/29/07, Jeffrey Yasskin <jyasskin at gmail.com> wrote:
> On 4/28/07, Baptiste Carvello <baptiste13 at altern.org> wrote:
> > 2) In the PEP, the concepts are used *inconsistently*. Complex derives from Ring
> > because the set of complex numbers *is a* ring. Int derives from Complex because
> > integer are complex numbers (or, alternatively, the set of integers *is included
> > in* the set of complex numbers). The consistent way could be to make the Complex
> > class an instance of Ring, not a subclass.
>
> Good point. In this structure, isinstance(3, Ring) really means that 3
> is a member of some (unspecified) ring, not that 3 isa Ring,

To ask whether x is a Ring, you'd use issubclass(x, Ring). Now, in a
different context (still in Python) you might define a class Ring
whose *instances* are rings; but in the current draft of PEP 3141,
Ring is a class whose subclasses are rings.

[BTW "isa" is not an English word, nor used anywhere in Python. If
"isa" means "is a" you might as well write proper English; if it has
additional connotations, they're likely lost to this crowd so you
can't count on your readers knowing the distinction.]

>  but the ambiguity there is probably the root cause of the problem with
> mixed-mode operations.

(Where by mixed-mode operations you mean e.g. trying to add or
multiply members of two *different* rings, right?)

> We should also say that isinstance(3, Complex)
> means that 3 is a member of some subring of the complex numbers, which
> preserves the claim that Complex is a subtype of Ring.

Hm... it's beginning to look like binary operations in the
mathematical sense just don't have an exact equivalent in OO type
theory. (Or vice versa.) There really isn't a way to define an
operation binop(a: T, b: T) -> T in such a way that it is clear what
should happen if a and b are members of two different subtypes of T,
named T1 and T2. Classic OO seems to indicate that this must be
defined, and there are plenty of examples that seem to agree, e.g.
when T1 and T2 are trivial subtypes of T (maybe each adding a
different inessential method). OTOH we have the counter-example where
T==Ring and T1 and T2 are two different, unrelated Rings, and the
result may not be defined.

Hmm... Maybe the conclusion to draw from this is that we shouldn't
make Ring a class? Maybe it ought to be a metaclass, so we could ask
isinstance(Complex, Ring)?

Perhaps a similar line of reasoning migtht apply to PartiallyOrdered
and TotallyOrdered.

> Up to here, things make sense, but because of how ABCs work, we need
> issubclass(rational, Complex). I suppose that's true too, since
> isinstance(3.4, rational) means "3.4 is a member of the rational
> subring of the complex numbers", which implies that "3.4 is a member
> of some subring of the complex numbers."
>
> There may be better names for these concepts. Perhaps suffixing every
> numeric ABC with "Element"? Do you have suggestions?

Maybe we should stop trying to capture radically different
mathematical number systems using classes or types, and limit
ourselves to capturing the systems one learns in high school: C, R, Q,
Z, and (perhaps) N (really N0). The concrete types would be complex <:
C, float<:R, Decimal<:R, int<:Z. NumPy would have many more. One could
argue that float and Decimal are <:Q, but I'm not sure if that makes
things better pragmatically; I guess I'm coming from the old Algol
school where float was actually called real (and in retrospect I wish
I'd called it that in Python). I'd rather reserve membership of Q for
an infinite precision rational type (which many people have
independently implemented).

> Jason Orendorff points out that Haskell typeclasses capture the fact
> that complex is an instance of Ring. I think imitating them as much as
> possible would indeed imply making the numeric ABCs into metaclasses
> (in Haskell terminology, "kinds"). To tell if the arguments to a
> function were in the same total order, you'd check if they had any
> common superclasses that were themselves instances of TotallyOrdered.
> I don't know enough about how metaclasses are typically used to know
> how that would conflict.

The more I think about it, it sounds like the right thing to do. To
take PartiallyOrdered (let's say PO for brevity) as an example, the
Set class should specify PO as a metaclass. The PO metaclass could
require that the class implement __lt__ and __le__. If it found a
class that didn't implement them, it could make the class abstract by
adding the missing methods to its __abstractmethods__ attribute. Or,
if it found that the class implemented one but not the other, it could
inject a default implementation of the other in terms of the one and
__eq__.

This leaves us with the question of how to check whether an object is
partially orderable. Though that may really be the wrong question --
perhaps you should ask whether two objects are partially orderable
relative to each other. For that, you would first have to find the
most derived common base class (if that is even always a defined
operation(*)), and then check whether that class is an instance of PO.
It seems easier to just try the comparison -- duck typing isn't dead
yet! I don't think this is worth introducing a new inspection
primitive ('ismetainstance(x, PO)').

The PO class may still be useful for introspection: at the meta-level,
it may be useful occasionally to insist that or inquire whether a
given *class* is PO. (Or TO, or a Ring, etc.)

Now, you could argue that Complex should also be a metaclass. While
that may mathematically meaningful (for all I know there are people
doing complex number theory using Complex[Z/n]), for Python's numeric
classes I think it's better to make Complex a regular class
representing all the usual complex numbers (i.e. a pair of Real
numbers). I expect that the complex subclasses used in practice are
all happy under mixed arithmetic using the usual definition of mixed
arithmetic: convert both arguments to a common base class and compute
the operation in that domain.

(*) consider classes AB derived from (B, A) and BA derived from (A,
B). Would A or B be the most derived base class? Or would we have to
skip both and continue the search with A's and B's base classes?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Mon Apr 30 00:49:54 2007
From: collinw at gmail.com (Collin Winter)
Date: Sun, 29 Apr 2007 15:49:54 -0700
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
	<fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
	<76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com>
Message-ID: <43aa6ff70704291549v50a11ebah1a96cc213d2d5177@mail.gmail.com>

[cc:-python-dev]

On 4/29/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > As long as you can be explicit, should the shortcut be a full
> > shortcut?  That is,
> >
> >     def f(self, a, b=c, *args, **kwargs):
> >         super()    # passes the exact arglist that f got
>
> I sure wish my previous complaints didn't hinder this, because I
> really love the idea of being able to this, which would really
> encourage more compatible method signatures, so you can use the
> shortcut! I'm desperate for a solution that satisfies all the sides of
> the equation.

I hate this. super() calls would be completely different from other
function calls in that what appears to be an empty argument list is
actually a filled-out argument list.

Please stick to the original topic of figuring out how to remove the
class name from super calls, rather than inventing new magical,
spooky-action-at-a-distance toys.

Collin Winter

From guido at python.org  Mon Apr 30 01:01:47 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 16:01:47 -0700
Subject: [Python-3000] ABC PEP isinstance issue Was: PEP 31XX: A Type
	Hierarchy for Numbers (and other algebraic entities)
In-Reply-To: <fb6fbf560704290840i3b758489q6eb6a3fba3931d8d@mail.gmail.com>
References: <fb6fbf560704251510i2ba00c73if2aa5d1784df190d@mail.gmail.com>
	<ca471dc20704261325k2853cebdg179c07326c154cd4@mail.gmail.com>
	<76fd5acf0704280613w38c273b6xd3073d0076caa0d5@mail.gmail.com>
	<fb6fbf560704290840i3b758489q6eb6a3fba3931d8d@mail.gmail.com>
Message-ID: <ca471dc20704291601q7298e183q54edea4b9f9a2893@mail.gmail.com>

On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> Guido wrote:
> > Note though that only the second argument to either function can
> > overload the rules. IOW if you write isinstance(x, C), there is no way
> > that x could attempt to lie; but C could.
>
> As Barry pointed out, this means a class can't easily say "my parent
> may well implement ABC, but I don't."  That special case could be
> added to the default overriding behavior, but ... it starts to get
> fragile.

That use case is very questionable, and I see it more as an
anti-pattern; I suspect that it's a case of inheriting implementation
where you really ought to be using containment instead of subclassing.
It's why I'm uncomfortable with string.Template inheriting from a
string type, and and ditto for the OO os.path proposals that have the
path object inherit from a string type (I know not all of them do).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Mon Apr 30 01:17:46 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 29 Apr 2007 17:17:46 -0600
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
	<fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
	<76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com>
Message-ID: <d11dcfba0704291617o62196b36qbeb73978ea576d8a@mail.gmail.com>

On 4/29/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/29/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
> > > 4. super objects are callable, and calling them will execute the super
> > > method with the same name as the instance method currently being executed.
> > > Lookup of this method occurs when the instance method is entered.
> > >
> > >     class A(object):
> > >         def f(self):
> > >             pass
> > >
> > >     class B(A):
> > >         def f(self):
> > >             super() # Calls A.f(self)
>
> This might run into the same issue I had to cover, where you get an
> ambiguous situation trying to distinguish between calling super and
> calling the __call__ method of the next class in the MRO.

Note that it's at least not backwards incompatible because super
objects are not currently callable. You have to make the explicit
.__call__() invocation::

>>> class C(object):
...     def __call__(self):
...         return 'C'
...
>>> class D(C):
...     def __call__(self):
...         sup = super(D, self)
...         return 'D' + sup()
...
>>> d = D()
>>> d()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
  File "<interactive input>", line 4, in __call__
TypeError: 'super' object is not callable
>>> class D(C):
...     def __call__(self):
...         sup = super(D, self)
...         return 'D' + sup.__call__()
...
>>> d = D()
>>> d()
'DC'

That said, you're probably right here:

> We should absolutely avoid a situation in python now where X() differs
> from X.__call__()

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Mon Apr 30 01:27:07 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 16:27:07 -0700
Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword
In-Reply-To: <43aa6ff70704291549v50a11ebah1a96cc213d2d5177@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
	<fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
	<76fd5acf0704291523q78a14a26lb421d927252495a8@mail.gmail.com>
	<43aa6ff70704291549v50a11ebah1a96cc213d2d5177@mail.gmail.com>
Message-ID: <ca471dc20704291627x7be9bd0euf0ffdf4a6b3bc351@mail.gmail.com>

On 4/29/07, Collin Winter <collinw at gmail.com> wrote:
> On 4/29/07, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > As long as you can be explicit, should the shortcut be a full
> > > shortcut?  That is,
> > >
> > >     def f(self, a, b=c, *args, **kwargs):
> > >         super()    # passes the exact arglist that f got
> >
> > I sure wish my previous complaints didn't hinder this, because I
> > really love the idea of being able to this, which would really
> > encourage more compatible method signatures, so you can use the
> > shortcut! I'm desperate for a solution that satisfies all the sides of
> > the equation.
>
> I hate this. super() calls would be completely different from other
> function calls in that what appears to be an empty argument list is
> actually a filled-out argument list.
>
> Please stick to the original topic of figuring out how to remove the
> class name from super calls, rather than inventing new magical,
> spooky-action-at-a-distance toys.

Amen, brother!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 30 01:30:18 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 16:30:18 -0700
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<76fd5acf0704232111q313c6ed7h5f271864faa201e7@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
	<fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
Message-ID: <ca471dc20704291630q7108d0ccl80c09baa57f98d6e@mail.gmail.com>

On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> So it is a "keyword" in the sense that None is a keyword; not in the
> stronger sense that "if" is a keyword?

Um, how do you see those two differ? Is 'if' a keyword in the same
sense as 'or', or in a different sense?

I realize that in Python 2.5, None is not a full-fledged keyword but
cannot be used as an assignment target. But that's only transitional.
In 3.0 I imagine it becoming a keyword in the grammar (whose only
appearance would be as one of the alternatives for 'atom'). And we're
talking 3.0 here.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From talin at acm.org  Mon Apr 30 02:11:44 2007
From: talin at acm.org (Talin)
Date: Sun, 29 Apr 2007 17:11:44 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
 algebraic entities)
In-Reply-To: <ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>	<f10g7k$fag$1@sea.gmane.org>	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
Message-ID: <46353440.5060402@acm.org>

Guido van Rossum wrote:
> Maybe we should stop trying to capture radically different
> mathematical number systems using classes or types, and limit
> ourselves to capturing the systems one learns in high school: C, R, Q,
> Z, and (perhaps) N (really N0). The concrete types would be complex <:
> C, float<:R, Decimal<:R, int<:Z. NumPy would have many more. One could
> argue that float and Decimal are <:Q, but I'm not sure if that makes
> things better pragmatically; I guess I'm coming from the old Algol
> school where float was actually called real (and in retrospect I wish
> I'd called it that in Python). I'd rather reserve membership of Q for
> an infinite precision rational type (which many people have
> independently implemented).

I haven't really been following this discussion, given my lack of 
understanding of the issues involved, but I want to make one observation 
about the discussion.

Normally, when someone suggests an idea for a major addition to Python 
of this scope, the standard response is that they should go develop it 
as a 3rd-party package and see if becomes popular, and if it does it can 
be considered for inclusion in the standard library.

Unfortunately, we are somewhat constrained in this case, because we're 
talking about altering the behavior of some fairly fundamental built-in 
types - which means that it's going to be hard to implement it as an 
add-on library.

And yet, it seems to me that this particular set of features really does 
need a long gestation period compared to some others that we've 
discussed. Most of the 3000-series PEPs are really about a fairly small 
set of base decisions. Even the long PEPs are more about descriptions of 
the logical consequences of those decisions than about the decisions 
themselves. The ABC PEPs are different, in that they are standardizing a 
whole slew of things all at once. Moreover, I think there is a real 
danger here of a kind of ivory-tower decision making, isolated from 
day-to-day writing of applications to keep them grounded.

The question of whether to limit to the "high school" number classes, or 
to go with the more mathematically abstract set of things is a decision 
which, it seems to me, ought to be made in the context of actual use 
cases, rather than abstract theorizing about use cases. (I'm also 
generally supportive about copying what other languages have done, on 
the theory that they too have been tested by real-world use.)

If it were technically possible, I would recommend that this PEP have to 
run the same gauntlet that any other large library addition would, which 
is to go through a long period of community feedback and criticism, 
during which a large number of people actually attempt to use the 
feature for real work. I also think, in this case, that the special 
power of "core python developer fiat" should not be invoked unless it 
has to be, because I don't think that there is a firm basis for making 
such a judgment yet.

I would also suggest that some thought be given to ways to allow for 
experimentation with different variations of this feature. If it is not 
possible to make these numeric classes definable in Python at runtime, 
then perhaps it is possible instead to allow for custom builds of the 
Python 3000 executable with different arrangements and configurations of 
these built-in classes.

-- Talin

From steven.bethard at gmail.com  Mon Apr 30 02:19:14 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Sun, 29 Apr 2007 18:19:14 -0600
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <46353440.5060402@acm.org>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
Message-ID: <d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>

On 4/29/07, Talin <talin at acm.org> wrote:
> If it were technically possible, I would recommend that this PEP have to
> run the same gauntlet that any other large library addition would, which
> is to go through a long period of community feedback and criticism,
> during which a large number of people actually attempt to use the
> feature for real work.

This sounds like a pretty good reason to add __isinstance__() and
__issubclass__().  Then the various ABCs can be distributed as
third-party modules but can still make sure that things like
isinstance(42, Real) and issubclass(int, Complex) are True (or
whatever other assertions people want to make).

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From greg.ewing at canterbury.ac.nz  Mon Apr 30 02:53:39 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 30 Apr 2007 12:53:39 +1200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <46346DD9.1020101@v.loewis.de>
References: <46327CC9.4040901@v.loewis.de> <f0u4nh$nqt$1@sea.gmane.org>
	<4632DD83.2030301@v.loewis.de> <4633F918.1010205@canterbury.ac.nz>
	<46346DD9.1020101@v.loewis.de>
Message-ID: <46353E13.5080501@canterbury.ac.nz>

Martin v. L?wis wrote:
> That doesn't work. For variable-sized objects, it would need
> to be ob_base.ob_base.ob_type.

I was wondering about that. It seems that each level
of inheritance is going to add another level of
required member access. Or is it legal to short-
circuit all that and just cast directly to the
ultimate base object?

--
Greg

From ironfroggy at gmail.com  Mon Apr 30 03:01:42 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sun, 29 Apr 2007 21:01:42 -0400
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <ca471dc20704291630q7108d0ccl80c09baa57f98d6e@mail.gmail.com>
References: <ca471dc20704181457h9ba0fe1wefcebd52da74e391@mail.gmail.com>
	<d11dcfba0704232122w2685c513g3aa23b8b94d1b5e7@mail.gmail.com>
	<76fd5acf0704240711p22f8060k25d787c0e85b6fb8@mail.gmail.com>
	<d11dcfba0704241035y55f0d111x898ad67898b7a4ae@mail.gmail.com>
	<d11dcfba0704242226s7983b664j5f5a5b78b185a3df@mail.gmail.com>
	<ca471dc20704251133o2baa5053xc8dc24e65ea4155a@mail.gmail.com>
	<002401c78778$75fb7eb0$0201a8c0@ryoko>
	<00b601c78a9f$38ec9390$0201a8c0@ryoko>
	<fb6fbf560704291425o7c84d72cqe2eafb041c43d088@mail.gmail.com>
	<ca471dc20704291630q7108d0ccl80c09baa57f98d6e@mail.gmail.com>
Message-ID: <76fd5acf0704291801o47733e29u634ffa317d32a0a7@mail.gmail.com>

On 4/29/07, Guido van Rossum <guido at python.org> wrote:
> On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > So it is a "keyword" in the sense that None is a keyword; not in the
> > stronger sense that "if" is a keyword?
>
> Um, how do you see those two differ? Is 'if' a keyword in the same
> sense as 'or', or in a different sense?
>
> I realize that in Python 2.5, None is not a full-fledged keyword but
> cannot be used as an assignment target. But that's only transitional.
> In 3.0 I imagine it becoming a keyword in the grammar (whose only
> appearance would be as one of the alternatives for 'atom'). And we're
> talking 3.0 here.

I think any concerns about it not being fit as a keyword would fall
under two catagories or varying validity:
1) Too many keywords is a valid concern, because it complicates the language.
2) It just doesn't "feel" like a keyword. Less valid, unless it
_really_ doesn't feel like a keyword.

It doesn't feel like a keyword. But it doesn't feel too much not like a keyword.

Anyway, I tried to address the concerns laid out, and I'm more than
happy to alter the PEP to actually say "Lets implement this as a
keyword", and I actually meant to keep more agnostic on that point in
the proposal itself. I was more interested in covering the interface,
at least to begin, than the actual implementation. Although, being
able to have a solid, working reference implementation based on the
frame lookups and such is nice, so we can see how it will actually
work in real code, and even use it to backport code using the new
super to just about any recent Python version.

I also checked and PyPy does implement a sys._getframe() and a
IronPython currently doesn't, but seems to plan on it (there is a
placeholder, at present). I am not sure if notes on this belongs in
the PEP or not.

Draft Three follows for all. I think I'm turning off e-mail for the
rest of this evening, so I'll catch up tomorrow.

-------------------------------------------------------

PEP: XXX
Title: New Super
Version: $Revision$
Last-Modified: $Date$
Author: Calvin Spealman <ironfroggy at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 28-Apr-2007
Python-Version: 2.6
Post-History: 28-Apr-2007, 29-Apr-2007 (1), 29-Apr-2007 (2)


Abstract
========

The PEP defines the proposal to enhance the super builtin to work implicitly
upon the class within which it is used and upon the instance the current
function was called on. The premise of the new super usage suggested is as
follows:

    super.foo(1, 2)

to replace the old:

    super(Foo, self).foo(1, 2)


Rationale
=========

The current usage of super requires an explicit passing of both the class and
instance it must operate from, requiring a breaking of the DRY (Don't Repeat
Yourself) rule. This hinders any change in class name, and is often considered
a wart by many.


Specification
=============

Within the specification section, some special terminology will be used to
distinguish similar and closely related concepts. "Super type" will refer to
the actual builtin type named "super". "Next Class/Type in the MRO" will refer
to the class where attribute lookups will be performed by super, for example,
in the following, A is the "Next class in the MRO" for the use of super.

    ::

        class A(object):
            def f(self):
                return 'A'

        class B(A):
            def f(self):
                super(B, self).f() # Here, A would be out "Next class in the
                                   # MRO", of course.

A "super object" is simply an instance of the super type, which is associated
with a class and possibly with an instance of that class. Finally, "new super"
refers to the new super type, which will replace the original.

Replacing the old usage of super, calls to the next class in the MRO (method
resolution order) will be made without an explicit super object creation,
by simply accessing an attribute on the super type directly, which will
automatically apply the class and instance to perform the proper lookup. The
following example demonstrates the use of this.

    ::

        class A(object):
            def f(self):
                return 'A'

        class B(A):
            def f(self):
                return 'B' + super.f()

        class C(A):
            def f(self):
                return 'C' + super.f()

        class D(B, C):
            def f(self):
                return 'D' + super.f()

        assert D().f() == 'DBCA'

The proposal adds a dynamic attribute lookup to the super type, which will
automatically determine the proper class and instance parameters. Each super
attribute lookup identifies these parameters and performs the super lookup on
the instance, as the current super implementation does with the explicit
invokation of a super object upon a class and instance.

The enhancements to the super type will define a new __getattr__ classmethod
of the super type, which must look backwards to the previous frame and locate
the instance object. This can be naively determined by located the local named
by the first argument to the function. Using super outside of a function where
this is a valid lookup for the instance can be considered undocumented in its
behavior. This special method will actually be invoked on attribute lookups to
the super type itself, as opposed to super objects, as the current
implementation works. This may pose open issues, which are detailed below.

"Every class will gain a new special attribute, __super__, which refers to an
instance of the associated super object for that class" In this capacity, the
new super also acts as its own descriptor, create an instance-specific super
upon lookup.

Much of this was discussed in the thread of the python-dev list, "Fixing super
anyone?" [1]_.

Open Issues
-----------

__call__ methods
''''''''''''''''

Backward compatability of the super type API raises some issues. Names, the
lookup of the __call__ of the super type itself, which means a conflict with
doing an actual super lookup of the __call__ attribute. Namely, the following
is ambiguous in the current proposal:

    ::

        super.__call__(arg)

Which means the backward compatible API, which involves instansiating the super
type, will either not be possible, because it will actually do a super lookup
on the __call__ attribute, or there will be no way to perform a super lookup on
the __call__ attribute. Both seem unacceptable, so any suggestions are welcome.

Actually keeping the old super around in 2.x and creating a completely new super
type seperately may be the best option. A future import or even a simple import
in 2.x of the new super type from some builtin module may offer a way to choose
which each module uses, even mixing uses by binding to different names. Such a
builtin module might be called 'newsuper'. This module is also the reference
implementation, which I will present below.

super type's new getattr
''''''''''''''''''''''''

To give the behavior needed, the super type either needs a way to do dynamic
lookup of attributes on the super type object itself or define a metaclass for
the builtin type. This author is unsure which, if either, is possible with C-
defined types.

When should we create __super__ attributes?
'''''''''''''''''''''''''''''''''''''''''''

They either need to be created on class creation or on __super__ attribute
lookup. For the second, they could be cached, of course, which seems like it
may be the best idea, if implicit creation of a super object for every class is
considered too much overhead.

How does it work in inner functions?
''''''''''''''''''''''''''''''''''''

If a method defines a function and super is used inside of it, how does this
work? The frame looking and instance detection breaks here. However,  if there
can be some unambiguous way to use both the new super form and still be able to
explicitly name the type and instance, I think its an acceptable tradeoff to
simply be explicit in these cases, rather than add weird super-specific lookup
rules in these cases.

An example of such a problematic bit of code is:

    ::

        class B(A):
            def f(self):
                def g():
                    return super.f()
                return g()

Should super actually become a keyword?
'''''''''''''''''''''''''''''''''''''''

This would solve many of the problems and allow more direct implementation of
super into the language proper. However, some are against the actual keyword-
ization of super. The simplest solution is often the correct solution and the
simplest solution may well not be adding additional keywords to the language
when they are not needed. Still, it may solve many of the other open issues.

Can we also allow super()?
''''''''''''''''''''''''''

There is strong sentiment for and against this, but implementation and style
concerns are obvious. Particularly, that its "magical" and that super() would
differ from super.__call__(), being very unpythonic.


Reference Implementation
========================

This implementation was a cooperative contribution in the original thread [1]_.

    ::

        #!/usr/bin/env python
        #
        # newsuper.py

        import sys

        class SuperMetaclass(type):
            def __getattr__(cls, attr):
                calling_frame = sys._getframe().f_back
                instance_name = calling_frame.f_code.co_varnames[0]
                instance = calling_frame.f_locals[instance_name]
                return getattr(instance.__super__, attr)

        class Super(object):
            __metaclass__ = SuperMetaclass
            def __init__(self, type, obj=None):
                if isinstance(obj, Super):
                    obj = obj.__obj__
                self.__type__ = type
                self.__obj__ = obj
            def __get__(self, obj, cls=None):
                if obj is None:
                    raise Exception('only supports instances')
                else:
                    return Super(self.__type__, obj)
            def __getattr__(self, attr):
                mro = iter(self.__obj__.__class__.__mro__)
                for cls in mro:
                    if cls is self.__type__:
                        break
                for cls in mro:
                    if attr in cls.__dict__:
                        x = cls.__dict__[attr]
                        if hasattr(x, '__get__'):
                            x = x.__get__(self, cls)
                        return x
                raise AttributeError, attr

        class autosuper(type):
            def __init__(cls, name, bases, clsdict):
                cls.__super__ = Super(cls)

        if __name__ == '__main__':
            class A(object):
                __metaclass__ = autosuper
                def f(self):
                    return 'A'

            class B(A):
                def f(self):
                    return 'B' + Super.f()

            class C(A):
                def f(self):
                    return 'C' + Super.f()

            class D(B, C):
                def f(self, arg=None):
                    var = None
                    return 'D' + Super.f()

            assert D().f() == 'DBCA'


Alternative Proposals
=====================

No Changes
----------

Although its always attractive to just keep things how they are, people have
sought a change in the usage of super calling for some time, and for good
reason, all mentioned previously.

    * Decoupling from the class name (which might not even be bound to the
      right class anymore! [2]_)
    * Simpler looking, cleaner super calls would be better

super(__this_class__, self)
---------------------------

This is nearly an anti-proposal, as it basically relies on the acceptance of
the __this_class__ PEP, which proposes a special name that would always be
bound to the class within which it is used. If that is accepted, __this_class__
could simply be used instead of the class' name explicitly, solving the name
binding issues [2]_.

self.__super__.foo(*args)
-------------------------

The __super__ attribute is mentioned in this PEP in several places, and could
be a candidate for the completel solution, actually using it explicitly instead
of any super usage directly. However, double-underscore names are usually an
internal detail, and attempted to be kept out of everyday code.

super(self, *args) or __super__(self, *args)
--------------------------------------------

This solution only solves the problem of the type indication, does not handle
differently named super methods, and is explicit about the name of the
instance. It is less flexable without being able to enacted on other method
names, in cases where that is needed. One use case this fails is where a base-
class has a factory classmethod and a subclass has two factory classmethods,
both of which needing to properly make super calls to the one in the base-
class.

super.foo(self, *args)
----------------------

This variation actually eliminates the problems with locating the proper
instance, and if any of the alternatives were pushed into the spotlight, I
would want it to be this one.

super or super()
----------------

This proposal leaves no room for different names, signatures, or application
to other classes, or instances. A way to allow some similar use alongside the
normal proposal would be favorable, encouraging good design of multiple
inheritence trees and compatible methods.


History
=======
29-Apr-2007 - Changed title from "Super As A Keyword" to "New Super"
            - Updated much of the language and added a terminology section
              for clarification in confusing places.
            - Added reference implementation and history sections.

References
==========

.. [1] Fixing super anyone?
   (http://mail.python.org/pipermail/python-3000/2007-April/006667.html)


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

From guido at python.org  Mon Apr 30 03:26:45 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 18:26:45 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <46353440.5060402@acm.org>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
Message-ID: <ca471dc20704291826p6cb8e001gf3df7a3dc6160667@mail.gmail.com>

On 4/29/07, Talin <talin at acm.org> wrote:
> Guido van Rossum wrote:
> > Maybe we should stop trying to capture radically different
> > mathematical number systems using classes or types, and limit
> > ourselves to capturing the systems one learns in high school: C, R, Q,
> > Z, and (perhaps) N (really N0). The concrete types would be complex <:
> > C, float<:R, Decimal<:R, int<:Z. NumPy would have many more. One could
> > argue that float and Decimal are <:Q, but I'm not sure if that makes
> > things better pragmatically; I guess I'm coming from the old Algol
> > school where float was actually called real (and in retrospect I wish
> > I'd called it that in Python). I'd rather reserve membership of Q for
> > an infinite precision rational type (which many people have
> > independently implemented).
>
> I haven't really been following this discussion, given my lack of
> understanding of the issues involved, but I want to make one observation
> about the discussion.
>
> Normally, when someone suggests an idea for a major addition to Python
> of this scope, the standard response is that they should go develop it
> as a 3rd-party package and see if becomes popular, and if it does it can
> be considered for inclusion in the standard library.
>
> Unfortunately, we are somewhat constrained in this case, because we're
> talking about altering the behavior of some fairly fundamental built-in
> types - which means that it's going to be hard to implement it as an
> add-on library.

Not entirely true in the latest incarnation -- the proposed
overloading of isinstance() and issubclass() will make it possible to
add Ring-ness to float and int as an afterthough from a user-defined
class.

> And yet, it seems to me that this particular set of features really does
> need a long gestation period compared to some others that we've
> discussed. Most of the 3000-series PEPs are really about a fairly small
> set of base decisions. Even the long PEPs are more about descriptions of
> the logical consequences of those decisions than about the decisions
> themselves. The ABC PEPs are different, in that they are standardizing a
> whole slew of things all at once. Moreover, I think there is a real
> danger here of a kind of ivory-tower decision making, isolated from
> day-to-day writing of applications to keep them grounded.
>
> The question of whether to limit to the "high school" number classes, or
> to go with the more mathematically abstract set of things is a decision
> which, it seems to me, ought to be made in the context of actual use
> cases, rather than abstract theorizing about use cases. (I'm also
> generally supportive about copying what other languages have done, on
> the theory that they too have been tested by real-world use.)

If that's the criterion (and I agree) it should be fairly obvious by
now that Ring and MonoidUnderPlus and everything in between should be
cast out, and we should stick with high school math. (Just note that
Travis Oliphant, Mr. NumPy Himself, pretty much confessed being
confused by the algebra stuff).

> If it were technically possible, I would recommend that this PEP have to
> run the same gauntlet that any other large library addition would, which
> is to go through a long period of community feedback and criticism,
> during which a large number of people actually attempt to use the
> feature for real work. I also think, in this case, that the special
> power of "core python developer fiat" should not be invoked unless it
> has to be, because I don't think that there is a firm basis for making
> such a judgment yet.
>
> I would also suggest that some thought be given to ways to allow for
> experimentation with different variations of this feature. If it is not
> possible to make these numeric classes definable in Python at runtime,
> then perhaps it is possible instead to allow for custom builds of the
> Python 3000 executable with different arrangements and configurations of
> these built-in classes.

So how about we reduce the scope of our (!) PEP (or perhaps of a new
one) to two items: (a) add @abstractmethod, and (b) overload
isinstance() and issubclass()? Library authors can do everything they
want with those, and we can always add a specific set of ABCs for
containers and/or numbers later in the 3.0 development cycle.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Mon Apr 30 03:40:41 2007
From: collinw at gmail.com (Collin Winter)
Date: Sun, 29 Apr 2007 18:40:41 -0700
Subject: [Python-3000] Traits/roles instead of ABCs
Message-ID: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>

[cc:-numpy]

On 4/29/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/29/07, Talin <talin at acm.org> wrote:
> > If it were technically possible, I would recommend that this PEP have to
> > run the same gauntlet that any other large library addition would, which
> > is to go through a long period of community feedback and criticism,
> > during which a large number of people actually attempt to use the
> > feature for real work.
>
> This sounds like a pretty good reason to add __isinstance__() and
> __issubclass__().  Then the various ABCs can be distributed as
> third-party modules but can still make sure that things like
> isinstance(42, Real) and issubclass(int, Complex) are True (or
> whatever other assertions people want to make).

To me, having to mess with something as fundamental as isinstance()
and issubclass() sounds like a pretty good reason to prefer a
different solution over ABCs.

The mechanism I'm most familiar with for solving this problem (which,
unless I've missed something, is, "how do I make sure this object does
what I expect?") is Perl 6's roles system; if you know about Squeak
Smalltalk's "traits" system, you're on the same page.

(I discussed this with Jeffery Yasskin at lunch the other day, and he
seemed interested, so here follows a medium-length explanation of
roles/traits.)

The idea is that we leave the traditional class system to manage an
object's implementation and use a more-or-less orthogonal system to
manage that same object's behavior.

Quoting from an article on Perl 6's roles,

"""
A role is a named collection of behavior ? a set of methods identified
by a unique name. This resembles a class or a type, in that referring
to the role refers to the combined set of behaviors, but it is more
general than a class and more concrete than a type.

Put another way, a role is an assertion about a set of capabilities.
"""
[http://www.oreillynet.com/onlamp/blog/2006/08/roles_composable_units_of_obje.html]

The key part of traits/roles is that, because the system is separate
from classes, you can do runtime role composition without a) mucking
with __bases__, or b) making isinstance() and issubclass() squishy and
ill-defined. By "runtime role composition", I mean it would be
possible to do something like this at runtime:

py> int.implements(Ring)

or (depending on your spelling preferences)

py> Ring.implemented_by(int)

That is, it would be possible to distribute Jeffery's numeric kinds as
a third-party library and still have them affect the built-in numeric
types.


Now, how do roles/traits differ from interfaces? The primary
distinction is that interfaces only define behavior, whereas roles can
provide a dummy implementation. It would be possible for an Equality
role to be defined like this (strawman syntax):

class Equality(Role):
  def __eq__(self, other):
    return not self != other

  def __ne__(self, other):
    return not self == other


How would all this work in Python? Here's a collection of strawman proposals:

1. For declaring roles statically, either class decorators or class
arguments could be used:

@does(Complex)
class A:
  ...


class A(does=Complex):
  ...


2. I gave a strawman syntax proposal for runtime role composition
above (modulo the method name): "Ring.done_by(int)" or
"int.does(Ring)". (Perl 6 speaks in terms of "an object/class *does* a
role".) Perl 6 also allows individual instances to do different roles
than their class does, but I'm not sure we want to go that far.

3. For querying role performance, something like this could work:
"isdoneby(x, Ring)".

If class decorators and the "role.method(object)" spellings were used,
I think this could probably be issued as a third-party package with no
need to modify the interpreter at all.


For the interested, here's some more reading about roles/traits in
roughly descending order of readability:

http://www.perlmonks.org/?node_id=384858
http://www.oreillynet.com/onlamp/blog/2006/08/roles_composable_units_of_obje.html
http://www.iam.unibe.ch/~scg/Archive/Papers/Scha03aTraits.pdf

If there's interest in this, I could probably whip up a PEP before the deadline.

Collin Winter

From greg.ewing at canterbury.ac.nz  Mon Apr 30 03:48:45 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 30 Apr 2007 13:48:45 +1200
Subject: [Python-3000] Could isinstance/issubclass overriding be dangerous?
In-Reply-To: <d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
Message-ID: <46354AFD.5070702@canterbury.ac.nz>

Currently, isinstance(x, C) implies that the
C struct layout of x is compatible with that
defined by C. I'm worried that changing this
could cause difficulties for extension modules
that use PyObject_IsInstance do decide what
to do with things, e.g. in a binary operator
method.

--
Greg

From jimjjewett at gmail.com  Mon Apr 30 03:51:52 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 21:51:52 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
Message-ID: <fb6fbf560704291851o7ca66a79g1cca54b13523cfbb@mail.gmail.com>

On 4/29/07, Guido van Rossum <guido at python.org> wrote:

> Hmm... Maybe the conclusion to draw from this is that we shouldn't
> make Ring a class? Maybe it ought to be a metaclass, so we could ask
> isinstance(Complex, Ring)?

Yes; all the ABCs are assertions about the class.  (Zope interfaces do
support instance-specific interfaces, which has been brought up as a
relative weakness of ABCs.)

The only thing two subclasses of an *Abstract* class need to have in
common is that they both (independently) meet the requirements of the
ABC.  If not for complexity of implementation, that would be better
described as a common metaclass.

Using a metaclass would also solve the "when to gripe" issue; the
metaclass would gripe if it couldn't make every method concrete.  If
this just used the standard metaclass machinery, then it would mean a
much deeper metaclass hierarchy than we're used to; MutableSet would a
have highly dervived metaclass.

> The more I think about it, it sounds like the right thing to do. To
> take PartiallyOrdered (let's say PO for brevity) as an example, the
> Set class should specify PO as a metaclass. The PO metaclass could
> require that the class implement __lt__ and __le__. If it found a
> class that didn't implement them, it could make the class abstract by
> adding the missing methods to its __abstractmethods__ attribute.

Or by making it a sub(meta)class, instead of a (regular instance) class.

> if it found that the class implemented one but not the other, it could
> inject a default implementation of the other in terms of the one and
> __eq__.

This also allows greater freedom in specifying which subsets of
methods must be defined.

> Now, you could argue that Complex should also be a metaclass. While
> that may mathematically meaningful (for all I know there are people
> doing complex number theory using Complex[Z/n]), for Python's numeric
> classes I think it's better to make Complex a regular class
> representing all the usual complex numbers (i.e. a pair of Real
> numbers).

complex already meets that need.  Complex would be the metaclass
representing the restrictions on the class, so that independing
implementations wouldn't have to fake-inherit from complex.

> I expect that the complex subclasses used in practice are
> all happy under mixed arithmetic using the usual definition of mixed
> arithmetic: convert both arguments to a common base class and compute
> the operation in that domain.

It is reasonable to insist that all Complex classes have a way to
tranform their instances into (builtin) complex instances, if only as
a final fallback.  There is no need for complex to be a base class.

-jJ

From jimjjewett at gmail.com  Mon Apr 30 04:01:57 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 22:01:57 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
Message-ID: <fb6fbf560704291901o2bad63d9x51142f4ed5cace6f@mail.gmail.com>

On 4/29/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/29/07, Talin <talin at acm.org> wrote:
> > If it were technically possible, I would recommend that this PEP have to
> > run the same gauntlet that any other large library addition would, which
> > is to go through a long period of community feedback and criticism,
> > during which a large number of people actually attempt to use the
> > feature for real work.

> This sounds like a pretty good reason to add __isinstance__() and
> __issubclass__().  Then the various ABCs can be distributed as
> third-party modules but can still make sure that things like
> isinstance(42, Real) and issubclass(int, Complex) are True (or
> whatever other assertions people want to make).

Or isexample, so that we aren't locked into implementing ABCs as base classes.

    def isexample(val, ABC):
        return ABC.__example__(val)

    class ABC(object):
        def __example__(cls, val):
            if val in cls.__instance:  return True
            if val in cls.__non_instance:  return False
            for instclass in type(val).__mro__:
                if instclass in __class:  return True
                if instclass in __non_class: return False
            return False

        ... methods to register classes, unregister subclasses, etc

-jJ

From guido at python.org  Mon Apr 30 04:31:53 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 19:31:53 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <fb6fbf560704291901o2bad63d9x51142f4ed5cace6f@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
	<fb6fbf560704291901o2bad63d9x51142f4ed5cace6f@mail.gmail.com>
Message-ID: <ca471dc20704291931h24df36c8tb0869f48a4cce02b@mail.gmail.com>

On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> Or isexample, so that we aren't locked into implementing ABCs as base classes.

You don't have to use the feature even if it exists. :-)

I think there are good reasons to support overriding
isinstance/issubclass beyond ABCs.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 30 05:02:49 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 20:02:49 -0700
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <fb6fbf560704291851o7ca66a79g1cca54b13523cfbb@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<fb6fbf560704291851o7ca66a79g1cca54b13523cfbb@mail.gmail.com>
Message-ID: <ca471dc20704292002i4bb4696fw115b41210679610b@mail.gmail.com>

On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/29/07, Guido van Rossum <guido at python.org> wrote:
> > Hmm... Maybe the conclusion to draw from this is that we shouldn't
> > make Ring a class? Maybe it ought to be a metaclass, so we could ask
> > isinstance(Complex, Ring)?
>
> Yes; all the ABCs are assertions about the class.

I don't think so. Many are quite useful for introspection of instances
as well, e.g. Hashable/Iterable (the whole "One Trick Ponies" section)
as well as the distinction between Sequence and Mapping. It's the
binary operations where the class comes into play.

> (Zope interfaces do
> support instance-specific interfaces, which has been brought up as a
> relative weakness of ABCs.)
>
> The only thing two subclasses of an *Abstract* class need to have in
> common is that they both (independently) meet the requirements of the
> ABC.  If not for complexity of implementation, that would be better
> described as a common metaclass.

Again, not so fast; it depends. The way the Set section of the PEP is
currently written, all sets are comparable (in the subset/superset
sense) to all other sets, and for ComposableSet instances the union,
intersection and both types of differences are also computable across
class boundaries.

> Using a metaclass would also solve the "when to gripe" issue; the
> metaclass would gripe if it couldn't make every method concrete.  If
> this just used the standard metaclass machinery, then it would mean a
> much deeper metaclass hierarchy than we're used to; MutableSet would a
> have highly dervived metaclass.

I think you're going way too fast here.

> > The more I think about it, it sounds like the right thing to do. To
> > take PartiallyOrdered (let's say PO for brevity) as an example, the
> > Set class should specify PO as a metaclass. The PO metaclass could
> > require that the class implement __lt__ and __le__. If it found a
> > class that didn't implement them, it could make the class abstract by
> > adding the missing methods to its __abstractmethods__ attribute.
>
> Or by making it a sub(meta)class, instead of a (regular instance) class.

That makes no sense. Deciding on the fly whether something should be a
class or a metaclass sounds like a fine recipe for end-user confusion.

> > if it found that the class implemented one but not the other, it could
> > inject a default implementation of the other in terms of the one and
> > __eq__.
>
> This also allows greater freedom in specifying which subsets of
> methods must be defined.
>
> > Now, you could argue that Complex should also be a metaclass. While
> > that may mathematically meaningful (for all I know there are people
> > doing complex number theory using Complex[Z/n]), for Python's numeric
> > classes I think it's better to make Complex a regular class
> > representing all the usual complex numbers (i.e. a pair of Real
> > numbers).
>
> complex already meets that need.  Complex would be the metaclass
> representing the restrictions on the class, so that independing
> implementations wouldn't have to fake-inherit from complex.

I was thinking of other representations of complex numbers as found
e.g. in numpy. These vary mostly by using fewer (or more?) bits for
the real and imag parts. They can't realistically subclass complex, as
their implementation is independent; they should subclass Complex, to
indicate that they implement the Complex API. I really think you're
going too far with the metaclass idea.

Now, if we had parameterizable types (for which I've proposed a
notation, e.g. list[int] would be a list of integers, and
Mapping[String, Real] would be a mapping from strings to real numbers;
but I don't expect this to be in py3k, as it needs more
experimentation), Complex might be a parameterizable type, and e.g.
the current concrete complex type could be equated to Complex[float];
but without that, I think it's fine to see Complex as the Abstract
Base Class and complex as one concrete representation.

> > I expect that the complex subclasses used in practice are
> > all happy under mixed arithmetic using the usual definition of mixed
> > arithmetic: convert both arguments to a common base class and compute
> > the operation in that domain.
>
> It is reasonable to insist that all Complex classes have a way to
> tranform their instances into (builtin) complex instances, if only as
> a final fallback.  There is no need for complex to be a base class.

I agree complex shouldn't be a base class (apologies if I implied that
by using lowercase) but I still think Complex should be a base class.

To be honest, I'm not sure what should happen with mixed operations
between classes that only have an abstract common base class. The
normal approach for binary operations is that each side gets a try.
For pairs like int+float this is easy; int.__add__ returns
NotImplemented in this case, and then float.__radd__ is called which
converts the first argument to float and returns a float. For pairs
like numpy's complex 32-bit float and numpy's complex 64-bit float it
should also be easy (numpy is aware of both types and hence always
gets to choose); and for numpy's complex combined with the built-in
complex it's easy enough too (again, numpy always gets to choose, this
time because the built-in complex doesn't know about numpy).

But what if I wrote my own complex type based on decimal.Decimal, and
I encountered a numpy complex? numpy doesn't know about my type and
hence passes the ball to me; but perhaps I don't know about numpy
either, and then a TypeError will be raised. So, to be a good citizen,
I could check if the other arg was a Complex of unknown provenance,
and then I could convert to the built-in complex and pass the ball to
that type. But if all good citizens lived by that rule (and it
*appears* to be a reasonable rule), then numpy, being the ultimate
good citizen, would also convert *my* type to the built-in complex.
But that would mean that if my type *did* know about numpy (but numpy
didn't know about mine), I wouldn't get to choose what to do in half
the cases (if the numpy instance was on the left, it would get and
take the first opportunity). Perhaps we need to extend the built-in
operation processing so that if both sides return NotImplemented,
before raising TypeError, we look for some common base type
implementing the same operation. The abstract Complex type could
provide abstract implementations of the binary operators that would
convert their arguments to the concrete complex type and compute the
result that way. (Hah! Another use case for abstract methods with a
useful implementation! :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Mon Apr 30 05:29:25 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 23:29:25 -0400
Subject: [Python-3000] PEP 30XZ: Simplified Parsing
Message-ID: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>

PEP: 30xz
Title: Simplified Parsing
Version: $Revision$
Last-Modified: $Date$
Author: Jim J. Jewett <JimJJewett at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 29-Apr-2007
Post-History: 29-Apr-2007


Abstract

    Python initially inherited its parsing from C.  While this has
    been generally useful, there are some remnants which have been
    less useful for python, and should be eliminated.

    + Implicit String concatenation

    + Line continuation with "\"

    + 034 as an octal number (== decimal 28).  Note that this is
      listed only for completeness; the decision to raise an
      Exception for leading zeros has already been made in the
      context of PEP XXX, about adding a binary literal.


Rationale for Removing Implicit String Concatenation

    Implicit String concatentation can lead to confusing, or even
    silent, errors. [1]

        def f(arg1, arg2=None): pass

        f("abc" "def")  # forgot the comma, no warning ...
                        # silently becomes f("abcdef", None)

    or, using the scons build framework,

        sourceFiles = [
        'foo.c',
        'bar.c',
        #...many lines omitted...
        'q1000x.c']

    It's a common mistake to leave off a comma, and then scons complains
    that it can't find 'foo.cbar.c'.  This is pretty bewildering behavior
    even if you *are* a Python programmer, and not everyone here is.

    Note that in C, the implicit concatenation is more justified; there
    is no other way to join strings without (at least) a function call.

    In Python, strings are objects which support the __add__ operator;
    it is possible to write:

        "abc" + "def"

    Because these are literals, this addition can still be optimized
    away by the compiler.

    Guido indicated [2] that this change should be handled by PEP, because
    there were a few edge cases with other string operators, such as the %.
    The resolution is to treat them the same as today.

        ("abc %s def" + "ghi" % var)  # fails like today.
                                      # raises TypeError because of
                                      # precedence.  (% before +)

        ("abc" + "def %s ghi" % var)  # works like today; precedence makes
                                      # the optimization more difficult to
                                      # recognize, but does not change the
                                      # semantics.

        ("abc %s def" + "ghi") % var  # works like today, because of
                                      # precedence:  () before %
                                      # CPython compiler can already
                                      # add the literals at compile-time.


Rationale for Removing Explicit Line Continuation

    A terminal "\" indicates that the logical line is continued on the
    following physical line (after whitespace).

    Note that a non-terminal "\" does not have this meaning, even if the
    only additional characters are invisible whitespace.  (Python depends
    heavily on *visible* whitespace at the beginning of a line; it does
    not otherwise depend on *invisible* terminal whitespace.)  Adding
    whitespace after a "\" will typically cause a syntax error rather
    than a silent bug, but it still isn't desirable.

    The reason to keep "\" is that occasionally code looks better with
    a "\" than with a () pair.

        assert True, (
            "This Paren is goofy")

    But realistically, that paren is no worse than a "\".  The only
    advantage of "\" is that it is slightly more familiar to users of
    C-based languages.  These same languages all also support line
    continuation with (), so reading code will not be a problem, and
    there will be one less rule to learn for people entirely new to
    programming.


Rationale for Removing Implicit Octal Literals

    This decision should be covered by PEP ???, on numeric literals.
    It is mentioned here only for completeness.

    C treats integers beginning with "0" as octal, rather than decimal.
    Historically, Python has inherited this usage.  This has caused
    quite a few annoying bugs for people who forgot the rule, and
    tried to line up their constants.

        a = 123
        b = 024   # really only 20, because octal
        c = 245

    In Python 3.0, the second line will instead raise a SyntaxError,
    because of the ambiguity.  Instead, the line should be written
    as in one of the following ways:

        b = 24    # PEP 8
        b =  24   # columns line up, for quick scanning
        b = 0t24  # really did want an Octal!


References

    [1] Implicit String Concatenation, Jewett, Orendorff
        http://mail.python.org/pipermail/python-ideas/2007-April/000397.html

    [2] PEP 12, Sample reStructuredText PEP Template, Goodger, Warsaw
        http://www.python.org/peps/pep-0012

    [3] http://www.opencontent.org/openpub/



Copyright

    This document has been placed in the public domain.



Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:

From jimjjewett at gmail.com  Mon Apr 30 05:52:32 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 29 Apr 2007 23:52:32 -0400
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704292002i4bb4696fw115b41210679610b@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<fb6fbf560704291851o7ca66a79g1cca54b13523cfbb@mail.gmail.com>
	<ca471dc20704292002i4bb4696fw115b41210679610b@mail.gmail.com>
Message-ID: <fb6fbf560704292052u3e599539m1b85f636040bcd0b@mail.gmail.com>

On 4/29/07, Guido van Rossum <guido at python.org> wrote:
> On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 4/29/07, Guido van Rossum <guido at python.org> wrote:
> > > Hmm... Maybe the conclusion to draw from this is that we shouldn't
> > > make Ring a class? Maybe it ought to be a metaclass, so we could ask
> > > isinstance(Complex, Ring)?

> > Yes; all the ABCs are assertions about the class.

> I don't think so. Many are quite useful for introspection of instances
> as well, e.g. Hashable/Iterable (the whole "One Trick Ponies" section)
> as well as the distinction between Sequence and Mapping. It's the
> binary operations where the class comes into play.

I think those are one of the reasons it seemed like we should use
inheritance.  Treating them as assertions about each instance makes
sense -- but it makes just as much sense to treat them as assertions
about the class.  (Is it meaningful to ask about the size of this?
Well, for objects of this class, it is..)

> > The only thing two subclasses of an *Abstract* class need to have in
> > common is that they both (independently) meet the requirements of the
> > ABC.  If not for complexity of implementation, that would be better
> > described as a common metaclass.

> Again, not so fast; it depends. The way the Set section of the PEP is
> currently written, all sets are comparable (in the subset/superset
> sense) to all other sets, and for ComposableSet instances the union,
> intersection and both types of differences are also computable across
> class boundaries.

That is an additional constraint which the Set metaclass imposes --
and it does this by effectively coercing instances of both Set classes
to (buitin) frozenset instances to create the return value.  (It
doesn't actually create the intermediate frozenset instance, but
mySet() | mySet() will return a frozenset rather than a MySet.

> > Using a metaclass would also solve the "when to gripe" issue; the
> > metaclass would gripe if it couldn't make every method concrete.  If
> > this just used the standard metaclass machinery, then it would mean a
> > much deeper metaclass hierarchy than we're used to; MutableSet would a
> > have highly dervived metaclass.

> I think you're going way too fast here.

To be more precise, classes implementing MutabSet would have
MutableSet as a metaclass, which would mean (through inheritance) than
they would also have ComposableSet, Set, Sized, Iterable, and
PartiallyOrdered as metaclasses.

This is legal today, but *I* haven't seen code in the wild with a
metaclass hierarchy that deep.

> > > The more I think about it, it sounds like the right thing to do. To
> > > take PartiallyOrdered (let's say PO for brevity) as an example, the
> > > Set class should specify PO as a metaclass. The PO metaclass could
> > > require that the class implement __lt__ and __le__. If it found a
> > > class that didn't implement them, it could make the class abstract by
> > > adding the missing methods to its __abstractmethods__ attribute.

> > Or by making it a sub(meta)class, instead of a (regular instance) class.

> That makes no sense. Deciding on the fly whether something should be a
> class or a metaclass sounds like a fine recipe for end-user confusion.

Perhaps.  But how is that different from deciding on the fly whether
the class will be instantiable?  Either way, the user does need to
keep track of whether it is abstract; the difference is that when
inheriting, they would have to say

    (metaclass=ABC)   # I know it is an abstract class

instead of

    (ABC)  # Might be a regular class, but I can never instantiate it directly


> I was thinking of other representations of complex numbers as found
> e.g. in numpy. These vary mostly by using fewer (or more?) bits for
> the real and imag parts. They can't realistically subclass complex, as
> their implementation is independent; they should subclass Complex, to
> indicate that they implement the Complex API. I really think you're
> going too far with the metaclass idea.

Or they could have Compex as a metaclass, which would serve the same
introspective needs.

> > > I expect that the complex subclasses used in practice are
> > > all happy under mixed arithmetic using the usual definition of mixed
> > > arithmetic: convert both arguments to a common base class and compute
> > > the operation in that domain.
> >
> > It is reasonable to insist that all Complex classes have a way to
> > tranform their instances into (builtin) complex instances, if only as
> > a final fallback.  There is no need for complex to be a base class.

> I agree complex shouldn't be a base class (apologies if I implied that
> by using lowercase) but I still think Complex should be a base class.

Why?

What do you get by inheriting from it, that you couldn't get by
letting it inject any missing methods?

> take the first opportunity). Perhaps we need to extend the built-in
> operation processing so that if both sides return NotImplemented,
> before raising TypeError, we look for some common base type
> implementing the same operation. The abstract Complex type could
> provide abstract implementations of the binary operators that would
> convert their arguments to the concrete complex type and compute the
> result that way. (Hah! Another use case for abstract methods with a
> useful implementation! :-)

That seems sensible -- but you could also do just do it in the _r* method,
since the regular method already returned NotImplemented.

So Complex could inject a __radd__ method that tried

    self.__add__(complex(other))   # since addition is commutative for Complex

If the method weren't in either class, I would expect it to be a more
general fallback, like "if we don't have __lt__, try __cmp__"

-jJ

From guido at python.org  Mon Apr 30 05:58:28 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 20:58:28 -0700
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com>
Message-ID: <ca471dc20704292058s4a0d73nf7c82fd7f85d741a@mail.gmail.com>

On 4/29/07, Delaney, Timothy (Tim) <tdelaney at avaya.com> wrote:
> I think the current PEP draft is way too complicated - I don't think
> there's any need for descriptors, etc. I think we can make things work
> in the following way:
>
> 1. When a method is defined, the class is bound to it via an attribute
> (which in my version is called func_class).

In Py3k all the func_XXX attrs are renamed __XXX__, so this would be
__class__; but that's a name reserved for something else, so it would
need to be something else.  E.g. __containing_class__.

Also, this would have to be done when the class is defined; when the
method is being defined the class doesn't exist yet.

> 2. Every non-static method has an implicit cell variable called 'super'.

I think you're using 'cell' in a different sense than it is normally
used in Python's implementation. What you are looking for is called a
local variable (I deduce this from your initialization of the "cell"
with something computed from the first argument). A 'cell' (as I
introduced in my original back-of-the-envelop proposal; apparently you
totally missed this reference) is a special kind of object containing
a reference to another object and used to implement nonlocal variable
references (the Python 2 read-only nonlocals as well as the Python 3
writable nonlocals). The cell is initialized in the surrounding scope
and attached to the function object (as func_closure in 2.x, or
__closure__ in 3.0). When the function is called, a reference to the
cell is stored in the call frame.

If anything in your proposal resembles a cell (in the sense that I and
other Pythonistas mean it), it's the func_class attribute. In fact, in
this sense our proposals are equivalent modulo slight implementation
details; I proposed using a cell because there's an existing mechanism
to get these mapped into a local variable, and because cells are
cheaper than function attributes -- the first function attribute
requires creating a dict, which is one of the larger built-in objects.

> This would preferably not be able to be rebound. I also think it would
> be beneficial if the first parameter to the method couldn't be rebound
> (point 7 in my original email in this thread).

The latter part sounds irrelevant to me.

> 3. When a method is entered, the 'super' cell variable is populated by a
> call equivalent to:
>
>     super = __builtin__.super(func_class, first_parameter)
>
> This would result in 'super' being a constant object, within the scope
> of the currently-executing method. 'keyword' was perhaps too strong - I
> was thinking this would only need to be done if 'super' were actually
> used, which would be easier to determine if 'super' actually were a
> keyword. This could still be done by not emitting the above call unless
> the 'super' cell variable were ever actually used.

This sounds about right to me, with the strong preference that super
*should* be made a keyword, and the built-in of the same name renamed
to __super__, as it has a somewhat similar relationship to the super
keyword as the built-in __import__ has to the import keyword.

> I've done bytecode-hacky stuff to do the equivalent of the above (as
> much as I've been able to), but a real implementation would just emit
> the correct bytecode (or java bytecode, or whatever) in the compiled
> code object.

Shouldn't be too hard for someone who's hacked Python/compile.c before.

> The issue of super() vs. super.__call__() ambiguity - I'll need to look
> at that when I get home.

Sounds irrelevant -- if super is equivalent to
__builtin__.__super__(<class>, <firstarg>) then super never gets
called; the only thing ever done to it (in the normal course of
things) is to request an attribute of it.

> I'm a strong -1 against super() automatically passing the parameters
> that were passed to the currently-executing method.

Same here.

There seems to be the remaining controversy that there are three forms
of super calls currently in use:

super(ThisClass, self).method(...)  # in a regular method
super(ThisClass, cls).method(...)  # in a class method
super(ThisClass).method(...)  # ???

The first two are handled by your "use the first argument" approach.

The third exists so that you can create an "unbound" super instance
which is useful for the oft-misunderstood autosuper example in my
"descrintro" essay:
http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_examples
.

But since the latter is the hack that we're trying to replace with a
proper implementation here, I suspect we can get away with only
supporting the first two forms (and the third form is still supported
using __builtin__.__super__).

IOW I think you're on to something. Keep up the good word.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 30 06:05:23 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 21:05:23 -0700
Subject: [Python-3000] Could isinstance/issubclass overriding be
	dangerous?
In-Reply-To: <46354AFD.5070702@canterbury.ac.nz>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
	<46354AFD.5070702@canterbury.ac.nz>
Message-ID: <ca471dc20704292105n423c5b0dg5938b3d6b12ffcfb@mail.gmail.com>

On 4/29/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Currently, isinstance(x, C) implies that the
> C struct layout of x is compatible with that
> defined by C. I'm worried that changing this
> could cause difficulties for extension modules
> that use PyObject_IsInstance do decide what
> to do with things, e.g. in a binary operator
> method.

Unless they were looking at classic classes, why wouldn't they be
using the specific Py<Type>_Check() macros? If you want this argument
to be taken serious, use Google code search (code.google.com) or a
similar source code search engine to find examples that would break.

Again, the only envisioned overloading is where the 2nd argument is an ABC.

Also, note that PyObject_IsInstance already returns abstract results
when the first argument is a proxy instance.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Mon Apr 30 06:05:28 2007
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 29 Apr 2007 21:05:28 -0700
Subject: [Python-3000] Traits/roles instead of ABCs
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
Message-ID: <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>

[Collin Winter]
> Put another way, a role is an assertion about a set of capabilities.
 . . .
> If there's interest in this, I could probably whip up a PEP before the deadline.

+100 I'm very interested in seeing a lighter weight alternative to abc.py that:

1) is dynamic
2) doesn't require inheritance to work
3) doesn't require mucking with isinstance or other existing mechansims
4) makes a limited, useful set of assertions rather than broadly covering a whole API.
5) that leaves the notion of duck-typing as the rule rather than the exception
6) that doesn't freeze all of the key APIs in concrete

I'm concerned that the current ABC proposal will quickly evolve from optional
to required and create somewhat somewhat java-esque landscape where
inheritance and full-specification are the order of the day.

IMHO, the ABC approach is using a cannon to shoot a mosquito.  My day-to-day
problems are much smaller are could be solved by a metadata attribute or a
role/trait solution:

* knowing whether a __getitem__ method implements a mapping or a sequence
* knowing whether an object can have more that one iterator (i.e a file has one
   but a list can have many)
* knowing whether a sequence, file, cursor, etc is writable or just readonly.


Raymond


From guido at python.org  Mon Apr 30 06:08:20 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 29 Apr 2007 21:08:20 -0700
Subject: [Python-3000] PEP 30XZ: Simplified Parsing
In-Reply-To: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
References: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
Message-ID: <ca471dc20704292108w6a72bee3w17ceab7f2d3d55d9@mail.gmail.com>

[-python-dev]

I think these should be two separate proposals, with more specific
names (e.g. "remove implicit string concatenation" and "remove
backslash continuation"). There's no need to mention the octal thing
if it's already a separate PEP.

On 4/29/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> PEP: 30xz
> Title: Simplified Parsing
> Version: $Revision$
> Last-Modified: $Date$
> Author: Jim J. Jewett <JimJJewett at gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/plain
> Created: 29-Apr-2007
> Post-History: 29-Apr-2007
>
>
> Abstract
>
>     Python initially inherited its parsing from C.  While this has
>     been generally useful, there are some remnants which have been
>     less useful for python, and should be eliminated.
>
>     + Implicit String concatenation
>
>     + Line continuation with "\"
>
>     + 034 as an octal number (== decimal 28).  Note that this is
>       listed only for completeness; the decision to raise an
>       Exception for leading zeros has already been made in the
>       context of PEP XXX, about adding a binary literal.
>
>
> Rationale for Removing Implicit String Concatenation
>
>     Implicit String concatentation can lead to confusing, or even
>     silent, errors. [1]
>
>         def f(arg1, arg2=None): pass
>
>         f("abc" "def")  # forgot the comma, no warning ...
>                         # silently becomes f("abcdef", None)
>
>     or, using the scons build framework,
>
>         sourceFiles = [
>         'foo.c',
>         'bar.c',
>         #...many lines omitted...
>         'q1000x.c']
>
>     It's a common mistake to leave off a comma, and then scons complains
>     that it can't find 'foo.cbar.c'.  This is pretty bewildering behavior
>     even if you *are* a Python programmer, and not everyone here is.
>
>     Note that in C, the implicit concatenation is more justified; there
>     is no other way to join strings without (at least) a function call.
>
>     In Python, strings are objects which support the __add__ operator;
>     it is possible to write:
>
>         "abc" + "def"
>
>     Because these are literals, this addition can still be optimized
>     away by the compiler.
>
>     Guido indicated [2] that this change should be handled by PEP, because
>     there were a few edge cases with other string operators, such as the %.
>     The resolution is to treat them the same as today.
>
>         ("abc %s def" + "ghi" % var)  # fails like today.
>                                       # raises TypeError because of
>                                       # precedence.  (% before +)
>
>         ("abc" + "def %s ghi" % var)  # works like today; precedence makes
>                                       # the optimization more difficult to
>                                       # recognize, but does not change the
>                                       # semantics.
>
>         ("abc %s def" + "ghi") % var  # works like today, because of
>                                       # precedence:  () before %
>                                       # CPython compiler can already
>                                       # add the literals at compile-time.
>
>
> Rationale for Removing Explicit Line Continuation
>
>     A terminal "\" indicates that the logical line is continued on the
>     following physical line (after whitespace).
>
>     Note that a non-terminal "\" does not have this meaning, even if the
>     only additional characters are invisible whitespace.  (Python depends
>     heavily on *visible* whitespace at the beginning of a line; it does
>     not otherwise depend on *invisible* terminal whitespace.)  Adding
>     whitespace after a "\" will typically cause a syntax error rather
>     than a silent bug, but it still isn't desirable.
>
>     The reason to keep "\" is that occasionally code looks better with
>     a "\" than with a () pair.
>
>         assert True, (
>             "This Paren is goofy")
>
>     But realistically, that paren is no worse than a "\".  The only
>     advantage of "\" is that it is slightly more familiar to users of
>     C-based languages.  These same languages all also support line
>     continuation with (), so reading code will not be a problem, and
>     there will be one less rule to learn for people entirely new to
>     programming.
>
>
> Rationale for Removing Implicit Octal Literals
>
>     This decision should be covered by PEP ???, on numeric literals.
>     It is mentioned here only for completeness.
>
>     C treats integers beginning with "0" as octal, rather than decimal.
>     Historically, Python has inherited this usage.  This has caused
>     quite a few annoying bugs for people who forgot the rule, and
>     tried to line up their constants.
>
>         a = 123
>         b = 024   # really only 20, because octal
>         c = 245
>
>     In Python 3.0, the second line will instead raise a SyntaxError,
>     because of the ambiguity.  Instead, the line should be written
>     as in one of the following ways:
>
>         b = 24    # PEP 8
>         b =  24   # columns line up, for quick scanning
>         b = 0t24  # really did want an Octal!
>
>
> References
>
>     [1] Implicit String Concatenation, Jewett, Orendorff
>         http://mail.python.org/pipermail/python-ideas/2007-April/000397.html
>
>     [2] PEP 12, Sample reStructuredText PEP Template, Goodger, Warsaw
>         http://www.python.org/peps/pep-0012
>
>     [3] http://www.opencontent.org/openpub/
>
>
>
> Copyright
>
>     This document has been placed in the public domain.
>
>
>
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> sentence-end-double-space: t
> fill-column: 70
> coding: utf-8
> End:
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Mon Apr 30 07:12:28 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 30 Apr 2007 07:12:28 +0200
Subject: [Python-3000] Draft PEP: Dropping PyObject_HEAD
In-Reply-To: <46353E13.5080501@canterbury.ac.nz>
References: <46327CC9.4040901@v.loewis.de> <f0u4nh$nqt$1@sea.gmane.org>
	<4632DD83.2030301@v.loewis.de> <4633F918.1010205@canterbury.ac.nz>
	<46346DD9.1020101@v.loewis.de> <46353E13.5080501@canterbury.ac.nz>
Message-ID: <46357ABC.10305@v.loewis.de>

>> That doesn't work. For variable-sized objects, it would need
>> to be ob_base.ob_base.ob_type.
> 
> I was wondering about that. It seems that each level
> of inheritance is going to add another level of
> required member access. Or is it legal to short-
> circuit all that and just cast directly to the
> ultimate base object?

As the PEP explains: you can cast a struct pointer to pointer
to the first field. That works recursively: if the first field
is a struct again, you can also cast to a pointer to that
struct's first field.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Mon Apr 30 08:27:23 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 30 Apr 2007 18:27:23 +1200
Subject: [Python-3000] Could isinstance/issubclass overriding be
	dangerous?
In-Reply-To: <ca471dc20704292105n423c5b0dg5938b3d6b12ffcfb@mail.gmail.com>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
	<46354AFD.5070702@canterbury.ac.nz>
	<ca471dc20704292105n423c5b0dg5938b3d6b12ffcfb@mail.gmail.com>
Message-ID: <46358C4B.2020803@canterbury.ac.nz>

Guido van Rossum wrote:

> Unless they were looking at classic classes, why wouldn't they be
> using the specific Py<Type>_Check() macros?

I'm thinking of Pyrex code. One of the goals of Pyrex
is that you should be able to write it without needing
to know about the Python/C API. One of the places that's
not possible at the moment is in binary operator methods,
where you need to write things like

   cdef class C:

     def __add(x, y):
       if PyObject_TypeCheck(x, C):
         # we're the left operand
       elif PyObject_TypeCheck(y, C):
         # we're the right operand

I'd like to be able to provide optimised access to
isinstance() so that you can think in Python instead of
C and write

     def __add(x, y):
       if isinstance(x, C):
         # we're the left operand
       elif isinstance(y, C):
         # we're the right operand

But it seems like isinstance() is already the wrong thing
to use for this, and there is currently *no* Python-level
function that does what is needed here.

So can we please have another couple of functions that just
do a simple, reliable concrete type test?

--
Greg

From tdelaney at avaya.com  Mon Apr 30 04:48:26 2007
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Mon, 30 Apr 2007 12:48:26 +1000
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1ED71@au3010avexu1.global.avaya.com>

Jim Jewett wrote:

> On 4/29/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
>> I've been intending to write up a PEP for fixing super, but I
>> haven't had time to get to it.
> 
> Calvin Spealman has the most recent draft. I hope he will incorporate
> this into his draft.

Sorry about this - wasn't receiving python-dev at home, so didn't
realise Calvin had released the PEP.

I think the current PEP draft is way too complicated - I don't think
there's any need for descriptors, etc. I think we can make things work
in the following way:

1. When a method is defined, the class is bound to it via an attribute
(which in my version is called func_class).

2. Every non-static method has an implicit cell variable called 'super'.
This would preferably not be able to be rebound. I also think it would
be beneficial if the first parameter to the method couldn't be rebound
(point 7 in my original email in this thread).

3. When a method is entered, the 'super' cell variable is populated by a
call equivalent to:

    super = __builtin__.super(func_class, first_parameter)

This would result in 'super' being a constant object, within the scope
of the currently-executing method. 'keyword' was perhaps too strong - I
was thinking this would only need to be done if 'super' were actually
used, which would be easier to determine if 'super' actually were a
keyword. This could still be done by not emitting the above call unless
the 'super' cell variable were ever actually used.

I've done bytecode-hacky stuff to do the equivalent of the above (as
much as I've been able to), but a real implementation would just emit
the correct bytecode (or java bytecode, or whatever) in the compiled
code object.

The issue of super() vs. super.__call__() ambiguity - I'll need to look
at that when I get home.

I'm a strong -1 against super() automatically passing the parameters
that were passed to the currently-executing method.

Tim Delaney

From tdelaney at avaya.com  Mon Apr 30 06:56:20 2007
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Mon, 30 Apr 2007 14:56:20 +1000
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>> 1. When a method is defined, the class is bound to it via an
attribute
>> (which in my version is called func_class).

> In Py3k all the func_XXX attrs are renamed __XXX__, so this would be
> __class__; but that's a name reserved for something else, so it would
> need to be something else.  E.g. __containing_class__.

Yep - I've just used a placeholder name.

> Also, this would have to be done when the class is defined; when the
> method is being defined the class doesn't exist yet.

Good point. Change that to "at the first opportunity" ;)

>> 2. Every non-static method has an implicit cell variable called
>> 'super'. 
> 
> I think you're using 'cell' in a different sense than it is normally
> used in Python's implementation. What you are looking for is called a
> local variable (I deduce this from your initialization of the "cell"
> with something computed from the first argument).

Actually, I think I'm using the terminology correctly - I'm talking
about an entry in co_cellvars. Given the following class:

    class A(object):
        def f(self):
            super = super_factory()

            def inner():
                return 'A' + super.f()

            print inner()

the use of 'super' in both A.f and A.f.inner will produce an entry in
A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm
proposing is that the `super = super_factory()` line be implicit in this
case, resulting in the following code behaving identically:

    class A(object):
        def f(self):
            def inner():
                return 'A' + super.f()

            print inner()

>> The issue of super() vs. super.__call__() ambiguity - I'll need to
>> look at that when I get home.
> 
> Sounds irrelevant -- if super is equivalent to
> __builtin__.__super__(<class>, <firstarg>) then super never gets
> called; the only thing ever done to it (in the normal course of
> things) is to request an attribute of it.

Sorry - this is related to my proposal that the following two bits of
code behave the same:

    class A(object):
        def f(self, *p, **kw):
            super.f(*p, **kw)

    class A(object):
        def f(self, *p, **kw):
            super(*p, **kw)

But as has been pointed out, this creates an ambiguity with:

    class A(object):
        def f(self, *p, **kw):
            super.__call__(*p, **kw)

so I want to see if I can resolve it.

> super(ThisClass).method(...)  # ???
> 
> The third exists so that you can create an "unbound" super instance
> which is useful for the oft-misunderstood autosuper example in my
> "descrintro" essay:
>
http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam
ples
> .
> 
> But since the latter is the hack that we're trying to replace with a
> proper implementation here, I suspect we can get away with only
> supporting the first two forms (and the third form is still supported
> using __builtin__.__super__).

Yep - that's my thought as well. I think it would be very rare to need
super(ThisClass), although it makes some sense that that would be what
super means in a static method ...

Tim Delaney

From tcdelaney at optusnet.com.au  Mon Apr 30 13:38:30 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Mon, 30 Apr 2007 21:38:30 +1000
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
Message-ID: <00d701c78b1c$143b3c40$0201a8c0@ryoko>

From: "Delaney, Timothy (Tim)" <tdelaney at avaya.com>

> Sorry - this is related to my proposal that the following two bits of
> code behave the same:
>
>    class A(object):
>        def f(self, *p, **kw):
>            super.f(*p, **kw)
>
>    class A(object):
>        def f(self, *p, **kw):
>            super(*p, **kw)
>
> But as has been pointed out, this creates an ambiguity with:
>
>    class A(object):
>        def f(self, *p, **kw):
>            super.__call__(*p, **kw)
>
> so I want to see if I can resolve it.

A 'super' instance would be callable, without being able to access it's 
__call__ method (because super.__call__ would refer to the base class method 
of that name).

But I find I really don't care. The only place where that would really 
matter IMO is if you want to find out if a 'super' instance is callable. 
Calling a base class __call__ method would not be ambiguous - the following 
two classes would work the same:

    class A(object):
        def __call__(self, *p, **kw):
            return super.__call__(*p, **kw)

    class A(object):
        def __call__(self, *p, **kw):
            return super(*p, **kw)

So, I guess my question is whether the most common case of calling the base 
class method with the same name is worth having some further syntactic sugar 
to avoid repetition? I think it is, but that would be your call Guido.

Cheers,

Tim Delaney 


From ironfroggy at gmail.com  Mon Apr 30 14:29:22 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Mon, 30 Apr 2007 08:29:22 -0400
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
Message-ID: <76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com>

On 4/30/07, Delaney, Timothy (Tim) <tdelaney at avaya.com> wrote:
> Guido van Rossum wrote:
>
> >> 1. When a method is defined, the class is bound to it via an
> attribute
> >> (which in my version is called func_class).
>
> > In Py3k all the func_XXX attrs are renamed __XXX__, so this would be
> > __class__; but that's a name reserved for something else, so it would
> > need to be something else.  E.g. __containing_class__.
>
> Yep - I've just used a placeholder name.
>
> > Also, this would have to be done when the class is defined; when the
> > method is being defined the class doesn't exist yet.
>
> Good point. Change that to "at the first opportunity" ;)
>
> >> 2. Every non-static method has an implicit cell variable called
> >> 'super'.
> >
> > I think you're using 'cell' in a different sense than it is normally
> > used in Python's implementation. What you are looking for is called a
> > local variable (I deduce this from your initialization of the "cell"
> > with something computed from the first argument).
>
> Actually, I think I'm using the terminology correctly - I'm talking
> about an entry in co_cellvars. Given the following class:
>
>     class A(object):
>         def f(self):
>             super = super_factory()
>
>             def inner():
>                 return 'A' + super.f()
>
>             print inner()
>
> the use of 'super' in both A.f and A.f.inner will produce an entry in
> A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm
> proposing is that the `super = super_factory()` line be implicit in this
> case, resulting in the following code behaving identically:
>
>     class A(object):
>         def f(self):
>             def inner():
>                 return 'A' + super.f()
>
>             print inner()

I believe the direction my PEP took with all this is a good bit
primitive compared to this approach, although I still find value in it
because at least a prototype came out of it that can be used to test
the waters, regardless of if a more direct-in-the-language approach
would be superior.

However, I seem to think that if the __this_class__ PEP goes through,
your version can be simplified as well. No tricky stuffy things in
cells would be needed, but we can just expand the super 'keyword' to
__super__(__this_class__, self), which has been suggested at least
once. It seems this would be much simpler to implement, and it also
brings up a second point.

Also, I like that the super object is created at the beginning of the
function, which my proposal couldn't even do. It is more efficient if
you have multiple super calls, and gets around a problem I completely
missed: what happens if the instance name were rebound before the
implicit lookup of the instance object at the time of the super call?

> >> The issue of super() vs. super.__call__() ambiguity - I'll need to
> >> look at that when I get home.
> >
> > Sounds irrelevant -- if super is equivalent to
> > __builtin__.__super__(<class>, <firstarg>) then super never gets
> > called; the only thing ever done to it (in the normal course of
> > things) is to request an attribute of it.
>
> Sorry - this is related to my proposal that the following two bits of
> code behave the same:
>
>     class A(object):
>         def f(self, *p, **kw):
>             super.f(*p, **kw)
>
>     class A(object):
>         def f(self, *p, **kw):
>             super(*p, **kw)
>
> But as has been pointed out, this creates an ambiguity with:
>
>     class A(object):
>         def f(self, *p, **kw):
>             super.__call__(*p, **kw)
>
> so I want to see if I can resolve it.

Turns out, it doesn't. A lot of people expect it, but it turns out to
be simple. super(*p, **kw) is not equivalent to super.__call__(*p,
**kw) but to type(super).__call__.__get__(super, type(super))(*p,
**kw), due to operator methods being looked up on the type directly
and bound by the descriptor, without any lookup on the super object
itself. Thus, no attribute is ever done on the super object itself,
type(super).__getattribute__ is never invoked, and there is no problem
at all.

This also means we can still invoke super the old, explicit way.

I've already got my copy of the PEP updated to reflect this.

> > super(ThisClass).method(...)  # ???
> >
> > The third exists so that you can create an "unbound" super instance
> > which is useful for the oft-misunderstood autosuper example in my
> > "descrintro" essay:
> >
> http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam
> ples
> > .
> >
> > But since the latter is the hack that we're trying to replace with a
> > proper implementation here, I suspect we can get away with only
> > supporting the first two forms (and the third form is still supported
> > using __builtin__.__super__).
>
> Yep - that's my thought as well. I think it would be very rare to need
> super(ThisClass), although it makes some sense that that would be what
> super means in a static method ...

Does super mean anything in a static method today?

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From rrr at ronadam.com  Mon Apr 30 13:58:58 2007
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 30 Apr 2007 06:58:58 -0500
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
Message-ID: <4635DA02.2060802@ronadam.com>

Delaney, Timothy (Tim) wrote:

> What I'm proposing is that the `super = super_factory()` line be
> implicit in this case, resulting in the following code behaving
> identically:
> 
>     class A(object):
>         def f(self):
>             def inner():
>                 return 'A' + super.f()
> 
>             print inner()


As Guido pointed out it has some resemblance to how import works, but I
also think there is resemblance to the context of how global is used.  So
if it is made into a keyword, could it work like the global keyword?

    class A(object):
        def f(self):
            def inner():
            	super f
            	return 'A' + f()

            print inner()

Cheers,
    Ron



From rasky at develer.com  Mon Apr 30 14:32:22 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Mon, 30 Apr 2007 14:32:22 +0200
Subject: [Python-3000] PEP 30XZ: Simplified Parsing
In-Reply-To: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
References: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
Message-ID: <f14nkm$3gt$1@sea.gmane.org>

(-CC python-dev)

On 30/04/2007 5.29, Jim Jewett wrote:

> Rationale for Removing Explicit Line Continuation
> 
>     A terminal "\" indicates that the logical line is continued on the
>     following physical line (after whitespace).
> 
>     Note that a non-terminal "\" does not have this meaning, even if the
>     only additional characters are invisible whitespace.  (Python depends
>     heavily on *visible* whitespace at the beginning of a line; it does
>     not otherwise depend on *invisible* terminal whitespace.)  Adding
>     whitespace after a "\" will typically cause a syntax error rather
>     than a silent bug, but it still isn't desirable.
> 
>     The reason to keep "\" is that occasionally code looks better with
>     a "\" than with a () pair.
> 
>         assert True, (
>             "This Paren is goofy")
> 
>     But realistically, that paren is no worse than a "\".  The only
>     advantage of "\" is that it is slightly more familiar to users of
>     C-based languages.  These same languages all also support line
>     continuation with (), so reading code will not be a problem, and
>     there will be one less rule to learn for people entirely new to
>     programming.

I was in favor of one of the alternatives that were proposed here: line 
continuation through indentation:

    a = 123 *
        (12 + 4) / 8

    assert True,
         "No goofy paren" +
         ", dude"

... and don't tell me that you need a character there because using just 
indentation is hard to read :)

Even if you don't want to champion this solution in your PEP, you should at 
least list it among the alternatives.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com


From rasky at develer.com  Mon Apr 30 14:43:26 2007
From: rasky at develer.com (Giovanni Bajo)
Date: Mon, 30 Apr 2007 14:43:26 +0200
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
Message-ID: <f14o9f$5o5$1@sea.gmane.org>

On 30/04/2007 3.40, Collin Winter wrote:

> The mechanism I'm most familiar with for solving this problem (which,
> unless I've missed something, is, "how do I make sure this object does
> what I expect?") is Perl 6's roles system; if you know about Squeak
> Smalltalk's "traits" system, you're on the same page.

+1. I like those, *specifically* the fact that they can be dynamic (you can 
"add" a role to a class or an instance of a class and "remove" it later). This 
is basically mixins they way they should be done.

> The idea is that we leave the traditional class system to manage an
> object's implementation and use a more-or-less orthogonal system to
> manage that same object's behavior.

Yes. In other words, traditional inheritance is very good for *reusing* an 
implementation, but it's not very flexible for defining behaviour. It works 
for basic cases of course, but gets limited fast when your object network gets 
more complex.

> 2. I gave a strawman syntax proposal for runtime role composition
> above (modulo the method name): "Ring.done_by(int)" or
> "int.does(Ring)". (Perl 6 speaks in terms of "an object/class *does* a
> role".) Perl 6 also allows individual instances to do different roles
> than their class does, but I'm not sure we want to go that far.

Well, I do! I know there have been times I have needed it badly, and when 
you're there there's little you can do. If you want to champion a "role" 
proposal for Python, please don't forget about this.
-- 
Giovanni Bajo


From tcdelaney at optusnet.com.au  Mon Apr 30 14:47:41 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Mon, 30 Apr 2007 22:47:41 +1000
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
	<76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com>
Message-ID: <00e701c78b25$be456c20$0201a8c0@ryoko>

From: "Calvin Spealman" <ironfroggy at gmail.com>

> I believe the direction my PEP took with all this is a good bit
> primitive compared to this approach, although I still find value in it
> because at least a prototype came out of it that can be used to test
> the waters, regardless of if a more direct-in-the-language approach
> would be superior.

I've been working on improved super syntax for quite a while now - my 
original approach was 'self.super' which used _getframe() and mro crawling 
too. I hit on using bytecode hacking to instantiate a super object at the 
start of the method to gain performance, which required storing the class in 
co_consts, etc. It turns out that using a metaclass then makes this a lot 
cleaner.

> However, I seem to think that if the __this_class__ PEP goes through,
> your version can be simplified as well. No tricky stuffy things in
> cells would be needed, but we can just expand the super 'keyword' to
> __super__(__this_class__, self), which has been suggested at least
> once. It seems this would be much simpler to implement, and it also
> brings up a second point.
>
> Also, I like that the super object is created at the beginning of the
> function, which my proposal couldn't even do. It is more efficient if
> you have multiple super calls, and gets around a problem I completely
> missed: what happens if the instance name were rebound before the
> implicit lookup of the instance object at the time of the super call?

You could expand it inline, but I think your second point is a strong 
argument against it. Also, sticking the super instance into a cell means 
that inner classes get access to it for free. Otherwise each inner class 
would *also* need to instantiate a super instance, and __this_class__ (or 
whatever it's called) would need to be in a cell for them to get access to 
it instead.

BTW, one of my test cases involves multiple super calls in the same method - 
there is a *very* large performance improvement by instantiating it once.

>> I think it would be very rare to need
>> super(ThisClass), although it makes some sense that that would be what
>> super means in a static method ...
>
> Does super mean anything in a static method today?

Well, since all super instantiations are explicit currently, it can mean 
whatever you want it to.

    class A(object):

        @staticmethod
        def f():
            print super(A)
            print super(A, A)

Cheers,

Tim Delaney 


From eric+python-dev at trueblade.com  Mon Apr 30 14:39:51 2007
From: eric+python-dev at trueblade.com (Eric V. Smith)
Date: Mon, 30 Apr 2007 08:39:51 -0400
Subject: [Python-3000] PEP 30XZ: Simplified Parsing
In-Reply-To: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
References: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
Message-ID: <4635E397.9000403@trueblade.com>

Jim Jewett wrote:

> Rationale for Removing Implicit String Concatenation
> 
>     Implicit String concatentation can lead to confusing, or even
>     silent, errors. [1]
> 
>         def f(arg1, arg2=None): pass
> 
>         f("abc" "def")  # forgot the comma, no warning ...
>                         # silently becomes f("abcdef", None)
> 
>     or, using the scons build framework,
> 
>         sourceFiles = [
>         'foo.c',
>         'bar.c',
>         #...many lines omitted...
>         'q1000x.c']
> 

Since your first example omits the comma, I think this one should, too.

          sourceFiles = [
          'foo.c'
          'bar.c',
          #...many lines omitted...
          'q1000x.c']

That is, either both examples should show an error, or both examples 
should work, but point out how easy it is to make an error.

Eric.



From ironfroggy at gmail.com  Mon Apr 30 15:09:35 2007
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Mon, 30 Apr 2007 09:09:35 -0400
Subject: [Python-3000] [Python-Dev] Pre-pre PEP for 'super' keyword
In-Reply-To: <00e701c78b25$be456c20$0201a8c0@ryoko>
References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
	<76fd5acf0704300529r1801bc6at9a2f3d5329d1ea0d@mail.gmail.com>
	<00e701c78b25$be456c20$0201a8c0@ryoko>
Message-ID: <76fd5acf0704300609o4285b9d6p4895a891420aa88d@mail.gmail.com>

On 4/30/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
> I've been working on improved super syntax for quite a while now - my
> original approach was 'self.super' which used _getframe() and mro crawling
> too. I hit on using bytecode hacking to instantiate a super object at the
> start of the method to gain performance, which required storing the class in
> co_consts, etc. It turns out that using a metaclass then makes this a lot
> cleaner.
>
> > However, I seem to think that if the __this_class__ PEP goes through,
> > your version can be simplified as well. No tricky stuffy things in
> > cells would be needed, but we can just expand the super 'keyword' to
> > __super__(__this_class__, self), which has been suggested at least
> > once. It seems this would be much simpler to implement, and it also
> > brings up a second point.
> >
> > Also, I like that the super object is created at the beginning of the
> > function, which my proposal couldn't even do. It is more efficient if
> > you have multiple super calls, and gets around a problem I completely
> > missed: what happens if the instance name were rebound before the
> > implicit lookup of the instance object at the time of the super call?
>
> You could expand it inline, but I think your second point is a strong
> argument against it. Also, sticking the super instance into a cell means
> that inner classes get access to it for free. Otherwise each inner class
> would *also* need to instantiate a super instance, and __this_class__ (or
> whatever it's called) would need to be in a cell for them to get access to
> it instead.
>
> BTW, one of my test cases involves multiple super calls in the same method -
> there is a *very* large performance improvement by instantiating it once.

Note that I would now advocate the creation of the super object at the
beginning of function, and when I talk about expanding super to
super(__this_class__, self) I do mean doing it at the beginning of the
function, just like you are saying, so we're in agreement here.

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From jimjjewett at gmail.com  Mon Apr 30 16:49:06 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 30 Apr 2007 10:49:06 -0400
Subject: [Python-3000] octal literals PEP
Message-ID: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>

On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> I think these should be two separate proposals, with more specific
> names (e.g. "remove implicit string concatenation" and "remove
> backslash continuation"). There's no need to mention the octal thing
> if it's already a separate PEP.

Patrick

Guido had set an Apr 30 deadline for Py3000 PEPs that can't be
implemented in pure python.

Are you still working on the "Integer literal syntax and radices ",
which included the octal literal?  I would much prefer to leave octal
literals with the rest of that PEP, (and to let you do it :D),  but I
will submit a much-simplified "023 raises SyntaxError" if you have
abandoned the rest.

-jJ

From pje at telecommunity.com  Mon Apr 30 17:16:34 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 30 Apr 2007 11:16:34 -0400
Subject: [Python-3000] Could isinstance/issubclass overriding be
 dangerous?
In-Reply-To: <46354AFD.5070702@canterbury.ac.nz>
References: <d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
	<5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070430111601.02e5e6c0@sparrow.telecommunity.com>

At 01:48 PM 4/30/2007 +1200, Greg Ewing wrote:
>Currently, isinstance(x, C) implies that the
>C struct layout of x is compatible with that
>defined by C.

No, it doesn't, and hasn't since Python 2.3.


>  I'm worried that changing this
>could cause difficulties for extension modules
>that use PyObject_IsInstance do decide what
>to do with things, e.g. in a binary operator
>method.
>
>--
>Greg
>_______________________________________________
>Python-3000 mailing list
>Python-3000 at python.org
>http://mail.python.org/mailman/listinfo/python-3000
>Unsubscribe: 
>http://mail.python.org/mailman/options/python-3000/pje%40telecommunity.com


From alan.mcintyre at gmail.com  Mon Apr 30 17:16:06 2007
From: alan.mcintyre at gmail.com (Alan McIntyre)
Date: Mon, 30 Apr 2007 11:16:06 -0400
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
Message-ID: <1d36917a0704300816ma3bf9c2o4dd674cfcefa9172@mail.gmail.com>

On 4/30/07, Raymond Hettinger <python at rcn.com> wrote:
> I'm concerned that the current ABC proposal will quickly evolve from optional
> to required and create somewhat somewhat java-esque landscape where
> inheritance and full-specification are the order of the day.

+1 for preferring simple solutions to complex ones

+inf for not turning Python into Java or C++ <0.5 wink>

From pje at telecommunity.com  Mon Apr 30 17:30:17 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 30 Apr 2007 11:30:17 -0400
Subject: [Python-3000] Could isinstance/issubclass overriding be
 dangerous?
In-Reply-To: <46358C4B.2020803@canterbury.ac.nz>
References: <ca471dc20704292105n423c5b0dg5938b3d6b12ffcfb@mail.gmail.com>
	<5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<d11dcfba0704291719n27527c25w5f2ac07e14ea00e3@mail.gmail.com>
	<46354AFD.5070702@canterbury.ac.nz>
	<ca471dc20704292105n423c5b0dg5938b3d6b12ffcfb@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070430112626.04ef35d8@sparrow.telecommunity.com>

At 06:27 PM 4/30/2007 +1200, Greg Ewing wrote:
>So can we please have another couple of functions that just
>do a simple, reliable concrete type test?

I believe the operation you're looking for (i.e., C-level layout 
compatibility) is:

C in type.__mro__.__get__(type(x))

or in Pyrex:

C in (((PyTypeObject *) type(x)).tp_mro)




From janssen at parc.com  Mon Apr 30 18:01:09 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 30 Apr 2007 09:01:09 PDT
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <ca471dc20704291826p6cb8e001gf3df7a3dc6160667@mail.gmail.com> 
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<ca471dc20704291826p6cb8e001gf3df7a3dc6160667@mail.gmail.com>
Message-ID: <07Apr30.090110pdt."57996"@synergy1.parc.xerox.com>

GvR wrote:
> So how about we reduce the scope of our (!) PEP (or perhaps of a new
> one) to two items: (a) add @abstractmethod, and (b) overload
> isinstance() and issubclass()? Library authors can do everything they
> want with those, and we can always add a specific set of ABCs for
> containers and/or numbers later in the 3.0 development cycle.

-1.  Adding mechanism without content seems less than ideal, despite
Talin's misgivings.  I'd recommend adding the base classes in, and see
how they work in the earliest 3.0 releases, then modify them as
necessary in subsequent releases.

Bill

From janssen at parc.com  Mon Apr 30 18:08:40 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 30 Apr 2007 09:08:40 PDT
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <014201c78adc$ca70d960$f101a8c0@RaymondLaptop1> 
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
Message-ID: <07Apr30.090845pdt."57996"@synergy1.parc.xerox.com>

> [Collin Winter]
> +100 I'm very interested in seeing a lighter weight alternative to abc.py that:
> 
> 1) is dynamic

There's no reason ABC's can't be dynamic.

> 2) doesn't require inheritance to work
> 3) doesn't require mucking with isinstance or other existing mechansims

Using existing mechanisms that work for the issue is always preferable
to adding new ones that *also* work for the issue.

> 4) makes a limited, useful set of assertions rather than broadly covering a whole API.

That's what an API is, isn't it?

> 5) that leaves the notion of duck-typing as the rule rather than the exception

Here we disagree.

> 6) that doesn't freeze all of the key APIs in concrete

After 15 years of experience with the key APIs, we could perhaps freeze some of them?

> IMHO, the ABC approach is using a cannon to shoot a mosquito.  My day-to-day
> problems are much smaller are could be solved by a metadata attribute or a
> role/trait solution:

And then you cite a few symptoms of not having an API-based system:

> * knowing whether a __getitem__ method implements a mapping or a sequence

In other words, what APIs does this value support?

> * knowing whether an object can have more that one iterator (i.e a file has one
>    but a list can have many)

In other words, what APIs does this value support?

> * knowing whether a sequence, file, cursor, etc is writable or just readonly.

In other words, what APIs does this value support?

If we simply had a small standard extensible set of APIs, and a way to
indicate that a value does or doesn't support one or more of them,
we'd be done.  We could either add a new mechanism to do this, or simply
actually use the one we already have, which should work perfectly well for
this purpose.

Bill

From janssen at parc.com  Mon Apr 30 18:11:24 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 30 Apr 2007 09:11:24 PDT
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <1d36917a0704300816ma3bf9c2o4dd674cfcefa9172@mail.gmail.com> 
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<1d36917a0704300816ma3bf9c2o4dd674cfcefa9172@mail.gmail.com>
Message-ID: <07Apr30.091130pdt."57996"@synergy1.parc.xerox.com>

> On 4/30/07, Raymond Hettinger <python at rcn.com> wrote:
> > I'm concerned that the current ABC proposal will quickly evolve from optional
> > to required and create somewhat somewhat java-esque landscape where
> > inheritance and full-specification are the order of the day.
> 
> +1 for preferring simple solutions to complex ones

Me, too.  But which is the simple solution?  I tend to think ABCs are.

> +inf for not turning Python into Java or C++ <0.5 wink>

Not into the bad parts of either, to be sure.

Bill

From talin at acm.org  Mon Apr 30 18:27:29 2007
From: talin at acm.org (Talin)
Date: Mon, 30 Apr 2007 09:27:29 -0700
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
Message-ID: <463618F1.6070309@acm.org>

Collin Winter wrote:
> The key part of traits/roles is that, because the system is separate
> from classes, you can do runtime role composition without a) mucking
> with __bases__, or b) making isinstance() and issubclass() squishy and
> ill-defined. By "runtime role composition", I mean it would be
> possible to do something like this at runtime:
> 
> py> int.implements(Ring)
> 
> or (depending on your spelling preferences)
> 
> py> Ring.implemented_by(int)
> 
> That is, it would be possible to distribute Jeffery's numeric kinds as
> a third-party library and still have them affect the built-in numeric
> types.

The question I have about all this runtime composition stuff is, what 
happens if different modules have different ideas about how things 
should be composed? How are conflicts resolved?

With ABCs, we have a limitation that is also a strength: The mapping 
between ABC and its bases is either built-in to the ABC or built-in to 
the class, meaning that these decisions have to all be made up front - 
which forces the ABC implementer to pre-decide the outcome of all such 
conflicts before they happen.

-- Talin

From collinw at gmail.com  Mon Apr 30 18:50:09 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 30 Apr 2007 09:50:09 -0700
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <-3604285971386182607@unknownmsgid>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<-3604285971386182607@unknownmsgid>
Message-ID: <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com>

On 4/30/07, Bill Janssen <janssen at parc.com> wrote:
> > [Collin Winter]
> > +100 I'm very interested in seeing a lighter weight alternative to abc.py that:
> >
> > 1) is dynamic
>
> There's no reason ABC's can't be dynamic.
>
> > 2) doesn't require inheritance to work
> > 3) doesn't require mucking with isinstance or other existing mechansims
>
> Using existing mechanisms that work for the issue is always preferable
> to adding new ones that *also* work for the issue.

No existing mechanism would allow PEP 3141 to be distributed as a
third-party module and still impact the built-in types; that's what
this whole business of __isinstance__ and __issubclass__ is about. In
order to achieve dynamic composition, we're having to monkey with
fundamental, bedrock properties of inheritance-based object
orientation. The definition of "subclass" and "instance" is no longer
objective; the answer to "is X an instance of Y?" becomes, "I dunno,
ask Y".

Collin Winter

From python3now at gmail.com  Mon Apr 30 18:53:16 2007
From: python3now at gmail.com (James Thiele)
Date: Mon, 30 Apr 2007 09:53:16 -0700
Subject: [Python-3000] Addition to PEP 3101
Message-ID: <8f01efd00704300953t6154d7e1j7ef18cead1acb344@mail.gmail.com>

For PEP: 3101  - Advanced String Formatting - I would like to suggest
the following additions.

First, that an additional pair of forms be added to format strings so
that callable objects could be used as parameters to the format string
method:
{0()} for positional arguments and
{keyword()} for keyword parameters.
The proposal is that if the format string had this form that the
format string method would treat the parameter as a callable and use
the value of the return for the formatting.

The second addition would be the forms
{0.method()} and
{keyword.method()}
where "method" is a method of the object.

This would allow things such as:
"I want to shout '{0.upper()}'".format("hello world")
which would yield:
 "I want to shout 'HELLO WORLD'"

From p.f.moore at gmail.com  Mon Apr 30 19:01:19 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 30 Apr 2007 18:01:19 +0100
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <438708814690534630@unknownmsgid>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<438708814690534630@unknownmsgid>
Message-ID: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com>

On 30/04/07, Bill Janssen <janssen at parc.com> wrote:
> > 6) that doesn't freeze all of the key APIs in concrete
>
> After 15 years of experience with the key APIs, we could perhaps freeze some of
> them?

After 15 years not being able to clearly state what "file-like" or
"mapping-like" means to different people, perhaps we should accept
that there is no clear-cut answer...?

> In other words, what APIs does this value support?
> In other words, what APIs does this value support?
> In other words, what APIs does this value support?

Yes, but the ABC proposal is about establishing a vocabulary for
answering those questions, not about asking them. It's not at all
clear (to me, at least) that there is agreement that the vocabulary
proposed is helpful.

> If we simply had a small standard extensible set of APIs, and a way to
> indicate that a value does or doesn't support one or more of them,
> we'd be done.  We could either add a new mechanism to do this, or simply
> actually use the one we already have, which should work perfectly well for
> this purpose.

And which has been around for many years, but has never actually been
used in practice (unlike, say, interfaces). Does this not say
something?

Paul.

From collinw at gmail.com  Mon Apr 30 19:14:34 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 30 Apr 2007 10:14:34 -0700
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <463618F1.6070309@acm.org>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<463618F1.6070309@acm.org>
Message-ID: <43aa6ff70704301014y3dfbac77yaa42a07a8dd57499@mail.gmail.com>

On 4/30/07, Talin <talin at acm.org> wrote:
> Collin Winter wrote:
> > The key part of traits/roles is that, because the system is separate
> > from classes, you can do runtime role composition without a) mucking
> > with __bases__, or b) making isinstance() and issubclass() squishy and
> > ill-defined. By "runtime role composition", I mean it would be
> > possible to do something like this at runtime:
> >
> > py> int.implements(Ring)
> >
> > or (depending on your spelling preferences)
> >
> > py> Ring.implemented_by(int)
> >
> > That is, it would be possible to distribute Jeffery's numeric kinds as
> > a third-party library and still have them affect the built-in numeric
> > types.
>
> The question I have about all this runtime composition stuff is, what
> happens if different modules have different ideas about how things
> should be composed? How are conflicts resolved?
>
> With ABCs, we have a limitation that is also a strength: The mapping
> between ABC and its bases is either built-in to the ABC or built-in to
> the class, meaning that these decisions have to all be made up front -
> which forces the ABC implementer to pre-decide the outcome of all such
> conflicts before they happen.

What do you mean by "different ideas about how things should be
composed"? Do you mean the case where two modules say the same class
should do different roles? Roles are purely additive, so if module X
says that class Z does one role, and module Y says class Z does
another role, there's no conflict: the class does both roles
simultaneously.

Things get interesting when you try to compose roles that both offer
default implementations of the same method. The standard response (and
the one chosen by Perl 6) is to just throw up your hands and shriek
wildly and let the programmer sort it out.
http://www.perlmonks.org/?node_id=384909 has a good example that
addresses this exact question.

Collin Winter

From guido at python.org  Mon Apr 30 19:19:52 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Apr 2007 10:19:52 -0700
Subject: [Python-3000] octal literals PEP
In-Reply-To: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
References: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
Message-ID: <ca471dc20704301019x56ae6debn586ea9a944925b73@mail.gmail.com>

On 4/30/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> > I think these should be two separate proposals, with more specific
> > names (e.g. "remove implicit string concatenation" and "remove
> > backslash continuation"). There's no need to mention the octal thing
> > if it's already a separate PEP.
>
> Patrick
>
> Guido had set an Apr 30 deadline for Py3000 PEPs that can't be
> implemented in pure python.

I don't recall limiting the deadline in that way. Is this a general
understanding?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Apr 30 19:30:28 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Apr 2007 10:30:28 -0700
Subject: [Python-3000] PEP 30XZ: Simplified Parsing
In-Reply-To: <f14nkm$3gt$1@sea.gmane.org>
References: <fb6fbf560704292029g118592bpb19a87a2ee5ea312@mail.gmail.com>
	<f14nkm$3gt$1@sea.gmane.org>
Message-ID: <ca471dc20704301030v63b318bbrfefd2a9ced0f5bf4@mail.gmail.com>

On 4/30/07, Giovanni Bajo <rasky at develer.com> wrote:
> I was in favor of one of the alternatives that were proposed here:

Was? Are you no longer in favor of it? (Then I should shut up. :-)

> line continuation through indentation:
>
>     a = 123 *
>         (12 + 4) / 8
>
>     assert True,
>          "No goofy paren" +
>          ", dude"
>
> ... and don't tell me that you need a character there because using just
> indentation is hard to read :)
>
> Even if you don't want to champion this solution in your PEP, you should at
> least list it among the alternatives.

Then please add my -1 to it: I don't think this is feasible. It would
require allowing matching INDENT and DEDENT tokens *anywhere* in the
grammar; *or* you would have to limit where in the grammar you allow
the indent to occur (e.g. after '*' but not before it; after the ','
but not before it). It seems a really big departure from the current
Python parsing methodology.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Mon Apr 30 19:38:14 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 30 Apr 2007 11:38:14 -0600
Subject: [Python-3000] PEP 31XX: A Type Hierarchy for Numbers (and other
	algebraic entities)
In-Reply-To: <-5382129169035830252@unknownmsgid>
References: <5d44f72f0704250136l6683b7ffv3585fb00d86861aa@mail.gmail.com>
	<43aa6ff70704250954o6147a56br7f05a6af7ae6056e@mail.gmail.com>
	<ca471dc20704251119vaaccf3ao981fac7660b19cb5@mail.gmail.com>
	<f10g7k$fag$1@sea.gmane.org>
	<5d44f72f0704291025n2543cc47v6bb6d8f1f9e716b6@mail.gmail.com>
	<ca471dc20704291546t7e9db466ha50f713e1004de0e@mail.gmail.com>
	<46353440.5060402@acm.org>
	<ca471dc20704291826p6cb8e001gf3df7a3dc6160667@mail.gmail.com>
	<-5382129169035830252@unknownmsgid>
Message-ID: <d11dcfba0704301038h57a07a20g2a60cf7f336c63d4@mail.gmail.com>

On 4/30/07, Bill Janssen <janssen at parc.com> wrote:
> GvR wrote:
> > So how about we reduce the scope of our (!) PEP (or perhaps of a new
> > one) to two items: (a) add @abstractmethod, and (b) overload
> > isinstance() and issubclass()? Library authors can do everything they
> > want with those, and we can always add a specific set of ABCs for
> > containers and/or numbers later in the 3.0 development cycle.
>
> -1.  Adding mechanism without content seems less than ideal, despite
> Talin's misgivings.  I'd recommend adding the base classes in, and see
> how they work in the earliest 3.0 releases, then modify them as
> necessary in subsequent releases.

+1 for Guido's simplified PEP. There are clearly use cases for
@abstractmethod, __isinstance__() and __issubclass__(), so I see no
reason why the simplified PEP shouldn't be acceptable on its own.
Complaining about "adding mechanism without content" seems like
saying, for example, that we shouldn't have introduced
"functools.partial" without *using* it somewhere in the standard
library. Whether the use cases are in the standard library or
somewhere else should not determine whether a PEP is acceptable or
not.

I personally think the simplified PEP is a great compromise --
proponents of ABCs will now be able to introduce them (even for
existing types like 'int' and 'list') and folks that don't want ABCs
can simply never install the ABC module. Plus, it's much easier to
*add* a module to the standard library than it is to *remove* one
(even given Python 3.0's somewhat laxer first release plans).

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Mon Apr 30 19:43:18 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Apr 2007 10:43:18 -0700
Subject: [Python-3000] [Python-Dev]  Pre-pre PEP for 'super' keyword
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F074481@au3010avexu1.global.avaya.com>
Message-ID: <ca471dc20704301043x6009c244t4171fa3b87fc236b@mail.gmail.com>

On 4/29/07, Delaney, Timothy (Tim) <tdelaney at avaya.com> wrote:
> Guido van Rossum wrote:
> >> 2. Every non-static method has an implicit cell variable called
> >> 'super'.
> >
> > I think you're using 'cell' in a different sense than it is normally
> > used in Python's implementation. What you are looking for is called a
> > local variable (I deduce this from your initialization of the "cell"
> > with something computed from the first argument).
>
> Actually, I think I'm using the terminology correctly - I'm talking
> about an entry in co_cellvars. Given the following class:
>
>     class A(object):
>         def f(self):
>             super = super_factory()
>
>             def inner():
>                 return 'A' + super.f()
>
>             print inner()
>
> the use of 'super' in both A.f and A.f.inner will produce an entry in
> A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm
> proposing is that the `super = super_factory()` line be implicit in this
> case, resulting in the following code behaving identically:
>
>     class A(object):
>         def f(self):
>             def inner():
>                 return 'A' + super.f()
>
>             print inner()

OK, I see now. I thought you meant for the cell-ness to be related to
the basic implementation; but you meant it so that it woks correctly
inside nested functions. That's fine of course. You might clarify this
somewhere.

> >> The issue of super() vs. super.__call__() ambiguity - I'll need to
> >> look at that when I get home.
> >
> > Sounds irrelevant -- if super is equivalent to
> > __builtin__.__super__(<class>, <firstarg>) then super never gets
> > called; the only thing ever done to it (in the normal course of
> > things) is to request an attribute of it.
>
> Sorry - this is related to my proposal that the following two bits of
> code behave the same:
>
>     class A(object):
>         def f(self, *p, **kw):
>             super.f(*p, **kw)
>
>     class A(object):
>         def f(self, *p, **kw):
>             super(*p, **kw)
>
> But as has been pointed out, this creates an ambiguity with:
>
>     class A(object):
>         def f(self, *p, **kw):
>             super.__call__(*p, **kw)
>
> so I want to see if I can resolve it.

I think the shortcut is more confusing than helpful; I propose to drop
it for the sake of focusing on the essential.

> > super(ThisClass).method(...)  # ???
> >
> > The third exists so that you can create an "unbound" super instance
> > which is useful for the oft-misunderstood autosuper example in my
> > "descrintro" essay:
> >
> > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exampples
> > .
> >
> > But since the latter is the hack that we're trying to replace with a
> > proper implementation here, I suspect we can get away with only
> > supporting the first two forms (and the third form is still supported
> > using __builtin__.__super__).
>
> Yep - that's my thought as well. I think it would be very rare to need
> super(ThisClass), although it makes some sense that that would be what
> super means in a static method ...

But that doesn't work with the current (2.x) super, and hasn't been
requested AFAIK, so I'd suggest dropping the use case -- KISS again.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Mon Apr 30 19:46:31 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 30 Apr 2007 11:46:31 -0600
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<-3604285971386182607@unknownmsgid>
	<43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com>
Message-ID: <d11dcfba0704301046g663cb8f8i57824b2d3041383d@mail.gmail.com>

On 4/30/07, Collin Winter <collinw at gmail.com> wrote:
> No existing mechanism would allow PEP 3141 to be distributed as a
> third-party module and still impact the built-in types; that's what
> this whole business of __isinstance__ and __issubclass__ is about. In
> order to achieve dynamic composition, we're having to monkey with
> fundamental, bedrock properties of inheritance-based object
> orientation. The definition of "subclass" and "instance" is no longer
> objective; the answer to "is X an instance of Y?" becomes, "I dunno,
> ask Y".

I guess I don't understand why this bothers you more than how you can
currently mess with __class__::

    >>> class X(object):
    ...     __class__ = property(lambda self: Y)
    ...
    >>> class Y(object):
    ...     pass
    ...
    >>> isinstance(X(), Y)
    True

The X class does *not* inherit from the Y class, yet it tells us that
it does. To me, that certainly looks like "monkey[ing] with
fundamental, bedrock properties of inheritance-based object
orientation".

Maybe you could give a couple realistic examples of where you think
people using __isinstance__ and __issubclass__ are going to break
things unintentionally?

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From steven.bethard at gmail.com  Mon Apr 30 19:49:34 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 30 Apr 2007 11:49:34 -0600
Subject: [Python-3000] octal literals PEP
In-Reply-To: <ca471dc20704301019x56ae6debn586ea9a944925b73@mail.gmail.com>
References: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
	<ca471dc20704301019x56ae6debn586ea9a944925b73@mail.gmail.com>
Message-ID: <d11dcfba0704301049t647854d9n52e474a391992cf3@mail.gmail.com>

On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> On 4/30/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be
> > implemented in pure python.
>
> I don't recall limiting the deadline in that way. Is this a general
> understanding?

FWIW, my understanding was only standard library changes would be
allowed after Apr 30. I don't recall hearing anything about "pure
python" vs. otherwise, but I guess "pure python" corresponds to
"standard library"...?

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Mon Apr 30 19:52:49 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Apr 2007 10:52:49 -0700
Subject: [Python-3000] octal literals PEP
In-Reply-To: <d11dcfba0704301049t647854d9n52e474a391992cf3@mail.gmail.com>
References: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
	<ca471dc20704301019x56ae6debn586ea9a944925b73@mail.gmail.com>
	<d11dcfba0704301049t647854d9n52e474a391992cf3@mail.gmail.com>
Message-ID: <ca471dc20704301052r29f7774dvd797a39bd3e5f877@mail.gmail.com>

On 4/30/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/30/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be
> > > implemented in pure python.
> >
> > I don't recall limiting the deadline in that way. Is this a general
> > understanding?
>
> FWIW, my understanding was only standard library changes would be
> allowed after Apr 30. I don't recall hearing anything about "pure
> python" vs. otherwise, but I guess "pure python" corresponds to
> "standard library"...?

Not at all. There's a whole slew of C extensions that are considered
part of the standard library.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From collinw at gmail.com  Mon Apr 30 20:01:58 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 30 Apr 2007 11:01:58 -0700
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <d11dcfba0704301046g663cb8f8i57824b2d3041383d@mail.gmail.com>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<-3604285971386182607@unknownmsgid>
	<43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com>
	<d11dcfba0704301046g663cb8f8i57824b2d3041383d@mail.gmail.com>
Message-ID: <43aa6ff70704301101w7707fd8m735442cda9d14eb9@mail.gmail.com>

On 4/30/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/30/07, Collin Winter <collinw at gmail.com> wrote:
> > No existing mechanism would allow PEP 3141 to be distributed as a
> > third-party module and still impact the built-in types; that's what
> > this whole business of __isinstance__ and __issubclass__ is about. In
> > order to achieve dynamic composition, we're having to monkey with
> > fundamental, bedrock properties of inheritance-based object
> > orientation. The definition of "subclass" and "instance" is no longer
> > objective; the answer to "is X an instance of Y?" becomes, "I dunno,
> > ask Y".
>
> I guess I don't understand why this bothers you more than how you can
> currently mess with __class__::
[snip]
> Maybe you could give a couple realistic examples of where you think
> people using __isinstance__ and __issubclass__ are going to break
> things unintentionally?

You assume that being able to change __class__ doesn't bother me: it
does. The thing that mitigates my unease is that it comes with a
gigantic red flag that boils down to, "Don't even think about this
unless you're implementing proxy objects".

By pushing capabilities tests into the core object system and making
isinstance() and issubclass() the primary mechanisms for determining
an object's behaviour, you're making it standard practice to monkey
around with inheritance. Do I have specific examples of what I'm
afraid of? No. All I can offer is a deep distrust of a system where
metaclasses and a redefined concept of inheritance are pulled out so
early in the debate in order to solve issues that other mechanisms
(like roles) make trivial.

Collin Winter

From steven.bethard at gmail.com  Mon Apr 30 20:21:24 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Mon, 30 Apr 2007 12:21:24 -0600
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <43aa6ff70704301101w7707fd8m735442cda9d14eb9@mail.gmail.com>
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<-3604285971386182607@unknownmsgid>
	<43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com>
	<d11dcfba0704301046g663cb8f8i57824b2d3041383d@mail.gmail.com>
	<43aa6ff70704301101w7707fd8m735442cda9d14eb9@mail.gmail.com>
Message-ID: <d11dcfba0704301121i7cdedd8ck86443a5fce569317@mail.gmail.com>

On 4/30/07, Collin Winter <collinw at gmail.com> wrote:
> On 4/30/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> > On 4/30/07, Collin Winter <collinw at gmail.com> wrote:
> > > No existing mechanism would allow PEP 3141 to be distributed as a
> > > third-party module and still impact the built-in types; that's what
> > > this whole business of __isinstance__ and __issubclass__ is about. In
> > > order to achieve dynamic composition, we're having to monkey with
> > > fundamental, bedrock properties of inheritance-based object
> > > orientation. The definition of "subclass" and "instance" is no longer
> > > objective; the answer to "is X an instance of Y?" becomes, "I dunno,
> > > ask Y".
> >
> > I guess I don't understand why this bothers you more than how you can
> > currently mess with __class__::
> [snip]
> > Maybe you could give a couple realistic examples of where you think
> > people using __isinstance__ and __issubclass__ are going to break
> > things unintentionally?
>
> You assume that being able to change __class__ doesn't bother me: it
> does. The thing that mitigates my unease is that it comes with a
> gigantic red flag that boils down to, "Don't even think about this
> unless you're implementing proxy objects".

Ahh.  I guess I see __isinstance__ and __issubclass__ as having
gigantic red flags saying "Don't even think about these unless you're
implementing ABCs". ;-)

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From pje at telecommunity.com  Mon Apr 30 20:46:47 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 30 Apr 2007 14:46:47 -0400
Subject: [Python-3000] super(), class decorators, and PEP 3115
Message-ID: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>

While working on the generic function PEP, I came across an unexpected (to 
me, anyway) consequence of PEP 3115: it breaks current methods (circa 
Python 2.2+) of implementing class decorators.  However, there does not 
appear to be a class decorator PEP outstanding.

In current Python versions, the only way to implement class decorators is 
to call a "magic" function in the body of the class, that inserts a special 
(temporary) metaclass to handle the decoration.  This is implemented in the 
DecoratorTools package as well as in zope.interface (where its most common 
use case is to supply interface information about a class).

Of course, in Py 3.x, some of these features could be handled using a 
metaclass with keyword arguments, due to PEP 3115.  However, the other use 
cases cannot.  Notably, any code that relies on in-class decorators or 
conditional class decorators (such as method decorators that add class 
decorators), will not be able to work.

However, I have an idea for a small modification to PEP 3115 that would 
re-enable support for both static and dynamic class decorators, and make it 
possible to implement an improved super() as well.

Suppose that we reserved a special name, '__decorators__', that would be 
contained in the mapping object returned by '__prepare__', and initialized 
to contain any static decorators defined outside the class, e.g., this code:

     @implements(IFoo)
     @otherdecorator
     class X:
         print __decorators__

would print a list equal to [implements(IFoo), otherdecorator].  The idea 
here is that following the execution of metaclass(name, bases, classdict, 
**kw), the __decorators__ list would be repeatedly popped to yield 
decorators to be invoked on the "finished" class.  In this way, code that 
needs dynamic access to the finished class can simply add objects to 
__decorators__.

As a result, we could then implement a super() enhancement of the form 
"super(__class__, self).methodname(args)", where "__class__" would be a 
compiler-defined variable (ala "__debug__").  If any function inside a 
class body uses "__class__", a preamble is added to the class body that 
defines an anonymous callback function that sets a closure variable to 
point to the class, and returns the class.  This anonymous callback 
function is then appended to __decorators__.

So, this approach could be used to enhance super() directly (using 
__class__ instead of a specific class), or as a behind-the-scenes part of 
the implementation of a more "sugary" syntax.  It would also continue to 
enable dynamic class decorators for other uses.

Thoughts?


From guido at python.org  Mon Apr 30 21:17:53 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Apr 2007 12:17:53 -0700
Subject: [Python-3000] super(), class decorators, and PEP 3115
In-Reply-To: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
Message-ID: <ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.com>

Hm... Where *is* the class decorators PEP? I was expecting it.

Assuming class decorators are added, can't you do all of this using a
custom metaclass?

I'm not sure that your proposal for implementing an improved super has
anything over the currently most-favored proposal by Timothy Delaney.

--Guido

On 4/30/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> While working on the generic function PEP, I came across an unexpected (to
> me, anyway) consequence of PEP 3115: it breaks current methods (circa
> Python 2.2+) of implementing class decorators.  However, there does not
> appear to be a class decorator PEP outstanding.
>
> In current Python versions, the only way to implement class decorators is
> to call a "magic" function in the body of the class, that inserts a special
> (temporary) metaclass to handle the decoration.  This is implemented in the
> DecoratorTools package as well as in zope.interface (where its most common
> use case is to supply interface information about a class).
>
> Of course, in Py 3.x, some of these features could be handled using a
> metaclass with keyword arguments, due to PEP 3115.  However, the other use
> cases cannot.  Notably, any code that relies on in-class decorators or
> conditional class decorators (such as method decorators that add class
> decorators), will not be able to work.
>
> However, I have an idea for a small modification to PEP 3115 that would
> re-enable support for both static and dynamic class decorators, and make it
> possible to implement an improved super() as well.
>
> Suppose that we reserved a special name, '__decorators__', that would be
> contained in the mapping object returned by '__prepare__', and initialized
> to contain any static decorators defined outside the class, e.g., this code:
>
>      @implements(IFoo)
>      @otherdecorator
>      class X:
>          print __decorators__
>
> would print a list equal to [implements(IFoo), otherdecorator].  The idea
> here is that following the execution of metaclass(name, bases, classdict,
> **kw), the __decorators__ list would be repeatedly popped to yield
> decorators to be invoked on the "finished" class.  In this way, code that
> needs dynamic access to the finished class can simply add objects to
> __decorators__.
>
> As a result, we could then implement a super() enhancement of the form
> "super(__class__, self).methodname(args)", where "__class__" would be a
> compiler-defined variable (ala "__debug__").  If any function inside a
> class body uses "__class__", a preamble is added to the class body that
> defines an anonymous callback function that sets a closure variable to
> point to the class, and returns the class.  This anonymous callback
> function is then appended to __decorators__.
>
> So, this approach could be used to enhance super() directly (using
> __class__ instead of a specific class), or as a behind-the-scenes part of
> the implementation of a more "sugary" syntax.  It would also continue to
> enable dynamic class decorators for other uses.
>
> Thoughts?
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Mon Apr 30 21:25:39 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 30 Apr 2007 12:25:39 -0700
Subject: [Python-3000] octal literals PEP
In-Reply-To: <d11dcfba0704301049t647854d9n52e474a391992cf3@mail.gmail.com>
References: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
	<ca471dc20704301019x56ae6debn586ea9a944925b73@mail.gmail.com>
	<d11dcfba0704301049t647854d9n52e474a391992cf3@mail.gmail.com>
Message-ID: <bbaeab100704301225m393b4b80qefcffd3dbfaa4092@mail.gmail.com>

On 4/30/07, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> > On 4/30/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > Guido had set an Apr 30 deadline for Py3000 PEPs that can't be
> > > implemented in pure python.
> >
> > I don't recall limiting the deadline in that way. Is this a general
> > understanding?
>
> FWIW, my understanding was only standard library changes would be
> allowed after Apr 30.

Ditto from me.  I never thought there was any connection to how the
solution would eventually be implemented.

-Brett

From pje at telecommunity.com  Mon Apr 30 21:35:36 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 30 Apr 2007 15:35:36 -0400
Subject: [Python-3000] super(), class decorators, and PEP 3115
In-Reply-To: <ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.co
 m>
References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
	<5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20070430152320.02d31868@sparrow.telecommunity.com>

At 12:17 PM 4/30/2007 -0700, Guido van Rossum wrote:
>Hm... Where *is* the class decorators PEP? I was expecting it.
>
>Assuming class decorators are added, can't you do all of this using a
>custom metaclass?

The only thing I need for the GF PEP is a way for a method decorator to get 
a callback after the class is created, so that overloading will work 
correctly in cases where overloaded methods are defined in a subclass.

In essence, when you define an overloaded method inside a class body, you 
would like to be able to treat it as if it were defined with 
"self:__class__", where __class__ is the enclosing class.  In practice, 
this means that the actual overloading has to wait until the class 
definition is finished.

In Python 2.x, RuleDispatch implements this by temporary tinkering with 
__metaclass__, but if I understand correctly this would not be possible 
with PEP 3115.  I didn't make this connection until I was fleshing out my 
PEP's explanation of how precedence works when you are overloading instance 
methods (as opposed to standalone functions).

If PEP 3115 were changed to restore support for __metaclass__, I could 
continue to use that approach.  Otherwise, some other sort of hook is required.

The class decorator thing isn't an issue for the GF PEP as such; it doesn't 
use them directly, only via the __metaclass__ hack.  I just brought it up 
because I was looking for the class decorator PEP when I realized that the 
old way of doing them wouldn't be possible any more.


>I'm not sure that your proposal for implementing an improved super has
>anything over the currently most-favored proposal by Timothy Delaney.

It's merely another use for the hook, that would save on having another 
special-purpose mechanism strictly for super(); I figured that having other 
uses for it (besides mine) would be a plus.


From brett at python.org  Mon Apr 30 21:34:55 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 30 Apr 2007 12:34:55 -0700
Subject: [Python-3000] super(), class decorators, and PEP 3115
In-Reply-To: <ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.com>
References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
	<ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.com>
Message-ID: <bbaeab100704301234q3f5aca24je9831d9089f0f68d@mail.gmail.com>

On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> Hm... Where *is* the class decorators PEP? I was expecting it.
>

Jack Diedrich was working on it.

Jack, you going to get this in to meet the April 30th deadline for
Py3K PEPs (which is obviously today)?

-Brett

From pmaupin at gmail.com  Mon Apr 30 21:51:24 2007
From: pmaupin at gmail.com (Patrick Maupin)
Date: Mon, 30 Apr 2007 14:51:24 -0500
Subject: [Python-3000] octal literals PEP
In-Reply-To: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
References: <fb6fbf560704300749r538c8674lc965f2dfcb67162e@mail.gmail.com>
Message-ID: <d09829f50704301251r35f05bf8wf7cf5397e3751faa@mail.gmail.com>

I sent an email with an initial PEP to the PEP editors a few weeks
ago.  Never got a reply.  I noticed some traffic about this recently
but was too busy to follow it really carefully.

Pat

On 4/30/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 4/30/07, Guido van Rossum <guido at python.org> wrote:
> > I think these should be two separate proposals, with more specific
> > names (e.g. "remove implicit string concatenation" and "remove
> > backslash continuation"). There's no need to mention the octal thing
> > if it's already a separate PEP.
>
> Patrick
>
> Guido had set an Apr 30 deadline for Py3000 PEPs that can't be
> implemented in pure python.
>
> Are you still working on the "Integer literal syntax and radices ",
> which included the octal literal?  I would much prefer to leave octal
> literals with the rest of that PEP, (and to let you do it :D),  but I
> will submit a much-simplified "023 raises SyntaxError" if you have
> abandoned the rest.
>
> -jJ
>

From pmaupin at gmail.com  Mon Apr 30 21:56:12 2007
From: pmaupin at gmail.com (Patrick Maupin)
Date: Mon, 30 Apr 2007 14:56:12 -0500
Subject: [Python-3000] Addition to PEP 3101
In-Reply-To: <8f01efd00704300953t6154d7e1j7ef18cead1acb344@mail.gmail.com>
References: <8f01efd00704300953t6154d7e1j7ef18cead1acb344@mail.gmail.com>
Message-ID: <d09829f50704301256n2b08b1fav7ac64b4fcc6a742c@mail.gmail.com>

On 4/30/07, James Thiele <python3now at gmail.com> wrote:
> For PEP: 3101  - Advanced String Formatting - I would like to suggest
> the following additions.
>
> First, that an additional pair of forms be added to format strings so
> that callable objects could be used as parameters to the format string
> method:
> {0()} for positional arguments and
> {keyword()} for keyword parameters.
> The proposal is that if the format string had this form that the
> format string method would treat the parameter as a callable and use
> the value of the return for the formatting.
>
> The second addition would be the forms
> {0.method()} and
> {keyword.method()}
> where "method" is a method of the object.
>
> This would allow things such as:
> "I want to shout '{0.upper()}'".format("hello world")
> which would yield:
>  "I want to shout 'HELLO WORLD'"

Method calls are deliberately disallowed by the PEP, so that the
implementation has some hope of being securable.

Regards,
Pat

From tcdelaney at optusnet.com.au  Mon Apr 30 22:59:29 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Tue, 1 May 2007 06:59:29 +1000
Subject: [Python-3000] super(), class decorators, and PEP 3115
References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
	<ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.com>
Message-ID: <011b01c78b6a$72098810$0201a8c0@ryoko>


From: "Guido van Rossum" <guido at python.org>


> I'm not sure that your proposal for implementing an improved super has
> anything over the currently most-favored proposal by Timothy Delaney.

Call me Tim ;) Unfortunately, I can't change the format that work sends my 
name out as - Delaney, Timothy.

I'm not going to be able to submit a spearate PEP for my proposal before COB 
April 30th (it's already the 1st over here ...) and I have to head to work.

Would you prefer me to work with Calvin to get his existing PEP to match my 
proposal, or would you prefer a competing PEP?

Tim Delaney 


From collinw at gmail.com  Mon Apr 30 23:03:39 2007
From: collinw at gmail.com (Collin Winter)
Date: Mon, 30 Apr 2007 14:03:39 -0700
Subject: [Python-3000] super(), class decorators, and PEP 3115
In-Reply-To: <011b01c78b6a$72098810$0201a8c0@ryoko>
References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
	<ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.com>
	<011b01c78b6a$72098810$0201a8c0@ryoko>
Message-ID: <43aa6ff70704301403q5bf557c2wf43148f7a339353d@mail.gmail.com>

On 4/30/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
> Would you prefer me to work with Calvin to get his existing PEP to match my
> proposal, or would you prefer a competing PEP?

Please work together with Calvin. One PEP is enough.

Thanks,
Collin Winter

From janssen at parc.com  Mon Apr 30 23:15:02 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 30 Apr 2007 14:15:02 PDT
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> 
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<438708814690534630@unknownmsgid>
	<79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com>
Message-ID: <07Apr30.141511pdt."57996"@synergy1.parc.xerox.com>

> > If we simply had a small standard extensible set of APIs, and a way to
> > indicate that a value does or doesn't support one or more of them,
> > we'd be done.  We could either add a new mechanism to do this, or simply
> > actually use the one we already have, which should work perfectly well for
> > this purpose.
> 
> And which has been around for many years, but has never actually been
> used in practice (unlike, say, interfaces). Does this not say
> something?

Paul,

Actually, it hasn't been around in workable form for years.  I'm not
sure of the actual chronology, but it wasn't really workable till 2.3,
with reasonable unification of built-in and user-defined types, and
the ability to subclass basic types like "dict".  So there are lots of
existing kludges that try to get around older limitations (like
interfaces).  The existence of these kludges is a symptom, not a fix.

Bill

From janssen at parc.com  Mon Apr 30 23:19:15 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 30 Apr 2007 14:19:15 PDT
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com> 
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<438708814690534630@unknownmsgid>
	<79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com>
Message-ID: <07Apr30.141916pdt."57996"@synergy1.parc.xerox.com>

> On 30/04/07, Bill Janssen <janssen at parc.com> wrote:
> > > 6) that doesn't freeze all of the key APIs in concrete
> >
> > After 15 years of experience with the key APIs, we could perhaps freeze some of
> > them?
> 
> After 15 years not being able to clearly state what "file-like" or
> "mapping-like" means to different people, perhaps we should accept
> that there is no clear-cut answer...?

And that's a problem -- people are confused.  Instead of throwing up
our hands, I think we should define what "file-like" means.

> > In other words, what APIs does this value support?
> > In other words, what APIs does this value support?
> > In other words, what APIs does this value support?
> 
> Yes, but the ABC proposal is about establishing a vocabulary for
> answering those questions, not about asking them.

That's what "isinstance" does.  The mechanism for asking is already
there; we're just trying to standardize a few answers.

And, by the way, I'm not sure adding the ability to override
"isinstance" is a good idea.

Bill

From janssen at parc.com  Mon Apr 30 23:22:36 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 30 Apr 2007 14:22:36 PDT
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com> 
References: <43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<-3604285971386182607@unknownmsgid>
	<43aa6ff70704300950t4b8179beob3b8df7001a2819b@mail.gmail.com>
Message-ID: <07Apr30.142241pdt."57996"@synergy1.parc.xerox.com>

> No existing mechanism would allow PEP 3141 to be distributed as a
> third-party module and still impact the built-in types; that's what
> this whole business of __isinstance__ and __issubclass__ is about.

Perhaps 3143 should not be distributed as a third-party module.
Perhaps it should be built into the base Python.

I think this "end-of-April" deadline, and Guido's ABC PEP, has spurred
some useful discussion.  I don't see why it has to cripple Python 3000
by ending that discussion prematurely.

Bill

From tcdelaney at optusnet.com.au  Mon Apr 30 23:30:49 2007
From: tcdelaney at optusnet.com.au (Tim Delaney)
Date: Tue, 1 May 2007 07:30:49 +1000
Subject: [Python-3000] super() PEP
References: <5.1.1.6.0.20070430142844.03c96240@sparrow.telecommunity.com>
	<ca471dc20704301217h1c117375r635bcae0034d593f@mail.gmail.com>
	<011b01c78b6a$72098810$0201a8c0@ryoko>
	<43aa6ff70704301403q5bf557c2wf43148f7a339353d@mail.gmail.com>
Message-ID: <014901c78b6e$d2d66d80$0201a8c0@ryoko>

From: "Collin Winter" <collinw at gmail.com>

> On 4/30/07, Tim Delaney <tcdelaney at optusnet.com.au> wrote:
>> Would you prefer me to work with Calvin to get his existing PEP to match 
>> my
>> proposal, or would you prefer a competing PEP?
>
> Please work together with Calvin. One PEP is enough.

Fine with me. Calvin - want to send me your latest draft, and I'll do some 
modifications? I think we've got to the point now where we can take this 
off-list.

Cheers,

Tim Delaney 


From pje at telecommunity.com  Mon Apr 30 23:42:52 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 30 Apr 2007 17:42:52 -0400
Subject: [Python-3000] Traits/roles instead of ABCs
In-Reply-To: <07Apr30.141511pdt."57996"@synergy1.parc.xerox.com>
References: <79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com>
	<43aa6ff70704291840s3384824et44ebfd360c15eda@mail.gmail.com>
	<014201c78adc$ca70d960$f101a8c0@RaymondLaptop1>
	<438708814690534630@unknownmsgid>
	<79990c6b0704301001ga0d2429sdaded9ac75fa15c5@mail.gmail.com>
Message-ID: <5.1.1.6.0.20070430173220.045a1648@sparrow.telecommunity.com>

At 02:15 PM 4/30/2007 -0700, Bill Janssen wrote:
>Actually, it hasn't been around in workable form for years.

Correction: Zope's Interface package has been around since 2002; PEAK's 
originated in 2003.  I'd say those count as both "workable form" and "for 
years".  :)


>I'm not sure of the actual chronology, but it wasn't really workable till 2.3,

You probably mean 2.2, but I'm fairly certain that the earliest Interface 
packages implemented by Jim Fulton actually predated Python 2.2, using a 
different metaclass hook mechanism.