From talin at acm.org  Wed Nov  1 04:26:47 2006
From: talin at acm.org (Talin)
Date: Tue, 31 Oct 2006 19:26:47 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
Message-ID: <454813F7.8040901@acm.org>

The thread on python-dev with respect to PEP 355 seems to have died out, 
so I thought it might be a good time to move things over here.

What I'd like to do is to start by examining the most fundamental 
assumptions about refactoring os.path and the various other path-related 
functions. Before we even begin to write a new PEP, I want to at least 
make sure that consensus is even possible - even though we may not all 
be on the same page (to use a tired metaphor), we're at least reading 
from the same book :)

Let me break this down into a number of questions:

1) Does os.path need to be refactored at all? I think most people agree 
that os.path isn't perfect, but IMHO it isn't terrible either - you can 
get work done with it, and its not really that hard to learn.

 From my own experience, I've often run into cases where I overlooked 
some critical feature of the standard library, and ended up looking 
foolish after posting my question on the python mailing lists, only to 
discover that the answer was there all along (kind of like Dorothy's red 
slippers in that way.) But I've not yet had an occasion, as far as I can 
recall, where the os.path module was the culprit.

2) Putting aside for the moment the issue of syntactic sugar, 
convenience, code beauty, and ease of learning, is there anything that 
the existing os.path *won't do* that we desperately need it to do?

3) Assuming that the answer to #1 is "yes", the next question is: 
"evolution or revolution?" Do we want something that looks 95%, 80%, or 
0% the same as the existing library? Is this just a case of moving 
around a few stray functions, or is are we going to take and axe and 
give os.path a reprogramming job it will never forget?

4) My third question is: Who are we going to steal our ideas from? 
Boost, Java, C# and others - all are worthy of being the, ahem, target 
of our inspiration. Or we have some alternative that's so cool that it 
makes sense to "Think Different(ly)"?

5) Must there be one ring to rule them all? I suggested earlier that we 
might have a "low-level" and a "high-level" API, one built on top of the 
other. Is this a good idea, or a non-starter?

OK lets start with that and see what happens.

-- Talin

From mbk.lists at gmail.com  Wed Nov  1 06:38:47 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Tue, 31 Oct 2006 22:38:47 -0700
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <00b101c6fb23$ba0bf210$0301010a@VIMES>
References: <00b101c6fb23$ba0bf210$0301010a@VIMES>
Message-ID: <da7032ce0610312138r2af314bbrf9fc9b0f8bb00a54@mail.gmail.com>

> Is anybody working [on a] PEP?

This has gone unanswered for several days, so I wanted to mention that
Ka-Ping Yee said in an off-list email from mid-October that he is
indeed working on a PEP for this, but it would take a while due to
other priorities.

   Mike

From sluggoster at gmail.com  Wed Nov  1 07:22:29 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 31 Oct 2006 22:22:29 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
Message-ID: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>

Talin wrote:
> 1) Does os.path need to be refactored at all?

Yes.  Functions are scattered arbitrarily across six modules: os,
os.path, shutil, stat, glob, fnmatch.  You have to search through five
scattered doc pages in the Python library to find your function, plus
the os module doc is split into five sections.  You may think
'shlutil' has to do with shells, not paths.  shutil.copy2 is
riduculously named: what's so "2" about it?  Why is 'split' in os.path
but 'stat' and 'mkdir' and 'remove' are in os?  Don't they all operate
on paths?

The lack of method chaning means you have to use nested functions,
which must be read "inside out" rather than left-to-right like paths
normally go. Say you want to add the absolute path of "../../lib" to
the Python path in a platform-independent manner, relative to an
absolute path (__file__):

    # Assume __file__ is "/toplevel/app1/bin/main_program.py".
    # Result is "/toplevel/app1/lib".
    p = os.path.join(os.path.dirname(os.path.dirname(__file__)), "lib")

PEP 355 proposes a much easier-to-read:

    # The Path object emulates "/toplevel/app1/bin/main_program.py".
    p = Path(__file__).parent.parent.join("lib")

Noam Raphael's directory-component object would make this even more
straightforward:

    # The Path object emulates ("/", "toplevel", "app1", "bin",
"main_program.py")
    p = Path(__file__)[:-2] + "lib"

Stat handling has grown cruft over the years. To check the modify time
of a file:

    os.path.getmtime("/FOO")
    os.stat("/FOO").st_mtime
    os.stat("/FOO")[stat.ST_MTIME]  # List subscript, deprecated usage.

If you want to check whether a file is a type for which there is no
os.path.is*() method:

    stat.S_ISSOCK( os.stat("/FOO").st_mode )  # Is the file a socket?

Compare to the directory-component proposal:

    Path("/foo").stat().issock

os.path functions are too low-level.  Say you want to recursively
delete a path you're about to overwrite, no matter whether it exists
or is a file or directory.  You can't do it in one line of code, darn,
you gotta write this function or inline the code everywhere you use
it:

    def purge(p):
        if os.path.isdir(p):
            shutil.rmtree(p)       # Raises error if nonexistent or
not a directory.
        elif os.path.exists():
            # isfile follows symlinks and returns False for special
files, so it's
            # not a reliable guide of whether we can call os.remove.
            os.remove(p)          # Raises error if nonexistent or a directory.
        if os.path.isfile(p):  # Includes all symlinks.
            os.remove(p)

> 2) is there anything that the existing os.path *won't do* that we desperately need it to do?

For filesystem files, no.  Though you really mean all six modules
above and not just os.path.  It has been proposed to support
non-filesystem directories (zip files, CSV/Subversion sandboxes, URLs,
FTP objects) under a new Path API.

>  3) Assuming that the answer to #1 is "yes", the next question is:
"evolution or revolution?"

Revolution.  It needs a clean new API.  However, this can live
alongside the existing functions if necessary:  posixpath.PosixPath,
path.Path, etc.

> 4) My third question is: Who are we going to steal our ideas from?
Boost, Java, C# and others - all are worthy of being the, ahem, target
of our inspiration. Or we have some alternative that's so cool that it
makes sense to "Think Different(ly)"?

Java is the only one I'm familiar with.  The existing Python proposals
are summarized below.

> 5) Must there be one ring to rule them all? I suggested earlier that we
might have a "low-level" and a "high-level" API, one built on top of the
other. Is this a good idea, or a non-starter?

It's worth discussing.  One question is whether the dichotomy does
anything useful or just adds unnecessary complexity.  But that can
only be answered for a specific API proposal.  Whatever we do will be
"low-level" compared to third-party extensions that will be built on
top of it, so we should plan for extensibility.

* * * *
Here's a summary of the existing Python proposals in chronological order.

Jason Orendorff's path.py
http://www.jorendorff.com/articles/python/path/src/path.py
This provides an OO path object encompassing the six modules above.
The object is a string subclass.

PEP 355
http://www.python.org/dev/peps/pep-0355/
An update to Orendorff's code.  This was rejected by the BDFL because:
(A) we haven't proven an OO interface is superior to os.path et al,
(B) it mixes abstract path operations and filesystem-dependent
operations, and (C) it's not radical enough for many OO proponents.
However, it does separate filesystem-dependent operations to an
extent: they must be methods, while abstract operations may be
attributes.  Orendorff's module does not separate these at all.

Noam Raphael's directory-based class
Introduction:  http://wiki.python.org/moin/AlternativePathClass
Feature discussion:  http://wiki.python.org/moin/AlternativePathDiscussion
Reference implementation:  http://wiki.python.org/moin/AlternativePathModule

Noam's emulates a sequence of components (a la os.path.split) rather
than a string.  This expresses slicing and joining by Python's [] and
+ operators, eliminating several named methods. Each component
emulates a unicode with methods to extract the name and extension(s).
The first component of absolute paths is a "root object" representing
the Posix root ("/"), a Windows drive-absolute ("C:\"), a Windows
drive-relative ("C:"), a network share, etc.

This proposal has a reference implementation but a PEP has not been
written, and the discussion page has many feature alternatives that
aren't coded.  A robust reference implementation would have methods
for all alternatives so they can be compared in real programs.

There have been a few additional ideas on python-dev but no code.
Nick Coghlan suggested separate classes for (A) string manipulation,
(B) abstract path operations, (C) read-only inspection of filesystem,
(D) add/remove files/directories/links.  Another suggested four
classes for (A) abstract path operations, (B) file, (C) directory, (D)
symlink.  Talin proposed an elaborate set of classes and functions to
do this.  I pointed out that each class would need 3+ versions to
accomodate platform differences, so somebody wanting to make a generic
subclass would potentially have to make 12 versions to accommodate all
the superclass possibilities (4 path classes * 3 platforms). Functions
would cut down the need for multiple classes and duplicated methods
between them, but functions would make "subclassing Path" more
difficult.

I would like to see one or more implementations tested and widely used
as soon as possible, so that we'd have confidence using them in our
programs until a general Python solution emerges.  But first we need
to see if we can achieve a common API, as Talin started this thread
saying.

-- 
Mike Orr <sluggoster at gmail.com>

From fredrik at pythonware.com  Wed Nov  1 08:04:37 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 01 Nov 2006 08:04:37 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
Message-ID: <ei9gu4$ti7$1@sea.gmane.org>

Mike Orr wrote:

 > You may think 'shutil' has to do with shells, not paths.

well, it has; it provides Python implementations of selected shell commands.

 > Why is 'split' in os.path but 'stat' and 'mkdir' and 'remove' are in
 > os?  Don't they all operate on paths?

no.  are you saying that you're unable to see the conceptual difference 
between a name and an object?

 > There have been a few additional ideas on python-dev but no code.
 > Nick Coghlan suggested separate classes for (A) string manipulation,
 > (B) abstract path operations, (C) read-only inspection of filesystem,
 > (D) add/remove files/directories/links.

that's the only remotely sane path-related proposal I've seen this far. 
  here's mine; it's fully backwards compatible, can go right into 2.6, 
and can be incrementally improved in future releases:

    1) add a pathname wrapper to "os.path", which lets you do basic
       path "algebra".  this should probably be a subclass of unicode,
       and should *only* contain operations on names.

    2) make selected "shutil" operations available via the "os" name-
       space; the old POSIX API vs. POSIX SHELL distinction is pretty
       irrelevant.  also make the os.path predicates available via the
       "os" namespace.

this gives a very simple conceptual model for the user; to manipulate 
path *names*, use "os.path.<op>(string)" functions or the "<path>" 
wrapper.  to manipulate *objects* identified by a path, given either as 
a string or a path wrapper, use "os.<op>(path)".  this can be taught in 
less than a minute.

</F>


From talin at acm.org  Wed Nov  1 08:41:41 2006
From: talin at acm.org (Talin)
Date: Tue, 31 Oct 2006 23:41:41 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
Message-ID: <45484FB5.3060608@acm.org>

Mike Orr wrote:
> Talin wrote:
>> 1) Does os.path need to be refactored at all?
> 
> Yes.  Functions are scattered arbitrarily across six modules: os,
> os.path, shutil, stat, glob, fnmatch.  You have to search through five
> scattered doc pages in the Python library to find your function, plus
> the os module doc is split into five sections.  You may think
> 'shlutil' has to do with shells, not paths.  shutil.copy2 is
> riduculously named: what's so "2" about it?  Why is 'split' in os.path
> but 'stat' and 'mkdir' and 'remove' are in os?  Don't they all operate
> on paths?

'stat' and 'mkdir' do not operate on paths. To my mind, that's kind of 
like saying that "toupper" and "encrypt" and "SQLObject.select" should 
all go together, because they all operate on strings.

-- Talin


From sluggoster at gmail.com  Wed Nov  1 09:04:20 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Wed, 1 Nov 2006 00:04:20 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <ei9gu4$ti7$1@sea.gmane.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<ei9gu4$ti7$1@sea.gmane.org>
Message-ID: <6e9196d20611010004y6ebce483l6f18f6e1bcdd802e@mail.gmail.com>

On 10/31/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
>  > Why is 'split' in os.path but 'stat' and 'mkdir' and 'remove' are in
>  > os?  Don't they all operate on paths?
>
> no.  are you saying that you're unable to see the conceptual difference
> between a name and an object?

I see the difference between modifying a path name (which doesn't
depend on the filesystem) vs doing something with it (which requires
the file to exist and be the right type).  But both os and os.path
contain both of these operations.

>     1) add a pathname wrapper to "os.path", which lets you do basic
>        path "algebra".  this should probably be a subclass of unicode,
>        and should *only* contain operations on names.

I assume this means the P.parent and P.join("lib") functionality that
I and others have been clamoring for.

>     2) make selected "shutil" operations available via the "os" name-
>        space; the old POSIX API vs. POSIX SHELL distinction is pretty
>        irrelevant.  also make the os.path predicates available via the
>        "os" namespace.

I assume this means os.path.is*(), os.path.get*(), and shutil.* would
be duplicated in the os module, as well as os.path.exists and
os.path.lexists.  os.walk() already supercedes os.path.walk(),
although I find neither of these as convenient as a generator yielding
individual path objects for every file and/or directory.

> this gives a very simple conceptual model for the user; to manipulate
> path *names*, use "os.path.<op>(string)" functions or the "<path>"
> wrapper.  to manipulate *objects* identified by a path, given either as
> a string or a path wrapper, use "os.<op>(path)".  this can be taught in
> less than a minute.

It would be better than the current situation at least.  It's mainly
the name-manipulation functions that are nested, and thus in need of
an OO paradigm and method chaining (P.basename().splitext()[0]).
Object-manipulation functions can't really be nested so OO isn't as
critical, although os.stat() is nested sometimes.

By the way, the whole stat concept is arbitrary too.  Just because C
makes some operations dependent on stat() doesn't mean we necessarily
should.  In that vein, the os.path.is* and os.path.get* functions are
an improvement. However, there is one good thing about stat():
P.stat().mtime and P.lstat().mtime look a lot better than P.mtime()
and P.lmtime() -- especially considering the two dozen other
functions/methods that would accompany the latter.

-- 
Mike Orr <sluggoster at gmail.com>

From talin at acm.org  Wed Nov  1 09:17:25 2006
From: talin at acm.org (Talin)
Date: Wed, 01 Nov 2006 00:17:25 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
Message-ID: <45485815.1030001@acm.org>

More comments...

Mike Orr wrote:
> Talin wrote:
>> 1) Does os.path need to be refactored at all?
> 
> Yes.  Functions are scattered arbitrarily across six modules: os,
> os.path, shutil, stat, glob, fnmatch.  You have to search through five
> scattered doc pages in the Python library to find your function, plus
> the os module doc is split into five sections.  You may think
> 'shlutil' has to do with shells, not paths.  shutil.copy2 is
> riduculously named: what's so "2" about it?  Why is 'split' in os.path
> but 'stat' and 'mkdir' and 'remove' are in os?  Don't they all operate
> on paths?
> 
> The lack of method chaning means you have to use nested functions,
> which must be read "inside out" rather than left-to-right like paths
> normally go. Say you want to add the absolute path of "../../lib" to
> the Python path in a platform-independent manner, relative to an
> absolute path (__file__):
> 
>     # Assume __file__ is "/toplevel/app1/bin/main_program.py".
>     # Result is "/toplevel/app1/lib".
>     p = os.path.join(os.path.dirname(os.path.dirname(__file__)), "lib")
> 
> PEP 355 proposes a much easier-to-read:
> 
>     # The Path object emulates "/toplevel/app1/bin/main_program.py".
>     p = Path(__file__).parent.parent.join("lib")

Actually I generally use:

       p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )

or even:

       p = os.path.normpath( os.path.join( __file__, "../../lib" ) )

...which isn't quite as concise as what you wrote, but is better than 
the first example. (The reason this works is because 'normpath' doesn't 
know whether the last component is a file or a directory -- it simply 
interprets the ".." as an instruction to strip off the last component.)

What I'd like to see is a version of "join" that automatically 
simplifies as it goes. Lets call it "combine":

       p = os.path.combine( __file__, "../..", "lib" )

or:

       p = os.path.combine( __file__, "../../lib" )

That's even easier to read than any of the above versions IMHO.

> Noam Raphael's directory-component object would make this even more
> straightforward:
> 
>     # The Path object emulates ("/", "toplevel", "app1", "bin",
> "main_program.py")
>     p = Path(__file__)[:-2] + "lib"

I don't know if I would describe this as 'straightforward'. 'Concise', 
certainly; 'terse', yes, and 'clever'. But also 'cute', and 'tricky'. I 
see a couple of problems with it:

-- Its only intuitive if you remember that array elements are path 
components and not strings. In other words, if you attempt to "read" the 
[:-2] as if the path were a string, you'll get the wrong answer.

-- Is path[ 0 ] a string or a path? What if I really do want to get the 
first two *characters* of the path, and not the first to components? Do 
I have to say something like:

    str( path )[ :2 ]

> Stat handling has grown cruft over the years. To check the modify time
> of a file:
> 
>     os.path.getmtime("/FOO")
>     os.stat("/FOO").st_mtime
>     os.stat("/FOO")[stat.ST_MTIME]  # List subscript, deprecated usage.
> 
> If you want to check whether a file is a type for which there is no
> os.path.is*() method:
> 
>     stat.S_ISSOCK( os.stat("/FOO").st_mode )  # Is the file a socket?
> 
> Compare to the directory-component proposal:
> 
>     Path("/foo").stat().issock

This is the part I really don't like. A path is not a file.

Imagine that if instead of paths we were doing SQL queries. Take 
SQLObject for example; say we have a table of addresses:

    Address.select( query_string )

Now, suppose you say that you want to be able to perform manipulations 
on the query string, and therefore it should be an object. So we'll 
define a new class, SQLQuery( string ):

    Address.select( SQLQuery( string ) )

And we will allow queries to be conjoined using boolean operators:

    Address.select( SQLQuery( string ) | SQLQuery( string2 ) )

Nothing controversial so far - this is the way many such systems work 
already.

But now you say "Well, since SQLQuery is an object, it would be more 
elegant to have all of the query-related functions be methods of the 
query object." So for example, if you wanted to run the query string 
against the Address table, and see how many records came back, you would 
have to do something like:

    SQLQuery( string ).select( Address ).count()

..which is exactly backwards, and here's why: Generally when creating 
member functions of objects, you arrange them in the form:

    actor.operation( subject )

Where 'actor' is considered to be the 'active agent' of the operation, 
while the 'subject' is the passive input parameter.

I would argue that both paths and query strings are passive, whereas 
tables and file systems are, if not exactly lively, at least more 
'actor-like' than paths or queries.

Now, that being said, I wouldn't have a problem with there being an 
"abstract filesystem object" that represents an entity on disk (be it 
file, directory, etc.), which would have a path inside it that would do 
some of the things you suggest.

> os.path functions are too low-level.  Say you want to recursively
> delete a path you're about to overwrite, no matter whether it exists
> or is a file or directory.  You can't do it in one line of code, darn,
> you gotta write this function or inline the code everywhere you use
> it:
> 
>     def purge(p):
>         if os.path.isdir(p):
>             shutil.rmtree(p)       # Raises error if nonexistent or
> not a directory.
>         elif os.path.exists():
>             # isfile follows symlinks and returns False for special
> files, so it's
>             # not a reliable guide of whether we can call os.remove.
>             os.remove(p)          # Raises error if nonexistent or a directory.
>         if os.path.isfile(p):  # Includes all symlinks.
>             os.remove(p)

I don't deny that such a function ought to exist. But it shouldn't be a 
member function on a path object IMHO.

>> 2) is there anything that the existing os.path *won't do* that we desperately need it to do?
> 
> For filesystem files, no.  Though you really mean all six modules
> above and not just os.path.  It has been proposed to support
> non-filesystem directories (zip files, CSV/Subversion sandboxes, URLs,
> FTP objects) under a new Path API.
> 
>>  3) Assuming that the answer to #1 is "yes", the next question is:
> "evolution or revolution?"
> 
> Revolution.  It needs a clean new API.  However, this can live
> alongside the existing functions if necessary:  posixpath.PosixPath,
> path.Path, etc.
> 
>> 4) My third question is: Who are we going to steal our ideas from?
> Boost, Java, C# and others - all are worthy of being the, ahem, target
> of our inspiration. Or we have some alternative that's so cool that it
> makes sense to "Think Different(ly)"?
> 
> Java is the only one I'm familiar with.  The existing Python proposals
> are summarized below.
> 
>> 5) Must there be one ring to rule them all? I suggested earlier that we
> might have a "low-level" and a "high-level" API, one built on top of the
> other. Is this a good idea, or a non-starter?
> 
> It's worth discussing.  One question is whether the dichotomy does
> anything useful or just adds unnecessary complexity.  But that can
> only be answered for a specific API proposal.  Whatever we do will be
> "low-level" compared to third-party extensions that will be built on
> top of it, so we should plan for extensibility.

Actually, I was considering the PEP 355 to be "high-level" and the 
current os.path to be "low-level".

-- Talin

From murman at gmail.com  Wed Nov  1 15:03:55 2006
From: murman at gmail.com (Michael Urman)
Date: Wed, 1 Nov 2006 08:03:55 -0600
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <45485815.1030001@acm.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
Message-ID: <dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>

>        p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )

Shouldn't an example avoid using the path separator directly within a
string literal?

    p = path.normpath(path.join(__file__, "..", "..", "lib"))

-- 
Michael Urman

From phd at phd.pp.ru  Wed Nov  1 15:10:11 2006
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 1 Nov 2006 17:10:11 +0300
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
Message-ID: <20061101141011.GA16013@phd.pp.ru>

On Wed, Nov 01, 2006 at 08:03:55AM -0600, Michael Urman wrote:
> >        p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )
> 
> Shouldn't an example avoid using the path separator directly within a
> string literal?
> 
>     p = path.normpath(path.join(__file__, "..", "..", "lib"))

   It also should use platform-independent constants:

p = path.normpath(path.join(__file__, os.pardir, os.pardir, "lib"))

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From jimjjewett at gmail.com  Wed Nov  1 15:27:22 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 1 Nov 2006 09:27:22 -0500
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <00b101c6fb23$ba0bf210$0301010a@VIMES>
References: <00b101c6fb23$ba0bf210$0301010a@VIMES>
Message-ID: <fb6fbf560611010627t72f87fc0ic83881b703648839@mail.gmail.com>

On 10/29/06, Gary Stephenson <garys at ihug.com.au> wrote:

> I would like to suggest the spelling "free", as I believe the term "free
> variable" has been widely used to describe the concept (albeit
> usually in reference to undeclared variables.)

They are currently called free variables (or cell variables).  This
isn't helpful for the beginner, and was confusing for me.  On the
other hand, maybe beginners shouldn't be using them anyhow.

I'll suggest "reuse"

     reuse myoutvar

A simple "use" is too generic, but "reuse" implies that it already
exists elsewhere, which (other than "not global") is the main point.

-jJ


>
> > What this discussion needs now is not further keyword suggestions, but
> > a PEP and an implementation.
>
> Is anybody working on such a PEP?
>
> gary
>
> http://www.stephensong.com.au
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/jimjjewett%40gmail.com
>

From python-dev at zesty.ca  Wed Nov  1 16:17:47 2006
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Wed, 1 Nov 2006 09:17:47 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
Message-ID: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>

Hi folks,

I have finally completed a draft of a PEP on rebinding of names
in outer scopes.  I've tried to go back and gather all of the
(amazingly numerous) proposals -- if i've forgotten or misattributed
any, let me know and i'll be happy to correct them.

I look forward to your thoughts on it:

    http://zesty.ca/python/pep-3104.txt

(Could i get an official PEP number, please?)


-- ?!ng

From python at zesty.ca  Wed Nov  1 16:27:06 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 1 Nov 2006 09:27:06 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
Message-ID: <Pine.LNX.4.58.0611010925410.3499@server1.LFW.org>

I wrote:
> I have finally completed a draft of a PEP on rebinding of names
[...]
>     http://zesty.ca/python/pep-3104.txt

I've turned this into HTML also, in case you find that easier to read:

    http://zesty.ca/python/pep-3104.html


-- ?!ng

From jcarlson at uci.edu  Wed Nov  1 17:32:08 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 01 Nov 2006 08:32:08 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <20061101141011.GA16013@phd.pp.ru>
References: <dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
	<20061101141011.GA16013@phd.pp.ru>
Message-ID: <20061101083046.C0E3.JCARLSON@uci.edu>


Oleg Broytmann <phd at phd.pp.ru> wrote:
> On Wed, Nov 01, 2006 at 08:03:55AM -0600, Michael Urman wrote:
> > >        p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )
> > 
> > Shouldn't an example avoid using the path separator directly within a
> > string literal?
> > 
> >     p = path.normpath(path.join(__file__, "..", "..", "lib"))
> 
>    It also should use platform-independent constants:
> 
> p = path.normpath(path.join(__file__, os.pardir, os.pardir, "lib"))

What operating systems that Python currently supports doesn't have ".."
mean "parent directory"?  And/or What systems currently in development
don't have ".." meaning "parent directory"?  Are they significant enough
for us to care?


 - Josiah


From talin at acm.org  Wed Nov  1 17:45:41 2006
From: talin at acm.org (Talin)
Date: Wed, 01 Nov 2006 08:45:41 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>	
	<45485815.1030001@acm.org>
	<dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
Message-ID: <4548CF35.8050000@acm.org>

Michael Urman wrote:
>>        p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )
> 
> Shouldn't an example avoid using the path separator directly within a
> string literal?
> 
>    p = path.normpath(path.join(__file__, "..", "..", "lib"))

No need. 'normpath' will automatically convert the forward-slash into a 
backward-slash on Win32 platforms.

More generally: My defacto solution to the problem of path manipulation 
is that the "path algebra" operators are strings, rather than object 
methods or functions. When you join two paths, all of the instructions 
as to how to combine the two paths are contained in the string.

-- Talin


From rrr at ronadam.com  Wed Nov  1 18:42:22 2006
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 01 Nov 2006 11:42:22 -0600
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <45485815.1030001@acm.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
Message-ID: <eiamgs$3sv$1@sea.gmane.org>

Talin wrote:

> Now, that being said, I wouldn't have a problem with there being an 
> "abstract filesystem object" that represents an entity on disk (be it 
> file, directory, etc.), which would have a path inside it that would do 
> some of the things you suggest.

I think this option should be explored a bit more for python 3000.

Is this a matter of just adding name spaces, or do you have something more 
specific in mind?

    filesys.path
    filesys.curdir
    filesys.pardir
    filesys.open
    etc...

If this, then what methods and attributes under 'os' would be moved and which 
would stay?

Or is this too big a change?

And also if this, would name spaces be used to organize methods to other IO 
objects such as local or WWW networks, and etc.? (net.path, inet.path)

Cheers,
   Ron




From phd at phd.pp.ru  Wed Nov  1 19:20:51 2006
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 1 Nov 2006 21:20:51 +0300
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <20061101083046.C0E3.JCARLSON@uci.edu>
References: <dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
	<20061101141011.GA16013@phd.pp.ru>
	<20061101083046.C0E3.JCARLSON@uci.edu>
Message-ID: <20061101182051.GA13184@phd.pp.ru>

On Wed, Nov 01, 2006 at 08:32:08AM -0800, Josiah Carlson wrote:
> > p = path.normpath(path.join(__file__, os.pardir, os.pardir, "lib"))
> 
> What operating systems that Python currently supports doesn't have ".."
> mean "parent directory"?

   macpath.py defines pardir = '::'. MacOS 9, probably.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From jcarlson at uci.edu  Wed Nov  1 20:07:17 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 01 Nov 2006 11:07:17 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <20061101182051.GA13184@phd.pp.ru>
References: <20061101083046.C0E3.JCARLSON@uci.edu>
	<20061101182051.GA13184@phd.pp.ru>
Message-ID: <20061101110513.C0E9.JCARLSON@uci.edu>


Oleg Broytmann <phd at phd.pp.ru> wrote:
> 
> On Wed, Nov 01, 2006 at 08:32:08AM -0800, Josiah Carlson wrote:
> > > p = path.normpath(path.join(__file__, os.pardir, os.pardir, "lib"))
> > 
> > What operating systems that Python currently supports doesn't have ".."
> > mean "parent directory"?
> 
>    macpath.py defines pardir = '::'. MacOS 9, probably.

MacOS 9 isn't a supported platform for Python anymore.  The continued
existance of macpath.py is either a mistake, or a kindness to those who
still use OS 9.


 - Josiah


From mark.russell at zen.co.uk  Wed Nov  1 20:05:01 2006
From: mark.russell at zen.co.uk (Mark Russell)
Date: Wed, 1 Nov 2006 19:05:01 +0000
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
Message-ID: <3B97E289-3548-4795-84D8-620DEB304B7E@zen.co.uk>

On 1 Nov 2006, at 15:17, Ka-Ping Yee wrote:
> I have finally completed a draft of a PEP on rebinding of names
> in outer scopes.

Nice summary - thanks for all the work.

> I look forward to your thoughts on it:

Minor point: it might be worth mentioning (in the section "Rebinding  
Operator") that := has been explicitly rejected in PEP 3099.  (A  
shame since IMHO it's the best solution, but there you go).

Mark Russell


From mcherm at mcherm.com  Wed Nov  1 20:28:21 2006
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed, 01 Nov 2006 11:28:21 -0800
Subject: [Python-3000] Draft PEP for outer scopes
Message-ID: <20061101112821.qd0fq3c7jnk0kows@login.werra.lunarpages.com>

?!ng writes:
> I have finally completed a draft of a PEP on rebinding of names
> in outer scopes.  I've tried to go back and gather all of the
> (amazingly numerous) proposals

And you've done an astonishingly good job of it. The amount of
history in this PEP is impressive, and it demonstrates the degree
to which this is seen as a "wart".

> I look forward to your thoughts on it

I think it's great... let's approve and implement it immediately!

-- Michael Chermside


From sluggoster at gmail.com  Wed Nov  1 21:47:12 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Wed, 1 Nov 2006 12:47:12 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <45485815.1030001@acm.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
Message-ID: <6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>

The thread on python-dev has been revived, so those interested in this
subject will want to look in both places.

On 11/1/06, Talin <talin at acm.org> wrote:
> Actually I generally use:
>
>        p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )
>
> or even:
>
>        p = os.path.normpath( os.path.join( __file__, "../../lib" ) )
>
> ...which isn't quite as concise as what you wrote, but is better than
> the first example. (The reason this works is because 'normpath' doesn't
> know whether the last component is a file or a directory -- it simply
> interprets the ".." as an instruction to strip off the last component.)

This illustrates two problems with os.path.  The reason I use all
these nested functions instead of a simple normpath/join is one is
told "it's bad to use platform-specific separators".  Perhaps this
disrecommendation should be lifted, especially since both Mac and
Windows do the right thing with "/", "..", and "." now.

The other thing is, ".." and "." seem to be smarter than
os.path.dirname.  I can't quite articulate the rules but '.' off a
file chops the filename component, while '.' off a directory does
nothing.  '..' off a file goes to the file's directory, while '..' off
a directory goes to the directory's parent.  Dirname() just chops the
final component without asking what it is, while '..' and '.' do
different things depending on whether the final component is a
directory.  I think a method like .ancestor(N) would be useful,
meaning "do '..' N times.

/a/b/../c    # Previous component is always a directory, so eliminate.
/a/b/./c     # Swallow the '.'.
/a/directory/..

> What I'd like to see is a version of "join" that automatically
> simplifies as it goes. Lets call it "combine":
>
>        p = os.path.combine( __file__, "../..", "lib" )
>
> or:
>
>        p = os.path.combine( __file__, "../../lib" )
>
> That's even easier to read than any of the above versions IMHO.

I wouldn't mind that actually.  But the feedback I've gotten is the
fewer variations from os.path functions, the better. I disagree with
that though.


[In Noam Raphael's propsal:]
> -- Is path[ 0 ] a string or a path? What if I really do want to get the
> first two *characters* of the path, and not the first to components? Do
> I have to say something like:
>
>     str( path )[ :2 ]

That's something we've gone back and forth on: how to add characters
to a path, how to add an extension, etc.  We decided component-slicing
was too important to give up.  str(p) or unicode(p) of any Path will
give the string representation.  Converting to a string and back may
not be the most elegant thing in the world but it's more
straightforward than having special methods for character slicing.  I
have also proposed

p[0] is a special "root object", which may be '', '/', 'c:\', 'c:',
'\\abc' depending on the platform.  So when joining there's no
separator before the next component.

p[1:] individually are each a subclass of unicode, with extra methods
to extract basename and extension, delete N extensions from the end,
add N extensions, etc.

Any slice of a path object is a new path object.  If the root is
chopped off it becomes a relative path.

> I would argue that both paths and query strings are passive, whereas
> tables and file systems are, if not exactly lively, at least more
> 'actor-like' than paths or queries.

I can see your point. The only reason I went with a "monolithic" OO
class is because that's what all the proposals have been for the past
three years until last month, and I didn't think another way was
possible or desirable.

-- 
Mike Orr <sluggoster at gmail.com>

From qrczak at knm.org.pl  Wed Nov  1 22:16:18 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Wed, 01 Nov 2006 22:16:18 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	(Mike Orr's message of "Wed, 1 Nov 2006 12:47:12 -0800")
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
Message-ID: <87velysw9p.fsf@qrnik.zagroda>

"Mike Orr" <sluggoster at gmail.com> writes:

> /a/b/../c    # Previous component is always a directory, so eliminate.

This is not equivalent to a/c on Unix if b is a symlink to a directory,
because .. goes to the parent of the target of the symlink.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From ntoronto at cs.byu.edu  Wed Nov  1 23:29:08 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Wed, 01 Nov 2006 15:29:08 -0700
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
Message-ID: <45491FB4.3000507@cs.byu.edu>

Ka-Ping Yee wrote:
> Hi folks,
>
> I have finally completed a draft of a PEP on rebinding of names
> in outer scopes.  I've tried to go back and gather all of the
> (amazingly numerous) proposals -- if i've forgotten or misattributed
> any, let me know and i'll be happy to correct them.
>
> I look forward to your thoughts on it:
>   
It's beautiful. Like Michael said, an impressive amount of history.

I wondered about this specifically:

 > A shorthand form is also permitted, in which nonlocal is prepended to 
an assignment or augmented assignment statement:
 >
 >     nonlocal x = 3

Is a similar statement for globals legal in Py3k? It's not in 2.4 
(according to my own testing) or 2.5 (according to the grammar). The 
syntax for 'global' and 'nonlocal' should be almost identical.

Neil


From guido at python.org  Wed Nov  1 23:52:06 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 1 Nov 2006 14:52:06 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <45491FB4.3000507@cs.byu.edu>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
Message-ID: <ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>

On 11/1/06, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> Ka-Ping Yee wrote:
> > I have finally completed a draft of a PEP on rebinding of names
> > in outer scopes.  I've tried to go back and gather all of the
> > (amazingly numerous) proposals -- if i've forgotten or misattributed
> > any, let me know and i'll be happy to correct them.

> It's beautiful. Like Michael said, an impressive amount of history.

Agreed.

> I wondered about this specifically:
>
>  > A shorthand form is also permitted, in which nonlocal is prepended to
>  > an assignment or augmented assignment statement:
>  >
>  >     nonlocal x = 3
>
> Is a similar statement for globals legal in Py3k? It's not in 2.4
> (according to my own testing) or 2.5 (according to the grammar). The
> syntax for 'global' and 'nonlocal' should be almost identical.

It's been proposed and I would endorse it.

My personal preference is still to abuse 'global' instead of adding a
new, ugly keyword. That would make the syntax for global and nonlocal
completely identical. :-) But I seem to be alone in this preference.
:-(

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Thu Nov  2 00:07:28 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 1 Nov 2006 15:07:28 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
Message-ID: <bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>

On 11/1/06, Guido van Rossum <guido at python.org> wrote:
>
> On 11/1/06, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> > Ka-Ping Yee wrote:
> > > I have finally completed a draft of a PEP on rebinding of names
> > > in outer scopes.  I've tried to go back and gather all of the
> > > (amazingly numerous) proposals -- if i've forgotten or misattributed
> > > any, let me know and i'll be happy to correct them.
>
> > It's beautiful. Like Michael said, an impressive amount of history.
>
> Agreed.
>
> > I wondered about this specifically:
> >
> >  > A shorthand form is also permitted, in which nonlocal is prepended to
> >  > an assignment or augmented assignment statement:
> >  >
> >  >     nonlocal x = 3
> >
> > Is a similar statement for globals legal in Py3k? It's not in 2.4
> > (according to my own testing) or 2.5 (according to the grammar). The
> > syntax for 'global' and 'nonlocal' should be almost identical.
>
> It's been proposed and I would endorse it.
>
> My personal preference is still to abuse 'global' instead of adding a
> new, ugly keyword. That would make the syntax for global and nonlocal
> completely identical. :-) But I seem to be alone in this preference.
> :-(


Seeing Guido have a sad face is enough to force me to have an opinon.  I
personally always viewed 'global' as "this variable is not local", so making
it truly mean that works for me.  Otherwise I would like the 'free' keyword
(I understand why Ping prefers 'nonlocal', but i just don't like it without
a hyphen in it and that isn't about to happen and I have no problem going
with the lambda calculus terminology).

Then again Guido could still get his way since this could end up being quite
the little argument as demonstrated by the list of possible keyword names
and thus require a BDFL pronouncement the way Guido wants it.  =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061101/71c6697a/attachment.html 

From greg.ewing at canterbury.ac.nz  Thu Nov  2 00:14:43 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2006 12:14:43 +1300
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
Message-ID: <45492A63.9030703@canterbury.ac.nz>

Guido van Rossum wrote:

> My personal preference is still to abuse 'global' instead of adding a
> new, ugly keyword. That would make the syntax for global and nonlocal
> completely identical. :-) But I seem to be alone in this preference.

You're not alone -- I'd be happy with that, too.
I don't even think it's particularly abusive,
since I regard "nonlocal" as a legitimate meaning
of "global" in a language that can have more than
two nested scopes.

--
Greg

From mahs at telcopartners.com  Thu Nov  2 00:28:23 2006
From: mahs at telcopartners.com (Michael Spencer)
Date: Wed, 01 Nov 2006 15:28:23 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
Message-ID: <eibahp$gvm$1@sea.gmane.org>

Ka-Ping Yee wrote:
> Hi folks,
> 
> I have finally completed a draft of a PEP on rebinding of names
> in outer scopes.  I've tried to go back and gather all of the
> (amazingly numerous) proposals -- if i've forgotten or misattributed
> any, let me know and i'll be happy to correct them.
> 
> I look forward to your thoughts on it:
> 
>     http://zesty.ca/python/pep-3104.txt
> 
> (Could i get an official PEP number, please?)
> 
> 
> -- ?!ng
Looks good.  I tried an experimental implementation using the compiler2 package. 
  (http://groups.google.com/group/comp.lang.python/msg/f10db8a1ca2047b4)

compiler2 doesn't do parsing, so instead of `nonlocal a,b` I abuse existing 
syntax and write `global __nonlocal__,a,b`

I did not try to implement `nonlocal a=3` form, since the existing global syntax 
does not allow it.  If it's going to be legal, then will global be similarly 
adjusted?

I assume nonlocal statements should not nest the way global does i.e., a 
nonlocal statement in one scope has no affect on assignment in an *enclosed* scope.

Michael

Here it is:

"""Experimental implementation of 'PEP 3104', using compiler2.
Requires:
   Python 2.5
   compiler2 from http://svn.brownspencer.com/pycompiler/branches/new_ast/
   See readme.txt for installation
   This file also available from:
    http://svn.brownspencer.com/pycompiler/branches/new_ast/sandbox/nonlocal.py
"""

from compiler2.pyassem import (CompilerUnit, FunctionMixin, ClassMixin,
     ModuleMixin, SymbolVisitor, CodeGenerator, OrderedSet, SYMBOLS, parse)

from _ast import AST, stmt, Global
import _ast

class NonLocal(stmt):
     """A new node type to represent the PEP3104 nonlocal statement"""
     _fields = ('names',)
     _attributes = ['lineno', 'col_offset']

# push NonLocal into _ast to satisfy SymbolVisitor
_ast.NonLocal = NonLocal


def convert_nonlocal(tree):
     """We have no syntax support for 'nonlocal' keyword.  So, instead,
     use global __nonlocal__, target[s].  This conversion function then
     turns the Global nodes into NonLocal, and removes the __nonlocal__
     flag"""

     stack = [tree]
     while stack:
         node = stack.pop()
         if isinstance(node, Global):
             if node.names[0] == "__nonlocal__":
                 node.__class__ = NonLocal
                 del node.names[0]
         else:
             for attr in node.__dict__.values():
                 if type(attr) is list:
                     stack.extend(elt for elt in attr if isinstance(elt, AST))
                 elif isinstance(attr, AST):
                     stack.append(attr)

class MySymbolVisitor(SymbolVisitor):

     def visitNonLocal(self, node, cu):
         for name in node.names:
             cu.add_free(name)


class MyCompilerUnit(CompilerUnit):

     def __init__(self, *args):
        CompilerUnit.__init__(self, *args)
        self.frees = set() # track names marked explicitly free (via nonlocal)

     # adjust add_def to ignore frees, like globals
     def add_def(self, name):
         m_name = self.mangle(name)
         if m_name not in self.globals and m_name not in self.frees:
             if m_name in self.defs:
                 self.redefs.add(m_name)
             self.defs.add(m_name)
             self.symbols.add(m_name)
             self.uses.discard(m_name)

     # new method - called by MySymbolVisitor.visitNonLocal
     def add_free(self, name):
         """Mark a name as explicitly free, i.e., nonlocal"""
         m_name = self.mangle(name)
         if m_name in self.uses or m_name in self.defs:
             pass # Warn about nonlocal following def/use
         elif m_name in self.params:
             raise SyntaxError, "Parameter can't be declared nonlocal"
         elif m_name in self.globals:
             raise SyntaxError, "Name can't be declared global and nonlocal"
         self.frees.add(m_name)
         self.symbols.add(m_name)

     def _get_frees(self, child_frees, outer_defs):
         """Helper for finish_symbol_analysis"""

         # only change from base method is to include self.frees
         self_frees = (self.frees | self.uses | child_frees) & outer_defs
         if self_frees:
             symbols=self.symbols
             fv = [n for n in symbols if n in self_frees]
             fv += (n for n in self_frees if n not in symbols)
             self.freevars = tuple(fv)
             self.closure.update(fv)
         return self_frees

     # Override required to use re-defined FunctionUnit
     def new_child_function(self, node, name, firstlineno):
         """Create a new function unit as a child of the current one."""
         return FunctionUnit(node, self.filename, name, firstlineno, self)

     # Override required to use re-defined ClassUnit
     def new_child_class(self, node, name, firstlineno):
         """Create a new function unit as a child of the current one."""
         return ClassUnit(node, self.filename, name, firstlineno, self)



class FunctionUnit(FunctionMixin, MyCompilerUnit):
     pass

class ClassUnit(ClassMixin, MyCompilerUnit):
     pass

class ModuleUnit(ModuleMixin, MyCompilerUnit):

     # Override to use our re-defined SymbolVisitor
     def make_scopes(self, visitor = MySymbolVisitor):
         s = visitor()
         s.visit(self.node, self)


# Adjust pycompile to take non_local flag
def pycompile(source, filename, mode, flags=None, dont_inherit=None, 
non_local=False):
     """Replacement for builtin compile() function"""
     tree = parse(source, filename, mode)
     if non_local:
         convert_nonlocal(tree)
     if mode == "exec":
         gen = ModuleUnit(tree, filename)
     else:
         raise ValueError("compile() 3rd arg must be 'exec' ["
                          "'eval' and 'single' not yet Implemented]")
     return gen.compile()


test_src = """\
# Note the behavior of the a=3 assignment depends on whether nonlocal
# is enabled.  If so, it rebinds f's cellvar.  If not, it rebinds
# global a

a=1
def f():
     a=2
     def g():
         global __nonlocal__, a # means nonlocal a
         a=3
     g()
     return a
f_a = f()
"""
"""
Examples and test:

  # using the nonlocal feature
  >>> nsp = {}
  >>> exec pycompile(test_src, "string", "exec", non_local = True) in nsp
  >>> assert nsp['a'] == 1
  >>> assert nsp['f_a'] == 3

  # compare with compiler2, nonlocal not active
  >>> nsp = {}
  >>> exec pycompile(test_src, "string", "exec", non_local = False) in nsp
  >>> assert nsp['a'] == 3
  >>> assert nsp['f_a'] == 2

  # compare with __builtin__.compile
  >>> nsp = {}
  >>> exec compile(test_src, "string", "exec") in nsp
  >>> assert nsp['a'] == 3
  >>> assert nsp['f_a'] == 2
  >>>
"""

def test():
     import doctest
     doctest.testmod()

if __name__ == "__main__":
     test()
     print 'ok'


From python at zesty.ca  Thu Nov  2 01:55:26 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 1 Nov 2006 18:55:26 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <45491FB4.3000507@cs.byu.edu>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
Message-ID: <Pine.LNX.4.58.0611011847130.3499@server1.LFW.org>

Hi again.

I've updated the PEP to incorporate some suggestions i received.
This revision also adds a Ruby example, a quote from 1994, a
security argument, and and an additional error case (attempting
to declare a parameter name nonlocal).  Thanks, everyone!

    http://zesty.ca/python/pep-3104.txt
    http://zesty.ca/python/pep-3104.html


-- ?!ng

From python at zesty.ca  Thu Nov  2 02:01:12 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 1 Nov 2006 19:01:12 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>

Guido wrote:
> My personal preference is still to abuse 'global' instead of adding a
> new, ugly keyword. That would make the syntax for global and nonlocal
> completely identical. :-) But I seem to be alone in this preference.

Brett wrote:
> Seeing Guido have a sad face is enough to force me to have an opinon.  I
> personally always viewed 'global' as "this variable is not local", so making
> it truly mean that works for me.

I'm convinced that "global variable" means top-level for most
programmers and so this usage would be confusing -- but i think we're
all just repeating what we've said before.

Would it help at all to survey some folks to see how many interpret
"global variable" to mean "top-level" vs. "anything nonlocal"?


-- ?!ng

From andrewm at object-craft.com.au  Thu Nov  2 02:40:59 2006
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 02 Nov 2006 12:40:59 +1100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611011857150.3499@server1.LFW.org> 
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
Message-ID: <20061102014059.BC4BB6F4198@longblack.object-craft.com.au>

>Guido wrote:
>> My personal preference is still to abuse 'global' instead of adding a
>> new, ugly keyword. That would make the syntax for global and nonlocal
>> completely identical. :-) But I seem to be alone in this preference.
>
>Brett wrote:
>> Seeing Guido have a sad face is enough to force me to have an opinon.  I
>> personally always viewed 'global' as "this variable is not local", so making
>> it truly mean that works for me.
>
>I'm convinced that "global variable" means top-level for most
>programmers and so this usage would be confusing -- but i think we're
>all just repeating what we've said before.

But it doesn't mean "top-level" - it already comes with the qualifier
"module". Even after nearly a decade of python use, I still find that
slightly unnatural, and changing "global" to mean "enclosing scope"
feels only slightly more unnatural.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From tjreedy at udel.edu  Thu Nov  2 04:34:44 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 1 Nov 2006 22:34:44 -0500
Subject: [Python-3000] Draft PEP for outer scopes
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org><45491FB4.3000507@cs.byu.edu><ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com><bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com><Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
Message-ID: <eibp0h$e62$1@sea.gmane.org>


"Andrew McNamara" <andrewm at object-craft.com.au> wrote in message 
news:20061102014059.BC4BB6F4198 at longblack.object-craft.com.au...
> >Guido wrote:
>>> My personal preference is still to abuse 'global' instead of adding a
>>> new, ugly keyword. That would make the syntax for global and nonlocal
>>> completely identical. :-) But I seem to be alone in this preference.

No, I (and others, it seems) just never have reason before to clutter the 
list with a 'me-too' post.

> But it doesn't mean "top-level" - it already comes with the qualifier
> "module". Even after nearly a decade of python use, I still find that
> slightly unnatural,

Me too ;-)

> and changing "global" to mean "enclosing scope"
> feels only slightly more unnatural.

Me too ;-)

Actually, I would only want a separate keyword would be if one wanted to be 
able to write

x = 1
def f():
  x = 'a'
  def _f():
    global x = {}

and have that act differently (as at present) from the inside-out behaviour 
of
newkey x = 'whatever'.  But I will not write such code and do not want to 
see such.

With two different but similar keywords, I would expect to see requests to 
be able to write
    def _f():
        global x = 2
        print x
        nonlocal x = 'b'
        print x

Terry Jan Reedy




From steven.bethard at gmail.com  Thu Nov  2 04:39:35 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 1 Nov 2006 20:39:35 -0700
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
Message-ID: <d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>

Guido wrote:
> My personal preference is still to abuse 'global' instead of adding a
> new, ugly keyword. That would make the syntax for global and nonlocal
> completely identical. :-) But I seem to be alone in this preference.

Brett wrote:
> Seeing Guido have a sad face is enough to force me to have an opinon.  I
> personally always viewed 'global' as "this variable is not local", so making
> it truly mean that works for me.

Ka-Ping Yee wrote:
> Would it help at all to survey some folks to see how many interpret
> "global variable" to mean "top-level" vs. "anything nonlocal"?

I don't think that'll really be worth it.  I'd be amazed if people
didn't expect it to mean "top-level". The real question is, if people
see something like this::

    def f():
        n = 0
        def g(i):
            global n
            n += i
        return g
    func = f()
    print func(), func()

what would they expect it to do?  If you need to run a survey, that's
probably the one to run.  (Of course a different code example could be
used, but you get the idea).

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From python at zesty.ca  Thu Nov  2 05:28:13 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 1 Nov 2006 22:28:13 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org> 
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com> 
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>


Ka-Ping Yee wrote:
> Would it help at all to survey some folks to see how many interpret
> "global variable" to mean "top-level" vs. "anything nonlocal"?

Steven Bethard wrote:
> I don't think that'll really be worth it.  I'd be amazed if people
> didn't expect it to mean "top-level".

If that's as obvious to you as it is to me, i don't understand why
there's still any question.  I've never heard of a programming
language or conversation about programming where "global variable"
means a variable bound in an outer enclosing function; it always
means a variable bound outside of any functions (Wikipedia: "a
variable that does not belong to any subroutine or class").

> The real question is, if people see something like this::
[...]
> what would they expect it to do?

I think a fairer survey example would be something like this:

    n = 1
    def f():
        n = 2
        def g():
            global n = 99
            return n
        g()

    f()
    print n

Which 'n' do you expect g() to change?

(My answer: the global n.  That's what it says: "global n".
Which n is global?  Clearly the first one.)


-- ?!ng

From steven.bethard at gmail.com  Thu Nov  2 05:36:14 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 1 Nov 2006 21:36:14 -0700
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
Message-ID: <d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>

On 11/1/06, Ka-Ping Yee <python at zesty.ca> wrote:
> I think a fairer survey example would be something like this:
>
>     n = 1
>     def f():
>         n = 2
>         def g():
>             global n = 99
>             return n
>         g()
>
>     f()
>     print n
>
> Which 'n' do you expect g() to change?

The only reason I didn't use that in the first place is that is seems
extremely unlikely in real code.  Who uses the same name for a
module-level binding and a function-local binding?

So yes, that's the corner case, but I contend that the corner case
will almost never come up.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From python at zesty.ca  Thu Nov  2 05:52:05 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Wed, 1 Nov 2006 22:52:05 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
Message-ID: <Pine.LNX.4.58.0611012229080.3499@server1.LFW.org>

I wrote:
> I'm convinced that "global variable" means top-level for most
> programmers and so this usage would be confusing -- but i think we're
> all just repeating what we've said before.

Andrew McNamara wrote:
> But it doesn't mean "top-level" - it already comes with the qualifier
> "module".

This is exactly what "global variable" means in C, Perl, JavaScript,
Ruby, etc.

The global scope is the widest scope in which you can declare a variable.


-- ?!ng

From greg.ewing at canterbury.ac.nz  Thu Nov  2 06:15:18 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2006 18:15:18 +1300
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20611010004y6ebce483l6f18f6e1bcdd802e@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<ei9gu4$ti7$1@sea.gmane.org>
	<6e9196d20611010004y6ebce483l6f18f6e1bcdd802e@mail.gmail.com>
Message-ID: <45497EE6.6080202@canterbury.ac.nz>

Mike Orr wrote:

> P.stat().mtime and P.lstat().mtime look a lot better than P.mtime()
> and P.lmtime()

Another thing to consider is that stat() potentially
lets you enquire about multiple attributes without
making a system call for each one.

--
Greg

From andrewm at object-craft.com.au  Thu Nov  2 06:24:12 2006
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Thu, 02 Nov 2006 16:24:12 +1100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611012229080.3499@server1.LFW.org> 
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
	<Pine.LNX.4.58.0611012229080.3499@server1.LFW.org>
Message-ID: <20061102052412.A91B36F4198@longblack.object-craft.com.au>

>Andrew McNamara wrote:
>> But it doesn't mean "top-level" - it already comes with the qualifier
>> "module".
>
>This is exactly what "global variable" means in C, Perl, JavaScript,
>Ruby, etc.
>
>The global scope is the widest scope in which you can declare a variable.

This is inaccurate at best. The reality is:

Python's "global" is module-global, yet there's a wider scope that is
searched after the module namespace: the __builtin__ module.

C has a similar concept module-global (module "static" variables), but
if I remember correctly Perl does not (globals are interpreter-global),
and neither does Javascript (globals are global to the document, with
each document essentially a separate interpreter instance).

To be honest, I'm +0 on using "global" in the way GvR proposes. My
point is that python's globals are already different from other common
languages, and people cope with that.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From greg.ewing at canterbury.ac.nz  Thu Nov  2 06:25:08 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2006 18:25:08 +1300
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<dcbbbb410611010603s21a4b37i50cc231adee810ea@mail.gmail.com>
Message-ID: <45498134.1030009@canterbury.ac.nz>

Michael Urman wrote:
>>       p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )
> 
> Shouldn't an example avoid using the path separator directly within a
> string literal?

And avoid using a platform-specific parent
directory specifier:

   p = path.normpath(path.join(__file__, os.pardir, os.pardir, "lib"))

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov  2 06:27:47 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Nov 2006 18:27:47 +1300
Subject: [Python-3000] Alternatives to 'outer'
In-Reply-To: <fb6fbf560611010627t72f87fc0ic83881b703648839@mail.gmail.com>
References: <00b101c6fb23$ba0bf210$0301010a@VIMES>
	<fb6fbf560611010627t72f87fc0ic83881b703648839@mail.gmail.com>
Message-ID: <454981D3.5090909@canterbury.ac.nz>

Jim Jewett wrote:

> I'll suggest "reuse"
> 
>      reuse myoutvar

But to me that sounds like you're shadowing myoutvar,
which is completely the wrong idea.

--
Greg

From fredrik at pythonware.com  Thu Nov  2 07:23:49 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 02 Nov 2006 07:23:49 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
Message-ID: <eic2tk$489$1@sea.gmane.org>

Guido van Rossum wrote:

>>  > A shorthand form is also permitted, in which nonlocal is prepended to
>>  > an assignment or augmented assignment statement:
>>  >
>>  >     nonlocal x = 3
>>
>> Is a similar statement for globals legal in Py3k? It's not in 2.4
>> (according to my own testing) or 2.5 (according to the grammar). The
>> syntax for 'global' and 'nonlocal' should be almost identical.
> 
> It's been proposed and I would endorse it.

except that the syntax looks like you're declaring *and* defining 
something in *this* scope, which isn't what's going on.

> My personal preference is still to abuse 'global' instead of adding a
> new, ugly keyword. That would make the syntax for global and nonlocal
> completely identical. :-) But I seem to be alone in this preference.

not necessarily, but that would rule it out for 2.X, I suppose.

</F>


From sluggoster at gmail.com  Thu Nov  2 07:47:54 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Wed, 1 Nov 2006 22:47:54 -0800
Subject: [Python-3000] Mini Path object
Message-ID: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>

Posted to python-dev and python-3000.  Follow-ups to python-dev only please.

On 10/31/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
>   here's mine; it's fully backwards compatible, can go right into 2.6,
> and can be incrementally improved in future releases:
>
>     1) add a pathname wrapper to "os.path", which lets you do basic
>        path "algebra".  this should probably be a subclass of unicode,
>        and should *only* contain operations on names.
>
>     2) make selected "shutil" operations available via the "os" name-
>        space; the old POSIX API vs. POSIX SHELL distinction is pretty
>        irrelevant.  also make the os.path predicates available via the
>        "os" namespace.
>
> this gives a very simple conceptual model for the user; to manipulate
> path *names*, use "os.path.<op>(string)" functions or the "<path>"
> wrapper.  to manipulate *objects* identified by a path, given either as
> a string or a path wrapper, use "os.<op>(path)".  this can be taught in
> less than a minute.

Given the widely-diverging views on what, if anything, should be done
to os.path, how about we make a PEP and a standalone implementation of
(1) for now, and leave (2) and everything else for a later PEP.  This
will make people who want a reasonably forward-compatable object NOW
for their Python 2.4/2.5 programs happy, provide a common seed for
more elaborate libraries that may be proposed for the standard library
later (and eliminate the possibility of moving the other functions and
later deprecating them), and provide a module that will be well tested
by the time 2.6 is ready for finalization.

There's already a reference implementation in PEP 355, we'd just have
to strip out the non-pathname features.  There's a copy here
(http://wiki.python.org/moin/PathModule) that looks reasonably recent
(constructors are self.__class__() to make it subclassable), although
I wonder why the class is called path instead of Path.  There was
another copy in the Python CVS  although I can't find it now; was it
deleted in the move to Subversion?  (I thought it was in
/sandbox/trunk/: http://svn.python.org/view/sandbox/trunk/).

So, let's say we strip this Path class to:

class Path(unicode):
    Path("foo")
    Path(  Path("directory"),   "subdirectory", "file")    # Replaces
.joinpath().
    Path()
    Path.cwd()
    Path("ab") + "c"  => Path("abc")
    .abspath()
    .normcase()
    .normpath()
    .realpath()
    .expanduser()
    .expandvars()
    .expand()
    .parent
    .name                 # Full filename without path
    .namebase        # Filename without extension
    .ext
    .drive
    .splitpath()
    .stripext()
    .splitunc()
    .uncshare
    .splitall()
    .relpath()
    .relpathto()

Would this offend anyone?  Are there any attribute renames or method
enhancements people just can't live without?  'namebase' is the only
name I hate but I could live with it.

The multi-argument constructor is a replacement for joining paths.
(The PEP says .joinpath was "problematic" without saying why.)    This
could theoretically go either way, doing either the same thing as
os.path.join, getting a little smarter, or doing "safe" joins by
disallowing "/" embedded in string arguments.

I would say that a directory-tuple Path object with these features
could be maintained in parallel, but since the remaining functions
require string arguments you'd have to use unicode() a lot.

-- 
Mike Orr <sluggoster at gmail.com>

From ronaldoussoren at mac.com  Thu Nov  2 08:40:18 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 2 Nov 2006 08:40:18 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <20061101110513.C0E9.JCARLSON@uci.edu>
References: <20061101083046.C0E3.JCARLSON@uci.edu>
	<20061101182051.GA13184@phd.pp.ru>
	<20061101110513.C0E9.JCARLSON@uci.edu>
Message-ID: <15D45482-38F9-453A-9083-1C9F64C4A7BE@mac.com>


On Nov 1, 2006, at 8:07 PM, Josiah Carlson wrote:

>
> Oleg Broytmann <phd at phd.pp.ru> wrote:
>>
>> On Wed, Nov 01, 2006 at 08:32:08AM -0800, Josiah Carlson wrote:
>>>> p = path.normpath(path.join(__file__, os.pardir, os.pardir, "lib"))
>>>
>>> What operating systems that Python currently supports doesn't  
>>> have ".."
>>> mean "parent directory"?
>>
>>    macpath.py defines pardir = '::'. MacOS 9, probably.
>
> MacOS 9 isn't a supported platform for Python anymore.  The continued
> existance of macpath.py is either a mistake, or a kindness to those  
> who
> still use OS 9.

Classic MacOS paths are still used by Carbon applications on Mac OS X  
and by AppleScript, which means there's still a need to use macpath  
sometimes (especially for the application scripting).

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061102/ed3625c9/attachment.bin 

From p.f.moore at gmail.com  Thu Nov  2 09:46:09 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 2 Nov 2006 08:46:09 +0000
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
Message-ID: <79990c6b0611020046j9d95781i378b65a55ea016c3@mail.gmail.com>

On 11/2/06, Mike Orr <sluggoster at gmail.com> wrote:
> Given the widely-diverging views on what, if anything, should be done
> to os.path, how about we make a PEP and a standalone implementation of
> (1) for now, and leave (2) and everything else for a later PEP.

Why write a PEP at this stage? Just release your proposal as a module,
and see if people use it. If they do, write a PEP to include it in the
stdlib. (That's basically what happened with the original PEP - it
started off proposing Jason Orendorff's path module IIRC).

>From what you're proposing, I may well use such a module, if it helps
:-) (But I'm not sure I'd vote for it in to go the stdlib without
having survived as an external module first)

Paul.

From tanzer at swing.co.at  Thu Nov  2 10:03:24 2006
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 02 Nov 2006 10:03:24 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: Your message of "Wed, 01 Nov 2006 19:01:12 CST."
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
Message-ID: <E1GfYTs-0002nS-1v@swing.co.at>


Ka-Ping Yee <python at zesty.ca> wrote:

> Guido wrote:
> > My personal preference is still to abuse 'global' instead of adding a
> > new, ugly keyword. That would make the syntax for global and nonlocal
> > completely identical. :-) But I seem to be alone in this preference.
>
> Brett wrote:
> > Seeing Guido have a sad face is enough to force me to have an opinon.  I
> > personally always viewed 'global' as "this variable is not local", so making
> > it truly mean that works for me.
>
> I'm convinced that "global variable" means top-level for most
> programmers and so this usage would be confusing -- but i think we're
> all just repeating what we've said before.

Well, for many programmers `global` means visible everywhere in the
program, so Python's use of the word is already confusing for some
people. But that confusion usually doesn't last long and neither would
the proposed reinterpretation's, IMHO.

OTOH, having `global` and `nonlocal` mean the same thing for
module-bound variables violates TOOWTDI.

-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From tanzer at swing.co.at  Thu Nov  2 09:55:41 2006
From: tanzer at swing.co.at (Christian Tanzer)
Date: Thu, 02 Nov 2006 09:55:41 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: Your message of "Wed, 01 Nov 2006 18:55:26 CST."
	<Pine.LNX.4.58.0611011847130.3499@server1.LFW.org>
Message-ID: <E1GfYMP-0002mn-5b@swing.co.at>


Ka-Ping Yee <python at zesty.ca> wrote:

> A shorthand form is also permitted, in which ``nonlocal`` is
> prepended to an assignment or augmented assignment statement::
>
>     nonlocal x = 3
>
> The above has exactly the same meaning as ``nonlocal x; x = 3``.

Nice.

> The shorthand form does not allow multiple names.

As you allow

    nonlocal x, y, z

and

    x, y, z = 1, 2, 3

is legal Python (and does the right thing), why not allow

    nonlocal x, y, z = 1, 2, 3

too?

That's one less rule needed to be learned and enforced.



-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From fredrik at pythonware.com  Thu Nov  2 10:14:07 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 02 Nov 2006 10:14:07 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
Message-ID: <eiccsu$v65$1@sea.gmane.org>

Steven Bethard wrote:

> I don't think that'll really be worth it.  I'd be amazed if people
> didn't expect it to mean "top-level".

as anyone who's spent enough time on c.l.python knows, people expect it 
to mean "application global".  it isn't.

alternatively, they expect it to be a scope specifier to be used when 
creating the variable, rather than when referring to it.  it isn't.

the exact meaning of "global" is one of the well-known "you only need to 
learn this once" things in Python.  a reinterpreted global would fall in 
the same category.

</F>


From jan.grant at bristol.ac.uk  Thu Nov  2 10:27:25 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Thu, 2 Nov 2006 09:27:25 +0000 (GMT)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
Message-ID: <20061102092431.H52313@tribble.ilrt.bris.ac.uk>

On Wed, 1 Nov 2006, Steven Bethard wrote:

> I don't think that'll really be worth it.  I'd be amazed if people
> didn't expect it to mean "top-level". The real question is, if people
> see something like this::
> 
>     def f():
>         n = 0
>         def g(i):
>             global n
>             n += i
>         return g
>     func = f()
>     print func(), func()
> 
> what would they expect it to do?

Set "n" in the scope of "func", assuming func is top-level. That global 
is already a lie isn't a feature.

I'd rather see "outer" than "global" but even that can be understood to 
mean "outermost" rather than "the next one out".

jan

PS. If you're hunting for keywords to overload to drag variables into scope, 
"import" is also available, although would probably be very hard to make 
work.

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Not as randy or clumsom as a blaster.

From talin at acm.org  Thu Nov  2 10:42:57 2006
From: talin at acm.org (Talin)
Date: Thu, 02 Nov 2006 01:42:57 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <eiccsu$v65$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<eiccsu$v65$1@sea.gmane.org>
Message-ID: <4549BDA1.4070009@acm.org>

Fredrik Lundh wrote:
> Steven Bethard wrote:
> 
>> I don't think that'll really be worth it.  I'd be amazed if people
>> didn't expect it to mean "top-level".
> 
> as anyone who's spent enough time on c.l.python knows, people expect it 
> to mean "application global".  it isn't.
> 
> alternatively, they expect it to be a scope specifier to be used when 
> creating the variable, rather than when referring to it.  it isn't.
> 
> the exact meaning of "global" is one of the well-known "you only need to 
> learn this once" things in Python.  a reinterpreted global would fall in 
> the same category.

One thing I don't understand in this discussion of re-purposing the 
'global' keyword is how you handle the case of an inner function that 
*creates* a global.

Right now, you can create a global variable from within a scope, even if 
that variable hasn't been declared yet:

    def foo():
       global x
       x = 1

    foo()
    print x

However, if 'global' is simply a synonym for 'nonlocal', then how does 
it know *which* scope to create the variable in?

Let me anticipate one possible answer: "Don't do that". My response 
would be: first, being able to do that is occasionally useful, and 
second, if we can't do it, then we're back to the notion that global 
variables have to be "declared" before they can be used in inner 
functions...not exactly within the theme of Python.

However, it just occurred to me that there's another option, one that's 
not yet included in Ping's PEP: _Named Lexical Scopes_.

What if 'global' was not a keyword, but rather a name? So when you say 
"global x" you really mean that 'x' derives from a lexical scope that 
has the name 'global'.

Inner scopes would be named after the function that defines the scope. 
So for example:

    def foo():
       def bar():
          foo x    # means that 'x' is defined in the 'foo' scope
          x = 1

       bar()
       print x

Admittedly, using a bare non-keyword identifier is probably a bit too 
tricky in terms of parsing; Sure, it could be done, but its likely that 
it might create some parsing ambiguities that might bite us down the 
road. So I'd say go ahead and use a keyword to introduce a reference to 
a named scope, for example:

    def outer():
       def inner1():
          def inner2():
             scope inner1 x, y = 1, 2
             scope outer z = 3
             scope global a = "this is a test"

This syntax allows us to refer to any arbitrary outer scope, regardless 
of whether a local variable has been defined in that scope or not. Its 
also immune to variables changing meaning if the hierarchy of scopes is 
rearranged - for example, if we remove 'inner2' from inside 'inner1', 
the the references to 'outer' and 'global' still mean exactly the same 
thing as they did before.

Also, this scheme does not prevent the compiler from knowing at compile 
time exactly what local variables are defined in which scopes.

-- Talin

From fredrik at pythonware.com  Thu Nov  2 10:59:40 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 02 Nov 2006 10:59:40 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <4549BDA1.4070009@acm.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<eiccsu$v65$1@sea.gmane.org>
	<4549BDA1.4070009@acm.org>
Message-ID: <eicfib$7mj$1@sea.gmane.org>

Talin wrote:

> One thing I don't understand in this discussion of re-purposing the 
> 'global' keyword is how you handle the case of an inner function that 
> *creates* a global.
> 
> Right now, you can create a global variable from within a scope, even if 
> that variable hasn't been declared yet:
> 
>     def foo():
>        global x
>        x = 1
> 
>     foo()
>     print x
> 
> However, if 'global' is simply a synonym for 'nonlocal', then how does 
> it know *which* scope to create the variable in?

since what's a free variable and not is determined by static analysis, 
and free variables are "owned" by the innermost scope they're used in, 
I'm not sure why you even asking that question.

</F>


From solipsis at pitrou.net  Thu Nov  2 17:00:02 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 2 Nov 2006 17:00:02 +0100 (CET)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <E1GfYTs-0002nS-1v@swing.co.at>
References: Your message of "Wed, 01 Nov 2006 19:01:12 CST."
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<E1GfYTs-0002nS-1v@swing.co.at>
Message-ID: <11024.62.39.9.251.1162483202.squirrel@webmail.nerim.net>


> OTOH, having `global` and `nonlocal` mean the same thing for
> module-bound variables violates TOOWTDI.

Then you can disallow "nonlocal" for anything else than closure variables.
And disallow "global" for anything else than module-global variables.
It is explicit and unambiguous, and reduces the probability of scoping bugs.



From g.brandl at gmx.net  Thu Nov  2 17:09:26 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 02 Nov 2006 17:09:26 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <20061102092431.H52313@tribble.ilrt.bris.ac.uk>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<20061102092431.H52313@tribble.ilrt.bris.ac.uk>
Message-ID: <eid57o$m6v$1@sea.gmane.org>

Jan Grant wrote:
> On Wed, 1 Nov 2006, Steven Bethard wrote:
> 
>> I don't think that'll really be worth it.  I'd be amazed if people
>> didn't expect it to mean "top-level". The real question is, if people
>> see something like this::
>> 
>>     def f():
>>         n = 0
>>         def g(i):
>>             global n
>>             n += i
>>         return g
>>     func = f()
>>     print func(), func()
>> 
>> what would they expect it to do?
> 
> Set "n" in the scope of "func", assuming func is top-level. That global 
> is already a lie isn't a feature.
> 
> I'd rather see "outer" than "global" but even that can be understood to 
> mean "outermost" rather than "the next one out".
> 
> jan
> 
> PS. If you're hunting for keywords to overload to drag variables into scope, 
> "import" is also available, although would probably be very hard to make 
> work.

How should that possibly work?

Georg


From g.brandl at gmx.net  Thu Nov  2 17:13:46 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 02 Nov 2006 17:13:46 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
Message-ID: <eid5ft$m6v$2@sea.gmane.org>

Mike Orr wrote:
> The thread on python-dev has been revived, so those interested in this
> subject will want to look in both places.
> 
> On 11/1/06, Talin <talin at acm.org> wrote:
>> Actually I generally use:
>>
>>        p = os.path.normpath( os.path.join( __file__, "../..", "lib" ) )
>>
>> or even:
>>
>>        p = os.path.normpath( os.path.join( __file__, "../../lib" ) )
>>
>> ...which isn't quite as concise as what you wrote, but is better than
>> the first example. (The reason this works is because 'normpath' doesn't
>> know whether the last component is a file or a directory -- it simply
>> interprets the ".." as an instruction to strip off the last component.)
> 
> This illustrates two problems with os.path.  The reason I use all
> these nested functions instead of a simple normpath/join is one is
> told "it's bad to use platform-specific separators".  Perhaps this
> disrecommendation should be lifted, especially since both Mac and
> Windows do the right thing with "/", "..", and "." now.
> 
> The other thing is, ".." and "." seem to be smarter than
> os.path.dirname.  I can't quite articulate the rules but '.' off a
> file chops the filename component, while '.' off a directory does
> nothing.  '..' off a file goes to the file's directory, while '..' off
> a directory goes to the directory's parent.  Dirname() just chops the
> final component without asking what it is, while '..' and '.' do
> different things depending on whether the final component is a
> directory.  I think a method like .ancestor(N) would be useful,
> meaning "do '..' N times.
> 
> /a/b/../c    # Previous component is always a directory, so eliminate.

"." and ".." on a file is meaningless on Unix:

$ ls /etc/passwd/../
/bin/ls: cannot access /etc/passwd/../: Not a directory

Georg


From p.f.moore at gmail.com  Thu Nov  2 17:21:43 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 2 Nov 2006 16:21:43 +0000
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
Message-ID: <79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>

On 11/1/06, Mike Orr <sluggoster at gmail.com> wrote:
> [...] especially since both Mac and
> Windows do the right thing with "/", "..", and "." now.

Not always:

D:\Data>dir C:/
Invalid switch - "".

Paul.

From ronaldoussoren at mac.com  Thu Nov  2 17:25:43 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 2 Nov 2006 17:25:43 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	<79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
Message-ID: <1E120870-D70C-496F-9960-112A4066D02C@mac.com>


On Nov 2, 2006, at 5:21 PM, Paul Moore wrote:

> On 11/1/06, Mike Orr <sluggoster at gmail.com> wrote:
>> [...] especially since both Mac and
>> Windows do the right thing with "/", "..", and "." now.
>
> Not always:
>
> D:\Data>dir C:/
> Invalid switch - "".

And on Mac this depends on the API you use, if you use the Carbon  
API's you still have to use colons as directory separators (and the  
OS does the correct translation for you, which is why you can still  
have a file named "a/b" in the Finder).

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061102/023a8e32/attachment.bin 

From jan.grant at bristol.ac.uk  Thu Nov  2 17:46:06 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Thu, 2 Nov 2006 16:46:06 +0000 (GMT)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <eid57o$m6v$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<20061102092431.H52313@tribble.ilrt.bris.ac.uk>
	<eid57o$m6v$1@sea.gmane.org>
Message-ID: <20061102163948.R52313@tribble.ilrt.bris.ac.uk>

On Thu, 2 Nov 2006, Georg Brandl wrote:

> > PS. If you're hunting for keywords to overload to drag variables into scope, 
> > "import" is also available, although would probably be very hard to make 
> > work.
> 
> How should that possibly work?

Without treading on the toes of the existing import syntax, god knows 
(which is what I meant by "very hard").

The docs claim import exists to "(2) define a name or names in the local 
namespace (of the scope where the import statement occurs)" but since 
outer variable names may shadow module names,

	def f():
	 import x

couldn't be used :-(



-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
We thought time travel was impossible. But that was now and this is then.

From talin at acm.org  Thu Nov  2 18:53:53 2006
From: talin at acm.org (Talin)
Date: Thu, 02 Nov 2006 09:53:53 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <1E120870-D70C-496F-9960-112A4066D02C@mac.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>	<45485815.1030001@acm.org>	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>	<79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
	<1E120870-D70C-496F-9960-112A4066D02C@mac.com>
Message-ID: <454A30B1.6060709@acm.org>

Ronald Oussoren wrote:
> 
> On Nov 2, 2006, at 5:21 PM, Paul Moore wrote:
> 
>> On 11/1/06, Mike Orr <sluggoster at gmail.com> wrote:
>>> [...] especially since both Mac and
>>> Windows do the right thing with "/", "..", and "." now.
>>
>> Not always:
>>
>> D:\Data>dir C:/
>> Invalid switch - "".
> 
> And on Mac this depends on the API you use, if you use the Carbon API's 
> you still have to use colons as directory separators (and the OS does 
> the correct translation for you, which is why you can still have a file 
> named "a/b" in the Finder).

I think you folks are missing the point here - of *course* different 
platforms have different interpretations of path separators.

But that's not what we're talking about here - we are talking about the 
behavior of os.path.normpath().

Look, people have been writing cross-platform Python code for years. How 
is this possible when different platforms have such a radical 
interpretation of path strings? The answer is that Python's path 
manipulation functions are *better* than what the underlying platform 
supplies. os.path doesn't just slavishly copy the semantics of the local 
filesystem, it is far more generous (and more useful).

Using os.path.normpath(), one is able to manipulate path strings in a 
cross-platform way. So if say:

    os.path.normpath( os.path.join( __file__, "../../lib" )

...what I get on both Win32, Linux and OSX (I can't speak for any other 
platform) is exactly the same semantics.

And as far as things like os.pardir goes - you can't put 'os.pardir' in 
a configuration file! There are *lots* of applications where you want to 
share data files across platforms, and not have to revise the path 
strings every time you migrate between Unix and Win32.

In fact, I'll tell you a little secret about Win32 applications - 
although "officially" the path separator for Win32 is backslash, there 
are so many Unix and cross-platform programs ported to Win32 that a 
larger percentage of the programs available on Win32 accept paths in 
either format.

So lets not get bogged down in the strange quirks of how a particular 
platform interprets paths - I am much more interested in being able to 
combine paths together in an intuitive way, and have it "just work" on 
all platforms. And that includes path strings read from config files, 
not just path objects.

-- Talin

From talin at acm.org  Thu Nov  2 19:00:56 2006
From: talin at acm.org (Talin)
Date: Thu, 02 Nov 2006 10:00:56 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <eicfib$7mj$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<eiccsu$v65$1@sea.gmane.org>	<4549BDA1.4070009@acm.org>
	<eicfib$7mj$1@sea.gmane.org>
Message-ID: <454A3258.9010706@acm.org>

Fredrik Lundh wrote:
> Talin wrote:
> 
>> One thing I don't understand in this discussion of re-purposing the 
>> 'global' keyword is how you handle the case of an inner function that 
>> *creates* a global.
>>
>> Right now, you can create a global variable from within a scope, even if 
>> that variable hasn't been declared yet:
>>
>>     def foo():
>>        global x
>>        x = 1
>>
>>     foo()
>>     print x
>>
>> However, if 'global' is simply a synonym for 'nonlocal', then how does 
>> it know *which* scope to create the variable in?
> 
> since what's a free variable and not is determined by static analysis, 
> and free variables are "owned" by the innermost scope they're used in, 
> I'm not sure why you even asking that question.

Right now, 'global' allows you to create a global variable from within a 
function, even if that global does not yet exist:

 >>> def foo():
...    global x
...    x = 1
...
 >>> foo()
 >>> print x
1

If you change the behavior of 'global' to be the same as 'nonlocal' as 
has been proposed, then this effectively becomes impossible - you can no 
longer set any global that hasn't already been pre-declared.

-- Talin

From ronaldoussoren at mac.com  Thu Nov  2 19:12:17 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 2 Nov 2006 19:12:17 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <454A30B1.6060709@acm.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	<79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
	<1E120870-D70C-496F-9960-112A4066D02C@mac.com>
	<454A30B1.6060709@acm.org>
Message-ID: <C67B7AD9-E1C1-4218-AA61-D5BBB74C6C51@mac.com>


On Nov 2, 2006, at 6:53 PM, Talin wrote:

> Ronald Oussoren wrote:
>>
>> On Nov 2, 2006, at 5:21 PM, Paul Moore wrote:
>>
>>> On 11/1/06, Mike Orr <sluggoster at gmail.com> wrote:
>>>> [...] especially since both Mac and
>>>> Windows do the right thing with "/", "..", and "." now.
>>>
>>> Not always:
>>>
>>> D:\Data>dir C:/
>>> Invalid switch - "".
>>
>> And on Mac this depends on the API you use, if you use the Carbon  
>> API's
>> you still have to use colons as directory separators (and the OS does
>> the correct translation for you, which is why you can still have a  
>> file
>> named "a/b" in the Finder).
>
> I think you folks are missing the point here - of *course* different
> platforms have different interpretations of path separators.
>
> But that's not what we're talking about here - we are talking about  
> the
> behavior of os.path.normpath().
>
> Look, people have been writing cross-platform Python code for  
> years. How
> is this possible when different platforms have such a radical
> interpretation of path strings? The answer is that Python's path
> manipulation functions are *better* than what the underlying platform
> supplies. os.path doesn't just slavishly copy the semantics of the  
> local
> filesystem, it is far more generous (and more useful).
>
> Using os.path.normpath(), one is able to manipulate path strings in a
> cross-platform way. So if say:
>
>     os.path.normpath( os.path.join( __file__, "../../lib" )
>
> ...what I get on both Win32, Linux and OSX (I can't speak for any  
> other
> platform) is exactly the same semantics.

That happens to work because all tree use '..' to represent a parent  
directory and happen to accept forward slashes as a directory  
seperator.  Your example doesn't work when you use macpath to  
manipulate classic OS9-style paths.

I agree that the functions in os.path are a good abstraction for  
platform-independent path manipulation, even if your example is not.  
A better cross-platform way to write the call above is:
     os.path.join( os.path.dirname( os.path.dirname( os.path.abspath 
(__file__)))).


>
> And as far as things like os.pardir goes - you can't put  
> 'os.pardir' in
> a configuration file! There are *lots* of applications where you  
> want to
> share data files across platforms, and not have to revise the path
> strings every time you migrate between Unix and Win32.
>
> In fact, I'll tell you a little secret about Win32 applications -
> although "officially" the path separator for Win32 is backslash, there
> are so many Unix and cross-platform programs ported to Win32 that a
> larger percentage of the programs available on Win32 accept paths in
> either format.

Most windows APIs accept either separator, but IIRC the actual kernel  
API uses backslashes only, which leaks through at some points.  It  
has been a while since I last looked at this (and scared collegues  
who's code I was testing), but I think it was the "really" absolute  
paths that don't accept forward slashes (paths like \\.\C:\Windows).  
This was with NT 4 and from the top of my head, so I may well be wrong.


>
> So lets not get bogged down in the strange quirks of how a particular
> platform interprets paths - I am much more interested in being able to
> combine paths together in an intuitive way, and have it "just work" on
> all platforms. And that includes path strings read from config files,
> not just path objects.

If you want that you'll have to define some way to convert between a  
generic path representation to the native one.

Ronald, who doesn't really want to get sucked into this discussion.


P.S. I agree with the sentiment that several others have raised:  
please work on a better os.path as a seperate library and come back  
when that has proven itself in the field.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061102/3cbbb84b/attachment.bin 

From fredrik at pythonware.com  Thu Nov  2 19:32:19 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 02 Nov 2006 19:32:19 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454A3258.9010706@acm.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<eiccsu$v65$1@sea.gmane.org>	<4549BDA1.4070009@acm.org>	<eicfib$7mj$1@sea.gmane.org>
	<454A3258.9010706@acm.org>
Message-ID: <eiddjg$qtg$1@sea.gmane.org>

Talin wrote:

> If you change the behavior of 'global' to be the same as 'nonlocal' as 
> has been proposed, then this effectively becomes impossible - you can no 
> longer set any global that hasn't already been pre-declared.

of course you can, as long as it isn't shadowed in an intermediate scope.

are you sure you know how free variables are handled in today's Python?

</F>


From python at zesty.ca  Thu Nov  2 21:34:24 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Thu, 2 Nov 2006 14:34:24 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <eiddjg$qtg$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<eiccsu$v65$1@sea.gmane.org> <4549BDA1.4070009@acm.org>
	<eicfib$7mj$1@sea.gmane.org>
	<454A3258.9010706@acm.org> <eiddjg$qtg$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.58.0611021432060.3499@server1.LFW.org>

On Thu, 2 Nov 2006, Fredrik Lundh wrote:
> Talin wrote:
>
> > If you change the behavior of 'global' to be the same as 'nonlocal' as
> > has been proposed, then this effectively becomes impossible - you can no
> > longer set any global that hasn't already been pre-declared.
>
> of course you can, as long as it isn't shadowed in an intermediate scope.

I think the reason you wouldn't be able to create new globals is that
Guido wants this to be an error (for good reasons IMHO):

    http://mail.python.org/pipermail/python-dev/2006-July/066968.html

|   Perhaps the best solution would be to make it an error if there
|   wasn't a visible variable named a in an outer scope.


-- ?!ng

From tdelaney at avaya.com  Thu Nov  2 22:18:03 2006
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Fri, 3 Nov 2006 08:18:03 +1100
Subject: [Python-3000] Draft PEP for outer scopes
Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1EB3E@au3010avexu1.global.avaya.com>

Steven Bethard wrote:

> On 11/1/06, Ka-Ping Yee <python at zesty.ca> wrote:
>> I think a fairer survey example would be something like this:
>> 
>>     n = 1
>>     def f():
>>         n = 2
>>         def g():
>>             global n = 99
>>             return n
>>         g()
>> 
>>     f()
>>     print n
>> 
>> Which 'n' do you expect g() to change?
> 
> The only reason I didn't use that in the first place is that is seems
> extremely unlikely in real code.  Who uses the same name for a
> module-level binding and a function-local binding?
> 
> So yes, that's the corner case, but I contend that the corner case
> will almost never come up.

A much more likely corner case is (using existing syntax):

def f():
    def g()
        global n
        n = 99
        return n

    g()

f()
print n

What does this do? It depends on if f() has a binding of "n". If it
does, the above throws a NameError. If it doesn't, it prints 99.

For "nonlocal", this is a non-issue - it's a syntax error. But if we
reuse "global", the nested function is context-dependent.

If "global" were to change in Py3K to require an existing binding at the
time the "global" keyword was executed, this would then have the
semantics of "nonlocal" and be context-independent again. So I think any
proposal to reuse "global" has to include these semantics.

Tim Delaney

From jcarlson at uci.edu  Thu Nov  2 23:19:27 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 02 Nov 2006 14:19:27 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5FF1EB3E@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5FF1EB3E@au3010avexu1.global.avaya.com>
Message-ID: <20061102141514.C115.JCARLSON@uci.edu>


"Delaney, Timothy (Tim)" <tdelaney at avaya.com> wrote:
> If "global" were to change in Py3K to require an existing binding at the
> time the "global" keyword was executed, this would then have the
> semantics of "nonlocal" and be context-independent again. So I think any
> proposal to reuse "global" has to include these semantics.

Maybe, but it would break currently existing (and working) code like the
following...

#no foo = declaration

def init_foo():
    global foo
    foo = ...

I use variants of the above in a few places.  Could I initialize foo =
None in the module namespace prior to running init_foo()?  Sure, but
that may or may not clutter up definitions contained in the module.

 - Josiah


From greg.ewing at canterbury.ac.nz  Fri Nov  3 00:38:06 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Nov 2006 12:38:06 +1300
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
Message-ID: <454A815E.80704@canterbury.ac.nz>

Mike Orr wrote:
> Perhaps this
> disrecommendation should be lifted, especially since both Mac and
> Windows do the right thing with "/", "..", and "." now.

Is anyone still supporting VMS? It has a wildly
different pathname syntax.

In any case, if we all start writing code that
makes assumptions about pathname syntax, and
some platform comes along in the future that
does things differently, we're hosed. Do we
want to take that risk?

> The other thing is, ".." and "." seem to be smarter than
> os.path.dirname.

You can certainly get different results if you
interpret a path containing . or .. through the
file system than if you treat it as a pure
pathname manipulation. But the same will be
true of any OO replacement that also restricts
itself to pathname manipulation -- it won't be
able to treat . and .. any more smartly than
os.path does.

> '.' off a file chops the filename component, 

Not sure what you mean by that. On Unix, if you
try to open "foo.c/." where "foo.c" is not a
directory, you get an exception:

    >>> open("foo.c/.")
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    IOError: [Errno 20] Not a directory: 'foo.c/.'

But os.path has no idea what foo.c is, so it
just treats joining "." to a path and then
normalising as a no-op.

> '..' off a file goes to the file's directory, while '..' off
> a directory goes to the directory's parent.

These aren't different things -- in both cases
you're getting the directory that contains the
object in question.

 > I think a method like .ancestor(N) would be useful,
> meaning "do '..' N times.

That would be handy, yes.

--
Greg

From qrczak at knm.org.pl  Fri Nov  3 02:03:03 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Fri, 03 Nov 2006 02:03:03 +0100
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <454A815E.80704@canterbury.ac.nz> (Greg Ewing's message of
	"Fri, 03 Nov 2006 12:38:06 +1300")
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	<454A815E.80704@canterbury.ac.nz>
Message-ID: <87velxibp4.fsf@qrnik.zagroda>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

> In any case, if we all start writing code that makes assumptions
> about pathname syntax, and some platform comes along in the future
> that does things differently, we're hosed.

Some conventions in the world are converging.

For example a byte universally has 8 bits, and a machine word is
always 8 bits times some power of 2. This used to vary.

EBCDIC is nearly dead, everything is a superset of ASCII. Unicode
becomes more common.

It's very likely that any new OS, if it uses textual filenames at all,
will use either the Unix filename convention, or - if it's made by
Microsoft - a bastardized Unix convention.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From greg.ewing at canterbury.ac.nz  Fri Nov  3 02:20:47 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Nov 2006 14:20:47 +1300
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
Message-ID: <454A996F.6090400@canterbury.ac.nz>

Steven Bethard wrote:
> Ka-Ping Yee wrote:
> 
>>Would it help at all to survey some folks to see how many interpret
>>"global variable" to mean "top-level" vs. "anything nonlocal"?
> 
> I don't think that'll really be worth it.  I'd be amazed if people
> didn't expect it to mean "top-level".

Another confounding factor could be how familiar the
respondent is with languages that have more than two
scopes in the first place. If most of their programming
experience is with C, they're likely to equate global
with top-level simply because there's no other
possibility in C.

Besides, the BDFL is going to do whatever he wants
anyway, so let's just leave him to get on with it. :-)

--
Greg


From python at zesty.ca  Fri Nov  3 02:37:18 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Thu, 2 Nov 2006 19:37:18 -0600 (CST)
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org> 
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com> 
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org> 
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com> 
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>

Before it is reasonable to change the meaning of "global", we would
need to have coherent answers to these questions:

    1.  What is the global namespace?

    2.  Do global variables belong to the global namespace?

    3.  When it says "global x" does that mean x is a global variable?

I think the folks who support changing "global" to mean "innermost
existing nonlocal" really have to provide answers to these.  They
are bound to become FAQs the moment such a change takes place.

Can you imagine trying to teach this?

My main problem with changing the meaning of "global" is that doing
so causes one of these answers to be nonsense.  Maybe i fail to see
where our trains of thought diverge.  It seems to me, either you have
to say (#1) there is no longer such a thing as "the global namespace",
or (#2) global variables have nothing to do with the global namespace,
or (#3) "global x" has nothing to do with whether x is global.

In all cases, well established Python terminology is thrown into
confusion.  It'll be even worse if we don't all agree on *which* of
these three answers to make nonsense -- if we don't all choose the
*same* nonsense, no one will know what anyone means by "global
variable".  The phrase "the global namespace" becomes meaningless or
fraught with ambiguity.

Or, in a concrete example:

    def f():
        n = 2
        def spam():
            global n = 99
        spam()

    1.  Is the "n" affected by spam() a global variable?

    2.  In what namespace does this "n" reside?

    3.  Is that what you would call the "global namespace"?

    4.  If not, which namespace is the "global namespace"?

(You can't just say "oh, no one will ever write code like that."
There have to be sensible answers -- teaching examples look like
this, and tests test this, and curious students will want to know.)

If the intention is to persuade Python programmers that the word
"global" could refer to any of several namespaces, that would break
compatibility not just with earlier versions of the language but
with all the documentation and educational materials out there that
mention "global variable" or "global namespace".

Parts of the reference manual and even the codebase would also
become confusing or misleading as a result -- identifiers containing
the word "global" or comments mentioning the "global namespace"
would no longer make sense.

That seems like a lot of unnecessary work and a losing battle, IMHO.
The battle will only hurt Python users who, henceforth, whenever
they hear "global" can no longer tell which namespace you're
talking about.

And what are we gaining for this cost?  So far there's no positive
argument that "global" is actually the right word for this purpose --
defenders of "global" only seem to be saying that it's not a totally
awful word for this purpose.  Why tolerate something that is merely
barely understandable (if you try hard enough to ignore what the rest
of the world means by "global"), just to save a keyword?  This is a
new kind of variable access for Python; it deserves a new keyword.
(Though "nonlocal" is my preference, i would take just about anything
over "global".)  I thought Py3K was about a fresh chance to make good
choices, instead of getting by with tolerable ones.


-- ?!ng

From greg.ewing at canterbury.ac.nz  Fri Nov  3 02:53:17 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Nov 2006 14:53:17 +1300
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <4549BDA1.4070009@acm.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<eiccsu$v65$1@sea.gmane.org> <4549BDA1.4070009@acm.org>
Message-ID: <454AA10D.9070903@canterbury.ac.nz>

Talin wrote:

> Right now, you can create a global variable from within a scope, even if 
> that variable hasn't been declared yet:

That doesn't seem like a good practice from a
readability point of view to me, but in any
case, there's no reason it couldn't continue
to be supported:

If you don't find any assigment to the name
in an enclosing scope, just assume it's
module-level.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov  3 03:00:51 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Nov 2006 15:00:51 +1300
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <454A30B1.6060709@acm.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	<79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
	<1E120870-D70C-496F-9960-112A4066D02C@mac.com>
	<454A30B1.6060709@acm.org>
Message-ID: <454AA2D3.5020706@canterbury.ac.nz>

Talin wrote:
> And that includes path strings read from config files, 
> not just path objects.

This is something new. I hadn't realised anyone
was considering that to be within the scope of
any of the proposed path modules.

Obviously this is going to require defining a
standard external representation for paths.
What do you envisage such a representation
being like?

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov  3 03:19:33 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Nov 2006 15:19:33 +1300
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
Message-ID: <454AA735.4080406@canterbury.ac.nz>

Ka-Ping Yee wrote:
> Before it is reasonable to change the meaning of "global", we would
> need to have coherent answers to these questions:
> 
>     1.  What is the global namespace?

Under the proposal, there is no such thing as
"the" global namespace, so the question is
meaningless.

There's no such thing now, either -- there
are just (multiple) module-level namespaces.

>     2.  Do global variables belong to the global namespace?

Also meaningless as a corollary of 1.

>     3.  When it says "global x" does that mean x is a global variable?

Yes, from the point of view of the scope containing
the "global" declaration, it is. From the point of
view of a different scope, it might not be.

> Can you imagine trying to teach this?

With a few well-chosen examples, yes.

> either you have
> to say (#1) there is no longer such a thing as "the global namespace",

As I said, there isn't one now. Which namespace
is "the global namespace" depends on which module
you're looking from. Under the proposal, it would
depend on which scope you're looking from.

--
Greg

From talin at acm.org  Fri Nov  3 03:42:02 2006
From: talin at acm.org (Talin)
Date: Thu, 02 Nov 2006 18:42:02 -0800
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <454AA735.4080406@canterbury.ac.nz>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
Message-ID: <454AAC7A.30906@acm.org>

Greg Ewing wrote:
> Ka-Ping Yee wrote:
>> Before it is reasonable to change the meaning of "global", we would
>> need to have coherent answers to these questions:
>>
>>     1.  What is the global namespace?
> 
> Under the proposal, there is no such thing as
> "the" global namespace, so the question is
> meaningless.
> 
> There's no such thing now, either -- there
> are just (multiple) module-level namespaces.

I think that the argument that "there's no such thing as a global 
namespace, either now or in the future" are using an overly pedantic 
definition of the word "global".

In Python "global" means, and has always meant, the namespace of the 
current module. It is the namespace that is "outside of any function 
definition", and that is true regardless of which module we are talking 
about. It is a "special" namespace because it has different semantics 
than the namespaces that are defined within a function scope.

You can't just hand-wave this away - the 'global' namespace is far too 
important and useful to be just dismissed out of hand.

I think the confusion stems from the fact that in many contexts, the 
word 'global' is a synonym for 'universal'. However, there's more than 
one planet in the universe...and no one is arguing that global variables 
in Python are in any way 'universal' variables.

-- Talin

From talin at acm.org  Fri Nov  3 04:09:40 2006
From: talin at acm.org (Talin)
Date: Thu, 02 Nov 2006 19:09:40 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <454AA2D3.5020706@canterbury.ac.nz>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	<79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
	<1E120870-D70C-496F-9960-112A4066D02C@mac.com>
	<454A30B1.6060709@acm.org> <454AA2D3.5020706@canterbury.ac.nz>
Message-ID: <454AB2F4.8020108@acm.org>

Greg Ewing wrote:
> Talin wrote:
>> And that includes path strings read from config files, not just path 
>> objects.
> 
> This is something new. I hadn't realised anyone
> was considering that to be within the scope of
> any of the proposed path modules.
> 
> Obviously this is going to require defining a
> standard external representation for paths.
> What do you envisage such a representation
> being like?

Hmmm, maybe it is new.

Let me take a moment to describe my motivations in this discussion. One 
of the reasons I really enjoy writing in Python is that I don't have to 
care what platform I am on. I use OS X at home and Win32 at work -- I'd 
like to be able to write code that works transparently on both systems.

It doesn't matter to me whether paths are represented as objects, or as
strings. What I'd really like is to be able to represent and manipulate
paths in a platform-agnostic way, and do so with code that is both easy
to read and concise.

But that model can't be restricted to just the objects contained within
the Python interpreter, we have to be able to read and write paths that
are coming from the external environment. In many instances, those will 
be paths that are in a platform-specific format; but, given a choice, I
am sure that many of us would prefer our data to be platform-neutral.

One thing to note is that there doesn't have to be just one single 
platform neutral representation. For example, lets suppose we're writing 
a Python script to interpret Perforce configuration files (which happens 
a lot in my environment.) A typical Perforce mapping has two parts, a 
depot part and a local part:

//depot/projects/metabuilder/src/... //myroot/dev/src/...

These 'paths' are the same regardless of what host you happen to be 
running on. If you plan on doing anything intelligent with them (like 
scanning for differences with the locally checked-out files or 
something), you'll probably want to translate them into some form that 
the local filesystem can interpret.

So here we have an example of an already-existing, platform-neutral 
representation of paths, but I would never suggest that Python adopt 
wholesale the Perforce representation of paths as the only 
platform-neutral solution.

One might ask - is it the responsibility of the Python standard path 
module to even deal with such things? Since these are not sanctioned 
operating system paths, why should Python have a responsibility to 
support them? That's a good argument, but on the other hand - it's 
awfully convenient to be able to use the existing os.path to process 
stuff like this, especially considering the possibility that folks who 
roll their own Perforce-specific path functionality might 'get it wrong' 
in some subtle way.

One way to solve this would be to define a platform-neutral path 
representation that can be "rendered" into either os-specific *or* 
application-specific formats. I could read a Perforce path string, 
convert that into an abstract path, do some work on it, and then convert 
it back to a Perforce path - or convert to a local filesystem path if I 
choose. Basically we'd have importers and exporters (format converters) 
for paths. On Win32, for example, we could have an "NT" importer for 
standard NT-style paths; a "Cygwin" importer that was more relaxed about 
a mixture of forward- and back-slashes; and so on.

Of course, there would always be a "shortcut" for the local filesystem 
format - i.e., if no conversion was specified, then the local host 
format would be used as the default.

How such importers and exporters would actually look syntactically, I am 
not sure yet. I'm open to ideas...

-- Talin

From sluggoster at gmail.com  Fri Nov  3 04:23:43 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Thu, 2 Nov 2006 19:23:43 -0800
Subject: [Python-3000] Path Reform: Get the ball rolling
In-Reply-To: <454AB2F4.8020108@acm.org>
References: <6e9196d20610312222o1a348eaeo8c41ddba6c0331c9@mail.gmail.com>
	<45485815.1030001@acm.org>
	<6e9196d20611011247k1b918ba2ga3b12cfca1cb7469@mail.gmail.com>
	<79990c6b0611020821x6f7fcf12yd1c5d1a846be62ff@mail.gmail.com>
	<1E120870-D70C-496F-9960-112A4066D02C@mac.com>
	<454A30B1.6060709@acm.org> <454AA2D3.5020706@canterbury.ac.nz>
	<454AB2F4.8020108@acm.org>
Message-ID: <6e9196d20611021923l6faf2076u751969871e5eee8f@mail.gmail.com>

On 11/2/06, Talin <talin at acm.org> wrote:
> One way to solve this would be to define a platform-neutral path
> representation that can be "rendered" into either os-specific *or*
> application-specific formats. I could read a Perforce path string,
> convert that into an abstract path, do some work on it, and then convert
> it back to a Perforce path - or convert to a local filesystem path if I
> choose. Basically we'd have importers and exporters (format converters)
> for paths. On Win32, for example, we could have an "NT" importer for
> standard NT-style paths; a "Cygwin" importer that was more relaxed about
> a mixture of forward- and back-slashes; and so on.
>
> Of course, there would always be a "shortcut" for the local filesystem
> format - i.e., if no conversion was specified, then the local host
> format would be used as the default.
>
> How such importers and exporters would actually look syntactically, I am
> not sure yet. I'm open to ideas...

This sounds like a use case for a directory-component path object.  If
it treats paths as sequences like ['usr', 'local', 'lib', 'python2.4']
and you had to input them that way too, it would be a platform-neutral
format. You'd still need to specify absolute/relative path, and in
some cases choose the root for an absolute path, which is why Noam's
module uses a special "root object" for the first element.

This weekend I plan to code up a MiniPath containing the
pathname-manipulation features of PEP 355, a MediumPath subclass
adding the filesystem-dependent methods that some ppl frown on, and a
MaxiPath subclass with my favorite additional methods.  I'm thinking
of making the path module a class attribute, so that you can subclass
any of these and use ntpath or macpath instead of your default if you
want.  I'll also try my hand at Glyph's "safe join" feature if I can
get it to work right, but that will be an optional separate method:
either a .child instance method or a class method (not sure which is
better).

-- 
Mike Orr <sluggoster at gmail.com>

From python at zesty.ca  Fri Nov  3 08:50:46 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Fri, 3 Nov 2006 01:50:46 -0600 (CST)
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <454AA735.4080406@canterbury.ac.nz>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
Message-ID: <Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>

On Fri, 3 Nov 2006, Greg Ewing wrote:
> > Before it is reasonable to change the meaning of "global", we would
> > need to have coherent answers to these questions:
> >
> >     1.  What is the global namespace?
>
> Under the proposal, there is no such thing as
> "the" global namespace, so the question is
> meaningless.
>
> There's no such thing now, either -- there
> are just (multiple) module-level namespaces.

Okay... tell that to all the writers of Python books and webpages
that mention "the global namespace".

Should we remove any occurrences of this phrase from the official
Python documentation as well?

> > either you have
> > to say (#1) there is no longer such a thing as "the global namespace",
>
> As I said, there isn't one now. Which namespace
> is "the global namespace" depends on which module
> you're looking from. Under the proposal, it would
> depend on which scope you're looking from.

No -- it's much worse than that.  That's the problem: even if you
know which scope you're looking from, you wouldn't be able to point
to anything and say "that's the global namespace".  If we repurpose
"global" to mean "nonlocal", the declaration "global a, b, c" might
mean that a, b, and c all belong to different namespaces.

Right now, in any given context, "the global namespace" has a clear
meaning: the namespace shared by the whole file.  With the new meaning
of the "global" keyword, "global namespace" becomes meaningless no
matter what the context.  In Python, the concepts of "global", "global
variable", and "global namespace" are too well established to destroy.


-- ?!ng

From fredrik at pythonware.com  Fri Nov  3 09:53:18 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 03 Nov 2006 09:53:18 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454A996F.6090400@canterbury.ac.nz>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<454A996F.6090400@canterbury.ac.nz>
Message-ID: <eif01u$d85$1@sea.gmane.org>

Greg Ewing wrote:

> Besides, the BDFL is going to do whatever he wants
> anyway, so let's just leave him to get on with it. :-)

but where is he?  has anyone seen him lately ?

</F>


From nnorwitz at gmail.com  Fri Nov  3 09:57:18 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 3 Nov 2006 00:57:18 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <eif01u$d85$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<454A996F.6090400@canterbury.ac.nz> <eif01u$d85$1@sea.gmane.org>
Message-ID: <ee2a432c0611030057y6dfc3f54pbcd5cab5f663f0d0@mail.gmail.com>

On 11/3/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Greg Ewing wrote:
>
> > Besides, the BDFL is going to do whatever he wants
> > anyway, so let's just leave him to get on with it. :-)
>
> but where is he?  has anyone seen him lately ?

Sure, he's been busy at work and sick recently.  He'll probably
resurface in the next couple of weeks or so.

n

From ncoghlan at gmail.com  Fri Nov  3 11:12:05 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 03 Nov 2006 20:12:05 +1000
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
Message-ID: <454B15F5.2090303@gmail.com>

Ka-Ping Yee wrote:
> Right now, in any given context, "the global namespace" has a clear
> meaning: the namespace shared by the whole file.  With the new meaning
> of the "global" keyword, "global namespace" becomes meaningless no
> matter what the context.  In Python, the concepts of "global", "global
> variable", and "global namespace" are too well established to destroy.

Exactly.

A lookup of a bare name can be resolved in one of the following kinds of 
namespace:

1. function locals (either the current function or an enclosing function)
2. module globals
3. builtins

The new PEP is about being able to better distinguish the case where the name 
is being resolved in an enclosing function's local namespace instead of the 
current function's local namespace. Simply moving the ambiguity to the second 
entry in the list above is NOT an improvement!

A new keyword, on the other hand, allows the list to be split relatively 
cleanly into four possible locations:

1. function locals of the current function (local variables)
2. function locals of an enclosing function (closure variables)
3. module globals (global variables)
4. builtins (builtin names)

A name for the second category other than 'nonlocal' would be nice, but I find 
it infinitely preferable to trying to redefine global. The only requirement I 
would place on it is that "nonlocal n" when there is no 'n' defined in a 
surrounding scope should be a SyntaxError.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From talin at acm.org  Fri Nov  3 12:17:00 2006
From: talin at acm.org (Talin)
Date: Fri, 03 Nov 2006 03:17:00 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <eif01u$d85$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<454A996F.6090400@canterbury.ac.nz>
	<eif01u$d85$1@sea.gmane.org>
Message-ID: <454B252C.7060304@acm.org>

Fredrik Lundh wrote:
> Greg Ewing wrote:
> 
>> Besides, the BDFL is going to do whatever he wants
>> anyway, so let's just leave him to get on with it. :-)
> 
> but where is he?  has anyone seen him lately ?

Also bear in mind that the BDFL doesn't simply make pronouncements in a 
vacuum...he really does take these arguments into account (when he has 
time to read them) which influence his decisions.

(You see, instead of the classic angel and devil hovering over his 
shoulders, he has little Fredrik Lundhs and Greg Ewings and Nick 
Coghlans and all the others, arguing the fine points of pythonicity, 
each trying to sway him to to their point of view...

...the poor ba***rd.)

-- Talin

From rrr at ronadam.com  Fri Nov  3 20:55:50 2006
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 03 Nov 2006 13:55:50 -0600
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
Message-ID: <eig735$ovr$1@sea.gmane.org>

Ka-Ping Yee wrote:
> On Fri, 3 Nov 2006, Greg Ewing wrote:
>>> Before it is reasonable to change the meaning of "global", we would
>>> need to have coherent answers to these questions:
>>>
>>>     1.  What is the global namespace?
>> Under the proposal, there is no such thing as
>> "the" global namespace, so the question is
>> meaningless.
>>
>> There's no such thing now, either -- there
>> are just (multiple) module-level namespaces.
> 
> Okay... tell that to all the writers of Python books and webpages
> that mention "the global namespace".
> 
> Should we remove any occurrences of this phrase from the official
> Python documentation as well?
> 
>>> either you have
>>> to say (#1) there is no longer such a thing as "the global namespace",
>> As I said, there isn't one now. Which namespace
>> is "the global namespace" depends on which module
>> you're looking from. Under the proposal, it would
>> depend on which scope you're looking from.
> 
> No -- it's much worse than that.  That's the problem: even if you
> know which scope you're looking from, you wouldn't be able to point
> to anything and say "that's the global namespace".  If we repurpose
> "global" to mean "nonlocal", the declaration "global a, b, c" might
> mean that a, b, and c all belong to different namespaces.
> 
> Right now, in any given context, "the global namespace" has a clear
> meaning: the namespace shared by the whole file.  With the new meaning
> of the "global" keyword, "global namespace" becomes meaningless no
> matter what the context.  In Python, the concepts of "global", "global
> variable", and "global namespace" are too well established to destroy.
> 
> 
> -- ?!ng

I'll put in a 'Me Too' here.  (For what it's worth.)  You've convinced me that 
reusing global in verbatim for both global and nested parent scopes would create 
a confusing difference between 2.x and 3.x versions at least in documentation.


How about limiting nonlocal to just the immediate parent scope and using 
'parent' as the keyword?

Are there many common use cases for accessing grandparent name spaces?  That 
might still be done by repeating 'parent' declarations in nested functions for a 
name and thereby pulling access to it down the tree.  That would be vary 
explicit and since it's not something you would do very often, it wouldn't be 
much of a bother.

Limiting it to only the parent scope may also limit possible unintended side 
effects in the case a name is not in the scope that was expected.  (Possibly due 
to omission or deleting it at some point while editing.)

Cheers,
    Ron






From mbk.lists at gmail.com  Fri Nov  3 21:09:44 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Fri, 3 Nov 2006 13:09:44 -0700
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eig735$ovr$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<eig735$ovr$1@sea.gmane.org>
Message-ID: <da7032ce0611031209y37ac5b75t333fa99a301b2478@mail.gmail.com>

> How about [...] using 'parent' as the keyword?

I once proposed 'parent' in a similar thread a long time ago.

It's a non-starter because it's a commonly used variable name.

   Mike

From rrr at ronadam.com  Fri Nov  3 21:11:42 2006
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 03 Nov 2006 14:11:42 -0600
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <da7032ce0611031209y37ac5b75t333fa99a301b2478@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>
	<da7032ce0611031209y37ac5b75t333fa99a301b2478@mail.gmail.com>
Message-ID: <eig80s$rl6$1@sea.gmane.org>

Mike Krell wrote:
>> How about [...] using 'parent' as the keyword?
> 
> I once proposed 'parent' in a similar thread a long time ago.
> 
> It's a non-starter because it's a commonly used variable name.
> 
>    Mike


I thought it might be.

How about.   __parent__


Just kidding,  (I think),  of course that looks too much like a method name.

    Ron


From ntoronto at cs.byu.edu  Fri Nov  3 21:28:07 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 03 Nov 2006 13:28:07 -0700
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eig80s$rl6$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>	<da7032ce0611031209y37ac5b75t333fa99a301b2478@mail.gmail.com>
	<eig80s$rl6$1@sea.gmane.org>
Message-ID: <454BA657.6080404@cs.byu.edu>

Ron Adam wrote:
> Mike Krell wrote:
>   
>>> How about [...] using 'parent' as the keyword?
>>>       
>> I once proposed 'parent' in a similar thread a long time ago.
>>
>> It's a non-starter because it's a commonly used variable name.
>>
>>    Mike
>>     
>
>
> I thought it might be.
>
> How about.   __parent__
>
>
> Just kidding,  (I think),  of course that looks too much like a method name.
>   

Heck - just call it mijn_ouder or something. That even *looks* like 
"outer," and I'll bet it's not in anybody's code, either...

Neil


From qrczak at knm.org.pl  Fri Nov  3 21:33:18 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Fri, 03 Nov 2006 21:33:18 +0100
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eig735$ovr$1@sea.gmane.org> (Ron Adam's message of "Fri, 03
	Nov 2006 13:55:50 -0600")
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<eig735$ovr$1@sea.gmane.org>
Message-ID: <877iycp8xd.fsf@qrnik.zagroda>

Ron Adam <rrr at ronadam.com> writes:

> How about limiting nonlocal to just the immediate parent scope and
> using 'parent' as the keyword?

Please don't put unnecessary limits.

There are two ways to refer to a variable: read it or assign it.
Currently reading doesn't create a new variable, while assigning does
(at the beginning of the function which assigns it).

'nonlocal' should mean: refer to the same variable as would be
referred to if the current function did not assign it, only read it.

I don't care much whether 'global' will be retained as is, or whether
'nonlocal' will actually be spelled 'global'. If I had to decide,
I would keep 'global' with its current meaning because of the name:
"global" means here "a single variable which exists during the whole
duration of the program", not "visible from everywhere", which is
appropriate for the current meaning but not for the proposed 'nonlocal'.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From ntoronto at cs.byu.edu  Fri Nov  3 21:36:33 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 03 Nov 2006 13:36:33 -0700
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
Message-ID: <454BA851.4010200@cs.byu.edu>

Ka-Ping Yee wrote:
> Hi folks,
>
> I have finally completed a draft of a PEP on rebinding of names
> in outer scopes.  I've tried to go back and gather all of the
> (amazingly numerous) proposals -- if i've forgotten or misattributed
> any, let me know and i'll be happy to correct them.
>
> I look forward to your thoughts on it:

Is there any particular reason this *must* be a Py3k PEP? (Aside from 
the fact that we're discussing it on the Py3k list, of course.) Some of 
the solutions discussed for historical context would *definitely* be bad 
ideas for a 2.x (as would changing the semantics of 'global'), but I 
can't see why the solution proposed in the PEP couldn't be done sooner.

Neil


From python at zesty.ca  Fri Nov  3 21:58:58 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Fri, 3 Nov 2006 14:58:58 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454BA851.4010200@cs.byu.edu>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<454BA851.4010200@cs.byu.edu>
Message-ID: <Pine.LNX.4.58.0611031457400.6331@server1.LFW.org>

On Fri, 3 Nov 2006, Neil Toronto wrote:
> Is there any particular reason this *must* be a Py3k PEP? (Aside from
> the fact that we're discussing it on the Py3k list, of course.) Some of
> the solutions discussed for historical context would *definitely* be bad
> ideas for a 2.x (as would changing the semantics of 'global'), but I
> can't see why the solution proposed in the PEP couldn't be done sooner.

Well, i guess that's true.  It doesn't seem out of the question for 2.x.


-- ?!ng

From python at zesty.ca  Fri Nov  3 22:19:21 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Fri, 3 Nov 2006 15:19:21 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <E1GfYMP-0002mn-5b@swing.co.at>
References: <E1GfYMP-0002mn-5b@swing.co.at>
Message-ID: <Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>

On Thu, 2 Nov 2006, Christian Tanzer wrote:
> As you allow
>
>     nonlocal x, y, z
>
> and
>
>     x, y, z = 1, 2, 3
>
> is legal Python (and does the right thing), why not allow
>
>     nonlocal x, y, z = 1, 2, 3
>
> too?
>
> That's one less rule needed to be learned and enforced.

I suppose that's fine.  Also i guess

    nonlocal x = y = 0

seems okay.  We have to be clear that you can't assign to anything
that you could in a normal assignment though -- e.g.:

    nonlocal foo.bar, a[5] = 1, 2

is not allowed.

I've updated the PEP to allow multiple assignment and to specify
the exact grammar of the "nonlocal" statement.

    http://zesty.ca/python/pep-3104.html


-- ?!ng

From solipsis at pitrou.net  Fri Nov  3 22:51:56 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 03 Nov 2006 22:51:56 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
Message-ID: <1162590716.4428.23.camel@fsol>


Le vendredi 03 novembre 2006 ? 15:19 -0600, Ka-Ping Yee a ?crit :
> I suppose that's fine.  Also i guess
> 
>     nonlocal x = y = 0

That's ambiguous.
Is only "x" nonlocal, or are both "x" and "y"?

What's the point anyway? if you write "nonlocal x = 0", it means you
aren't reusing the variable value between calls...
Of course the variable could be re-used by another inner function, but
it's not common to have lots of inner functions sharing variables of a
single outer function (people would rather write a class).



From python at zesty.ca  Fri Nov  3 23:22:42 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Fri, 3 Nov 2006 16:22:42 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <1162590716.4428.23.camel@fsol>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
	<1162590716.4428.23.camel@fsol>
Message-ID: <Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>

On Fri, 3 Nov 2006, Antoine Pitrou wrote:
> Le vendredi 03 novembre 2006 ? 15:19 -0600, Ka-Ping Yee a ?crit :
> > I suppose that's fine.  Also i guess
> >
> >     nonlocal x = y = 0
>
> That's ambiguous.
> Is only "x" nonlocal, or are both "x" and "y"?

Only x is nonlocal.

    nonlocal x = y = 0

is equivalent to

    nonlocal x; x = y = 0

> What's the point anyway?

The point is to make the statement easy to explain.  As Christian
suggested, prohibiting multiple assignment is an exception: an
additional rule to remember.  I found his argument convincing, and
realized that prohibiting chained assignment is another exception.

Consistently allowing these forms of assignment makes the
description simple:

    If you see 'nonlocal' in front of an assignment, it means
    the same as a 'nonlocal' declaration (read from the beginning
    up to the assignment operator) followed by the assignment.

This tells you both what the statement means and how to tell
whether your syntax is valid.  No exceptions to memorize.


-- ?!ng

From solipsis at pitrou.net  Fri Nov  3 23:50:37 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 03 Nov 2006 23:50:37 +0100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
	<1162590716.4428.23.camel@fsol>
	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
Message-ID: <1162594237.4428.37.camel@fsol>

Le vendredi 03 novembre 2006 ? 16:22 -0600, Ka-Ping Yee a ?crit :
> Only x is nonlocal.
> 
>     nonlocal x = y = 0
> 
> is equivalent to
> 
>     nonlocal x; x = y = 0

Perhaps, but the point is that it's non-intuitive from reading.
Just like the old "char * x, y" annoyance in C.

> The point is to make the statement easy to explain.  As Christian
> suggested, prohibiting multiple assignment is an exception: an
> additional rule to remember.

You are just trading one rule for another rule: the rule that only the
leftmost variable in the assignment chain is nonlocal.

In both cases, you have a non-obvious rule to remember. But in one case,
the rule is enforced by syntax, while in the other, it can silently
trigger bugs if not respected.

OTOH if you forbid assignment altogether, there is nothing special to
explain that is not already explained for "global".



From greg.ewing at canterbury.ac.nz  Sat Nov  4 01:58:07 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 04 Nov 2006 13:58:07 +1300
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <454B15F5.2090303@gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<454B15F5.2090303@gmail.com>
Message-ID: <454BE59F.9030301@canterbury.ac.nz>

Nick Coghlan wrote:

> A new keyword, on the other hand, allows the list to be split relatively 
> cleanly into four possible locations:
> 
> 1. function locals of the current function (local variables)
> 2. function locals of an enclosing function (closure variables)
> 3. module globals (global variables)
> 4. builtins (builtin names)

I don't see what's so important about treating the
module namespace differently from other lexically
enclosing scopes for the purpose of resolving names.
Especially only for *assignment* to names.

For read access, we seem to get on just fine without
any keywords to give us a hint what level to look
in. What's different about assignment that we
suddenly feel such a need for them?

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Nov  4 02:07:44 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 04 Nov 2006 14:07:44 +1300
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
Message-ID: <454BE7E0.5010501@canterbury.ac.nz>

Ka-Ping Yee wrote:

>     nonlocal x = y = 0

I don't really like the idea of allowing assignment in
the nonlocal/global/whatever statement. It leads to
questions like what happens if some assignments in a
scope are qualified with nonlocal and others aren't.

Certainly answers can be given, but it's more
complexity in the rules and more to learn. If
declaration and assignment are kept separate, these
questions don't arise in the first place.

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Nov  4 02:11:55 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 04 Nov 2006 14:11:55 +1300
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
	<1162590716.4428.23.camel@fsol>
	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
Message-ID: <454BE8DB.20203@canterbury.ac.nz>

Ka-Ping Yee wrote:

>     nonlocal x = y = 0
> 
> is equivalent to
> 
>     nonlocal x; x = y = 0

This is far from obvious. And what happens if
you want both x and y to be nonlocal? Can you
write

   nonlocal x = nonlocal y = 0

If so, this is getting even more complicated
for very little benefit.

> The point is to make the statement easy to explain.

It's even easier to explain if assignment isn't
allowed at all.

--
Greg

From ntoronto at cs.byu.edu  Sat Nov  4 03:50:41 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 03 Nov 2006 19:50:41 -0700
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454BE8DB.20203@canterbury.ac.nz>
References: <E1GfYMP-0002mn-5b@swing.co.at>	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>	<1162590716.4428.23.camel@fsol>	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
	<454BE8DB.20203@canterbury.ac.nz>
Message-ID: <454C0001.1080301@cs.byu.edu>

Greg Ewing wrote:
> Ka-Ping Yee wrote:
>
>   
>>     nonlocal x = y = 0
>>
>> is equivalent to
>>
>>     nonlocal x; x = y = 0
>>     
>
> This is far from obvious. And what happens if
> you want both x and y to be nonlocal? Can you
> write
>
>    nonlocal x = nonlocal y = 0
>
> If so, this is getting even more complicated
> for very little benefit.
>
>   
>> The point is to make the statement easy to explain.
>>     
>
> It's even easier to explain if assignment isn't
> allowed at all.
>   

Sorry about the "mee to" post, but me too: +1 on no assignment allowed. 
I can't think of any way to do it that would be intuitive and obvious to 
*most* people and wouldn't require a separate FAQ entry besides the one 
that explains 'global' and 'nonlocal.'

Neil


From ncoghlan at gmail.com  Sat Nov  4 04:23:03 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 04 Nov 2006 13:23:03 +1000
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <454BE59F.9030301@canterbury.ac.nz>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<454B15F5.2090303@gmail.com> <454BE59F.9030301@canterbury.ac.nz>
Message-ID: <454C0797.9010708@gmail.com>

Greg Ewing wrote:
> I don't see what's so important about treating the
> module namespace differently from other lexically
> enclosing scopes for the purpose of resolving names.
> Especially only for *assignment* to names.
> 
> For read access, we seem to get on just fine without
> any keywords to give us a hint what level to look
> in. What's different about assignment that we
> suddenly feel such a need for them?

Because read access doesn't have any non-local effects, but write access does. 
In most cases, it doesn't matter all that much which namespace a value came 
from, but where you are putting it can matter a great deal in terms of its 
visibility and lifecycle.

  1. Assigning to x when x is a local variable:
      - only this function and enclosed functions can see the change
      - 'x' will now be found in locals() in this function
      - reference lasts until end of current function execution

  2. Assigning to x when x is a global variable:
      - all code in this module can see the change
      - every user of the module can see the change (as a module attribute)
      - 'x' will now be found in globals() in this module
      - reference lasts until module dictionary is cleared

Currently, these are the only 2 possible cases. The PEP proposes adding a 3rd 
case:

  3. Assigning to x when x is a closure variable:
      - the enclosing function and all enclosed functions can see the change
      - will not cause 'x' to appear in either globals() or locals()
      - reference lasts until function object is destroyed

The 3rd case is significantly different from the 2nd in terms of both the 
visibility of any changes (particularly the points about whether or not 
changes are going to be visible as module attributes and entries in globals()) 
and the lifecycle of the reference (whether it is released along with the 
function object or the module dictionary).

Obscuring these differences by re-using the keyword global to denote write 
access to closure variables as well as global variables wouldn't be doing 
anyone any favours.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Nov  4 04:36:52 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 04 Nov 2006 13:36:52 +1000
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454C0001.1080301@cs.byu.edu>
References: <E1GfYMP-0002mn-5b@swing.co.at>	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>	<1162590716.4428.23.camel@fsol>	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>	<454BE8DB.20203@canterbury.ac.nz>
	<454C0001.1080301@cs.byu.edu>
Message-ID: <454C0AD4.5090602@gmail.com>

Neil Toronto wrote:
>>> The point is to make the statement easy to explain.
>>>     
>> It's even easier to explain if assignment isn't
>> allowed at all.
>>   
> 
> Sorry about the "mee to" post, but me too: +1 on no assignment allowed. 
> I can't think of any way to do it that would be intuitive and obvious to 
> *most* people and wouldn't require a separate FAQ entry besides the one 
> that explains 'global' and 'nonlocal.'

'Me too' posts can be useful sometimes :)

Count me as another +1 on disallowing the shorthand version.

Rationale:
   1. We've lived without it in the case of global
   2. We can add it later if we decide we really need it, but if we start with 
it and it turns out to be confusing or error-prone, taking it away would be a 
serious pain.
   3. Leaving it out ducks some hard questions like "what does this do?":

   def weird(local=True):
       if local:
          n = 1    # Does this alter the closure variable?
       else:
          nonlocal n += i
       return n

Cheers,
Nick.

P.S. For those who asked "what's the point of the shorthand?", it's intended 
more for use with augmented assignment rather than basic assignment:

   def inc(i=1):
       nonlocal n += i
       return n

I can see the benefit in shortening toy examples, but I don't think it even 
comes close to being worth the extra complexity.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Nov  4 04:40:48 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 04 Nov 2006 13:40:48 +1000
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454BA851.4010200@cs.byu.edu>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<454BA851.4010200@cs.byu.edu>
Message-ID: <454C0BC0.20503@gmail.com>

Neil Toronto wrote:
> Ka-Ping Yee wrote:
>> Hi folks,
>>
>> I have finally completed a draft of a PEP on rebinding of names
>> in outer scopes.  I've tried to go back and gather all of the
>> (amazingly numerous) proposals -- if i've forgotten or misattributed
>> any, let me know and i'll be happy to correct them.
>>
>> I look forward to your thoughts on it:
> 
> Is there any particular reason this *must* be a Py3k PEP? (Aside from 
> the fact that we're discussing it on the Py3k list, of course.) Some of 
> the solutions discussed for historical context would *definitely* be bad 
> ideas for a 2.x (as would changing the semantics of 'global'), but I 
> can't see why the solution proposed in the PEP couldn't be done sooner.

I think you answered your own question - some of the options up for 
consideration in the PEP are *not* compatible with 2.x, so targeting Py3k 
makes for a freer discussion of the possibilities.

If the result turns out to be backwards compatible, then we can retarget 
Python 2.6, but that's actually the case for *all* of the Py3k PEPs (even the 
backwards incompatible PEPs will likely have elements which can be 
incorporated into the 2.x series to ease the eventual transition).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From greg.ewing at canterbury.ac.nz  Sat Nov  4 01:57:58 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 04 Nov 2006 13:57:58 +1300
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eig735$ovr$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<eig735$ovr$1@sea.gmane.org>
Message-ID: <454BE596.4020507@canterbury.ac.nz>

Ron Adam wrote:

> How about limiting nonlocal to just the immediate parent scope and using 
> 'parent' as the keyword?

That could lead to confusing situations. What should
the following do:

   def f():
     x = 42
     def g():
       def h():
       parent x
       x = 88

Should the assignment to x in h() create a name in
the scope of g() even though there's no assignment
in g() to establish that as its home scope? Should
it be an error?

--
Greg

From python at zesty.ca  Sat Nov  4 22:54:03 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Sat, 4 Nov 2006 15:54:03 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <20061102052412.A91B36F4198@longblack.object-craft.com.au>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
	<Pine.LNX.4.58.0611012229080.3499@server1.LFW.org>
	<20061102052412.A91B36F4198@longblack.object-craft.com.au>
Message-ID: <Pine.LNX.4.58.0611041537300.6331@server1.LFW.org>

On Thu, 2 Nov 2006, Andrew McNamara wrote:
> >The global scope is the widest scope in which you can declare a variable.
>
> This is inaccurate at best.

No, my statement is correct.  What you are disputing is not what i wrote:

The global scope is the widest scope in which you can declare a variable.

You cannot declare variables in builtin scope in Python.  You can stuff
variables into the builtin module with some manipulation, but it is a
very rare thing to do, it is not encouraged as a normal style of
programming, and it is not supported by the language syntax.

Python, C/C++, JavaScript, Ruby, and Perl all have this in common:

    A "global variable" is visible to the entire file and does
    not belong to any particular function.

> To be honest, I'm +0 on using "global" in the way GvR proposes. My
> point is that python's globals are already different from other common
> languages, and people cope with that.

That does not constitute an argument in favour of anything.

For example, Python's integers are already different from other common
languages, because they have unlimited range.  That is not a reason to
make "integers" include, say, floating point numbers.


-- ?ing

From rrr at ronadam.com  Sun Nov  5 05:28:23 2006
From: rrr at ronadam.com (Ron Adam)
Date: Sat, 04 Nov 2006 22:28:23 -0600
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <454BE596.4020507@canterbury.ac.nz>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>
	<454BE596.4020507@canterbury.ac.nz>
Message-ID: <eijpg6$jqe$1@sea.gmane.org>

Greg Ewing wrote:
> Ron Adam wrote:
> 
>> How about limiting nonlocal to just the immediate parent scope and using 
>> 'parent' as the keyword?
> 
> That could lead to confusing situations. What should
> the following do:
> 
>    def f():
>      x = 42
>      def g():
>        def h():
>        parent x
>        x = 88
> 
> Should the assignment to x in h() create a name in
> the scope of g() even though there's no assignment
> in g() to establish that as its home scope? Should
> it be an error?
> 
> --
> Greg

Not confusing at all.  It would work just like global does in the same 
situation.  The parent statement here does not create a name, it only designates 
the scope where a name will be written.

I presume you meant:

    def f():
      x = 42
      def g():
        def h():
          parent x
          x = 88


The x would be a new local x in function g and have no effect on the x in 
function f.  Parent would do exactly what it means in this case, its a reference 
to a name in only the parent scope.

Unless you were to do:

    def f():
      x = 42
      def g():
        parent x
        def h():
          parent x
          x = 88

Then all references to x would be the local x in function f.

This is what I meant that it might be possibly to pull a reference down to inner 
scopes.  It gives more control on what name space you are writing to and less 
chance of writing to a grandparent scope unintentionally because of a variable 
name being left out, having its name changed, or has been deleted in the 
immediate parent function while editing.

I also don't think this would be too much trouble as nested functions don't tend 
to be more than 2 or 3 levels deep very often and if it becomes a problem of 
having too many parent statements, it's probably a good sign a class would be 
better.

IMHO, and if a suitable keyword meaning parent could be found.

   Ron


From g.brandl at gmx.net  Sun Nov  5 10:49:34 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 05 Nov 2006 10:49:34 +0100
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eijpg6$jqe$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>	<454BE596.4020507@canterbury.ac.nz>
	<eijpg6$jqe$1@sea.gmane.org>
Message-ID: <eikc3g$mrl$1@sea.gmane.org>

Ron Adam wrote:

> I presume you meant:
> 
>     def f():
>       x = 42
>       def g():
>         def h():
>           parent x
>           x = 88
> 
> 
> The x would be a new local x in function g and have no effect on the x in 
> function f.  Parent would do exactly what it means in this case, its a reference 
> to a name in only the parent scope.

This *is* confusing, even if the word "parent" is a slight hint on what's
really going on.

> Unless you were to do:
> 
>     def f():
>       x = 42
>       def g():
>         parent x
>         def h():
>           parent x
>           x = 88
> 
> Then all references to x would be the local x in function f.

"So why do I have to declare x in g, if I don't use it there?" This is really
ugly.

Georg


From rrr at ronadam.com  Sun Nov  5 22:16:34 2006
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 05 Nov 2006 15:16:34 -0600
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eikc3g$mrl$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>	<454BE596.4020507@canterbury.ac.nz>	<eijpg6$jqe$1@sea.gmane.org>
	<eikc3g$mrl$1@sea.gmane.org>
Message-ID: <eilkih$ea2$1@sea.gmane.org>

Georg Brandl wrote:
> Ron Adam wrote:
> 
>> I presume you meant:
>>
>>     def f():
>>       x = 42
>>       def g():
>>         def h():
>>           parent x
>>           x = 88
>>
>>
>> The x would be a new local x in function g and have no effect on the x in 
>> function f.  Parent would do exactly what it means in this case, its a reference 
>> to a name in only the parent scope.
> 
> This *is* confusing, even if the word "parent" is a slight hint on what's
> really going on.

Why is it confusing to you, what would you expect to happen here?

     def f():
       def g():
         def h():
           parent x
           x = 42

By not limiting parent to just the parent scope you create exceptions.  The rule 
becomes:


     The keyword (*)nonlocal designates a name will be written to in the
     closest enclosing "parent" scope *except* when a pre-existing matching name
     exists in a scope further up.

To me that is more confusing than always referring to the closest enclosing 
scope without exception.

(*) Nonlocal is better a better term for implicit scope selection.  Parent is 
better suited for the explicit scope selection suggested.  See below.


>> Unless you were to do:
>>
>>     def f():
>>       x = 42
>>       def g():
>>         parent x
>>         def h():
>>           parent x
>>           x = 88
>>
>> Then all references to x would be the local x in function f.
> 
> "So why do I have to declare x in g, if I don't use it there?" This is really
> ugly.
> 
> Georg

It's a choice.

Explicit scope selection: This depends only on the existence of parent 
statements to determine which scope a name will be bound in.  Parent redirects 
an assignment to the parent scope.  Successive parent statements can redirect 
the binding of names to scopes further up.

Implicit scope selection: This depends on the pre-existence of a matching bound 
name in *any* parent scope to determine where write access will be allowed.  If 
no matching name if found write access will be the nearest enclosing scope.

There are more opportunities for silent bugs with the looser implicit scope 
selection.  That was the point I was making.  I'll leave it up to the experts to 
  decide which is better.

Cheers,
    Ron












From qrczak at knm.org.pl  Sun Nov  5 22:46:18 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Sun, 05 Nov 2006 22:46:18 +0100
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eilkih$ea2$1@sea.gmane.org> (Ron Adam's message of "Sun, 05
	Nov 2006 15:16:34 -0600")
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>
	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>
	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<eig735$ovr$1@sea.gmane.org> <454BE596.4020507@canterbury.ac.nz>
	<eijpg6$jqe$1@sea.gmane.org> <eikc3g$mrl$1@sea.gmane.org>
	<eilkih$ea2$1@sea.gmane.org>
Message-ID: <87odrla7o5.fsf@qrnik.zagroda>

Ron Adam <rrr at ronadam.com> writes:

> By not limiting parent to just the parent scope you create exceptions.  The rule 
> becomes:
>
>      The keyword (*)nonlocal designates a name will be written to in the
>      closest enclosing "parent" scope *except* when a pre-existing matching name
>      exists in a scope further up.
>
> To me that is more confusing than always referring to the closest enclosing 
> scope without exception.

The rule should be:

The keyword 'nonlocal' causes the lookup to be performed as if there
were no assignments to that variable in the scope containing the
'nonlocal' declaration.

No exceptions are needed, and no limitation to the immediately
surrounding scope is needed. Read accesses already have fine scoping
rules, as long as there is something to determine the scope. The only
problem with current rules is that assignment is coupled with creating
a new variable, and 'nonlocal' just allows to decouple that.

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From mbk.lists at gmail.com  Sun Nov  5 23:58:59 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Sun, 5 Nov 2006 15:58:59 -0700
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <87odrla7o5.fsf@qrnik.zagroda>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<eig735$ovr$1@sea.gmane.org> <454BE596.4020507@canterbury.ac.nz>
	<eijpg6$jqe$1@sea.gmane.org> <eikc3g$mrl$1@sea.gmane.org>
	<eilkih$ea2$1@sea.gmane.org> <87odrla7o5.fsf@qrnik.zagroda>
Message-ID: <da7032ce0611051458y479fade3u85e921ac9a32df1f@mail.gmail.com>

> The rule should be:
>
> The keyword 'nonlocal' causes the lookup to be performed as if there
> were no assignments to that variable in the scope containing the
> 'nonlocal' declaration.
>
> No exceptions are needed, and no limitation to the immediately
> surrounding scope is needed. Read accesses already have fine scoping
> rules, as long as there is something to determine the scope. The only
> problem with current rules is that assignment is coupled with creating
> a new variable, and 'nonlocal' just allows to decouple that.

Exactly.  There is no reason to place an artificial limitation on this
feature.  This is also why I advocate the spelling "use", because
ISTM "use x" pretty much means exactly what it says.  The main
argument against the spelling "use" that I've seen is that it's too
generic, i.e, one could "use" anything.  This argument doesn't sway me
because whatever we call the keyword, I think the programmer will have
to look it up in the docs the first time they encounter it.  Then
they'll say "aha, I get it" and move on -- a minor one-time bump on
the learning curve.  Once you know what the keyword means, I think
"use" is the spelling that makes the most sense, and it's easy to
type, too.

   Mike

From g.brandl at gmx.net  Sun Nov  5 23:59:35 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 05 Nov 2006 23:59:35 +0100
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <87odrla7o5.fsf@qrnik.zagroda>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>
	<454BE596.4020507@canterbury.ac.nz>	<eijpg6$jqe$1@sea.gmane.org>
	<eikc3g$mrl$1@sea.gmane.org>	<eilkih$ea2$1@sea.gmane.org>
	<87odrla7o5.fsf@qrnik.zagroda>
Message-ID: <eilqcp$u0a$1@sea.gmane.org>

Marcin 'Qrczak' Kowalczyk wrote:
> Ron Adam <rrr at ronadam.com> writes:
> 
>> By not limiting parent to just the parent scope you create exceptions.  The rule 
>> becomes:
>>
>>      The keyword (*)nonlocal designates a name will be written to in the
>>      closest enclosing "parent" scope *except* when a pre-existing matching name
>>      exists in a scope further up.
>>
>> To me that is more confusing than always referring to the closest enclosing 
>> scope without exception.
> 
> The rule should be:
> 
> The keyword 'nonlocal' causes the lookup to be performed as if there
> were no assignments to that variable in the scope containing the
> 'nonlocal' declaration.

Plus, if there's no binding in an enclosing scope, an error is raised.

(Which brings up the assymetry to today's global again, but is that really
a problem?)

Georg


From greg.ewing at canterbury.ac.nz  Mon Nov  6 00:21:26 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2006 12:21:26 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
Message-ID: <454E71F6.7090103@canterbury.ac.nz>

Mike Orr wrote:

>     .abspath()
>     .normpath()
>     .realpath()
>     .splitpath()
>     .relpath()
>     .relpathto()

Seeing as the whole class is about paths, having
"path" in the method names seems redundant. I'd
prefer to see terser method names without any
noise characters in them.

--
Greg

From python at zesty.ca  Mon Nov  6 00:57:14 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Sun, 5 Nov 2006 17:57:14 -0600 (CST)
Subject: [Python-3000] Backward compatibility for "nonlocal"
Message-ID: <Pine.LNX.4.58.0611051721400.6331@server1.LFW.org>

As an aside to the discussion about "nonlocal", here are a couple of
thoughts on backward compatibility.

For some of the proposed keywords, here's the number of occurrences
of the keyword in the current standard library (not including comments
and docstrings):

    nonlocal      0       0
    use           2       2
    using         3       3
    reuse         0       4
    free          8       8
    outer         5     147
    global      126     214

(I used 'tokenize' to do this; let me know if you'd like me to count
other possibilities.)

This may or may not be a good measure of the likely impact of adding
a new keyword.  At least it's something to think about.

It also occurred to me that this new keyword doesn't necessarily
have to break anything, if we are willing to tolerate a little
hackery in the Python lexer and parser.  The situation is a bit
reminiscent of 'as' -- we were able to add 'as' to the 'import'
statement as though 'as' were a keyword, without actually making
'as' a reserved word in all code.

In this case, the usage of "nonlocal" (or whatever we choose) as
a keyword never overlaps with its usage as a variable name.  The
parser could easily tell the difference by checking whether:

    - The keyword is at the beginning of a statement.
    - The keyword is immediately followed by an identifier.

If both of these things are true, then the keyword is a nonlocal
declaration.  Otherwise, it can be treated as a variable name.

I'm not saying this is necessarily a good idea; i just wanted to
note that it was an available option.  We might think about a
schedule for phasing in the feature:

    Phase 1.  If "nonlocal" is seen in code, issue a warning
              that it will become a keyword in a future version.

    Phase 2.  Have the parser distinguish when "nonlocal" is
              intended as a keyword and when as a variable name.

    Phase 3.  Make "nonlocal" an official reserved word.

All of these phases are optional -- we could skip straight to 3,
or do 1 and then 3, or stop at 2, etc.


-- ?!ng

From greg.ewing at canterbury.ac.nz  Mon Nov  6 01:00:28 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Nov 2006 13:00:28 +1300
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <da7032ce0611051458y479fade3u85e921ac9a32df1f@mail.gmail.com>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>
	<454AA735.4080406@canterbury.ac.nz>
	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>
	<eig735$ovr$1@sea.gmane.org> <454BE596.4020507@canterbury.ac.nz>
	<eijpg6$jqe$1@sea.gmane.org> <eikc3g$mrl$1@sea.gmane.org>
	<eilkih$ea2$1@sea.gmane.org> <87odrla7o5.fsf@qrnik.zagroda>
	<da7032ce0611051458y479fade3u85e921ac9a32df1f@mail.gmail.com>
Message-ID: <454E7B1C.8040204@canterbury.ac.nz>

Mike Krell wrote:
>  This is also why I advocate the spelling "use", because
> ISTM "use x" pretty much means exactly what it says.

I beg to differ -- it means nothing at all to
me. At least "global" or "nonlocal" sound like
they have *something* to do with scope, even
if it's not clear exactly what.

--
Greg

From python at zesty.ca  Mon Nov  6 01:17:03 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Sun, 5 Nov 2006 18:17:03 -0600 (CST)
Subject: [Python-3000] Backward compatibility for "nonlocal"
In-Reply-To: <Pine.LNX.4.58.0611051721400.6331@server1.LFW.org>
References: <Pine.LNX.4.58.0611051721400.6331@server1.LFW.org>
Message-ID: <Pine.LNX.4.58.0611051816260.6331@server1.LFW.org>

On Sun, 5 Nov 2006, Ka-Ping Yee wrote:
>     nonlocal      0       0
>     use           2       2
>     using         3       3
>     reuse         0       4
>     free          8       8
>     outer         5     147
>     global      126     214

Oops, i forgot to explain: the first column is the count not including
tests; the second column is the count in the entire Lib/ directory.


-- ?!ng

From rrr at ronadam.com  Mon Nov  6 02:35:16 2006
From: rrr at ronadam.com (Ron Adam)
Date: Sun, 05 Nov 2006 19:35:16 -0600
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
Message-ID: <eim3nk$jbk$1@sea.gmane.org>

Mike Orr wrote:

> The multi-argument constructor is a replacement for joining paths.
> (The PEP says .joinpath was "problematic" without saying why.)    This
> could theoretically go either way, doing either the same thing as
> os.path.join, getting a little smarter, or doing "safe" joins by
> disallowing "/" embedded in string arguments.

Some of this may be due to platform dependency.

Maybe adding some equivalent mechanism like unicode.encode() and str.decode() to 
path objects to enable easier use of foreign or different types of paths in some 
situations would be good?

In this case instead of a string encoding, it would be a path os encoding.

    Path("ABC").encode('windows') =>  a windows path string from a path object.

    Path().decode('ABC', 'windows') =>  a platform dependant path object

Don't take these examples too literally, I'm just pointing out a concept that 
might be useful where paths from differing platforms may touch each other.

You would probably want to use a different spelling for these so they don't get 
confused with string encodings.

Could paths be handled with a string encoding?  Just a thought.

Cheers,
   Ron











From andrewm at object-craft.com.au  Mon Nov  6 02:47:19 2006
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Mon, 06 Nov 2006 12:47:19 +1100
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611041537300.6331@server1.LFW.org> 
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
	<Pine.LNX.4.58.0611012229080.3499@server1.LFW.org>
	<20061102052412.A91B36F4198@longblack.object-craft.com.au>
	<Pine.LNX.4.58.0611041537300.6331@server1.LFW.org>
Message-ID: <20061106014719.61D6679419E@longblack.object-craft.com.au>

>Python, C/C++, JavaScript, Ruby, and Perl all have this in common:
>
>    A "global variable" is visible to the entire file and does
>    not belong to any particular function.

I note that you didn't say "a global variable is visible to the entire
application" - you've deliberately narrowed the definition to suit
your argument.

>> To be honest, I'm +0 on using "global" in the way GvR proposes. My
>> point is that python's globals are already different from other common
>> languages, and people cope with that.
>
>That does not constitute an argument in favour of anything.

Sigh. I'll spell it out more fully then:

Introducing new keywords is rude. Changing the behaviour of existing
language constructs is rude. To implement this proposal, we need to do one
or the other. Of the two, I lean very slightly in favour of making the
semantics of "global" slightly more complex, on the basis that they're
already slightly more complex than most other languages and people cope
with that.

>For example, Python's integers are already different from other common
>languages, because they have unlimited range.  That is not a reason to
>make "integers" include, say, floating point numbers.

I'm disappointed that you're resorting to straw-man arguments. I guess
that means we're done now.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From ntoronto at cs.byu.edu  Mon Nov  6 03:02:41 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Sun, 05 Nov 2006 19:02:41 -0700
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <454E7B1C.8040204@canterbury.ac.nz>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>
	<454BE596.4020507@canterbury.ac.nz>	<eijpg6$jqe$1@sea.gmane.org>
	<eikc3g$mrl$1@sea.gmane.org>	<eilkih$ea2$1@sea.gmane.org>
	<87odrla7o5.fsf@qrnik.zagroda>	<da7032ce0611051458y479fade3u85e921ac9a32df1f@mail.gmail.com>
	<454E7B1C.8040204@canterbury.ac.nz>
Message-ID: <454E97C1.5000204@cs.byu.edu>

Greg Ewing wrote:

>Mike Krell wrote:
>  
>
>> This is also why I advocate the spelling "use", because
>>ISTM "use x" pretty much means exactly what it says.
>>    
>>
>
>I beg to differ -- it means nothing at all to
>me. At least "global" or "nonlocal" sound like
>they have *something* to do with scope, even
>if it's not clear exactly what.
>  
>

And because it looks like the illegitimate child of a scabby old Pascal 
keyword, it'll make people look twice. I'm only being half facetious.

Another half-facetious (maybe quarter) argument in its favor is that, 
even though it's short(-ish), it's rare. Any future Google search for 
"python nonlocal" will find hundreds of relevant documents. Heck, the 
first seven hits for that *right now* are on subject.

This one's starting to grow on me... sort of like a cheese mold, but 
it's growing nonetheless.

Neil



From python at zesty.ca  Mon Nov  6 05:28:22 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Sun, 5 Nov 2006 22:28:22 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <20061106014719.61D6679419E@longblack.object-craft.com.au>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>
	<45491FB4.3000507@cs.byu.edu>
	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>
	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>
	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>
	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>
	<Pine.LNX.4.58.0611012229080.3499@server1.LFW.org>
	<20061102052412.A91B36F4198@longblack.object-craft.com.au>
	<Pine.LNX.4.58.0611041537300.6331@server1.LFW.org>
	<20061106014719.61D6679419E@longblack.object-craft.com.au>
Message-ID: <Pine.LNX.4.58.0611052154250.6331@server1.LFW.org>

On Mon, 6 Nov 2006, Andrew McNamara wrote:
> >Python, C/C++, JavaScript, Ruby, and Perl all have this in common:
> >
> >    A "global variable" is visible to the entire file and does
> >    not belong to any particular function.
>
> I note that you didn't say "a global variable is visible to the entire
> application" - you've deliberately narrowed the definition to suit
> your argument.

Let's be clear that we're discussing the same topic here.  If we're
trying to assess whether a particular meaning of "global" in Python
will be familiar, surprising, easy to understand, etc. then we have
to compare it to what "global" means in other contexts.  That requires
talking about what is in common among the other contexts.

Are you trying to assert that most of the other languages have a common
meaning for "global", from which Python has already diverged enough
that it doesn't matter for Python to diverge further?  But i think
we agree it is not always true in programming languages that "global"
means "visible to the entire application" -- you provided some good
examples, in fact.  So this cannot be part of the common meaning.

Or are you trying to show that other languages have meanings for
"global" that are all so different that there *is* no common meaning
for Python to follow?  That's the reason i offered, above, a statement
of something that is clearly common among them.  This is not intended
to be the complete definition of "global" in all these contexts, as each
language has its own way of integrating multiple modules.  But because
at least this much is common, this is part of the common meaning that
people have come to expect.  This is a meaning that Python conforms to
now, and that Python would no longer fit if we redefined "global" to
mean "belonging to any parent scope of the current scope".

> Introducing new keywords is rude. Changing the behaviour of existing
> language constructs is rude. To implement this proposal, we need to do one
> or the other. Of the two, I lean very slightly in favour of making the
> semantics of "global" slightly more complex, on the basis that they're
> already slightly more complex than most other languages and people cope
> with that.

I am trying to warn the participants in this discussion that the cost
of redefining "global" in this manner is rather higher than may seem
initially apparent.  The effect will be to confuse the meaning of
"global" not just as a language keyword but as a programming concept:
"global namespace" will not just have a different meaning -- it will
come to have no meaning at all (or, alternatively, the language itself
will be at odds with its own semantics), as i've explained [1].

This would be quite a bit more painful than introducing a keyword in
a new version of Python that has been specifically designated as
potentially backward incompatible, with time still remaining between
now and then for gradual migration.


-- ?!ng

[1] http://mail.python.org/pipermail/python-3000/2006-November/004215.html

From python at zesty.ca  Mon Nov  6 06:16:37 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Sun, 5 Nov 2006 23:16:37 -0600 (CST)
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <454C0AD4.5090602@gmail.com>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
	<1162590716.4428.23.camel@fsol>
	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
	<454BE8DB.20203@canterbury.ac.nz> <454C0001.1080301@cs.byu.edu>
	<454C0AD4.5090602@gmail.com>
Message-ID: <Pine.LNX.4.58.0611052306170.6331@server1.LFW.org>

On Sat, 4 Nov 2006, Nick Coghlan wrote:
> Count me as another +1 on disallowing the shorthand version.
>
> Rationale:
> 1. We've lived without it in the case of global
> 2. We can add it later if we decide we really need it, but if we
> start with it and it turns out to be confusing or error-prone,
> taking it away would be a serious pain.

That's a good point.  We can start without it, or perhaps with just
the simple form that Guido said he'd endorse, and find out later if
people want to be able to do the other forms of assignment.

>    3. Leaving it out ducks some hard questions like "what does this do?":
>
>    def weird(local=True):
>        if local:
>           n = 1    # Does this alter the closure variable?
>        else:
>           nonlocal n += i
>        return n

Maybe this will confuse people, maybe it won't.  I think we do know
that so far the possibility of writing:

   def weird(local=True):
       if local:
           n = 1
       else:
           global n
           n += 1
       return n

doesn't seem to have caused problems.  I can see how the shorthand
form makes it more likely that people will want to put "global" in
the middle of the function than at the beginning, though.


-- ?!ng

From talin at acm.org  Mon Nov  6 08:39:50 2006
From: talin at acm.org (Talin)
Date: Sun, 05 Nov 2006 23:39:50 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
Message-ID: <454EE6C6.4000701@acm.org>

Mike Orr wrote:
> Posted to python-dev and python-3000.  Follow-ups to python-dev only please.
> 
> So, let's say we strip this Path class to:

I'm finally taking the time to sit down and go over this in detail. Here 
are some suggestions.

> class Path(unicode):
>     Path("foo")
>     Path(  Path("directory"),   "subdirectory", "file")    # Replaces
> .joinpath().
>     Path()

For the constructor, I would write it as:

   Path( *components )

'components' is an arbitrary number of path components, either strings 
or path objects. The components are joined together into a single path.

Strings can also be wrapped with an object that indicates that the Path 
is in a platform- or application-specific format:

    # Explicitly indicate that the path string is in Windows NTFS format.
    Path( Path.format.NTFS( "C:\\Program Files" ) )

Note that it's OK to include path separators in the string that's passed 
to the format wrapper - these will get converted.

Not including a format wrapper is equivalent to using the "local" wrapper:

    Path( Path.format.local( "C:\\Program Files" ) )

Where 'local' is an alias to the native path format for the host's 
default filesystem.

The wrapper objects are themselves classes, and need not be in the 
"Path" namespace. For example:

    import p4
    Path( p4.path.format( "//depot/files/..." ) )

This makes the set of specific path formats open-ended and extensible. 
Path format wrappers need not be built into the "Path" module. Each 
format wrapper will have a "to_path" method, that converts the specific 
path encoding into the universal path representation.

Note that if there are multiple components, they don't have to be 
wrapped the same way:

    Path( Path.format.NTFS( "C:\\Program Files" ),
          Path.format.local( "Gimp" ) )

...because the conversion to universal representation is done before the 
components are combined.

One question to be asked is whether the path should be simplified or 
not. There are cases where you *don't* want the path to be simplified, 
and other cases where you do. Perhaps a keyword argument?

    Path( "C:\\Program Files", "../../Gimp", normalize = True )

>     Path.cwd()

No objection here.

>     Path("ab") + "c"  => Path("abc")

Wouldn't that be:

    Path( "ab" ) + "c" => Path( "ab", "c" )

?

It seems that the most common operation is concatenating components, not 
characters, although both should be easy.

>     .abspath()

I've always thought this was a strange function. To be honest, I'd 
rather explicitly pass in the cwd().

>     .normcase()

So the purpose of this function is to get around the fact that on some 
platforms, comparisons between paths are case-sensitive, and on other 
platforms not. However, the reason this function seems weird to me is 
that most case-insensitive filesystems are case-preserving, which makes 
me thing that the real solution is to fix the comparison functions 
rather than mangling the string. (Although there's a hitch - its hard to 
make a case-insensitive dictionary that doesn't require a downcase'd 
copy of the key; Something I've long wanted was a userdict that allowed 
both the comparison and hash functions to be replaceable, but that's a 
different topic.)

>     .normpath()

I'd rename this to "simplify", since it no longer needs to normalize the 
separator chars. (That's done by the wrappers.)

>     .realpath()

Rename to resolve() or resolvelinks().

>     .expanduser()
>     .expandvars()
>     .expand()

Replace with expand( user=True, vars=True )

>     .parent

If parent was a function, you could pass in the number of levels go to 
up, i.e. parent( 2 ) to get the grandparent.

>     .name                 # Full filename without path
>     .namebase        # Filename without extension

I find the term 'name' ambiguous. How about:

     .filepart
     .basepart

or:

     .filename
     .basename

>     .ext

No problem with this

>     .drive

Do we need to somehow unify the concept of 'drive' and 'unc' part? Maybe 
'.device' could return the part before the first directory name.

>     .splitpath()

I'd like to replace this with:

    .component( slice_object )

where the semantics of 'component' are identical to __getitem__ on an 
array or tuple. So for example:

    Path( "a", "b" ).component( 0 ) => "a"
    Path( "a", "b" ).component( 1 ) => "b"
    Path( "a", "b" ).component( -1 ) => "b"
    Path( "a", "b" ).component( 0:1 ) => Path( "a", "b" )
    Path( "a", "b" ).component( 1: ) => Path( "b" )

This is essentially the same as the "slice notation" proposal given 
earlier, except that explicitly tell the user that we are dealing with 
path components, not characters.

>     .stripext()

How about:

     path.ext = ''

>     .splitunc()
>     .uncshare

See above - UNC shouldn't be a special case.

>     .splitall()

Something sadly lacking in os.path.

>     .relpath()

Again, I'd rather that they pass in the cwd() explicitly. But I would 
like to see something like:

    .relativeto( path )

...which computes the minimal relative path that goes from 'self' to 'path'.

>     .relpathto()

Not sure what this does, since there's no argument defined.

Additional methods:

     .format( wrapper_class )

...converts the path into a filesystem-specific format. You can also get 
the same effect by "wrapping" the path object and calling str()

    str( Path.format.NTFS( Path( "a", "b", "c" ) ) )

Although it's a bit cumbersome.

-- Talin

From ncoghlan at gmail.com  Mon Nov  6 10:22:25 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 06 Nov 2006 19:22:25 +1000
Subject: [Python-3000] The meaning of "global variable"
In-Reply-To: <eilqcp$u0a$1@sea.gmane.org>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<d11dcfba0611011939y37317b76k844d6a285c9dc620@mail.gmail.com>	<Pine.LNX.4.58.0611012215280.3499@server1.LFW.org>	<d11dcfba0611012036y3e43972fyf6721f9d0b8b508@mail.gmail.com>	<Pine.LNX.4.58.0611012258390.3499@server1.LFW.org>	<454AA735.4080406@canterbury.ac.nz>	<Pine.LNX.4.58.0611030101150.3499@server1.LFW.org>	<eig735$ovr$1@sea.gmane.org>	<454BE596.4020507@canterbury.ac.nz>	<eijpg6$jqe$1@sea.gmane.org>	<eikc3g$mrl$1@sea.gmane.org>	<eilkih$ea2$1@sea.gmane.org>	<87odrla7o5.fsf@qrnik.zagroda>
	<eilqcp$u0a$1@sea.gmane.org>
Message-ID: <454EFED1.4080407@gmail.com>

Georg Brandl wrote:
> Marcin 'Qrczak' Kowalczyk wrote:
>> The rule should be:
>>
>> The keyword 'nonlocal' causes the lookup to be performed as if there
>> were no assignments to that variable in the scope containing the
>> 'nonlocal' declaration.
> 
> Plus, if there's no binding in an enclosing scope, an error is raised.
> 
> (Which brings up the assymetry to today's global again, but is that really
> a problem?)

I'd narrow the requirement such that module globals don't count - if you 
declare a variable nonlocal it *must* be initialised in an enclosing function 
scope or you will get a syntax error similar to this one:

 >>> def f():
...   x = 1
...   del x
...   def g():
...     return x
...   return g
...
SyntaxError: can not delete variable 'x' referenced in nested scope

Here, the compiler knows that g() references back to 'x' and hence considers 
the attempt to delete 'x' nonsensical. Attempting to reference 'nonlocal x' 
when x isn't a closure variable is also nonsensical - if you want a local 
variable then remove the declaration, if you want a global then declare x as a 
global.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Mon Nov  6 10:37:36 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 06 Nov 2006 19:37:36 +1000
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <20061106014719.61D6679419E@longblack.object-craft.com.au>
References: <Pine.LNX.4.58.0611010912100.3499@server1.LFW.org>	<45491FB4.3000507@cs.byu.edu>	<ca471dc20611011452v1a88082eu75bedc3e4fb1f573@mail.gmail.com>	<bbaeab100611011507r42d694e9nfd791d9cc2b5f5f0@mail.gmail.com>	<Pine.LNX.4.58.0611011857150.3499@server1.LFW.org>	<20061102014059.BC4BB6F4198@longblack.object-craft.com.au>	<Pine.LNX.4.58.0611012229080.3499@server1.LFW.org>	<20061102052412.A91B36F4198@longblack.object-craft.com.au>	<Pine.LNX.4.58.0611041537300.6331@server1.LFW.org>
	<20061106014719.61D6679419E@longblack.object-craft.com.au>
Message-ID: <454F0260.8020104@gmail.com>

Andrew McNamara wrote:
>> Python, C/C++, JavaScript, Ruby, and Perl all have this in common:
>>
>>    A "global variable" is visible to the entire file and does
>>    not belong to any particular function.
> 
> I note that you didn't say "a global variable is visible to the entire
> application" - you've deliberately narrowed the definition to suit
> your argument.

In C/C++ to use another module's 'globals' you must redeclare their names in 
the current module (usually with a #include of the appropriate header file). 
The linker then takes care of associating the two declarations with the same 
defining module.

In C, each symbol must be unique. In C++, the global variables may be in 
different namespaces, requiring either scope qualification or an appropriate 
using statement to access the names in the namespace.

In Python, to use another module's globals you must import the module. You can 
then accessing the name as an attribute of the imported module (or you can use 
the alternate form of import and retrieve only the globals you are interested in).

So tell me again how Python's globals are different from C++ ones? Sure, the 
linkage is done at runtime rather than build time, but the module cache means 
they're still global to the application (accessible from everywhere, always 
refer to the same object).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From murman at gmail.com  Mon Nov  6 15:33:24 2006
From: murman at gmail.com (Michael Urman)
Date: Mon, 6 Nov 2006 08:33:24 -0600
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <Pine.LNX.4.58.0611052306170.6331@server1.LFW.org>
References: <E1GfYMP-0002mn-5b@swing.co.at>
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>
	<1162590716.4428.23.camel@fsol>
	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>
	<454BE8DB.20203@canterbury.ac.nz> <454C0001.1080301@cs.byu.edu>
	<454C0AD4.5090602@gmail.com>
	<Pine.LNX.4.58.0611052306170.6331@server1.LFW.org>
Message-ID: <dcbbbb410611060633q371afbd7o53d5457d433cad83@mail.gmail.com>

>    def weird(local=True):
>        if local:
>            n = 1
>        else:
>            global n   # [ or nonlocal n, in Nick's ]
>            n += 1
>        return n

I'm not sure what you're both trying to explain here. First off, the
above code yields a SyntaxWarning. Secondly all its accesses to n are
global accesses. The global keyword is currently a modifier for the
whole function. Thus the local parameter is misleading; it should be
called reset.

<stdin>:1: SyntaxWarning: name 'n' is assigned to before global declaration
>>> dis.dis(weird)
  2           0 LOAD_FAST                0 (local)
              3 JUMP_IF_FALSE           10 (to 16)
              6 POP_TOP

  3           7 LOAD_CONST               1 (1)
             10 STORE_GLOBAL             1 (n)
             13 JUMP_FORWARD            11 (to 27)
        >>   16 POP_TOP

  6          17 LOAD_GLOBAL              1 (n)
             20 LOAD_CONST               1 (1)
             23 INPLACE_ADD
             24 STORE_GLOBAL             1 (n)

  7     >>   27 LOAD_GLOBAL              1 (n)
             30 RETURN_VALUE

Would you expect the overloaded global keyword to apply differently,
or just have slightly different scoping semantics? Would a nonlocal
keyword apply differently? Would we allow multiple function-global
parameter modifiers?

I personally expect that while there is a theoretical clash between
variable names in nested scopes, that's already a poor coding
decision. The module level globals should not unintentionally collide
with function-local non-local access. Thus reusing the global keyword
is not a practical limitation.
-- 
Michael Urman  http://www.tortall.net/mu/blog

From jcarlson at uci.edu  Mon Nov  6 17:05:57 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 06 Nov 2006 08:05:57 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <454EE6C6.4000701@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
Message-ID: <20061106080304.8229.JCARLSON@uci.edu>


Talin <talin at acm.org> wrote:
> I'd like to replace this with:
> 
>     .component( slice_object )
> 
> where the semantics of 'component' are identical to __getitem__ on an 
> array or tuple. So for example:
> 
>     Path( "a", "b" ).component( 0 ) => "a"
>     Path( "a", "b" ).component( 1 ) => "b"
>     Path( "a", "b" ).component( -1 ) => "b"
>     Path( "a", "b" ).component( 0:1 ) => Path( "a", "b" )
>     Path( "a", "b" ).component( 1: ) => Path( "b" )

Use a property or list or something.  Something with __getitem__.  In
2.5 it is a syntax error to use slices like that, and it would seem
silly to change 3.x syntax to be able to pass slices directly like that.


 - Josiah


From jcarlson at uci.edu  Mon Nov  6 17:16:37 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 06 Nov 2006 08:16:37 -0800
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <dcbbbb410611060633q371afbd7o53d5457d433cad83@mail.gmail.com>
References: <Pine.LNX.4.58.0611052306170.6331@server1.LFW.org>
	<dcbbbb410611060633q371afbd7o53d5457d433cad83@mail.gmail.com>
Message-ID: <20061106080829.822C.JCARLSON@uci.edu>


"Michael Urman" <murman at gmail.com> wrote:
> I personally expect that while there is a theoretical clash between
> variable names in nested scopes, that's already a poor coding
> decision. The module level globals should not unintentionally collide
> with function-local non-local access. Thus reusing the global keyword
> is not a practical limitation.

Just saying that they shouldn't be colliding doesn't mean that they
won't.  I've seen horrendous code from some otherwise smart people where
locals shadow globals, even in closures.  I don't have a link for you
because I've tried to block out the experience.  However, in my opinion,
relying on good coding practices (never shadow variables) to guarantee
correctness of the use of the global keyword in Python seems to me to be
silly.

I hope we can all at least agree that *requiring* good coding practices
to make a feature work may be a bit more anal retentive than we want
Python to be.  While I personally don't want the feature in the first
place, if I'm going to be seeing it in code, I would very much rather
that a second keyword were introduced.  If only because then I wouldn't
tear out my hair trying to figure out whether a variable was really in
the module globals, or in just another containing scope.


 - Josiah


From talin at acm.org  Mon Nov  6 17:55:46 2006
From: talin at acm.org (Talin)
Date: Mon, 06 Nov 2006 08:55:46 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <20061106080304.8229.JCARLSON@uci.edu>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org> <20061106080304.8229.JCARLSON@uci.edu>
Message-ID: <454F6912.1030503@acm.org>

Josiah Carlson wrote:
> Talin <talin at acm.org> wrote:
>> I'd like to replace this with:
>>
>>     .component( slice_object )
>>
>> where the semantics of 'component' are identical to __getitem__ on an 
>> array or tuple. So for example:
>>
>>     Path( "a", "b" ).component( 0 ) => "a"
>>     Path( "a", "b" ).component( 1 ) => "b"
>>     Path( "a", "b" ).component( -1 ) => "b"
>>     Path( "a", "b" ).component( 0:1 ) => Path( "a", "b" )
>>     Path( "a", "b" ).component( 1: ) => Path( "b" )
> 
> Use a property or list or something.  Something with __getitem__.  In
> 2.5 it is a syntax error to use slices like that, and it would seem
> silly to change 3.x syntax to be able to pass slices directly like that.

That's unfortunate. I specifically want to avoid saying something like 
path[ 1:2 ], because its ambiguous and confusing, especially if "Path" 
is derived from unicode. I suppose path.components[ 1:2 ] would be all 
right.

>  - Josiah


From ntoronto at cs.byu.edu  Mon Nov  6 19:16:15 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 06 Nov 2006 11:16:15 -0700
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <20061106080829.822C.JCARLSON@uci.edu>
References: <Pine.LNX.4.58.0611052306170.6331@server1.LFW.org>	<dcbbbb410611060633q371afbd7o53d5457d433cad83@mail.gmail.com>
	<20061106080829.822C.JCARLSON@uci.edu>
Message-ID: <454F7BEF.4080903@cs.byu.edu>

Josiah Carlson wrote:

>"Michael Urman" <murman at gmail.com> wrote:
>  
>
>>I personally expect that while there is a theoretical clash between
>>variable names in nested scopes, that's already a poor coding
>>decision. The module level globals should not unintentionally collide
>>with function-local non-local access. Thus reusing the global keyword
>>is not a practical limitation.
>>    
>>
>
>Just saying that they shouldn't be colliding doesn't mean that they
>won't.  I've seen horrendous code from some otherwise smart people where
>locals shadow globals, even in closures.  I don't have a link for you
>because I've tried to block out the experience.  However, in my opinion,
>relying on good coding practices (never shadow variables) to guarantee
>correctness of the use of the global keyword in Python seems to me to be
>silly.
>
>I hope we can all at least agree that *requiring* good coding practices
>to make a feature work may be a bit more anal retentive than we want
>Python to be.  While I personally don't want the feature in the first
>place, if I'm going to be seeing it in code, I would very much rather
>that a second keyword were introduced.  If only because then I wouldn't
>tear out my hair trying to figure out whether a variable was really in
>the module globals, or in just another containing scope.
>  
>

And that's the most important implication of this decision: the health 
of Josiah's hair.

Um, yeah - I agree, actually. This is my biggest beef with reusing 
"global" as well: code using it doesn't say what most people would think 
it says. (My second beef is that there would be no chance of seeing it 
in 2.x.)

There's *always* going to be some mismatch between what the programmer 
believes the code says and what it actually does say. That doesn't mean 
we need to exacerbate the problem. The weirdest thing I've seen in this 
discussion is arguments along the lines of, "Well, it's ambiguous in the 
first place, so why not make it worse?" I honestly can't fathom that 
kind of reasoning, especially given that Python is supposed to be a 
language that's partly designed to be a good user interface.

(Michael, I know you were talking about whether there was any technical 
reason it couldn't be done. My comment is aimed laterally along the 
discussion tree.)

It'd be like saying, "You know, a checkbox is technically a toggle 
button, so why not call this color wheel a button, too? I mean, it has a 
discrete state, and you click on it. It's not that different."

Neil


From brett at python.org  Mon Nov  6 20:33:00 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 6 Nov 2006 11:33:00 -0800
Subject: [Python-3000] Backward compatibility for "nonlocal"
In-Reply-To: <Pine.LNX.4.58.0611051721400.6331@server1.LFW.org>
References: <Pine.LNX.4.58.0611051721400.6331@server1.LFW.org>
Message-ID: <bbaeab100611061133o3b0e7599p405dbcddd227c717@mail.gmail.com>

On 11/5/06, Ka-Ping Yee <python at zesty.ca> wrote:
>
> As an aside to the discussion about "nonlocal", here are a couple of
> thoughts on backward compatibility.
>
> For some of the proposed keywords, here's the number of occurrences
> of the keyword in the current standard library (not including comments
> and docstrings):
>
>     nonlocal      0       0
>     use           2       2
>     using         3       3
>     reuse         0       4
>     free          8       8
>     outer         5     147
>     global      126     214
>
> (I used 'tokenize' to do this; let me know if you'd like me to count
> other possibilities.)
>
> This may or may not be a good measure of the likely impact of adding
> a new keyword.  At least it's something to think about.
>
> It also occurred to me that this new keyword doesn't necessarily
> have to break anything, if we are willing to tolerate a little
> hackery in the Python lexer and parser.  The situation is a bit
> reminiscent of 'as' -- we were able to add 'as' to the 'import'
> statement as though 'as' were a keyword, without actually making
> 'as' a reserved word in all code.
>
> In this case, the usage of "nonlocal" (or whatever we choose) as
> a keyword never overlaps with its usage as a variable name.  The
> parser could easily tell the difference by checking whether:
>
>     - The keyword is at the beginning of a statement.
>     - The keyword is immediately followed by an identifier.
>
> If both of these things are true, then the keyword is a nonlocal
> declaration.  Otherwise, it can be treated as a variable name.
>
> I'm not saying this is necessarily a good idea; i just wanted to
> note that it was an available option.  We might think about a
> schedule for phasing in the feature:
>
>     Phase 1.  If "nonlocal" is seen in code, issue a warning
>               that it will become a keyword in a future version.
>
>     Phase 2.  Have the parser distinguish when "nonlocal" is
>               intended as a keyword and when as a variable name.
>
>     Phase 3.  Make "nonlocal" an official reserved word.
>
> All of these phases are optional -- we could skip straight to 3,
> or do 1 and then 3, or stop at 2, etc.


Any new keyword would require a __future__ statement which automatically
implies a warning in the version that introduces the __future__ statement.
Then the next version makes it a keyword and the __future__ statement
redundant.  So the transition to a new keyword is already established.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061106/2730b3b4/attachment.html 

From ncoghlan at gmail.com  Mon Nov  6 23:31:11 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 07 Nov 2006 08:31:11 +1000
Subject: [Python-3000] Draft PEP for outer scopes
In-Reply-To: <dcbbbb410611060633q371afbd7o53d5457d433cad83@mail.gmail.com>
References: <E1GfYMP-0002mn-5b@swing.co.at>	
	<Pine.LNX.4.58.0611031513560.6331@server1.LFW.org>	
	<1162590716.4428.23.camel@fsol>	
	<Pine.LNX.4.58.0611031556250.6331@server1.LFW.org>	
	<454BE8DB.20203@canterbury.ac.nz>
	<454C0001.1080301@cs.byu.edu>	 <454C0AD4.5090602@gmail.com>	
	<Pine.LNX.4.58.0611052306170.6331@server1.LFW.org>
	<dcbbbb410611060633q371afbd7o53d5457d433cad83@mail.gmail.com>
Message-ID: <454FB7AF.3030909@gmail.com>

Michael Urman wrote:
>>    def weird(local=True):
>>        if local:
>>            n = 1
>>        else:
>>            global n   # [ or nonlocal n, in Nick's ]
>>            n += 1
>>        return n
> 
> I'm not sure what you're both trying to explain here. First off, the
> above code yields a SyntaxWarning. Secondly all its accesses to n are
> global accesses. The global keyword is currently a modifier for the
> whole function. Thus the local parameter is misleading; it should be
> called reset.

We weren't really talking about global/nonlocal in general - I was merely 
pointing out the specific problem that permitting the shortcut form 
(declaration & assignment on the same line) increases the temptation to put 
the scope declaration somewhere other than at the start of the function.

As you point out, the compiler won't care (beyond possibly issuing a warning 
like the one you mention), but it can be misleading for human readers and writers.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From sluggoster at gmail.com  Mon Nov  6 23:37:01 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Mon, 6 Nov 2006 14:37:01 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <454EE6C6.4000701@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
Message-ID: <6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>

My latest idea is something like this:

#### BEGIN
class Path(unicode):
    """Pathname-manipulation methods."""
    pathlib = os.path              # Subclass can specify (posix|nt|mac)path.
    safe_args_only = False    # Glyph can set this to True in a subclass.

class FSPath(object):
    """Filesystem-access methods.

         The constructor takes a path arg and sets self.path to it.  It also
         accepts any other combination of positional args, which are passed
         to the path constructor to create the path.
    """
    path_class = Path    # Subclass can specify an alternate path class.

    def __init__(klass, *args):
        if len(args) == 1 and isinstance(args[0], klass.path_class):
            self.path = args[0]
        else:
            self.path = self.path_class(*args)

    @classmethod
    def cwd(klass):
        """Convenience method for this common operation."""
        path = klass.path_klass.cwd()
        rerutn klass(path)
#### END

This should be versatile enough to handle several of the alternatives
that have been proposed.  Those who want Path and FSPath separate have
it.  Those who want them together can use FSPath and FSPath.path.
FSPath can itself be broken into two levels: a "medium" level that
minimally expresses the os.*, os.path.*, and shutil.*
filesystem-access functions, and an "enhanced" level that has all my
favorite methods. Those who think one or another level is evil can use
just the lower levels.

This would satisfy those (like me) who want something we can use now
in our Python 2.4/2.5 programs, and there will be working code useful
for lobbying that one or more levels should be adopted into the
stdlib.  If Path contains minimal enhancements, it would have the best
chance.

Subclassing unicode would be the simplest implementation.  The PEP 355
implementation does a unicode-or-str dance in case
os.path.supports_unicode_filenames is false.  Is it really necessary
to support this nowadays?  I'd like to promote all str's to unicode in
the constructor so that any possible UnicodeDecodeErrors are localized
in one place.


On 11/5/06, Talin <talin at acm.org> wrote:
> Mike Orr wrote:
> >     Path(  Path("directory"),   "subdirectory", "file")    # Replaces
> > .joinpath().
>
> For the constructor, I would write it as:
>
>    Path( *components )

Yes.  I was just trying to show what the argument objects where.

> Strings can also be wrapped with an object that indicates that the Path
> is in a platform- or application-specific format:
>
>     # Explicitly indicate that the path string is in Windows NTFS format.
>     Path( Path.format.NTFS( "C:\\Program Files" ) )

This (and all your other .format examples) sounds a bit complicated.
The further we stray from the syntax/semantics of the existing stdlib
modules, the harder it will be to achieve consensus, and thus the less
chance we'll get anything into the stdlib.  So I'm for a scalable
solution, where the lower levels are less controversial and well
tested, and we can experiment in the higher levels.  Can you make your
.format proposal into an optional high-level component and we'll see
how well it works in practice?

> One question to be asked is whether the path should be simplified or
> not. There are cases where you *don't* want the path to be simplified,
> and other cases where you do. Perhaps a keyword argument?
>
>     Path( "C:\\Program Files", "../../Gimp", normalize = True )

Maybe.  I'm inclined to let an .only_safe_args attribute or a SafePath
subclass enforce normalizing, and let the main class do whatever
os.path.join() does.

> >     Path("ab") + "c"  => Path("abc")
>
> Wouldn't that be:
>
>     Path( "ab" ) + "c" => Path( "ab", "c" )

If we want string compatibility we can't redefine the '+' operator.
If we ditch string compatibility we can't pass Paths to functions
expecting a string.  We can't have it both ways.  This also applies to
character slicing vs component slicing.

> >     .abspath()
>
> I've always thought this was a strange function. To be honest, I'd
> rather explicitly pass in the cwd().

I use it; it's convenient.  The method name could be improved.

> >     .normcase()
> >     .normpath()

... and other methods.  Your proposals are all in the context of "how
much do we want to improve os.path's syntax/semantics vs keeping them
as-is?"  I would certainly like improvement, but improvement works
against consensus. Some people don't want anything to change beyond a
minimal class wrapper.  Others want improvements but have differing
views about what the best "improvements" are.  So how far do you want
to go, and how does this impact your original question, "Is consensus
possible?"

Then, if consensus is not possible, what do we do?  Each go into our
corners and make our favorite incompatible module?  Or can we come up
with a master plan containing alternatives, to bring at least some
unity to the differing modules without cramping anybody's style.

In this vein, a common utility module with back-end functions would be
good.  Then we can solve the difficult problems *once* and have a test
suite that proves it, and people would have confidence using any OO
classes that are built over them.  We can start by gathering the
existing os.*, os.path.*, and shutil.* functions, and then add
whatever other functions our various OO classes might need.

However, due to the problem of supporting (posix|nt|mac)path, we may
need to express this as a class of classmethods rather than a set of
functions, so they can be defined relative to a platform library.

> >     .realpath()
>
> Rename to resolve() or resolvelinks().

Good idea.

> >     .expanduser()
> >     .expandvars()
> >     .expand()
>
> Replace with expand( user=True, vars=True )

Perhaps.  There was one guy in the discussion about Noam's path module
who didn't like .expand() at all; he thought it did too many things
implicitly and was thus too magical.

> >     .parent
>
> If parent was a function, you could pass in the number of levels go to
> up, i.e. parent( 2 ) to get the grandparent.

I'd like .ancestor(N) for that.  Parent as a property is nice when
it's only one or two levels.

>
> >     .name                 # Full filename without path
> >     .namebase        # Filename without extension
>
> I find the term 'name' ambiguous. How about:
>
>      .filepart
>      .basepart
>
> or:
>
>      .filename
>      .basename

.name/.namebase isn't great, but nothing else that's been proposed is better.

> >     .drive
>
> Do we need to somehow unify the concept of 'drive' and 'unc' part? Maybe
> '.device' could return the part before the first directory name.

This gets into the "root object" in Noam's proposal.  I'd say just
read that and the discussion, and see if it approaches what you want.
I find this another complicated and potential bog-down point, like
.format.

http://wiki.python.org/moin/AlternativePathClass
http://wiki.python.org/moin/AlternativePathDiscussion

> >     .splitpath()
>
> I'd like to replace this with:
>
>     .component( slice_object )
>
> where the semantics of 'component' are identical to __getitem__ on an
> array or tuple. So for example:
>
>     Path( "a", "b" ).component( 0 ) => "a"
>     Path( "a", "b" ).component( 1 ) => "b"
>     Path( "a", "b" ).component( -1 ) => "b"
>     Path( "a", "b" ).component( 0:1 ) => Path( "a", "b" )
>     Path( "a", "b" ).component( 1: ) => Path( "b" )
>
> This is essentially the same as the "slice notation" proposal given
> earlier, except that explicitly tell the user that we are dealing with
> path components, not characters.

    Path("a/b").components[0:1] => Path("a/b")

Is there a problem with .component returning a Path instead of a list
of components?

In some ways I still like Noam's Path-as-components idea.  It
eliminates all slicing methods, and '+' does join.  The problem is
you'd have to explicitly unicode() it when passing it to functions
that expect a string. I guess the advantage of Path being unicode
still outweigh the disadvantages.

Here's one possibility for splitting the absolute/relative part of a path:

    Path("/a/b").absolute_prefix => "/"
    relative_start_point = len(Path("/a/b").absolute_prefix)

It would have to be defined for all platforms.  Or we can have a
splitroot method:

    Path("/a/b").splitroot()  =>  [Path("/"), Path("a/b")]

Not sure which way would be most useful overall.


> >     .stripext()
>
> How about:
>
>      path.ext = ''

The discussion in Noam's proposal has .add_exts(".tar", ".gz") and
.del_exts(N).  Remember that any component can have extension(s), not
just the last.  Also, it's up to the user which apparent extensions
should be considered extensions.  How many extensions does
"python-2.4.5-i386.2006-12-12.orig.tar.gz" have?

> >     .splitall()
>
> Something sadly lacking in os.path.

I thought this was what .splitpath() would do.

> >     .relpathto()
>
> Not sure what this does, since there's no argument defined.

>From Orendorff's commentary.
"The method p1.relpathto(p2) returns a relative path to p2, starting from p1."
http://www.jorendorff.com/articles/python/path/

I've always found it confusing to remember which one is 'from' and
which one is 'to'?

-- 
Mike Orr <sluggoster at gmail.com>

From solipsis at pitrou.net  Tue Nov  7 00:08:08 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 07 Nov 2006 00:08:08 +0100
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
Message-ID: <1162854488.4384.29.camel@fsol>

Le lundi 06 novembre 2006 ? 14:37 -0800, Mike Orr a ?crit :
>     def __init__(klass, *args):
>         if len(args) == 1 and isinstance(args[0], klass.path_class):
>             self.path = args[0]
>         else:
>             self.path = self.path_class(*args)

s/klass/self/, I suppose ?

> Subclassing unicode would be the simplest implementation.  The PEP 355
> implementation does a unicode-or-str dance in case
> os.path.supports_unicode_filenames is false.  Is it really necessary
> to support this nowadays?

I'm not sure what you mean, but on Mandriva Linux:

$ python
Python 2.4.3 (#2, Sep 18 2006, 21:07:35) 
[GCC 4.1.1 20060724 (prerelease) (4.1.1-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.supports_unicode_filenames
False

However:
>>> os.path.exists(u"/etc/passwd")
True
>>> os.path.exists("/etc/passwd")
True
>>> os.path.exists(u"?l?phant")
True
>>> os.path.exists("?l?phant")
True

(after having run "touch ?l?phant")


>     Path("a/b").components[0:1] => Path("a/b")
> 
> Is there a problem with .component returning a Path instead of a list
> of components?

Having a "list slice" returning a non-sequence result is a bit
surprising; the plural "components" is also misleading. Why not simply a
method, and why not name it "subpath" ?

	Path("a/b/c").subpath(0, 2) => Path("a/b")
	Path("a/b/c").subpath(0, -1) => Path("a/b")

By the way, is the absolute root a separate "component"?

	Path("/a/b/c").subpath(0, 1) => Path("/")
	Path("/a/b/c").subpath(1, 2) => Path("a")
	Path("/a/b/c").subpath(1) => Path("a/b/c")




From rrr at ronadam.com  Tue Nov  7 02:41:06 2006
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 06 Nov 2006 19:41:06 -0600
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
Message-ID: <eiooeh$urg$1@sea.gmane.org>

> [Mike Orr wrote:]

> In this vein, a common utility module with back-end functions would be
> good.  Then we can solve the difficult problems *once* and have a test
> suite that proves it, and people would have confidence using any OO
> classes that are built over them.  We can start by gathering the
> existing os.*, os.path.*, and shutil.* functions, and then add
> whatever other functions our various OO classes might need.


This really sounds like a situation where writing the test cases first would be 
worth while.

So maybe it would be good to start by gathering the relevant examples and 
agreeing on the desired behavior in various situations as a doctest suite.

Cheers,
    Ron


From talin at acm.org  Tue Nov  7 10:15:48 2006
From: talin at acm.org (Talin)
Date: Tue, 07 Nov 2006 01:15:48 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
Message-ID: <45504EC4.50609@acm.org>

Mike Orr wrote:
> My latest idea is something like this:
> 
> #### BEGIN
> class Path(unicode):
>     """Pathname-manipulation methods."""
>     pathlib = os.path              # Subclass can specify (posix|nt|mac)path.
>     safe_args_only = False    # Glyph can set this to True in a subclass.
>

I'm a little confused here about the model of how platform-specific and 
application-specific formats are represented. Is it the case that the 
creation function converts the platform-specific path into a generic, 
universal path object, or does it create a platform-specific path object?

In the former case -- where you create a platform-agnostic "universal" 
path object - the question is, how do you then 'render' that into a 
platform-specific string format?

And in the latter case, where the 'path objects' are themselves in 
platform-specific form, how do you cross-convert from one format to another?

One thing I really want to avoid is the situation where the only 
available formats are those that are built-in to the path module.

> On 11/5/06, Talin <talin at acm.org> wrote:
>> Mike Orr wrote:
>>>     Path(  Path("directory"),   "subdirectory", "file")    # Replaces
>>> .joinpath().
>> For the constructor, I would write it as:
>>
>>    Path( *components )
> 
> Yes.  I was just trying to show what the argument objects where.
> 
>> Strings can also be wrapped with an object that indicates that the Path
>> is in a platform- or application-specific format:
>>
>>     # Explicitly indicate that the path string is in Windows NTFS format.
>>     Path( Path.format.NTFS( "C:\\Program Files" ) )
> 
> This (and all your other .format examples) sounds a bit complicated.
> The further we stray from the syntax/semantics of the existing stdlib
> modules, the harder it will be to achieve consensus, and thus the less
> chance we'll get anything into the stdlib.  So I'm for a scalable
> solution, where the lower levels are less controversial and well
> tested, and we can experiment in the higher levels.  Can you make your
> .format proposal into an optional high-level component and we'll see
> how well it works in practice?

First, ignore the syntax of my examples - its really the semantics that 
I want to get across.

I don't think you need to follow too closely the syntax of os.path - 
rather, we should concentrate on the semantics, and even more 
importantly, the scope of the existing module. In other words don't try 
to do too much more than os.path did.

>> One question to be asked is whether the path should be simplified or
>> not. There are cases where you *don't* want the path to be simplified,
>> and other cases where you do. Perhaps a keyword argument?
>>
>>     Path( "C:\\Program Files", "../../Gimp", normalize = True )
> 
> Maybe.  I'm inclined to let an .only_safe_args attribute or a SafePath
> subclass enforce normalizing, and let the main class do whatever
> os.path.join() does.

For my own code, I want to simplify 100% of the time. Having to insert 
the extra call to normpath() everywhere clutters up the code fairly 
egregiously.

The whole issue of path simplification is really an "end user debugging" 
issue. Since simplified and non-simplified paths are equivalent, the 
only time it really matters whether or not a path is simplified is when 
the end-user sees a path; And in my experience, this generally only 
happens in error messages such as "File xxx not found" or in batch 
programs that take a list of files as input. Moreover, GUI programs that 
use a file open dialog to choose files generally don't show the full 
path when an error occurs. So really we're limited to the case where you 
have some sort of app that is either script-driven or has a user-edited 
config file (like Make) in which there are fragments of paths being 
joined together.

As the designer of such an app, the question you want to ask yourself 
is, will it be easier for the end user to diagnose the error if we show 
them the simplified result of the concatenation, or the unsimplified, 
concatenated strings. In my case, I almost always prefer to be shown the 
simplified path so that I don't have to mentally simplify all of the 
../../.. bits before looking at that particular location in the 
filesystem to see what the problem is.

>>>     Path("ab") + "c"  => Path("abc")
>> Wouldn't that be:
>>
>>     Path( "ab" ) + "c" => Path( "ab", "c" )
> 
> If we want string compatibility we can't redefine the '+' operator.
> If we ditch string compatibility we can't pass Paths to functions
> expecting a string.  We can't have it both ways.  This also applies to
> character slicing vs component slicing.

If the '+' operator can't be used, then we need to specify how paths are 
joined. Or is it your intent that the only way to concatenate paths be 
via the constructor?

I'm in favor of the verb 'combine' being used to indicate both a joining 
and a simplification of a path.

>>>     .abspath()
>> I've always thought this was a strange function. To be honest, I'd
>> rather explicitly pass in the cwd().
> 
> I use it; it's convenient.  The method name could be improved.

The thing that bugs me about it is that it relies on a hidden variable, 
in this case cwd(). Although that's partly my personal experience 
speaking, since a lot of the work I do is writing library code on game 
consoles such as Xbox360 and PS3: These platforms have a filesystem, but 
no concept of a "current working directory", so if the app wants to have 
something like a cwd, it has to maintain the variable itself.

>>>     .normcase()
>>>     .normpath()
> 
> ... and other methods.  Your proposals are all in the context of "how
> much do we want to improve os.path's syntax/semantics vs keeping them
> as-is?"  I would certainly like improvement, but improvement works
> against consensus. Some people don't want anything to change beyond a
> minimal class wrapper.  Others want improvements but have differing
> views about what the best "improvements" are.  So how far do you want
> to go, and how does this impact your original question, "Is consensus
> possible?"
> 
> Then, if consensus is not possible, what do we do?  Each go into our
> corners and make our favorite incompatible module?  Or can we come up
> with a master plan containing alternatives, to bring at least some
> unity to the differing modules without cramping anybody's style.

What I'm trying to do is refactor os.path as we go - it seems to me that 
it would be a shame to simply shovel over the existing functions and 
simply make them object methods, especially (as several people have 
pointed out) since many of them can be combined into more powerful and 
more general methods.

As far as consensus goes: Consensus is not hard to get if you can get it 
piecemeal. People with nitpick at little details of the proposal 
forever, but that's not a huge obstacle to adoption. However, getting 
them to swallow one huge lump of a concept is much harder.

It seems that once you can get people to accept the basic premise of a 
path object, you are home free - the rest is niggling over details. So I 
don't see that this harms the chance of adoption, as long as people are 
allowed to tweak the details of the proposal.

> In this vein, a common utility module with back-end functions would be
> good.  Then we can solve the difficult problems *once* and have a test
> suite that proves it, and people would have confidence using any OO
> classes that are built over them.  We can start by gathering the
> existing os.*, os.path.*, and shutil.* functions, and then add
> whatever other functions our various OO classes might need.
> 
> However, due to the problem of supporting (posix|nt|mac)path, we may
> need to express this as a class of classmethods rather than a set of
> functions, so they can be defined relative to a platform library.
> 
>>>     .realpath()
>> Rename to resolve() or resolvelinks().
> 
> Good idea.
> 
>>>     .expanduser()
>>>     .expandvars()
>>>     .expand()
>> Replace with expand( user=True, vars=True )
> 
> Perhaps.  There was one guy in the discussion about Noam's path module
> who didn't like .expand() at all; he thought it did too many things
> implicitly and was thus too magical.

Well I'd say leave it in then, and see who objects.

>>>     .parent
>> If parent was a function, you could pass in the number of levels go to
>> up, i.e. parent( 2 ) to get the grandparent.
> 
> I'd like .ancestor(N) for that.  Parent as a property is nice when
> it's only one or two levels.

Reasonable

>>>     .name                 # Full filename without path
>>>     .namebase        # Filename without extension
>> I find the term 'name' ambiguous. How about:
>>
>>      .filepart
>>      .basepart
>>
>> or:
>>
>>      .filename
>>      .basename
> 
> .name/.namebase isn't great, but nothing else that's been proposed is better.

If all else fails, copy what someone else has done:

http://msdn2.microsoft.com/en-us/library/system.io.path_members(VS.80).aspx

>>>     .drive
>> Do we need to somehow unify the concept of 'drive' and 'unc' part? Maybe
>> '.device' could return the part before the first directory name.
> 
> This gets into the "root object" in Noam's proposal.  I'd say just
> read that and the discussion, and see if it approaches what you want.
> I find this another complicated and potential bog-down point, like
> .format.
> 
> http://wiki.python.org/moin/AlternativePathClass
> http://wiki.python.org/moin/AlternativePathDiscussion
> 
>>>     .splitpath()
>> I'd like to replace this with:
>>
>>     .component( slice_object )
>>
>> where the semantics of 'component' are identical to __getitem__ on an
>> array or tuple. So for example:
>>
>>     Path( "a", "b" ).component( 0 ) => "a"
>>     Path( "a", "b" ).component( 1 ) => "b"
>>     Path( "a", "b" ).component( -1 ) => "b"
>>     Path( "a", "b" ).component( 0:1 ) => Path( "a", "b" )
>>     Path( "a", "b" ).component( 1: ) => Path( "b" )
>>
>> This is essentially the same as the "slice notation" proposal given
>> earlier, except that explicitly tell the user that we are dealing with
>> path components, not characters.
> 
>     Path("a/b").components[0:1] => Path("a/b")
> 
> Is there a problem with .component returning a Path instead of a list
> of components?

I don't see why we can't have both:

    path.component[ a:b ] # returns a list of components
    path.subpath[ a:b ] # returns a (possibly non-rooted) Path object

Note also that this would mean that ".splitall()" is no longer needed, 
since it can be replaced by "path.component[:]". (More refactoring...woot!)

Another thing I like about the slice syntax is that you can also control 
whether or not the return value is a subsequence or not:

    path.component[ a ] # Gets the ath component as a string
    path.component[ a:a+1 ] # Gets a list containing the ath component

This distinction is not as easily replicated using a function-call 
syntax such as path.component( a, a+1 ). Although in the case of 
subpath, I am not sure what the distinction is.

(On the naming of 'component' vs. 'components' - my general naming 
convention is that array names are plurals - so a table of primes is 
called 'primes' not 'prime'.)

> In some ways I still like Noam's Path-as-components idea.  It
> eliminates all slicing methods, and '+' does join.  The problem is
> you'd have to explicitly unicode() it when passing it to functions
> that expect a string. I guess the advantage of Path being unicode
> still outweigh the disadvantages.
> 
> Here's one possibility for splitting the absolute/relative part of a path:
> 
>     Path("/a/b").absolute_prefix => "/"
>     relative_start_point = len(Path("/a/b").absolute_prefix)
> 
> It would have to be defined for all platforms.  Or we can have a
> splitroot method:
> 
>     Path("/a/b").splitroot()  =>  [Path("/"), Path("a/b")]
> 
> Not sure which way would be most useful overall.
> 
> 
>>>     .stripext()
>> How about:
>>
>>      path.ext = ''
> 
> The discussion in Noam's proposal has .add_exts(".tar", ".gz") and
> .del_exts(N).  Remember that any component can have extension(s), not
> just the last.  Also, it's up to the user which apparent extensions
> should be considered extensions.  How many extensions does
> "python-2.4.5-i386.2006-12-12.orig.tar.gz" have?

A directory name with a '.' in it isn't really an extension, at least 
not as interpreted by most filesystems. For example, if you create a 
folder in MSWindows, and name it "foo.bar" and then look at it in 
windows explorer, it will still say it's a folder; It won't try and 
display it as a "folder of type 'bar'". Similarly, if you are using 
LSCOLORS under posix, and you have a directory with a dot in the name, 
it still shows up in the same color as other dirs.

In any case, I really don't think we need to support any special 
accessors for accessing the part after the '.' in any component but the 
last.

As far as multiple extensions go - the easiest thing would be to simply 
treat the very last part - '.gz' in your example - as the extension, and 
let the user worry about the rest. I only know of one program - GNU tar 
- that attempts to interpret multiple file extensions in this way. (And 
you'll notice that most of the dots in the example are version number 
separators, not file extension separators.)

I'll go further out on a limb here, and say that interpreting the file 
extension really isn't the path library's job, and the only reason why 
this function is here at all is to prevent novice programmers from 
erroneously calling str.rfind( '.' ) on the path string, which will of 
course yield the wrong answer if the filename has no dot in it but a 
directory name does.

>>>     .splitall()
>> Something sadly lacking in os.path.
> 
> I thought this was what .splitpath() would do.

.splitpath only splits into two pieces, a dirname and a filename.

>>>     .relpathto()
>> Not sure what this does, since there's no argument defined.
> 
>>From Orendorff's commentary.
> "The method p1.relpathto(p2) returns a relative path to p2, starting from p1."
> http://www.jorendorff.com/articles/python/path/
> 
> I've always found it confusing to remember which one is 'from' and
> which one is 'to'?


From sluggoster at gmail.com  Tue Nov  7 23:06:42 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 7 Nov 2006 14:06:42 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <45504EC4.50609@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
Message-ID: <6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>

On 11/7/06, Talin <talin at acm.org> wrote:
> Mike Orr wrote:
> > My latest idea is something like this:
> >
> > #### BEGIN
> > class Path(unicode):
> >     """Pathname-manipulation methods."""
> >     pathlib = os.path              # Subclass can specify (posix|nt|mac)path.
> >     safe_args_only = False    # Glyph can set this to True in a subclass.
> >
>
> I'm a little confused here about the model of how platform-specific and
> application-specific formats are represented. Is it the case that the
> creation function converts the platform-specific path into a generic,
> universal path object, or does it create a platform-specific path object?
>
> In the former case -- where you create a platform-agnostic "universal"
> path object - the question is, how do you then 'render' that into a
> platform-specific string format?
>
> And in the latter case, where the 'path objects' are themselves in
> platform-specific form, how do you cross-convert from one format to another?
>
> One thing I really want to avoid is the situation where the only
> available formats are those that are built-in to the path module.


My Path object is platform-specific.  By default it uses the "native"
path module (os.path).  If you need non-native paths, subclass Path
and set .pathlib to the other path module (e.g., ntpath, posixpath).
To convert paths we can make the constructor smarter; e.g.,

    class NTPath(Path):
        pathlib = ntpath
    foo = Path("foo")
    my_nt_path = NTPath(foo)

The constructor might recognize that Path.pathlib != NTPath.pathlib
and call foo.to_universal() to get a universal format, which it would
then convert.

Thinking further, why not use a component tuple as a universal format,
with the first element being the root (or '' for a relative path).
This is easy to convert to/from any platform string, won't be mistaken
for a platform string, and won't give one platform an advantage over
another.  We can make a pseudo Path class for the universal format,
the only restriction being that you can't pass it directly to a
filesystem-access function.  Actually, you can't pass *any* non-native
path to a filesystem-access function anyway.

Remember that 99% of Python programmers are concerned only with native
paths.  I have never used a non-native path or multiple-platform paths
in an application.  So we need to make the native case easy and clear.
 For that reason I'd rather keep the non-native cases and conversion
code in separate classes.


> I don't think you need to follow too closely the syntax of os.path -
> rather, we should concentrate on the semantics, and even more
> importantly, the scope of the existing module. In other words don't try
> to do too much more than os.path did.


I think a layered approach is important.  Use the code in the existing
modules because it's well tested.  Add a native-path object on top of
that.  Put non-native paths and conversions on the side.  Then put a
filesystem-access class (or functions) on top of the native-path
object.  Then high-level functions/methods on top of that.  Then when
we lobby for stdlib inclusion it'll be "one level and everything below
it".  People can see and test that level, and ignore the (possibly
more controversial) levels above it.

Of course, the layers can be collapsed at a later date if desired.
And the os.path/os/shutil functions can be inlined at some point if
it's decided to deprecate them.  But that's too much for now.

I would really like to improve on os.path and os/shutil.  That's a
separate issue from the class structure I've proposed.  My only reason
for making a thin wrapper would be for greater acceptability (lowest
common denominator).  But I'd rather work on a thick wrapper whose
methods corresponds to what the application programer conceptually
wants to do, rather than having to specify every piddly step (rmtree
if a directory exists, remove if a file exists, nothing if nothing
exists).

> >> One question to be asked is whether the path should be simplified or
> >> not. There are cases where you *don't* want the path to be simplified,
> >> and other cases where you do. Perhaps a keyword argument?
> >>
> >>     Path( "C:\\Program Files", "../../Gimp", normalize = True )
> >
> > Maybe.  I'm inclined to let an .only_safe_args attribute or a SafePath
> > subclass enforce normalizing, and let the main class do whatever
> > os.path.join() does.
>
> For my own code, I want to simplify 100% of the time. Having to insert
> the extra call to normpath() everywhere clutters up the code fairly
> egregiously.


That is what I'd prefer.  Let the constructor call normpath().  Maybe
we can just say we don't support non-normalized paths, and if you need
that you can write a NonNormalizedPath subclass or use os.path
directly.

My pet peeve is paths beginning with "./" .  I want that out of error
messages and diagnostic messages!


> >>>     Path("ab") + "c"  => Path("abc")
> >> Wouldn't that be:
> >>
> >>     Path( "ab" ) + "c" => Path( "ab", "c" )
> >
> > If we want string compatibility we can't redefine the '+' operator.
> > If we ditch string compatibility we can't pass Paths to functions
> > expecting a string.  We can't have it both ways.  This also applies to
> > character slicing vs component slicing.
>
> If the '+' operator can't be used, then we need to specify how paths are
> joined. Or is it your intent that the only way to concatenate paths be
> via the constructor?
>
> I'm in favor of the verb 'combine' being used to indicate both a joining
> and a simplification of a path.


I like the syntax of a join method.  With a multi-arg constructor it's
not necessary though. PEP 355 hints at problems with the .joinpath()
method though it didn't say what they were.   .combine() is an OK
name, and there's a precedent in the datetime module.  But people are
used to calling it "joining paths" so I'm not sure we should rename it
so easily.  We can't call it .join due to string compatibility.
.joinpath is ugly if we delete "path" from the other method names.
.pjoin comes to mind but people might find it too cryptic and too
similar to .join.  By just using the constructor we avoid the debate
over the method name.


> >>>     .abspath()
> >> I've always thought this was a strange function. To be honest, I'd
> >> rather explicitly pass in the cwd().
> >
> > I use it; it's convenient.  The method name could be improved.
>
> The thing that bugs me about it is that it relies on a hidden variable,
> in this case cwd(). Although that's partly my personal experience
> speaking, since a lot of the work I do is writing library code on game
> consoles such as Xbox360 and PS3: These platforms have a filesystem, but
> no concept of a "current working directory", so if the app wants to have
> something like a cwd, it has to maintain the variable itself.


Again, 99.9% of Python users have a functioning cwd, and .abspath() is
one of those things people expect in any programming language. Nobody
is forcing you to call it on the Xbox.  However, I'm not completely
opposed to dropping .abspath().

> It seems that once you can get people to accept the basic premise of a
> path object, you are home free - the rest is niggling over details. So I
> don't see that this harms the chance of adoption, as long as people are
> allowed to tweak the details of the proposal.


The syntax details do matter.  They won't necessarily make or break a
proposal but they do make people's support for it stronger or weaker.
But flexibility is a good thing.  If we have a well-chosen structure
but make it easy for people to subclass it and rename methods if they
can't stand our choices, they might accept it anyway.  This also means
not hardwiring class relationships.  For instance, Path should create
paths via self.__class__ rather than calling Path literally, and
FSPath should call self.path_class rather than calling Path directly.
This makes it easy for somebody to write a "better" Path class and
have FSPath use it.


> > In this vein, a common utility module with back-end functions would be
> > good.  Then we can solve the difficult problems *once* and have a test
> > suite that proves it, and people would have confidence using any OO
> > classes that are built over them.  We can start by gathering the
> > existing os.*, os.path.*, and shutil.* functions, and then add
> > whatever other functions our various OO classes might need.
> >
> > However, due to the problem of supporting (posix|nt|mac)path, we may
> > need to express this as a class of classmethods rather than a set of
> > functions, so they can be defined relative to a platform library.


Actually, this may not be a problem after all.  The filesystem-access
functions are universal; e.g., there's only one os.remove().  And they
only make sense with native paths.  So we can call os.path.exists()
from our purge() or super_duper_copy() function and be happy.


>     path.component[ a:b ] # returns a list of components
>     path.subpath[ a:b ] # returns a (possibly non-rooted) Path object


Great.  I was wondering how to handle the two cases.

    path.components[a:b]    =>  list of components
    path.components[a]        =>  one component
    path.subpath(a, b)          =>  a path   (relative if a != 0)

Now if

     Path("/usr/bin/python").components  => ["/", "usr", "bin", "python"]

then we have the universal format I described above.  That's two birds
down with one stone.

I think .subpath should be a method though.  I can't think of another
case where slicing returns a non-sequence.  If a defaults to 0 and b
to -1, it'll be easy for people to get the beginning or end of the
path using normal Python subscripts.  Or we can even allow None for
the endpoinds to make it super easy.

>     path.component[ a ] # Gets the ath component as a string
>     path.component[ a:a+1 ] # Gets a list containing the ath component
>
> This distinction is not as easily replicated using a function-call
> syntax such as path.component( a, a+1 ). Although in the case of
> subpath, I am not sure what the distinction is.


There is no distinction.  It's not a sequence in either case.  (Or at
least it's not *that* kind of sequence.  It's a subclass of unicode so
it's a sequence of characters.)

> (On the naming of 'component' vs. 'components' - my general naming
> convention is that array names are plurals - so a table of primes is
> called 'primes' not 'prime'.)

So we agree that .components is better, if I understand you.


> > The discussion in Noam's proposal has .add_exts(".tar", ".gz") and
> > .del_exts(N).  Remember that any component can have extension(s), not
> > just the last.  Also, it's up to the user which apparent extensions
> > should be considered extensions.  How many extensions does
> > "python-2.4.5-i386.2006-12-12.orig.tar.gz" have?
>
> A directory name with a '.' in it isn't really an extension, at least
> not as interpreted by most filesystems. For example, if you create a
> folder in MSWindows, and name it "foo.bar" and then look at it in
> windows explorer, it will still say it's a folder; It won't try and
> display it as a "folder of type 'bar'". Similarly, if you are using
> LSCOLORS under posix, and you have a directory with a dot in the name,
> it still shows up in the same color as other dirs.
>
> In any case, I really don't think we need to support any special
> accessors for accessing the part after the '.' in any component but the
> last.
>
> As far as multiple extensions go - the easiest thing would be to simply
> treat the very last part - '.gz' in your example - as the extension, and
> let the user worry about the rest. I only know of one program - GNU tar
> - that attempts to interpret multiple file extensions in this way. (And
> you'll notice that most of the dots in the example are version number
> separators, not file extension separators.)
>
> I'll go further out on a limb here, and say that interpreting the file
> extension really isn't the path library's job, and the only reason why
> this function is here at all is to prevent novice programmers from
> erroneously calling str.rfind( '.' ) on the path string, which will of
> course yield the wrong answer if the filename has no dot in it but a
> directory name does.


People need to add/delete/replace extensions, and they don't want to
use character slicing / .rfind / .endswith / len() / + to do it.  They
expect the library to at least handle extension splitting as well as
os.path does.  Adding a few convenience methods would be unobtrusive
and express people really want to do:

    p2 = p.add_ext(".tar")
    p2 = p.del_ext()
    p2 = Path("foo.gzip").replace_ext(".bz2")

But what harm is there in making them scalable to multiple extensions?

    .add_exts(*exts)
    .del_exts(N)
    .replace_exts(N, *exts)

You're right that directory extensions are rare.  Maybe we should just
support extensions on the last path component.


-- 
Mike Orr <sluggoster at gmail.com>

From sluggoster at gmail.com  Tue Nov  7 23:36:38 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 7 Nov 2006 14:36:38 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
	<6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
Message-ID: <6e9196d20611071436hebe17bfl9049f2af45ae3311@mail.gmail.com>

How do you convert an absolute path anyway?

PosixPath(NTPath("C:\winnt\system")) => ??
NTPath(PosixPath("/mnt/cdrom")  => ??

You can convert them to "/winnt/system" and "\mnt\cdrom", but what's
the point?  They won't exist anyway.

-- 
Mike Orr <sluggoster at gmail.com>

From greg.ewing at canterbury.ac.nz  Wed Nov  8 02:38:23 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 08 Nov 2006 14:38:23 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <45504EC4.50609@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
Message-ID: <4551350F.7070903@canterbury.ac.nz>

Talin wrote:

> I'm a little confused here about the model of how platform-specific and 
> application-specific formats are represented. Is it the case that the 
> creation function converts the platform-specific path into a generic, 
> universal path object, or does it create a platform-specific path object?

I don't think it makes sense to mix different platform's
path objects at all. They're not just different ways of
representing the same thing, they're actually different
things.

I would drop any notion of format specifiers when
constructing or joining paths. If you really want to do
something like that, you can split it into components and
construct a new path object of a different type from those
components. And then it's up to you to deal with whatever
problems arise, e.g. how to get a Windows drive letter into
a Unix path object.

> The whole issue of path simplification is really an "end user debugging" 
> issue. Since simplified and non-simplified paths are equivalent

Actually, they're not -- it's possible for simplified and
non-simplified versions of the same path to refer to different
filesystem objects. So I'd say that the path object should
never try to simplify on its own initiative, except in ways
that can't possibly alter the semantics of the path.

>>>>    .abspath()
>>>
>>>I've always thought this was a strange function. To be honest, I'd
>>>rather explicitly pass in the cwd().
>>
>>I use it; it's convenient.  The method name could be improved.

It does violate the constraint of the path object being
restricted to path algebra operations, though, since the
result depends on the cwd of the process.

>>Perhaps.  There was one guy in the discussion about Noam's path module
>>who didn't like .expand() at all; he thought it did too many things
>>implicitly and was thus too magical.

And again, it strays outside the domain of path algebra
operations.

>>>Do we need to somehow unify the concept of 'drive' and 'unc' part? Maybe
>>>'.device' could return the part before the first directory name.

These sound like they belong in the realm of extensions
belonging to platform-specific path subclasses.

> A directory name with a '.' in it isn't really an extension, at least 
> not as interpreted by most filesystems. For example, if you create a 
> folder in MSWindows, and name it "foo.bar" and then look at it in 
> windows explorer, it will still say it's a folder; It won't try and 
> display it as a "folder of type 'bar'".

Not true on MacOSX, where some kinds of directories
do indeed have meaningful extensions -- eg. .app,
.framework.

> In any case, I really don't think we need to support any special 
> accessors for accessing the part after the '.' in any component but the 
> last.

That's probably true.

--
Greg

From sluggoster at gmail.com  Wed Nov  8 03:46:59 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 7 Nov 2006 18:46:59 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551350F.7070903@canterbury.ac.nz>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org> <4551350F.7070903@canterbury.ac.nz>
Message-ID: <6e9196d20611071846l1f0b4dddjc689a9991cdab9eb@mail.gmail.com>

On 11/7/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> >>>>    .abspath()
> >>>
> >>>I've always thought this was a strange function. To be honest, I'd
> >>>rather explicitly pass in the cwd().
> >>
> >>I use it; it's convenient.  The method name could be improved.
>
> It does violate the constraint of the path object being
> restricted to path algebra operations, though, since the
> result depends on the cwd of the process.

What do we do with Path.cwd() then?  It also violates the
path-algrebra-only logic.  How else will people get the current
directory?  We can put it on FSPath, but then it's the only FSPath
method that returns a Path, so you'd be passing FSPath to Path to
FSPath.

> >>Perhaps.  There was one guy in the discussion about Noam's path module
> >>who didn't like .expand() at all; he thought it did too many things
> >>implicitly and was thus too magical.
>
> And again, it strays outside the domain of path algebra
> operations.

This is also the same issue.  Where do we put the .expand*() methods
if not on Path?

Is there an actual case where calling normpath() would change which
file the path referred to?  Any case that's not handled by
(posix|nt|mac)path.normpath itself?

-- 
Mike Orr <sluggoster at gmail.com>

From ncoghlan at gmail.com  Wed Nov  8 11:07:10 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 08 Nov 2006 20:07:10 +1000
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611071846l1f0b4dddjc689a9991cdab9eb@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>	<454EE6C6.4000701@acm.org>	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>	<45504EC4.50609@acm.org>
	<4551350F.7070903@canterbury.ac.nz>
	<6e9196d20611071846l1f0b4dddjc689a9991cdab9eb@mail.gmail.com>
Message-ID: <4551AC4E.8090608@gmail.com>

Mike Orr wrote:
>> And again, it strays outside the domain of path algebra
>> operations.
> 
> This is also the same issue.  Where do we put the .expand*() methods
> if not on Path?

For the moment, I'd say that if an operation can raise any flavour of 
EnvironmentError, don't make it a method of the path algebra.

I realise this means normalisation and symbol expansion will be defined only 
on FSPath's. I'd say that's OK because at least the syntax for environment 
variable substitution is OS dependent anyway (note that these methods should 
return the appropriate FSPath objects, rather than the abstract versions).

> Is there an actual case where calling normpath() would change which
> file the path referred to?  Any case that's not handled by
> (posix|nt|mac)path.normpath itself?
> 

To quote the docs for os.normpath itself:

   "It should be understood that this may change the meaning of the path if it 
contains symbolic links!"

Specifically, using path algebra to normalise a path that backtracks after 
following a symlink will actually give a different answer than asking the 
filesystem to do it.

To steal an explanation from the effbot [1]:

"""
if BAR is a symbolic link, FOO/BAR/../DIR isn't necessarily the same
thing as FOO/DIR.

a simple example:

     $ ln -s /etc etc
     $ ls -ld etc
     lrwxrwxrwx    1 fredrik  fredrik         4 Dec  2 23:22 etc -> /etc
     $ etc/../usr/local/bin/python2.4
     Python 2.4.1 (#1, Sep 12 2005, 19:35:02)
     ...
     >>> import os
     >>> os.path.normpath("etc/../usr/local/bin/python2.4")
     'usr/local/bin/python2.4'
     >>> <control-D>
     $ usr/local/bin/python2.4
     -bash: usr/local/bin/python2.4: No such file or directory
"""

Cheers,
Nick.

[1]
http://mail.python.org/pipermail/python-dev/2005-December/058453.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From talin at acm.org  Wed Nov  8 11:21:18 2006
From: talin at acm.org (Talin)
Date: Wed, 08 Nov 2006 02:21:18 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551350F.7070903@canterbury.ac.nz>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org> <4551350F.7070903@canterbury.ac.nz>
Message-ID: <4551AF9E.9070409@acm.org>

Greg Ewing wrote:
> Talin wrote:
> 
>> I'm a little confused here about the model of how platform-specific 
>> and application-specific formats are represented. Is it the case that 
>> the creation function converts the platform-specific path into a 
>> generic, universal path object, or does it create a platform-specific 
>> path object?
> 
> I don't think it makes sense to mix different platform's
> path objects at all. They're not just different ways of
> representing the same thing, they're actually different
> things.

Only true if we're talking about fully qualified paths. For relative 
paths, every file system I know of supports the 'name (sep name)*' 
syntax to indicate subdirectory structure.

If this were not true, 'make' could never work across filesystems. You'd 
have to have a different makefile for every different filesystem type.

-- Talin

From talin at acm.org  Wed Nov  8 11:21:29 2006
From: talin at acm.org (Talin)
Date: Wed, 08 Nov 2006 02:21:29 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>	<454EE6C6.4000701@acm.org>	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>	<45504EC4.50609@acm.org>
	<6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
Message-ID: <4551AFA9.9090905@acm.org>

Mike Orr wrote:
> Remember that 99% of Python programmers are concerned only with native
> paths.  I have never used a non-native path or multiple-platform paths
> in an application.  So we need to make the native case easy and clear.
>  For that reason I'd rather keep the non-native cases and conversion
> code in separate classes.

This is only true if one is speaking of fully-qualified paths.

Most of the time, however, we deal with relative paths. Relative paths 
are, for the most part, universal already - if they weren't, then it 
would be impossible to create a makefile that works on both Linux and 
Windows, because the embedded file paths would be incompatible. If 
relative paths couldn't be used in conjunction with different path 
types, then there would be no way for make, Apache, Scons, svn, and a 
whole bunch of other apps I can think of to work cross-platform.

What I want is a way to have data files that contain embedded paths 
encoded in a way that allows those data files to be transported across 
platform. Generally those embedded paths will be relative to something, 
rather than fully-qualfied, such as ${root}/something/or/other.

What I want to avoid is a situation where I have to edit my config file 
to switch all the path separators from '/' to '\' when I move my 
application from OS X to Win32.

As a Python programmer, my ideal is to be able to write as if I *didn't 
know* what the underlying platform was.

>> I don't think you need to follow too closely the syntax of os.path -
>> rather, we should concentrate on the semantics, and even more
>> importantly, the scope of the existing module. In other words don't try
>> to do too much more than os.path did.
> 
> I think a layered approach is important.  Use the code in the existing
> modules because it's well tested.  Add a native-path object on top of
> that.  Put non-native paths and conversions on the side.  Then put a
> filesystem-access class (or functions) on top of the native-path
> object.  Then high-level functions/methods on top of that.  Then when
> we lobby for stdlib inclusion it'll be "one level and everything below
> it".  People can see and test that level, and ignore the (possibly
> more controversial) levels above it.

BTW, I don't think that the "filesystem object" is going to fly in the 
way you describe it. Specifically, it sounds like yet another swiss army 
knife class.

I think that there's a reasonable chance of acceptance for an object 
that does filesystem-like operations that *doesn't overlap* with what 
the Path object does. But what you are proposing is a *superset* of what 
Path does (because you're saying its a subclass), and I don't think that 
will go over well.

The basic rationale is simple: "Things" and "Names for things" are 
distinct from each other, and shouldn't be conflated into the same 
object. A path is a name for a thing, but it is not the same as the 
thing it is naming.

>> I'm in favor of the verb 'combine' being used to indicate both a joining
>> and a simplification of a path.
> 
> 
> I like the syntax of a join method.  With a multi-arg constructor it's
> not necessary though. PEP 355 hints at problems with the .joinpath()
> method though it didn't say what they were.   .combine() is an OK
> name, and there's a precedent in the datetime module.  But people are
> used to calling it "joining paths" so I'm not sure we should rename it
> so easily.  We can't call it .join due to string compatibility.
> .joinpath is ugly if we delete "path" from the other method names.
> .pjoin comes to mind but people might find it too cryptic and too
> similar to .join.  By just using the constructor we avoid the debate
> over the method name.

The reason I don't like the word "join" is that it implies some sort of 
concatenation, like an "append" operation - i.e. the combination of "a" 
and "b" joined together is "a/b". However, when we combine paths, we're 
really doing "path algebra", which is a more sophisticated mixing of 
paths. Thus "a" joined with "/b" is just "/b", not "a/b".

> Again, 99.9% of Python users have a functioning cwd, and .abspath() is
> one of those things people expect in any programming language. Nobody
> is forcing you to call it on the Xbox.  However, I'm not completely
> opposed to dropping .abspath().

And I'm not completely opposed to keeping it either. I'm just a 
minimalist :) :)

> People need to add/delete/replace extensions, and they don't want to
> use character slicing / .rfind / .endswith / len() / + to do it.  They
> expect the library to at least handle extension splitting as well as
> os.path does.  Adding a few convenience methods would be unobtrusive
> and express people really want to do:
> 
>     p2 = p.add_ext(".tar")
>     p2 = p.del_ext()
>     p2 = Path("foo.gzip").replace_ext(".bz2")
> 
> But what harm is there in making them scalable to multiple extensions?
> 
>     .add_exts(*exts)
>     .del_exts(N)
>     .replace_exts(N, *exts)

Someone in another message pointed out that paths, being based on 
strings, are immutable, so this whole handling of extensions will have 
to be done another way.

-- Talin

From ncoghlan at gmail.com  Wed Nov  8 11:22:43 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 08 Nov 2006 20:22:43 +1000
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551AC4E.8090608@gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>	<454EE6C6.4000701@acm.org>	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>	<45504EC4.50609@acm.org>	<4551350F.7070903@canterbury.ac.nz>	<6e9196d20611071846l1f0b4dddjc689a9991cdab9eb@mail.gmail.com>
	<4551AC4E.8090608@gmail.com>
Message-ID: <4551AFF3.8020604@gmail.com>

Nick Coghlan wrote:
> I realise this means normalisation and symbol expansion will be defined only 
> on FSPath's.

Scratch the bit about normalisation not being defined on the path algebra 
object - as the latter part of the previous email noted, normalisation *is* 
path algebra based. It just has the problem that whether or not the normalised 
version means the same as the original on a given filesystem depends on 
whether or not the original involves backtracking after following a symlink.

It's abspath() and cwd() that don't belong on the abstract Path object - the 
former would be a zero argument instance method of FSPath objects, while the 
latter would be a class method of FSPath. In both cases, the result would be 
an FSPath since it relates specifically to the current filesystem (and is 
OS-dependent, as an absolute path on Windows starts with a drive specifier or 
UNC name, while one on a *nix system will start at the root directory).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From p.f.moore at gmail.com  Wed Nov  8 15:47:19 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 8 Nov 2006 14:47:19 +0000
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551AFA9.9090905@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
	<6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
	<4551AFA9.9090905@acm.org>
Message-ID: <79990c6b0611080647s3f337c6dl875e02d693fb59a8@mail.gmail.com>

On 11/8/06, Talin <talin at acm.org> wrote:

> What I want to avoid is a situation where I have to edit my config file
> to switch all the path separators from '/' to '\' when I move my
> application from OS X to Win32.

This requirement conflicts with that of a user who only uses one
platform, and wants (expects) to just enter paths in native format in
the config file.

> As a Python programmer, my ideal is to be able to write as if I *didn't
> know* what the underlying platform was.

As a user, *my* ideal is to use native format everywhere, and not have
to worry about other platforms.

I don't think there's a one-size-fits-all answer to this - it's a
human issue, not a technical one.

Paul.

From sluggoster at gmail.com  Wed Nov  8 22:21:05 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Wed, 8 Nov 2006 13:21:05 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551AFA9.9090905@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
	<6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
	<4551AFA9.9090905@acm.org>
Message-ID: <6e9196d20611081321u46f0f4baoc42e09c7e464d5b4@mail.gmail.com>

On 11/8/06, Talin <talin at acm.org> wrote:
> What I want is a way to have data files that contain embedded paths
> encoded in a way that allows those data files to be transported across
> platform. Generally those embedded paths will be relative to something,
> rather than fully-qualfied, such as ${root}/something/or/other.
>
> What I want to avoid is a situation where I have to edit my config file
> to switch all the path separators from '/' to '\' when I move my
> application from OS X to Win32.
>
> As a Python programmer, my ideal is to be able to write as if I *didn't
> know* what the underlying platform was.

I think my proposal yesterday will do it for you.

    p = Path(  PosixPath("something/or/other")  )

This would convert a Posix relative path to the native format if
different, using .components as a universal intermediary.

I'm inclined to raise an exception if the user tries to convert from
an absolute path since there's no one obvious way to do it.  Let the
user make the path relative and then explicitly attach it to a new
base directory.

> I think that there's a reasonable chance of acceptance for an object
> that does filesystem-like operations that *doesn't overlap* with what
> the Path object does. But what you are proposing is a *superset* of what
> Path does (because you're saying its a subclass), and I don't think that
> will go over well.

It's a "contains" relationship, not a subclass.  FSPath contains a
path in its .path attribute.

Path(object):
    .pathlib = os.path        # Class attribute.

PosixPath(Path):
    .pathlib = posixpath

FSPath(object):
    .path = Path("foo")       # Instance attribute.

FSPath will call the Path constructor whenever it needs to create a
path object.  That's the only relationship between the two.  Three
occasions have been mentioned:

    - When FSPath() is called with any arguments other than a single
Path instance.  It passes the arguments to Path() and sets self.path
to the result.
    - FSPath.cwd() does 'return FSPath(   Path(os.getcwd())   )'.
    - If .norm and .expand are moved to FSPath, they will return new
FSPath objects initialized to new Path objects.

(Previously I said you could make FSPath use an alternate *Path class.
On reflection that's unnecessary, since FSPath can't do anything with
a non-native path anyway.)

I would personally like Path and FSPath combined into one class, so
this is a compromise on my part.  The argument for a "pure path
algebra" class seems academic: who really needs it?  (Talin's
cross-platform example could be handled by a conversion class designed
for that purpose.)  os.path mixes FS-independent and FS-dependent
functions together, and I've never heard anybody say that's a problem.
 So why is it suddenly a problem now?  We've seen that path joining,
the current directory, absolute paths, normalization, and FS-sensitive
".." all imply an interaction between Path and FSPath, so it's a messy
split.

I'm thinking about proposing .absolute() and .relative() to replace
.aspath(), where .absolute() prefixes the cwd and .relative() chops
off the root only.  Although with the Path/FSPath distinction,
.absolute() would have to go on FSPath and .relative() belongs on
Path.  Users might wonder why the two aren't together.

> > People need to add/delete/replace extensions, and they don't want to
> > use character slicing / .rfind / .endswith / len() / + to do it.  They
> > expect the library to at least handle extension splitting as well as
> > os.path does.  Adding a few convenience methods would be unobtrusive
> > and express people really want to do:
> >
> >     p2 = p.add_ext(".tar")
> >     p2 = p.del_ext()
> >     p2 = Path("foo.gzip").replace_ext(".bz2")
> >
> > But what harm is there in making them scalable to multiple extensions?
> >
> >     .add_exts(*exts)
> >     .del_exts(N)
> >     .replace_exts(N, *exts)
>
> Someone in another message pointed out that paths, being based on
> strings, are immutable, so this whole handling of extensions will have
> to be done another way.

They would return new Path's, just like str.replace() does.  I do want
them to be dictionary keys so they'd have to be immutable.  I'm
veering toward this syntax now:

    .add_ext(*exts)    # Mainly for one but can unobtrusively add more.
    .del_ext(N=1)       # Mainly for one but can unobtrusively delete more.
    .replace_ext(ext)    # Replace one extension only (e.g., ".jpg" to ".gif").

-- 
Mike Orr <sluggoster at gmail.com>

From greg.ewing at canterbury.ac.nz  Thu Nov  9 00:20:26 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 09 Nov 2006 12:20:26 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611071846l1f0b4dddjc689a9991cdab9eb@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org> <4551350F.7070903@canterbury.ac.nz>
	<6e9196d20611071846l1f0b4dddjc689a9991cdab9eb@mail.gmail.com>
Message-ID: <4552663A.8040002@canterbury.ac.nz>

Mike Orr wrote:

> What do we do with Path.cwd() then?  It also violates the
> path-algrebra-only logic. We can put it on FSPath, but then
 > it's the only FSPath method that returns a Path

I don't see why it should be a problem for an FSPath
method to return a Path.

> Where do we put the .expand*() methods if not on Path?

This case might be excusable, since it can require
platform-specific knowledge of the path format.

> Is there an actual case where calling normpath() would change which
> file the path referred to?

If the path crosses a symbolic link to a directory,
then .. no longer means what it appears to mean
from just looking at the path.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov  9 00:43:23 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 09 Nov 2006 12:43:23 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551AF9E.9070409@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org> <4551350F.7070903@canterbury.ac.nz>
	<4551AF9E.9070409@acm.org>
Message-ID: <45526B9B.2010705@canterbury.ac.nz>

Talin wrote:

> If this were not true, 'make' could never work across filesystems. You'd 
> have to have a different makefile for every different filesystem type.

Ever tried to use a Unix makefile on classic MacOS
or VMS? It wouldn't work very well at all...

Even relative pathnames can have semantic differences
that don't translate across systems. For example,
in VMS, the pathname of a directory looks quite
different from the pathname of a file. You couldn't
round-trip that to a Unix pathname and back without
losing the distinction.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov  9 01:02:31 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 09 Nov 2006 13:02:31 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <4551AFA9.9090905@acm.org>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
	<6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
	<4551AFA9.9090905@acm.org>
Message-ID: <45527017.4020402@canterbury.ac.nz>

Talin wrote:

> I think that there's a reasonable chance of acceptance for an object 
> that does filesystem-like operations that *doesn't overlap* with what 
> the Path object does. But what you are proposing is a *superset* of what 
> Path does (because you're saying its a subclass), and I don't think that 
> will go over well.

I agree. File system operations should be functions that
operate on paths, not methods of a special kind of path.

> Mike Orr wrote:

>>I like the syntax of a join method.  With a multi-arg constructor it's
>>not necessary though.

True, but seeing as we're talking about an algebra, it
would be nice to have some operators for the most
frequent operations. Perhaps

    p1 / p2     #  combine paths
    p & ext     #  add extension
    p | ext     #  replace last extension
    p | ""      #  remove last extension

>>But what harm is there in making them scalable to multiple extensions?
>>
>>    .add_exts(*exts)
>>    .del_exts(N)
>>    .replace_exts(N, *exts)

Not sure if this is worth it, since even when a
file has multiple extensions, you're usually dealing
with them one at a time. E.g. given foo.tar.gz, you
first ungzip it to give foo.tar, and then untar it
to give foo (conceptually, at least).

> Someone in another message pointed out that paths, being based on 
> strings, are immutable, so this whole handling of extensions will have 
> to be done another way.

The relevant methods would just have to return new
paths instead of modifying in-place.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov  9 01:07:01 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 09 Nov 2006 13:07:01 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <79990c6b0611080647s3f337c6dl875e02d693fb59a8@mail.gmail.com>
References: <6e9196d20611012247w51d740fm68116bd98b6591d9@mail.gmail.com>
	<454EE6C6.4000701@acm.org>
	<6e9196d20611061437h35756e6fm1afa267f1048870b@mail.gmail.com>
	<45504EC4.50609@acm.org>
	<6e9196d20611071406j4d98a3ete3b464966d532e1c@mail.gmail.com>
	<4551AFA9.9090905@acm.org>
	<79990c6b0611080647s3f337c6dl875e02d693fb59a8@mail.gmail.com>
Message-ID: <45527125.8060407@canterbury.ac.nz>

Paul Moore wrote:

> This requirement conflicts with that of a user who only uses one
> platform, and wants (expects) to just enter paths in native format in
> the config file.

If the standard format were designed so as to be
unambiguously distinguishable from all native
formats, paths in either standard or platform
format could be used as desired.

--
Greg

From janssen at parc.com  Thu Nov  9 03:23:30 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 8 Nov 2006 18:23:30 PST
Subject: [Python-3000] Mini Path object
In-Reply-To: Your message of "Wed, 08 Nov 2006 16:07:01 PST."
	<45527125.8060407@canterbury.ac.nz> 
Message-ID: <06Nov8.182336pst."58648"@synergy1.parc.xerox.com>

Greg Ewing writes:
> If the standard format were designed so as to be
> unambiguously distinguishable from all native
> formats, ...

All native formats both past and future.

Bill


From talin at acm.org  Thu Nov  9 19:14:11 2006
From: talin at acm.org (Talin)
Date: Thu, 09 Nov 2006 10:14:11 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <06Nov8.182336pst."58648"@synergy1.parc.xerox.com>
References: <06Nov8.182336pst."58648"@synergy1.parc.xerox.com>
Message-ID: <45536FF3.4070105@acm.org>

Bill Janssen wrote:
> Greg Ewing writes:
>> If the standard format were designed so as to be
>> unambiguously distinguishable from all native
>> formats, ...
> 
> All native formats both past and future.

That's not difficult.

I use 'posix' paths as my universal format. I convert them to native
paths just before writing them out or passing to a subsystem that
requires native paths.

For reading paths, it depends on whether the paths are already in
universal (posix) style or native style. If they are native, I use one
method for reading them, if they are universal I use a different method
(Well, in the current code base that's not true - since os.path always
handles paths in posix format as well as native format, even when
running on non-posix systems. So I just pass everything to os.path and
it just works.)

What this means is that universal paths need never be serialized - they
are always converted to native form before being written anywhere. Which
in turn implies that the 'universal' path format can be an
in-memory-only format. And as such, it can express its 'universalness'
by additional object properties or object types, rather than via any
kind of weird string encoding.

Thus, a universal path is nothing more than a posix path, except with a
different object type.

Representing non-posix concepts such as drive letters is done simply by
not interpreting them; Or in some cases, deferring any interpretation
until the user specifically calls a function to split off that part. So
if you have a path beginning with "C:", it doesn't try to interpret what
that means until you ask it to via splitdrive().

(This is a good reason to have paths represented as strings instead of
as a tuple, since you can't defer interpretation this way with
pre-parsed paths.)

-- Talin



From python at zesty.ca  Thu Nov  9 20:57:52 2006
From: python at zesty.ca (Ka-Ping Yee)
Date: Thu, 9 Nov 2006 13:57:52 -0600 (CST)
Subject: [Python-3000] PEP 3104 added
Message-ID: <Pine.LNX.4.58.0611091353370.14217@server1.LFW.org>

I've committed PEP 3104 to the SVN repository.  It should appear at

    http://www.python.org/dev/peps/pep-3104/

shortly.  The PEP is updated to add a note on the amount of code
breakage in the standard library that we might expect for some of
the possible keywords we might choose.  The updated version is
also available at

    http://zesty.ca/python/pep-3104.html

if you want to see it now.


-- ?!ng

From p.f.moore at gmail.com  Thu Nov  9 21:45:10 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 9 Nov 2006 20:45:10 +0000
Subject: [Python-3000] Mini Path object
In-Reply-To: <45536FF3.4070105@acm.org>
References: <45536FF3.4070105@acm.org>
Message-ID: <79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>

On 11/9/06, Talin <talin at acm.org> wrote:
> Bill Janssen wrote:
> > Greg Ewing writes:
> >> If the standard format were designed so as to be
> >> unambiguously distinguishable from all native
> >> formats, ...
> >
> > All native formats both past and future.
>
> That's not difficult.
>
> I use 'posix' paths as my universal format. I convert them to native
> paths just before writing them out or passing to a subsystem that
> requires native paths.

Not a valid option. C:\Data\a_file.txt is a perfectly valid posix
format filename. It's a file in the current directory (with some very
strange characters in it). But it's also a native path in Windows. So
how do you interpret it?

And yes, I have seen precisely this problem occur with an application
which took (posix) paths for use on a Unix server, but in a Windows
client GUI. So we ended up with some really funky filenames on Unix to
sort out :-(

I stand by my assertion, it's a human problem, not a technical one.

> What this means is that universal paths need never be serialized - they
> are always converted to native form before being written anywhere.

Again, I don't believe this is possible for all corner cases: what is
the drive for /my/file on a Windows system?

I'm not at all sure what you're trying to achieve here - if I'm
running a program on a Windows system, I want to see Windows native
paths for all external information (whether it's input from command
line args or ini files, or output in messages etc). If I'm running
that program on a Posix system, I want to see and use Posix files. If
I want to write an ini file for the program which contains filenames,
and which can be used unchanged on a Windows and a Posix system, I
assert that I'm misguided (because I can't do it - drive letters will
kill me as /data/xxx (or \data\xxx if you prefer) is ambiguous on
Windows.

If you mandate that all filenames must be relative, you could manage,
but enforcing that is probably harder than just using native format
everywhere and giving up on portable ini files...

(I think this has strayed so far off topic as to be meaningless, so
I'll leave it at that, and avoid perpetuating the thread...).

Paul.

From solipsis at pitrou.net  Thu Nov  9 22:05:47 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 09 Nov 2006 22:05:47 +0100
Subject: [Python-3000] Mini Path object
In-Reply-To: <79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
Message-ID: <1163106347.5190.20.camel@fsol>


Le jeudi 09 novembre 2006 ? 20:45 +0000, Paul Moore a ?crit :
> Again, I don't believe this is possible for all corner cases: what is
> the drive for /my/file on a Windows system?

Why not the current drive?

> If you mandate that all filenames must be relative, you could manage,
> but enforcing that is probably harder than just using native format
> everywhere and giving up on portable ini files...

Portable ini files can only use relative paths anyway.
If you want to play nice with the OS, you have to follow its existing
conventions for file locations, and it isn't the same everywhere (you
don't use /usr/share under Windows, or /Programs/Shared Files under
Linux). So it's impossible to come up with portable absolute paths.




From mbk.lists at gmail.com  Thu Nov  9 22:12:48 2006
From: mbk.lists at gmail.com (Mike Krell)
Date: Thu, 9 Nov 2006 14:12:48 -0700
Subject: [Python-3000] PEP 3104 added
In-Reply-To: <Pine.LNX.4.58.0611091353370.14217@server1.LFW.org>
References: <Pine.LNX.4.58.0611091353370.14217@server1.LFW.org>
Message-ID: <da7032ce0611091312q4a97aa8ay957d9bb7f30aedd9@mail.gmail.com>

Ka-Ping,

Thanks for all of your hard work on this.  Well done!

   Mike

From sluggoster at gmail.com  Fri Nov 10 00:41:52 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Thu, 9 Nov 2006 15:41:52 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <45536FF3.4070105@acm.org>
References: <45536FF3.4070105@acm.org>
Message-ID: <6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>

On 11/9/06, Talin <talin at acm.org> wrote:
> Bill Janssen wrote:
> > Greg Ewing writes:
> >> If the standard format were designed so as to be
> >> unambiguously distinguishable from all native
> >> formats, ...
> >
> > All native formats both past and future.
>
> That's not difficult.
>
> I use 'posix' paths as my universal format. I convert them to native
> paths just before writing them out or passing to a subsystem that
> requires native paths.
>
> For reading paths, it depends on whether the paths are already in
> universal (posix) style or native style. If they are native, I use one
> method for reading them, if they are universal I use a different method
> (Well, in the current code base that's not true - since os.path always
> handles paths in posix format as well as native format, even when
> running on non-posix systems. So I just pass everything to os.path and
> it just works.)
>
> What this means is that universal paths need never be serialized - they
> are always converted to native form before being written anywhere. Which
> in turn implies that the 'universal' path format can be an
> in-memory-only format. And as such, it can express its 'universalness'
> by additional object properties or object types, rather than via any
> kind of weird string encoding.
>
> Thus, a universal path is nothing more than a posix path, except with a
> different object type.
>
> Representing non-posix concepts such as drive letters is done simply by
> not interpreting them; Or in some cases, deferring any interpretation
> until the user specifically calls a function to split off that part. So
> if you have a path beginning with "C:", it doesn't try to interpret what
> that means until you ask it to via splitdrive().
>
> (This is a good reason to have paths represented as strings instead of
> as a tuple, since you can't defer interpretation this way with
> pre-parsed paths.)

Er, is there an API recommendation in there somewhere, or is this just
philosophy or usage style?

Posix is the lingua franca of the current age, and the non-compliant
OSes (Windows and Mac) have various levels of compatibility with it.
So using Posix in portable config files is justifiable.

However, a list of components works better as an intermediate format
for autoconverting.  You're going to get lists anyway with .split()
and .join().  There's no reason to convert from format A to Posix to
format B.  If we define the first component as the absolute root or
'', the source format knows how to convert it to that, and the
destination format will at least recognize it as an  even if it
doesn't know what it means.  That's enough information to raise an
exception or convert the path to relative.

-- 
Mike Orr <sluggoster at gmail.com>

From sluggoster at gmail.com  Fri Nov 10 00:44:31 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Thu, 9 Nov 2006 15:44:31 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>
Message-ID: <6e9196d20611091544x44766339h1d091b30ad076f38@mail.gmail.com>

On 11/9/06, Mike Orr <sluggoster at gmail.com> wrote:
> the destination format will at least recognize it as an  even if it
> doesn't know what it means.

... recognize it as an absolute path ...

-- 
Mike Orr <sluggoster at gmail.com>

From greg.ewing at canterbury.ac.nz  Fri Nov 10 01:18:34 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 10 Nov 2006 13:18:34 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <45536FF3.4070105@acm.org>
References: <06Nov8.182336pst.58648@synergy1.parc.xerox.com>
	<45536FF3.4070105@acm.org>
Message-ID: <4553C55A.5020500@canterbury.ac.nz>

Talin wrote:

> Bill Janssen wrote:
>
> > All native formats both past and future.
>
> That's not difficult.
 >
 > I use 'posix' paths as my universal format.

That assumes you can always distinguish a posix path from
some other kind of path. That's not always the case,
e.g. consider

   /hello:wide/world

Is that a posix path or not? It could be, but it could
also be a classic MacOS pathname referring to a file
called "wide/world" on a volume called "/hello".

It's fine if there's a rule that all paths in your
config file are in the universal format. But we were
talking about allowing the user to use either native
or universal format.

> What this means is that universal paths need never be serialized - they
> are always converted to native form before being written anywhere. Which
> in turn implies that the 'universal' path format can be an
> in-memory-only format.

Hmmm, we seem to have crossed topics again -- I thought
we were talking about storing paths in config files.

> os.path always
> handles paths in posix format as well as native format, even when
> running on non-posix systems.

Really? I wasn't aware of that. I don't see how it *can*,
given what I said above.

If you're talking about the way you can use forward or
backward slashes on Windows, that's a property of the
Windows kernel, not os.path. And it only applies to
Unix vs. Windows, not in general.

> Representing non-posix concepts such as drive letters is done simply by
> not interpreting them; Or in some cases, deferring any interpretation
> until the user specifically calls a function to split off that part. So
> if you have a path beginning with "C:", it doesn't try to interpret what
> that means until you ask it to via splitdrive().

What if you need to know whether it's an absolute or
relative path, e.g. so you can join it to another path?
You can't do that without knowing something about the
semantics.

--
Greg

From sluggoster at gmail.com  Fri Nov 10 03:55:37 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Thu, 9 Nov 2006 18:55:37 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <4553C55A.5020500@canterbury.ac.nz>
References: <06Nov8.182336pst.58648@synergy1.parc.xerox.com>
	<45536FF3.4070105@acm.org> <4553C55A.5020500@canterbury.ac.nz>
Message-ID: <6e9196d20611091855h53e910c5w3a8e6019c64c79a1@mail.gmail.com>

On 11/9/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Talin wrote:
>  > I use 'posix' paths as my universal format.
>
> That assumes you can always distinguish a posix path from
> some other kind of path. That's not always the case,
> e.g. consider
>
>    /hello:wide/world
>
> Is that a posix path or not? It could be, but it could
> also be a classic MacOS pathname referring to a file
> called "wide/world" on a volume called "/hello".

The user will tell us what the source format is.    If they don't
know, they've got bigger problems than we can handle.  One can imagine
a guesspath() or any2posix() function, but I can't imagine it would be
widely used... or 100% correct.

-- 
Mike Orr <sluggoster at gmail.com>

From talin at acm.org  Fri Nov 10 05:49:04 2006
From: talin at acm.org (Talin)
Date: Thu, 09 Nov 2006 20:49:04 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>
Message-ID: <455404C0.70605@acm.org>

Mike Orr wrote:
> On 11/9/06, Talin <talin at acm.org> wrote:
>> Bill Janssen wrote:
>>> Greg Ewing writes:
>>>> If the standard format were designed so as to be
>>>> unambiguously distinguishable from all native
>>>> formats, ...
>>> All native formats both past and future.
>> That's not difficult.
>>
>> I use 'posix' paths as my universal format. I convert them to native
>> paths just before writing them out or passing to a subsystem that
>> requires native paths.
>>

(much useless Talin blathering omitted)

>> (This is a good reason to have paths represented as strings instead of
>> as a tuple, since you can't defer interpretation this way with
>> pre-parsed paths.)
> 
> Er, is there an API recommendation in there somewhere, or is this just
> philosophy or usage style?

I am indeed trying to describe a usage style, one that is not only used 
by me but by a large number of pre-existing apps. My goal in describing 
this is to insure that any new path system allows similar kinds of usage 
patterns.

What I am arguing against is an overly strict and pedantic 
interpretation of the differences in path representation across 
platforms. True, in theory, you can't compare a windows path with a 
posix path, but in practice it generally "just works".

I sometimes notice a tendency on this list that anything which is not 
100% provably correct should be forbidden, no matter how useful it is.

Let me give a concrete example: In my TurboGears project, there is a 
file called "thes.egg-info/sqlobject.txt" that contains this string:

	history_dir=$base/thes/sqlobject-history

I didn't write this, it was generated by the 'quickstart' script. I 
fully expect to be able to package up my TG app from my OSX machine, 
transport it to my Windows box, and have it work. And I would fully 
expect to be able to do the same in any future version of TurboGears 
running on a future version of Python.

Now, one can argue that this string is meaningless when taken from one 
platform to another, but the only thing that is accomplished by that 
argument is to remove functionality that current users find useful and 
valuable.

I could bring up other examples, but do I really need to? (And would you 
really want to read them?)

> Posix is the lingua franca of the current age, and the non-compliant
> OSes (Windows and Mac) have various levels of compatibility with it.
> So using Posix in portable config files is justifiable.

I don't have a problem with this.

> However, a list of components works better as an intermediate format
> for autoconverting.  You're going to get lists anyway with .split()
> and .join().  There's no reason to convert from format A to Posix to
> format B.  If we define the first component as the absolute root or
> '', the source format knows how to convert it to that, and the
> destination format will at least recognize it as an  even if it
> doesn't know what it means.  That's enough information to raise an
> exception or convert the path to relative.

The way I tend to deal with these kinds of issues is to let the 
"absolute" part of the path be an opaque blob that is uninterpreted by 
me or my code. Code that does path algebra is really only interested in 
the 'tail' of the path (except in cases where you have an absolute 
override during a combine). So it really doesn't matter to me what the 
'head' part looks like - I just leave it alone and don't worry about it.

-- Talin

From sluggoster at gmail.com  Fri Nov 10 07:11:10 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Thu, 9 Nov 2006 22:11:10 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <455404C0.70605@acm.org>
References: <45536FF3.4070105@acm.org>
	<6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>
	<455404C0.70605@acm.org>
Message-ID: <6e9196d20611092211r2c16c734l52bb6448c69190c7@mail.gmail.com>

On 11/9/06, Talin <talin at acm.org> wrote:
> Mike Orr wrote:
> > On 11/9/06, Talin <talin at acm.org> wrote:
> >> (This is a good reason to have paths represented as strings instead of
> >> as a tuple, since you can't defer interpretation this way with
> >> pre-parsed paths.)
> >
> > Er, is there an API recommendation in there somewhere, or is this just
> > philosophy or usage style?
>
> I am indeed trying to describe a usage style, one that is not only used
> by me but by a large number of pre-existing apps. My goal in describing
> this is to insure that any new path system allows similar kinds of usage
> patterns.
>
> What I am arguing against is an overly strict and pedantic
> interpretation of the differences in path representation across
> platforms. True, in theory, you can't compare a windows path with a
> posix path, but in practice it generally "just works".
>
> I sometimes notice a tendency on this list that anything which is not
> 100% provably correct should be forbidden, no matter how useful it is.
>
> Let me give a concrete example: In my TurboGears project, there is a
> file called "thes.egg-info/sqlobject.txt" that contains this string:
>
>         history_dir=$base/thes/sqlobject-history
>
> I didn't write this, it was generated by the 'quickstart' script. I
> fully expect to be able to package up my TG app from my OSX machine,
> transport it to my Windows box, and have it work. And I would fully
> expect to be able to do the same in any future version of TurboGears
> running on a future version of Python.
>
> Now, one can argue that this string is meaningless when taken from one
> platform to another, but the only thing that is accomplished by that
> argument is to remove functionality that current users find useful and
> valuable.

I'm looking for "we need a method named foo with signature bar that
does this..."     Or "make sure method blah doesn't do this...."
Otherwise I don't know what to do with the information.  Can you make
a use case showing how you'd ideally like to interact with the module
(what you'd call) and what it should do, starting wtih a config file
fragment and ending with a file operation (mkdir, open, etc).

In the case above, the question is how TurboGears would interact with
the new path module.  First we'd need to know what TG is doing
internally now; then we can verify we're not losing any functionality.
  I would guess TG is doing a text substitution of $base and not even
using a join function.  That's outside of our control.  We know that
our join/combine method is tied to a particular platform, and if given
two path fragments from that same platform will join them properly.
Beyond that all bets are off -- no using the wrong platform class, and
no mixing path fragments from different platforms.  You can't combine
"C:\" and "thes/sqlobject-history" into anything useful.  But
os.path.join() has the same restrictions.

On Windows you're depending on "something below Python" to interpret
the forward slashes properly when you do a file operation.  This is
outside the path module's scope.  But it looks like we'll be adding
the ability to convert at least some paths from Posix to Windows --
something os.path doesn't do.

-- 
Mike Orr <sluggoster at gmail.com>

From talin at acm.org  Fri Nov 10 08:10:13 2006
From: talin at acm.org (Talin)
Date: Thu, 09 Nov 2006 23:10:13 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611092211r2c16c734l52bb6448c69190c7@mail.gmail.com>
References: <45536FF3.4070105@acm.org>	<6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>	<455404C0.70605@acm.org>
	<6e9196d20611092211r2c16c734l52bb6448c69190c7@mail.gmail.com>
Message-ID: <455425D5.9040002@acm.org>

Mike Orr wrote:
> I'm looking for "we need a method named foo with signature bar that
> does this..."     Or "make sure method blah doesn't do this...."
> Otherwise I don't know what to do with the information.  Can you make
> a use case showing how you'd ideally like to interact with the module
> (what you'd call) and what it should do, starting wtih a config file
> fragment and ending with a file operation (mkdir, open, etc).

I could answer this better if there was an actual spec document to look 
at - right now, there are so many suggestions in play that I've kind of 
lost track of where we are.

-- Talin

From p.f.moore at gmail.com  Fri Nov 10 09:59:42 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Nov 2006 08:59:42 +0000
Subject: [Python-3000] Mini Path object
In-Reply-To: <455404C0.70605@acm.org>
References: <45536FF3.4070105@acm.org>
	<6e9196d20611091541y1780accfy82167ae14759fe34@mail.gmail.com>
	<455404C0.70605@acm.org>
Message-ID: <79990c6b0611100059y7ce30d3exd3886b6d93645cf1@mail.gmail.com>

On 11/10/06, Talin <talin at acm.org> wrote:
> What I am arguing against is an overly strict and pedantic
> interpretation of the differences in path representation across
> platforms. True, in theory, you can't compare a windows path with a
> posix path, but in practice it generally "just works".
>
> I sometimes notice a tendency on this list that anything which is not
> 100% provably correct should be forbidden, no matter how useful it is.

That's a fair point, and you're right that 99% of the time, things
work fine. I was reacting against the implication in your comments of
"use Posix and you're fine" as if that precluded the need for thinking
about the issue. As I've been bitten many, many times by code which
assumes Posix is all that matters, and gives unnatural (or indeed
simply incorrect) results on Windows, this is a bit of a sore point
for me.

As long as you think about what you're doing, and don't assume
portability is free, then all of the proposed APIs will work fine.
Which one is *better* is a different discussion... :-)

Paul.

From niki.spahiev at gmail.com  Fri Nov 10 10:18:28 2006
From: niki.spahiev at gmail.com (Niki Spahiev)
Date: Fri, 10 Nov 2006 11:18:28 +0200
Subject: [Python-3000] Mini Path object
In-Reply-To: <4553C55A.5020500@canterbury.ac.nz>
References: <06Nov8.182336pst.58648@synergy1.parc.xerox.com>	<45536FF3.4070105@acm.org>
	<4553C55A.5020500@canterbury.ac.nz>
Message-ID: <ej1g4i$6kl$1@sea.gmane.org>

Greg Ewing wrote:
> What if you need to know whether it's an absolute or
> relative path, e.g. so you can join it to another path?
> You can't do that without knowing something about the
> semantics.

Maybe use file: URLs for universal format. They become more and more 
used in internet era.

Niki Spahiev


From jimjjewett at gmail.com  Fri Nov 10 18:37:21 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 10 Nov 2006 12:37:21 -0500
Subject: [Python-3000] Mini Path object
In-Reply-To: <ej1g4i$6kl$1@sea.gmane.org>
References: <06Nov8.182336pst.58648@synergy1.parc.xerox.com>
	<45536FF3.4070105@acm.org> <4553C55A.5020500@canterbury.ac.nz>
	<ej1g4i$6kl$1@sea.gmane.org>
Message-ID: <fb6fbf560611100937p4df31df3ve1aa66e18c725a2@mail.gmail.com>

On 11/10/06, Niki Spahiev <niki.spahiev at gmail.com> wrote:
> Greg Ewing wrote:
> > What if you need to know whether it's an absolute or
> > relative path, e.g. so you can join it to another path?
> > You can't do that without knowing something about the
> > semantics.

> Maybe use file: URLs for universal format. They become more and more
> used in internet era.

the file: protocol has a long and messy history; it is now probably
impossible to use in a consistent way without breaking something.  It
works fairly well for the tail of the path, but so does an ordinary
list of path components.

In general practice, even just using a string with a slash in either
direction as the separator will work.  If that has too many corner
cases for your taste, then you should treat "file:" as undefined
behavior.

-jJ

From greg.ewing at canterbury.ac.nz  Sat Nov 11 01:20:27 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 11 Nov 2006 13:20:27 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <ej1g4i$6kl$1@sea.gmane.org>
References: <06Nov8.182336pst.58648@synergy1.parc.xerox.com>
	<45536FF3.4070105@acm.org> <4553C55A.5020500@canterbury.ac.nz>
	<ej1g4i$6kl$1@sea.gmane.org>
Message-ID: <4555174B.9020602@canterbury.ac.nz>

Niki Spahiev wrote:

> Maybe use file: URLs for universal format. They become more and more 
> used in internet era.

You still need some outside way of telling whether
you're dealing with a universal format, since
"file:" is a valid pathname component in Unix,
a valid volume name in classic MacOS, etc.

--
Greg

From qrczak at knm.org.pl  Sun Nov 12 11:58:27 2006
From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk)
Date: Sun, 12 Nov 2006 11:58:27 +0100
Subject: [Python-3000] Mini Path object
In-Reply-To: <79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	(Paul Moore's message of "Thu, 9 Nov 2006 20:45:10 +0000")
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
Message-ID: <87ac2x7wz0.fsf@qrnik.zagroda>

"Paul Moore" <p.f.moore at gmail.com> writes:

>> I use 'posix' paths as my universal format. I convert them to native
>> paths just before writing them out or passing to a subsystem that
>> requires native paths.
>
> Not a valid option. C:\Data\a_file.txt is a perfectly valid posix
> format filename. It's a file in the current directory (with some very
> strange characters in it). But it's also a native path in Windows. So
> how do you interpret it?

The Boost Filesystem library has solved this problem by using the
POSIX-based syntax by default, with system-dependent modifications
possible, and by specifying a portable subset which avoids all the
system dependencies (but possibly can't express some filenames).
There is a separate constructor which accepts the native path syntax.

http://www.boost.org/libs/filesystem/doc/path.htm

-- 
   __("<         Marcin Kowalczyk
   \__/       qrczak at knm.org.pl
    ^^     http://qrnik.knm.org.pl/~qrczak/

From gsakkis at rutgers.edu  Mon Nov 13 19:44:10 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Mon, 13 Nov 2006 13:44:10 -0500
Subject: [Python-3000] Builtin iterator type
Message-ID: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>

Following up on a recent c.l.py thread
(http://groups.google.com/group/comp.lang.python/browse_frm/thread/42818717b400bcd4/#),
I'd like to get an idea from python-dev folks on how much of a chance
is there for a builtin iterator type. Although there might be a slight
possibility to make this proposal backwards compatible, I am mainly
targetting Py3K, so compatibility is not a strict requirement.

The key points are:
1. Formalize the currently informal concept of iterator by changing
iter() from a builtin function to a type.
2. Augment this type with operators to implement operations currently
done explicitly by itertools.*.
3. Make this type a base for all other (existing and future) iterator
types, e.g. xrange and generator.

As a proof of concept, I provide below a sample implementation of how
I imagine this type to work (also posted in the original c.l.py.
thread):


from itertools import chain, tee, islice

import __builtin__
_builtin_iter = __builtin__.iter

class iter(object):

    def __init__(self, iterable):
        self._it = _builtin_iter(iterable)

    def __iter__(self):
        return self
    def next(self):
        return self._it.next()

    def __getitem__(self, index):
        if isinstance(index, int):
            try: return islice(self._it, index, index+1).next()
            except StopIteration:
                raise IndexError('Index %d out of range' % index)
        else:
            start,stop,step = index.start, index.stop, index.step
            if start is None: start = 0
            if step is None: step = 1
            return islice(self._it, start, stop, step)

    def __add__(self, other):
        return chain(self._it, other)
    def __radd__(self,other):
        return chain(other, self._it)

    def __mul__(self, num):
        return chain(*tee(self._it,num))

    __rmul__ = __mul__

__builtin__.iter = iter

if __name__ == '__main__':
    def irange(*args):
        return iter(xrange(*args))

    assert list(irange(5)[:3]) == range(5)[:3]
    assert list(irange(5)[3:])  == range(5)[3:]
    assert list(irange(5)[1:3]) == range(5)[1:3]
    assert list(irange(5)[3:1]) == range(5)[3:1]
    assert list(irange(5)[:])   == range(5)[:]
    assert irange(5)[3]         == range(5)[3]

    s = range(5) + range(7,9)
    assert list(irange(5) + irange(7,9)) == s
    assert list(irange(5) +  range(7,9)) == s
    assert list(range(5)  + irange(7,9)) == s

    s = range(5) * 3
    assert list(irange(5) * 3) == s
    assert list(3 * irange(5)) == s


Thoughts, opinions, suggestions, objections, all welcome.

Regards,
George

From guido at python.org  Mon Nov 13 20:31:50 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2006 11:31:50 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
Message-ID: <ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>

Hm. Without knowing much of the background, this appears to be a
worrysome trend away from duck typing. Why would I have to inherit
from a standard class just so that I can implement next()? What's the
advantage of the proposed change? Are you going to propose similar
changes for all standard de-facto interfaces, like sequences,
mappings, files etc.?

Unconvinced,

--Guido

On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> Following up on a recent c.l.py thread
> (http://groups.google.com/group/comp.lang.python/browse_frm/thread/42818717b400bcd4/#),
> I'd like to get an idea from python-dev folks on how much of a chance
> is there for a builtin iterator type. Although there might be a slight
> possibility to make this proposal backwards compatible, I am mainly
> targetting Py3K, so compatibility is not a strict requirement.
>
> The key points are:
> 1. Formalize the currently informal concept of iterator by changing
> iter() from a builtin function to a type.
> 2. Augment this type with operators to implement operations currently
> done explicitly by itertools.*.
> 3. Make this type a base for all other (existing and future) iterator
> types, e.g. xrange and generator.
>
> As a proof of concept, I provide below a sample implementation of how
> I imagine this type to work (also posted in the original c.l.py.
> thread):
>
>
> from itertools import chain, tee, islice
>
> import __builtin__
> _builtin_iter = __builtin__.iter
>
> class iter(object):
>
>     def __init__(self, iterable):
>         self._it = _builtin_iter(iterable)
>
>     def __iter__(self):
>         return self
>     def next(self):
>         return self._it.next()
>
>     def __getitem__(self, index):
>         if isinstance(index, int):
>             try: return islice(self._it, index, index+1).next()
>             except StopIteration:
>                 raise IndexError('Index %d out of range' % index)
>         else:
>             start,stop,step = index.start, index.stop, index.step
>             if start is None: start = 0
>             if step is None: step = 1
>             return islice(self._it, start, stop, step)
>
>     def __add__(self, other):
>         return chain(self._it, other)
>     def __radd__(self,other):
>         return chain(other, self._it)
>
>     def __mul__(self, num):
>         return chain(*tee(self._it,num))
>
>     __rmul__ = __mul__
>
> __builtin__.iter = iter
>
> if __name__ == '__main__':
>     def irange(*args):
>         return iter(xrange(*args))
>
>     assert list(irange(5)[:3]) == range(5)[:3]
>     assert list(irange(5)[3:])  == range(5)[3:]
>     assert list(irange(5)[1:3]) == range(5)[1:3]
>     assert list(irange(5)[3:1]) == range(5)[3:1]
>     assert list(irange(5)[:])   == range(5)[:]
>     assert irange(5)[3]         == range(5)[3]
>
>     s = range(5) + range(7,9)
>     assert list(irange(5) + irange(7,9)) == s
>     assert list(irange(5) +  range(7,9)) == s
>     assert list(range(5)  + irange(7,9)) == s
>
>     s = range(5) * 3
>     assert list(irange(5) * 3) == s
>     assert list(3 * irange(5)) == s
>
>
> Thoughts, opinions, suggestions, objections, all welcome.
>
> Regards,
> George
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Mon Nov 13 23:51:29 2006
From: janssen at parc.com (Bill Janssen)
Date: Mon, 13 Nov 2006 14:51:29 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: Your message of "Mon, 13 Nov 2006 11:31:50 PST."
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com> 
Message-ID: <06Nov13.145133pst."58648"@synergy1.parc.xerox.com>

> this appears to be a worrysome trend away from duck typing

Duck typing is a seriously bad idea, forced on Python by the now
obsolete split between built-in types and user-defined types.

> Are you going to propose similar
> changes for all standard de-facto interfaces, like sequences,
> mappings, files etc.?

Good idea.

Bill

From exarkun at divmod.com  Tue Nov 14 00:15:38 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 13 Nov 2006 18:15:38 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov13.145133pst."58648"@synergy1.parc.xerox.com>
Message-ID: <20061113231538.20948.713332044.divmod.quotient.26448@ohm>

On Mon, 13 Nov 2006 14:51:29 PST, Bill Janssen <janssen at parc.com> wrote:
>> this appears to be a worrysome trend away from duck typing
>
>Duck typing is a seriously bad idea, forced on Python by the now
>obsolete split between built-in types and user-defined types.

Wow.  Just wow.

Anyway, the proposal was in no way suggesting moving away from duck
typing.  The new iter was to be a _wrapper_ around any existing
iterator, not a base class for iterators.

Jean-Paul

From gsakkis at rutgers.edu  Mon Nov 13 21:39:43 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Mon, 13 Nov 2006 15:39:43 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
Message-ID: <91ad5bf80611131239g72afbfaahac86c2387e1f10cf@mail.gmail.com>

On 11/13/06, Guido van Rossum <guido at python.org> wrote:

> Hm. Without knowing much of the background, this appears to be a
> worrysome trend away from duck typing. Why would I have to inherit
> from a standard class just so that I can implement next()? What's the
> advantage of the proposed change?

Duck typing will still apply of course; an object that just implements
next() without inheriting from iter will continue to be iterable. The
advantage is the iterator methods and syntax sugar you get for free,
instead of importing them explicitly from itertools (I've noticed that
most of my modules lately start with a variation of "import sys,
itertools as it").

The way I see it, the trend is not away from duck typing but towards
iterability (generators, itertools, gen. expressions). A builtin iter
type replacing for the most part the need for itertools is a natural
next step to this direction.

> Are you going to propose similar
> changes for all standard de-facto interfaces, like sequences,
> mappings, files etc.?

No, I won't (at least not for now ;-)). Notice however that most
user-defined sequences, mappings, etc. usually don't start from
scratch; they either inherit from an existing type or from an
appropriate mixin. For a baseline iterator that just implements next()
there's no need for a base type or mixin, but if you consider that
itertools.* functionality should/would-be-nice to be provided as
methods and/or syntax instead of imported functions, the parallel
becomes apparent.

George

> On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > Following up on a recent c.l.py thread
> > (http://groups.google.com/group/comp.lang.python/browse_frm/thread/42818717b400bcd4/#),
> > I'd like to get an idea from python-dev folks on how much of a chance
> > is there for a builtin iterator type. Although there might be a slight
> > possibility to make this proposal backwards compatible, I am mainly
> > targetting Py3K, so compatibility is not a strict requirement.
> >
> > The key points are:
> > 1. Formalize the currently informal concept of iterator by changing
> > iter() from a builtin function to a type.
> > 2. Augment this type with operators to implement operations currently
> > done explicitly by itertools.*.
> > 3. Make this type a base for all other (existing and future) iterator
> > types, e.g. xrange and generator.
> >
> > As a proof of concept, I provide below a sample implementation of how
> > I imagine this type to work (also posted in the original c.l.py.
> > thread):
> >
> >
> > from itertools import chain, tee, islice
> >
> > import __builtin__
> > _builtin_iter = __builtin__.iter
> >
> > class iter(object):
> >
> >     def __init__(self, iterable):
> >         self._it = _builtin_iter(iterable)
> >
> >     def __iter__(self):
> >         return self
> >     def next(self):
> >         return self._it.next()
> >
> >     def __getitem__(self, index):
> >         if isinstance(index, int):
> >             try: return islice(self._it, index, index+1).next()
> >             except StopIteration:
> >                 raise IndexError('Index %d out of range' % index)
> >         else:
> >             start,stop,step = index.start, index.stop, index.step
> >             if start is None: start = 0
> >             if step is None: step = 1
> >             return islice(self._it, start, stop, step)
> >
> >     def __add__(self, other):
> >         return chain(self._it, other)
> >     def __radd__(self,other):
> >         return chain(other, self._it)
> >
> >     def __mul__(self, num):
> >         return chain(*tee(self._it,num))
> >
> >     __rmul__ = __mul__
> >
> > __builtin__.iter = iter
> >
> > if __name__ == '__main__':
> >     def irange(*args):
> >         return iter(xrange(*args))
> >
> >     assert list(irange(5)[:3]) == range(5)[:3]
> >     assert list(irange(5)[3:])  == range(5)[3:]
> >     assert list(irange(5)[1:3]) == range(5)[1:3]
> >     assert list(irange(5)[3:1]) == range(5)[3:1]
> >     assert list(irange(5)[:])   == range(5)[:]
> >     assert irange(5)[3]         == range(5)[3]
> >
> >     s = range(5) + range(7,9)
> >     assert list(irange(5) + irange(7,9)) == s
> >     assert list(irange(5) +  range(7,9)) == s
> >     assert list(range(5)  + irange(7,9)) == s
> >
> >     s = range(5) * 3
> >     assert list(irange(5) * 3) == s
> >     assert list(3 * irange(5)) == s
> >
> >
> > Thoughts, opinions, suggestions, objections, all welcome.
> >
> > Regards,
> > George
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From george.sakkis at gmail.com  Tue Nov 14 00:49:49 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Mon, 13 Nov 2006 18:49:49 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
Message-ID: <91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>

On 11/13/06, Guido van Rossum <guido at python.org> wrote:

> [Adding back python-3000 -- I'm not going to have private conversations]

Sorry, I accidentally hit reply instead of reply-all; I realized it a
minute later and I sent a cc to the list but it hasn't appeared yet..
strange.

> On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > On 11/13/06, Guido van Rossum <guido at python.org> wrote:
> >
> > > Hm. Without knowing much of the background, this appears to be a
> > > worrysome trend away from duck typing. Why would I have to inherit
> > > from a standard class just so that I can implement next()? What's the
> > > advantage of the proposed change?
> >
> > Duck typing will still apply of course; an object that just implements
> > next() without inheriting from iter will continue to be iterable. The
> > advantage is the iterator methods and syntax sugar you get for free,
> > instead of importing them explicitly from itertools (I've noticed that
> > most of my modules lately start with a variation of "import sys,
> > itertools as it").
>
> Well, if you move the itertools functions to the base class, without
> inheriting from that base class you'd lose the itertools
> functionality. (Although many itertools functions take multiple
> generator arguments so I'm not sure how e.g. imap or izip can become
> methods without losing functionality.)

It's actually straightforward; see below.

> > 2. Augment this type with operators to implement operations currently
> > done explicitly by itertools.*.
>
> Please go over all of the itertools functions and show how they can be
> reimplemented as methods. I think you'll find most of them don't work
> this way.

There are currently 14 itertools functions; from those, only 2 don't
take an iterable argument, count() and repeat(). The rest can be
trivially rephrased as methods. Below is the updated iter class
(renamed to Iter for backwards compatibility). I didn't add imap,
ifilter, ifilterfalse and starmap as I think these should be
deprecated in favor of gencomps. Instead I added enumerate and
reversed, as I don't see why these are builtins and not itertools in
the first place.


from itertools import *

class Iter(object):

    def __init__(self, iterable): self._it = iter(iterable)
    def next(self): return self._it.next()
    def __iter__(self): return self
    def __add__(self, other): return Iter(chain(self._it, other))
    def __radd__(self,other): return Iter(chain(other, self._it))
    def __mul__(self, num): return Iter(chain(*tee(self._it,num)))
    __rmul__ = __mul__

    def __getitem__(self, index):
        if isinstance(index, int):
            try: return islice(self._it, index, index+1).next()
            except StopIteration:
                raise IndexError('Index %d out of range' % index)
        else:
            start,stop,step = index.start, index.stop, index.step
            if start is None: start = 0
            if step is None: step = 1
            return Iter(islice(self._it, start, stop, step))

    def cycle(self): return Iter(cycle(self._it))
    def takewhile(self, predicate): return Iter(takewhile(predicate, self._it))
    def dropwhile(self, predicate): return Iter(dropwhile(predicate, self._it))
    def groupby(self, keyfunc=None): return Iter(groupby(self._it, keyfunc))
    def zip(self, *others): return Iter(izip(self._it, *others))
    def enumerate(self): return Iter(enumerate(self._it))
    def reversed(self): return Iter(reversed(self._it))
    def copy(self):
        self._it, new = tee(self._it)
        return Iter(new)


Unit tests omitted for brevity.

George

From guido at python.org  Mon Nov 13 23:17:36 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2006 14:17:36 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
Message-ID: <ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>

[Adding back python-3000 -- I'm not going to have private conversations]

On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> On 11/13/06, Guido van Rossum <guido at python.org> wrote:
>
> > Hm. Without knowing much of the background, this appears to be a
> > worrysome trend away from duck typing. Why would I have to inherit
> > from a standard class just so that I can implement next()? What's the
> > advantage of the proposed change?
>
> Duck typing will still apply of course; an object that just implements
> next() without inheriting from iter will continue to be iterable. The
> advantage is the iterator methods and syntax sugar you get for free,
> instead of importing them explicitly from itertools (I've noticed that
> most of my modules lately start with a variation of "import sys,
> itertools as it").

Well, if you move the itertools functions to the base class, without
inheriting from that base class you'd lose the itertools
functionality. (Although many itertools functions take multiple
generator arguments so I'm not sure how e.g. imap or izip can become
methods without losing functionality.)

> The way I see it, the trend is not away from duck typing but towards
> iterability (generators, itertools, gen. expressions). A builtin iter
> type replacing for the most part the need for itertools is a natural
> next step to this direction.

You need to have a better (more technical) argument than that to convince me.

Let me go back to your original arguments:

> 1. Formalize the currently informal concept of iterator by changing
> iter() from a builtin function to a type.

That surely sounds like a step away from duck typing. Or what else
does "formalize" mean to you?

> 2. Augment this type with operators to implement operations currently
> done explicitly by itertools.*.

Please go over all of the itertools functions and show how they can be
reimplemented as methods. I think you'll find most of them don't work
this way.

> 3. Make this type a base for all other (existing and future) iterator
> types, e.g. xrange and generator.

Here again you're contradicting your "duck typing isn't dead" claim --
you do say "all other (existing and future) iterator types".

> > Are you going to propose similar
> > changes for all standard de-facto interfaces, like sequences,
> > mappings, files etc.?
>
> No, I won't (at least not for now ;-)). Notice however that most
> user-defined sequences, mappings, etc. usually don't start from
> scratch; they either inherit from an existing type or from an
> appropriate mixin. For a baseline iterator that just implements next()
> there's no need for a base type or mixin, but if you consider that
> itertools.* functionality should/would-be-nice to be provided as
> methods and/or syntax instead of imported functions, the parallel
> becomes apparent.

Despite invoking c.l.py I don't think you have broad support for this idea.

It's not too late to drop it.

> George
>
> > On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > Following up on a recent c.l.py thread
> > > (http://groups.google.com/group/comp.lang.python/browse_frm/thread/42818717b400bcd4/#),
> > > I'd like to get an idea from python-dev folks on how much of a chance
> > > is there for a builtin iterator type. Although there might be a slight
> > > possibility to make this proposal backwards compatible, I am mainly
> > > targetting Py3K, so compatibility is not a strict requirement.
> > >
> > > The key points are:
> > > 1. Formalize the currently informal concept of iterator by changing
> > > iter() from a builtin function to a type.
> > > 2. Augment this type with operators to implement operations currently
> > > done explicitly by itertools.*.
> > > 3. Make this type a base for all other (existing and future) iterator
> > > types, e.g. xrange and generator.
> > >
> > > As a proof of concept, I provide below a sample implementation of how
> > > I imagine this type to work (also posted in the original c.l.py.
> > > thread):
> > >
> > >
> > > from itertools import chain, tee, islice
> > >
> > > import __builtin__
> > > _builtin_iter = __builtin__.iter
> > >
> > > class iter(object):
> > >
> > >     def __init__(self, iterable):
> > >         self._it = _builtin_iter(iterable)
> > >
> > >     def __iter__(self):
> > >         return self
> > >     def next(self):
> > >         return self._it.next()
> > >
> > >     def __getitem__(self, index):
> > >         if isinstance(index, int):
> > >             try: return islice(self._it, index, index+1).next()
> > >             except StopIteration:
> > >                 raise IndexError('Index %d out of range' % index)
> > >         else:
> > >             start,stop,step = index.start, index.stop, index.step
> > >             if start is None: start = 0
> > >             if step is None: step = 1
> > >             return islice(self._it, start, stop, step)
> > >
> > >     def __add__(self, other):
> > >         return chain(self._it, other)
> > >     def __radd__(self,other):
> > >         return chain(other, self._it)
> > >
> > >     def __mul__(self, num):
> > >         return chain(*tee(self._it,num))
> > >
> > >     __rmul__ = __mul__
> > >
> > > __builtin__.iter = iter
> > >
> > > if __name__ == '__main__':
> > >     def irange(*args):
> > >         return iter(xrange(*args))
> > >
> > >     assert list(irange(5)[:3]) == range(5)[:3]
> > >     assert list(irange(5)[3:])  == range(5)[3:]
> > >     assert list(irange(5)[1:3]) == range(5)[1:3]
> > >     assert list(irange(5)[3:1]) == range(5)[3:1]
> > >     assert list(irange(5)[:])   == range(5)[:]
> > >     assert irange(5)[3]         == range(5)[3]
> > >
> > >     s = range(5) + range(7,9)
> > >     assert list(irange(5) + irange(7,9)) == s
> > >     assert list(irange(5) +  range(7,9)) == s
> > >     assert list(range(5)  + irange(7,9)) == s
> > >
> > >     s = range(5) * 3
> > >     assert list(irange(5) * 3) == s
> > >     assert list(3 * irange(5)) == s
> > >
> > >
> > > Thoughts, opinions, suggestions, objections, all welcome.
> > >
> > > Regards,
> > > George
> > > _______________________________________________
> > > Python-3000 mailing list
> > > Python-3000 at python.org
> > > http://mail.python.org/mailman/listinfo/python-3000
> > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
> > >
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> >
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ferringb at gmail.com  Tue Nov 14 01:52:53 2006
From: ferringb at gmail.com (Brian Harring)
Date: Mon, 13 Nov 2006 16:52:53 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
Message-ID: <20061114005252.GA9730@seldon>

On Mon, Nov 13, 2006 at 01:44:10PM -0500, George Sakkis wrote:
> As a proof of concept, I provide below a sample implementation of how
> I imagine this type to work (also posted in the original c.l.py.
> thread):
>
> from itertools import chain, tee, islice
> 
> import __builtin__
> _builtin_iter = __builtin__.iter
> 
> class iter(object):
> 
>     def __init__(self, iterable):
>         self._it = _builtin_iter(iterable)
> 
>     def __iter__(self):
>         return self
>     def next(self):
>         return self._it.next()
> 
>     def __getitem__(self, index):
>         if isinstance(index, int):
>             try: return islice(self._it, index, index+1).next()
>             except StopIteration:
>                 raise IndexError('Index %d out of range' % index)
>         else:
>             start,stop,step = index.start, index.stop, index.step
>             if start is None: start = 0
>             if step is None: step = 1
>             return islice(self._it, start, stop, step)

Tend to think __getitem__ on base iter is a rather bad idea due to the 
fact it implies the underlying iter is a mapping; it's not.

Additionally... islice *does* have issues when working against an 
iterator that hasn't been protected via tee; that's not a complaint 
against islice mind you, it's just reality that if you islice an 
iterator, make damn sure you pull from that islice before going 
back to the original iterator; intermixing them is a grand way to 
shoot yourself in the foot.

Current usage, it's easy to spot it (look for islice); overloading 
getitem however means it gets far harder, thus more likely to trip 
folks up (in my opinion).


>     def __add__(self, other):
>         return chain(self._it, other)

The change to the iterator protocol here, that iter1 + iter2 == 
chain(iter1, iter2) breaks down for self iterating types; trying to 
shoe horn in add/radd means that for self-iters, add/radd/mul are now 
taken away from them.

Duck typing works (imo) when the hook points all behave similiar; not 
convinced that abusing __add__ is wise, definitely not convinced 
__getitem__ is sane, especially since islice doesn't guarntee that 
it'll return that specific slice- just gurantees that it'll follow a 
specific pattern for consuming from the wrapped iter.

So... nuking those, you wind up with existing iter.  Perhaps providing 
sane examples of __getitem__ usage would help; add sort of makes sense 
sans it getting whacky for self-iters.

~harring
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061113/018cee7e/attachment.pgp 

From guido at python.org  Tue Nov 14 02:45:11 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2006 17:45:11 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
Message-ID: <ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>

As my final word, I think this is a seriously bad idea, and as you're
not answering my challenge about duck typing I don't think you
understand your own proposal.

On 11/13/06, George Sakkis <george.sakkis at gmail.com> wrote:
> On 11/13/06, Guido van Rossum <guido at python.org> wrote:
>
> > [Adding back python-3000 -- I'm not going to have private conversations]
>
> Sorry, I accidentally hit reply instead of reply-all; I realized it a
> minute later and I sent a cc to the list but it hasn't appeared yet..
> strange.
>
> > On 11/13/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > On 11/13/06, Guido van Rossum <guido at python.org> wrote:
> > >
> > > > Hm. Without knowing much of the background, this appears to be a
> > > > worrysome trend away from duck typing. Why would I have to inherit
> > > > from a standard class just so that I can implement next()? What's the
> > > > advantage of the proposed change?
> > >
> > > Duck typing will still apply of course; an object that just implements
> > > next() without inheriting from iter will continue to be iterable. The
> > > advantage is the iterator methods and syntax sugar you get for free,
> > > instead of importing them explicitly from itertools (I've noticed that
> > > most of my modules lately start with a variation of "import sys,
> > > itertools as it").
> >
> > Well, if you move the itertools functions to the base class, without
> > inheriting from that base class you'd lose the itertools
> > functionality. (Although many itertools functions take multiple
> > generator arguments so I'm not sure how e.g. imap or izip can become
> > methods without losing functionality.)
>
> It's actually straightforward; see below.
>
> > > 2. Augment this type with operators to implement operations currently
> > > done explicitly by itertools.*.
> >
> > Please go over all of the itertools functions and show how they can be
> > reimplemented as methods. I think you'll find most of them don't work
> > this way.
>
> There are currently 14 itertools functions; from those, only 2 don't
> take an iterable argument, count() and repeat(). The rest can be
> trivially rephrased as methods. Below is the updated iter class
> (renamed to Iter for backwards compatibility). I didn't add imap,
> ifilter, ifilterfalse and starmap as I think these should be
> deprecated in favor of gencomps. Instead I added enumerate and
> reversed, as I don't see why these are builtins and not itertools in
> the first place.
>
>
> from itertools import *
>
> class Iter(object):
>
>     def __init__(self, iterable): self._it = iter(iterable)
>     def next(self): return self._it.next()
>     def __iter__(self): return self
>     def __add__(self, other): return Iter(chain(self._it, other))
>     def __radd__(self,other): return Iter(chain(other, self._it))
>     def __mul__(self, num): return Iter(chain(*tee(self._it,num)))
>     __rmul__ = __mul__
>
>     def __getitem__(self, index):
>         if isinstance(index, int):
>             try: return islice(self._it, index, index+1).next()
>             except StopIteration:
>                 raise IndexError('Index %d out of range' % index)
>         else:
>             start,stop,step = index.start, index.stop, index.step
>             if start is None: start = 0
>             if step is None: step = 1
>             return Iter(islice(self._it, start, stop, step))
>
>     def cycle(self): return Iter(cycle(self._it))
>     def takewhile(self, predicate): return Iter(takewhile(predicate, self._it))
>     def dropwhile(self, predicate): return Iter(dropwhile(predicate, self._it))
>     def groupby(self, keyfunc=None): return Iter(groupby(self._it, keyfunc))
>     def zip(self, *others): return Iter(izip(self._it, *others))
>     def enumerate(self): return Iter(enumerate(self._it))
>     def reversed(self): return Iter(reversed(self._it))
>     def copy(self):
>         self._it, new = tee(self._it)
>         return Iter(new)
>
>
> Unit tests omitted for brevity.
>
> George
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From george.sakkis at gmail.com  Tue Nov 14 04:26:54 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Mon, 13 Nov 2006 22:26:54 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
Message-ID: <91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>

On 11/13/06, Guido van Rossum <guido at python.org> wrote:

> As my final word, I think this is a seriously bad idea, and as you're
> not answering my challenge about duck typing I don't think you
> understand your own proposal.

I think I do, though I can't tell the same about the reasons of your
objections to it. In your previous reply you mentioned that I need to
have a more technical argument to convince you, and that's why I chose
to reply with the most technical argument, a proof of concept
implementation that addresses your concerns about how to incorporate
the itertools functions as methods.

As for the duck typing, I mentioned already that nobody forces you to
extend this type to make some class an iterator, as nobody forces you
to extend dict or dictmixin to write a user-defined mapping. You may
well start from scratch implementing just next(); if you don't plan to
use "+", "*" or any of the itertools operations on this type,
extending Iter is useless. If you do plan to provide these operations
though, you may either write them from scratch every time, or extend
Iter.

I honestly fail to understand your current objections. Is my analogy
with dictmixin flawed ? Would anything change if I named it
"itermixin" instead of iter or Iter ? I'm ok with the idea being
rejected, but at least I'd like to understand the reasons.

George

From alexander.belopolsky at gmail.com  Tue Nov 14 05:02:24 2006
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Tue, 14 Nov 2006 04:02:24 +0000 (UTC)
Subject: [Python-3000] Builtin iterator type
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
Message-ID: <loom.20061114T045406-213@post.gmane.org>

George Sakkis <george.sakkis <at> gmail.com> writes:
...
> As for the duck typing, I mentioned already that nobody forces you to
> extend this type to make some class an iterator, as nobody forces you
> to extend dict or dictmixin to write a user-defined mapping. You may
> well start from scratch implementing just next(); if you don't plan to
> use "+", "*" or any of the itertools operations on this type,
> extending Iter is useless. If you do plan to provide these operations
> though, you may either write them from scratch every time, or extend
> Iter.
> 

Unfortunately, an iterator that only defines next() will not be
compatible with the functions that would take advantage of
your proposal.


From gsakkis at rutgers.edu  Tue Nov 14 05:19:51 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Mon, 13 Nov 2006 23:19:51 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061114005252.GA9730@seldon>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<20061114005252.GA9730@seldon>
Message-ID: <91ad5bf80611132019m54de49d2t2ceefbfe181d2aac@mail.gmail.com>

On 11/13/06, Brian Harring <ferringb at gmail.com> wrote:

> Tend to think __getitem__ on base iter is a rather bad idea due to the
> fact it implies the underlying iter is a mapping; it's not.

Only mappings define __getitem__ ?!? Now that's a new one.

> Additionally... islice *does* have issues when working against an
> iterator that hasn't been protected via tee; that's not a complaint
> against islice mind you, it's just reality that if you islice an
> iterator, make damn sure you pull from that islice before going
> back to the original iterator; intermixing them is a grand way to
> shoot yourself in the foot.
>
> Current usage, it's easy to spot it (look for islice); overloading
> getitem however means it gets far harder, thus more likely to trip
> folks up (in my opinion).

That might be a valid point, but according to this argument, Numpy
slices are evil too as they return a view and not a copy, like most
other sequences. There is a limit to polymorphism and duck typing; you
can't always pretend you don't care about the actual type.

> >     def __add__(self, other):
> >         return chain(self._it, other)
>
> The change to the iterator protocol here, that iter1 + iter2 ==
> chain(iter1, iter2) breaks down for self iterating types; trying to
> shoe horn in add/radd means that for self-iters, add/radd/mul are now
> taken away from them.

Excuse my ignorance, but what's a "self-iter" ?

> Duck typing works (imo) when the hook points all behave similiar; not
> convinced that abusing __add__ is wise, definitely not convinced
> __getitem__ is sane, especially since islice doesn't guarntee that
> it'll return that specific slice- just gurantees that it'll follow a
> specific pattern for consuming from the wrapped iter.

Care to provide an example of what you're talking about ?

> So... nuking those, you wind up with existing iter.  Perhaps providing
> sane examples of __getitem__ usage would help; add sort of makes sense
> sans it getting whacky for self-iters.

If by sane example you mean expectable, not necessarily useful, here's
a simple one off the top of my head:

>>> from itertools import count
>>> x = Iter(count(10))
>>> y = Iter('abcdefghij')
>>> list((x[:3] + y[7:]) * 2)
[10, 11, 12, 'h', 'i', 'j', 10, 11, 12, 'h', 'i', 'j']

Try rewriting it using explicit itertools calls and see what looks
more readable.

George

From guido at python.org  Tue Nov 14 05:21:16 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 13 Nov 2006 20:21:16 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
Message-ID: <ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>

On 11/13/06, George Sakkis <george.sakkis at gmail.com> wrote:
> On 11/13/06, Guido van Rossum <guido at python.org> wrote:
>
> > As my final word, I think this is a seriously bad idea, and as you're
> > not answering my challenge about duck typing I don't think you
> > understand your own proposal.
>
> I think I do, though I can't tell the same about the reasons of your
> objections to it. In your previous reply you mentioned that I need to
> have a more technical argument to convince you, and that's why I chose
> to reply with the most technical argument, a proof of concept
> implementation that addresses your concerns about how to incorporate
> the itertools functions as methods.
>
> As for the duck typing, I mentioned already that nobody forces you to
> extend this type to make some class an iterator, as nobody forces you
> to extend dict or dictmixin to write a user-defined mapping. You may
> well start from scratch implementing just next(); if you don't plan to
> use "+", "*" or any of the itertools operations on this type,
> extending Iter is useless. If you do plan to provide these operations
> though, you may either write them from scratch every time, or extend
> Iter.
>
> I honestly fail to understand your current objections. Is my analogy
> with dictmixin flawed ? Would anything change if I named it
> "itermixin" instead of iter or Iter ? I'm ok with the idea being
> rejected, but at least I'd like to understand the reasons.

The flaw is that you're creating two categories of iterators: those
that support the various methods you're adding, and those that don't.
This means that the itertools module can't be discarded, because it is
still needed to support those operations for the iterators that don't
have them natively. Thus, you're introducing two ways of doing the
same thing -- using itertools (works for all iterators) or using the
methods (only works for iterators that inherit from your base class).

Please stop wasting everybody's time.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ronaldoussoren at mac.com  Tue Nov 14 07:46:02 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 14 Nov 2006 07:46:02 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
Message-ID: <8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>


On  14 Nov 2006, at 4:26 AM, George Sakkis wrote:

>
> I honestly fail to understand your current objections. Is my analogy
> with dictmixin flawed ? Would anything change if I named it
> "itermixin" instead of iter or Iter ? I'm ok with the idea being
> rejected, but at least I'd like to understand the reasons.

You basically propose to extend the iterator interface from 1 method  
to len(dir(itertools)) methods without any clear benefits. Saving a  
single import line is not a clear benefit.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061114/47b3efb4/attachment-0001.bin 

From george.sakkis at gmail.com  Tue Nov 14 07:59:57 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 01:59:57 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
Message-ID: <91ad5bf80611132259v13c7e13ap911d201e189cc4bf@mail.gmail.com>

On 11/14/06, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On  14 Nov 2006, at 4:26 AM, George Sakkis wrote:
>
> >
> > I honestly fail to understand your current objections. Is my analogy
> > with dictmixin flawed ? Would anything change if I named it
> > "itermixin" instead of iter or Iter ? I'm ok with the idea being
> > rejected, but at least I'd like to understand the reasons.
>
> You basically propose to extend the iterator interface from 1 method
> to len(dir(itertools)) methods without any clear benefits. Saving a
> single import line is not a clear benefit.

The original motivation is to provide syntax sugar (+, *, indexing,
slicing) to iterators, which you can't do with functions. The rest
methods that would be spelled out with their current itertools names
is a fortunate side-effect, not the primary motivation.

George

From george.sakkis at gmail.com  Tue Nov 14 08:03:08 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 02:03:08 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>
Message-ID: <91ad5bf80611132303y33ecee81k8d7b2093d4b6806d@mail.gmail.com>

On 11/13/06, Guido van Rossum <guido at python.org> wrote:

> On 11/13/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > I honestly fail to understand your current objections. Is my analogy
> > with dictmixin flawed ? Would anything change if I named it
> > "itermixin" instead of iter or Iter ? I'm ok with the idea being
> > rejected, but at least I'd like to understand the reasons.
>
> The flaw is that you're creating two categories of iterators: those
> that support the various methods you're adding, and those that don't.
> This means that the itertools module can't be discarded, because it is
> still needed to support those operations for the iterators that don't
> have them natively. Thus, you're introducing two ways of doing the
> same thing -- using itertools (works for all iterators) or using the
> methods (only works for iterators that inherit from your base class).

Understood. Any _technical_ reasons then why shouldn't all iterators
inherit from it, or "duck typing is The Right Way" should be taken as
an axiom ? Why do we somehow _need_ itertools when we don't need
sequencetools, mappingtools, etc ? What's so special about the
iterator protocol ?

George

From jcarlson at uci.edu  Tue Nov 14 09:09:31 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 14 Nov 2006 00:09:31 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611132259v13c7e13ap911d201e189cc4bf@mail.gmail.com>
References: <8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
	<91ad5bf80611132259v13c7e13ap911d201e189cc4bf@mail.gmail.com>
Message-ID: <20061113235118.82E8.JCARLSON@uci.edu>


"George Sakkis" <george.sakkis at gmail.com> wrote:
> The original motivation is to provide syntax sugar (+, *, indexing,
> slicing) to iterators, which you can't do with functions. The rest
> methods that would be spelled out with their current itertools names
> is a fortunate side-effect, not the primary motivation.

I've never needed (the equivalent of) +, *, indexing, or slicing on
arbitrary iterators (I make them iterators so that I can iterate over
them), so I don't see the need to make the current object more
cumbersome than necessary.

For those who would like the equivalent of +, *, indexing, or slicing
arbitrary iterators, there is the itertools module.  -1

 - Josiah


From gsakkis at rutgers.edu  Tue Nov 14 05:36:25 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Mon, 13 Nov 2006 23:36:25 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>
Message-ID: <91ad5bf80611132036obdb042i1b02a860569d86ac@mail.gmail.com>

On 11/13/06, Guido van Rossum <guido at python.org> wrote:

> On 11/13/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > On 11/13/06, Guido van Rossum <guido at python.org> wrote:
> >
> > > As my final word, I think this is a seriously bad idea, and as you're
> > > not answering my challenge about duck typing I don't think you
> > > understand your own proposal.
> >
> > I think I do, though I can't tell the same about the reasons of your
> > objections to it. In your previous reply you mentioned that I need to
> > have a more technical argument to convince you, and that's why I chose
> > to reply with the most technical argument, a proof of concept
> > implementation that addresses your concerns about how to incorporate
> > the itertools functions as methods.
> >
> > As for the duck typing, I mentioned already that nobody forces you to
> > extend this type to make some class an iterator, as nobody forces you
> > to extend dict or dictmixin to write a user-defined mapping. You may
> > well start from scratch implementing just next(); if you don't plan to
> > use "+", "*" or any of the itertools operations on this type,
> > extending Iter is useless. If you do plan to provide these operations
> > though, you may either write them from scratch every time, or extend
> > Iter.
> >
> > I honestly fail to understand your current objections. Is my analogy
> > with dictmixin flawed ? Would anything change if I named it
> > "itermixin" instead of iter or Iter ? I'm ok with the idea being
> > rejected, but at least I'd like to understand the reasons.
>
> The flaw is that you're creating two categories of iterators: those
> that support the various methods you're adding, and those that don't.
> This means that the itertools module can't be discarded, because it is
> still needed to support those operations for the iterators that don't
> have them natively. Thus, you're introducing two ways of doing the
> same thing -- using itertools (works for all iterators) or using the
> methods (only works for iterators that inherit from your base class).

Understood. Any _technical_ reasons then why shouldn't all iterators
inherit from it, or "duck typing is The Right Way" should be taken as
axiom ? Why do we somehow _need_ itertools when we don't need
sequencetools, mappingtools, etc ? What's special about the iterator
protocol ?

George

From bjourne at gmail.com  Tue Nov 14 10:21:24 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Tue, 14 Nov 2006 09:21:24 +0000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
Message-ID: <740c3aec0611140121t6fac8dfcm4a58f3c56ae9daef@mail.gmail.com>

On 11/14/06, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
> On  14 Nov 2006, at 4:26 AM, George Sakkis wrote:
> > I honestly fail to understand your current objections. Is my analogy
> > with dictmixin flawed ? Would anything change if I named it
> > "itermixin" instead of iter or Iter ? I'm ok with the idea being
> > rejected, but at least I'd like to understand the reasons.
>
> You basically propose to extend the iterator interface from 1 method
> to len(dir(itertools)) methods without any clear benefits. Saving a
> single import line is not a clear benefit.

But why is both the dict and list protocol so fat then? Is it hard to
create your own dict or list-derived types in Python?

-- 
mvh Bj?rn

From fredrik at pythonware.com  Tue Nov 14 10:59:55 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2006 10:59:55 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <740c3aec0611140121t6fac8dfcm4a58f3c56ae9daef@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
	<740c3aec0611140121t6fac8dfcm4a58f3c56ae9daef@mail.gmail.com>
Message-ID: <ejc42s$b6f$1@sea.gmane.org>

BJ?rn Lindqvist wrote:

> But why is both the dict and list protocol so fat then? Is it hard to
> create your own dict or list-derived types in Python?

don't confuse things like lists and dictionaries with things like 
sequences and mappings.  iterators and iterables belong to the second 
category.

</F>


From fredrik at pythonware.com  Tue Nov 14 13:36:08 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2006 13:36:08 +0100
Subject: [Python-3000] Builtin iterator type
References: <06Nov13.145133pst."58648"@synergy1.parc.xerox.com>
	<20061113231538.20948.713332044.divmod.quotient.26448@ohm>
Message-ID: <ejcd7o$cbd$1@sea.gmane.org>

Jean-Paul Calderone wrote:

> Anyway, the proposal was in no way suggesting moving away from duck
> typing.  The new iter was to be a _wrapper_ around any existing
> iterator, not a base class for iterators.

the OP seems to think otherwise:

    The key points are:
        /.../
    3. Make this type a base for all other (existing and future) iterator
    types, e.g. xrange and generator.

</F> 




From ark-mlist at att.net  Tue Nov 14 16:31:37 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue, 14 Nov 2006 10:31:37 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov13.145133pst."58648"@synergy1.parc.xerox.com>
Message-ID: <002801c70801$fd33d310$6402a8c0@arkdesktop>

> Duck typing is a seriously bad idea

Why?


From ncoghlan at gmail.com  Tue Nov 14 16:30:15 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 15 Nov 2006 01:30:15 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611132303y33ecee81k8d7b2093d4b6806d@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>
	<91ad5bf80611132303y33ecee81k8d7b2093d4b6806d@mail.gmail.com>
Message-ID: <4559E107.9090207@gmail.com>

George Sakkis wrote:
> Understood. Any _technical_ reasons then why shouldn't all iterators
> inherit from it, or "duck typing is The Right Way" should be taken as
> an axiom ? Why do we somehow _need_ itertools when we don't need
> sequencetools, mappingtools, etc ? What's so special about the
> iterator protocol ?

I have sympathy for the idea of syntactic sugar for working with iterators, I 
really do (just read the thread I started about iterator slicing early last 
year [1]).

However, since that time we've moved on to the idea of returning richer view 
objects rather than bare iterators for some of the methods being changed in 
Py3k to return something other than a list (such as dict.keys() and friends).

The underlying realisation behind that change is that an iterable is free to 
implement whichever aspects of the sequence and mapping interfaces it wants 
to, depending on what makes sense for the particular iterator.

By taking this route the basic iterator protocol is kept simple with minimal 
assumptions about the underlying data storage (which is exactly what leads to 
the protocol's wide applicability), while the richer mapping and sequence 
interfaces are supported as appropriate in order to make the objects easier to 
work with.

The difficult with pulling views out into a generic mixin class such as the 
one you propose is that they are typically strongly coupled to their 
associated concrete container class. Arbitrary iterators don't make strong 
enough guarantees to allow that to work without suprises.

For example, it would be somewhat surprising if (a, b, c), (x, y, z) = it[:3], 
it[:3] puts different results into a & x, but that is exactly what happens if 
'it' is a sliceable iterator like the one you suggest. This didn't bother me 
last year, but I've since grown to dislike it (primarily due to the 
realisation mentioned above that returning view objects where appropriate is 
likely to be a better answer, even if it lacks "one idiom to rule them all" 
status).

Regards,
Nick.

[1]
http://mail.python.org/pipermail/python-dev/2005-January/051257.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From george.sakkis at gmail.com  Tue Nov 14 16:29:42 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 10:29:42 -0500
Subject: [Python-3000] Python-3000 Digest, Vol 9, Issue 27
In-Reply-To: <mailman.71.1163502028.12842.python-3000@python.org>
References: <mailman.71.1163502028.12842.python-3000@python.org>
Message-ID: <91ad5bf80611140729o55570666gb8e07c761487c0ce@mail.gmail.com>

On 11/14/06, Fredrik Lundh <fredrik at pythonware.com> wrote:

> BJ?rn Lindqvist wrote:
>
> > But why is both the dict and list protocol so fat then? Is it hard to
> > create your own dict or list-derived types in Python?
>
> don't confuse things like lists and dictionaries with things like
> sequences and mappings.  iterators and iterables belong to the second
> category.

This doesn't answer my last question: why do we need itertools when we
can live without sequencetools, mappingtools, fileliketools, etc. ?

George

From gsakkis at rutgers.edu  Tue Nov 14 16:26:03 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Tue, 14 Nov 2006 10:26:03 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061114144840.GA24358@panix.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<20061114005252.GA9730@seldon>
	<91ad5bf80611132019m54de49d2t2ceefbfe181d2aac@mail.gmail.com>
	<20061114144840.GA24358@panix.com>
Message-ID: <91ad5bf80611140726x5ace4eefpe7ae481600417c56@mail.gmail.com>

On 11/14/06, Aahz <aahz at pythoncraft.com> wrote:

> [off-list to avoid more CLUTTER]

Sorry, I don't consider a well-intentioned discussion to understand
how things work at the design level of python CLUTTER.

> On Mon, Nov 13, 2006, George Sakkis wrote:
> >
> > Excuse my ignorance, but what's a "self-iter" ?
>
> Files

# iterate over the first 10 lines of two files
f1, f2 = map(Iter, (open('foo.txt'), open('bar.txt'))
for line in f1[:10] + f2[:10]:
    print line,


Pretty pythonic if you ask me.

George

From fredrik at pythonware.com  Tue Nov 14 16:54:10 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2006 16:54:10 +0100
Subject: [Python-3000] Python-3000 Digest, Vol 9, Issue 27
References: <mailman.71.1163502028.12842.python-3000@python.org>
	<91ad5bf80611140729o55570666gb8e07c761487c0ce@mail.gmail.com>
Message-ID: <ejcor2$qj8$1@sea.gmane.org>

George Sakkis wrote:

>> don't confuse things like lists and dictionaries with things like
>> sequences and mappings.  iterators and iterables belong to the second
>> category.
>
> This doesn't answer my last question: why do we need itertools when we
> can live without sequencetools, mappingtools, fileliketools, etc. ?

try listing the sequence and mapping and file tools that Python actually provides, and
look at how they're used in contemporary Python code.  do you see a pattern ?

</F> 




From ncoghlan at gmail.com  Tue Nov 14 17:01:51 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 15 Nov 2006 02:01:51 +1000
Subject: [Python-3000] Python-3000 Digest, Vol 9, Issue 27
In-Reply-To: <91ad5bf80611140729o55570666gb8e07c761487c0ce@mail.gmail.com>
References: <mailman.71.1163502028.12842.python-3000@python.org>
	<91ad5bf80611140729o55570666gb8e07c761487c0ce@mail.gmail.com>
Message-ID: <4559E86F.4010805@gmail.com>

George Sakkis wrote:
> On 11/14/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> 
>> BJ?rn Lindqvist wrote:
>>
>>> But why is both the dict and list protocol so fat then? Is it hard to
>>> create your own dict or list-derived types in Python?
>> don't confuse things like lists and dictionaries with things like
>> sequences and mappings.  iterators and iterables belong to the second
>> category.
> 
> This doesn't answer my last question: why do we need itertools when we
> can live without sequencetools, mappingtools, fileliketools, etc. ?

Because the iterator protocol is simple enough to be easily composable - there 
are a wide variety of things that can be done with an object when all you know 
about it is that it is some form of iterable. The more assumptions you have to 
start making about the iterable, the less widely applicable the resulting 
operation will be.

And I think the number one reason we don't see a compelling need for the extra 
tool libraries you describe is the fact that sequences, mappings and files are 
themselves all iterables.

That said, there are actually a number of modules for working with sequences 
in the standard library, like bisect and heapq. They just aren't lumped into 
one place the way itertools and functools are.

Having a rich method API vs having a narrow method API and duck-typed support 
functions is a design trade-off. In the case of sequences and mappings, the 
trade-off went towards a richer API because the de facto reference 
implementations were the builtin dict and list classes (which is why DictMixin 
and ListMixin are so useful when implementing your own containers). In the 
case of iterables and iterators, the trade-off went towards the narrow API so 
that the interface could be used in a wide variety of situations (lines in a 
file, records in a database, characters in a string, bytes from a serial port, 
frames in a bowling game, active players in a MMORPG, etc, etc, etc).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From george.sakkis at gmail.com  Tue Nov 14 17:49:35 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 11:49:35 -0500
Subject: [Python-3000] Builtin iterator type
Message-ID: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>

On 11/14/06, Nick Coghlan <ncoghlan at gmail.com> wrote:

> George Sakkis wrote:
> > On 11/14/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> >
> >> BJ?rn Lindqvist wrote:
> >>
> >>> But why is both the dict and list protocol so fat then? Is it hard to
> >>> create your own dict or list-derived types in Python?
> >> don't confuse things like lists and dictionaries with things like
> >> sequences and mappings.  iterators and iterables belong to the second
> >> category.
> >
> > This doesn't answer my last question: why do we need itertools when we
> > can live without sequencetools, mappingtools, fileliketools, etc. ?
>
> Because the iterator protocol is simple enough to be easily composable - there
> are a wide variety of things that can be done with an object when all you know
> about it is that it is some form of iterable. The more assumptions you have to
> start making about the iterable, the less widely applicable the resulting
> operation will be.
>
> And I think the number one reason we don't see a compelling need for the extra
> tool libraries you describe is the fact that sequences, mappings and files are
> themselves all iterables.
>
> That said, there are actually a number of modules for working with sequences
> in the standard library, like bisect and heapq. They just aren't lumped into
> one place the way itertools and functools are.
>
> Having a rich method API vs having a narrow method API and duck-typed support
> functions is a design trade-off. In the case of sequences and mappings, the
> trade-off went towards a richer API because the de facto reference
> implementations were the builtin dict and list classes (which is why DictMixin
> and ListMixin are so useful when implementing your own containers). In the
> case of iterables and iterators, the trade-off went towards the narrow API so
> that the interface could be used in a wide variety of situations (lines in a
> file, records in a database, characters in a string, bytes from a serial port,
> frames in a bowling game, active players in a MMORPG, etc, etc, etc).

Given the overly negative reaction to a base iterator type, I withdraw
the part of my proposal that suggests this type as the base of all
iterators. Instead I propose a builtin Iter (or even better iter, if
there is no objection to change iter's current behavior) as an OO
replacement of the (functional) itertools API. In other words, instead
of:

from itertools import chain, islice, groupby
for k,sub in groupby(chain(islice(it1, 1, None), islice(it2, 5)),
key=str.lower):
    print k, list(sub)

you will write:

for k,sub in (Iter(it1)[1:] + Iter(it2)[:5]).groupby(str.lower):
    print k, list(sub)

it1, it2 can be arbitrary iterators, no restrictions imposed. Iter()
(or iter()) will just return a thin wrapper around them to provide
itertools functionality in a less verbose, more pythonic, way. We can
decide on the exact API of this wrapper later, but for now I'd like to
get a general impression of its acceptance chances.

Thoughts ?

George

From fredrik at pythonware.com  Tue Nov 14 18:00:48 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 14 Nov 2006 18:00:48 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
Message-ID: <ejcsnv$bma$1@sea.gmane.org>

George Sakkis wrote:

> Given the overly negative reaction to a base iterator type, I withdraw
> the part of my proposal that suggests this type as the base of all
> iterators. Instead I propose a builtin Iter (or even better iter, if
> there is no objection to change iter's current behavior) as an OO
> replacement of the (functional) itertools API. In other words, instead
> of:
> 
> from itertools import chain, islice, groupby
> for k,sub in groupby(chain(islice(it1, 1, None), islice(it2, 5)),
> key=str.lower):
>     print k, list(sub)
> 
> you will write:
> 
> for k,sub in (Iter(it1)[1:] + Iter(it2)[:5]).groupby(str.lower):
>     print k, list(sub)

Uhuh.  And why would this more important than adding, say, Array, 
AtExit, Bz2, Cgi, CgiTB, Cmath, Code, Codecs, Collections, Copy, Csv, 
DateTime, Decimal, ElementTree, ErrNo, Ftplib, Gc, GetOpt, GetText, 
Glob, Gzip, HashLib, HMac, Inspect, Locale, Logging, Marshal, Math, 
Mmap, Operator, Os, OsPath, Pickle, Platform, Random, Re, Select, 
ShUtil, SmtpLib, Socket, Struct, SubProcess, Sys, TempFile, Threading, 
Time, Traceback, UnicodeData, UrlLib, UrlLib2, UrlParse, Warnings, 
WeakRef, XmlDom, XmlSax, ZipFile, or Zlib builtins?  Most of these are 
definitely used a lot more often than itertools in typical Python 
programs...

</F>


From jcarlson at uci.edu  Tue Nov 14 18:17:17 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 14 Nov 2006 09:17:17 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
Message-ID: <20061114090956.82EB.JCARLSON@uci.edu>


"George Sakkis" <george.sakkis at gmail.com> wrote:
> 
> On 11/14/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> > George Sakkis wrote:
> > > On 11/14/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > >
> > >> BJ?rn Lindqvist wrote:
> > >>
> > >>> But why is both the dict and list protocol so fat then? Is it hard to
> > >>> create your own dict or list-derived types in Python?
> > >> don't confuse things like lists and dictionaries with things like
> > >> sequences and mappings.  iterators and iterables belong to the second
> > >> category.
> > >
> > > This doesn't answer my last question: why do we need itertools when we
> > > can live without sequencetools, mappingtools, fileliketools, etc. ?
> >
> > Because the iterator protocol is simple enough to be easily composable - there
> > are a wide variety of things that can be done with an object when all you know
> > about it is that it is some form of iterable. The more assumptions you have to
> > start making about the iterable, the less widely applicable the resulting
> > operation will be.
> >
> > And I think the number one reason we don't see a compelling need for the extra
> > tool libraries you describe is the fact that sequences, mappings and files are
> > themselves all iterables.
> >
> > That said, there are actually a number of modules for working with sequences
> > in the standard library, like bisect and heapq. They just aren't lumped into
> > one place the way itertools and functools are.
> >
> > Having a rich method API vs having a narrow method API and duck-typed support
> > functions is a design trade-off. In the case of sequences and mappings, the
> > trade-off went towards a richer API because the de facto reference
> > implementations were the builtin dict and list classes (which is why DictMixin
> > and ListMixin are so useful when implementing your own containers). In the
> > case of iterables and iterators, the trade-off went towards the narrow API so
> > that the interface could be used in a wide variety of situations (lines in a
> > file, records in a database, characters in a string, bytes from a serial port,
> > frames in a bowling game, active players in a MMORPG, etc, etc, etc).
> 
> Given the overly negative reaction to a base iterator type, I withdraw
> the part of my proposal that suggests this type as the base of all
> iterators. Instead I propose a builtin Iter (or even better iter, if
> there is no objection to change iter's current behavior) as an OO
> replacement of the (functional) itertools API. In other words, instead
> of:

-1.  There is nothing wrong with a functional approach.  If you dislike
the verbosity of the names of the operations, and/or the arguments they
take, you are free to rename them:
    from itertools import chain as CH
... or write wrappers:
    def IS(it, start):
        return islice(it, start, None)

You can even have your itertools automatically inserted into
__builtins__ on startup with a small manipulation of site.py .

Heck, if _you_ want them, _you_ are free to have your Iter object
inserted into the __builtins__ namespace by site.py .  But due to the
overwhelming -1s from _everyone_, _including Guido_, your odds of
getting anything like Iter into mainline Python in the next 18 months 
(when Python 2.6 is expected to be released), is very low.

 - Josiah


From steven.bethard at gmail.com  Tue Nov 14 18:16:59 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Tue, 14 Nov 2006 10:16:59 -0700
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
Message-ID: <d11dcfba0611140916x1af5425aoa1dc030895695227@mail.gmail.com>

On 11/14/06, George Sakkis <george.sakkis at gmail.com> wrote:
> Given the overly negative reaction to a base iterator type, I withdraw
> the part of my proposal that suggests this type as the base of all
> iterators. Instead I propose a builtin Iter (or even better iter, if
> there is no objection to change iter's current behavior) as an OO
> replacement of the (functional) itertools API.
[snip]
> for k,sub in (Iter(it1)[1:] + Iter(it2)[:5]).groupby(str.lower):
>     print k, list(sub)

Why don't you distribute this class as a third-party library for a
while, and see if people use it.  If it turns out that everyone loves
the functionality that it provides, then you can lobby for its
inclusion in the standard library.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From george.sakkis at gmail.com  Tue Nov 14 16:49:54 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 10:49:54 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <4559E107.9090207@gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<ca471dc20611132021y5840cd4fpd1385c17a578b608@mail.gmail.com>
	<91ad5bf80611132303y33ecee81k8d7b2093d4b6806d@mail.gmail.com>
	<4559E107.9090207@gmail.com>
Message-ID: <91ad5bf80611140749t59fe2067ra5452ac6505ea97b@mail.gmail.com>

On 11/14/06, Nick Coghlan <ncoghlan at gmail.com> wrote:

> George Sakkis wrote:
> > Understood. Any _technical_ reasons then why shouldn't all iterators
> > inherit from it, or "duck typing is The Right Way" should be taken as
> > an axiom ? Why do we somehow _need_ itertools when we don't need
> > sequencetools, mappingtools, etc ? What's so special about the
> > iterator protocol ?
>
> I have sympathy for the idea of syntactic sugar for working with iterators, I
> really do (just read the thread I started about iterator slicing early last
> year [1]).
>
> However, since that time we've moved on to the idea of returning richer view
> objects rather than bare iterators for some of the methods being changed in
> Py3k to return something other than a list (such as dict.keys() and friends).
>
> The underlying realisation behind that change is that an iterable is free to
> implement whichever aspects of the sequence and mapping interfaces it wants
> to, depending on what makes sense for the particular iterator.
>
> By taking this route the basic iterator protocol is kept simple with minimal
> assumptions about the underlying data storage (which is exactly what leads to
> the protocol's wide applicability), while the richer mapping and sequence
> interfaces are supported as appropriate in order to make the objects easier to
> work with.
>
> The difficult with pulling views out into a generic mixin class such as the
> one you propose is that they are typically strongly coupled to their
> associated concrete container class. Arbitrary iterators don't make strong
> enough guarantees to allow that to work without suprises.
>
> For example, it would be somewhat surprising if (a, b, c), (x, y, z) = it[:3],
> it[:3] puts different results into a & x, but that is exactly what happens if
> 'it' is a sliceable iterator like the one you suggest. This didn't bother me
> last year, but I've since grown to dislike it (primarily due to the
> realisation mentioned above that returning view objects where appropriate is
> likely to be a better answer, even if it lacks "one idiom to rule them all"
> status).

Nick,

thank you for the most useful and insightful reply so far. Maybe you
are a year ahead of me because the example you showed doesn't bother
me either, the same way I'm not bothered by, say, infinite iterators
and the risk they introduce when someone writes list(iterator). I'm
also not bothered at all with numpy arrays returning views instead of
copies. Is it really part of the informal sequence protocol that
slicing must return a copy ? That would be new to me.

I haven't read the thread you mention about rich views, I probably
should do that, but from the sound of it, I think I'll like it.

Regards,
George

From george.sakkis at gmail.com  Tue Nov 14 18:55:06 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 12:55:06 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061114090956.82EB.JCARLSON@uci.edu>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
Message-ID: <91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>

On 11/14/06, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "George Sakkis" <george.sakkis at gmail.com> wrote:
> >
> > On 11/14/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > > George Sakkis wrote:
> > > > On 11/14/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > > >
> > > >> BJ?rn Lindqvist wrote:
> > > >>
> > > >>> But why is both the dict and list protocol so fat then? Is it hard to
> > > >>> create your own dict or list-derived types in Python?
> > > >> don't confuse things like lists and dictionaries with things like
> > > >> sequences and mappings.  iterators and iterables belong to the second
> > > >> category.
> > > >
> > > > This doesn't answer my last question: why do we need itertools when we
> > > > can live without sequencetools, mappingtools, fileliketools, etc. ?
> > >
> > > Because the iterator protocol is simple enough to be easily composable - there
> > > are a wide variety of things that can be done with an object when all you know
> > > about it is that it is some form of iterable. The more assumptions you have to
> > > start making about the iterable, the less widely applicable the resulting
> > > operation will be.
> > >
> > > And I think the number one reason we don't see a compelling need for the extra
> > > tool libraries you describe is the fact that sequences, mappings and files are
> > > themselves all iterables.
> > >
> > > That said, there are actually a number of modules for working with sequences
> > > in the standard library, like bisect and heapq. They just aren't lumped into
> > > one place the way itertools and functools are.
> > >
> > > Having a rich method API vs having a narrow method API and duck-typed support
> > > functions is a design trade-off. In the case of sequences and mappings, the
> > > trade-off went towards a richer API because the de facto reference
> > > implementations were the builtin dict and list classes (which is why DictMixin
> > > and ListMixin are so useful when implementing your own containers). In the
> > > case of iterables and iterators, the trade-off went towards the narrow API so
> > > that the interface could be used in a wide variety of situations (lines in a
> > > file, records in a database, characters in a string, bytes from a serial port,
> > > frames in a bowling game, active players in a MMORPG, etc, etc, etc).
> >
> > Given the overly negative reaction to a base iterator type, I withdraw
> > the part of my proposal that suggests this type as the base of all
> > iterators. Instead I propose a builtin Iter (or even better iter, if
> > there is no objection to change iter's current behavior) as an OO
> > replacement of the (functional) itertools API. In other words, instead
> > of:
>
> -1.  There is nothing wrong with a functional approach.  If you dislike
> the verbosity of the names of the operations, and/or the arguments they
> take, you are free to rename them:
>     from itertools import chain as CH
> ... or write wrappers:
>     def IS(it, start):
>         return islice(it, start, None)
>
> You can even have your itertools automatically inserted into
> __builtins__ on startup with a small manipulation of site.py .
>
> Heck, if _you_ want them, _you_ are free to have your Iter object
> inserted into the __builtins__ namespace by site.py .  But due to the
> overwhelming -1s from _everyone_, _including Guido_, your odds of
> getting anything like Iter into mainline Python in the next 18 months
> (when Python 2.6 is expected to be released), is very low.

This is the python-3000 list; I don't aim at Python 2.x. Given that I
propose to *replace* itertools, not add an extra builtin (having them
both would be redundant and violates the one obvious way to do it
principle), it wouldn't be backwards compatible. As for the -1s, so
far most of them seem to oppose the inheritance idea as a worse option
than duck typing; let's see if removing this part changes anything.

As for your other arguments about renaming the itertools functions to
smaller names, let me remind you that in python syntax sugar *does*
matter. If I am not mistaken, the list comprehension syntax will not
be removed in python-3K, although it is completely redundant with
generator comprehensions. Heck, writing list(gencomp) is only 4
characters longer than [gencomp]. My suggested proposal does much
better, both in saving keystrokes and readability.

George

From jimjjewett at gmail.com  Tue Nov 14 19:06:12 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 14 Nov 2006 13:06:12 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
Message-ID: <fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>

On 11/14/06, George Sakkis <george.sakkis at gmail.com> wrote:

> This is the python-3000 list; I don't aim at Python 2.x. Given that I
> propose to *replace* itertools, not add an extra builtin (having them
> both would be redundant and violates the one obvious way to do it
> principle), it wouldn't be backwards compatible. As for the -1s, so
> far most of them seem to oppose the inheritance idea as a worse option
> than duck typing; let's see if removing this part changes anything.

Yes, and at first glance, an IterMixin would be nice.  But only at first glance.

(1)  Things that are already iterators won't implement the entire
newly expanded iteration API.   I don't want people to start
mechanically replacing

    for var in seq

with

    for var in Iter(seq)

just because they can't remember whether or not it matters.

(2)  Some of the functionality in itertools (particularly islice) is
awkward enough that maybe it should be hidden in a library.  That way
people are less likely to stumble on it without seeing the
documentation.

(3)  IterMixin would be the only mixin exposed as a builtin -- the
other mixins must be imported.  The builtin type factories are
typically the stripped down versions.

-jJ

From gsakkis at rutgers.edu  Tue Nov 14 19:06:33 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Tue, 14 Nov 2006 13:06:33 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
Message-ID: <91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>

On 11/14/06, Fredrik Lundh <fredrik at pythonware.com> wrote:

> Uhuh.  And why would this more important than adding, say, Array,
> AtExit, Bz2, Cgi, CgiTB, Cmath, Code, Codecs, Collections, Copy, Csv,
> DateTime, Decimal, ElementTree, ErrNo, Ftplib, Gc, GetOpt, GetText,
> Glob, Gzip, HashLib, HMac, Inspect, Locale, Logging, Marshal, Math,
> Mmap, Operator, Os, OsPath, Pickle, Platform, Random, Re, Select,
> ShUtil, SmtpLib, Socket, Struct, SubProcess, Sys, TempFile, Threading,
> Time, Traceback, UnicodeData, UrlLib, UrlLib2, UrlParse, Warnings,
> WeakRef, XmlDom, XmlSax, ZipFile, or Zlib builtins?  Most of these are
> definitely used a lot more often than itertools in typical Python
> programs...
>
> </F>

I understand you are exaggerating (can't believe you are seriously
claiming that cmath or traceback are more frequently used than
itertools), but if your objection is on adding yet another builtin,
what would be the objection to boosting up the existing iter() to
provide  this extra functionality ? This might even be backwards
compatible (but even if it's not, that's not a main concern for
py-3k).

George

From george.sakkis at gmail.com  Tue Nov 14 19:19:17 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 13:19:17 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
Message-ID: <91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>

On 11/14/06, Jim Jewett <jimjjewett at gmail.com> wrote:

> Yes, and at first glance, an IterMixin would be nice.  But only at first glance.
>
> (1)  Things that are already iterators won't implement the entire
> newly expanded iteration API.   I don't want people to start
> mechanically replacing
>
>     for var in seq
>
> with
>
>     for var in Iter(seq)
>
> just because they can't remember whether or not it matters.

Short answer: it doesn't. Only if you actually use the new API it matters, e.g.

for var in Iter(it1) + Iter(it2):


> (2)  Some of the functionality in itertools (particularly islice) is
> awkward enough that maybe it should be hidden in a library.  That way
> people are less likely to stumble on it without seeing the
> documentation.

You can argue the same for a gazillion other python features. There's
no way to know that list slicing returns a copy and not a view, or
that numpy arrays do the opposite, without reading the docs.

> (3)  IterMixin would be the only mixin exposed as a builtin -- the
> other mixins must be imported.  The builtin type factories are
> typically the stripped down versions.

That problem is solved if we boost up the existing iter() builtin.

George

From fdrake at acm.org  Tue Nov 14 19:53:18 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 14 Nov 2006 13:53:18 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
Message-ID: <200611141353.18690.fdrake@acm.org>

On Tuesday 14 November 2006 13:06, George Sakkis wrote:
 > I understand you are exaggerating (can't believe you are seriously
 > claiming that cmath or traceback are more frequently used than
 > itertools), 

I certainly use traceback far more than itertools.  I use traceback 
occaissionally, but I've never actually had reason to use itertools at all.

 > but if your objection is on adding yet another builtin, 
 > what would be the objection to boosting up the existing iter() to
 > provide  this extra functionality ? This might even be backwards
 > compatible (but even if it's not, that's not a main concern for
 > py-3k).

The real issue seems to be that there's no benefit.  Iterators are nice 
because they're composable; that doesn't make the compositions part of the 
iterator, though.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From george.sakkis at gmail.com  Tue Nov 14 20:32:22 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 14:32:22 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <200611141353.18690.fdrake@acm.org>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
	<200611141353.18690.fdrake@acm.org>
Message-ID: <91ad5bf80611141132v3cac4035nfa92b52f394ebd1a@mail.gmail.com>

On 11/14/06, Fred L. Drake, Jr. <fdrake at acm.org> wrote:
> On Tuesday 14 November 2006 13:06, George Sakkis wrote:
>  > I understand you are exaggerating (can't believe you are seriously
>  > claiming that cmath or traceback are more frequently used than
>  > itertools),
>
> I certainly use traceback far more than itertools.  I use traceback
> occaissionally, but I've never actually had reason to use itertools at all.

I believe you, but I doubt your usage patterns are close to the
average python user. I am sure cmath is invaluable to some people too.
There's also a chicken and egg problem; the reason itertools are not
used as often as they could is exactly the overheard of importing a
module and use a verbose function, rather than having them for free as
methods of a builtin object. If instead of

    for k,v in some_dict.iteritems():

I had to do something like

    from dictutils import iteritems
    for k,v in iteritems(some_dict):

I'd probably woudn't bother and just use some_dict.items().

>  > but if your objection is on adding yet another builtin,
>  > what would be the objection to boosting up the existing iter() to
>  > provide  this extra functionality ? This might even be backwards
>  > compatible (but even if it's not, that's not a main concern for
>  > py-3k).
>
> The real issue seems to be that there's no benefit.  Iterators are nice
> because they're composable; that doesn't make the compositions part of the
> iterator, though.

Seems like a classic case for OOP to me: combine state (iterables)
with behavior (iter) in handly little packets (objects).

George

From gsakkis at rutgers.edu  Tue Nov 14 20:44:06 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Tue, 14 Nov 2006 14:44:06 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
Message-ID: <91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>

On 11/14/06, George Sakkis <george.sakkis at gmail.com> wrote:

> On 11/14/06, Jim Jewett <jimjjewett at gmail.com> wrote:

> > (3)  IterMixin would be the only mixin exposed as a builtin -- the
> > other mixins must be imported.  The builtin type factories are
> > typically the stripped down versions.
>
> That problem is solved if we boost up the existing iter() builtin.
>
> George

And for those objecting to touching the existing iter() or bloating
the builtin namespace with yet another builtin, let me mention that we
can get rid of *two* existing functions which for some reason were
promoted to builtin status, although they conceptually belong in
itertools: enumerate and reversed.

George

From guido at python.org  Tue Nov 14 20:56:41 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2006 11:56:41 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
Message-ID: <ca471dc20611141156t27291f5fsd23d21b909c82d0f@mail.gmail.com>

We can all save a lot of time by not continuing this conversation.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From sluggoster at gmail.com  Tue Nov 14 22:38:54 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 14 Nov 2006 13:38:54 -0800
Subject: [Python-3000] Python-3000 Digest, Vol 9, Issue 27
In-Reply-To: <4559E86F.4010805@gmail.com>
References: <mailman.71.1163502028.12842.python-3000@python.org>
	<91ad5bf80611140729o55570666gb8e07c761487c0ce@mail.gmail.com>
	<4559E86F.4010805@gmail.com>
Message-ID: <6e9196d20611141338s706a0daao77e5833a22884716@mail.gmail.com>

On 11/14/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Having a rich method API vs having a narrow method API and duck-typed support
> functions is a design trade-off. In the case of sequences and mappings, the
> trade-off went towards a richer API because the de facto reference
> implementations were the builtin dict and list classes (which is why DictMixin
> and ListMixin are so useful when implementing your own containers). In the
> case of iterables and iterators, the trade-off went towards the narrow API so
> that the interface could be used in a wide variety of situations (lines in a
> file, records in a database, characters in a string, bytes from a serial port,
> frames in a bowling game, active players in a MMORPG, etc, etc, etc).

To augment Nick's excellent explanation, note that you can't restart
an iterator or go back ("unyield" a value).  There was a conscious
decision to limit the iterator interface to make it as easy as
possible for some unknown future object to implement an iterator.
Now, anything with a compliant .next() method is an iterator.  Nothing
prevents a particular iterator from having reset or backstepping
methods, but these are not required for all iterators.  I sense this
is the same reason people are opposed to your proposal.  I'll not
comment on the merits of having iter() return a rich Iterator object
because I don't understand well enough what we might lose.  The real
problem is that iterator is an interface, and there's no formal way to
express interfaces in Python; it's all in the documentation.  That's
why the relationship between dict and "mapping type" seems so
nebulous, and also why an iterator looks like the same kind of object
as a dict or list but it isn't.

The tradeoff between rich objects + methods and minimal objects +
functions is pervasive in Python.  len(x) is a function because it
applies to a wide variety of objects -- not all of them known yet --
rather than to a certain class hierarchy.  One major success of the
iterator protocol was when file objects became iterable.  No more
"while loop with break in the middle": now you can just do a simple
"for line in file".  If the file iterator were required to support "+"
and your other rich methods, it may be difficult to implement.  What
if the file is tied to a socket-like stream instead of a disk file?

When you need a mapping, the obvious answer is, "Use a dict!"  A dict
is an empty container with infrastructure for adding and managing
items.  When a file object needs to iterate likes in a file, it can't
"use an iterator", it has to *be* an iterator.    A generic iterator
object doesn't know *how* to iterate over a file; that's what your
class has to do.  But in order to do that it has to implement the
iterator interface... and we're right back where we started.

A "listtools" package is not a bad idea, actually.  A place to gather
list-like objects and their functions so they aren't scattered
throughout the library.  "collections" almost does it, although it's
defined a bit too narrowly for function libraries (but that could be
changed).  But this is a minor issue.

The thing with "mappingtools" is, what would such a package contain?
I've sometimes wondered why there's only one mapping type after so
many years.  But the real question is, what kind of "mapping type"
functionality is there that a dict doesn't provide?  None that I can
think of.  "ordered dict" and "default dict" can be handled by
subclasses.

The problem with .__getitem__ is, you can't tell whether an object is
a sequence or a mapping.  If it has .__getitem__, it's one or the
other, but you don't know whether it accepts sequential integers
starting from 0, or arbitrary keys.  This is not really .__getitem__'s
fault since there's only one [] operator to access both.  It's the
lack of interfaces again: the lack of a universal way to say "this is
a sequence type".  All the routine can do is document what it expects
in arguments, and hope that the caller heeds it.

As for itertools, I rarely use it.  If so it's one or two functions at
a time.  And often I'm not actually using the functions, just studying
the implementation so I can write a custom function that does what I
need.  I suppose it would be nice to chain iterators with "+", but I
chain iterators so rarely it's no big deal.  I suspect many other
programmers are the same way.  What I'd most like to see in itertools
are the functions on the "Recipes" page.  Why should everyone have to
paste the code for no/quantify/flatten rather than having one central
function for them?  But I expect this will happen one one by one as
certain recipes get popular.

-- 
Mike Orr <sluggoster at gmail.com>

From mike.klaas at gmail.com  Tue Nov 14 22:59:36 2006
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 14 Nov 2006 13:59:36 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
Message-ID: <3d2ce8cb0611141359s278c6d4cjd01e1aa6759e6c4@mail.gmail.com>

On 11/14/06, George Sakkis <gsakkis at rutgers.edu> wrote:

> And for those objecting to touching the existing iter() or bloating
> the builtin namespace with yet another builtin, let me mention that we
> can get rid of *two* existing functions which for some reason were
> promoted to builtin status, although they conceptually belong in
> itertools: enumerate and reversed.

Well, "conceptually" all itertools are not equal.  enumerate and
reversed (and let's not forget xrange) are among my most-used
"itertools"--enumerate in particular is used is virtually every
module.  Some tools in itertools also belong to that category--izip in
particular is used heavily in my code.  Some of the heavily-used
itertools are already being pseudo-promoted to builtins: zip, map,
(and maybe? filter) are turning into izip, imap, and ifilter.

I will admit that itertools are among the few modules that are
"first-class" import citizens in my code, that is, I tend to import
the symbols directly:

from itertools import groupby, chain

Another such module is collections.  It is possible that once I'm more
familiar with functools and contextlib that they will occupy this
hallowed place.

I don't see the problem of importing important language functionality.
 Most languages since c have required somthing similar.

-Mike

From sluggoster at gmail.com  Tue Nov 14 23:11:48 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 14 Nov 2006 14:11:48 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <002801c70801$fd33d310$6402a8c0@arkdesktop>
References: <002801c70801$fd33d310$6402a8c0@arkdesktop>
Message-ID: <6e9196d20611141411x57a4d3f0tdbc81a18b9590889@mail.gmail.com>

On 11/14/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > Duck typing is a seriously bad idea
>
> Why?

And more importantly, are we all talking about the same thing when we
say "duck typing"?  Duck typing as I know it means accepting an
argument that exhibits certain behaviors rather than being a certain
type.  It's widely considered to be one of Python's main features
because it's flexible for unforeseen circumstances. Sometimes a better
implementation is found that can't be expressed as a subclass.
Sometimes an object can't be one class because it has to be another,
yet it can still "behave like" the other object.

-- 
Mike Orr <sluggoster at gmail.com>

From george.sakkis at gmail.com  Tue Nov 14 23:28:36 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 17:28:36 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <3d2ce8cb0611141359s278c6d4cjd01e1aa6759e6c4@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
	<3d2ce8cb0611141359s278c6d4cjd01e1aa6759e6c4@mail.gmail.com>
Message-ID: <91ad5bf80611141428r6f9313edq48c95acacf174059@mail.gmail.com>

On 11/14/06, Mike Klaas <mike.klaas at gmail.com> wrote:

> On 11/14/06, George Sakkis <gsakkis at rutgers.edu> wrote:
>
> > And for those objecting to touching the existing iter() or bloating
> > the builtin namespace with yet another builtin, let me mention that we
> > can get rid of *two* existing functions which for some reason were
> > promoted to builtin status, although they conceptually belong in
> > itertools: enumerate and reversed.
>
> Well, "conceptually" all itertools are not equal.  enumerate and
> reversed (and let's not forget xrange) are among my most-used
> "itertools"--enumerate in particular is used is virtually every
> module.  Some tools in itertools also belong to that category--izip in
> particular is used heavily in my code.  Some of the heavily-used
> itertools are already being pseudo-promoted to builtins: zip, map,
> (and maybe? filter) are turning into izip, imap, and ifilter.
>
> I will admit that itertools are among the few modules that are
> "first-class" import citizens in my code, that is, I tend to import
> the symbols directly:
>
> from itertools import groupby, chain

I do the same, and that's exactly my point. Iterators, generators,
generator comprehensions are among the core python features. Their
status is promoted steadily in every new version; the latest 2.5 is no
exception. Itertools is not yet another obscure module; it includes
functions that provide fundamental operations that apply on any
iterable.

> I don't see the problem of importing important language functionality.
>  Most languages since c have required somthing similar.
>
> -Mike

Most languages since C are not as readable and elegant as Python. I
think we all agree that list1[5:] + list2 is much better than the
hypothetical
    list1.getslice(5,None).concatenate(list2)
or
    from sequenceutils import concatenate, getslice
    concatenate(getslice(list1,5,None), list2)

Unfortunately, people here seem to consider natural the latter when it
comes to itertools. Oh well, whatever.

George

From mike.klaas at gmail.com  Tue Nov 14 23:44:07 2006
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 14 Nov 2006 14:44:07 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141428r6f9313edq48c95acacf174059@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
	<3d2ce8cb0611141359s278c6d4cjd01e1aa6759e6c4@mail.gmail.com>
	<91ad5bf80611141428r6f9313edq48c95acacf174059@mail.gmail.com>
Message-ID: <3d2ce8cb0611141444oe9284a9w74574e3d12e8c073@mail.gmail.com>

On 11/14/06, George Sakkis <george.sakkis at gmail.com> wrote:
> On 11/14/06, Mike Klaas <mike.klaas at gmail.com> wrote:

> > I don't see the problem of importing important language functionality.
> >  Most languages since c have required somthing similar.
>
> Most languages since C are not as readable and elegant as Python. I
> think we all agree that list1[5:] + list2 is much better than the
> hypothetical
>     list1.getslice(5,None).concatenate(list2)
> or
>     from sequenceutils import concatenate, getslice
>     concatenate(getslice(list1,5,None), list2)
>
> Unfortunately, people here seem to consider natural the latter when it
> comes to itertools. Oh well, whatever.

Two differences:
   - slicing an iterator is a rare activity; slicing a list is common
   - I often use .extend rather than += when appending iterables to lists

Finally, your analogy between sequences and iterators is flawed.  +
does not work for sequences, but only for concrete types:

>>> [8] + (9,)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only concatenate list (not "tuple") to list

In python, only concrete types tend to implement syntatic operator
support (another example: sets only support +, -, |, & with other
sets, but their methods accept iterables).

-Mike

From george.sakkis at gmail.com  Wed Nov 15 01:23:24 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Tue, 14 Nov 2006 19:23:24 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <3d2ce8cb0611141444oe9284a9w74574e3d12e8c073@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
	<3d2ce8cb0611141359s278c6d4cjd01e1aa6759e6c4@mail.gmail.com>
	<91ad5bf80611141428r6f9313edq48c95acacf174059@mail.gmail.com>
	<3d2ce8cb0611141444oe9284a9w74574e3d12e8c073@mail.gmail.com>
Message-ID: <91ad5bf80611141623r6771b14bg5ea55efceb36e820@mail.gmail.com>

On 11/14/06, Mike Klaas <mike.klaas at gmail.com> wrote:

> On 11/14/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > On 11/14/06, Mike Klaas <mike.klaas at gmail.com> wrote:
>
> > > I don't see the problem of importing important language functionality.
> > >  Most languages since c have required somthing similar.
> >
> > Most languages since C are not as readable and elegant as Python. I
> > think we all agree that list1[5:] + list2 is much better than the
> > hypothetical
> >     list1.getslice(5,None).concatenate(list2)
> > or
> >     from sequenceutils import concatenate, getslice
> >     concatenate(getslice(list1,5,None), list2)
> >
> > Unfortunately, people here seem to consider natural the latter when it
> > comes to itertools. Oh well, whatever.
>
> Two differences:
>    - slicing an iterator is a rare activity; slicing a list is common

Slicing an infinite iterator is *very* common, to which you'll
probably reply that infinite iterators are rare. I guess neither of us
has numbers to support his claim, so let's leave it at that.

>    - I often use .extend rather than += when appending iterables to lists
>
> Finally, your analogy between sequences and iterators is flawed.  +
> does not work for sequences, but only for concrete types:
>
> >>> [8] + (9,)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: can only concatenate list (not "tuple") to list
>
> In python, only concrete types tend to implement syntatic operator
> support (another example: sets only support +, -, |, & with other
> sets, but their methods accept iterables).
>
> -Mike
>

The TypeError is not raised because the arguments are not of a
concrete type (all python types are concrete; interfaces or protocols
are informal descriptions) but because they're not the *same* type,
and I don't object to this. This is not the case in what I propose: in
the expression Iter(file('foo')) + Iter(some_string), both operands
have the same type, the Iter wrapper that wraps an underlying
iterator.

George

From greg.ewing at canterbury.ac.nz  Wed Nov 15 03:34:06 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Nov 2006 15:34:06 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov13.145133pst.58648@synergy1.parc.xerox.com>
References: <06Nov13.145133pst.58648@synergy1.parc.xerox.com>
Message-ID: <455A7C9E.2050904@canterbury.ac.nz>

Bill Janssen wrote:

> Duck typing is a seriously bad idea, forced on Python by the now
> obsolete split between built-in types and user-defined types.

Non-duck typing is a seriously bad idea, forced
on some other languages by static typing. Python
is mercifully free of such constraints.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 15 03:34:57 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Nov 2006 15:34:57 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
Message-ID: <455A7CD1.5090503@canterbury.ac.nz>

George Sakkis wrote:

> I think I do, though I can't tell the same about the reasons of your
> objections to it.

Perhaps I can fill in some of the things that Guido
is not explicitly saying.

You've shown that it *can* be done, but you haven't
provided a compelling reason why it *should* be done.

The main benefit seems to be that itertools operations
could then be done using method calls instead of
function calls. But that's not automatically better
than what we have now. Python does not follow the
"everything must be a method of something" ideology
found in some other languages such as Java and Ruby.

The itertools functions have generic implementations
that work with any iterator. They don't need access
to the iterator's internals, or need to be dynamically
dispatched for any reason, so there is no obvious
benefit to making them methods instead of stand-alone
functions.

The one technical benefit appears to be the ability
to use operators instead of named functions. But
considering how infrequently itertools operations
are used, having to spell them out doesn't seem
like a great hardship, so this argument is weak.
All other things being equal, it might hold sway,
but other things are not equal.

You claim that duck typing would not be inhibited.
While technically this is true, in practice it would
mean that any iterator that didn't inherit from this
base class would be a second-class citizen, unable
to be used by code that expected it to have all the
"standard" itertools methods.

Finally, consider that there are infinitely many
possible functions of the kind that are found in
the itertools module. As stand-alone functions,
they all have equal status -- new ones can be added
and used in the same way by any code on any
iterator. Some good arguments would have to be
put forward as to why some particular ones should
be picked out and elevated to the status of methods.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 15 03:35:13 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Nov 2006 15:35:13 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141132v3cac4035nfa92b52f394ebd1a@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
	<200611141353.18690.fdrake@acm.org>
	<91ad5bf80611141132v3cac4035nfa92b52f394ebd1a@mail.gmail.com>
Message-ID: <455A7CE1.4020005@canterbury.ac.nz>

George Sakkis wrote:

> Seems like a classic case for OOP to me: combine state (iterables)
> with behavior (iter) in handly little packets (objects).

Except that it's not really behaviour. Chaining, e.g., is
something you do *to* iterators, not something that the
iterators themselves do.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 15 03:35:21 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Nov 2006 15:35:21 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
Message-ID: <455A7CE9.5030105@canterbury.ac.nz>

George Sakkis wrote:

> Short answer: it doesn't. Only if you actually use the new API it matters, e.g.
> 
> for var in Iter(it1) + Iter(it2):

Which doesn't look any more readable to me than

   for var in chain(it1, it2):

In fact, it looks considerably *less* readable.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 15 03:35:26 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Nov 2006 15:35:26 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <740c3aec0611140121t6fac8dfcm4a58f3c56ae9daef@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>
	<740c3aec0611140121t6fac8dfcm4a58f3c56ae9daef@mail.gmail.com>
Message-ID: <455A7CEE.9000701@canterbury.ac.nz>

BJ?rn Lindqvist wrote:

> But why is both the dict and list protocol so fat then?

Because there are a variety of things you need to
be able to do to sequences and mappings, and the
implementations of them are closely tied to the
internals of the type concerned.

If there were a substantial number of commonly
used operations on them that could be implemented
efficiently in a generic way, then we probably
would have listtools and dicttools modules
(or more properly sequencetools and mappingtools).
That doesn't seem to have happened, though.

--
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 15 03:35:33 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Nov 2006 15:35:33 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
Message-ID: <455A7CF5.6000409@canterbury.ac.nz>

George Sakkis wrote:

> I understand you are exaggerating (can't believe you are seriously
> claiming that cmath or traceback are more frequently used than
> itertools)

I don't think he's exaggerating much -- I'm sure
many of those modules are used a lot more than
itertools. I don't know about generally, but
personally I *do* use traceback more often than
itertools. And I can easily see someone who's
into number crunching using cmath a *lot* more
than itertools.

--
Greg

From janssen at parc.com  Wed Nov 15 04:46:39 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 14 Nov 2006 19:46:39 PST
Subject: [Python-3000] Python-3000 Digest, Vol 9, Issue 27
In-Reply-To: Your message of "Tue, 14 Nov 2006 13:38:54 PST."
	<6e9196d20611141338s706a0daao77e5833a22884716@mail.gmail.com> 
Message-ID: <06Nov14.194639pst."58648"@synergy1.parc.xerox.com>

> The real
> problem is that iterator is an interface, and there's no formal way to
> express interfaces in Python; it's all in the documentation.
> ...
> The problem with .__getitem__ is, you can't tell whether an object is
> a sequence or a mapping.  If it has .__getitem__, it's one or the
> other, but you don't know whether it accepts sequential integers
> starting from 0, or arbitrary keys.

Actually, Mike, there is a formal way to express interfaces in Python.

We just don't use it.

The mechanism is to define all interfaces clearly as base types, and
use the type mechanism to indicate "typeness" instead of the pervasive
use of duck-typing.  Then, to indicate that a type "implements" the
"foo" interface, just inherit from "foo".

But wait, there's more!  Since Python usefully supports multiple
inheritance in way that Java doesn't, Python "interface" classes can
actually provide useful generic implementations of functionality,
where possible, or raise NotImplemented where not possible.

And that's not all!  You could then see if a value implements a
particular interface with isinstance(), instead of having to check for
the presence of "__getitem__", and wondering if the implementor's
"__getitem__" is the same "__getitem__" you were thinking of.
Suddenly you can do non-fragile reflective programming.

Bill

From janssen at parc.com  Wed Nov 15 04:53:36 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 14 Nov 2006 19:53:36 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: Your message of "Tue, 14 Nov 2006 18:34:06 PST."
	<455A7C9E.2050904@canterbury.ac.nz> 
Message-ID: <06Nov14.195344pst."58648"@synergy1.parc.xerox.com>

Greg Ewing wrote:
> Bill Janssen wrote:
> 
> > Duck typing is a seriously bad idea, forced on Python by the now
> > obsolete split between built-in types and user-defined types.
> 
> Non-duck typing is a seriously bad idea, forced
> on some other languages by static typing. Python
> is mercifully free of such constraints.

I'm thinking that Mike Orr's question, "are we all talking about the
same duck-typing", makes sense.  Greg, I'm not suggesting static
typing -- I much prefer dynamic strong typing.  But what Python has
now is dynamic weak typing, which makes programs (particularly
frameworks) fragile.  And it's mainly due to the historical accident
of not being able to inherit from C-based types in Python 1.x.

I'd like to be able to look at a value and determine which interfaces
I can safely invoke on it, even if I don't understand its full type.
I can't (safely) do that by string-matching method names.

Bill

From guido at python.org  Wed Nov 15 07:18:20 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 14 Nov 2006 22:18:20 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <-7764899780500803165@unknownmsgid>
References: <455A7C9E.2050904@canterbury.ac.nz>
	<-7764899780500803165@unknownmsgid>
Message-ID: <ca471dc20611142218n3923b4b2k77e09ca0cb7540ff@mail.gmail.com>

On 11/14/06, Bill Janssen <janssen at parc.com> wrote:
> Greg Ewing wrote:
> > Bill Janssen wrote:
> >
> > > Duck typing is a seriously bad idea, forced on Python by the now
> > > obsolete split between built-in types and user-defined types.
> >
> > Non-duck typing is a seriously bad idea, forced
> > on some other languages by static typing. Python
> > is mercifully free of such constraints.
>
> I'm thinking that Mike Orr's question, "are we all talking about the
> same duck-typing", makes sense.  Greg, I'm not suggesting static
> typing -- I much prefer dynamic strong typing.  But what Python has
> now is dynamic weak typing, which makes programs (particularly
> frameworks) fragile.  And it's mainly due to the historical accident
> of not being able to inherit from C-based types in Python 1.x.
>
> I'd like to be able to look at a value and determine which interfaces
> I can safely invoke on it, even if I don't understand its full type.
> I can't (safely) do that by string-matching method names.

I can't completely disagree with everything you say (introspection of
interfaces makes sense, and this is proven by that several frameworks
have implemented an explicit notion of interfaces).

But I object to your claim that it was invented because of the
difference between C and python types/classes in Python versions <=
2.1. The real reason was (and is) that there are no type declarations
in Python, so there is absolutely no reason why you *couldn't* pass
anything with an appropriate append() method to the following
function:

 def foo(x):
  x.append(42)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From george.sakkis at gmail.com  Wed Nov 15 07:33:53 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 01:33:53 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455A7CD1.5090503@canterbury.ac.nz>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
Message-ID: <91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>

On 11/14/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> George Sakkis wrote:
>
> > I think I do, though I can't tell the same about the reasons of your
> > objections to it.
>
> Perhaps I can fill in some of the things that Guido
> is not explicitly saying.

Indeed, another high quality reply. Thanks for sharing.

> You've shown that it *can* be done, but you haven't
> provided a compelling reason why it *should* be done.
>
> The main benefit seems to be that itertools operations
> could then be done using method calls instead of
> function calls. But that's not automatically better
> than what we have now. Python does not follow the
> "everything must be a method of something" ideology
> found in some other languages such as Java and Ruby.

I won't open another can of worms here, but I'll just say that as much
as I hate Java's stubborn insistence on OO purity, I am equally
disturbed by Python's arbitrary-looking choices on whether some
callable ends up a function or a method. When I talk about or
introduce Python to outsiders, one of the toughest questions is often
something along the lines of "why len() is a function and not a
method?"

> The itertools functions have generic implementations
> that work with any iterator. They don't need access
> to the iterator's internals, or need to be dynamically
> dispatched for any reason, so there is no obvious
> benefit to making them methods instead of stand-alone
> functions.

Did you look at my implementation ? The only attribute is a reference
to the underlying iterator. No access to its internal is needed.

> The one technical benefit appears to be the ability
> to use operators instead of named functions. But
> considering how infrequently itertools operations
> are used, having to spell them out doesn't seem
> like a great hardship, so this argument is weak.
> All other things being equal, it might hold sway,
> but other things are not equal.

This seems to be the most central, and at the same time most
unexpected for me, point of resistance. Given the prominence of
iterators, generators, generator comprehensions and generic iterables
in Python, I've been considering itertools the module that makes all
these "talk" to each other seamlessly and transparently. I am totally
amazed that I have itertools in higher esteem than many/most core
Python developers.

> You claim that duck typing would not be inhibited.
> While technically this is true, in practice it would
> mean that any iterator that didn't inherit from this
> base class would be a second-class citizen, unable
> to be used by code that expected it to have all the
> "standard" itertools methods.

The promotion from second class to first would require 6 extra
characters: Iter(x) (or iter(x) if it's ok to augment the existing
iter()). Just spelling "itertools" takes more than that, let alone
importing a function or two. The runtime cost would be negligible too
since Iter is a thinnest wrapper around x (or it could be x itself if
it was already "first-class").

> Finally, consider that there are infinitely many
> possible functions of the kind that are found in
> the itertools module. As stand-alone functions,
> they all have equal status -- new ones can be added
> and used in the same way by any code on any
> iterator. Some good arguments would have to be
> put forward as to why some particular ones should
> be picked out and elevated to the status of methods.

I'm not sure I understand your point here. If something was deemed
useful enough to make it to itertools, it would be a candidate for
being a method to a base Iter type. I don't have any strong feelings
on the exact set of methods that I want to see included apart from the
most basic ones (chain as +, islice as [], izip as zip and enumerate -
yes, I count enumerate as an iter tool). Besides, types, like modules,
are not carved in stone. It's not as if no builtin type or function in
Python has ever grown (in a backwards compatible way) more methods,
extra named arguments, etc. I guess the good arguments to be put
forward for every candidate method would be similar to the ones that
elevated enumerate() and reversed() to builtin status.

George

From fredrik at pythonware.com  Wed Nov 15 08:55:27 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 08:55:27 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455A7CEE.9000701@canterbury.ac.nz>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<8E8031E5-BCE9-4FBF-BA9D-487AA3316E31@mac.com>	<740c3aec0611140121t6fac8dfcm4a58f3c56ae9daef@mail.gmail.com>
	<455A7CEE.9000701@canterbury.ac.nz>
Message-ID: <ejeh5e$7ag$1@sea.gmane.org>

Greg Ewing wrote:

> If there were a substantial number of commonly
> used operations on them that could be implemented
> efficiently in a generic way, then we probably
> would have listtools and dicttools modules
> (or more properly sequencetools and mappingtools).
> That doesn't seem to have happened, though.

oh, of course it has happened.  Python's full of sequence tools, and 
there are plenty of generic operations that can work on arbitrary mappings.

and the *very* *first* *statement* a Python newcomer learns is a generic 
operation on a file-like object.

</F>


From sluggoster at gmail.com  Wed Nov 15 08:56:13 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 14 Nov 2006 23:56:13 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
Message-ID: <6e9196d20611142356m72642bc6qd2821457617285d8@mail.gmail.com>

On 11/14/06, George Sakkis <george.sakkis at gmail.com> wrote:
> When I talk about or
> introduce Python to outsiders, one of the toughest questions is often
> something along the lines of "why len() is a function and not a
> method?"

I came to Python from Java and originally thought this. The answer I
received was as I described: methods are for a certain class
hierarchy, while functions are for an unknown number of potential
types that exhibit a certain behavior.  So you can call len() on both
lists and strings.  Here Java has an inconsistency: array.length vs
String.length().  This in itself illustrates my point.  Different
types necessarily have different implementations.  But if the left
hand isn't watching closely what the right hand is doing, you end up
with cross-class inconsistencies and incompatibilities.  len() here is
a unifier: it says "thou shalt do this, .__len__ method, or users will
be pissed that len() doesn't work on your type".

Python made mistakes along the way in terms of method/function
distribution, but these are gradually being corrected.  The most
notorious were string functions.  Who wants to call string.find(s,
sub) or string.split(s)?  But these have long been replaced by
methods.  My work on a Path object is along the same lines.

The #1 complaint I hear about Python from non-users is indentation,
not too many functions.  People generally praise Python's clean method
interface in its file objects, list/str/dict objects, and its stdlib
namespaces.  These aren't perfect but they're better than many
languages.  I was very pleased that typecasting in Python is as
straightforward as int() and str(), and calling a function
instantiates it.  Much more straightforward than " Foo x = new Foo()".

> I am totally
> amazed that I have itertools in higher esteem than many/most core
> Python developers.

I esteem itertools, I just don't have much need for those operations.
I'm glad it's there because it has better (less memory intensive)
implementations than I'd likely make on my own, and it shows ways to
use iterators that I'd never considered.

> > Finally, consider that there are infinitely many
> > possible functions of the kind that are found in
> > the itertools module. As stand-alone functions,
> > they all have equal status -- new ones can be added
> > and used in the same way by any code on any
> > iterator. Some good arguments would have to be
> > put forward as to why some particular ones should
> > be picked out and elevated to the status of methods.
>
> I'm not sure I understand your point here. If something was deemed
> useful enough to make it to itertools, it would be a candidate for
> being a method to a base Iter type.

That's a good point, the overhead of adding a method is not really
worse than adding a function.  I wouldn't object to an iter() function
that returns a SuperIterator, able to slice tall iterables in a single
bound, er, cut.  But I'm not a core developer so I don't understand
all the ramifications it might have. That's a point worth pressing
though: what's the harm in having iter() return an object that's
"sma-a-a-rter than your average bear!", er, than your average
iterator.

> I guess the good arguments to be put
> forward for every candidate method would be similar to the ones that
> elevated enumerate() and reversed() to builtin status.

Actually, it's easier to add a method or package function than to add
a builtin.  Every additional builtin makes it harder to keep all the
builtins in your head, and runs the risk of colliding with a variable
in existing code.  And as Cheetah discovered, it's not safe to
autogenerate Python code based on what the builtins are, because a
future version of Python will add ones you can't predict.  enumerate()
and zip() were added because they solve extremely widespread problems.
 I can't say I like sorted() and reversed(), but they're there because
somebody thought they'd be very widely used.

-- 
Mike Orr <sluggoster at gmail.com>

From fredrik at pythonware.com  Wed Nov 15 08:57:22 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 08:57:22 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
Message-ID: <ejeh91$7ag$2@sea.gmane.org>

George Sakkis wrote:

> I won't open another can of worms here, but I'll just say that as much
> as I hate Java's stubborn insistence on OO purity, I am equally
> disturbed by Python's arbitrary-looking choices on whether some
> callable ends up a function or a method. When I talk about or
> introduce Python to outsiders, one of the toughest questions is often
> something along the lines of "why len() is a function and not a
> method?"

I guess those outsiders don't really understand the concept of a 
"generic operation".  are you sure *you* understand what that is?
it's not obvious from your posts.

</F>


From fredrik at pythonware.com  Wed Nov 15 09:02:22 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 09:02:22 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <6e9196d20611142356m72642bc6qd2821457617285d8@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<455A7CD1.5090503@canterbury.ac.nz>	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<6e9196d20611142356m72642bc6qd2821457617285d8@mail.gmail.com>
Message-ID: <ejehie$7ag$3@sea.gmane.org>

Mike Orr wrote:

 > len() here is a unifier: it says "thou shalt do this, .__len__ method,
 > or users will be pissed that len() doesn't work on your type".

that's an excellent observation.  mind if I borrow it for the related 
FAQ entry?

     http://tinyurl.com/y6vavp

</F>


From george.sakkis at gmail.com  Wed Nov 15 09:47:22 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 03:47:22 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
Message-ID: <91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>

Fredrik Lundh <fredrik at pythonware.com> wrote:

> George Sakkis wrote:
>
> > I won't open another can of worms here, but I'll just say that as much
> > as I hate Java's stubborn insistence on OO purity, I am equally
> > disturbed by Python's arbitrary-looking choices on whether some
> > callable ends up a function or a method. When I talk about or
> > introduce Python to outsiders, one of the toughest questions is often
> > > something along the lines of "why len() is a function and not a
> > method?"
>
> I guess those outsiders don't really understand the concept of a
> "generic operation".  are you sure *you* understand what that is?
> it's not obvious from your your posts.

Perhaps I don't, but if you or anyone else wants to enlighten me, please make
sure you include in your explanation

1) why having a "generic operation" len() that ends up looking for an
ugly special *method* called  __len__() makes sense, while calling
directly a method len() doesn't, and

2) if "generic operations" are such a cool idea, why there aren't more
of them, such as count() or index() (the former for any container
type, the latter for sequence types).

I don't think that defending a decision by pointing out a different bad decision
in another language (Java's inconsistency between string.length and
list.length())
is a particularly compelling argument, especially in an FAQ page. The ending
sentences though are more convincing: "...but it's a part of Python,
and it's too
late to make such fundamental changes now. The functions have to remain to avoid
massive code breakage". That I can buy (at least for 2.x).

George

From fredrik at pythonware.com  Wed Nov 15 10:12:28 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 10:12:28 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<455A7CD1.5090503@canterbury.ac.nz>	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
Message-ID: <ejells$m2u$1@sea.gmane.org>

George Sakkis wrote:

> 1) why having a "generic operation" len() that ends up looking for an
> ugly special *method* called  __len__() makes sense, while calling
> directly a method len() doesn't

for the very reason Mike explained: keeping implementation interfaces 
separate from public interfaces has allowed Python to avoid a certain 
category of design fragmentation that other languages suffer from.

> 2) if "generic operations" are such a cool idea, why there aren't more
> of them, such as count() or index() (the former for any container
> type, the latter for sequence types).

count() isn't that common, really (and can be written as sum(1 for item 
in iterable if condition) if you really want), and things like "in" and 
"any" and "enumerate" are already built-ins.

> I don't think that defending a decision by pointing out a different
 > bad decision in another language (Java's inconsistency between
 > string.length and > list.length()) > is a particularly compelling
 > argument

see above.

> The ending sentences though are more convincing: "...but it's a part
 > of Python, and it's too late to make such fundamental changes now.
 > The functions have to remain to avoid massive code breakage". That
 > I can buy (at least for 2.x).

yeah, it's clear that most of your argumentation is based on a rather 
common "I haven't thought this through very deeply, but I'm sure I'm 
smarter than those guys so that won't stop me" approach.  trust me, the 
design of Python is a *lot* more carefully put together than you appear 
to think.

</F>


From george.sakkis at gmail.com  Wed Nov 15 10:40:12 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 04:40:12 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
Message-ID: <91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>

Fredrik Lundh <fredrik at pythonware.com> wrote:

> George Sakkis wrote:
>
> > 1) why having a "generic operation" len() that ends up looking for an
> > ugly special *method* called  __len__() makes sense, while calling
> > directly a method len() doesn't
>
> for the very reason Mike explained: keeping implementation interfaces
> separate from public interfaces has allowed Python to avoid a certain
> category of design fragmentation that other languages suffer from.

Fredrik, I am not arguing for the argument's sake, I just don't get
it: Python requires my class to define __len__, otherwise len() fails.
Why not require len() as a method instead and forget about __len__ ?
Does len() (the function) do anything smarter behind the scenes than
just passing the ball to __len__ ? That could justify its role but
AFAIK it doesn't.

George

From solipsis at pitrou.net  Wed Nov 15 11:10:44 2006
From: solipsis at pitrou.net (Antoine)
Date: Wed, 15 Nov 2006 11:10:44 +0100 (CET)
Subject: [Python-3000] duck typing
In-Reply-To: <455A7C9E.2050904@canterbury.ac.nz>
References: <06Nov13.145133pst.58648@synergy1.parc.xerox.com>
	<455A7C9E.2050904@canterbury.ac.nz>
Message-ID: <49981.62.39.9.251.1163585444.squirrel@webmail.nerim.net>


> Non-duck typing is a seriously bad idea, forced
> on some other languages by static typing.

Static != non-duck.
One could imagine static duck typing (is it the same as structural
typing?) with type inference. I wonder if some existing languages have
static duck typing (boo? haskell?).




From fredrik at pythonware.com  Wed Nov 15 11:28:54 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 11:28:54 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<455A7CD1.5090503@canterbury.ac.nz>	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
Message-ID: <ejeq56$65k$1@sea.gmane.org>

George Sakkis wrote:

> Fredrik, I am not arguing for the argument's sake, I just don't get
> it: Python requires my class to define __len__, otherwise len() fails.
> Why not require len() as a method instead and forget about __len__ ?

and have people complain about a len/__getitem__ naming inconsistency? 
(cf. the __iter__/next vs. __iter__/__next__ discussions.)

> Does len() (the function) do anything smarter behind the scenes than
> just passing the ball to __len__ ?

yes.  len() can use alternative approaches to determine the length for 
objects implemented in the host language.  this is used by CPython to 
efficiently implement len() for C-level objects without having to do 
full Python method resolution and dispatch for each call.

you could of course turn len() into a __len__() helper (or even 
sequencetools.len(), if you prefer) for cases where this matters, or
even reserve the "len" method name, but I'm not sure how that would
make things *simpler* than they are today.

I'm convinced that it's better for people to get over that silly notion 
that writing x.foo() is somehow always "better" than writing foo(x), in 
a language that actually supports both forms.

(if I'd have to chose between foo(x) and x.foo(), I'd rather get foo(x) 
with multiple dispatch than today's x.foo() single dispatch, but that's 
probably too radical for Python 3000 ;-)

</F>


From ncoghlan at gmail.com  Wed Nov 15 11:34:44 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 15 Nov 2006 20:34:44 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<455A7CD1.5090503@canterbury.ac.nz>	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
Message-ID: <455AED44.5090402@gmail.com>

George Sakkis wrote:
> Fredrik Lundh <fredrik at pythonware.com> wrote:
> 
>> George Sakkis wrote:
>>
>>> 1) why having a "generic operation" len() that ends up looking for an
>>> ugly special *method* called  __len__() makes sense, while calling
>>> directly a method len() doesn't
>> for the very reason Mike explained: keeping implementation interfaces
>> separate from public interfaces has allowed Python to avoid a certain
>> category of design fragmentation that other languages suffer from.
> 
> Fredrik, I am not arguing for the argument's sake, I just don't get
> it: Python requires my class to define __len__, otherwise len() fails.
> Why not require len() as a method instead and forget about __len__ ?
> Does len() (the function) do anything smarter behind the scenes than
> just passing the ball to __len__ ? That could justify its role but
> AFAIK it doesn't.

The benefit of the magic method approach is that it leaves control of the 
public namespace of the class instance in the hands of the class developer. 
For a given class, it may make sense for the class API to expose its length 
under a different name like "size()" or "count()" (e.g. exposing an existing 
Java or C++ class to Python).

However, regardless of what the class API itself looks like, it can be mapped 
behind the scenes into the standard Python idiom by doing "__len__ = size" (or 
whatever). If the Python idioms for these protocols were based on public API 
methods instead of the magic methods then it would greatly interfere with the 
developer's control over their own class interface.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Wed Nov 15 11:51:04 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 15 Nov 2006 20:51:04 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>	<20061114090956.82EB.JCARLSON@uci.edu>	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
Message-ID: <455AF118.1040903@gmail.com>

George Sakkis wrote:
> And for those objecting to touching the existing iter() or bloating
> the builtin namespace with yet another builtin, let me mention that we
> can get rid of *two* existing functions which for some reason were
> promoted to builtin status, although they conceptually belong in
> itertools: enumerate and reversed.

FWIW, reversed isn't based on the iterator protocol - it needs a real sequence 
(i.e. provides __getitem__ supporting indices from 0 to len(seq)-1), or 
something that implements the __reversed__ special method. enumerate() was 
made a builtin as it is a direct replacement for the zip(range(len(x)), x) idiom.

As far as the 'flexible iterator' idea goes, I've had a rough implementation 
of such a beast sitting on my hard drive for the last year and a half (since 
shortly after that January '05 thread I linked to earlier in this discussion). 
My conclusion from writing it was that depending on the implementation choices 
you make in developing the class you will end up in one of two situations:

either 1. The entire iterator is soon loaded into memory as you manipulate it

or 2. The underlying iterator is consumed unexpectedly as you manipulate it

Which lead me to the conclusion that if you want richer access than the 
iterator protocol provides then your best option is to dump the contents of 
the iterator into an appropriate container and work with the container instead 
of the iterator (possible segmenting the retrieval of values if you want to 
limit peak memory usage).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From murman at gmail.com  Wed Nov 15 15:48:33 2006
From: murman at gmail.com (Michael Urman)
Date: Wed, 15 Nov 2006 08:48:33 -0600
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
Message-ID: <dcbbbb410611150648gf0166ar1a7e7513a2dbf8df@mail.gmail.com>

On 11/15/06, George Sakkis <george.sakkis at gmail.com> wrote:
> Why not require len() as a method instead and forget about __len__ ?
> Does len() (the function) do anything smarter behind the scenes than
> just passing the ball to __len__ ? That could justify its role but
> AFAIK it doesn't.

It most certainly does. It not only unifies the name, it makes an
interface guarantee you couldn't make on a custom method: return an
integer or raise an exception.

>>> class Len(object):
...   def __len__(self): return 'length'
...
>>> obj = Len()
>>> obj.__len__()
'length'
>>> len(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: an integer is required

And, if you don't like this interface, it gives you a single point to
override it. (Shoving it into your builtins left as an exercise for
the interested.)

>>> def len(thing):
...   try: return int(thing.__len__())
...   except (ValueError, TypeError, AttributeError): return 0
...
>>> len(obj)
0

-- 
Michael Urman  http://www.tortall.net/mu/blog

From george.sakkis at gmail.com  Wed Nov 15 16:26:48 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 10:26:48 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455AED44.5090402@gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<455AED44.5090402@gmail.com>
Message-ID: <91ad5bf80611150726m7d6f187bhf59593aba111e298@mail.gmail.com>

On 11/15/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> George Sakkis wrote:
> > Fredrik Lundh <fredrik at pythonware.com> wrote:
> >
> >> George Sakkis wrote:
> >>
> >>> 1) why having a "generic operation" len() that ends up looking for an
> >>> ugly special *method* called  __len__() makes sense, while calling
> >>> directly a method len() doesn't
> >> for the very reason Mike explained: keeping implementation interfaces
> >> separate from public interfaces has allowed Python to avoid a certain
> >> category of design fragmentation that other languages suffer from.
> >
> > Fredrik, I am not arguing for the argument's sake, I just don't get
> > it: Python requires my class to define __len__, otherwise len() fails.
> > Why not require len() as a method instead and forget about __len__ ?
> > Does len() (the function) do anything smarter behind the scenes than
> > just passing the ball to __len__ ? That could justify its role but
> > AFAIK it doesn't.
>
> The benefit of the magic method approach is that it leaves control of the
> public namespace of the class instance in the hands of the class developer.
> For a given class, it may make sense for the class API to expose its length
> under a different name like "size()" or "count()" (e.g. exposing an existing
> Java or C++ class to Python).
>
> However, regardless of what the class API itself looks like, it can be mapped
> behind the scenes into the standard Python idiom by doing "__len__ = size" (or
> whatever). If the Python idioms for these protocols were based on public API
> methods instead of the magic methods then it would greatly interfere with the
> developer's control over their own class interface.

On the one hand this makes sense, but on the other it goes back to
Bill's reply on that Python should formalize some widely used
interfaces (iterators, sequence, mappings, etc.), exactly for
extending the "one obvious way to do it" rule to interfaces and
prevent anyone who misses Java too much to use getIndex() for his
container instead of __getitem__.

George

From gsakkis at rutgers.edu  Wed Nov 15 16:39:26 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 15 Nov 2006 10:39:26 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455AF118.1040903@gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
	<455AF118.1040903@gmail.com>
Message-ID: <91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>

On 11/15/06, Nick Coghlan <ncoghlan at gmail.com> wrote:

> As far as the 'flexible iterator' idea goes, I've had a rough implementation
> of such a beast sitting on my hard drive for the last year and a half (since
> shortly after that January '05 thread I linked to earlier in this discussion).
> My conclusion from writing it was that depending on the implementation choices
> you make in developing the class you will end up in one of two situations:
>
> either 1. The entire iterator is soon loaded into memory as you manipulate it
>
> or 2. The underlying iterator is consumed unexpectedly as you manipulate it

For (2), I'd say it depends on your expectations. If you expect x[:5]
to have always the same semantics you expect from a list, then yes,
you will be surprized if x is an iterator, as you may be surprized if
x is a numpy array. However, it seems contradictory for Python to
insist on specific method semantics, while at the same time is
reluctant to enforce APIs for omnipresent interfaces like sequences
and iterators.

George

From gsakkis at rutgers.edu  Wed Nov 15 17:08:54 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 15 Nov 2006 11:08:54 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
	<455AF118.1040903@gmail.com>
	<91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>
Message-ID: <91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>

Michael Urman <murman at gmail.com> wrote:

> On 11/15/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > Why not require len() as a method instead and forget about __len__ ?
> > Does len() (the function) do anything smarter behind the scenes than
> > just passing the ball to __len__ ? That could justify its role but
> > AFAIK it doesn't.
>
> It most certainly does. It not only unifies the name, it makes an
> interface guarantee you couldn't make on a custom method: return an
> integer or raise an exception.
>
> >>> class Len(object):
> ...   def __len__(self): return 'length'
> ...
> >>> obj = Len()
> >>> obj.__len__()
> 'length'
> >>> len(obj)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: an integer is required
>
> And, if you don't like this interface, it gives you a single point to
> override it. (Shoving it into your builtins left as an exercise for
> the interested.)
>
> >>> def len(thing):
> ...   try: return int(thing.__len__())
> ...   except (ValueError, TypeError, AttributeError): return 0

Thank you for your explanation; I'd rather read this in the FAQ rather than a
moot comparison with Java's inconsistent design.

As I wrote in my last reply to Nick though, I question Python's right to perform
such limited forms of design-by-contract-like assertions ("why not add
a precondition
on __add__(self,other) to enforce isinstance(other, self.__class__)
?") when it refuses to formalize interfaces for sequences, iterators
et al.

George

From ark-mlist at att.net  Wed Nov 15 17:50:47 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 15 Nov 2006 11:50:47 -0500
Subject: [Python-3000] duck typing
In-Reply-To: <49981.62.39.9.251.1163585444.squirrel@webmail.nerim.net>
Message-ID: <000001c708d6$39d0fe90$6402a8c0@arkdesktop>

> Static != non-duck.
> One could imagine static duck typing (is it the same as structural
> typing?) with type inference. I wonder if some existing languages have
> static duck typing (boo? haskell?).

C++ (using templates).



From fredrik at pythonware.com  Wed Nov 15 17:59:41 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 17:59:41 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>	<20061114090956.82EB.JCARLSON@uci.edu>	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>	<455AF118.1040903@gmail.com>	<91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>
	<91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
Message-ID: <ejfh1s$ps5$1@sea.gmane.org>

George Sakkis wrote:

> Thank you for your explanation; I'd rather read this in the FAQ rather than a
> moot comparison with Java's inconsistent design.

if you have comments on the FAQ entries, post them over there.

note that the FAQ entry isn't about len(), though; it's about generic 
functions in general.

(and I completely disagree that Mike's observation is moot: consistency 
is a fundamental part of Python's design, and mechanisms that helps 
people keep their code consistent without even noticing is an important 
part of why things are that way.)

</F>


From gsakkis at rutgers.edu  Wed Nov 15 18:39:02 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 15 Nov 2006 12:39:02 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<20061114090956.82EB.JCARLSON@uci.edu>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>
	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>
	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>
	<455AF118.1040903@gmail.com>
	<91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>
	<91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
Message-ID: <91ad5bf80611150939s416c7a28n7277cc795e35c82c@mail.gmail.com>

Fredrik Lundh <fredrik at pythonware.com> wrote:

> > Thank you for your explanation; I'd rather read this in the FAQ rather than a
> > moot comparison with Java's inconsistent design.
>
> if you have comments on the FAQ entries, post them over there.
>
> note that the FAQ entry isn't about len(), though; it's about generic
> functions in general.

Something along the lines of "a generic function might also do other
smart things
apart from just calling the appropriate methods on its argument(s),
such as ..." would help (rephrased to match the FAQ's style).

> (and I completely disagree that Mike's observation is moot: consistency
> is a fundamental part of Python's design, and mechanisms that helps
> people keep their code consistent without even noticing is an important
> part of why things are that way.)

Striving for consistency without enforcing interfaces seems self defeating.

George

From jcarlson at uci.edu  Wed Nov 15 18:13:41 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 15 Nov 2006 09:13:41 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
References: <91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>
	<91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
Message-ID: <20061115090151.82FE.JCARLSON@uci.edu>


"George Sakkis" <gsakkis at rutgers.edu> wrote:
> As I wrote in my last reply to Nick though, I question Python's right to perform
> such limited forms of design-by-contract-like assertions ("why not add
> a precondition
> on __add__(self,other) to enforce isinstance(other, self.__class__)
> ?") when it refuses to formalize interfaces for sequences, iterators
> et al.

Because 5, 5+0j, 5.0 are different types.  To not be able to add any of
them because they are different types, would be silly.  Note that
Decimal is a different beast, which is why I don't include it.  There's
also string + unicode (at least in 2.x), and if some sort of
unicode view makes it into 3.x, view + unicode.

 - Josiah


From fredrik at pythonware.com  Wed Nov 15 18:48:29 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 15 Nov 2006 18:48:29 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150939s416c7a28n7277cc795e35c82c@mail.gmail.com>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>	<20061114090956.82EB.JCARLSON@uci.edu>	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>	<fb6fbf560611141006s4bae7cdam95f4b58f632bf89a@mail.gmail.com>	<91ad5bf80611141019i316747b1t45222bf50ecd0446@mail.gmail.com>	<91ad5bf80611141144l58569e26wf5633a849e07c9ad@mail.gmail.com>	<455AF118.1040903@gmail.com>	<91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>	<91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
	<91ad5bf80611150939s416c7a28n7277cc795e35c82c@mail.gmail.com>
Message-ID: <ejfjtc$50h$1@sea.gmane.org>

George Sakkis wrote:

>> (and I completely disagree that Mike's observation is moot: consistency
>> is a fundamental part of Python's design, and mechanisms that helps
>> people keep their code consistent without even noticing is an important
>> part of why things are that way.)
> 
> Striving for consistency without enforcing interfaces seems self defeating.

seems to, perhaps, but Python's been around for a quite some time by 
now.  if the approach didn't work, we'd probably noticed by now.

</F>


From janssen at parc.com  Wed Nov 15 18:54:47 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 15 Nov 2006 09:54:47 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com> 
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
Message-ID: <06Nov15.095454pst."58648"@synergy1.parc.xerox.com>

> Fredrik Lundh <fredrik at pythonware.com> wrote:
> 
> > George Sakkis wrote:
> >
> > > 1) why having a "generic operation" len() that ends up looking for an
> > > ugly special *method* called  __len__() makes sense, while calling
> > > directly a method len() doesn't
> >
> > for the very reason Mike explained: keeping implementation interfaces
> > separate from public interfaces has allowed Python to avoid a certain
> > category of design fragmentation that other languages suffer from.
> 
> Fredrik, I am not arguing for the argument's sake, I just don't get
> it: Python requires my class to define __len__, otherwise len() fails.
> Why not require len() as a method instead and forget about __len__ ?
> Does len() (the function) do anything smarter behind the scenes than
> just passing the ball to __len__ ? That could justify its role but
> AFAIK it doesn't.
> 
> George

So you're suggesting something like:

   class Lengthable:

      def len(self):
         raise NotImplemented

(presumably defined in some "standard" or "builtins" module) be a
mix-in to every type which it currently makes sense to call "len()"
on?

Bill


From george.sakkis at gmail.com  Wed Nov 15 19:36:33 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 13:36:33 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <-9052809295998148604@unknownmsgid>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
Message-ID: <91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>

On 11/15/06, Bill Janssen <janssen at parc.com> wrote:

> > Fredrik Lundh <fredrik at pythonware.com> wrote:
> >
> > > George Sakkis wrote:
> > >
> > > > 1) why having a "generic operation" len() that ends up looking for an
> > > > ugly special *method* called  __len__() makes sense, while calling
> > > > directly a method len() doesn't
> > >
> > > for the very reason Mike explained: keeping implementation interfaces
> > > separate from public interfaces has allowed Python to avoid a certain
> > > category of design fragmentation that other languages suffer from.
> >
> > Fredrik, I am not arguing for the argument's sake, I just don't get
> > it: Python requires my class to define __len__, otherwise len() fails.
> > Why not require len() as a method instead and forget about __len__ ?
> > Does len() (the function) do anything smarter behind the scenes than
> > just passing the ball to __len__ ? That could justify its role but
> > AFAIK it doesn't.
> >
> > George
>
> So you're suggesting something like:
>
>    class Lengthable:
>
>       def len(self):
>          raise NotImplemented
>
> (presumably defined in some "standard" or "builtins" module) be a
> mix-in to every type which it currently makes sense to call "len()"
> on?

No, having an interface for every single method is obviously an
overkill. Unless I am convinced otherwise, I am thinking something
like:

class Container(object):
   def len(self): raise NotImplementedError
   def __iter__(self): raise NotImplementedError
   # other generic container methods

class Sequence(Container):
   def __getitem__(self, index_or_slice): raise NotImplementedError
   # more methods

class Mapping(Container):
   def __getitem__(self, key): raise NotImplementedError
   # more methods

class list(Sequence):
    # blah blah

class tuple(Sequence):
    # blah blah

class MyFancySequence(Sequence):
    # blah blah


Note that this does _not_ disallow duck typing; anyone is perfectly
free to define a method len() in a class that does not extend any of
the above so that a client can call x.len() successfully. If not, a
normal AttributeError will be raised, instead of the TypeError you get
today.

George

From gsakkis at rutgers.edu  Wed Nov 15 19:47:50 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 15 Nov 2006 13:47:50 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061115090151.82FE.JCARLSON@uci.edu>
References: <91ad5bf80611150739h6c19a130ve065a773b0b08855@mail.gmail.com>
	<91ad5bf80611150808m8d72ddanf30bfef495c8475d@mail.gmail.com>
	<20061115090151.82FE.JCARLSON@uci.edu>
Message-ID: <91ad5bf80611151047k57046c0ct84ab731b8701c7cd@mail.gmail.com>

On 11/15/06, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "George Sakkis" <gsakkis at rutgers.edu> wrote:
> > As I wrote in my last reply to Nick though, I question Python's right to perform
> > such limited forms of design-by-contract-like assertions ("why not add
> > a precondition
> > on __add__(self,other) to enforce isinstance(other, self.__class__)
> > ?") when it refuses to formalize interfaces for sequences, iterators
> > et al.
>
> Because 5, 5+0j, 5.0 are different types.  To not be able to add any of
> them because they are different types, would be silly.  Note that
> Decimal is a different beast, which is why I don't include it.  There's
> also string + unicode (at least in 2.x), and if some sort of
> unicode view makes it into 3.x, view + unicode.

You missed my point; I wasn't seriously suggesting that there should
be more checks (hence the quotes), but rather the contrary, that
checking whether len() returns a non-negative integer does very little
to the overall consistency scheme of things.

Pre-conditions, post-conditions, invariants are a stricter form of API
conformance than just checking whether a method with a specific name
is there, and Python is unwilling to check even that.

George

From jcarlson at uci.edu  Wed Nov 15 20:36:09 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 15 Nov 2006 11:36:09 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611151047k57046c0ct84ab731b8701c7cd@mail.gmail.com>
References: <20061115090151.82FE.JCARLSON@uci.edu>
	<91ad5bf80611151047k57046c0ct84ab731b8701c7cd@mail.gmail.com>
Message-ID: <20061115111318.8301.JCARLSON@uci.edu>


"George Sakkis" <gsakkis at rutgers.edu> wrote:
> You missed my point; I wasn't seriously suggesting that there should
> be more checks (hence the quotes), but rather the contrary, that
> checking whether len() returns a non-negative integer does very little
> to the overall consistency scheme of things.
> 
> Pre-conditions, post-conditions, invariants are a stricter form of API
> conformance than just checking whether a method with a specific name
> is there, and Python is unwilling to check even that.

No, it's not that Python is unwilling, we just haven't found the need. 
For people who want/need to _enforce_ preconditions, postconditions,
invariants, etc., Python 2.4 introduced a decorator syntax that makes it
(arguably) trivial to declare such things...

@typecheck((int, long), (int, long), (int, long, float))
@precondition(ARG > 0, 0 < ARG < 10, NOTIN(inf, NaN, -inf))
@postcondition(NOTIN(inf, NaN, -inf))
def fcn(a, b, c):
    ...

Given proper definitions of typecheck, precondition, postcondition, ARG
and NOTIN.

For the rest of us who document, use unit testing (with or without the
unittest framework), etc., thinking of development in terms of what we
learned in school for developing C, C++, or Java seems backwards (at
least in my opinion).  I can spend days writing all of that crap, or I
can spend a few hours writing the software with unittests in Python.

 - Josiah


From gsakkis at rutgers.edu  Wed Nov 15 20:53:48 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 15 Nov 2006 14:53:48 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061115111318.8301.JCARLSON@uci.edu>
References: <20061115090151.82FE.JCARLSON@uci.edu>
	<91ad5bf80611151047k57046c0ct84ab731b8701c7cd@mail.gmail.com>
	<20061115111318.8301.JCARLSON@uci.edu>
Message-ID: <91ad5bf80611151153m6fc76cefu66960901f6fb58c1@mail.gmail.com>

On 11/15/06, Josiah Carlson <jcarlson at uci.edu> wrote:
>
> "George Sakkis" <gsakkis at rutgers.edu> wrote:
> > You missed my point; I wasn't seriously suggesting that there should
> > be more checks (hence the quotes), but rather the contrary, that
> > checking whether len() returns a non-negative integer does very little
> > to the overall consistency scheme of things.
> >
> > Pre-conditions, post-conditions, invariants are a stricter form of API
> > conformance than just checking whether a method with a specific name
> > is there, and Python is unwilling to check even that.
>
> No, it's not that Python is unwilling, we just haven't found the need.
> For people who want/need to _enforce_ preconditions, postconditions,
> invariants, etc., Python 2.4 introduced a decorator syntax that makes it
> (arguably) trivial to declare such things...
>
> @typecheck((int, long), (int, long), (int, long, float))
> @precondition(ARG > 0, 0 < ARG < 10, NOTIN(inf, NaN, -inf))
> @postcondition(NOTIN(inf, NaN, -inf))
> def fcn(a, b, c):
>     ...
>
> Given proper definitions of typecheck, precondition, postcondition, ARG
> and NOTIN.
>
> For the rest of us who document, use unit testing (with or without the
> unittest framework), etc., thinking of development in terms of what we
> learned in school for developing C, C++, or Java seems backwards (at
> least in my opinion).  I can spend days writing all of that crap, or I
> can spend a few hours writing the software with unittests in Python.

To avoid leading this to a digression about optional typing, my point
was that what len() does is a builtin postcondition, and if, for some
perverse reason, someone wants to define negative or complex lengths
for his perverse class, he'll hit into a wall. Is this yet another
case of practicality beating purity hands down ?

George

From george.sakkis at gmail.com  Wed Nov 15 21:34:24 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 15:34:24 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <fb6fbf560611151205w5601381fwb07af1f03ff1a967@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<fb6fbf560611151205w5601381fwb07af1f03ff1a967@mail.gmail.com>
Message-ID: <91ad5bf80611151234x7d7f6b27j6c7ef217c197c2dc@mail.gmail.com>

On 11/15/06, Jim Jewett <jimjjewett at gmail.com> wrote:

> (list clipped)

Sorry, I'd rather not get into private conversations.

> On 11/15/06, George Sakkis <george.sakkis at gmail.com> wrote:
> > Note that this does _not_ disallow duck typing; anyone is perfectly
> > free to define a method len() in a class that does not extend any of
> > the above so that a client can call x.len() successfully.
>
>
> Not if
>     >>> if instance(x, Sequence):
>
> becomes a standard idiom

It doesn't have to, and it shouldn't, become the standard; after all,
whatever you do in the "else:" part of the isinstance check, you can
do in a try/except block. This has nothing to do with type checking
per se, it's a general LBYL vs EAFP error handling strategy. Python in
general encourages the latter and I totally agree, there's no reason
for this to change. In the few cases where LBYL is deemed better and
you'd use isinstance, you can do the same today with
    if hasattr(x, '__getitem__'):
or whatever attributes you need.

George

From george.sakkis at gmail.com  Wed Nov 15 23:29:05 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 17:29:05 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455B7FE7.8040301@ilowe.net>
References: <91ad5bf80611140849t25509976q5f396cec9d3c7370@mail.gmail.com>
	<91ad5bf80611140955s7cd3cfe8qee85b898e98230de@mail.gmail.com>
	<91ad5bf80611141006k431695e4l1d7752cb1f027aea@mail.gmail.com>
	<200611141353.18690.fdrake@acm.org>
	<91ad5bf80611141132v3cac4035nfa92b52f394ebd1a@mail.gmail.com>
	<455B7FE7.8040301@ilowe.net>
Message-ID: <91ad5bf80611151429u3c5153dkeffe7b45be7b8cf0@mail.gmail.com>

On 11/15/06, Iain Lowe <me at ilowe.net> wrote:

> Hi George,
>
> I've been reading this thread with some interest since you seem to be
> really adamant about the "features" you are requesting. I went back into
> the c.l.p thread and did a little more searching before finding this
> thread: http://shrunklink.com/?nnq

Glad you ran a full investigation on me before replying, that's reassuring :-)

> I don't really understand how you can be advocating the opposite of what
> you explain rather clearly in the referenced thread. Specifically you
> say "there's no particular reason this should be a method of class A
> since it can be used for any arbitrary object with no extra overhead.
> Now, if you intend to use it only for instances of A and its subclasses,
> the only difference would be syntactic."
>
> Now, clearly we are not talking about "any arbitrary object" but rather
> "any arbitrary object that implements the iterable protocol" but it
> seems to me that there is little difference between what you are
> proposing here and the OOP solution the poster in the referenced thread
> was proposing.
>
> In both cases, the implementation of a series of constraints as an
> *interface* instead of as a *protocol* un-necessarily limits future
> code. It also seems clear to me that you would like to be able to
> slice/chain/etc. *any* iterable, not just instances of (subclasses of)
> `Iter`. So why advocate the narrowing of the *protocol* into an
> *interface* just to add some syntax sugar. Indeed, the two most-quoted
> examples so far (slicing and indexing) can be better written as
> "list(it)[:3]" and "list(it)[6]". The only possible caveat I can see is
> that this will not work for infinite iterables. But *even if* you did
> implement this "class-wrapper" for iterables you *still* would be
> breaking the implementation of slices (since you would not be able to do
> "Iter(infinite_it)[-5]").
>
> What am I missing here?

Several things:

1. In the thread you refer to, the whole point was *one* callable
(listMethods) that takes a single arbitrary object. I wouldn't propose
a class for holding a single method of course, but here we're talking
about a dozen or so callables that take an iterable (and optionally
other parameters).

2. If one wanted to attach listMethods to an existing type, the only
obvious choice would be the base object type. That callable isn't so
important though to pollute the base object namespace with a new
method. OTOH in this thread:
  (1) I don't propose to pollute object or any other existing type; I
propose a brand new "rich iterator" type, and
  (2) I consider several itertools functions (chain, islice, izip,
enumerate) fundamental in getting the most out of iterables.
Surprizingly, as it turns out nobody else here seems to share this
feeling. I'd say this is the only argument so far I have nothing to
respond to. If people consider itertools a small obscure module they
rarely need to import anyway, the whole proposal fails flat.

3. I don't know if you've followed the thread for the last day or so,
but I've dropped the part that suggested the proposed type as a
required base for all iterators. Instead I suggest it as a wrapper (or
"adaptor" if you prefer) to provide a rich-iterator API to the code
that needs it. The overhead of wrapping an arbitrary iterable is
trivial, both in keystrokes ("Iter(x)") and in runtime, so objections
of the kind "how should a client know whether he is passed a simple
iterator or a rich one?" are easily responded by "never assume you are
passed a rich iterator; just wrap it with Iter() yourself if you need
it".

4. As for the negative indices that don't work for iterators, let me
remind you that indexing and slicing, like pretty much everything else
in python, are an informal API. Numpy array slices also have different
semantics from list slices, and for good reason. Another hypothetical
example would be an automatically extensible sequence type defined so
that s[i] for i>=len(s) doesn't raise IndexError, but it extends the
sequence with some default value up to i and the returns s[i]. The
bottom line is, you can't assume you know what x[:5] does without
knowing x's type, the same way you don't if x.shoot() will shoot a
photo or a bullet without knowing what x is.

> P.S. I notice in a couple of places you ask what the objections are
> whereas I think that in this case it's up to the proposal's author to
> explain why the community can't live without the feature rather than for
> the community to explain why they don't need it. Clearly if they needed
> it they would have built it before (or at least you would just get a
> stream +1's)

Right, and that's the part I mistakenly took for granted, the
prevalence of itertools. My bad.

George

From tomerfiliba at gmail.com  Thu Nov 16 00:06:04 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Thu, 16 Nov 2006 01:06:04 +0200
Subject: [Python-3000] yes to class decorators
Message-ID: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>

i understand there's a green light for class decorators in py3k,
so i wanted to give the issue a slight push.

one problem with metaclasses is they are non-nestable, i.e., a
function can be wrapped by a number of decorators, while
classes can have only one metaclass.

not only that, but metaclasses complicate inheritance,
because their are not "componentized" the way decorators
are (i.e., decorator A does not impose restrictions on
how decorator B will work, whereas metaclasses must
be subclasses of one another, which means they are not
isolated *components*)

first i want to differentiate between ``metaclasses`` and
``pseudo metaclasses``. real metaclasses subclass type
and change it's behavior. pseudo metaclasses just use
the __metaclass__ magic syntax to modify the class
being returned, but not it's type (i.e., it's type is //type//)

for example -- this is what i call pseudo-metaclass:
>>> def singleton(*args):
...     return type(*args)() # create the only instance
...
>>> class OnlyOne(object):
...     __metaclass__ = singleton
...
>>> OnlyOne
<__main__.OnlyOne object at 0x009F5270>

of course we want to keep metaclasses as a facility as
well as a concept, but we do not want to have to use the
__metaclass__ syntax when it's not about real metaclasses.

class decorators -- like function decorators -- wrap the object
(class in this case), which makes them stackable and
componentized.

here are some usecases to class decorators:

def singleton(cls):
    return cls() # create the "single instance"

@singleton
class OnlyOne(object):
    pass

another example: @staticclass - automatically make all
methods staticmethods, or whatever (dotNET has them,
for instance). essentially this turns the class into a module.

def staticclass(cls):
    newdict = {}
    for name, obj in cls.__dict__.iteritems():
        if isinstance(getattr(cls, name), MethodType):
            newdict[name] = staticmethod(obj)
        else:
           newdict[name] = obj
    return type(cls.__name__, cls.__bases__, newdict)

@staticclass
class Eggs(objects):
    def foo():
        print "foo"
    def goo():
        print "goo"

Eggs.foo()
e = Eggs()
e.goo()

i'm not saying this particular example is useful, but i have
had many times when i thought "argh, i wish i had class
decorators for that".

sadly though, i don't remember many of them, since i always
had to find workarounds :)

ANYWAY, if we went this far already, we might as well just
trash the __metaclass__ syntax (*) altogether. we can easily
implement a metaclass decorator that does the trick:

def metaclass(meta):
    def wrapper(cls):
        return meta(cls.__name__, cls.__bases__, dict(cls.__dict__))
    return wrapper

class FooMeta(type):
   ....

@singleton
@metaclass(FooMeta)
class Bar(object):
    ....

or even something like

@FooMeta
class Bar(object):
    pass

although that would require some hackery. forget it, it's better
to be explicit.

(*) throwing only the syntax. subclassing type is still required
of course, but IMHO it's much more clear to understand where
the "magic" comes from when a class has a @metaclass
decorator instead of a __metaclass__ attribute.

moreover, as shown above, we can simplify the way type_new()
works, which is a blessing on its own (considering its current
complexity)


- tomer

From steven.bethard at gmail.com  Thu Nov 16 00:38:41 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 15 Nov 2006 16:38:41 -0700
Subject: [Python-3000] yes to class decorators
In-Reply-To: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
Message-ID: <d11dcfba0611151538taa51c1cg95a8259563b5b4ab@mail.gmail.com>

On 11/15/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> i understand there's a green light for class decorators in py3k,
> so i wanted to give the issue a slight push.

If you haven't already, you should look at PEP 359, which tried to
address many of the same use-cases and was withdrawn after feedback on
python-dev:

    http://www.python.org/dev/peps/pep-0359/

> def singleton(cls):
>     return cls() # create the "single instance"
>
> @singleton
> class OnlyOne(object):
>     pass

That would have been something like::

    make singleton OnlyOne:
        pass

> @staticclass
> class Eggs(objects):
>     def foo():
>         print "foo"
>     def goo():
>         print "goo"

And this would have been something like::

    make namespace Eggs:
        def foo():
            print "foo"
        def goo():
            print "goo"

> ANYWAY, if we went this far already, we might as well just
> trash the __metaclass__ syntax (*) altogether.

That was one of the open issues in the PEP too.

Basically, I think that the use-cases you've given are not a great
motivation -- they weren't enough to motivate the "make" statement of
PEP 359, so they're probably not enough to motivate class decorators.

FWIW, most of the arguments against PEP 359 were along the lines of,
"well you can do that with a metaclass already, so we don't really
need any new syntax", but you may be able to get around those
arguments because the decorator syntax already exists.


STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From george.sakkis at gmail.com  Thu Nov 16 01:52:57 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 19:52:57 -0500
Subject: [Python-3000] yes to class decorators
Message-ID: <91ad5bf80611151652w51a4446ar1bf1e1307d1c6ba9@mail.gmail.com>

tomer filiba <tomerfiliba at gmail.com> wrote:

> i'm not saying this particular example is useful, but i have
> had many times when i thought "argh, i wish i had class
> decorators for that".

Same here, but like Steve, I'm not very optimistic this will be
greeted with open arms.

> sadly though, i don't remember many of them, since i always
> had to find workarounds :)

Off the top of my head, @abstractclass was one of my workarounds :)

George

From greg.ewing at canterbury.ac.nz  Thu Nov 16 02:09:51 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 16 Nov 2006 14:09:51 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
Message-ID: <455BBA5F.5070106@canterbury.ac.nz>

George Sakkis wrote:

> class Container(object):
>
> class Sequence(Container):

> class Mapping(Container):
>
> Note that this does _not_ disallow duck typing; anyone is perfectly
> free to define a method len() in a class that does not extend any of
> the above so that a client can call x.len() successfully.

What's the point, then? What benefit do I get from
extending Sequence if I can achieve the same thing
without bothering?

--
Greg


From george.sakkis at gmail.com  Thu Nov 16 03:49:12 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Wed, 15 Nov 2006 21:49:12 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455BBA5F.5070106@canterbury.ac.nz>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
Message-ID: <91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>

On 11/15/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> George Sakkis wrote:
>
> > class Container(object):
> >
> > class Sequence(Container):
>
> > class Mapping(Container):
> >
> > Note that this does _not_ disallow duck typing; anyone is perfectly
> > free to define a method len() in a class that does not extend any of
> > the above so that a client can call x.len() successfully.
>
> What's the point, then? What benefit do I get from
> extending Sequence if I can achieve the same thing
> without bothering?

For one thing, the implementations you get for free for the methods
that can be implemented without needing to know the specifics of the
concrete class (think DictMixin, ListMixin and friends; all these
would move there). So yes, for the nitpickers out there, these are not
interfaces in the Java sense (or pure virtual classes in C++) but
abstract classes.

And for two, not everyone feels comfortable with duck typing. People
who consider (for better or for worse) isinstance() safer than
hasattr()/getattr() would be accomodated too. Why do we have to
alienate those folks telling them they are wrong when both approaches
can coexist, as it happens for LBYL vs EAFP error strategies,
procedural vs OO vs functional paradigms, and other dichotomies and
n-otomies that Python has successfully brought together ?

George

From jcarlson at uci.edu  Thu Nov 16 02:43:12 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 15 Nov 2006 17:43:12 -0800
Subject: [Python-3000] yes to class decorators
In-Reply-To: <d11dcfba0611151538taa51c1cg95a8259563b5b4ab@mail.gmail.com>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
	<d11dcfba0611151538taa51c1cg95a8259563b5b4ab@mail.gmail.com>
Message-ID: <20061115172512.8305.JCARLSON@uci.edu>


"Steven Bethard" <steven.bethard at gmail.com> wrote:
> On 11/15/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> > i understand there's a green light for class decorators in py3k,
> > so i wanted to give the issue a slight push.
> FWIW, most of the arguments against PEP 359 were along the lines of,
> "well you can do that with a metaclass already, so we don't really
> need any new syntax", but you may be able to get around those
> arguments because the decorator syntax already exists.

That's neither here nor there.  Here's a post from Guido in response to
Phillip and Greg in which he says more or less; someone write a PEP so
that we can get them into 2.6 and Py3k...
http://mail.python.org/pipermail/python-dev/2006-March/062942.html

The use-cases that Tomer brought up were also brought up in various
posts within that same thread, and in the 3+ previous threads extending
prior to March 2005 in the python-dev list (when I noticed that class
decorators didn't make it into 2.4); namespaces, singletons, easy
'metaclass' chaining, whole class manipulations, properties, etc.

The syntax already exists and I would imagine that there's a patch
around somewhere.  If Tomer (or someone else) writes a PEP, I don't see
why (the previously overlooked) class decorators shouldn't make it into
2.6 and 3.0 .

 - Josiah


From steven.bethard at gmail.com  Thu Nov 16 04:55:56 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 15 Nov 2006 20:55:56 -0700
Subject: [Python-3000] yes to class decorators
In-Reply-To: <20061115172512.8305.JCARLSON@uci.edu>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
	<d11dcfba0611151538taa51c1cg95a8259563b5b4ab@mail.gmail.com>
	<20061115172512.8305.JCARLSON@uci.edu>
Message-ID: <d11dcfba0611151955l575dad56l4100964df49bba7d@mail.gmail.com>

On 11/15/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> i understand there's a green light for class decorators in py3k,
> so i wanted to give the issue a slight push.

"Steven Bethard" <steven.bethard at gmail.com> wrote:
> FWIW, most of the arguments against PEP 359 were along the lines of,
> "well you can do that with a metaclass already, so we don't really
> need any new syntax", but you may be able to get around those
> arguments because the decorator syntax already exists.

On 11/15/06, Josiah Carlson <jcarlson at uci.edu> wrote:
> Here's a post from Guido in response to Phillip and Greg in which he
> says more or less; someone write a PEP so that we can get them
> into 2.6 and Py3k...
> http://mail.python.org/pipermail/python-dev/2006-March/062942.html

Thanks for the link.

> If Tomer (or someone else) writes a PEP, I don't see why (the
> previously overlooked) class decorators shouldn't make it into
> 2.6 and 3.0 .

So the purpose of this thread then is to write the PEP?  If so, my
comments about the use cases are still valid.  If they weren't
convincing use cases before, they're not likely to be convincing use
cases for a PEP.

Or was there another purpose of the thread?

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From jcarlson at uci.edu  Thu Nov 16 05:09:29 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Wed, 15 Nov 2006 20:09:29 -0800
Subject: [Python-3000] yes to class decorators
In-Reply-To: <d11dcfba0611151955l575dad56l4100964df49bba7d@mail.gmail.com>
References: <20061115172512.8305.JCARLSON@uci.edu>
	<d11dcfba0611151955l575dad56l4100964df49bba7d@mail.gmail.com>
Message-ID: <20061115200359.830A.JCARLSON@uci.edu>


"Steven Bethard" <steven.bethard at gmail.com> wrote:
> On 11/15/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> > i understand there's a green light for class decorators in py3k,
> > so i wanted to give the issue a slight push.
> 
> "Steven Bethard" <steven.bethard at gmail.com> wrote:
> > FWIW, most of the arguments against PEP 359 were along the lines of,
> > "well you can do that with a metaclass already, so we don't really
> > need any new syntax", but you may be able to get around those
> > arguments because the decorator syntax already exists.
> 
> On 11/15/06, Josiah Carlson <jcarlson at uci.edu> wrote:
> > Here's a post from Guido in response to Phillip and Greg in which he
> > says more or less; someone write a PEP so that we can get them
> > into 2.6 and Py3k...
> > http://mail.python.org/pipermail/python-dev/2006-March/062942.html
> 
> Thanks for the link.
> 
> > If Tomer (or someone else) writes a PEP, I don't see why (the
> > previously overlooked) class decorators shouldn't make it into
> > 2.6 and 3.0 .
> 
> So the purpose of this thread then is to write the PEP?  If so, my
> comments about the use cases are still valid.  If they weren't
> convincing use cases before, they're not likely to be convincing use
> cases for a PEP.

Except that they *were* convincing, which is why Guido green lighted it
for 2.6 and 3.0 .  One of the purposes of the PEP, like all other PEPs,
is so that in the future when someone comes up with some so-called
"enhancement" they can't point to a previous feature addition (like
class decorators) and say, "but _they_ didn't have to write a PEP to get
the feature in, why do _I_?"


> Or was there another purpose of the thread?

I believe that Tomer just wanted to remind people that class decorators
are slated for 2.6 and 3.0 if there was a PEP, and perhaps that he is
encouraging someone to write one.

 - Josiah


From talin at acm.org  Thu Nov 16 06:41:43 2006
From: talin at acm.org (Talin)
Date: Wed, 15 Nov 2006 21:41:43 -0800
Subject: [Python-3000] Class Decorators + ctypes (Was: yes to class
	decorators)
In-Reply-To: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
Message-ID: <455BFA17.1000205@acm.org>

tomer filiba wrote:
> here are some usecases to class decorators:

What I would like is a way to use class decorators, combined with 
ctypes, to effectively 'declare' a C structure using Python syntax. 
Right now you can use ctypes to declare a C struct, but the syntax isn't 
very Python-like:

    class RECT(Structure):
      _fields_ = [("upperleft", POINT),
                  ("lowerright", POINT)]

Even with class decorators, however, there is still a missing piece - 
there's no way to decorate a variable declaration. That's because 
there's no equivalent of 'def' for variables. (Thinking about it, it 
does seem like there's an asymmetry here - you can define functions 
using 'def' or using assignment, but you can define variables only using 
assignment.)

Here's a more specific proposal: The token '->' has been proposed (as 
part of the argument decorator syntax) as a way to indicate the return 
type of a function definition. What if a similar syntax could be used to 
  define and create a class variable?

Here's an example. I realize that the syntax may look strange, but it's 
all a logical extension of things that have already been proposed and 
for the most part accepted:

@cstruct
class CPoint:

    def x -> float
    def y -> float

Under the hood, here's what's happening:

1) 'def' without an argument list defines a variable instead of a function.

2) The '-> float' is merely advisory - it has no meaning except to the 
decorator, just as has been proposed with argument decorators.

3) The @cstruct decorator changes the way that the class is evaluated, 
adding an implicit decorator to each definition. In this case, it 
evaluates the argument decorators and adds a field definition for each one.

One unresolved issue is what exactly gets assigned to 'x' and 'y' at the 
time the class definition is evaluated. Ordinarily, 'def' adds a named 
entry to the class dict. In a normal function declaration, the part 
after the colon defines a suite, which is the value assigned to the name.

I first thought about allowing the same colon to be used to define the 
value to assign to the variable, i.e.:

    def x -> float: 0

However, this is inconsistent with the use of the colon in other places 
in Python, which generally introduces a suite.

An alternative would be to use an assignment operator:

    def x -> float = 0

The only difference between this and simply 'x = 0' is that the 'def' 
statement, unlike an assignment, can be decorated.

A different approach would be to say that the class decorator can 
overload the '__setattr__' method on the class itself during class creation:

@cstruct
class CPoint:

    x = FLOAT
    y = FLOAT

I think that you would *only* want to do this override during the 
initial evaluation of the class. Once the class was fully created, I 
would think that you'd want to be able to do "CPoint.x = value" without 
triggering the decorator's override.

(As a bit of trivia that may be of interest: The latest proposals for 
extending the C++ language allow a function return type to be declared 
using the "(args) -> returntype" syntax. The primary reason for doing 
this is to avoid certain syntactical ambiguities and to allow support 
for lambda functions in C++, but there are some other advantages to 
doing so when using functions as template arguments.)

-- Talin


From talin at acm.org  Thu Nov 16 06:55:15 2006
From: talin at acm.org (Talin)
Date: Wed, 15 Nov 2006 21:55:15 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ejeq56$65k$1@sea.gmane.org>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>	<455A7CD1.5090503@canterbury.ac.nz>	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<ejeq56$65k$1@sea.gmane.org>
Message-ID: <455BFD43.6070701@acm.org>

Fredrik Lundh wrote:
> I'm convinced that it's better for people to get over that silly notion 
> that writing x.foo() is somehow always "better" than writing foo(x), in 
> a language that actually supports both forms.
> 
> (if I'd have to chose between foo(x) and x.foo(), I'd rather get foo(x) 
> with multiple dispatch than today's x.foo() single dispatch, but that's 
> probably too radical for Python 3000 ;-)

I'm in agreement with this. Since OOP was first popularized, there have 
been various schools of thought which taught that object oriented 
programming was "good style", and any reversion to a more functional or 
procedural syntax was "bad style". I really wish that people would get 
over this.

At the moment, there are, by my count, at least four and perhaps as many 
as a dozen "styles" or program organization, all of which have more or 
less validity in any given situation.

As far as multiple dispatch goes: I would agree here as well, especially 
when we talk about binary operations. For example, suppose we have two 
objects, a and b, which are of types A and B respectively. And suppose 
each of those types overloads both __add__ and __radd__:

   class A:
      def __add__( self, other ):
         ...
      def __radd__( self, other ):
         ...

   class B:
      def __add__( self, other ):
         ...
      def __radd__( self, other ):
         ...

   a = A()
   b = B()

   print a + b   # __add__ wins over __radd__, but should it?
   print b + a

My conclusion: Single dispatch systems are a poor way to specify 
operator overloading. Compare this to the corresponding generic dispatch 
case:

    @generic( A, A )
    def __add__( p0, p1 ):
       ...

    @generic( A, B )
    def __add__( p0, p1 ):
       ...

    @generic( B, B )
    def __add__( p0, p1 ):
       ...

    @generic( B, A )
    def __add__( p0, p1 ):
       ...

With multiple dispatch, its easy to spell out what will happen in each 
possible permutation of arguments - and if you get an ambiguity, the 
dispatcher will let you know about it, instead of simply choosing an 
implementation arbitrarily.

-- Talin


From gsakkis at rutgers.edu  Thu Nov 16 07:32:36 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Thu, 16 Nov 2006 01:32:36 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455BFD43.6070701@acm.org>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<ejeq56$65k$1@sea.gmane.org> <455BFD43.6070701@acm.org>
Message-ID: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>

On 11/16/06, Talin <talin at acm.org> wrote:

> As far as multiple dispatch goes: I would agree here as well, especially
> when we talk about binary operations. For example, suppose we have two
> objects, a and b, which are of types A and B respectively. And suppose
> each of those types overloads both __add__ and __radd__:
>
>    class A:
>       def __add__( self, other ):
>          ...
>       def __radd__( self, other ):
>          ...
>
>    class B:
>       def __add__( self, other ):
>          ...
>       def __radd__( self, other ):
>          ...
>
>    a = A()
>    b = B()
>
>    print a + b   # __add__ wins over __radd__, but should it?
>    print b + a
>
> My conclusion: Single dispatch systems are a poor way to specify
> operator overloading. Compare this to the corresponding generic dispatch
> case:
>
>     @generic( A, A )
>     def __add__( p0, p1 ):
>        ...
>
>     @generic( A, B )
>     def __add__( p0, p1 ):
>        ...
>
>     @generic( B, B )
>     def __add__( p0, p1 ):
>        ...
>
>     @generic( B, A )
>     def __add__( p0, p1 ):
>        ...
>
> With multiple dispatch, its easy to spell out what will happen in each
> possible permutation of arguments - and if you get an ambiguity, the
> dispatcher will let you know about it, instead of simply choosing an
> implementation arbitrarily.

Interesting idea, certainly not mainstream for the moment though.
Short question: how do multiple dispatch systems deal with the
combinatorial explosion ? I guess there should be a way to avoid
having to spell out all combinations every time, or it will soon be
unpractical.

George

From talin at acm.org  Thu Nov 16 07:37:31 2006
From: talin at acm.org (Talin)
Date: Wed, 15 Nov 2006 22:37:31 -0800
Subject: [Python-3000] A plea for anonymous functions
Message-ID: <455C072B.503@acm.org>

 From the beat-that-dead-horse department: I've never really been 
satisfied with the outcome of the discussion on lambda and anonymous 
functions.

As much as I love the use of syntactic indentation in Python, it does 
have one serious drawback, which is that it creates a wall between the 
realms of 'statements' and 'expressions'. Statements are delimited by 
line breaks (generally), while expressions are free-format. The result 
is that while statements can contain expressions, expressions cannot 
contain statements.

None of the scripting languages which are direct competitors to Python 
(in particular, Ruby, Javascript, Perl, Lua, etc.) have this limitation, 
and all of them are able to take very powerful advantage of the ability 
to freely mix statements within expressions.

Javascript, for example, doesn't need special syntactical sugar for 
generator expressions:

    /* Return a list of the square of numbers.
      ('map' appears courtesy of Mochikit.js) */

    result = map( [1, 2, 3, 4], function (a) {
       return a * a;
    })

No, its not quite as concise as a generator expression, but it is far 
more general in what it can do.

Now, there are those who say "Just use a named function", but that 
argument doesn't hold - because the *same* argument can be levied 
against generator expressions, the 'with' statement, and a number of 
other recent syntactical innovations in Python.

 From my point of view, both 'with' and generator expressions are 
limited, special-case solutions to a general problem - the desire to be 
able to use and manipulate unnamed blocks of code as first-class 
objects. Conversely, the same arguments that make 'with' important 
enough to change Python syntax - instead of just saying 'use a named 
function' - apply here as well, only in this case we're not talking 
about a specific use case, but rather a whole class of use cases which 
has yet to be fully explored.

There are also those who would claim that, given the lack of use of 
'lambda' in existing Python code, that there's no compelling use case. 
To this, I have two counter-arguments:

1) This is IMHO a chicken-and-egg problem. Anonymous functions aren't a 
solution to an existing problem, so much as they are an opportunity to 
explore new kinds of solutions to new kinds of problems in Python. As 
long as functions must be named (or crippled, as in the case of Lambda), 
people are going to find workarounds.

2) Many of those 'workarounds' are going to come back in the form of 
PEPs to add new language features. I predict that we'll see a 
never-ending parade of enhancement proposals - lazy evaluation 
expressions, asynchronous callback expressions, and so on - all of which 
are in a sense compensating for the lack of the ability to treat code 
like data.

Instead of attempting to address the problem piecemeal, and thereby 
obfuscating the clean, elegant minimalism of Python (more than it 
already has been), I say we cut to the chase and deal with the root problem.

Of course, I realize that this isn't an easy problem to solve - its 
inherent in Python's syntax choices, and certainly I don't want to give 
those up. But I want to specifically argue against the idea that "it's 
hard to solve, and it's not important, so let's not bother with it". My 
argument is that it may be hard, but it *is* important.

I'm going to refrain from adding any hare-brained syntax proposals of my 
own here - I'm more interested in trying to convince people that this is 
a problem worth bothering with.

I would like, however, to be clear in defining the parameters of the 
problem I would like to see addressed. I am not arguing *specifically* 
for anonymous functions of the 'lambda' type, although that would be one 
possible solution to the problem. What I want is "the ability to treat 
anonymous code blocks as first-class objects", which is a larger space 
of solutions than just lambda functions. For example, a solution that 
didn't involve parameter binding would not be a lambda function, but 
would still meet my criteria.

-- Talin

From jackdied at jackdied.com  Thu Nov 16 07:31:18 2006
From: jackdied at jackdied.com (Jack Diederich)
Date: Thu, 16 Nov 2006 01:31:18 -0500
Subject: [Python-3000] yes to class decorators
In-Reply-To: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
Message-ID: <20061116063118.GC26748@performancedrivers.com>

On Thu, Nov 16, 2006 at 01:06:04AM +0200, tomer filiba wrote:
[tomer filiba wrote stuff]

I was hoping this thread would go unremarked but it looks like
it is picking up some steam.  So I'll throw in my two bits:
Please for the love of god abandon this thread.  It has been done.  
You weren't there.  If you were there you have already said your 
piece.  If you weren't there you are wet behind the ears and won't 
be contributing anything meaningful.

If you really want to post then first:

1) Please go to gmane.org and read all the threads that match
   "class decorators" in the group "gmane.comp.python.devel"
   before posting.

.. and then ..

2a) Feel free to suggest ideas for a class decorator library
    (which is what I think Filiba was offering)
2b) If you want to suggest a new language feature like "make"
   then write a PEP like the "make" folks.
2c) Don't post

I realize this is the py3k list and asking people to keep their
crazy to themselves is an act of utopian optimism but I'm asking
anyway.  Don't poison the well for a simple, consistent, and
pythonic feature like class decorators by piggy backing your
amazing new idea on its adoption.

Did I mention I care about this feature and don't want
a raft of crazies screwing it up with their own vices?

-Jack

NB, if this thread is allowed to die peacefully I'll post a 
patch by the new year (mainly because Christmas is the next
time I'll have a couple days free).

From fredrik at pythonware.com  Thu Nov 16 08:33:50 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 08:33:50 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C072B.503@acm.org>
References: <455C072B.503@acm.org>
Message-ID: <ejh48u$k7n$1@sea.gmane.org>

Talin wrote:

> Javascript, for example, doesn't need special syntactical sugar for 
> generator expressions:
> 
>     /* Return a list of the square of numbers.
>       ('map' appears courtesy of Mochikit.js) */
> 
>     result = map( [1, 2, 3, 4], function (a) {
>        return a * a;
>     })
> 
> No, its not quite as concise as a generator expression, but it is far 
> more general in what it can do.

Are you sure you know what a generator expression is?  Mochikit's map is 
just a JavaScript version of Python's function with the same name; it 
takes an Array or an Array-like object, maps it though a function, and 
returns an Array.  Compare

     http://mochikit.com/doc/html/MochiKit/Base.html#fn-map

with

     http://effbot.org/pyref/map.htm

> Now, there are those who say "Just use a named function", but that 
> argument doesn't hold - because the *same* argument can be levied 
> against generator expressions, the 'with' statement, and a number of 
> other recent syntactical innovations in Python.

Oh, please: Generator expressions (which you might need to study a bit 
further) is generalization of list comprehensions (which is turn is 
syntactical sugar for a for-in statement); and the the "with" statement 
was added mostly because it's *not* very practical to wrap try/finally 
handlers in functions; see

     http://online.effbot.org/2006_10_01_archive.htm#with

for more on this.

> From my point of view, both 'with' and generator expressions are 
> limited, special-case solutions to a general problem - the desire to be 
> able to use and manipulate unnamed blocks of code as first-class 
> objects.

Python has two full block constructs: "for-in" (which has been in there 
from the start) and "with" (added in 2.5).  The "for-in" block construct 
is discussed here:

     http://online.effbot.org/2006_11_01_archive.htm#for

As I mention in a comment in the with-article, it's all about use cases: 
do 'for-in' blocks and 'with' blocks cover most of the use cases for 
which a full-blown callback isn't a conceptually cleaner thing to use 
anyway?

The current Python hypothesis is 'yes, they do'.

To prove that they don't, you *have* to provide use cases, and show how 
they would look and feel.  Hand-waving arguments and vague promises of 
"once you reach my level of sophistication, you'll understand" isn't 
good enough.

(And frankly, it would be *wonderful* if someone could come up with a 
new proposal that actually enabled Python programmers to do things they 
*cannot* do today, instead of just rehashing old "let's move the sofa 
over there" threads.  How about doing something along the lines of 
LINQ's "give me this expression as an AST so I can optimize it myself" 
model or looking at COmega's parallelization/synchronization stuff or 
digging deeper into how PJE's RuleDispatch could be fit into Python or 
stealing some cool idea from some functional research language that I 
haven't even heard of.  I want to get *new* things when I upgrade from 
2.X to 3.0, not just silly syntax tweaks that would only give me the 
ability to change two lines of Python code to one line of code plus a 
comment that explains what the heck that line is doing.  Let's do some 
*hard* stuff, for a change.)

</F>


From krstic at solarsail.hcs.harvard.edu  Thu Nov 16 09:33:23 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Thu, 16 Nov 2006 03:33:23 -0500
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejh48u$k7n$1@sea.gmane.org>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
Message-ID: <455C2253.5090909@solarsail.hcs.harvard.edu>

Fredrik Lundh wrote:
> And frankly, it would be *wonderful* if someone could come up with a 
> new proposal that actually enabled Python programmers to do things they 
> *cannot* do today, instead of just rehashing old "let's move the sofa 
> over there" threads.  

Hear, hear.

> I want to get *new* things when I upgrade from 
> 2.X to 3.0, not just silly syntax tweaks that would only give me the 
> ability to change two lines of Python code to one line of code plus a 
> comment that explains what the heck that line is doing. 

>From my understanding of the purpose of 3.0, you're looking at the wrong
release. 3.0 is the "accumulated cruft removal" release, with features
becoming the focus of the 3.1 cycle and subsequent releases.

> Let's do some *hard* stuff, for a change.

I couldn't agree more. The fundamental problem that I see, though, is
that Guido seems strongly opposed to adding any additional
metaprogramming features to the language [0], and solving much of the
hard stuff elegantly and without special-cased, one-off solutions,
requires these.

LINQ-like features certainly require having first-class code blocks. And
for a couple of months, I've been jotting down notes, in preparation for
a PEP, on how best to add multiprocessing support to Python given the
GIL and the fact that it's not going away in the near future. There are
a number of attractive solutions -- some of them quite beautiful, and I
can provide examples -- that can be used here, but they all make the
same requirement as LINQ if they're to not look like utter warts.

The bottom line, I think, is that we need generic language support for
"hard stuff", and as Lispers figured out a while back, that support /is/
metaprogramming. Guido does not wish to make Python syntax any more
programmable, however, while at the same time features like metaclasses
tempt us just enough to feel the possibilities are within reach.




[0] See second to last paragraph:
http://mail.python.org/pipermail/python-3000/2006-April/000286.html

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From fredrik at pythonware.com  Thu Nov 16 09:55:11 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 09:55:11 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C2253.5090909@solarsail.hcs.harvard.edu>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
	<455C2253.5090909@solarsail.hcs.harvard.edu>
Message-ID: <ejh91f$2dj$1@sea.gmane.org>

Ivan Krsti? wrote:

>> Let's do some *hard* stuff, for a change.
> 
> I couldn't agree more. The fundamental problem that I see, though, is
> that Guido seems strongly opposed to adding any additional
> metaprogramming features to the language [0], and solving much of the
> hard stuff elegantly and without special-cased, one-off solutions,
> requires these.
> 
> LINQ-like features certainly require having first-class code blocks.

I'm not convinced that they do -- you can emulate LINQ today with 
generators and iterator tools and good old for-in statements:

http://sayspy.blogspot.com/2006/02/why-python-doesnt-need-something-like.html

but as noted in the comments, what's missing is a way for the container 
to do query optimizations based on the *entire* filter expression.  if 
we can come up with a decent mechanism (*) for that, the existing for-in 
block construct is good enough for the rest.

more later.

</F>

*) there are plenty of hacks to address parts of this; everything from
algebra on custom objects (used by various SQL construction kits, RE 
construction kits, etc), code that analyzes an expression by passing 
special AST-building objects through it (used in PIL's point method, for 
example), and code that requires you to pass in the expression as a text 
string (or a ready-made "compiler" module AST-tree).


From talin at acm.org  Thu Nov 16 09:58:47 2006
From: talin at acm.org (Talin)
Date: Thu, 16 Nov 2006 00:58:47 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejh48u$k7n$1@sea.gmane.org>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
Message-ID: <455C2847.50903@acm.org>

Fredrik Lundh wrote:
> Talin wrote:
> 
>> Javascript, for example, doesn't need special syntactical sugar for 
>> generator expressions:
>>
>>     /* Return a list of the square of numbers.
>>       ('map' appears courtesy of Mochikit.js) */
>>
>>     result = map( [1, 2, 3, 4], function (a) {
>>        return a * a;
>>     })
>>
>> No, its not quite as concise as a generator expression, but it is far 
>> more general in what it can do.
> 
> Are you sure you know what a generator expression is?  Mochikit's map is 
> just a JavaScript version of Python's function with the same name; it 
> takes an Array or an Array-like object, maps it though a function, and 
> returns an Array.  Compare

You are correct that 'map' isn't equivalent to a generator expression - 
my mistake. However, replacing 'map' with 'itertools.imap' gives you 
something roughly equivalent.

> To prove that they don't, you *have* to provide use cases, and show how 
> they would look and feel.  Hand-waving arguments and vague promises of 
> "once you reach my level of sophistication, you'll understand" isn't 
> good enough.

The problem with providing use cases is that each one individually 
doesn't carry much weight. When I do Javascript/AJAX programming, I use 
anonymous functions quite a bit, as in the following example:

    do_command( 'get_synset_links', {}, function( response_data ) {
       format_expanded( li.content, response_data.related )
    })

'do_command' calls a function on the server side via an async HTTP 
request, and calls the callback when the data is ready. By using an 
anonymous function as the callback, I can group the request for the data 
and the use of that data together in a logical way.

Could I do the same thing using a named callback? Sure. Would it be as 
easy to read and as clear? Not in my opinion. With a named callback, we 
would have to put the code that uses the data in a different place than 
the code that fetches the data. We would have to write the code so that 
the part that executes later appears earlier in the source file.

But it doesn't matter, because no matter what cases I come up with, 
someone will claim "You're just saving 2 lines of code". The fact that 
it saves 2 lines of code in many different places in my program can't 
easily be illustrated in an email message.

> (And frankly, it would be *wonderful* if someone could come up with a 
> new proposal that actually enabled Python programmers to do things they 
> *cannot* do today, instead of just rehashing old "let's move the sofa 
> over there" threads.  How about doing something along the lines of 
> LINQ's "give me this expression as an AST so I can optimize it myself" 
> model or looking at COmega's parallelization/synchronization stuff or 
> digging deeper into how PJE's RuleDispatch could be fit into Python or 
> stealing some cool idea from some functional research language that I 
> haven't even heard of.  I want to get *new* things when I upgrade from 
> 2.X to 3.0, not just silly syntax tweaks that would only give me the 
> ability to change two lines of Python code to one line of code plus a 
> comment that explains what the heck that line is doing.  Let's do some 
> *hard* stuff, for a change.)

I would disagree with one bit: I think most of those things can be done 
in Python today, they are just complicated and ugly under the current 
syntax. As much as people decry 'syntactic sugar', I would argue that 
those things you mentioned (even the AST example, although it's tricky) 
really just need sugar to make them feasible and useful to ordinary 
programmers, rather than any fundamental change to the execution model.

In my experience, most of the 'hard stuff' is built out of powerful, 
low-level language primitives - most of which Python already has. Those 
low-level primitives don't seem particularly powerful until you start 
combining them in synergetic ways. This is particularly well illustrated 
in Scheme, where many of the modern programming features - such as 
OOP-style single method dispatch - can be 'built up' out of more 
fundamental features like closures.

And I would argue that being able to treat 'code as data' is one of 
those fundamentals. We can already do this to some extent, but there's a 
long way to go still as far as intuitiveness and ease of use goes.

(Although I just read the last paragraph Ivan Krsti?'s response as I was 
typing this, and I think he says it better than I can, so I'll stop.)

-- Talin

From krstic at solarsail.hcs.harvard.edu  Thu Nov 16 10:35:19 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Thu, 16 Nov 2006 04:35:19 -0500
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejh91f$2dj$1@sea.gmane.org>
References: <455C072B.503@acm.org>
	<ejh48u$k7n$1@sea.gmane.org>	<455C2253.5090909@solarsail.hcs.harvard.edu>
	<ejh91f$2dj$1@sea.gmane.org>
Message-ID: <455C30D7.1020701@solarsail.hcs.harvard.edu>

Fredrik Lundh wrote:
> I'm not convinced that they do -- you can emulate LINQ today with 
> generators and iterator tools and good old for-in statements:

No, you can't. You can emulate a tiny subset of it, as you yourself
note. For people following from home: when you query a database with
LINQ, C# can construct the actual SQL *query* by introspecting the AST;
you can't do that without first-class code blocks, unless you get ugly.
And because the language is aware of what you're querying, it can
compile the same LINQ query to SQL, or XQuery, or something else
entirely, as needed.

> *) there are plenty of hacks to address parts of this; everything from
> algebra on custom objects (used by various SQL construction kits, RE 
> construction kits, etc), code that analyzes an expression by passing 
> special AST-building objects through it (used in PIL's point method, for 
> example), and code that requires you to pass in the expression as a text 
> string (or a ready-made "compiler" module AST-tree).

Please distinguish hacks from language support, and note what I said:
"... but they all make the same requirement as LINQ if they're to not
look like utter warts." So yes, these can be done (to a degree), but
they're all nasty solutions, and the fact we have a legitimate need for
them -- I brought up SQLObject before in this context -- tells me people
are already making Python behave non-obviously (which was Guido's fear),
just in exceedingly ugly and arbitrarily limited ways.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From fredrik at pythonware.com  Thu Nov 16 10:50:56 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 10:50:56 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C2847.50903@acm.org>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
	<455C2847.50903@acm.org>
Message-ID: <ejhca0$cqt$1@sea.gmane.org>

Talin wrote:

> The problem with providing use cases is that each one individually 
> doesn't carry much weight. When I do Javascript/AJAX programming, I use 
> anonymous functions quite a bit, as in the following example:
> 
>     do_command( 'get_synset_links', {}, function( response_data ) {
>        format_expanded( li.content, response_data.related )
>     })
> 
> 'do_command' calls a function on the server side via an async HTTP 
> request, and calls the callback when the data is ready. By using an 
> anonymous function as the callback, I can group the request for the data 
> and the use of that data together in a logical way.

Your specific example would probably be written:

     server.get_synset_links(
         lambda rec: format_expanded(li.content, rec.related)
     )

in today's Python, which I find a bit easier to read than your slightly 
bloated JavaScript example.

How do you handle errors, btw?

> Could I do the same thing using a named callback? Sure. Would it be as 
> easy to read and as clear?

Well, if you find that easy to read and clear, I suspect you're more of 
a JavaScript programmer than a Python programmer.  Here's how a more 
general form of the above would look in Python:

     response = (yield server.get_synset_links())
     format_expanded(li.content, response.related)

or

     def handle_response(record):
         format_expanded(li.content, response.related)
     server.get_synset_link(handle_response)

or perhaps

     with async server.get_synset_link() as response:
         format_expanded(li.content, response.related)

(all of which are shorter than your JavaScript example, by the way)

> But it doesn't matter, because no matter what cases I come up with, 
> someone will claim "You're just saving 2 lines of code".

No, I said that under your proposal, you're trading two lines of code 
that looks like Python for two (or three) lines of code that looks like 
something else.

 > The fact that it saves 2 lines of code in many different places in
 > my program can't easily be illustrated in an email message.

Well, this far, you haven't even been able to illustrate how your 
unspecified block syntax would have saved you a single line even in your 
three-line example.  That's not a good start, really.

> I would disagree with one bit: I think most of those things can be done 
> in Python today, they are just complicated and ugly under the current 
> syntax. As much as people decry 'syntactic sugar', I would argue that 
> those things you mentioned (even the AST example, although it's tricky) 
> really just need sugar to make them feasible and useful to ordinary 
> programmers, rather than any fundamental change to the execution model.

So show us some sugar, then.  After all, Python's design is driven by by 
an urge to simplify real usage patterns that's been observed in the 
wild, rather than theoretically-useful-for-everything-maybe language 
constructs that's been dreamed up in some laboratory.  And all the 
things I've mentioned *are* used in real code.

Also see the "Patterns are signs of weakness in programming languages" 
discussions:

    http://newbabe.pobox.com/~mjd/blog/prog/design-patterns.html

and note that Python's usually aiming for "(nearly) invisible", not 
"formal", in Norvig's taxonomy.

Also see the part about use cases for block constructs in my original 
post to this thread.

</F>


From fredrik at pythonware.com  Thu Nov 16 11:01:05 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 11:01:05 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C30D7.1020701@solarsail.hcs.harvard.edu>
References: <455C072B.503@acm.org>	<ejh48u$k7n$1@sea.gmane.org>	<455C2253.5090909@solarsail.hcs.harvard.edu>	<ejh91f$2dj$1@sea.gmane.org>
	<455C30D7.1020701@solarsail.hcs.harvard.edu>
Message-ID: <ejhct1$est$1@sea.gmane.org>

Ivan Krsti? wrote:

>> I'm not convinced that they do -- you can emulate LINQ today with 
>> generators and iterator tools and good old for-in statements:
> 
> No, you can't.

my sentence didn't end at that colon, though.

> For people following from home: when you query a database with
> LINQ, C# can construct the actual SQL *query* by introspecting the AST;
> you can't do that without first-class code blocks

of course you can.  all you need is the AST for the *query*, not the 
rest of the code.

     for record in (<this part is important>):
         process record

     gen = (for record in (<this part is important>))

e.g. something like

     for record in (select table: <generator expression on table>):
         process record

where the table is able to either execute the generator expression on 
itself, or analyze the AST and do an optimized query.

> And because the language is aware of what you're querying, it can
> compile the same LINQ query to SQL, or XQuery, or something else
> entirely, as needed.

well, Python's a dynamic language, so that has to be done by the 
collection, on the fly.  but that's not a problem, really.

>> *) there are plenty of hacks to address parts of this; everything from
>> algebra on custom objects (used by various SQL construction kits, RE 
>> construction kits, etc), code that analyzes an expression by passing 
>> special AST-building objects through it (used in PIL's point method, for 
>> example), and code that requires you to pass in the expression as a text 
>> string (or a ready-made "compiler" module AST-tree).
> 
> Please distinguish hacks from language support

oh, my use of "decent mechanism" implied that the things I labelled as 
"hacks" in the footnote weren't decent enough.

if you read my post again, you'll find that I said pretty much the same 
thing as you're saying, minus the "we absolutely must have full support 
for anonymous blocks to be able to do this" part.

</F>


From krstic at solarsail.hcs.harvard.edu  Thu Nov 16 11:10:40 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Thu, 16 Nov 2006 05:10:40 -0500
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejhct1$est$1@sea.gmane.org>
References: <455C072B.503@acm.org>	<ejh48u$k7n$1@sea.gmane.org>	<455C2253.5090909@solarsail.hcs.harvard.edu>	<ejh91f$2dj$1@sea.gmane.org>	<455C30D7.1020701@solarsail.hcs.harvard.edu>
	<ejhct1$est$1@sea.gmane.org>
Message-ID: <455C3920.4030509@solarsail.hcs.harvard.edu>

Fredrik Lundh wrote:
>      for record in (select table: <generator expression on table>):
>          process record

This is precisely what I had in mind when I referred to "special-cased,
one-off solutions" two posts back.

> if you read my post again, you'll find that I said pretty much the same 
> thing as you're saying, minus the "we absolutely must have full support 
> for anonymous blocks to be able to do this" part.

That's not what I'm saying. What I'm saying is "we absolutely must have
full support for anonblocks to be able to do this *elegantly*", i.e.
while avoiding special-cased solutions. The LINQ special case given
above won't help me in building proper multiprocessing support. Now,
maybe special-cased solutions are preferred to more metaprogramming, in
which case there's not much point in continuing the discussion.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From fredrik at pythonware.com  Thu Nov 16 11:22:52 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 11:22:52 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C3920.4030509@solarsail.hcs.harvard.edu>
References: <455C072B.503@acm.org>	<ejh48u$k7n$1@sea.gmane.org>	<455C2253.5090909@solarsail.hcs.harvard.edu>	<ejh91f$2dj$1@sea.gmane.org>	<455C30D7.1020701@solarsail.hcs.harvard.edu>	<ejhct1$est$1@sea.gmane.org>
	<455C3920.4030509@solarsail.hcs.harvard.edu>
Message-ID: <ejhe5r$j68$1@sea.gmane.org>

Ivan Krsti? wrote:

> That's not what I'm saying. What I'm saying is "we absolutely must have
> full support for anonblocks to be able to do this *elegantly*", i.e.
> while avoiding special-cased solutions.

As I've tried to say several times in my other posts in this thread, 
from a theoretical computer science perspective, Python's all about 
"special cases".

Python's design values elegant solutions for groups of *practical* 
problems observed in the wild, over hyper-generalized solutions to 
problems designed in a laboratory.  That's what makes the language so 
darn practical to use ;-)

</F>


From talin at acm.org  Thu Nov 16 11:50:50 2006
From: talin at acm.org (Talin)
Date: Thu, 16 Nov 2006 02:50:50 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejhca0$cqt$1@sea.gmane.org>
References: <455C072B.503@acm.org>
	<ejh48u$k7n$1@sea.gmane.org>	<455C2847.50903@acm.org>
	<ejhca0$cqt$1@sea.gmane.org>
Message-ID: <455C428A.6020207@acm.org>

Fredrik Lundh wrote:
> Talin wrote:
> 
>> The problem with providing use cases is that each one individually 
>> doesn't carry much weight. When I do Javascript/AJAX programming, I use 
>> anonymous functions quite a bit, as in the following example:
>>
>>     do_command( 'get_synset_links', {}, function( response_data ) {
>>        format_expanded( li.content, response_data.related )
>>     })
>>
>> 'do_command' calls a function on the server side via an async HTTP 
>> request, and calls the callback when the data is ready. By using an 
>> anonymous function as the callback, I can group the request for the data 
>> and the use of that data together in a logical way.
> 
> Your specific example would probably be written:
> 
>      server.get_synset_links(
>          lambda rec: format_expanded(li.content, rec.related)
>      )

Only if the code in the block is a one-liner.

> in today's Python, which I find a bit easier to read than your slightly 
> bloated JavaScript example.
> 
> How do you handle errors, btw?

For this app, internally in the do_command function. In any case, error 
handling code can be assumed to have been omitted for clarity.

>> Could I do the same thing using a named callback? Sure. Would it be as 
>> easy to read and as clear?
> 
> Well, if you find that easy to read and clear, I suspect you're more of 
> a JavaScript programmer than a Python programmer.  Here's how a more 
> general form of the above would look in Python:
> 
>      response = (yield server.get_synset_links())
>      format_expanded(li.content, response.related)
> 
> or
> 
>      def handle_response(record):
>          format_expanded(li.content, response.related)
>      server.get_synset_link(handle_response)
> 
> or perhaps
> 
>      with async server.get_synset_link() as response:
>          format_expanded(li.content, response.related)

None of these are functionally equivalent to my code example, if I am 
understanding them correctly.

But I guess I need to explain a bit more about my example.

First, 'get_synset_links' is a URL, not a method name. (More 
technically, it's a fragment of a URL). I suppose you could play tricks 
with __getitem__ to turn a method call into a URL, but I wouldn't do 
that - for one thing, not all URLs can be represented as method names.

More importantly, the flow of execution is different:

    do_command( stuff ) {
       // This code executes when the HTTP response from the
       // server returns.
       ... code ...
    }

    // This code executes immediately after the request has
    // been sent.
    ... code ...

In other words, 'do_command' sends a request to the server and continues 
processing after the block. When the response returns, the code inside 
the block is executed. I don't think you can emulate this behavior with 
either yield or with - they are both synchronous operations.

So I can see how your example #2 might be made to work this way, but not 
#1 or #3. And I'm not convinced that #2 is really better.

More generally, the notion here is to implement a rough kind of 
continuation, based on external asynchronous events. There are lots and 
lots of potential use cases for code that handles asynchronous responses 
to program events.

> (all of which are shorter than your JavaScript example, by the way)
> 
>> But it doesn't matter, because no matter what cases I come up with, 
>> someone will claim "You're just saving 2 lines of code".
> 
> No, I said that under your proposal, you're trading two lines of code 
> that looks like Python for two (or three) lines of code that looks like 
> something else.

Lets not get confused here - I'm illustrating in Javascript code because 
this is something that's easy to do in Javascript (or Lua or Perl or 
whatever) but hard to do in Python.

Since I haven't actually proposed what it would look like in Python, I'm 
not sure that it makes sense to argue that it doesn't "look like" 
Python. That remains to be seen.

>  > The fact that it saves 2 lines of code in many different places in
>  > my program can't easily be illustrated in an email message.
> 
> Well, this far, you haven't even been able to illustrate how your 
> unspecified block syntax would have saved you a single line even in your 
> three-line example.  That's not a good start, really.

Lets try that one again, now that I (hopefully) have given a clearer 
explanation of what I am trying to do.

>> I would disagree with one bit: I think most of those things can be done 
>> in Python today, they are just complicated and ugly under the current 
>> syntax. As much as people decry 'syntactic sugar', I would argue that 
>> those things you mentioned (even the AST example, although it's tricky) 
>> really just need sugar to make them feasible and useful to ordinary 
>> programmers, rather than any fundamental change to the execution model.
> 
> So show us some sugar, then.  After all, Python's design is driven by by 
> an urge to simplify real usage patterns that's been observed in the 
> wild, rather than theoretically-useful-for-everything-maybe language 
> constructs that's been dreamed up in some laboratory.  And all the 
> things I've mentioned *are* used in real code.

Well, I'm not going to give solutions for all of the examples you 
raised, because (a) its off-topic, and (b) it's not necessary. But I 
will take two examples from your list: The LINQ example and the 
RuleDispatch example.

For the LINQ example, all I need to do is point to SQLObject, which does 
  something very similar - it builds an SQL query using Python 
expression syntax. We could improve SQLObjects's syntax quite a bit by 
allowing overloading of the 'and' and 'or' operator - right now, it has 
to use '&' and '|', which have the wrong precedence and require extra 
parens.

For RuleDispatch, we already know it works in Python, the only question 
is how to make it easier to use. And that's been the subject of a number 
of threads on this list, of which there are two parts: How to write the 
dispatcher, and how to express the dispatch constrains in syntax.

Writing the dispatcher isn't a problem - I wrote one for my embedded 
mini-language, Saga, in a day or two just by looking up the 
Chambers/Chen paper on ACM library.

So really it all boils down to syntax.

> Also see the "Patterns are signs of weakness in programming languages" 
> discussions:
> 
>     http://newbabe.pobox.com/~mjd/blog/prog/design-patterns.html

OK I read it - I think the author is overgeneralizing.

> and note that Python's usually aiming for "(nearly) invisible", not 
> "formal", in Norvig's taxonomy.
> 
> Also see the part about use cases for block constructs in my original 
> post to this thread.

-- Talin

From fredrik at pythonware.com  Thu Nov 16 12:03:05 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 12:03:05 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C428A.6020207@acm.org>
References: <455C072B.503@acm.org>	<ejh48u$k7n$1@sea.gmane.org>	<455C2847.50903@acm.org>	<ejhca0$cqt$1@sea.gmane.org>
	<455C428A.6020207@acm.org>
Message-ID: <ejhgh9$r8p$1@sea.gmane.org>

Talin wrote:

> Since I haven't actually proposed what it would look like in Python, I'm 
> not sure that it makes sense to argue that it doesn't "look like" 
> Python. That remains to be seen.

wow.

</F>


From tomerfiliba at gmail.com  Thu Nov 16 12:24:13 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Thu, 16 Nov 2006 13:24:13 +0200
Subject: [Python-3000] yes to class decorators
Message-ID: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>

several things:

[Josiah]
> I believe that Tomer just wanted to remind people that class decorators
> are slated for 2.6 and 3.0 if there was a PEP, and perhaps that he is
> encouraging someone to write one.

yeah, i just wanted to bring it up again. however, i do have some
spare time at the moment, so i can write the pep myself.
but before someone writes a pep, it's better to discuss the issue
at hand.

currently, to my understanding, this is what we have:

("@" expression "\n")*
"class" name ["(" expression "):"]
    clause

we should find some more/better use-cases than what i came
up with, and show why class decorators are the right solution,
and compare with function decorators.

we should also note that it's possible to replace "__metaclass__"
by a @metaclass decorator. of course this requires more thinking.

- - - - - - - - -

[Jack Diederich]
> I realize this is the py3k list and asking people to keep their
> crazy to themselves is an act of utopian optimism but I'm asking
> anyway.

MWHAHAAHAHAHAHAHAHA!!!111!!^&%^#%&@
*shoves own hand into the oven*

> Did I mention I care about this feature and don't want
> a raft of crazies screwing it up with their own vices?

i don't get your point. i DID read many posts about class
decorators, the most recent of which i've seen were at least half
a year old.  since we have Guido's blessing on the subject,
i wanted to get the wheels moving.

of course i couldn't read EVERYTHING that was said about
class decorators, so i added a couple of use-cases i found.
in fact, the reason why i posted yesterday was because i wrote a
singleton class, and wished i had @singleton instead of

class Foo(object):
    pass
Foo = Foo()

so if we put aside your Stalinistic behavior for the moment,
do YOU want to write the pep? please do.

i don't see why this thread bothers you so much, but you could
have prevented it by writing a pep months ago.
now isn't that motivating? :)


-tomer

From fredrik at pythonware.com  Thu Nov 16 12:34:33 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 12:34:33 +0100
Subject: [Python-3000] yes to class decorators
In-Reply-To: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
References: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
Message-ID: <ejhic8$19n$1@sea.gmane.org>

tomer filiba wrote:

>> Did I mention I care about this feature and don't want
>> a raft of crazies screwing it up with their own vices?
> 
> i don't get your point.

maybe your news server is dropping posts ?

</F>


From meyer at acm.org  Thu Nov 16 12:36:47 2006
From: meyer at acm.org (Andre Meyer)
Date: Thu, 16 Nov 2006 12:36:47 +0100
Subject: [Python-3000] yes to class decorators
In-Reply-To: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
References: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
Message-ID: <7008329d0611160336u5e8e71fex68a9b376ad5fb1af@mail.gmail.com>

On 11/16/06, tomer filiba <tomerfiliba at gmail.com> wrote:
>
>
> we should find some more/better use-cases than what i came
> up with, and show why class decorators are the right solution,
> and compare with function decorators.


A while ago I was trying to implement the Singleton or Borg design pattern
and thought it could be elegantly made accessible via a class decorator. It
proved otherwise. Maybe this is a useful use-case for you.

regards
Andr?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061116/99f34dde/attachment.htm 

From fredrik at pythonware.com  Thu Nov 16 12:44:30 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 12:44:30 +0100
Subject: [Python-3000] native support for multiple dispatch in 3.0 ?
Message-ID: <ejhiuu$3ej$1@sea.gmane.org>

since it seems to be impossible to bring up new ideas in subthreads, I 
though I'd give this one its own thread:

- has there been any discussions on making RuleDispatch-type mechanisms
   available on a more fundamental level?

could someone perhaps come up with a list of real-life uses for PJE's 
existing dispatcher implementation?

or should we go all the way: how about using this mechanism for *all* 
dispatching, even the usual dispatch-on-instance handler?  and poly- 
morphic builtins too, while we're at it?

(the first person to bring up anonymous blocks in this thread will be 
duly *plonked* ;-)

</F>


From solipsis at pitrou.net  Thu Nov 16 14:49:10 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 16 Nov 2006 14:49:10 +0100 (CET)
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejhca0$cqt$1@sea.gmane.org>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
	<455C2847.50903@acm.org> <ejhca0$cqt$1@sea.gmane.org>
Message-ID: <52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>


Hi,

> Well, if you find that easy to read and clear, I suspect you're more of
> a JavaScript programmer than a Python programmer.  Here's how a more
> general form of the above would look in Python:
>
>      response = (yield server.get_synset_links())
>      format_expanded(li.content, response.related)

This assumes there is an underlying event loop of some sort that accepts
and is able to schedule generators.
Much more annoyingly, all your "yield"'s must be in the same scope, they
can't be spread in various subfunctions called from the generator. It is a
massive limitation to writing clean code, unless your generators are very
short.

>      def handle_response(record):
>          format_expanded(li.content, response.related)
>      server.get_synset_link(handle_response)

As already mentioned, the problem with this idiom is that code that is
executed *after* appears *before* the "get_synset_link".
It makes the code less straightforward to write and especially to read
(and having readable code is important).

>      with async server.get_synset_link() as response:
>          format_expanded(li.content, response.related)

How is this one supposed to work ? What is "with async blah()" ?

> No, I said that under your proposal, you're trading two lines of code
> that looks like Python for two (or three) lines of code that looks like
> something else.

The problem is not merely related to the number of lines of code. It is a
readibility problem. Having the callback enclosed in a the function call
which registers it makes the code more readable than defining the callback
beforehand.
Saying this does *not* mean one is a Javascript programmer (I hate
Javascript, but this particular point of Javascript is superior to what
Python has to offer).

> So show us some sugar, then.  After all, Python's design is driven by by
> an urge to simplify real usage patterns that's been observed in the
> wild,

Real usage patterns of event-driven code can be seen everywhere from GUI
applications to network applications (written with Twisted).
In many applications it is arguably much more common than the
"try..finally" constructs that "with" attempts to simplify.

I'm not sure what more is needed here? Addresses of SVN repositories?

> Also see the "Patterns are signs of weakness in programming languages"
> discussions:

I would agree with this argument if I could understand how it relates to
the current discussion. Are you saying anonymous functions are a "design
pattern" ? (then what is *not* a design pattern ?).

> and note that Python's usually aiming for "(nearly) invisible", not
> "formal", in Norvig's taxonomy.

What is more "invisible", including the callback straight at the point
where it is registered, or defining it beforehand?


The only thing I can agree with here is that anonymous functions or code
blocks are very hard to design inside the current Python syntax. But
saying there is no point in anonymous functions is a plain lie.

Regards

Antoine.



From murman at gmail.com  Thu Nov 16 15:06:31 2006
From: murman at gmail.com (Michael Urman)
Date: Thu, 16 Nov 2006 08:06:31 -0600
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
	<455C2847.50903@acm.org> <ejhca0$cqt$1@sea.gmane.org>
	<52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
Message-ID: <dcbbbb410611160606i4941521fp11aafce50d68bdda@mail.gmail.com>

> The problem is not merely related to the number of lines of code. It is a
> readibility problem. Having the callback enclosed in a the function call
> which registers it makes the code more readable than defining the callback
> beforehand.

And, unless I'm missing something, much harder to unit test. Is that
really worth encouraging for handlers with complexity beyond the
current lambda?

-- 
Michael Urman  http://www.tortall.net/mu/blog

From fredrik at pythonware.com  Thu Nov 16 15:54:20 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 16 Nov 2006 15:54:20 +0100
Subject: [Python-3000] A plea for anonymous functions
References: <455C072B.503@acm.org>
	<ejh48u$k7n$1@sea.gmane.org><455C2847.50903@acm.org>
	<ejhca0$cqt$1@sea.gmane.org>
	<52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
Message-ID: <ejhu2t$gli$1@sea.gmane.org>

Antoine wrote:

> >      response = (yield server.get_synset_links())
> >      format_expanded(li.content, response.related)
>
> This assumes there is an underlying event loop of some sort that accepts
> and is able to schedule generators.

no, it only assumes that whoever called your code is set up to schedule
communication tasks on your behalf.  that's how asyncronous network
stuff is usually done, so I'm not sure why you're using this as a counter-
argument.

> Much more annoyingly, all your "yield"'s must be in the same scope, they
> can't be spread in various subfunctions called from the generator

dealing with nested generators is simple; a bit tedious, but far from impossible.

>>      def handle_response(record):
>>          format_expanded(li.content, response.related)
>>      server.get_synset_link(handle_response)
>
> As already mentioned, the problem with this idiom is that code that is
> executed *after* appears *before* the "get_synset_link".
> It makes the code less straightforward to write and especially to read
> (and having readable code is important).

why?  it *separates* the two things, and lets you concentrate on one thing at a
time.  I'm far from convinced that that's a drawback, especially when both the
request construction and the response handlers gets more complex than what
you can fit in a line of code or three.

it also makes it a *lot* easier to test and debug the callbacks, makes it easier
to refactor both the request-generating code and the response handlers, makes
it easier to reuse response handlers for related tasks, is self-documenting, given
appropriate callback names, and generally, in my experience, leads to much
better code quality.

>>      with async server.get_synset_link() as response:
>>          format_expanded(li.content, response.related)
>
> How is this one supposed to work ? What is "with async blah()" ?

a magic thing that runs the code block behind the covers, of course.  requires a
non-released version of the from __future__ statement, at the time.  unlike the
following syntax:

@server.get_synset_link()
def block(response):
    format_expanded(li.content, response.related)

which actually works in today's Python.

>> No, I said that under your proposal, you're trading two lines of code
>> that looks like Python for two (or three) lines of code that looks like
>> something else.
>
> The problem is not merely related to the number of lines of code. It is a
> readibility problem. Having the callback enclosed in a the function call
> which registers it makes the code more readable than defining the callback
> beforehand.

but here we're back to the "it seems to me" kind of argumentation.  show us the
code, and don't just argue that you really meant to post something else when people
point out how lousy your example is.

> Saying this does *not* mean one is a Javascript programmer (I hate
> Javascript, but this particular point of Javascript is superior to what
> Python has to offer).
>
>> So show us some sugar, then.  After all, Python's design is driven by by
>> an urge to simplify real usage patterns that's been observed in the
>> wild,
>
> Real usage patterns of event-driven code can be seen everywhere from GUI
> applications to network applications (written with Twisted).
> In many applications it is arguably much more common than the
> "try..finally" constructs that "with" attempts to simplify.

yeah, exactly.  so what kind of sugar would help event-driven programmers to
write better, shorter, more maintainable code ?  you and tomer want more onions
in your soup; perhaps there are other Twisted hackers out there with other ideas ?

>> Also see the "Patterns are signs of weakness in programming languages"
>> discussions:
>
> I would agree with this argument if I could understand how it relates to
> the current discussion. Are you saying anonymous functions are a "design
> pattern" ? (then what is *not* a design pattern ?).

no, the code that you argue would improved by anonymous blocks implement
a specific pattern, of course (a handler for an asyncronous event).  you're still
stuck thinking on mechanisms, I'm trying to get you to talk about use cases.

> What is more "invisible", including the callback straight at the point
> where it is registered, or defining it beforehand?

an invisible solution would be let you write asyncronous event handlers in a way
where you weren't even noticing that you were creating callbacks.  I have no idea
what it would look like, but ideally, it should be a bit like anonymous blocks in
for-in loops: they're been in there for ages, everyone is using them, but people
still claim that Python doesn't have anonymous blocks at all.

</F> 




From jimjjewett at gmail.com  Thu Nov 16 17:37:48 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 16 Nov 2006 11:37:48 -0500
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C072B.503@acm.org>
References: <455C072B.503@acm.org>
Message-ID: <fb6fbf560611160837k287b8652oba0fc2eef1430448@mail.gmail.com>

On 11/16/06, Talin <talin at acm.org> wrote:
>  From my point of view, both 'with' and generator expressions are
> limited, special-case solutions to a general problem - the desire to be
> able to use and manipulate unnamed blocks of code as first-class
> objects.

Other than your callback of a curried function, I don't think the
suites really need to be anonymous; first class is enough.

And even that may be overkill; I didn't see any examples that didn't
boil down to "If I wrote this normally, it would execute in the wrong
namespace."   (I may be undervaluing the "but I haven't defined it
*yet*" portion of your examples.)

> Conversely, the same arguments that make 'with' important
> enough to change Python syntax - instead of just saying 'use a named
> function'

The wrapping portions often are named.  The point of "with" is to tie
them together as a single named function instead of two paired
functions.

The center suite (inside the "with") can't always be replaced by a
function, and that is because it needs to be evaluated in the same
scope as the suites before and after the with block.

> 2) Many of those 'workarounds' are going to come back in the form of
> PEPs to add new language features. I predict that we'll see a
> never-ending parade of enhancement proposals - lazy evaluation
> expressions, asynchronous callback expressions, and so on - all of which
> are in a sense compensating for the lack of the ability to treat code
> like data.

If not for the special-case treatment of locals and globals, I can't
quite think of an example that couldn't be rewritten as a suite, and
called with exec.

Just for nostalgia...

    suite core:
        stmt1
        stmt2

-jJ

From gmccaughan at synaptics-uk.com  Thu Nov 16 16:39:11 2006
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu, 16 Nov 2006 16:39:11 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<455BFD43.6070701@acm.org>
	<91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
Message-ID: <200611161539.12577.gmccaughan@synaptics-uk.com>

[Talin:]
> > With multiple dispatch, its easy to spell out what will happen in each
> > possible permutation of arguments - and if you get an ambiguity, the
> > dispatcher will let you know about it, instead of simply choosing an
> > implementation arbitrarily.

[George Sakkis:]
> Interesting idea, certainly not mainstream for the moment though.

(Who cares whether it's "mainstream"? For what it's worth, the
idea has been around for decades.)

> Short question: how do multiple dispatch systems deal with the
> combinatorial explosion ? I guess there should be a way to avoid
> having to spell out all combinations every time, or it will soon be
> unpractical.

What combinatorial explosion?

If you restrict yourself to what can be expressed in a traditional
single-dispatch OO system, there is no combinatorial explosion;
the complexity is no greater than for single dispatch.

Multiple dispatch allows you to express much more. Some versions
of "much more" would involve defining an enormous number of
methods, but that's no more a problem with multiple dispatch
than the fact that multidimensional arrays can use a huge
amount of memory is a problem with multidimensional arrays.

-- 
g


From solipsis at pitrou.net  Thu Nov 16 18:21:31 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 16 Nov 2006 18:21:31 +0100 (CET)
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <dcbbbb410611160606i4941521fp11aafce50d68bdda@mail.gmail.com>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
	<455C2847.50903@acm.org> <ejhca0$cqt$1@sea.gmane.org>
	<52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
	<dcbbbb410611160606i4941521fp11aafce50d68bdda@mail.gmail.com>
Message-ID: <33914.62.39.9.251.1163697691.squirrel@webmail.nerim.net>


>> The problem is not merely related to the number of lines of code. It is
>> a
>> readibility problem. Having the callback enclosed in a the function call
>> which registers it makes the code more readable than defining the
>> callback
>> beforehand.
>
> And, unless I'm missing something, much harder to unit test.

Not harder than any nested function, or am I missing something?




From solipsis at pitrou.net  Thu Nov 16 18:24:38 2006
From: solipsis at pitrou.net (Antoine)
Date: Thu, 16 Nov 2006 18:24:38 +0100 (CET)
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejhu2t$gli$1@sea.gmane.org>
References: <455C072B.503@acm.org>
	<ejh48u$k7n$1@sea.gmane.org><455C2847.50903@acm.org>
	<ejhca0$cqt$1@sea.gmane.org>
	<52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
	<ejhu2t$gli$1@sea.gmane.org>
Message-ID: <43166.62.39.9.251.1163697878.squirrel@webmail.nerim.net>

>> How is this one supposed to work ? What is "with async blah()" ?
>
> a magic thing that runs the code block behind the covers, of course.
> requires a
> non-released version of the from __future__ statement, at the time.

;-))

> unlike the
> following syntax:
>
> @server.get_synset_link()
> def block(response):
>     format_expanded(li.content, response.related)

Ok, that's nearly perfect!
Only "nearly" because:
1. There are cases where you want the return value of the
"get_synset_link". 2. You sometimes need more than one callback, and you
cannot decorate two functions at once. For example, with Twisted you can
define a callback (called on success) and an errback (called on error).

But I agree that decorators help to solve a lot of problems.

Thanks for the answer

Antoine.



From exarkun at divmod.com  Thu Nov 16 18:45:58 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 16 Nov 2006 12:45:58 -0500
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <43166.62.39.9.251.1163697878.squirrel@webmail.nerim.net>
Message-ID: <20061116174558.20948.797356069.divmod.quotient.29882@ohm>

On Thu, 16 Nov 2006 18:24:38 +0100 (CET), Antoine <solipsis at pitrou.net> wrote:
> [snip]
>
>Ok, that's nearly perfect!
>Only "nearly" because:
>1. There are cases where you want the return value of the
>"get_synset_link". 2. You sometimes need more than one callback, and you
>cannot decorate two functions at once. For example, with Twisted you can
>define a callback (called on success) and an errback (called on error).
>
>But I agree that decorators help to solve a lot of problems.
>
>Thanks for the answer
>
>Antoine.
>

    >>> @foo().addCallback
      File "<stdin>", line 1
        @foo().addCallback
              ^
    SyntaxError: invalid syntax
    >>> 

Jean-Paul

From jason.orendorff at gmail.com  Thu Nov 16 19:14:19 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Thu, 16 Nov 2006 13:14:19 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ejells$m2u$1@sea.gmane.org>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<91ad5bf80611131231x14d85c75jd1daf49dd47cbe58@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<ejells$m2u$1@sea.gmane.org>
Message-ID: <bb8868b90611161014x69e1fc32gc18ba6f9317c80e5@mail.gmail.com>

On 11/15/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> George Sakkis wrote:
> > The ending sentences though are more convincing: "...but it's a part
> > of Python, and it's too late to make such fundamental changes now.
> > The functions have to remain to avoid massive code breakage". That
> > I can buy (at least for 2.x).
>
> yeah, it's clear that most of your argumentation is based on a rather
> common "I haven't thought this through very deeply, but I'm sure I'm
> smarter than those guys so that won't stop me" approach.  trust me, the
> design of Python is a *lot* more carefully put together than you appear
> to think.

Yikes.  Fredrik, if you can't say something nice, please just drop the
conversation.

(Actually, even if you *can* say something nice, this conversation
might be worth dropping.  Just a thought.)

-j

From pje at telecommunity.com  Thu Nov 16 19:23:30 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 16 Nov 2006 13:23:30 -0500
Subject: [Python-3000] native support for multiple dispatch in 3.0 ?
In-Reply-To: <mailman.649.1163689159.32030.python-3000@python.org>
Message-ID: <5.1.1.6.0.20061116123144.03425708@sparrow.telecommunity.com>

At 12:44 PM 11/16/2006 +0100, Fredrik Lundh <fredrik at pythonware.com> wrote:
>since it seems to be impossible to bring up new ideas in subthreads, I
>though I'd give this one its own thread:
>
>- has there been any discussions on making RuleDispatch-type mechanisms
>    available on a more fundamental level?

Only the discussion several months ago, where I tried to show that you 
could treat all functions as extensible, by using extensible functions to 
define them.  (This makes it possible to keep optimizations like 
tp_as_sequence->tp_len in place at the C level to support fast builtins.)

I was also trying to push for something like a 'defop' statement of the form:

     defop expr(...):
        ...

Which, instead of binding the resulting function to 'expr', would invoke 
something like:

     addmethod(expr, func, containing_class_or_None)

Where addmethod is a builtin generic function that can be extended to 
support user-defined generic function implementation.  (Of course, this 
proposal assumes type information of some kind is available in the function 
signature, making simple implementations simple.)

One purpose of the proposal is to let us get rid of most __special__ names 
syntactically, even though the implementation would retain them "under the 
hood".  Instead of 'def __iter__(self)', one would 'defop iter(self)', 
'defop operator.add(self, other)', etc.



>could someone perhaps come up with a list of real-life uses for PJE's
>existing dispatcher implementation?

http://peak.telecommunity.com/DevCenter/SecurityRules is one; in fact it's 
the major reason I wrote RuleDispatch in the first place.  I know that 
TurboGears is said to be doing something similar of their own.  There are 
also other people doing enterprisey things that they don't tell me much 
about.  People doing virtual world stuff or classic adventure games have 
also expressed interest.

I myself have found that I don't use full predicate dispatch as often as I 
use multiple or even single-dispatch generic functions.  I even went so far 
as to create this package so as to be able to use single-dispatch functions 
in code I'm doing for Chandler:

    http://cheeseshop.python.org/pypi/simplegeneric

And here's what I'm doing with it:

http://svn.osafoundation.org/chandler/trunk/chandler/parcels/osaf/sharing/EIM.txt
http://svn.osafoundation.org/chandler/trunk/chandler/parcels/osaf/sharing/eim.py

The place I'm mostly using multiple dispatch, however, is in the work I'm 
doing on RuleDispatch's better-documented and better-tested replacement-to-be:

    http://svn.eby-sarna.com/PEAK-Rules/

The new package uses an enhanced version of Guido's type-tuple dispatching 
to do multiple dispatch, and I'm using that to bootstrap the more advanced 
features.

For more "real world" usage, however, note that adaptation is a subset of 
single-dispatch generic functions, so everything in Zope and Twisted that 
uses adapters would qualify.  Zope 3 also does what they call "tuple 
adapters" which is basically a rather twisted way of implementing multiple 
dispatch generic functions without actually having a function.  :)  They 
also have event adapters that are a lot like the method combination 
features of RuleDispatch and PEAK-Rules.

(And of course, for further single-dispatch use cases, one need look no 
further than the builtin polymorphics and stdlib stuff like pickle and pprint.)


>or should we go all the way: how about using this mechanism for *all*
>dispatching, even the usual dispatch-on-instance handler?  and poly-
>morphic builtins too, while we're at it?

I wouldn't go that far.  I think we should keep most of the existing 
C-level dispatch machinery: slots are fast.  Instead, I think we should 
just have One Obvious Way to *add methods* togeneric functions, regardless 
of how they are implemented.  If there is a builtin 'addmethod()' generic, 
then it can be defined to do the right thing for builtin slot-based 
generics.  And if you implement your own specialty generics (ala 
RuleDispatch), you would simply:

     addmethod(addmethod, addmethod_for_my_new_type, my_new_type)

Or more succinctly:

     class my_new_type:
         ...
         defop addmethod(self, func, cls=None):
             # code to add a method to this generic function object

Actually, it would be nice if you could subclass 'function', just so that 
we could have 'inspect' and 'pydoc' functions treat your instances 
sanely.  All of my generic functions go to insane lengths to *be* normal 
function objects so that inspection and documentation tools will treat them 
sanely.  OTOH, I suppose if the inspect and pydoc code were refactored to 
generics, that wouldn't be a problem, because you could just 'defop 
inspect.getdoc(...)' or whatever...


From rhamph at gmail.com  Thu Nov 16 20:39:59 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 16 Nov 2006 12:39:59 -0700
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
References: <455C072B.503@acm.org> <ejh48u$k7n$1@sea.gmane.org>
	<455C2847.50903@acm.org> <ejhca0$cqt$1@sea.gmane.org>
	<52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
Message-ID: <aac2c7cb0611161139s433e4916wd1d3a935abf6210f@mail.gmail.com>

On 11/16/06, Antoine <solipsis at pitrou.net> wrote:
>
> Hi,
>
> > Well, if you find that easy to read and clear, I suspect you're more of
> > a JavaScript programmer than a Python programmer.  Here's how a more
> > general form of the above would look in Python:
> >
> >      response = (yield server.get_synset_links())
> >      format_expanded(li.content, response.related)
>
> This assumes there is an underlying event loop of some sort that accepts
> and is able to schedule generators.
> Much more annoyingly, all your "yield"'s must be in the same scope, they
> can't be spread in various subfunctions called from the generator. It is a
> massive limitation to writing clean code, unless your generators are very
> short.
>
> >      def handle_response(record):
> >          format_expanded(li.content, response.related)
> >      server.get_synset_link(handle_response)
>
> As already mentioned, the problem with this idiom is that code that is
> executed *after* appears *before* the "get_synset_link".
> It makes the code less straightforward to write and especially to read
> (and having readable code is important).

This is a bit biased by my dislike of event-driven programming, but:

def x():
    response = yield server.do_command('get_synset_links')
    format_expanded(li.content, response.related)
event_loop.add(x())

It may be possible to clean it up further with decorators, anonymous
functions, or "suites", but it's interesting to note that the bulk of
the code contains no functions (anonymous or named) at all?one is only
used to wrap it all in a new "thread".

Another benefit is that any error handling can use traditional python
methods: a try/except wrapped around the operation that may fail.
TOOWTDI.

And yes, it does still require an event loop, but so does your
original javascript example; what do you think calls your callback?

-- 
Adam Olsen, aka Rhamphoryncus

From gsakkis at rutgers.edu  Thu Nov 16 20:56:51 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Thu, 16 Nov 2006 14:56:51 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <bb8868b90611161014x69e1fc32gc18ba6f9317c80e5@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<ejells$m2u$1@sea.gmane.org>
	<bb8868b90611161014x69e1fc32gc18ba6f9317c80e5@mail.gmail.com>
Message-ID: <91ad5bf80611161156t59857119y90ebab5f966f31b1@mail.gmail.com>

On 11/16/06, Jason Orendorff <jason.orendorff at gmail.com> wrote:

> (Actually, even if you *can* say something nice, this conversation
> might be worth dropping.  Just a thought.)

I agree. For me, the bottom line and the main argument for dropping
the proposal is "itertools is not as big as you think it is to
introduce a builtin rich-iterator object". It might be a bad analogy
for some, but it reminds me of the "why have a path object, os.path.*
is fine" reactions a while back [1]. All other arguments can be
rebutted. Perhaps when we start seeing more "from itertools import
..." all over the place, we can reopen it. For now let's leave this
thread R.I.P.

George


[1] I haven't followed the recent path and mini path object
discussions to know if things have changed since then. One thing I
know though is that Jason's path module has practically replaced os.*
and glob.* in my code.

From tjreedy at udel.edu  Thu Nov 16 23:46:44 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 16 Nov 2006 17:46:44 -0500
Subject: [Python-3000] A plea for anonymous functions
References: <455C072B.503@acm.org>
	<ejh48u$k7n$1@sea.gmane.org><455C2847.50903@acm.org>
	<ejhca0$cqt$1@sea.gmane.org>
	<52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
Message-ID: <ejipoi$f32$1@sea.gmane.org>


"Antoine" <solipsis at pitrou.net> wrote in message 
news:52661.62.39.9.251.1163684950.squirrel at webmail.nerim.net...
>>      def handle_response(record):
>>          format_expanded(li.content, response.related)
>>      server.get_synset_link(handle_response)
>
> As already mentioned, the problem with this idiom is that code that is
> executed *after* appears *before* the "get_synset_link".

*To me* you have this backwards.  The function-object creation code is 
executed *before* the call that passes that object.  This is true whether 
it is placed before, as above, or nested within, as you and Tomer prefer.

> It makes the code less straightforward to write

*To you*.

> and especially to read

*To you*.

I (and many others, apparently) am less comfortable reading deeply nested 
code inside out than perhaps you and Tomer.  If a function body is more 
complicated than 'return <expression>', I prefer to have it factored out.

> (and having readable code is important).

To both of us, but we differ at least slightly in what is most readable.

Terry Jan Reedy




From mike.klaas at gmail.com  Fri Nov 17 00:01:44 2006
From: mike.klaas at gmail.com (Mike Klaas)
Date: Thu, 16 Nov 2006 15:01:44 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611161156t59857119y90ebab5f966f31b1@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<ejells$m2u$1@sea.gmane.org>
	<bb8868b90611161014x69e1fc32gc18ba6f9317c80e5@mail.gmail.com>
	<91ad5bf80611161156t59857119y90ebab5f966f31b1@mail.gmail.com>
Message-ID: <3d2ce8cb0611161501x64092187u4d867f3c8d1bd987@mail.gmail.com>

On 11/16/06, George Sakkis <gsakkis at rutgers.edu> wrote:
>
> I agree. For me, the bottom line and the main argument for dropping
> the proposal is "itertools is not as big as you think it is to
> introduce a builtin rich-iterator object".
<>

You should also consider that some people see that as a false dichotomy.

-Mike

From greg.ewing at canterbury.ac.nz  Fri Nov 17 03:52:52 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Nov 2006 15:52:52 +1300
Subject: [Python-3000] Class Decorators + ctypes (Was: yes to
	class	decorators)
In-Reply-To: <455BFA17.1000205@acm.org>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
	<455BFA17.1000205@acm.org>
Message-ID: <455D2404.6010504@canterbury.ac.nz>

Talin wrote:

> One unresolved issue is what exactly gets assigned to 'x' and 'y' at the 
> time the class definition is evaluated.

Seems to me it ought to be some kind of descriptor.
It might be possible to do this reasonably without
any new syntax, e.g.

   @cstruct
   class MyStruct:
     x = cfield("f")
     y = cfield("f")

given a function cfield() that returns a suitable
descriptor.

> A different approach would be to say that the class decorator can 
> overload the '__setattr__' method on the class itself during class creation:

But it doesn't work that way -- class attributes don't
go through __setattr__ during class creation. They go
into a dict that is passed to the type object's
constructor. The class doesn't even exist while the
class body is being evaluated.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov 17 03:53:41 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Nov 2006 15:53:41 +1300
Subject: [Python-3000] yes to class decorators
In-Reply-To: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
References: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
Message-ID: <455D2435.1000704@canterbury.ac.nz>

tomer filiba wrote:

> we should also note that it's possible to replace "__metaclass__"
> by a @metaclass decorator.

I don't think so. Your proposed technique for that seems to
involve creating an ordinary class first, then taking it
apart and recreating it using the specified metaclass.
That smells very wrong to me.

--
Greg


From greg.ewing at canterbury.ac.nz  Fri Nov 17 03:54:25 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Nov 2006 15:54:25 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
Message-ID: <455D2461.8000602@canterbury.ac.nz>

George Sakkis wrote:

> And for two, not everyone feels comfortable with duck typing. People
> who consider (for better or for worse) isinstance() safer than
> hasattr()/getattr() would be accomodated too.

The trouble is that building things into the core to
"accommodate" these people ends up making things worse
for people who don't subscribe to that school of
thought, because they will trip over places where
some type won't be acceptable because it doesn't
inherit from the right bases, even though it
implements all the functionality required for their
application.

I would suggest that if someone is that uncomfortable
with duck typing, then perhaps Python is not the
right language for them. Even if the language itself
is neutral on the matter, the culture and the body
of code libraries that has grown up around it is
not. There's a clear preference in favour of duck
typing and against LYBL techniques. If you don't
go with that flow, your code will have trouble
interoperating with that of others.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov 17 03:54:31 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Nov 2006 15:54:31 +1300
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455C072B.503@acm.org>
References: <455C072B.503@acm.org>
Message-ID: <455D2467.2030409@canterbury.ac.nz>

Talin wrote:

> From my point of view, both 'with' and generator expressions are 
> limited, special-case solutions to a general problem - the desire to be 
> able to use and manipulate unnamed blocks of code as first-class 
> objects.

I don't think it's as simple as that. Back when the with
statement was first being discussed, I suggested that it
should be implemented by passing the body as an anonymous
function. That would have been a very simple way of doing
it, but there were problems, such as what to do if the
body contained break or continue statements. In the end,
Guido rejected the anonymous-function approach, and we
ended up with something that is rather more elaborate.

So we have a situation where there was no syntactical
barrier to treating a code block as an anonymous function,
but we chose not to do so. That seems to cast doubt on
the idea that first-class code blocks would solve all
the problems you suggest they would solve.

--
Greg

From guido at python.org  Fri Nov 17 05:29:33 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Nov 2006 20:29:33 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455D2467.2030409@canterbury.ac.nz>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
Message-ID: <ca471dc20611162029x77b332d1uabb7a531881ec542@mail.gmail.com>

If you want anonymous blocks, by all means use Ruby. Python has
first-class callables, Ruby has anonymous blocks. Pick your favorite,
but don't whine that neither language has both. It ain't gonna happen.

On 11/16/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Talin wrote:
>
> > From my point of view, both 'with' and generator expressions are
> > limited, special-case solutions to a general problem - the desire to be
> > able to use and manipulate unnamed blocks of code as first-class
> > objects.
>
> I don't think it's as simple as that. Back when the with
> statement was first being discussed, I suggested that it
> should be implemented by passing the body as an anonymous
> function. That would have been a very simple way of doing
> it, but there were problems, such as what to do if the
> body contained break or continue statements. In the end,
> Guido rejected the anonymous-function approach, and we
> ended up with something that is rather more elaborate.
>
> So we have a situation where there was no syntactical
> barrier to treating a code block as an anonymous function,
> but we chose not to do so. That seems to cast doubt on
> the idea that first-class code blocks would solve all
> the problems you suggest they would solve.
>
> --
> Greg
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From george.sakkis at gmail.com  Fri Nov 17 05:57:30 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Thu, 16 Nov 2006 23:57:30 -0500
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ca471dc20611162029x77b332d1uabb7a531881ec542@mail.gmail.com>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<ca471dc20611162029x77b332d1uabb7a531881ec542@mail.gmail.com>
Message-ID: <91ad5bf80611162057x578fbc56l314d5a8dfd405d81@mail.gmail.com>

On 11/16/06, Guido van Rossum <guido at python.org> wrote:

> If you want anonymous blocks, by all means use Ruby. Python has
> first-class callables, Ruby has anonymous blocks. Pick your favorite,
> but don't whine that neither language has both. It ain't gonna happen.

Should there be a new addition for this in PEP 3099 or does the
"lambda will not be renamed" entry covers it sufficiently ?

George

From jcarlson at uci.edu  Fri Nov 17 07:33:47 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Thu, 16 Nov 2006 22:33:47 -0800
Subject: [Python-3000] yes to class decorators
In-Reply-To: <455D2435.1000704@canterbury.ac.nz>
References: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
	<455D2435.1000704@canterbury.ac.nz>
Message-ID: <20061116222220.8325.JCARLSON@uci.edu>


Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> tomer filiba wrote:
> 
> > we should also note that it's possible to replace "__metaclass__"
> > by a @metaclass decorator.
> 
> I don't think so. Your proposed technique for that seems to
> involve creating an ordinary class first, then taking it
> apart and recreating it using the specified metaclass.
> That smells very wrong to me.

I more or less agree, at least on the side of 'creating and destroying a
class is a waste', but here's some abuse that I couldn't help chuckling
over. No need to even bother constructing a class, but don't forget that
'return locals()' at the end!  (not that I'm advocating its [ab]use,
just something that made me smile)

 - Josiah

>>> def metaclass(*bases):
...     def class_factory(fcn):
...         return type(fcn.__name__, bases, fcn())
...     return class_factory
...
>>> @metaclass(object)
... def newclass():
...     def foo(self, arg):
...         print arg
...     return locals()
...
>>> newclass().foo('hello!')
hello!
>>>


From talin at acm.org  Fri Nov 17 10:00:46 2006
From: talin at acm.org (Talin)
Date: Fri, 17 Nov 2006 01:00:46 -0800
Subject: [Python-3000] Class Decorators + ctypes (Was: yes to
	class	decorators)
In-Reply-To: <455D2404.6010504@canterbury.ac.nz>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>
	<455BFA17.1000205@acm.org> <455D2404.6010504@canterbury.ac.nz>
Message-ID: <455D7A3E.6030209@acm.org>

Greg Ewing wrote:
> Talin wrote:
>> A different approach would be to say that the class decorator can 
>> overload the '__setattr__' method on the class itself during class 
>> creation:
> 
> But it doesn't work that way -- class attributes don't
> go through __setattr__ during class creation. They go
> into a dict that is passed to the type object's
> constructor. The class doesn't even exist while the
> class body is being evaluated.

I see that as a shortcoming of the current metaclass implementation. It 
would be much more useful IMHO if the the metaclass had access to the 
declaration order. Perhaps the class decorator protocol could somehow 
take this into account?

-- Talin

From talin at acm.org  Fri Nov 17 10:41:53 2006
From: talin at acm.org (Talin)
Date: Fri, 17 Nov 2006 01:41:53 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455D2467.2030409@canterbury.ac.nz>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
Message-ID: <455D83E1.4060601@acm.org>

Greg Ewing wrote:
> Talin wrote:
> 
>> From my point of view, both 'with' and generator expressions are 
>> limited, special-case solutions to a general problem - the desire to 
>> be able to use and manipulate unnamed blocks of code as first-class 
>> objects.
> 
> I don't think it's as simple as that. Back when the with
> statement was first being discussed, I suggested that it
> should be implemented by passing the body as an anonymous
> function. That would have been a very simple way of doing
> it, but there were problems, such as what to do if the
> body contained break or continue statements. In the end,
> Guido rejected the anonymous-function approach, and we
> ended up with something that is rather more elaborate.
> 
> So we have a situation where there was no syntactical
> barrier to treating a code block as an anonymous function,
> but we chose not to do so. That seems to cast doubt on
> the idea that first-class code blocks would solve all
> the problems you suggest they would solve.

Well, it seems that my argument about being able to construct the 'with' 
statement with anon. code blocks may not be air-tight; However I still 
stand by my basic argument that anon. code is a fundamental 
building-block of a number of interesting language extensions, and that 
I expect to see a series of special-case syntactical work-arounds that 
compensate for the lack of such a feature.

To give a concrete example, suppose we decide to add to Python a 
special-case syntactical structure to deal with asynchronous events, 
similar to the code example that I posted earlier. The simplest possible 
example I can think of is an an alarm-clock function:

    # Create a callback timer
    alarm = Timer()
    alarm.SetDuration( 100, Timer.Milliseconds )

    upon alarm:
       print "It's time to get up, silly-head!"

    print "Alarm has been set!"

In the example, the hypothetical 'upon' statement passes a reference to 
the following suite to the object resulting from the expression - so in 
this case, the suite containing the print statement is passed as a 
callable to the 'alarm' object. Thus it is equivalent to:

    # Create a callback timer
    alarm = Timer()
    alarm.SetDuration( 100, Timer.Milliseconds )

    def upon_alarm():
       print "It's time to get up, silly-head!"

    alarm.set_callback( upon_alarm )

    print "Alarm has been set!"

The latter approach is a common pattern in Python. However, if we take 
the argument presented in the blog entry cited by Fredrik, which 
proposes that 'design patterns are symptomatic of a weakness in a 
programming language' (an argument which I don't entirely buy - absence 
of a feature is not always a weakness), then in order to make the 
language 'stronger', we ought to make the design pattern disappear into 
the language. In this case, we do so by absorbing the setup of the 
function machinery into the syntax of the 'upon' statement.

However, this special-case approach is flawed in this example, because 
its use is so narrow. If you think about it for a minute, you 
immediately start to imagine all of the things that 'upon' ought to deal 
with that it doesn't. For example, it has no way to deal with error 
callbacks; It has no way to pass arguments from the asynchronous process 
to the code block. It doesn't let me build a hash table of code snippets 
(which is my solution to the problem of 'switch' statements in Python.)

But even if we spend the next month writing a PEP for an 'upon' 
statement that overcomes all these limitations, I will still most likely 
be able to think of some use case that it doesn't handle. No matter what 
I add to the PEP, there will eventually be another PEP that adds yet 
another special-case solution to some aspect that I haven't covered. 
Instead, I'd rather work at a more fundamental level, that solves a 
larger class of problems.

(To the folks that would say that this is only a trivial optimization, I 
would respond - if that's the case, then why is it used so heavily in 
languages which do support it? And why does practically every other 
popular scripting language except Python support it at all? And no, the 
answer 'you shouldn't be programming in Python' is not an acceptable 
response, it's a lame cop-out IMHO.)

-- Talin


From fredrik at pythonware.com  Fri Nov 17 10:55:18 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 17 Nov 2006 10:55:18 +0100
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455D83E1.4060601@acm.org>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<455D83E1.4060601@acm.org>
Message-ID: <ejk0u7$o6k$1@sea.gmane.org>

Talin wrote:

> I expect to see a series of special-case syntactical work-arounds that 
> compensate for the lack of such a feature.

yeah, because the "special-case syntactical work-arounds" are care-
fully designed to be *usable* for a well-defined group of *practical* 
problems.  it's about HCI, not CS.

please get over this "all I have is a hammer that my CS teacher told
me to use" mode of thinking; we're designing for humans, not wannabe 
language designers who believe in "the one true mechanism".  there are 
plenty of other reli^h^h^h^hlanguages for that kind of thinking.

</F>


From p.f.moore at gmail.com  Fri Nov 17 10:57:05 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 17 Nov 2006 09:57:05 +0000
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455D83E1.4060601@acm.org>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<455D83E1.4060601@acm.org>
Message-ID: <79990c6b0611170157w72955f93t988c536cd84e2add@mail.gmail.com>

On 11/17/06, Talin <talin at acm.org> wrote:
>     # Create a callback timer
>     alarm = Timer()
>     alarm.SetDuration( 100, Timer.Milliseconds )
>
>     upon alarm:
>        print "It's time to get up, silly-head!"
>
>     print "Alarm has been set!"


Why invent a new special case syntax - the ones we have work fine.

# This goes in a library
def upon(event):
    def wrapper(fn):
        event.set_callback(fn)
        return fn
    return wrapper

# Create a callback timer
alarm = Timer()
alarm.SetDuration( 100, Timer.Milliseconds )

@upon(alarm)
def callback():
    print "It's time to get up, silly-head!"

Looks pretty readable to me...

Paul

From fredrik at pythonware.com  Fri Nov 17 10:59:41 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 17 Nov 2006 10:59:41 +0100
Subject: [Python-3000] native support for multiple dispatch in 3.0 ?
In-Reply-To: <5.1.1.6.0.20061116123144.03425708@sparrow.telecommunity.com>
References: <mailman.649.1163689159.32030.python-3000@python.org>
	<5.1.1.6.0.20061116123144.03425708@sparrow.telecommunity.com>
Message-ID: <ejk16c$o6k$2@sea.gmane.org>

Phillip J. Eby wrote:

> Only the discussion several months ago, where I tried to show that you 
> could treat all functions as extensible, by using extensible functions to 
> define them.

/snip many useful references/

thanks.  more than enough to keep me busy over the weekend ;-)

has anyone looked at "unifying" the twisted and zope models, btw?

</F>


From talin at acm.org  Fri Nov 17 11:09:52 2006
From: talin at acm.org (Talin)
Date: Fri, 17 Nov 2006 02:09:52 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejk0u7$o6k$1@sea.gmane.org>
References: <455C072B.503@acm.org>
	<455D2467.2030409@canterbury.ac.nz>	<455D83E1.4060601@acm.org>
	<ejk0u7$o6k$1@sea.gmane.org>
Message-ID: <455D8A70.8010707@acm.org>

Fredrik Lundh wrote:
> Talin wrote:
> 
>> I expect to see a series of special-case syntactical work-arounds that 
>> compensate for the lack of such a feature.
> 
> yeah, because the "special-case syntactical work-arounds" are care-
> fully designed to be *usable* for a well-defined group of *practical* 
> problems.  it's about HCI, not CS.
> 
> please get over this "all I have is a hammer that my CS teacher told
> me to use" mode of thinking; we're designing for humans, not wannabe 
> language designers who believe in "the one true mechanism".  there are 
> plenty of other reli^h^h^h^hlanguages for that kind of thinking.

I've never taken a CS course BTW - the only formal training I've had was 
in the Air Force, in 1976, where I learned assembly language, Fortran, 
and COBOL :)

So be careful what assumptions you make here :)

-- Talin

From solipsis at pitrou.net  Fri Nov 17 11:26:59 2006
From: solipsis at pitrou.net (Antoine)
Date: Fri, 17 Nov 2006 11:26:59 +0100 (CET)
Subject: [Python-3000] upon
In-Reply-To: <455D83E1.4060601@acm.org>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<455D83E1.4060601@acm.org>
Message-ID: <7924.62.39.9.251.1163759219.squirrel@webmail.nerim.net>

> However, this special-case approach is flawed in this example, because
> its use is so narrow. If you think about it for a minute, you
> immediately start to imagine all of the things that 'upon' ought to deal
> with that it doesn't.
> For example, it has no way to deal with error
> callbacks;

Arguably you could write :

    # Create a callback timer
    alarm = Timer()
    alarm.SetDuration( 100, Timer.Milliseconds )

    upon alarm.success:
       print "It's time to get up, silly-head!"
    upon alarm.failure:
       print "The clock is broken, you can go back to bed"

(or @upon(alarm.success) for the suggested decorator equivalent)

Which would resolve to something like :

    def _cb():
       print "It's time to get up, silly-head!"
    alarm.success.__upon__(_cb)
    def _cb():
       print "The clock is broken, you can go back to bed"
    alarm.failure.__upon__(_cb)




From behnel_ml at gkec.informatik.tu-darmstadt.de  Fri Nov 17 11:35:34 2006
From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel)
Date: Fri, 17 Nov 2006 11:35:34 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611131239g72afbfaahac86c2387e1f10cf@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>	<ca471dc20611131131j29d0139dja2aa77b1639d5f4f@mail.gmail.com>
	<91ad5bf80611131239g72afbfaahac86c2387e1f10cf@mail.gmail.com>
Message-ID: <455D9076.6010206@gkec.informatik.tu-darmstadt.de>

Hi,

George Sakkis wrote:
> On 11/13/06, Guido van Rossum <guido at python.org> wrote:
>> Are you going to propose similar
>> changes for all standard de-facto interfaces, like sequences,
>> mappings, files etc.?
> 
> No, I won't (at least not for now ;-)). Notice however that most
> user-defined sequences, mappings, etc. usually don't start from
> scratch; they either inherit from an existing type or from an
> appropriate mixin.

I absolutely disagree. One advantage of the way Python handles protocols like
sequences, mappings and file-likes is that you do not have to implement the
entire protocol if all you want is a thing to be, say, indexable.

In most cases I've come across, I did not need to use any special base class
or mixin at all, partly because I didn't need the complete protocol anyway,
partly for performance reasons, partly because the the behaviour of the class
was so different from a list that there was no big gain in having a partial
implementation.

A superclass like UserDict etc. can be helpful, but requiring you to inherit
from it seems plain wrong in my eyes.

Stefan

From ncoghlan at gmail.com  Fri Nov 17 13:35:06 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 17 Nov 2006 22:35:06 +1000
Subject: [Python-3000] Class Decorators + ctypes (Was: yes
	to	class	decorators)
In-Reply-To: <455D7A3E.6030209@acm.org>
References: <1d85506f0611151506n48e1ee48q91de76b05754435a@mail.gmail.com>	<455BFA17.1000205@acm.org>
	<455D2404.6010504@canterbury.ac.nz> <455D7A3E.6030209@acm.org>
Message-ID: <455DAC7A.8060601@gmail.com>

Talin wrote:
> Greg Ewing wrote:
>> Talin wrote:
>>> A different approach would be to say that the class decorator can 
>>> overload the '__setattr__' method on the class itself during class 
>>> creation:
>> But it doesn't work that way -- class attributes don't
>> go through __setattr__ during class creation. They go
>> into a dict that is passed to the type object's
>> constructor. The class doesn't even exist while the
>> class body is being evaluated.
> 
> I see that as a shortcoming of the current metaclass implementation. It 
> would be much more useful IMHO if the the metaclass had access to the 
> declaration order. Perhaps the class decorator protocol could somehow 
> take this into account?

Not really, no: currently, no code other than the compiler has access to the 
order of declaration of class attributes (and the compiler only has access 
because it executes the code to populate that namespace).

The only way I could see it working is to replace the dictionary used during 
class instantiation with a dict subclass that kept track of the order that 
keys were added to the namespace (for example, storing a list of keys in an 
"ordered_keys" attribute).

Adding new keys to and deleting keys from such a dict would be slower than 
usual, though (as it would have to update the key list as well as the hash map).

However, actually doing anything like this would require some mechanism for 
the metaclass to get class instantiation to use that dict subtype in the first 
place...

Regardless, as Jack pointed out: this has *nothing* to do with class 
decorators, which operate only on fully constructed classes.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From talin at acm.org  Fri Nov 17 14:13:24 2006
From: talin at acm.org (Talin)
Date: Fri, 17 Nov 2006 05:13:24 -0800
Subject: [Python-3000] upon
In-Reply-To: <7924.62.39.9.251.1163759219.squirrel@webmail.nerim.net>
References: <455C072B.503@acm.org>
	<455D2467.2030409@canterbury.ac.nz>	<455D83E1.4060601@acm.org>
	<7924.62.39.9.251.1163759219.squirrel@webmail.nerim.net>
Message-ID: <455DB574.6000207@acm.org>

Antoine wrote:
>> However, this special-case approach is flawed in this example, because
>> its use is so narrow. If you think about it for a minute, you
>> immediately start to imagine all of the things that 'upon' ought to deal
>> with that it doesn't.
>> For example, it has no way to deal with error
>> callbacks;
> 
> Arguably you could write :
> 
>     # Create a callback timer
>     alarm = Timer()
>     alarm.SetDuration( 100, Timer.Milliseconds )
> 
>     upon alarm.success:
>        print "It's time to get up, silly-head!"
>     upon alarm.failure:
>        print "The clock is broken, you can go back to bed"
> 
> (or @upon(alarm.success) for the suggested decorator equivalent)
> 
> Which would resolve to something like :
> 
>     def _cb():
>        print "It's time to get up, silly-head!"
>     alarm.success.__upon__(_cb)
>     def _cb():
>        print "The clock is broken, you can go back to bed"
>     alarm.failure.__upon__(_cb)

Interesting. Very interesting.

You could even do a switch statement with this idea:

    # Create the dispatcher
    s = Switch()

    # Set up the cases
    upon s.case(1):
       ... code for case 1
    upon s.case(2):
       ... code for case 2
    upon s.case(3):
       ... code for case 3

    # Invoke the case corresponding to value 'n'
    s( n )

Where 'case' associates assigns the suite to a dictionary, using the 
argument (1, 2, ...) as the key. Once the case handlers are set up, you 
can invoke the 'switch' as many times as you want, without having to 
re-evaluate the case expressions. You can even pass the dispatcher 
around to other objects and invoke it remotely.

Hmmm, a single syntax that handles both async callbacks, async errors, 
and switch/case constructs...maybe this would be more useful than I 
thought ... :)

Now, can we figure out how to handle arguments to the code block...?

-- Talin

From fredrik at pythonware.com  Fri Nov 17 14:46:19 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 17 Nov 2006 14:46:19 +0100
Subject: [Python-3000] upon
References: <455C072B.503@acm.org><455D2467.2030409@canterbury.ac.nz>	<455D83E1.4060601@acm.org><7924.62.39.9.251.1163759219.squirrel@webmail.nerim.net>
	<455DB574.6000207@acm.org>
Message-ID: <ejkefc$772$1@sea.gmane.org>

"Talin" wrote:

> Hmmm, a single syntax that handles both async callbacks, async errors,
> and switch/case constructs...maybe this would be more useful than I
> thought ... :)

done much BASIC programming lately ?

</F> 




From bob at redivi.com  Fri Nov 17 16:07:39 2006
From: bob at redivi.com (Bob Ippolito)
Date: Fri, 17 Nov 2006 07:07:39 -0800
Subject: [Python-3000] native support for multiple dispatch in 3.0 ?
In-Reply-To: <ejk16c$o6k$2@sea.gmane.org>
References: <mailman.649.1163689159.32030.python-3000@python.org>
	<5.1.1.6.0.20061116123144.03425708@sparrow.telecommunity.com>
	<ejk16c$o6k$2@sea.gmane.org>
Message-ID: <6a36e7290611170707q601bcd2eo532ab9b97735c9a4@mail.gmail.com>

On 11/17/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Phillip J. Eby wrote:
>
> > Only the discussion several months ago, where I tried to show that you
> > could treat all functions as extensible, by using extensible functions to
> > define them.
>
> /snip many useful references/
>
> thanks.  more than enough to keep me busy over the weekend ;-)
>
> has anyone looked at "unifying" the twisted and zope models, btw?

Yes. Twisted has been using Zope.Interface for a few years now.

-bob

From nas at arctrix.com  Fri Nov 17 17:35:59 2006
From: nas at arctrix.com (Neil Schemenauer)
Date: Fri, 17 Nov 2006 09:35:59 -0700
Subject: [Python-3000] Fwd: Python bytes object
Message-ID: <20061117163559.GA13713@mems-exchange.org>

----- Forwarded message from Antti Louko <alo at louko.com> -----

Date: Fri, 17 Nov 2006 08:25:03 +0200
From: Antti Louko <alo at louko.com>
Subject: Python bytes object
To: nas at arctrix.com

Python bytes object is useful.

I would add bitwise logical operations. They would be most useful in
cryptographic and graphics operations.

Also, encoding and decoding (long) integers in binary fromat to and from
bytes type would be useful. Currently, there is no way to efficiently
convert long integers to binary formats.

Regards,

	Antti Louko

----- End forwarded message -----

From fumanchu at amor.org  Fri Nov 17 17:47:04 2006
From: fumanchu at amor.org (Robert Brewer)
Date: Fri, 17 Nov 2006 08:47:04 -0800
Subject: [Python-3000] A plea for anonymous functions
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<455D83E1.4060601@acm.org> <ejk0u7$o6k$1@sea.gmane.org>
Message-ID: <435DF58A933BA74397B42CDEB8145A86224CA8@ex9.hostedexchange.local>

Fredrik Lundh wrote:
> it's about HCI, not CS.

+1 QOTF.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061117/c0552e1b/attachment.htm 

From guido at python.org  Fri Nov 17 18:25:03 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 17 Nov 2006 09:25:03 -0800
Subject: [Python-3000] Fwd: Python bytes object
In-Reply-To: <20061117163559.GA13713@mems-exchange.org>
References: <20061117163559.GA13713@mems-exchange.org>
Message-ID: <ca471dc20611170925w2148766auf8756135ac9f4bfd@mail.gmail.com>

Sounds good to me -- I don't have this need myself but it seems to fit
well.  I'm looking forward to a patch. :-)

On 11/17/06, Neil Schemenauer <nas at arctrix.com> wrote:
> ----- Forwarded message from Antti Louko <alo at louko.com> -----
>
> Date: Fri, 17 Nov 2006 08:25:03 +0200
> From: Antti Louko <alo at louko.com>
> Subject: Python bytes object
> To: nas at arctrix.com
>
> Python bytes object is useful.
>
> I would add bitwise logical operations. They would be most useful in
> cryptographic and graphics operations.
>
> Also, encoding and decoding (long) integers in binary fromat to and from
> bytes type would be useful. Currently, there is no way to efficiently
> convert long integers to binary formats.
>
> Regards,
>
>         Antti Louko
>
> ----- End forwarded message -----
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steven.bethard at gmail.com  Fri Nov 17 18:39:54 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 17 Nov 2006 10:39:54 -0700
Subject: [Python-3000] yes to class decorators
In-Reply-To: <20061116222220.8325.JCARLSON@uci.edu>
References: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>
	<455D2435.1000704@canterbury.ac.nz>
	<20061116222220.8325.JCARLSON@uci.edu>
Message-ID: <d11dcfba0611170939j2bdfdec1p7016e6896d2e2a8a@mail.gmail.com>

On 11/16/06, Josiah Carlson <jcarlson at uci.edu> wrote:
> but here's some abuse that I couldn't help chuckling
> over. No need to even bother constructing a class, but don't forget that
> 'return locals()' at the end!  (not that I'm advocating its [ab]use,
> just something that made me smile)
>
>  - Josiah
>
> >>> def metaclass(*bases):
> ...     def class_factory(fcn):
> ...         return type(fcn.__name__, bases, fcn())
> ...     return class_factory
> ...
> >>> @metaclass(object)
> ... def newclass():
> ...     def foo(self, arg):
> ...         print arg
> ...     return locals()
> ...
> >>> newclass().foo('hello!')
> hello!
> >>>

Of course, this is almost exactly what the class statement does under
the covers.  It makes a function object for the body of the class
statement, inserting a ``return locals()`` at the end.  When the class
statement is executed, that function is called::

>>> code = compile('class C(object): x = 1', '<string>', 'exec')
>>> dis.dis(code)
  1           0 LOAD_CONST               0 ('C')
              3 LOAD_NAME                0 (object)
              6 BUILD_TUPLE              1
              9 LOAD_CONST               1 (<code object C at
012EFF08, file "<string>", line 1>)
             12 MAKE_FUNCTION            0
             15 CALL_FUNCTION            0
             18 BUILD_CLASS
             19 STORE_NAME               1 (C)
             22 LOAD_CONST               2 (None)
             25 RETURN_VALUE
>>> dis.dis(code.co_consts[1])
  1           0 LOAD_NAME                0 (__name__)
              3 STORE_NAME               1 (__module__)
              6 LOAD_CONST               0 (1)
              9 STORE_NAME               2 (x)
             12 LOAD_LOCALS
             13 RETURN_VALUE

The important points in the code above are the ``MAKE_FUNCTION`` and
``CALL_FUNCTION`` from the class statement itself, and the
``LOAD_LOCALS`` and ``RETURN VALUE`` that were inserted at the end of
the code for the class's body.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From tjreedy at udel.edu  Fri Nov 17 18:46:30 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 17 Nov 2006 12:46:30 -0500
Subject: [Python-3000] A plea for anonymous functions
References: <455C072B.503@acm.org><ejh48u$k7n$1@sea.gmane.org><455C2847.50903@acm.org><ejhca0$cqt$1@sea.gmane.org><52661.62.39.9.251.1163684950.squirrel@webmail.nerim.net>
	<ejipoi$f32$1@sea.gmane.org>
Message-ID: <ejkshk$1p4$1@sea.gmane.org>


"Terry Reedy" <tjreedy at udel.edu> wrote in message 
news:ejipoi$f32$1 at sea.gmane.org...
> *To me* you have this backwards.  The function-object creation code is 
> executed *before* the call that passes that object.  This is true whether 
> it is placed before, as above, or nested within, as you and Tomer prefer.

Correction: I meant Talin, the OP, not Tomer, who started a different 
thread on decorators.  My apologies to Tomer for putting words in his mouth 
that were not his.

tjr




From janssen at parc.com  Fri Nov 17 20:22:58 2006
From: janssen at parc.com (Bill Janssen)
Date: Fri, 17 Nov 2006 11:22:58 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455D2461.8000602@canterbury.ac.nz> 
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
Message-ID: <06Nov17.112300pst."58648"@synergy1.parc.xerox.com>

Greg Ewing wrote:
> George Sakkis wrote:
> 
> > And for two, not everyone feels comfortable with duck typing. People
> > who consider (for better or for worse) isinstance() safer than
> > hasattr()/getattr() would be accomodated too.
> 
> The trouble is that building things into the core to
> "accommodate" these people ends up making things worse
> for people who don't subscribe to that school of
> thought, because they will trip over places where
> some type won't be acceptable because it doesn't
> inherit from the right bases, even though it
> implements all the functionality required for their
> application.

I think that the issue is whether the caller can know whether their
type "implements all the functionality required for their
application".  If the library function they are calling is effectively
opaque, they somehow have to understand all the checking logic that
that function will perform, to build their type and pass their
parameters effectively.  The use of standard base types (interfaces)
is basically a communications mechanism that lets functionality
designers easily tell users of their functionality what's needed.

Using isinstance() instead of hasattr()/getattr() allows for more and
better communication, too.  Checking for some textual attributes of a
type can't capture semantics not expressed in those textual
attributes, which a "type" can capture that kind of intangible
attribute.  Finally, checking for one type instead of a vast variety
of possible syntactic checks is probably usually more efficient, as
well.

> I would suggest that if someone is that uncomfortable
> with duck typing, then perhaps Python is not the
> right language for them.

Wow.  I think that lots of folks aren't exactly uncomfortable with it.
It's more that they see it as a blemish on what's otherwise a pretty
good language, and what's more, they see it as an unnecessary blemish,
since Python's type system is (and this is a rarity among languages)
powerful and flexible enough not to *need* duck typing.  They
contribute suggestions about how to remove the necessity for it in a
spirit of public service, to improve the commonweal by improving
Python, to improve your life and the lives of all Python users.  Much
like your work on PyGUI.  And the suggestions I've seen aren't about
making it *impossible* to do duck typing in Python, but rather about
making it *unnecessary* to do duck typing in Python.

Bill

From talin at acm.org  Fri Nov 17 21:34:54 2006
From: talin at acm.org (Talin)
Date: Fri, 17 Nov 2006 12:34:54 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <ejk0u7$o6k$1@sea.gmane.org>
References: <455C072B.503@acm.org>
	<455D2467.2030409@canterbury.ac.nz>	<455D83E1.4060601@acm.org>
	<ejk0u7$o6k$1@sea.gmane.org>
Message-ID: <455E1CEE.5030006@acm.org>

Fredrik Lundh wrote:
> Talin wrote:
> 
>> I expect to see a series of special-case syntactical work-arounds that 
>> compensate for the lack of such a feature.
> 
> yeah, because the "special-case syntactical work-arounds" are care-
> fully designed to be *usable* for a well-defined group of *practical* 
> problems.  it's about HCI, not CS.
> 
> please get over this "all I have is a hammer that my CS teacher told
> me to use" mode of thinking; we're designing for humans, not wannabe 
> language designers who believe in "the one true mechanism".  there are 
> plenty of other reli^h^h^h^hlanguages for that kind of thinking.

I don't think that anyone has really answered my fundamental question 
yet, which is this: Python is my favorite language, but I use a lot of 
other languages as well, and there's one feature that I use a lot in 
those other languages which I miss not having in Python. How is it that 
people are so hostile to something that I, and apparently others, find 
so useful?

-- Talin


From brett at python.org  Fri Nov 17 22:44:33 2006
From: brett at python.org (Brett Cannon)
Date: Fri, 17 Nov 2006 13:44:33 -0800
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455E1CEE.5030006@acm.org>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<455D83E1.4060601@acm.org> <ejk0u7$o6k$1@sea.gmane.org>
	<455E1CEE.5030006@acm.org>
Message-ID: <bbaeab100611171344i5f2aa78dm283341d455baea2c@mail.gmail.com>

On 11/17/06, Talin <talin at acm.org> wrote:
>
> Fredrik Lundh wrote:
> > Talin wrote:
> >
> >> I expect to see a series of special-case syntactical work-arounds that
> >> compensate for the lack of such a feature.
> >
> > yeah, because the "special-case syntactical work-arounds" are care-
> > fully designed to be *usable* for a well-defined group of *practical*
> > problems.  it's about HCI, not CS.
> >
> > please get over this "all I have is a hammer that my CS teacher told
> > me to use" mode of thinking; we're designing for humans, not wannabe
> > language designers who believe in "the one true mechanism".  there are
> > plenty of other reli^h^h^h^hlanguages for that kind of thinking.
>
> I don't think that anyone has really answered my fundamental question
> yet, which is this: Python is my favorite language, but I use a lot of
> other languages as well, and there's one feature that I use a lot in
> those other languages which I miss not having in Python. How is it that
> people are so hostile to something that I, and apparently others, find
> so useful?


Because we have gone down this road on this topic so many times.  This is
why Guido declared that lambda would not change; it would neither gain nor
lose any abilities (except maybe requiring parentheses around its
arguments).  No one has come up with a good solution, but plenty of people
have said, "this is a marvelous feature and as soon as we have it we can do
so much!"  Well, that's fine but you need to have the good solution before
it will be considered. <South Park reference>You need to know what you are
going to do with the underpants before you can get rich</reference>.

As Fredrik said, we worry more about HCI than CS.  Stating multi-line
anonymous functions are a great solution to all of these issues is fine from
a theoretical point of view.  But the key point is that Python puts
usability/practicality before purity.  In terms of this argument, it is
manifesting itself as resistance to this idea without a concrete proposal
for the syntax that people actually like.  And that is going to be damn hard
when every proposal so far for multi-line anonymous functions has been ugly
and not very usable.

So one must worry about the usability aspect of this proposal first instead
of what it might buy us because if it ain't usable then it ain't worth
anything in Pythonland.  You have the rationale behind wanting this, which
is great and is a good start on a PEP.  But now you need to come up with how
this feature will manifest itself in the code and be usable.  Once you have
that all wrapped up in a PEP then people will probably be more receptive to
discussing this whole matter.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061117/38a4ad2e/attachment.html 

From jcarlson at uci.edu  Sat Nov 18 00:20:27 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Fri, 17 Nov 2006 15:20:27 -0800
Subject: [Python-3000] Fwd: Python bytes object
In-Reply-To: <ca471dc20611170925w2148766auf8756135ac9f4bfd@mail.gmail.com>
References: <20061117163559.GA13713@mems-exchange.org>
	<ca471dc20611170925w2148766auf8756135ac9f4bfd@mail.gmail.com>
Message-ID: <20061117151417.832B.JCARLSON@uci.edu>


"Guido van Rossum" <guido at python.org> wrote:
> Sounds good to me -- I don't have this need myself but it seems to fit
> well.  I'm looking forward to a patch. :-)

If I remember correctly, 2.5 was supposed to have binascii.long2b() and
binascii.b2long() which would have converted a long integer to a 2.x
string and vv, offering access to the underlying PyLong_AsString() and
PyLong_FromString() functions.  According to my Python 2.5 distribution
and the documentation, it didn't make it.

Note that there exists a patch against some 2.3 for the struct module
that implements a typecode for allowing the packing and unpacking of
arbitrarily lengthed integers, also offering access to the underlying
PyLong functions.  This would likely need to be updated for the post-2.5
struct module, if its functionality is desired.

 - Josiah


> On 11/17/06, Neil Schemenauer <nas at arctrix.com> wrote:
> > ----- Forwarded message from Antti Louko <alo at louko.com> -----
> >
> > Date: Fri, 17 Nov 2006 08:25:03 +0200
> > From: Antti Louko <alo at louko.com>
> > Subject: Python bytes object
> > To: nas at arctrix.com
> >
> > Python bytes object is useful.
> >
> > I would add bitwise logical operations. They would be most useful in
> > cryptographic and graphics operations.
> >
> > Also, encoding and decoding (long) integers in binary fromat to and from
> > bytes type would be useful. Currently, there is no way to efficiently
> > convert long integers to binary formats.
> >
> > Regards,
> >
> >         Antti Louko
> >
> > ----- End forwarded message -----
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
> >
> 
> 
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/jcarlson%40uci.edu


From ncoghlan at gmail.com  Sat Nov 18 02:18:10 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Nov 2006 11:18:10 +1000
Subject: [Python-3000] yes to class decorators
In-Reply-To: <d11dcfba0611170939j2bdfdec1p7016e6896d2e2a8a@mail.gmail.com>
References: <1d85506f0611160324x5d70967aha46c3eff47e85a96@mail.gmail.com>	<455D2435.1000704@canterbury.ac.nz>	<20061116222220.8325.JCARLSON@uci.edu>
	<d11dcfba0611170939j2bdfdec1p7016e6896d2e2a8a@mail.gmail.com>
Message-ID: <455E5F52.1070708@gmail.com>

Steven Bethard wrote:
> Of course, this is almost exactly what the class statement does under
> the covers.

That 'almost' you have there covers some pretty significant differences though.

> The important points in the code above are the ``MAKE_FUNCTION`` and
> ``CALL_FUNCTION`` from the class statement itself, and the
> ``LOAD_LOCALS`` and ``RETURN VALUE`` that were inserted at the end of
> the code for the class's body.

And for anyone tempted to take this parallel too far, the important 
*difference* is the fact that this scope gets ignored for the purposes of 
lexical scoping.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From greg.ewing at canterbury.ac.nz  Sat Nov 18 02:21:35 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Nov 2006 14:21:35 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov17.112300pst.58648@synergy1.parc.xerox.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
Message-ID: <455E601F.1090604@canterbury.ac.nz>

Bill Janssen wrote:
> The use of standard base types (interfaces)
> is basically a communications mechanism that lets functionality
> designers easily tell users of their functionality what's needed.

But it's a blunt tool, because it arbitrarily lumps
together sets of functionality that often are not
required in their entirety. Unless you break the
interfaces down to the level of single methods,
in which case you might as well just document which
methods are required -- as we do now, and it seems
to work fairly well.

> Checking for some textual attributes of a
> type

Now you're talking about LYBL, which is generally
considered an anti-pattern in Python. APIs should
be designed so that you don't need to test for the
presence of features.

> I think that lots of folks aren't exactly uncomfortable with it.
> It's more that they see it as a blemish on what's otherwise a pretty
> good language...  They
> contribute suggestions about how to remove the necessity for it

But this is a fundamental difference of opinion about
what is desirable and what is not. From what you've
said, they see duck typing as a necessary evil, something
to be put up with. Whereas the rest of us see it as a
*good* thing that rescues us from the hell of having to
tediously declare all our interfaces and variable types.
These are opposite and irreconcilable points of view,
as far as I can see.

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Nov 18 02:21:42 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Nov 2006 14:21:42 +1300
Subject: [Python-3000] A plea for anonymous functions
In-Reply-To: <455E1CEE.5030006@acm.org>
References: <455C072B.503@acm.org> <455D2467.2030409@canterbury.ac.nz>
	<455D83E1.4060601@acm.org> <ejk0u7$o6k$1@sea.gmane.org>
	<455E1CEE.5030006@acm.org>
Message-ID: <455E6026.1040300@canterbury.ac.nz>

Talin wrote:
> How is it that 
> people are so hostile to something that I, and apparently others, find 
> so useful?

I don't think it's hostility so much as a weary feeling
that this discussion is revisiting something that's been
hashed over many times before with no satisfactory
result.

If there were a simple and obvious way to fit code blocks
into Python's syntax, we'd probably have them already.
The reason we don't is not because we don't think they'd
be useful, but that we don't think they'd be useful
*enough* to warrant the syntactic awkwardness that would
be required to get them, unless we gave up on indentation-
based block structure, but we like that a *lot*, so it's
not really an option.

If you want to make any headway on this, you're going to
have to come up with an extremely clever syntactic idea.
And it would be a good idea to research previous discussions
on the topic to find out what sort of ideas were not
considered clever enough.

--
Greg

From ncoghlan at gmail.com  Sat Nov 18 02:24:41 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 18 Nov 2006 11:24:41 +1000
Subject: [Python-3000] upon
In-Reply-To: <455DB574.6000207@acm.org>
References: <455C072B.503@acm.org>	<455D2467.2030409@canterbury.ac.nz>	<455D83E1.4060601@acm.org>	<7924.62.39.9.251.1163759219.squirrel@webmail.nerim.net>
	<455DB574.6000207@acm.org>
Message-ID: <455E60D9.20807@gmail.com>

Talin wrote:
> Hmmm, a single syntax that handles both async callbacks, async errors, 
> and switch/case constructs...maybe this would be more useful than I 
> thought ... :)
> 
> Now, can we figure out how to handle arguments to the code block...?

PEP 343 already gives you an answer to this question :)

Cheers,
Nick.

P.S. I'm not saying I agree with the idea. As Paul pointed out, it doesn't let 
you do anything you can't do with a function definition and an appropriate 
decorator (particularly once the nonlocal PEP goes through).

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From janssen at parc.com  Sat Nov 18 04:51:05 2006
From: janssen at parc.com (Bill Janssen)
Date: Fri, 17 Nov 2006 19:51:05 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455E601F.1090604@canterbury.ac.nz> 
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
	<455E601F.1090604@canterbury.ac.nz>
Message-ID: <06Nov17.195106pst."58648"@synergy1.parc.xerox.com>

Greg Ewing writes:
> > Checking for some textual attributes of a
> > type
> 
> Now you're talking about LYBL, which is generally
> considered an anti-pattern in Python. APIs should
> be designed so that you don't need to test for the
> presence of features.

So you join me in wanting to make the use of "hasattr()" unnecessary?

Good.

But when you do have to test, it would be nice to be able to
test for a semantic item rather than a syntactic one.

Bill

From ronaldoussoren at mac.com  Sat Nov 18 16:37:28 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 18 Nov 2006 16:37:28 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov17.112300pst.58648@synergy1.parc.xerox.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
Message-ID: <363050DE-3CA0-445E-BB21-52C66406651F@mac.com>


On  17 Nov 2006, at 8:22 PM, Bill Janssen wrote:

> Greg Ewing wrote:
>> George Sakkis wrote:
>>
>>> And for two, not everyone feels comfortable with duck typing. People
>>> who consider (for better or for worse) isinstance() safer than
>>> hasattr()/getattr() would be accomodated too.
>>
>> The trouble is that building things into the core to
>> "accommodate" these people ends up making things worse
>> for people who don't subscribe to that school of
>> thought, because they will trip over places where
>> some type won't be acceptable because it doesn't
>> inherit from the right bases, even though it
>> implements all the functionality required for their
>> application.
>
> I think that the issue is whether the caller can know whether their
> type "implements all the functionality required for their
> application".  If the library function they are calling is effectively
> opaque, they somehow have to understand all the checking logic that
> that function will perform, to build their type and pass their
> parameters effectively.  The use of standard base types (interfaces)
> is basically a communications mechanism that lets functionality
> designers easily tell users of their functionality what's needed.

IMHO "checking for functionality" is part of the problem here.  Just  
try to use the argument and let the caller worry about supplying the  
right kind of object. IMHO there are only two valid reasons to  
deviate from this: check for optional functionality and some checking  
if the argument will be used a long time in the future (such as with  
callbacks). Even in the last case I usually don't bother.

BTW. Communication with users of a function is what documentation is  
for.

acronymly-challengedly-yours,
	
	Ronald


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061118/532279dd/attachment.bin 

From george.sakkis at gmail.com  Sat Nov 18 17:18:15 2006
From: george.sakkis at gmail.com (George Sakkis)
Date: Sat, 18 Nov 2006 11:18:15 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455E601F.1090604@canterbury.ac.nz>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
	<455E601F.1090604@canterbury.ac.nz>
Message-ID: <91ad5bf80611180818l290e0a3j9f7f3f0b0e42007b@mail.gmail.com>

On 11/17/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Bill Janssen wrote:
> > Checking for some textual attributes of a
> > type
>
> Now you're talking about LYBL, which is generally
> considered an anti-pattern in Python. APIs should
> be designed so that you don't need to test for the
> presence of features.

The end user may not have to explicitly test for it, but it happens
anyway behind the scenes; x.foo is functionally equivalent to
if hasattr(x,'foo'): return getattr(x,'foo')
else: raise AttributeError()

I think Bill's point is not whether you have to be explicit about it
or the language does it for you, but that the textual check must
happen at some point, and that this is inherently unsafer than a
semantic check.

George

From ark-mlist at att.net  Sat Nov 18 22:00:40 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sat, 18 Nov 2006 16:00:40 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
Message-ID: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>

I've just come up with an idea that I think may help clarify this discussion
in a new way.  I'm afraid the description is going to be a little long; but
I'd appreciate it if you would bear with me and read the whole thing.

I'd like to start by coining a usage, namely the "abilities" of a type.  By
an ability, I mean a claim such as "Objects of type Foo are iterators" or
"Objects of type Foo are copyable."  I would like to use the word "ability"
to refer to "iterator" and "copyable" in this context.

When we say that a an object of a given class (or the class itself) has an
ability, we are really saying that objects of that class behave in a
particular way when used in a particular way.  So, for example, when we say
that x is an iterator (or x has the iterator ability), we mean that we can
call x.next() to obtain successive values of a sequence, and that doing so
for long enough will eventually raise StopIteration.

So far, this discussion has been about the idea that a class should be able
to acquire an ability by inheriting it.  For quite a while, I thought this
was an excellent idea; but now I'm not so sure.  Here's why.

I'll begin by explaining why I think it's an excellent idea.  Suppose you
have an object x of unknown type, and you want to know whether x is an
iterator.  Some people in this discussion have suggested that the Pythonic
way to answer this question is to try to use it as an iterator and see if it
raises an exception.  The trouble with this technique is that it is
destructive if it succeeds, because it consumes an element of the sequence.
That means you can't use the technique until you have something to do with
that first element, which rules out some plausible usages.

I've raised this issue in discussion before, and gotten the response that
the right way to deal with this problem is to use getattr(x,"next").  Doing
so is certainly a way of determining whether x.next() is meaningful, but it
doesn't offer a general way of telling whether an object has a given
ability.  For example, there is no analogous technique for determining
whether len(x) will work without evaluating it.  Nor is there a way to know
whether someone just happened to define x.next() without intending to
implement the iterator protocol.

A good bit of the current discussion is about a way to solve this kind of
problem, namely to define types that represent abilities.  So, for example,
if "iterator" were such a type, we could evaluate isinstance(x,iterator) to
determine whether x is of a type that is intended to behave as an iterator.
I've proposed this idea myself a while ago, and had been quite fond of it.
Yes, there is an objection to this scheme, namely that existing user-defined
iterator types would not automatically inherit from the "iterator" type; but
maybe there is a way around that problem.

However, I want to argue against this idea for a completely different
reason: Inheritance should establish an "is-a" relationship, and that's not
the right kind of relationship here.  To put it bluntly:

	I can see no reason why the iterator type should
	have the iterator ability!

More generally, I see no reason why a type that signals the presence of a
given ability should itself possess that ability.  Because if it does, then
it becomes difficult to distinguish types that signal abilities from types
that do not.

In other words:  Suppose that we have a family of types with names such as
"iterator", "copyable", "equality_comparable", "comparable", and so on.
Each of these types signals the corresponding ability.  Some of these types
may be related by inheritance.  For example, I would expect "comparable" to
inherit from "equality_comparable", because any type that is comparable had
better be comparable for equality.

It seems to me to make sense for the notion of "signaling an ability" to be
an ability.  In other words, the types in the last paragraph can all be used
in a particular way, which is the hallmark of an ability.

Now, suppose that every type that signals the presence of an ability
inherits from "ability".  Then we've just broken the whole ability system.
Because if "iterator" inherits from "ability" and every iterator type
inherits from "iterator", then every iterator is also able to signal the
presence of an ability--and we most definitely do not want that!

In other words, whether or not we choose to define a family of types that
stand for particular abilities, I think we should not use inheritance as the
mechanism for specifying that a particular class has a particular ability.
I don't know what the mechanism should be, but it shouldn't be inheritance.

Am I missing something?




							--Andrew Koenig



From bob at redivi.com  Sat Nov 18 22:30:06 2006
From: bob at redivi.com (Bob Ippolito)
Date: Sat, 18 Nov 2006 13:30:06 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ejeq56$65k$1@sea.gmane.org>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131417if5008f4g177543111c08e03f@mail.gmail.com>
	<91ad5bf80611131549i7ce82745j39fd4313d759f5f4@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<ejeq56$65k$1@sea.gmane.org>
Message-ID: <6a36e7290611181330w728261c3tc148ef2a69f66c56@mail.gmail.com>

On 11/15/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> George Sakkis wrote:
>
> > Fredrik, I am not arguing for the argument's sake, I just don't get
> > it: Python requires my class to define __len__, otherwise len() fails.
> > Why not require len() as a method instead and forget about __len__ ?
>
> and have people complain about a len/__getitem__ naming inconsistency?
> (cf. the __iter__/next vs. __iter__/__next__ discussions.)
>
> > Does len() (the function) do anything smarter behind the scenes than
> > just passing the ball to __len__ ?
>
> yes.  len() can use alternative approaches to determine the length for
> objects implemented in the host language.  this is used by CPython to
> efficiently implement len() for C-level objects without having to do
> full Python method resolution and dispatch for each call.
>
> you could of course turn len() into a __len__() helper (or even
> sequencetools.len(), if you prefer) for cases where this matters, or
> even reserve the "len" method name, but I'm not sure how that would
> make things *simpler* than they are today.
>
> I'm convinced that it's better for people to get over that silly notion
> that writing x.foo() is somehow always "better" than writing foo(x), in
> a language that actually supports both forms.
>
> (if I'd have to chose between foo(x) and x.foo(), I'd rather get foo(x)
> with multiple dispatch than today's x.foo() single dispatch, but that's
> probably too radical for Python 3000 ;-)

Same here. I've tended to write mostly-functions for a long time, and
PJE's RuleDispatch made it even more compelling to do things that way.
It also made it really easy to learn Erlang and get accustomed to its
pattern matching. All they have is functions and modules, and it works
out rather well because of pattern matching (essentially generic
functions). Instead of classes you just keep the state as a parameter.

-bob

From sluggoster at gmail.com  Sat Nov 18 23:50:17 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Sat, 18 Nov 2006 14:50:17 -0800
Subject: [Python-3000] Fwd:  Builtin iterator type
In-Reply-To: <6e9196d20611181449wbc55cd2j206ae649bcd68c0f@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<ejeq56$65k$1@sea.gmane.org>
	<6a36e7290611181330w728261c3tc148ef2a69f66c56@mail.gmail.com>
	<6e9196d20611181449wbc55cd2j206ae649bcd68c0f@mail.gmail.com>
Message-ID: <6e9196d20611181450o6d6ba5d5pfe207dfdf904ce3@mail.gmail.com>

On 11/18/06, Bob Ippolito <bob at redivi.com> wrote:
> On 11/15/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > I'm convinced that it's better for people to get over that silly notion
> > that writing x.foo() is somehow always "better" than writing foo(x), in
> > a language that actually supports both forms.
> >
> Instead of classes you just keep the state as a parameter.

Classes make nice bundles of methods that you can pass around as a
unit. They show that "this goes with that". You can do the same thing
with modules but not as flexibly. For instance, you can have several
classes in one module, but not several modules in one module.

--
Mike Orr <sluggoster at gmail.com>

From greg.ewing at canterbury.ac.nz  Sun Nov 19 02:15:49 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 19 Nov 2006 14:15:49 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov17.195106pst.58648@synergy1.parc.xerox.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<ca471dc20611131745p8fdbf9fp2f6d2c3cc3834e3a@mail.gmail.com>
	<91ad5bf80611131926iaef336dk1f2daf5529123a33@mail.gmail.com>
	<455A7CD1.5090503@canterbury.ac.nz>
	<91ad5bf80611142233i6b1e60e6i65e79da831e152b9@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
	<455E601F.1090604@canterbury.ac.nz>
	<06Nov17.195106pst.58648@synergy1.parc.xerox.com>
Message-ID: <455FB045.6080908@canterbury.ac.nz>

Bill Janssen wrote:

> But when you do have to test, it would be nice to be able to
> test for a semantic item rather than a syntactic one.

In your own code, there's nothing to stop you from
creating your own interface classes to use in your
own tests.

But when it comes to interoperating with other
people's code, you can't rely on them having
inherited from all the classes you want to test
for, so the testing approach isn't much use.
Unless you require everyone to inherit from a
standard set of base classes, and then we're
back in Java-land.

Anyhow, I've yet to see a case where you really
*need* that sort of testing. Whenever someone asks
something like "how do I tell whether I've got a
sequence or not", usually it's a symptom of poor
API design, IMO.

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Nov 19 02:15:55 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 19 Nov 2006 14:15:55 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
Message-ID: <455FB04B.3010706@canterbury.ac.nz>

Andrew Koenig wrote:
> Some people in this discussion have suggested that the Pythonic
> way to answer this question is to try to use it as an iterator and see if it
> raises an exception.

That's not what I would say. I would say that you
should design your code so that you don't *need*
to find out whether it's an iterator or not. Instead,
you just require an iterator, always, and trust that
the calling code will give you one. If it doesn't,
your unit tests will catch that. (You *do* have
unit tests, don't you?-)

> I think we should not use inheritance as the
> mechanism for specifying that a particular class has a particular ability.
> I don't know what the mechanism should be, but it shouldn't be inheritance.

Your ability-signallers seem to be something akin to
the Haskell notion of typeclasses. A typeclass is not
itself a type, but represents a set of characteristics
that a type can have.

There have been various proposals before for ways of
registering interfaces outside of the inheritance
hierarchy. It wouldn't be hard to come up with something
if it were thought desirable. I'm still not convinced
that it solves a problem that really needs solving,
however.

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Nov 19 02:15:59 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 19 Nov 2006 14:15:59 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611180818l290e0a3j9f7f3f0b0e42007b@mail.gmail.com>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<91ad5bf80611150047p73bfbffaqb34d1eafa4618d52@mail.gmail.com>
	<91ad5bf80611150140r1b1c7d8cwcd8c730ca520d6c@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
	<455E601F.1090604@canterbury.ac.nz>
	<91ad5bf80611180818l290e0a3j9f7f3f0b0e42007b@mail.gmail.com>
Message-ID: <455FB04F.6050105@canterbury.ac.nz>

George Sakkis wrote:

> The end user may not have to explicitly test for it, but it happens
> anyway behind the scenes; x.foo is functionally equivalent to
> if hasattr(x,'foo'): return getattr(x,'foo')
> else: raise AttributeError()

Well, yes, but the point is that since this is being
done automatically anyway, doing it yourself as
well is redundant.

> the textual check must
> happen at some point, and that this is inherently unsafer than a
> semantic check.

But testing for a base class is a rather crude form
of semantic check, because it bundles a lot of semantics
together that you often don't need bundled.

The only completely accurate semantic check is to run
the program and see if it produces the right result.
In other words, don't LBYL, but Just Do It, and use
unit tests.

--
Greg

From ncoghlan at gmail.com  Sun Nov 19 05:00:00 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Nov 2006 14:00:00 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
Message-ID: <455FD6C0.7000105@gmail.com>

Andrew Koenig wrote:
> In other words, whether or not we choose to define a family of types that
> stand for particular abilities, I think we should not use inheritance as the
> mechanism for specifying that a particular class has a particular ability.
> I don't know what the mechanism should be, but it shouldn't be inheritance.
> 
> Am I missing something?

It seems to me to be an excellent write-up of why using concrete classes for 
interface definitions is potentially problematic. (Although I will point out 
that most  protocols for things like len() *do* involve checks for special 
methods by name, and the check for iterability is typically a non-destructive 
call to iter(x), rather than a destructive one to x.next()).

To make multiple dispatch practical, I suspect we will end up with something 
more along of the lines of Zope.interface to 'formalise' Python interface 
definitions.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Sun Nov 19 05:13:13 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 18 Nov 2006 20:13:13 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FD6C0.7000105@gmail.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<455FD6C0.7000105@gmail.com>
Message-ID: <ca471dc20611182013y496d7f2dt2bd96d98c45c765f@mail.gmail.com>

On 11/18/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> (Although I will point out
> that most  protocols for things like len() *do* involve checks for special
> methods by name, and the check for iterability is typically a non-destructive
> call to iter(x), rather than a destructive one to x.next()).

Ouch?! I would never check for iterability explicitly. I would just
require it, as Greg Ewing says. iter() could be fairly expensive,
depending on what is being iterated over.

Maybe this is unique to iterators and iterators are a bad example? I
don't feel as strongly about this when testing e.g. for string-ness,
file-ness or list-ness. While I tend to dislike those tests too, they
are sometimes inevitable when we want to overload an API. Especially
file/string is often overloaded, and I see no big problem with this (I
do dislike overloading list/dict, or string/list-of-strings).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Nov 19 05:28:48 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 18 Nov 2006 20:28:48 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
Message-ID: <ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>

On 11/18/06, Andrew Koenig <ark-mlist at att.net> wrote:
> I've just come up with an idea that I think may help clarify this discussion
> in a new way.  I'm afraid the description is going to be a little long; but
> I'd appreciate it if you would bear with me and read the whole thing.
>
> I'd like to start by coining a usage, namely the "abilities" of a type.  By
> an ability, I mean a claim such as "Objects of type Foo are iterators" or
> "Objects of type Foo are copyable."  I would like to use the word "ability"
> to refer to "iterator" and "copyable" in this context.
>
> When we say that a an object of a given class (or the class itself) has an
> ability, we are really saying that objects of that class behave in a
> particular way when used in a particular way.  So, for example, when we say
> that x is an iterator (or x has the iterator ability), we mean that we can
> call x.next() to obtain successive values of a sequence, and that doing so
> for long enough will eventually raise StopIteration.
>
> So far, this discussion has been about the idea that a class should be able
> to acquire an ability by inheriting it.  For quite a while, I thought this
> was an excellent idea; but now I'm not so sure.  Here's why.
>
> I'll begin by explaining why I think it's an excellent idea.  Suppose you
> have an object x of unknown type, and you want to know whether x is an
> iterator.  Some people in this discussion have suggested that the Pythonic
> way to answer this question is to try to use it as an iterator and see if it
> raises an exception.  The trouble with this technique is that it is
> destructive if it succeeds, because it consumes an element of the sequence.
> That means you can't use the technique until you have something to do with
> that first element, which rules out some plausible usages.
>
> I've raised this issue in discussion before, and gotten the response that
> the right way to deal with this problem is to use getattr(x,"next").  Doing
> so is certainly a way of determining whether x.next() is meaningful, but it
> doesn't offer a general way of telling whether an object has a given
> ability.  For example, there is no analogous technique for determining
> whether len(x) will work without evaluating it.  Nor is there a way to know
> whether someone just happened to define x.next() without intending to
> implement the iterator protocol.
>
> A good bit of the current discussion is about a way to solve this kind of
> problem, namely to define types that represent abilities.  So, for example,
> if "iterator" were such a type, we could evaluate isinstance(x,iterator) to
> determine whether x is of a type that is intended to behave as an iterator.
> I've proposed this idea myself a while ago, and had been quite fond of it.
> Yes, there is an objection to this scheme, namely that existing user-defined
> iterator types would not automatically inherit from the "iterator" type; but
> maybe there is a way around that problem.
>
> However, I want to argue against this idea for a completely different
> reason: Inheritance should establish an "is-a" relationship, and that's not
> the right kind of relationship here.  To put it bluntly:
>
>         I can see no reason why the iterator type should
>         have the iterator ability!
>
> More generally, I see no reason why a type that signals the presence of a
> given ability should itself possess that ability.  Because if it does, then
> it becomes difficult to distinguish types that signal abilities from types
> that do not.
>
> In other words:  Suppose that we have a family of types with names such as
> "iterator", "copyable", "equality_comparable", "comparable", and so on.
> Each of these types signals the corresponding ability.  Some of these types
> may be related by inheritance.  For example, I would expect "comparable" to
> inherit from "equality_comparable", because any type that is comparable had
> better be comparable for equality.
>
> It seems to me to make sense for the notion of "signaling an ability" to be
> an ability.  In other words, the types in the last paragraph can all be used
> in a particular way, which is the hallmark of an ability.
>
> Now, suppose that every type that signals the presence of an ability
> inherits from "ability".  Then we've just broken the whole ability system.
> Because if "iterator" inherits from "ability" and every iterator type
> inherits from "iterator", then every iterator is also able to signal the
> presence of an ability--and we most definitely do not want that!
>
> In other words, whether or not we choose to define a family of types that
> stand for particular abilities, I think we should not use inheritance as the
> mechanism for specifying that a particular class has a particular ability.
> I don't know what the mechanism should be, but it shouldn't be inheritance.
>
> Am I missing something?
>
>                                                         --Andrew Koenig

This looks like you're rediscovering or reinventing Java- or
Zope-style interfaces: those are not passed on via inheritance but via
a separate "implements" clause. I don't remember how it's done in Java
(and don't have time to look it up) but it makes sense to me to use
something akin to inheritance for constructing interfaces out of other
interfaces, like your example of comparable inheriting from
equality-comparable. But "being an interface" is not transmitted
through this same mechanism -- that's a meta-property. I think that it
would be a mistake to us isinstance to test for an interface. I
believe Zope already solves most of the deeper problems; the main
issue seems to be that the surface syntax for creating and using
interfaces isn't very elegant (due to lack of language support) and
that the standard types and common de-facto interfaces (e.g. sequence,
mapping, iterator, file) don't play along and to some extent defy
capturing their semantics in interfaces.

I realize that Java- and Zope-style interfaces *seem* to be all about
syntax (they define names and signatures of methods but not semantics)
but IMO that's only a theoretical objection; in *practice* these
interfaces have strong connotations of semantics (albeit written in
Enlish rather than using assertions or pre- and post-conditions) and
nobody would think of claiming to implement an interface without
implementing the proper semantics (or some interpretation thereof :-).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ntoronto at cs.byu.edu  Sun Nov 19 06:42:19 2006
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Sat, 18 Nov 2006 22:42:19 -0700
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
Message-ID: <455FEEBB.9010100@cs.byu.edu>

Guido van Rossum wrote:
> I realize that Java- and Zope-style interfaces *seem* to be all about
> syntax (they define names and signatures of methods but not semantics)
> but IMO that's only a theoretical objection; in *practice* these
> interfaces have strong connotations of semantics (albeit written in
> Enlish rather than using assertions or pre- and post-conditions) and
> nobody would think of claiming to implement an interface without
> implementing the proper semantics (or some interpretation thereof :-)

Actually, plenty of people would dream of it and even do it. I've seen 
some pretty evil implementations of Java interfaces. All they can 
enforce is static types and method signatures.

My main point is, unless you're willing to go whole-hog with pre- and 
post-conditions (and even that would be insufficient, as you can't check 
anything like "this eventually, on some call, responds with x" without 
spelling it out in temporal logic and running model checking software), 
no interface can enforce anything *semantically* meaningful. The most 
useful thing an interface can do is *communicate* semantics to another 
programmer.

I'm not sure I said anything Guido didn't already say, but I wanted to 
make the distinction between *enforcing* correct behavior and 
*communicating* correct behavior. Java (and Zope, apparently) simply 
have a more formal way of doing this. Stock Python doesn't.

Neil


From ncoghlan at gmail.com  Sun Nov 19 07:26:39 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 19 Nov 2006 16:26:39 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611182013y496d7f2dt2bd96d98c45c765f@mail.gmail.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>	
	<455FD6C0.7000105@gmail.com>
	<ca471dc20611182013y496d7f2dt2bd96d98c45c765f@mail.gmail.com>
Message-ID: <455FF91F.2000800@gmail.com>

Guido van Rossum wrote:
> On 11/18/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> (Although I will point out
>> that most  protocols for things like len() *do* involve checks for 
>> special
>> methods by name, and the check for iterability is typically a 
>> non-destructive
>> call to iter(x), rather than a destructive one to x.next()).
> 
> Ouch?! I would never check for iterability explicitly. I would just
> require it, as Greg Ewing says. iter() could be fairly expensive,
> depending on what is being iterated over.

I didn't mean to imply that the call to iter() couldn't be an implicit one in 
a for loop/genexp/listcomp/itertool. As you say, the need to check explicitly 
is rare.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From solipsis at pitrou.net  Sun Nov 19 11:13:26 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Nov 2006 11:13:26 +0100
Subject: [Python-3000] interfaces
In-Reply-To: <455FEEBB.9010100@cs.byu.edu>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu>
Message-ID: <1163931206.4510.30.camel@fsol>


Le samedi 18 novembre 2006 ? 22:42 -0700, Neil Toronto a ?crit :
> Actually, plenty of people would dream of it and even do it. I've seen 
> some pretty evil implementations of Java interfaces. All they can 
> enforce is static types and method signatures.

But protecting against erroneous use (rather than deliberate misuse) can
be a reasonable goal. There are cases where relying on duck-typing leads
to silent bugs instead of nice explicit tracebacks. Then as the
implementer of the API it is good to be able to enforce some aspects of
the API, so that your users don't lose time finding those bugs.


I had the problem recently when I wrote a decorator which took an
optional number as an argument. The decorator declaration goes like
this:

def deferred(timeout=None):
    def decorate(func):
        blah blah...
    return decorate

It can be used like this:

@deferred()
def test_case():
    ...

The problem was if you forgot the parentheses altogether:

@deferred
def test_case():
    ...

In that case, no exception was raised, but the test was silently
ignored. Also, "@deferred" doesn't strike at first sight like something
is missing (parameterless decorators do exist).

So to know if "deferred" was correctly used, I had to devise a test to
know if "timeout" is a number (without mandating that it is a e.g. float
rather than an int). I ended up with this:
    try:
        timeout is None or timeout + 0
    except TypeError:
        raise TypeError("'timeout' argument must be a number or None")

Instead of testing "timeout" against an arbitrary arithmetic operation,
it would be more robust to test it against an interface (say "Number").
I think that's the kind of use cases which makes people oppose the idea
of removing callable(). It feels more robust to test against an
interface than against a specific trait of that interface.




From jan.grant at bristol.ac.uk  Sun Nov 19 12:48:35 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Sun, 19 Nov 2006 11:48:35 +0000 (GMT)
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
Message-ID: <20061119113835.D14417@tribble.ilrt.bris.ac.uk>

On Sat, 18 Nov 2006, Andrew Koenig wrote:

> Now, suppose that every type that signals the presence of an ability
> inherits from "ability".  Then we've just broken the whole ability system.
> Because if "iterator" inherits from "ability" and every iterator type
> inherits from "iterator", then every iterator is also able to signal the
> presence of an ability--and we most definitely do not want that!

> Am I missing something?

I believe you have a type error in this argument.

Instance x of class X has ability A, you suggest, should be indicated by
having X derive from signalling_class(A). That's is-a: All Xs are 
signalling_class(A)s. Then sc(A) is the class of objects with 
ability A.

You want to indicate that sc(A) is a class that signals an ability. You 
suggest you do this by having sc(A) derive from a base class, 
sc(Signals_Abilities).

That is a type error: to indicate that sc(A) is an ability-signalling 
class, your original convention is that sc(A) should be an _instance_ of 
sc(Signals_Abilities). That is not represented by an inheritance 
relationship between sc(A) and sc(Signals_Abiliities), but by a 
membership relationship.

So, sc(Signals_Abilities) is actually a metaclass, I think.

jan

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Spreadsheet through network. Oh yeah.

From jan.grant at bristol.ac.uk  Sun Nov 19 12:57:38 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Sun, 19 Nov 2006 11:57:38 +0000 (GMT)
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FEEBB.9010100@cs.byu.edu>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu>
Message-ID: <20061119115009.V14417@tribble.ilrt.bris.ac.uk>

On Sat, 18 Nov 2006, Neil Toronto wrote:

> Guido van Rossum wrote:
> > I realize that Java- and Zope-style interfaces *seem* to be all about
> > syntax (they define names and signatures of methods but not semantics)
> > but IMO that's only a theoretical objection; in *practice* these
> > interfaces have strong connotations of semantics (albeit written in
> > Enlish rather than using assertions or pre- and post-conditions) and
> > nobody would think of claiming to implement an interface without
> > implementing the proper semantics (or some interpretation thereof :-)
> 
> Actually, plenty of people would dream of it and even do it. I've seen 
> some pretty evil implementations of Java interfaces. All they can 
> enforce is static types and method signatures.

This seems to be an argument that interfaces can be misused by people 
without taste. That's true; but the same arguments have been levelled 
against pretty much all language features (operator overloading, 
asepct-orientation, etc).

It strikes me that one aspect of "being Pythonic" is a strong reliance 
on politeness: that's what duck-typing is all about. Or conventions for 
private attributes. Such features could potentially be abused, but a 
Python programmer has the good taste to know what's abuse and what 
isn't.

> My main point is, unless you're willing to go whole-hog with pre- and 
> post-conditions (and even that would be insufficient, as you can't check 
> anything like "this eventually, on some call, responds with x" without 
> spelling it out in temporal logic and running model checking software), 
> no interface can enforce anything *semantically* meaningful. The most 
> useful thing an interface can do is *communicate* semantics to another 
> programmer.

Interfaces are more than just language constructs. They come with a 
semantics. It's true that that semantics may be spelled out in javadoc, 
but since the consumer of the interface is another programmer, that's 
ok: we want to communicate effectively to that programmer the contract 
that they are agreeing to uphold when they implement that interface.

It's true that lots of Java programmers may not quite grok this from the 
word go; but fundamentally an interface should be viewed as a type (or 
ability marker, as Andrew K called it) and comes with a corresponding 
set of semantics. That Java doesn't make an attempt to spell out 
explicitly in code those other semantics isn't necessarily a weakness.

Java interfaces are very useful, however. Java programming seems to be 
less and less about inheritance and more and more about implementing 
interfaces; at least it does amongst Java programmers with taste :-)

Cheers,
jan

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Whenever I see a dog salivate I get an insatiable urge to ring a bell.

From ark-mlist at att.net  Sun Nov 19 17:40:46 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun, 19 Nov 2006 11:40:46 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FB04B.3010706@canterbury.ac.nz>
Message-ID: <000501c70bf9$791c2b80$6402a8c0@arkdesktop>

> That's not what I would say. I would say that you
> should design your code so that you don't *need*
> to find out whether it's an iterator or not. Instead,
> you just require an iterator, always, and trust that
> the calling code will give you one. If it doesn't,
> your unit tests will catch that. (You *do* have
> unit tests, don't you?-)

That's a nice general sentiment, but not always feasible -- especially if
you're writing library code.  Remember, I'm thinking about abilities in
general, not just whether a type is an iterator.  For example, testing
whether something is an iterator by trying to call its next() method is
destructive if it succeeds.  So consider the following:

	def concat(x, y):
		for a in x: yield a
		for a in y: yield a

Simple as this function may be, it has a drawback: If x is an iterator and y
is not a valid sequence, the function consumes x before failing.  If it were
possible to determine nondestructively whether y is a sequence, the function
would not have this drawback.  The drawback in this particular example may
be trivial; but similar drawbacks can be serious in other contexts,
especially where networking is involved.

> Your ability-signallers seem to be something akin to
> the Haskell notion of typeclasses. A typeclass is not
> itself a type, but represents a set of characteristics
> that a type can have

To the extent that I'm familiar with Haskell, I think you're right.




From ark-mlist at att.net  Sun Nov 19 17:42:09 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun, 19 Nov 2006 11:42:09 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FB04F.6050105@canterbury.ac.nz>
Message-ID: <000601c70bf9$b015f620$6402a8c0@arkdesktop>

> The only completely accurate semantic check is to run
> the program and see if it produces the right result.

If that were possible, we could solve the halting problem :-)

> In other words, don't LBYL, but Just Do It, and use
> unit tests.

A fine idea when it's possible.  Unfortunately, it's not always possible.



From ark-mlist at att.net  Sun Nov 19 17:56:31 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun, 19 Nov 2006 11:56:31 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
Message-ID: <000701c70bfb$a9faef00$6402a8c0@arkdesktop>

> This looks like you're rediscovering or reinventing Java- or
> Zope-style interfaces: those are not passed on via inheritance but via
> a separate "implements" clause.

I don't think I'm quite reinventing Java interfaces.  In Java, where just
about everything is a method, it's temptingly easy to talk about the
interface to a type by making claims about its methods and the types they
return.  But in Python, such claims don't mesh well with the language's
dynamic nature.

So what I'm really looking for is a formal way of claiming that a type has
an informal property, and leaving it up to the programmers (and their unit
tests) to enforce the correspondence between the claims and the properties.

For iterators, for example, it's not just the presence of the __iter__ and
next methods that are relevant; it's also the notion that calling next can
raise StopIteration in the ordinary course of events, but if it raises any
other exception, it should be considered to be truly exceptional.  I don't
know how to capture such specifications formally, but I do think it would be
useful for a programmer to have a formal way of claiming that a given class
meets a given specification.


> I don't remember how it's done in Java
> (and don't have time to look it up) but it makes sense to me to use
> something akin to inheritance for constructing interfaces out of other
> interfaces, like your example of comparable inheriting from
> equality-comparable. But "being an interface" is not transmitted
> through this same mechanism -- that's a meta-property. I think that it
> would be a mistake to us isinstance to test for an interface.

Yes -- that's it exactly!

> I realize that Java- and Zope-style interfaces *seem* to be all about
> syntax (they define names and signatures of methods but not semantics)
> but IMO that's only a theoretical objection; in *practice* these
> interfaces have strong connotations of semantics (albeit written in
> Enlish rather than using assertions or pre- and post-conditions) and
> nobody would think of claiming to implement an interface without
> implementing the proper semantics (or some interpretation thereof :-).

I agree almost 100%.  Unfortunately, some people claim to implement
interfaces without appearing to think at all; but that's another subject
entirely :-)



From ark-mlist at att.net  Sun Nov 19 17:57:50 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun, 19 Nov 2006 11:57:50 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FEEBB.9010100@cs.byu.edu>
Message-ID: <000801c70bfb$d9161b70$6402a8c0@arkdesktop>


> I'm not sure I said anything Guido didn't already say, but I wanted to
> make the distinction between *enforcing* correct behavior and
> *communicating* correct behavior. Java (and Zope, apparently) simply
> have a more formal way of doing this. Stock Python doesn't.

What he said!



From ark-mlist at att.net  Sun Nov 19 18:03:28 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sun, 19 Nov 2006 12:03:28 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061119115009.V14417@tribble.ilrt.bris.ac.uk>
Message-ID: <000901c70bfc$a8bbe9e0$6402a8c0@arkdesktop>

> It strikes me that one aspect of "being Pythonic" is a strong reliance
> on politeness: that's what duck-typing is all about.

Part of my motivation for entering this discussion is that C++ templates use
duck-typing, and the C++ community has for several years been thinking about
how to add more explicit constraints to the type system.

One motivation is easier diagnostics: If by mistake you hand a non-sequence
to something that requires a sequence, it would be nice for the compiler to
be able to say "This argument does not satisfy the 'sequence' constraint"
rather than spewing forth a page of error messages generated deep in the
internals of the library.

I understand that this exact motivation doesn't apply in Python because of
its dynamic typing.  However, I've seen examples in Python that feel to me
like that address similar problems to the ones in C++.  Of course, such
feelings don't prove anything; but sometimes they help point the way to a
useful solution.



From gsakkis at rutgers.edu  Sun Nov 19 18:44:15 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Sun, 19 Nov 2006 12:44:15 -0500
Subject: [Python-3000] interfaces
In-Reply-To: <1163931206.4510.30.camel@fsol>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu> <1163931206.4510.30.camel@fsol>
Message-ID: <91ad5bf80611190944y66f4cbedyc4bf324d7dbda028@mail.gmail.com>

On 11/19/06, Antoine Pitrou <solipsis at pitrou.net> wrote:

> I had the problem recently when I wrote a decorator which took an
> optional number as an argument. The decorator declaration goes like
> this:
>
> def deferred(timeout=None):
>     def decorate(func):
>         blah blah...
>     return decorate
>
> It can be used like this:
>
> @deferred()
> def test_case():
>     ...
>
> The problem was if you forgot the parentheses altogether:
>
> @deferred
> def test_case():
>     ...
>
> In that case, no exception was raised, but the test was silently
> ignored. Also, "@deferred" doesn't strike at first sight like something
> is missing (parameterless decorators do exist).
>
> So to know if "deferred" was correctly used, I had to devise a test to
> know if "timeout" is a number (without mandating that it is a e.g. float
> rather than an int). I ended up with this:
>     try:
>         timeout is None or timeout + 0
>     except TypeError:
>         raise TypeError("'timeout' argument must be a number or None")

I understand this is not the point you're trying to make, but in such
cases I usually prefer to make @decorator be equivalent to
@decorator() by something like:

def deferred(timeout=None):
    if callable(timeout):
        f = timeout; timeout = None
    else:
        f = None
    def decorate(func):
        blah blah...
    return decorate if f is None else decorate(f)

Of course this doesn't work if it's valid for the timeout parameter to
be a callable, but usually, like in your example, it's not.

> Instead of testing "timeout" against an arbitrary arithmetic operation,
> it would be more robust to test it against an interface (say "Number").
> I think that's the kind of use cases which makes people oppose the idea
> of removing callable(). It feels more robust to test against an
> interface than against a specific trait of that interface.

The counter-argument is that instead of picking an arbitrary trait of
the interface to check, make sure your unit tests fail if the passed
value is not appropriate. The real problem you should look into is why
"in that case, no exception was raised, but the test was silently
ignored". I don't know how 'timeout' is used in your example, but from
a similar bad experience, I'd guess all you are doing with it is
compare it with a number ("if timeout < 3") and comparisons between
instances of different types don't raise an exception.

That's perhaps one of the top gotchas in Python today; thankfully it
will change in 3.0. Until then, I can't see how one can avoid an
explicit check, either by testing with isinstance() or trying to call
an arbitrary method of the expected protocol. Having said that both
are necessary evils, I'd go with the isinstance check as the lesser
evil. Checking for an arbitrary method is not only an ugly kludge; it
is both unnecessary (if the tested method is not used in the actual
code) and error-prone (e.g. for instances of a type that just happens
to define __add__ so that x + 0 doesn't raise an exception, although x
is an unappropriate argument in the specific context).

George

From python3now at gmail.com  Sun Nov 19 19:13:30 2006
From: python3now at gmail.com (James Thiele)
Date: Sun, 19 Nov 2006 10:13:30 -0800
Subject: [Python-3000] print() parameters in py3k
Message-ID: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>

The BDFL has said that print will change from a statement to a function in
Python 3.

I haven't found anything describing what parameters it will take and what it
will return. Has this been decided?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061119/c1a460ca/attachment.htm 

From sluggoster at gmail.com  Sun Nov 19 19:28:42 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Sun, 19 Nov 2006 10:28:42 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FB04F.6050105@canterbury.ac.nz>
References: <91ad5bf80611131044q44fa97eesdf37cdc0bac44fe8@mail.gmail.com>
	<-9052809295998148604@unknownmsgid>
	<91ad5bf80611151036p248fde9ate83b21f631fe7fc3@mail.gmail.com>
	<455BBA5F.5070106@canterbury.ac.nz>
	<91ad5bf80611151849v70f413e5s9085ae100a3a4741@mail.gmail.com>
	<455D2461.8000602@canterbury.ac.nz>
	<06Nov17.112300pst.58648@synergy1.parc.xerox.com>
	<455E601F.1090604@canterbury.ac.nz>
	<91ad5bf80611180818l290e0a3j9f7f3f0b0e42007b@mail.gmail.com>
	<455FB04F.6050105@canterbury.ac.nz>
Message-ID: <6e9196d20611191028u4b3ad027kf8e8031bb446cae@mail.gmail.com>

On 11/18/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> George Sakkis wrote:
>
> > The end user may not have to explicitly test for it, but it happens
> > anyway behind the scenes; x.foo is functionally equivalent to
> > if hasattr(x,'foo'): return getattr(x,'foo')
> > else: raise AttributeError()
>
> Well, yes, but the point is that since this is being
> done automatically anyway, doing it yourself as
> well is redundant.

*Something* is being done automatically but it's not the same thing.
The error message may be so unrelated to the actual problem as to give
no clue what's wrong.   Good code should reflect one's mental model,
as close to "one line per step" as feasable. If the language can have
features to facilitate this, why not? Many Pythoneers find interfaces
useful, or would if the language had them.  Making a concrete class
with dummy methods is kind of pointless when it'll never be used for
anything except isinstance(), and it's even worse at enforcing
attribute existence.  ("Yippee, I have a default attribute that's
None.")

There have long been discussions about adding interfaces and/or
adapters to Python, so it will probably happen someday. As has been
mentioned, Zope has had an interface implementation for years. PEPs
245 and 246 (both rejected) contain specific proposals for interfaces
and adapters. The last discussion I saw was in Guido's blog (April
2006):
http://www.artima.com/forums/flat.jsp?forum=106&thread=155123
There the discussion was on how generic functions could fulfill both
roles and more besides. The discussion seemed to fizzle out at, "Yes,
this would be nice to have," without a concrete proposal.

Simultaneously there was discussion on this list about it, although I
wasn't subscribed then so I haven't fully read it:
http://mail.python.org/pipermail/python-3000/2006-April/thread.html

-- 
Mike Orr <sluggoster at gmail.com>

From sluggoster at gmail.com  Sun Nov 19 19:56:01 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Sun, 19 Nov 2006 10:56:01 -0800
Subject: [Python-3000] interfaces
In-Reply-To: <91ad5bf80611190944y66f4cbedyc4bf324d7dbda028@mail.gmail.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu> <1163931206.4510.30.camel@fsol>
	<91ad5bf80611190944y66f4cbedyc4bf324d7dbda028@mail.gmail.com>
Message-ID: <6e9196d20611191056r6fbe71c0n32c0a740fbc806f6@mail.gmail.com>

On 11/19/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> That's perhaps one of the top gotchas in Python today; thankfully it
> will change in 3.0. Until then, I can't see how one can avoid an
> explicit check, either by testing with isinstance() or trying to call
> an arbitrary method of the expected protocol. Having said that both
> are necessary evils, I'd go with the isinstance check as the lesser
> evil. Checking for an arbitrary method is not only an ugly kludge; it
> is both unnecessary (if the tested method is not used in the actual
> code) and error-prone (e.g. for instances of a type that just happens
> to define __add__ so that x + 0 doesn't raise an exception, although x
> is an unappropriate argument in the specific context).

One of Python's biggest surprises is it has distinct concepts of
"sequence" and "mapping", but the API is intertwined so much you can't
even test whether an object is one or the other.  Say you want to
verify an object is a sequence:

    (1) isinstance(obj, list): fails for UserList et al, but many
users have gone this route of requiring a list subclass because it's
the most straighforward.

    (2) hasattr(obj, "__getitem__"): it may have that method, but that
doesn't mean it's list-like.

    (3) Does .__getitem__() accept int arguments?  All lists/dicts do.

    (4) Does .__getitem__() return a value for all ints within range
(0, len(obj))?  No way to test for this without calling with all
possible args and seeing if it raises LookupError.

I can't think of any way around this except a 'sequence' interface
that 'list' would implement.  That in itself wouldn't verify test 4,
but the fact that 'implements(obj, sequence) == True' would mean the
class author has promised it will.

However, requiring objects to be list subclasses hasn't been that much
of a burden in practical terms.

-- 
Mike Orr <sluggoster at gmail.com>

From solipsis at pitrou.net  Sun Nov 19 20:18:14 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 19 Nov 2006 20:18:14 +0100
Subject: [Python-3000] interfaces
In-Reply-To: <91ad5bf80611190944y66f4cbedyc4bf324d7dbda028@mail.gmail.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu> <1163931206.4510.30.camel@fsol>
	<91ad5bf80611190944y66f4cbedyc4bf324d7dbda028@mail.gmail.com>
Message-ID: <1163963894.4510.87.camel@fsol>


Le dimanche 19 novembre 2006 ? 12:44 -0500, George Sakkis a ?crit :
> I understand this is not the point you're trying to make, but in such
> cases I usually prefer to make @decorator be equivalent to
> @decorator() by something like:

I could do that, but it's not very clean. Also it doesn't invalidate my
point, since what callable() does is precisely to check that the
parameter conforms to a given interface (instead of explicitly testing
for the underlying implementation e.g. __call__).

I'm not an interface fan at all. Simply, in some cases, it would be
cleaner to check an object implements, or claims to implement, a
conventional interface (which might not even involve any syntactic
checks about the presence of a method), rather than testing for a
specific implementation trait in the hope that it's both necessary and
sufficient.




From guido at python.org  Sun Nov 19 22:33:46 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 19 Nov 2006 13:33:46 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
Message-ID: <ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>

On 11/19/06, James Thiele <python3now at gmail.com> wrote:
> The BDFL has said that print will change from a statement to a function in
> Python 3.
>
> I haven't found anything describing what parameters it will take and what it
> will return. Has this been decided?

Not really, I'm hoping someone would write a PEP. But (from memory, I
think there are some threads on python-dev or the py3k list about
this) here are some suggestions:

1. With only positional parameters (or none at all), print(x, y, z) is
equivalent to "print x, y, z" today. So it converts each argument
using str(), inserting spaces in between, and appending a newline.

2. A keyword parameter will direct it to a different file.
Probably print(x, y, z, file=<file-like object>).

3. Two more keyword parameters can change what gets inserted between
items and at the end.
Perhaps print(x, y, z, sep=<string>, end=<string>). These default to "
" and "\n", respectively.

4. The softspace feature (a semi-secret attribute on files currently
used to tell print whether to insert a space before the first item)
will be killed.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Sun Nov 19 22:37:06 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 19 Nov 2006 22:37:06 +0100
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
Message-ID: <ejqiqb$d38$1@sea.gmane.org>

Guido van Rossum wrote:
> On 11/19/06, James Thiele <python3now at gmail.com> wrote:
>> The BDFL has said that print will change from a statement to a function in
>> Python 3.
>>
>> I haven't found anything describing what parameters it will take and what it
>> will return. Has this been decided?
> 
> Not really, I'm hoping someone would write a PEP. But (from memory, I
> think there are some threads on python-dev or the py3k list about
> this) here are some suggestions:
> 
> 1. With only positional parameters (or none at all), print(x, y, z) is
> equivalent to "print x, y, z" today. So it converts each argument
> using str(), inserting spaces in between, and appending a newline.
> 
> 2. A keyword parameter will direct it to a different file.
> Probably print(x, y, z, file=<file-like object>).
> 
> 3. Two more keyword parameters can change what gets inserted between
> items and at the end.
> Perhaps print(x, y, z, sep=<string>, end=<string>). These default to "
> " and "\n", respectively.
> 
> 4. The softspace feature (a semi-secret attribute on files currently
> used to tell print whether to insert a space before the first item)
> will be killed.

Do we even need a PEP? This is a good, easy-to-explain interface. Is more
functionality needed?

Georg


From guido at python.org  Sun Nov 19 22:55:04 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 19 Nov 2006 13:55:04 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ejqiqb$d38$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
Message-ID: <ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>

On 11/19/06, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum wrote:
> > On 11/19/06, James Thiele <python3now at gmail.com> wrote:
> >> The BDFL has said that print will change from a statement to a function in
> >> Python 3.
> >>
> >> I haven't found anything describing what parameters it will take and what it
> >> will return. Has this been decided?
> >
> > Not really, I'm hoping someone would write a PEP. But (from memory, I
> > think there are some threads on python-dev or the py3k list about
> > this) here are some suggestions:
> >
> > 1. With only positional parameters (or none at all), print(x, y, z) is
> > equivalent to "print x, y, z" today. So it converts each argument
> > using str(), inserting spaces in between, and appending a newline.
> >
> > 2. A keyword parameter will direct it to a different file.
> > Probably print(x, y, z, file=<file-like object>).
> >
> > 3. Two more keyword parameters can change what gets inserted between
> > items and at the end.
> > Perhaps print(x, y, z, sep=<string>, end=<string>). These default to "
> > " and "\n", respectively.
> >
> > 4. The softspace feature (a semi-secret attribute on files currently
> > used to tell print whether to insert a space before the first item)
> > will be killed.
>
> Do we even need a PEP? This is a good, easy-to-explain interface. Is more
> functionality needed?

PEPs aren't only for difficult discussions. :-) They are also there
for reference and to record agreement. Referring to an email isn't
really a very good answer when someone asks (as happened here) "what
is the spec"?  A PEP may also discourage attempts to add more cruft,
and encourage someone with a few spare cycles to provide a patch so we
can put the whole thing behind it.

BTW I forgot to mention that it shouldn't return anything.

(So are you going to write that PEP? :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Mon Nov 20 00:18:27 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 20 Nov 2006 00:18:27 +0100
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
Message-ID: <ejqoob$sca$1@sea.gmane.org>

Guido van Rossum wrote:

> PEPs aren't only for difficult discussions. :-) They are also there
> for reference and to record agreement. Referring to an email isn't
> really a very good answer when someone asks (as happened here) "what
> is the spec"?  A PEP may also discourage attempts to add more cruft,
> and encourage someone with a few spare cycles to provide a patch so we
> can put the whole thing behind it.
> 
> BTW I forgot to mention that it shouldn't return anything.
> 
> (So are you going to write that PEP? :-)

It's in Subversion and numbered 3105, should be online with the next
website update cycle.

cheers,
Georg


From greg.ewing at canterbury.ac.nz  Mon Nov 20 00:33:06 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 20 Nov 2006 12:33:06 +1300
Subject: [Python-3000] interfaces
In-Reply-To: <1163931206.4510.30.camel@fsol>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu> <1163931206.4510.30.camel@fsol>
Message-ID: <4560E9B2.70801@canterbury.ac.nz>

Antoine Pitrou wrote:

> So to know if "deferred" was correctly used, I had to devise a test to
> know if "timeout" is a number

Another solution might have been to require it to
be passed as a keyword (which will become easier
if the proposal for keyword-only arguments gets
in).

--
Greg

From greg.ewing at canterbury.ac.nz  Mon Nov 20 01:01:10 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 20 Nov 2006 13:01:10 +1300
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <000501c70bf9$791c2b80$6402a8c0@arkdesktop>
References: <000501c70bf9$791c2b80$6402a8c0@arkdesktop>
Message-ID: <4560F046.304@canterbury.ac.nz>

Andrew Koenig wrote:

> That's a nice general sentiment, but not always feasible -- especially if
> you're writing library code.

I've written a fair bit of library code, and I've never
found a pressing need to test for an abstract interface.

In the rare cases where I've used type testing in an
API, it's always been for specific well-known concrete
types, such as strings and tuples -- never anything so
abstract as "iterator", "sequence", etc.

--
Greg

From janssen at parc.com  Mon Nov 20 02:02:40 2006
From: janssen at parc.com (Bill Janssen)
Date: Sun, 19 Nov 2006 17:02:40 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <455FD6C0.7000105@gmail.com> 
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<455FD6C0.7000105@gmail.com>
Message-ID: <06Nov19.170242pst."58648"@synergy1.parc.xerox.com>

> Andrew Koenig wrote:
> > In other words, whether or not we choose to define a family of types that
> > stand for particular abilities, I think we should not use inheritance as the
> > mechanism for specifying that a particular class has a particular ability.
> > I don't know what the mechanism should be, but it shouldn't be inheritance.
> > 
> > Am I missing something?
> 
> It seems to me to be an excellent write-up of why using concrete classes for 
> interface definitions is potentially problematic.

Huh?  Andrew pointed to exactly one "drawback", the inability to
disassociate an inherent characteristic of a type, its "typeness"
(which he refers to as ability-signalling), from a type.  I hardly see
that as problematic.

Am *I* missing something?

Bill

From brett at python.org  Mon Nov 20 02:04:05 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 19 Nov 2006 17:04:05 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ejqoob$sca$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
Message-ID: <bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>

On 11/19/06, Georg Brandl <g.brandl at gmx.net> wrote:
>
> Guido van Rossum wrote:
>
> > PEPs aren't only for difficult discussions. :-) They are also there
> > for reference and to record agreement. Referring to an email isn't
> > really a very good answer when someone asks (as happened here) "what
> > is the spec"?  A PEP may also discourage attempts to add more cruft,
> > and encourage someone with a few spare cycles to provide a patch so we
> > can put the whole thing behind it.
> >
> > BTW I forgot to mention that it shouldn't return anything.
> >
> > (So are you going to write that PEP? :-)
>
> It's in Subversion and numbered 3105, should be online with the next
> website update cycle.


The only thing I would prefer is instead of 'end' have 'newline' and have
that be a boolean since I don't see a need to support different line
endings.  I realize the difference between ``end=''`` and ``newline=False``
is minimal beyond more typing, but for some reason my head just keeps
telling me I prefer the latter for clarity reasons.

Otherwise +1 from me.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061119/2161efc3/attachment.html 

From sluggoster at gmail.com  Mon Nov 20 02:04:41 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Sun, 19 Nov 2006 17:04:41 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <4560F046.304@canterbury.ac.nz>
References: <000501c70bf9$791c2b80$6402a8c0@arkdesktop>
	<4560F046.304@canterbury.ac.nz>
Message-ID: <6e9196d20611191704k4a8a8e3dyf225c13d283a4461@mail.gmail.com>

On 11/19/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Andrew Koenig wrote:
>
> > That's a nice general sentiment, but not always feasible -- especially if
> > you're writing library code.
>
> I've written a fair bit of library code, and I've never
> found a pressing need to test for an abstract interface.
>
> In the rare cases where I've used type testing in an
> API, it's always been for specific well-known concrete
> types, such as strings and tuples -- never anything so
> abstract as "iterator", "sequence", etc.

Sometimes people don't realize how much they'd use a feature until it
exists.  I never thought I'd use generators until I started using
them, and then it's "Wow, this is neat!"  There have been several
cases like this with Python.

Another way people often test for sequence-ness:
    isinstance(obj, (list, tuple))
    isinstance(obj, (list, tuple, UserList))
Here one buldes up all the likely classes.  That's fine but we left
out str because you have to stop somewhere.  Usually I forego UserList
because that depends on a module that may not be used, and often I
forego tuple to make it shorter: "isinstance(obj, list)".  That's fine
but it's more restrictive than I want to me: my function *could* work
with a tuple or list-like object too.

Equality for tuples!  Down with sequence discrimination!  Support interfaces! :)

-- 
Mike Orr <sluggoster at gmail.com>

From janssen at parc.com  Mon Nov 20 02:07:29 2006
From: janssen at parc.com (Bill Janssen)
Date: Sun, 19 Nov 2006 17:07:29 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <20061119115009.V14417@tribble.ilrt.bris.ac.uk> 
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu>
	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>
Message-ID: <06Nov19.170737pst."58648"@synergy1.parc.xerox.com>

> Java interfaces are very useful, however. Java programming seems to be 
> less and less about inheritance and more and more about implementing 
> interfaces; at least it does amongst Java programmers with taste :-)

It seems to me that that's where Python has a real advantage.  With
real support for multiple inheritance, Python "interfaces" could be
real classes (either like real Java classes or Java abstract classes),
perhaps providing default implementations.  You get the goodness of
mix-ins, along with interface communication.

Bill

From guido at python.org  Mon Nov 20 03:03:32 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 19 Nov 2006 18:03:32 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
Message-ID: <ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>

On 11/19/06, Brett Cannon <brett at python.org> wrote:
>
>
> On 11/19/06, Georg Brandl <g.brandl at gmx.net> wrote:
> > Guido van Rossum wrote:
> >
> > > PEPs aren't only for difficult discussions. :-) They are also there
> > > for reference and to record agreement. Referring to an email isn't
> > > really a very good answer when someone asks (as happened here) "what
> > > is the spec"?  A PEP may also discourage attempts to add more cruft,
> > > and encourage someone with a few spare cycles to provide a patch so we
> > > can put the whole thing behind it.
> > >
> > > BTW I forgot to mention that it shouldn't return anything.
> > >
> > > (So are you going to write that PEP? :-)
> >
> > It's in Subversion and numbered 3105, should be online with the next
> > website update cycle.
>
> The only thing I would prefer is instead of 'end' have 'newline' and have
> that be a boolean since I don't see a need to support different line
> endings.  I realize the difference between ``end=''`` and ``newline=False``
> is minimal beyond more typing, but for some reason my head just keeps
> telling me I prefer the latter for clarity reasons.
>
> Otherwise +1 from me.

Oh, but I *do* see a use case for other line endings. E.g. when
writing a Windows file or a standards-compliant email from within
Unix, end="\r\n" would be helpful. In practice it probably would be a
variable set from a config option.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Mon Nov 20 10:23:42 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 20 Nov 2006 19:23:42 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov19.170242pst."58648"@synergy1.parc.xerox.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<455FD6C0.7000105@gmail.com>
	<06Nov19.170242pst."58648"@synergy1.parc.xerox.com>
Message-ID: <4561741E.1040102@gmail.com>

Bill Janssen wrote:
>> Andrew Koenig wrote:
>>> In other words, whether or not we choose to define a family of types that
>>> stand for particular abilities, I think we should not use inheritance as the
>>> mechanism for specifying that a particular class has a particular ability.
>>> I don't know what the mechanism should be, but it shouldn't be inheritance.
>>>
>>> Am I missing something?
>> It seems to me to be an excellent write-up of why using concrete classes for 
>> interface definitions is potentially problematic.
> 
> Huh?  Andrew pointed to exactly one "drawback", the inability to
> disassociate an inherent characteristic of a type, its "typeness"
> (which he refers to as ability-signalling), from a type.  I hardly see
> that as problematic.
> 
> Am *I* missing something?

Class inheritance implies a transitive relationship: if A is a B and B is a C, 
then A is also a C. Not all characteristics of a type are necessarily 
transitive in this fashion, so there are some things which cannot be 
effectively expressed by means of class inheritance.

Consider the classic example of a taxonomy of birds and the expression of the 
property "this bird can fly" in a world containing emus, penguins and ostriches.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From jan.grant at bristol.ac.uk  Mon Nov 20 10:39:37 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Mon, 20 Nov 2006 09:39:37 +0000 (GMT)
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu>
	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>
	<06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
Message-ID: <20061120093735.Q42166@tribble.ilrt.bris.ac.uk>

On Sun, 19 Nov 2006, Bill Janssen wrote:

> > Java interfaces are very useful, however. Java programming seems to be 
> > less and less about inheritance and more and more about implementing 
> > interfaces; at least it does amongst Java programmers with taste :-)
> 
> It seems to me that that's where Python has a real advantage.  With
> real support for multiple inheritance, Python "interfaces" could be
> real classes (either like real Java classes or Java abstract classes),
> perhaps providing default implementations.  You get the goodness of
> mix-ins, along with interface communication.

True, but the enforced separation of implementation and interface is 
pretty useful. Having said that, nothing to stop people declaring a 
completely abstract interface class, subclassing that to provide a 
default implementation. Again, comes down to a question of taste on a 
case-by-case basis.

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Lambda calculus? I hardly know 'er!

From bingham at cenix-bioscience.com  Mon Nov 20 10:46:16 2006
From: bingham at cenix-bioscience.com (Aaron Bingham)
Date: Mon, 20 Nov 2006 10:46:16 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <ca471dc20611182013y496d7f2dt2bd96d98c45c765f@mail.gmail.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>	<455FD6C0.7000105@gmail.com>
	<ca471dc20611182013y496d7f2dt2bd96d98c45c765f@mail.gmail.com>
Message-ID: <45617968.5050303@cenix-bioscience.com>

Guido van Rossum wrote:

>On 11/18/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>  
>
>>(Although I will point out
>>that most  protocols for things like len() *do* involve checks for special
>>methods by name, and the check for iterability is typically a non-destructive
>>call to iter(x), rather than a destructive one to x.next()).
>>    
>>
>
>Ouch?! I would never check for iterability explicitly. I would just
>require it, as Greg Ewing says. iter() could be fairly expensive,
>depending on what is being iterated over.
>
>Maybe this is unique to iterators and iterators are a bad example? I
>don't feel as strongly about this when testing e.g. for string-ness,
>file-ness or list-ness. While I tend to dislike those tests too, they
>are sometimes inevitable when we want to overload an API. Especially
>file/string is often overloaded, and I see no big problem with this (I
>do dislike overloading list/dict, or string/list-of-strings).
>
Might I point out that with Design by Contract, it is impossible to 
write a correct and complete precondition if nondestructive tests are 
not available.

Regards,

-- 
--------------------------------------------------------------------
Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH
--------------------------------------------------------------------


From bingham at cenix-bioscience.com  Mon Nov 20 10:47:59 2006
From: bingham at cenix-bioscience.com (Aaron Bingham)
Date: Mon, 20 Nov 2006 10:47:59 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <4561741E.1040102@gmail.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>	<455FD6C0.7000105@gmail.com>	<06Nov19.170242pst."58648"@synergy1.parc.xerox.com>
	<4561741E.1040102@gmail.com>
Message-ID: <456179CF.1060201@cenix-bioscience.com>

Nick Coghlan wrote:

>Bill Janssen wrote:
>  
>
>>>Andrew Koenig wrote:
>>>      
>>>
>>>>In other words, whether or not we choose to define a family of types that
>>>>stand for particular abilities, I think we should not use inheritance as the
>>>>mechanism for specifying that a particular class has a particular ability.
>>>>I don't know what the mechanism should be, but it shouldn't be inheritance.
>>>>
>>>>Am I missing something?
>>>>        
>>>>
>>>It seems to me to be an excellent write-up of why using concrete classes for 
>>>interface definitions is potentially problematic.
>>>      
>>>
>>Huh?  Andrew pointed to exactly one "drawback", the inability to
>>disassociate an inherent characteristic of a type, its "typeness"
>>(which he refers to as ability-signalling), from a type.  I hardly see
>>that as problematic.
>>
>>Am *I* missing something?
>>    
>>
>
>Class inheritance implies a transitive relationship: if A is a B and B is a C, 
>then A is also a C. Not all characteristics of a type are necessarily 
>transitive in this fashion, so there are some things which cannot be 
>effectively expressed by means of class inheritance.
>
>Consider the classic example of a taxonomy of birds and the expression of the 
>property "this bird can fly" in a world containing emus, penguins and ostriches.
>
You seem to be imagining a class hierarchy like this:

class Bird1(object):
    def fly(self):
        # default implementation
        ....

class Finch1(Bird1):
        pass

class Ostrich1(Bird1):
   def fly(self):
       # oops, this doesn't make sense, and there is nothing we can do
       # except raise an error which will surprise clients expecting a 
generic Bird1
       raise RuntimeError("Ostriches can't fly")

However, of you know ahead of time that not all birds can fly you can 
design for this.

class FlightlessBirdError(Exception):
    """Raised when a flightless bird attempts to fly"""
    pass

class Bird2(object):
    def flightless(self):
        return False
    def fly(self):
        """Only valid if not flightless, otherwise raises 
FlightlessBirdError"""
        # default implementation
        if self.flightless():
            raise FlightlessBirdError()
        ...

class Finch2(Bird2):
    pass

class Ostrich2(Bird2):
    def flightless(self):
        return True

Now the possibility of flightlessness has been made explicit in the 
design and a client will not be surprised by failures caused by attempts 
to call fly() on a flightless bird.

Regards,

-- 
--------------------------------------------------------------------
Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH
--------------------------------------------------------------------


From bingham at cenix-bioscience.com  Mon Nov 20 11:37:46 2006
From: bingham at cenix-bioscience.com (Aaron Bingham)
Date: Mon, 20 Nov 2006 11:37:46 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>	<455FEEBB.9010100@cs.byu.edu>	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>
	<06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
Message-ID: <4561857A.8080101@cenix-bioscience.com>

Bill Janssen wrote:

>>Java interfaces are very useful, however. Java programming seems to be 
>>less and less about inheritance and more and more about implementing 
>>interfaces; at least it does amongst Java programmers with taste :-)
>>    
>>
>
>It seems to me that that's where Python has a real advantage.  With
>real support for multiple inheritance, Python "interfaces" could be
>real classes (either like real Java classes or Java abstract classes),
>perhaps providing default implementations.  You get the goodness of
>mix-ins, along with interface communication.
>
I agree.  In Java, interfaces are necessary because multiple inheritance 
is not supported.  I see no good reason to add an additional language 
mechanism for interfaces when multiple inheritance would do the job, AFAICT.

Regards,

-- 
--------------------------------------------------------------------
Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH
--------------------------------------------------------------------


From and-dev at doxdesk.com  Mon Nov 20 11:59:20 2006
From: and-dev at doxdesk.com (Andrew Clover)
Date: Mon, 20 Nov 2006 10:59:20 +0000
Subject: [Python-3000] Extending warnings for Py3K - any interest?
Message-ID: <45618A88.1040905@doxdesk.com>

I've always felt slightly disappointed by the Python warnings interface. 
It's great for managing language feature deprecation, but I've often 
wanted something I could also use for recoverable errors in general.

I've written multiple application-specific mechanisms for allowing a 
caller to customise handling of potentially recoverable conditions in 
the past, so I think it's probably a fairly common use case, and maybe, 
if others how found the same, worth standardising as an extension of the 
existing Python warnings system.

Here's a first sketch of the sort of thing I'm thinking of:

   http://doxdesk.com/file/software/py/v/warnings2-0.1.py

Features:

   - warnings as instances with arbitrary properties, rather than being
     limited to a string message;

   - different severities with different default actions, so trivial
     warnings can be ignored by default and things which are by default
     errors can be recovered if a filter says so;

   - option to store warning instances that occur for later handling;

   - switchable warnings-handling contexts (like eg. the decimal module)
     for temporary changes to filters;

   - context manager helpers so you can say things like:

     with ignoring(DeprecationWarning):
         import oldmodule

     with storing(InvalidDataWarning) as problems:
         reghive.scanKeys()
     for problem in problems: ...

Anyone agree/disagree with the idea of adding such functionality to 
warnings, or what features are worthwhile?

-- 
And Clover
mailto:and at doxdesk.com
http://www.doxdesk.com/

From murman at gmail.com  Mon Nov 20 15:25:27 2006
From: murman at gmail.com (Michael Urman)
Date: Mon, 20 Nov 2006 08:25:27 -0600
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <456179CF.1060201@cenix-bioscience.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<455FD6C0.7000105@gmail.com> <4561741E.1040102@gmail.com>
	<456179CF.1060201@cenix-bioscience.com>
Message-ID: <dcbbbb410611200625l3629ca08jdaba454b94f906a5@mail.gmail.com>

> However, of you know ahead of time that not all birds can fly you can
> design for this.

To use a more relevant example, how about file-like-objects and
nameless files. Any class derived from file can be expected to have
the name member. However several files have nonsense names:

>>> f = tempfile.TemporaryFile()
>>> isinstance(f, file)
True
>>> f.name, sys.stdin.name, sys.stdout.name
('<fdopen>', '<stdin>', '<stdout>')

Furthermore arbitrary file-like objects may or may not see it as
necessary to provide a name attribute:

>>> f = urllib.urlopen('http://www.google.com')
>>> isinstance(f, file)
False
>>> f.read(62)
'<html><head><meta http-equiv="content-type" content="text/html'
>>> f.name
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: addinfourl instance has no attribute 'name'

I'm skeptical that class hierarchy design time is a good time to
target. Class hierarchies should be relatively rare in python,
maintenance is more common, and developers new to your class hierarchy
not knowing the special cases are likely to make incorrect
assumptions. Both of the following are wrong, but are easy assumptions
to have made:

  1) All file-like objects have a usable name
  2) All file-like objects have a name attribute

To tie this back to the duck vs. lawyer context, if a function created
by someone else is going to read data from a file object, which would
make it easiest for you to use in legitimate ways the other programmer
hadn't planned?

  1) It checks for isinstance(obj, file),
  2) It opens a copy with file(obj.name, 'r'),
  3) It allows filenames by opening file(obj, 'r'),
  4) It allows filedata by creating a StringIO(obj),
  5) It requires the (fuzzy) right thing and calls obj.read(), or
  6) It does 5 with a fallback of 4 or 3 if it can't find obj.read.

I've set this scenario to make 6 the obvious answer, but it should be
clear that there isn't a single winner between 3 and 4 across all
scenarios.
-- 
Michael Urman  http://www.tortall.net/mu/blog

From rasky at develer.com  Mon Nov 20 15:58:44 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Mon, 20 Nov 2006 15:58:44 +0100
Subject: [Python-3000] print() parameters in py3k
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com><ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com><ejqiqb$d38$1@sea.gmane.org><ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com><ejqoob$sca$1@sea.gmane.org><bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
Message-ID: <041601c70cb4$5ff11470$cf09f01b@bagio>

Guido van Rossum wrote:

>> The only thing I would prefer is instead of 'end' have 'newline' and
>> have that be a boolean since I don't see a need to support different
>> line endings.  I realize the difference between ``end=''`` and
>> ``newline=False`` is minimal beyond more typing, but for some reason
>> my head just keeps telling me I prefer the latter for clarity
>> reasons. 
>> 
>> Otherwise +1 from me.
> 
> Oh, but I *do* see a use case for other line endings. E.g. when
> writing a Windows file or a standards-compliant email from within
> Unix, end="\r\n" would be helpful. In practice it probably would be a
> variable set from a config option.

Uhm, but then, why not simply:

println(x,y,z) -> append "\n"
print(x,y,z) -> no terminator
print(x,y,z,"\r\n") -> custom terminator

Giovanni Bajo


From walter at livinglogic.de  Mon Nov 20 16:46:20 2006
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon, 20 Nov 2006 16:46:20 +0100
Subject: [Python-3000] Extending warnings for Py3K - any interest?
In-Reply-To: <45618A88.1040905@doxdesk.com>
References: <45618A88.1040905@doxdesk.com>
Message-ID: <4561CDCC.8050903@livinglogic.de>

Andrew Clover wrote:

> I've always felt slightly disappointed by the Python warnings interface. 
> It's great for managing language feature deprecation, but I've often 
> wanted something I could also use for recoverable errors in general.
> 
> I've written multiple application-specific mechanisms for allowing a 
> caller to customise handling of potentially recoverable conditions in 
> the past, so I think it's probably a fairly common use case, and maybe, 
> if others how found the same, worth standardising as an extension of the 
> existing Python warnings system.
> 
> Here's a first sketch of the sort of thing I'm thinking of:
> 
>    http://doxdesk.com/file/software/py/v/warnings2-0.1.py
> 
> Features:
> 
>    - warnings as instances with arbitrary properties, rather than being
>      limited to a string message;

That's already possible:

class MyWarning(Warning):
   ...

warnings.warn(MyWarning(...))

>    - different severities with different default actions, so trivial
>      warnings can be ignored by default and things which are by default
>      errors can be recovered if a filter says so;
>
>    - option to store warning instances that occur for later handling;
> 
>    - switchable warnings-handling contexts (like eg. the decimal module)
>      for temporary changes to filters;
> 
>    - context manager helpers so you can say things like:
> 
>      with ignoring(DeprecationWarning):
>          import oldmodule
> 
>      with storing(InvalidDataWarning) as problems:
>          reghive.scanKeys()
>      for problem in problems: ...

+1

> Anyone agree/disagree with the idea of adding such functionality to 
> warnings, or what features are worthwhile?

Another idea: Actions might be callables, so that they can be used
directly in handleWarning() instead of dispatching on the action name.

BTW, your source code is not PEP 8 compatible.

Servus,
   Walter

From barry at python.org  Mon Nov 20 16:54:51 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 20 Nov 2006 10:54:51 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <041601c70cb4$5ff11470$cf09f01b@bagio>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com><ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com><ejqiqb$d38$1@sea.gmane.org><ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com><ejqoob$sca$1@sea.gmane.org><bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
Message-ID: <FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 20, 2006, at 9:58 AM, Giovanni Bajo wrote:

> Guido van Rossum wrote:
>
>>> The only thing I would prefer is instead of 'end' have 'newline' and
>>> have that be a boolean since I don't see a need to support different
>>> line endings.  I realize the difference between ``end=''`` and
>>> ``newline=False`` is minimal beyond more typing, but for some reason
>>> my head just keeps telling me I prefer the latter for clarity
>>> reasons.
>>>
>>> Otherwise +1 from me.
>>
>> Oh, but I *do* see a use case for other line endings. E.g. when
>> writing a Windows file or a standards-compliant email from within
>> Unix, end="\r\n" would be helpful. In practice it probably would be a
>> variable set from a config option.
>
> Uhm, but then, why not simply:
>
> println(x,y,z) -> append "\n"
> print(x,y,z) -> no terminator
> print(x,y,z,"\r\n") -> custom terminator

I'd like to at least see a discussion of a more printf() or logging  
style for print function.  Specifically, I wonder if it wouldn't be  
useful to require a format string with positional and keyword  
arguments being used for automatic interpolation.  E.g.

print('%s: %s', field, value)

instead of

print('%s: %s' % (field, value))

There are lots of questions to answer, such as whether to use $- 
strings and require keyword arguments.

Another thought: what if 'print' weren't a function but a callable  
object. exposed in builtins.  I'm thinking something roughly parallel  
to stdout, stderr, stdin, or maybe better cout, cerr, cin.

The default 'print' object then would be a "print-to-sys.stdout-with- 
\n", but you would then be able to do something like:

stderr = print.clone(stream=sys.stderr)
stderr('here go all my error messages')

or

smtpout = print.clone(stream=mysock, end='\r\n')
smtpout('$code $msg', code=250, msg='Okay')

Perhaps you could even hook in i18n by doing something like:

print = print.clone(translate=gettext.gettext)
print('$who likes to eat $what', person, food)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRWHP0HEjvBPtnXfVAQJ8HgQAoKfkh94Js4MXMRgjAm3uFHvNxYBrb99k
Je8PkICx1QXhkSH9Kq3KJ3AbWXx/EKQFycfXun9ZxXy/hax7RcsDHJRsR9H1moR7
6IiWBEJJ1IqAjCfTNCKhP4VMxQtq6DFb5rRoN9jfe5pAJeA7SqCM0Fqcdf8F0stG
OWQtK+UIjLU=
=MhoH
-----END PGP SIGNATURE-----

From fredrik at pythonware.com  Mon Nov 20 17:56:03 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 20 Nov 2006 17:56:03 +0100
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <041601c70cb4$5ff11470$cf09f01b@bagio>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com><ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com><ejqiqb$d38$1@sea.gmane.org><ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com><ejqoob$sca$1@sea.gmane.org><bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
Message-ID: <ejsmn3$bvu$1@sea.gmane.org>

Giovanni Bajo wrote:

> Uhm, but then, why not simply:
> 
> println(x,y,z) -> append "\n"
> print(x,y,z) -> no terminator
> print(x,y,z,"\r\n") -> custom terminator

hey, I proposed that earlier today, but something ate my post ;-)

while we're at it, why not add a printf as well, btw?  (defined as
def printf(fmt, *args): print(fmt % args)).

</F>


From gsakkis at rutgers.edu  Mon Nov 20 18:06:58 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Mon, 20 Nov 2006 12:06:58 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <dcbbbb410611200625l3629ca08jdaba454b94f906a5@mail.gmail.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<455FD6C0.7000105@gmail.com> <4561741E.1040102@gmail.com>
	<456179CF.1060201@cenix-bioscience.com>
	<dcbbbb410611200625l3629ca08jdaba454b94f906a5@mail.gmail.com>
Message-ID: <91ad5bf80611200906y394e6257jd8bd5354d5b98a66@mail.gmail.com>

On 11/20/06, Michael Urman <murman at gmail.com> wrote:
> > However, of you know ahead of time that not all birds can fly you can
> > design for this.
>
> To use a more relevant example, how about file-like-objects and
> nameless files. Any class derived from file can be expected to have
> the name member. However several files have nonsense names:
>
> >>> f = tempfile.TemporaryFile()
> >>> isinstance(f, file)
> True
> >>> f.name, sys.stdin.name, sys.stdout.name
> ('<fdopen>', '<stdin>', '<stdout>')
>
> Furthermore arbitrary file-like objects may or may not see it as
> necessary to provide a name attribute:
>
> >>> f = urllib.urlopen('http://www.google.com')
> >>> isinstance(f, file)
> False
> >>> f.read(62)
> '<html><head><meta http-equiv="content-type" content="text/html'
> >>> f.name
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> AttributeError: addinfourl instance has no attribute 'name'
>
> I'm skeptical that class hierarchy design time is a good time to
> target. Class hierarchies should be relatively rare in python,
> maintenance is more common, and developers new to your class hierarchy
> not knowing the special cases are likely to make incorrect
> assumptions. Both of the following are wrong, but are easy assumptions
> to have made:
>
>   1) All file-like objects have a usable name
>   2) All file-like objects have a name attribute

One issue here is the (informal) name of the protocol "file-like",
which although it adds the suffix "like" and gives a clue it's not
necessarily about real files, it makes such erroneous assumptions more
likely. OTOH if we were talking about, say, "character streams", the
expectation of having a name would be less justified.

By the way, file-like is a good example of a protocol that would
benefit from being less fuzzy and informal. For one, it would give a
clear definition of the protocol attributes; you can't know this just
by looking at the concrete file type and assume that all file-like
objects should support the same attributes ('name' is one example that
makes sense to real files mainly). For two, it would give default
implementations for methods that can be defined through others, e.g.
readline() and readlines() can be implemented using read() (of course
a subclass may override these to provide more efficient
implementations, just as with dictmixin). Anyone else that would find
a 'stream' (or 'charstream' or 'StreamMixin' or whatever) class a good
idea?

George

From jcarlson at uci.edu  Mon Nov 20 18:14:55 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon, 20 Nov 2006 09:14:55 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
References: <041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
Message-ID: <20061120082547.8346.JCARLSON@uci.edu>


Barry Warsaw <barry at python.org> wrote:
> On Nov 20, 2006, at 9:58 AM, Giovanni Bajo wrote:
> > Uhm, but then, why not simply:
> >
> > println(x,y,z) -> append "\n"
> > print(x,y,z) -> no terminator
> > print(x,y,z,"\r\n") -> custom terminator

I don't like the idea of 2 builtins to print (-1).  It doesn't feel
right. Then again, I'm not particularly interested in a print function
either (-0).


> I'd like to at least see a discussion of a more printf() or logging  
> style for print function.  Specifically, I wonder if it wouldn't be  
> useful to require a format string with positional and keyword  
> arguments being used for automatic interpolation.  E.g.
> 
> print('%s: %s', field, value)
> 
> instead of
> 
> print('%s: %s' % (field, value))

How much print statement rewriting do you want to do?  Me, I'd like to
get away with as little as possible.  Adding parenthesis isn't that bad. 
Changing everything to use a printf-style would be 1) annoying, 2)
unnecessary, and 3) I hate printf.

Not to say that #3 is a sufficient reason for everyone else, but when I
sit down to write C, among the standard annoyances I wish I didn't have
to deal with is that printf _requires_ that damn format string.  I
understand it is a requirement due to underlying language limitations,
but Python has no such limitations, so let us not burdon Python with the
braindead nature of C's printf.

I would also mention that there would be no way to distinguish between
the two cases you specify above (within print()), unless we are going to
remove % formatting, which would be a mistake.


> There are lots of questions to answer, such as whether to use $- 
> strings and require keyword arguments.

Bad idea.  Unless we are changing all of string formatting to use $,
then changing the default string formatting of one of the likely most
used functions, is at best confusing, and certainly gratuitous breakage.
Then again, I never saw that using required parens and a specifier like
%(...)s was a burden...


> Another thought: what if 'print' weren't a function but a callable  
> object. exposed in builtins.  I'm thinking something roughly parallel  
> to stdout, stderr, stdin, or maybe better cout, cerr, cin.

We still wouldn't be able to get "cin << bar" without frame hacking, and
I would be content to never see "cout >> foo" or "cin << bar" ever again.
The 2.x "convenience syntax" of print >>foo, ... I think was a decent
feature, brought on by the desire to make print usable for non-stdout
streams.  cout/cerr/cin, almost by definition, are C++ hacks and/or
misfeatures, and adding them to Python seems to be silly to me;
regardless of the 5 (or fewer) C++ users who don't know how to use C's
printf, or who wouldn't be able to understand Python's print().

We've got raw_input() and print().  If users want alternate
functionality, they can write their own print function and even assign
it to __builtins__.print .


> The default 'print' object then would be a "print-to-sys.stdout-with- 
> \n", but you would then be able to do something like:
> 
> stderr = print.clone(stream=sys.stderr)
> stderr('here go all my error messages')
> 
> or
> 
> smtpout = print.clone(stream=mysock, end='\r\n')
> smtpout('$code $msg', code=250, msg='Okay')

I guess I don't really see either of these as significant improvements,
if any, to the guido-defined print function.


> Perhaps you could even hook in i18n by doing something like:
> 
> print = print.clone(translate=gettext.gettext)
> print('$who likes to eat $what', person, food)

Except that would need to be...

print('$who likes to eat $what', who=person, what=food)

This is the only feature that I raise my eyebrow and say, 'that would be
somewhat convenient', but with print as a function, users can _easily_
do that by themselves...

_print = print
def print_(line, **kwargs):
    _print(gettext.gettext(line), **kwargs)
__builtins__.print = print_

Heck, toss a couple fairly simple examples into the print()
documentation, the cookbook, and/or some module, and I think user desire
would be satisfied.

 - Josiah


From guido at python.org  Mon Nov 20 19:11:11 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 10:11:11 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ejsmn3$bvu$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio> <ejsmn3$bvu$1@sea.gmane.org>
Message-ID: <ca471dc20611201011i60155938r2986db4a9d843338@mail.gmail.com>

On 11/20/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Giovanni Bajo wrote:
>
> > Uhm, but then, why not simply:
> >
> > println(x,y,z) -> append "\n"
> > print(x,y,z) -> no terminator
> > print(x,y,z,"\r\n") -> custom terminator
>
> hey, I proposed that earlier today, but something ate my post ;-)

I don't like that as much; it doesn't have a way to change the
separator between items, so if you want your items flush together
you'll have to say

  print(x);  print(y);  println(z)

The third example above has a minor flaw in that it inserts a space
before the "\r\n".

FWIW, in case anyone proposes to "solve" this by not automatically
inserting spaces between items by default, that was quickly voted out
of the race the first time we went through this.

> while we're at it, why not add a printf as well, btw?  (defined as
> def printf(fmt, *args): print(fmt % args)).

Fine with me, although print-hating folks will probably hate it more.
BTW printf() should not automatically append a newline (can't tell
whether you intended it to do or not -- that depends on whose
definition of print() you were referring, mind or Giovanni's). Also,
it should have a file=... keyword parameter.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Nov 20 19:21:34 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 10:21:34 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
Message-ID: <ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>

On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> I'd like to at least see a discussion of a more printf() or logging
> style for print function.

Agreed, as long as it isn't presented as an alternative to the more
basic print() function. We don't want to force students making their
first forays into programming to have to learn format strings. Having
printf() for more advanced use together with print() as defined in PEP
3105 makes sense to me.

> Specifically, I wonder if it wouldn't be
> useful to require a format string with positional and keyword
> arguments being used for automatic interpolation.  E.g.
>
> print('%s: %s', field, value)
>
> instead of
>
> print('%s: %s' % (field, value))

Before going too far down this route, first read PEP 3101, which
proposes a different way out of the issues around the % operator. (Not
saying we couldn't adopt PEP 3101 *and* a printf() function.

> There are lots of questions to answer, such as whether to use $-
> strings and require keyword arguments.

IMO it should follow PEP 3101 exactly; I think that answers any
questions you might have in this realm.

> Another thought: what if 'print' weren't a function but a callable
> object. exposed in builtins.  I'm thinking something roughly parallel
> to stdout, stderr, stdin, or maybe better cout, cerr, cin.

I'm not sure I follow the difference. It was quite clear from earlier
discussions that we *don't* want print to be a bound method of a
hypothetical print method on sys.stdout; that's much less flexible,
and makes it much more work to implement a file-like type. print()
should dynamically look up sys.stdout each time it is called (and
exactly once per call).

(BTW I'm not sure why you like cin/cout/cerr better than stdin/stdout/stderr?)

> The default 'print' object then would be a "print-to-sys.stdout-with-
> \n", but you would then be able to do something like:
>
> stderr = print.clone(stream=sys.stderr)
> stderr('here go all my error messages')
>
> or
>
> smtpout = print.clone(stream=mysock, end='\r\n')
> smtpout('$code $msg', code=250, msg='Okay')

I'd like to firmly resist adding more mechanism for somethign that can
be addressed so simply by a tiny wrapper function:

def smtpout(*args):
  print(*args, file=mysock, end="\r\n")

> Perhaps you could even hook in i18n by doing something like:
>
> print = print.clone(translate=gettext.gettext)
> print('$who likes to eat $what', person, food)

Ditto**2.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Nov 20 19:27:05 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 10:27:05 -0800
Subject: [Python-3000] Abilities / Interfaces
Message-ID: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>

I'd like to have a discussion of abilities and interfaces but right
now I don't have time -- maybe tomorrow. I see Andrew's proposal as
mostly isomorphic to Zope's notion of interfaces, but the new
terminology may make it possible to clarify certain distinctions
better, e.g. the distinction between a class that provides an ability
to its instances, and the abilities that an instance has.

Feel free to riff on this theme here. I'll check in within 24 hours or so.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Mon Nov 20 19:42:58 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 20 Nov 2006 10:42:58 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
Message-ID: <bbaeab100611201042t7e405139u6009df3b45a5b2d8@mail.gmail.com>

On 11/19/06, Guido van Rossum <guido at python.org> wrote:
>
> On 11/19/06, Brett Cannon <brett at python.org> wrote:
> >
> >
> > On 11/19/06, Georg Brandl <g.brandl at gmx.net> wrote:
> > > Guido van Rossum wrote:
> > >
> > > > PEPs aren't only for difficult discussions. :-) They are also there
> > > > for reference and to record agreement. Referring to an email isn't
> > > > really a very good answer when someone asks (as happened here) "what
> > > > is the spec"?  A PEP may also discourage attempts to add more cruft,
> > > > and encourage someone with a few spare cycles to provide a patch so
> we
> > > > can put the whole thing behind it.
> > > >
> > > > BTW I forgot to mention that it shouldn't return anything.
> > > >
> > > > (So are you going to write that PEP? :-)
> > >
> > > It's in Subversion and numbered 3105, should be online with the next
> > > website update cycle.
> >
> > The only thing I would prefer is instead of 'end' have 'newline' and
> have
> > that be a boolean since I don't see a need to support different line
> > endings.  I realize the difference between ``end=''`` and
> ``newline=False``
> > is minimal beyond more typing, but for some reason my head just keeps
> > telling me I prefer the latter for clarity reasons.
> >
> > Otherwise +1 from me.
>
> Oh, but I *do* see a use case for other line endings. E.g. when
> writing a Windows file or a standards-compliant email from within
> Unix, end="\r\n" would be helpful. In practice it probably would be a
> variable set from a config option.


Right, I assumed it would always get set by some config option that was set
during compile.  But whatever, as I said it is not a big deal to me and I
can definitely be happy with the existing proposal.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061120/f068d41c/attachment.htm 

From barry at python.org  Mon Nov 20 20:03:47 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 20 Nov 2006 14:03:47 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
Message-ID: <64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 20, 2006, at 1:21 PM, Guido van Rossum wrote:

> On 11/20/06, Barry Warsaw <barry at python.org> wrote:
>> I'd like to at least see a discussion of a more printf() or logging
>> style for print function.
>
> Agreed, as long as it isn't presented as an alternative to the more
> basic print() function. We don't want to force students making their
> first forays into programming to have to learn format strings. Having
> printf() for more advanced use together with print() as defined in PEP
> 3105 makes sense to me.

That seems reasonable to me too (separate print() and printf()  
thingies).

>> Specifically, I wonder if it wouldn't be
>> useful to require a format string with positional and keyword
>> arguments being used for automatic interpolation.  E.g.
>>
>> print('%s: %s', field, value)
>>
>> instead of
>>
>> print('%s: %s' % (field, value))
>
> Before going too far down this route, first read PEP 3101, which
> proposes a different way out of the issues around the % operator. (Not
> saying we couldn't adopt PEP 3101 *and* a printf() function.

Thanks for the pointer.  I agree that if we adopt a companion printf 
() function it should definitely require (only) PEP 3101 format strings.

>> Another thought: what if 'print' weren't a function but a callable
>> object. exposed in builtins.  I'm thinking something roughly parallel
>> to stdout, stderr, stdin, or maybe better cout, cerr, cin.
>
> I'm not sure I follow the difference. It was quite clear from earlier
> discussions that we *don't* want print to be a bound method of a
> hypothetical print method on sys.stdout; that's much less flexible,
> and makes it much more work to implement a file-like type. print()
> should dynamically look up sys.stdout each time it is called (and
> exactly once per call).
>
> (BTW I'm not sure why you like cin/cout/cerr better than stdin/ 
> stdout/stderr?)

I was trying to make an analogy (probably poorly so) that 'print'  
would be bound to an instance as opposed to a function (current  
proposal) or structure-like thingie without methods as in std*.

>> The default 'print' object then would be a "print-to-sys.stdout-with-
>> \n", but you would then be able to do something like:
>>
>> stderr = print.clone(stream=sys.stderr)
>> stderr('here go all my error messages')
>>
>> or
>>
>> smtpout = print.clone(stream=mysock, end='\r\n')
>> smtpout('$code $msg', code=250, msg='Okay')
>
> I'd like to firmly resist adding more mechanism for somethign that can
> be addressed so simply by a tiny wrapper function:
>
> def smtpout(*args):
>  print(*args, file=mysock, end="\r\n")

What I was trying to address was the namespace collisions you'd get  
between printf-style keyword arguments and 'special' arguments that  
the print function would accept.  The thing is that if we were to add  
both a print and a printf, you'd like for their interfaces to be as  
similar as possible except that the latter takes a format string as  
its first argument.  Clearly 'special' arguments like 'file' and  
'end' will prohibit their use as keyword arguments for printf, and I  
don't like that (this isn't anything new -- the problem crops up  
occasionally in other places, see the email package API).

One way out would be to use special argument names, e.g. prepending  
and or appending some number of underscores.  This doesn't eliminate  
the problem, but it reduces its likelihood.  Another approach would  
for 'print' to be an object with attributes that would change  
behavior, such as the stream to write to or line-ending characters.   
A possible third approach might be some special singleton objects  
that, when seen positionally would modify the state of the underlying  
print object.  Of the three (maybe fourth: "sorry you can't use 'end'  
or 'file'"), it seems preferable to use a callable print object.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRWH8GXEjvBPtnXfVAQL0BQP/SoF2ETUnUlept5pPwzzeAQdOJjcKuEjL
3saKhz0QAVfCktxTTFK4hZQGeYHKLTSWemr3A+zl82lc58rmSGr8OemXDLXy1J7l
PplFkp4iBrT8mLKSFL2c397IKg0M/hbfb3Iw2NqNBu8h2pA1d1bAozBR+nNClUGP
z6vCi1zfYvk=
=7G56
-----END PGP SIGNATURE-----

From jimjjewett at gmail.com  Mon Nov 20 20:05:21 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 20 Nov 2006 14:05:21 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
Message-ID: <fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>

On 11/19/06, Guido van Rossum <guido at python.org> wrote:
> On 11/19/06, Georg Brandl <g.brandl at gmx.net> wrote:
> > Guido van Rossum wrote:

> > > 2. A keyword parameter will direct it to a different file.
> > > Probably print(x, y, z, file=<file-like object>).

> > > 3. Two more keyword parameters can change what gets inserted between
> > > items and at the end.
> > > Perhaps print(x, y, z, sep=<string>, end=<string>). These default to "
> > > " and "\n", respectively.

> A PEP may also discourage attempts to add more cruft,

(1)  Is this an explicit rejection of a keyword for a format string, such as

    print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")

(2)  What happens to additional keyword arguments that do get passed?
    (a)  A StandardError of some sort is raised.
    (b)  print is treated as though it had (but ignored) a **kwargs
argument, to support extensions in print-replacements.
    (c)  The extra arguments are printed at the end, perhaps with a label.

-jJ

From barry at python.org  Mon Nov 20 20:11:34 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 20 Nov 2006 14:11:34 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <20061120082547.8346.JCARLSON@uci.edu>
References: <041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<20061120082547.8346.JCARLSON@uci.edu>
Message-ID: <46B9A351-2664-4B58-86C6-BB2D9A8DDF7F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 20, 2006, at 12:14 PM, Josiah Carlson wrote:

> How much print statement rewriting do you want to do?  Me, I'd like to
> get away with as little as possible.  Adding parenthesis isn't that  
> bad.
> Changing everything to use a printf-style would be 1) annoying, 2)
> unnecessary, and 3) I hate printf.

Are we going to include a flawless transformation script for Python  
3k?  If not, then I'm pretty much resigned to rewriting them by hand  
anyway.

But I think this would be solved by adopting separate print() and  
printf() thingies.

>> There are lots of questions to answer, such as whether to use $-
>> strings and require keyword arguments.
>
> Bad idea.  Unless we are changing all of string formatting to use $,
> then changing the default string formatting of one of the likely most
> used functions, is at best confusing, and certainly gratuitous  
> breakage.
> Then again, I never saw that using required parens and a specifier  
> like
> %(...)s was a burden...

I withdraw the suggestion to use $-strings in the format string.  If  
we get printf, it should only use PEP 3101 format strings.  I still  
believe that $-strings will be useful because they target a different  
audience.  The motivating use case for PEP 292 was that translators  
are not (always) programmers.  Programmers can handle %()s  
substitutions or PEP 3101 format strings, but translators have lots  
of problems with them.  It makes sense to keep the uber-powerful PEP  
3101 format strings for programmers and simple PEP 292 strings for  
non-programmers.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRWH953EjvBPtnXfVAQIA2QQAtSF0djvOIJQCUegtifmIn3w+SG6epykC
BDL6mI0T7TbSKyQLQxgKBkqTzGnJCcCa4eWRi0ATOZyPMTU8YmZBB70SLzLOMaqT
LAYl77i+PCRxO6DCS2PQuCltv1LuMi6x36KtJ4BM0sMXDK51f/uykz6z8F+qfQpO
Okt98c0EiqY=
=eYRm
-----END PGP SIGNATURE-----

From brett at python.org  Mon Nov 20 20:30:31 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 20 Nov 2006 11:30:31 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
Message-ID: <bbaeab100611201130r3888c7ccy3f9b7fee52ed0fca@mail.gmail.com>

On 11/20/06, Barry Warsaw <barry at python.org> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 20, 2006, at 1:21 PM, Guido van Rossum wrote:
>
> > On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> >> I'd like to at least see a discussion of a more printf() or logging
> >> style for print function.
> >
> > Agreed, as long as it isn't presented as an alternative to the more
> > basic print() function. We don't want to force students making their
> > first forays into programming to have to learn format strings. Having
> > printf() for more advanced use together with print() as defined in PEP
> > 3105 makes sense to me.
>
> That seems reasonable to me too (separate print() and printf()
> thingies).
>
> >> Specifically, I wonder if it wouldn't be
> >> useful to require a format string with positional and keyword
> >> arguments being used for automatic interpolation.  E.g.
> >>
> >> print('%s: %s', field, value)
> >>
> >> instead of
> >>
> >> print('%s: %s' % (field, value))
> >
> > Before going too far down this route, first read PEP 3101, which
> > proposes a different way out of the issues around the % operator. (Not
> > saying we couldn't adopt PEP 3101 *and* a printf() function.
>
> Thanks for the pointer.  I agree that if we adopt a companion printf
> () function it should definitely require (only) PEP 3101 format strings.
>
> >> Another thought: what if 'print' weren't a function but a callable
> >> object. exposed in builtins.  I'm thinking something roughly parallel
> >> to stdout, stderr, stdin, or maybe better cout, cerr, cin.
> >
> > I'm not sure I follow the difference. It was quite clear from earlier
> > discussions that we *don't* want print to be a bound method of a
> > hypothetical print method on sys.stdout; that's much less flexible,
> > and makes it much more work to implement a file-like type. print()
> > should dynamically look up sys.stdout each time it is called (and
> > exactly once per call).
> >
> > (BTW I'm not sure why you like cin/cout/cerr better than stdin/
> > stdout/stderr?)
>
> I was trying to make an analogy (probably poorly so) that 'print'
> would be bound to an instance as opposed to a function (current
> proposal) or structure-like thingie without methods as in std*.
>
> >> The default 'print' object then would be a "print-to-sys.stdout-with-
> >> \n", but you would then be able to do something like:
> >>
> >> stderr = print.clone(stream=sys.stderr)
> >> stderr('here go all my error messages')
> >>
> >> or
> >>
> >> smtpout = print.clone(stream=mysock, end='\r\n')
> >> smtpout('$code $msg', code=250, msg='Okay')
> >
> > I'd like to firmly resist adding more mechanism for somethign that can
> > be addressed so simply by a tiny wrapper function:
> >
> > def smtpout(*args):
> >  print(*args, file=mysock, end="\r\n")
>
> What I was trying to address was the namespace collisions you'd get
> between printf-style keyword arguments and 'special' arguments that
> the print function would accept.  The thing is that if we were to add
> both a print and a printf, you'd like for their interfaces to be as
> similar as possible except that the latter takes a format string as
> its first argument.  Clearly 'special' arguments like 'file' and
> 'end' will prohibit their use as keyword arguments for printf, and I
> don't like that (this isn't anything new -- the problem crops up
> occasionally in other places, see the email package API).


Why do they need to be similar?  print() is just a really simple, quick
output function.  printf() seems to be proposed for advanced formatting
needs for sending to stdout.  As Guido said, the newline should be explicit
in printf(), so that already begins to deviate from print().  And 'sep'
would be useless.  So all that leaves is 'file', and that is just
convenience for passing a string to a 'write' method.  And for that I think
people can just learn not to use 'file' as a name of a variable.  Or we can
just lose that as well and have printf() be nothing more than a convenience
function that avoids having to specify the method name and sending it to
print with the desired arguments.

Then again, this to me seems to make printf() not that grand to lead to its
own built-in.  But I have been fine with the way things have been so far
anyway with having a separate line that stores some string I created before
calling 'print'.  If you don't have any keyword arguments then printf() is
just::

  def printf(format_string, *args, **kwargs):
      output = format_string.format(*args, **kwargs)
      print(output, end='')


One way out would be to use special argument names, e.g. prepending
> and or appending some number of underscores.  This doesn't eliminate
> the problem, but it reduces its likelihood.


I don't see this being a big problem if the only argument that is common is
'file'.

  Another approach would
> for 'print' to be an object with attributes that would change
> behavior, such as the stream to write to or line-ending characters.


Yuck.  Why would you want to suddenly set global semantics in a built-in
instead of a module?  That's a total shift in how Python does things at a
global level.  Plus I don't want to have to worry about setting 'sep' or
'end' for every function that uses print() just because some other function
decided that it didn't like single spaces but wanted to use tabs for a
separator.

A possible third approach might be some special singleton objects
> that, when seen positionally would modify the state of the underlying
> print object.


Yuck again. This seems to be getting rather specific just for printing when
we have already lived without all of this functionality for so long with
minor issue.

  Of the three (maybe fourth: "sorry you can't use 'end'
> or 'file'"),


I like this option.  =)

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061120/781abb99/attachment.htm 

From solipsis at pitrou.net  Mon Nov 20 20:39:55 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 20 Nov 2006 20:39:55 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
Message-ID: <1164051595.4864.25.camel@fsol>

Le lundi 20 novembre 2006 ? 10:27 -0800, Guido van Rossum a ?crit :
> I'd like to have a discussion of abilities and interfaces but right
> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
> mostly isomorphic to Zope's notion of interfaces, but the new
> terminology may make it possible to clarify certain distinctions
> better, e.g. the distinction between a class that provides an ability
> to its instances, and the abilities that an instance has.
> 
> Feel free to riff on this theme here. I'll check in within 24 hours or so.


It doesn't directly answer your post, but I think the way an
object/class is queried to know whether it implements a given interface
should be quite free-form, which means it's up to the interface creator
to decide what really he wants to check.

For example, the creator of the Number interface may just check that
Number is declared as an interface, without going any further. So, the
author of Complex class just has to write:

class Complex:
    __implements__ = (Number,)

An interface creator who has a taste for "stricter" typing may on the
other hand mandate that certain or attributes methods are defined, etc.
He could then raise an error if a class claiming to implement his
interface doesn't provide the required methods/attributes.


So to sum it up, to know whether an object-or-class implements a given
interface, you could call a standard function
implements(obj, interface), which in turn would call
interface.__implemented_by__(obj). It's up to each interface to decide
what breadth of tests it wants to perform, although it would be
practical to fallback on a default implementation (in the Interface base
class common to all interfaces) which just checks for presence of the
interface in obj.__implements__.


(I don't know if this post is useful. Sorry if it's not)

Antoine.



From sluggoster at gmail.com  Mon Nov 20 20:42:34 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Mon, 20 Nov 2006 11:42:34 -0800
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <91ad5bf80611200906y394e6257jd8bd5354d5b98a66@mail.gmail.com>
References: <002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<455FD6C0.7000105@gmail.com> <4561741E.1040102@gmail.com>
	<456179CF.1060201@cenix-bioscience.com>
	<dcbbbb410611200625l3629ca08jdaba454b94f906a5@mail.gmail.com>
	<91ad5bf80611200906y394e6257jd8bd5354d5b98a66@mail.gmail.com>
Message-ID: <6e9196d20611201142q7bd65ca8l622ba42e9b9c0fa5@mail.gmail.com>

On 11/20/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> One issue here is the (informal) name of the protocol "file-like",
> which although it adds the suffix "like" and gives a clue it's not
> necessarily about real files, it makes such erroneous assumptions more
> likely.

This is an issue througout Python, one that people have just learned
to live with.  Generally it works because routines use only the
minimum characteristic feature of the original type (file.read() or
file.write(), dict.__getitem__()), and the user has assumed correctly
which features are required (or has looked at the routine's source to
verify what it calls).  But obviously this fails if the routine calls
dict.keys() or dict.pop() and the object doesn't have 100% dict
compatibility.

Older parts of the Python docs describe exactly which methods are
called and with which arguments.  For instance, "a writelines method
that will be called with one argument, and a read method that will be
called with no arguments".  (Because some file-like objects leave out
the optional numeric argument to .read().)  But this is so wordy to
document that most programmers don't go to this level.

I can see the use for MinimalFileLikeForReading,
MinimalFileLikeForWriting, and CompletelyFileLike interfaces.

> Anyone else that would find
> a 'stream' (or 'charstream' or 'StreamMixin' or whatever) class a good
> idea?

Possibly, although we should distinguish between reading and writing.
The traceback module is a good example.  It accepts file-like objects
and calls only file.write.

interface IReadStream:
    def read(self, num_bytes=None)  =>  bytes object

interface IWriteStream:
    def write(self, bytes) => None    # Writes the bytes to the stream.

('bytes object' being the successor to 'str' strings in Python 3000.)

The word "stream" may connote more than this to C/Java programmers --
we'd have to check.

-- 
Mike Orr <sluggoster at gmail.com>

From g.brandl at gmx.net  Mon Nov 20 20:42:28 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 20 Nov 2006 20:42:28 +0100
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>	<ejqiqb$d38$1@sea.gmane.org>	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
Message-ID: <ejt0fc$jk9$1@sea.gmane.org>

Jim Jewett wrote:
> On 11/19/06, Guido van Rossum <guido at python.org> wrote:
>> On 11/19/06, Georg Brandl <g.brandl at gmx.net> wrote:
>> > Guido van Rossum wrote:
> 
>> > > 2. A keyword parameter will direct it to a different file.
>> > > Probably print(x, y, z, file=<file-like object>).
> 
>> > > 3. Two more keyword parameters can change what gets inserted between
>> > > items and at the end.
>> > > Perhaps print(x, y, z, sep=<string>, end=<string>). These default to "
>> > > " and "\n", respectively.
> 
>> A PEP may also discourage attempts to add more cruft,
> 
> (1)  Is this an explicit rejection of a keyword for a format string, such as
> 
>     print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")

-1. This is confusing, as normally the format string comes first, and then
the arguments to be formatted.

> (2)  What happens to additional keyword arguments that do get passed?
>     (a)  A StandardError of some sort is raised.

Why not the standard TypeError: "xyz" is an invalid keyword arg for this function?

>     (b)  print is treated as though it had (but ignored) a **kwargs
> argument, to support extensions in print-replacements.
>     (c)  The extra arguments are printed at the end, perhaps with a label.

What label do you have in mind?

Georg


From guido at python.org  Mon Nov 20 20:52:28 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 11:52:28 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
Message-ID: <ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>

On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> >> Another thought: what if 'print' weren't a function but a callable
> >> object. exposed in builtins.  I'm thinking something roughly parallel
> >> to stdout, stderr, stdin, or maybe better cout, cerr, cin.
> >
> > I'm not sure I follow the difference. It was quite clear from earlier
> > discussions that we *don't* want print to be a bound method of a
> > hypothetical print method on sys.stdout; that's much less flexible,
> > and makes it much more work to implement a file-like type. print()
> > should dynamically look up sys.stdout each time it is called (and
> > exactly once per call).
> >
> > (BTW I'm not sure why you like cin/cout/cerr better than stdin/
> > stdout/stderr?)
>
> I was trying to make an analogy (probably poorly so) that 'print'
> would be bound to an instance as opposed to a function (current
> proposal) or structure-like thingie without methods as in std*.

Sorry, I still don't understand your proposal.

> >> The default 'print' object then would be a "print-to-sys.stdout-with-
> >> \n", but you would then be able to do something like:
> >>
> >> stderr = print.clone(stream=sys.stderr)
> >> stderr('here go all my error messages')
> >>
> >> or
> >>
> >> smtpout = print.clone(stream=mysock, end='\r\n')
> >> smtpout('$code $msg', code=250, msg='Okay')
> >
> > I'd like to firmly resist adding more mechanism for somethign that can
> > be addressed so simply by a tiny wrapper function:
> >
> > def smtpout(*args):
> >  print(*args, file=mysock, end="\r\n")
>
> What I was trying to address was the namespace collisions you'd get
> between printf-style keyword arguments and 'special' arguments that
> the print function would accept.  The thing is that if we were to add
> both a print and a printf, you'd like for their interfaces to be as
> similar as possible except that the latter takes a format string as
> its first argument.  Clearly 'special' arguments like 'file' and
> 'end' will prohibit their use as keyword arguments for printf, and I
> don't like that (this isn't anything new -- the problem crops up
> occasionally in other places, see the email package API).
>
> One way out would be to use special argument names, e.g. prepending
> and or appending some number of underscores.  This doesn't eliminate
> the problem, but it reduces its likelihood.  Another approach would
> for 'print' to be an object with attributes that would change
> behavior, such as the stream to write to or line-ending characters.
> A possible third approach might be some special singleton objects
> that, when seen positionally would modify the state of the underlying
> print object.  Of the three (maybe fourth: "sorry you can't use 'end'
> or 'file'"), it seems preferable to use a callable print object.

I still don't quite see how making print a callable object solves that
issue. But personally would be totally happy with "You can't use
'file' as a variable name in your format". ('end' and 'sep' should not
be supported by printf(), these should just be made part of the format
string.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Mon Nov 20 21:13:58 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 20 Nov 2006 15:13:58 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
	<ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>
Message-ID: <46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 20, 2006, at 2:52 PM, Guido van Rossum wrote:

> I still don't quite see how making print a callable object solves that
> issue.

It solves it by making the stream an attribute on the callable  
instead of an argument to the call.

> But personally would be totally happy with "You can't use
> 'file' as a variable name in your format". ('end' and 'sep' should not
> be supported by printf(), these should just be made part of the format
> string.)

If 'file' is the only special keyword argument to printf, then let's  
call it '_file' or some such.  There's probably less chance that a  
newbie will want to change the output stream than they'll want to use  
a file variable in their format string.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRWIMi3EjvBPtnXfVAQJ2aQP/Qcq+J5ZN0p6aH0QJSO5W7HpJ/MpBQCWu
vRQTu13NHFscPNIC3SFvRIg/QPZdpMYSpEUZSl4BUuQMp+MEGSSKKFbq14DyDTfV
O7VZ8QjN94EL410cZAHS2PZSZzGo0eG96zwSiGkEK+pjuKVwML+nTSq0xQqJF8Yu
w7btiFT/ThY=
=SxCt
-----END PGP SIGNATURE-----

From exarkun at divmod.com  Mon Nov 20 21:48:46 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 20 Nov 2006 15:48:46 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
Message-ID: <20061120204846.20948.1503749203.divmod.quotient.36290@ohm>

On Mon, 20 Nov 2006 10:27:05 -0800, Guido van Rossum <guido at python.org> wrote:
>I'd like to have a discussion of abilities and interfaces but right
>now I don't have time -- maybe tomorrow. I see Andrew's proposal as
>mostly isomorphic to Zope's notion of interfaces, but the new
>terminology may make it possible to clarify certain distinctions
>better, e.g. the distinction between a class that provides an ability
>to its instances, and the abilities that an instance has.

Zope Interface makes this distinction very clear.  It has separate
terminology for each thing, and separate APIs for dealing with each.

Jean-Paul

From jimjjewett at gmail.com  Mon Nov 20 21:57:25 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 20 Nov 2006 15:57:25 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <46B9A351-2664-4B58-86C6-BB2D9A8DDF7F@python.org>
References: <041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<20061120082547.8346.JCARLSON@uci.edu>
	<46B9A351-2664-4B58-86C6-BB2D9A8DDF7F@python.org>
Message-ID: <fb6fbf560611201257y7a7f690cvd52fcc34e37b13a2@mail.gmail.com>

On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 20, 2006, at 12:14 PM, Josiah Carlson wrote:
>
> > How much print statement rewriting do you want to do?  Me, I'd like to
> > get away with as little as possible.  Adding parenthesis isn't that
> > bad.
> > Changing everything to use a printf-style would be 1) annoying, 2)
> > unnecessary, and 3) I hate printf.

> Are we going to include a flawless transformation script for Python
> 3k?  If not, then I'm pretty much resigned to rewriting them by hand
> anyway.

Assuming no further changes to PEP 3105, then it should be possible,
at least for the print statement.

Just parse the thing into statements; when you get to a print
statement make the string replacement before writing it back out.

  The "print" literal translates into "print(" ... ")"

  ">>" file_expression moves to the end and translates into
"file="file_expression

  the printable_expression stays the same

  a trailing comma translates into "end=''" (an empty string)

  (legacy code never uses the sep keyword)

-jJ

From tomerfiliba at gmail.com  Mon Nov 20 22:17:52 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Mon, 20 Nov 2006 23:17:52 +0200
Subject: [Python-3000] __nonzero__ vs. __bool__
Message-ID: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>

how about changing the name of __nonzero__ to __bool__?

today it looks like:
* int(x) --> x.__int__()
* long(x) --> x.__long__()
* float(x) --> x.__float__()
* str(x) --> x.__str__()

but for bool it's x.__nonzero__()

i'd guess __nonzero__ existed even before the bool type was introduced (2.2?),

but in the cleanup-old-design-issues spirit, perhaps it can be made
symmetrical
to the other builtin types.


-tomer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061120/38523dd6/attachment.htm 

From greg.ewing at canterbury.ac.nz  Tue Nov 21 01:23:45 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 21 Nov 2006 13:23:45 +1300
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <041601c70cb4$5ff11470$cf09f01b@bagio>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
Message-ID: <45624711.2060806@canterbury.ac.nz>

Giovanni Bajo wrote:

> println(x,y,z) -> append "\n"
> print(x,y,z) -> no terminator
> print(x,y,z,"\r\n") -> custom terminator

The third one won't work, because you'll get a
space before the terminator.

--
Greg

From guido at python.org  Tue Nov 21 02:21:10 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 17:21:10 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
Message-ID: <ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>

Looks about right. Patch?

On 11/20/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> how about changing the name of __nonzero__ to __bool__?
>
> today it looks like:
> * int(x) --> x.__int__()
> * long(x) --> x.__long__()
> * float(x) --> x.__float__()
> * str(x) --> x.__str__()
>
> but for bool it's x.__nonzero__()
>
> i'd guess __nonzero__ existed even before the bool type was introduced
> (2.2?),
> but in the cleanup-old-design-issues spirit, perhaps it can be made
> symmetrical
> to the other builtin types.
>
>
> -tomer
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 21 02:35:41 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 17:35:41 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <fb6fbf560611201257y7a7f690cvd52fcc34e37b13a2@mail.gmail.com>
References: <041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<20061120082547.8346.JCARLSON@uci.edu>
	<46B9A351-2664-4B58-86C6-BB2D9A8DDF7F@python.org>
	<fb6fbf560611201257y7a7f690cvd52fcc34e37b13a2@mail.gmail.com>
Message-ID: <ca471dc20611201735j42c09d25x4c521e6fb06ae201@mail.gmail.com>

On 11/20/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On Nov 20, 2006, at 12:14 PM, Josiah Carlson wrote:
> >
> > > How much print statement rewriting do you want to do?  Me, I'd like to
> > > get away with as little as possible.  Adding parenthesis isn't that
> > > bad.
> > > Changing everything to use a printf-style would be 1) annoying, 2)
> > > unnecessary, and 3) I hate printf.
>
> > Are we going to include a flawless transformation script for Python
> > 3k?  If not, then I'm pretty much resigned to rewriting them by hand
> > anyway.
>
> Assuming no further changes to PEP 3105, then it should be possible,
> at least for the print statement.
>
> Just parse the thing into statements; when you get to a print
> statement make the string replacement before writing it back out.
>
>   The "print" literal translates into "print(" ... ")"
>
>   ">>" file_expression moves to the end and translates into
> "file="file_expression
>
>   the printable_expression stays the same
>
>   a trailing comma translates into "end=''" (an empty string)

No, end=" " (i.e. a single space). And the translation can't be
perfect, due to the dropping of softspace. But IMO for code using
print statements few people will mind occasionally seeing a line
ending with a space before the newline.

>   (legacy code never uses the sep keyword)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 21 02:36:53 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 17:36:53 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ejqoob$sca$1@sea.gmane.org>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
	<ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>
	<46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>
Message-ID: <ca471dc20611201736l47e3b1d8gf64c2bdf7706573d@mail.gmail.com>

On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> On Nov 20, 2006, at 2:52 PM, Guido van Rossum wrote:
> > I still don't quite see how making print a callable object solves that
> > issue.
>
> It solves it by making the stream an attribute on the callable
> instead of an argument to the call.

I'm still confused. Are you proposing that in order to print to a
different file we do

save_file = print.file
try:
print.file = open("log.txt", "a")
print(x, y, z)

etc.?

> > But personally would be totally happy with "You can't use
> > 'file' as a variable name in your format". ('end' and 'sep' should not
> > be supported by printf(), these should just be made part of the format
> > string.)
>
> If 'file' is the only special keyword argument to printf, then let's
> call it '_file' or some such.  There's probably less chance that a
> newbie will want to change the output stream than they'll want to use
> a file variable in their format string.
>
> - -Barry
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (Darwin)
>
> iQCVAwUBRWIMi3EjvBPtnXfVAQJ2aQP/Qcq+J5ZN0p6aH0QJSO5W7HpJ/MpBQCWu
> vRQTu13NHFscPNIC3SFvRIg/QPZdpMYSpEUZSl4BUuQMp+MEGSSKKFbq14DyDTfV
> O7VZ8QjN94EL410cZAHS2PZSZzGo0eG96zwSiGkEK+pjuKVwML+nTSq0xQqJF8Yu
> w7btiFT/ThY=
> =SxCt
> -----END PGP SIGNATURE-----
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 21 02:38:20 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 20 Nov 2006 17:38:20 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611201736l47e3b1d8gf64c2bdf7706573d@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
	<ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>
	<46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>
	<ca471dc20611201736l47e3b1d8gf64c2bdf7706573d@mail.gmail.com>
Message-ID: <ca471dc20611201738o42ffc7f1v5d6fe7907b78382d@mail.gmail.com>

[Sorry, sent before its time]

On 11/20/06, Guido van Rossum <guido at python.org> wrote:
> On 11/20/06, Barry Warsaw <barry at python.org> wrote:
> > On Nov 20, 2006, at 2:52 PM, Guido van Rossum wrote:
> > > I still don't quite see how making print a callable object solves that
> > > issue.
> >
> > It solves it by making the stream an attribute on the callable
> > instead of an argument to the call.
>
I'm still confused. Are you proposing that in order to print to a
different file we do

save_file = print.file
try:
  print.file = open("log.txt", "a")
  print(x, y, z)
finally:
  print.file = save_file

etc.?
>
> > > But personally would be totally happy with "You can't use
> > > 'file' as a variable name in your format". ('end' and 'sep' should not
> > > be supported by printf(), these should just be made part of the format
> > > string.)
> >
> > If 'file' is the only special keyword argument to printf, then let's
> > call it '_file' or some such.  There's probably less chance that a
> > newbie will want to change the output stream than they'll want to use
> > a file variable in their format string.

-0. I really don't want it to be _file for the basic print(); and
making them different is asking for trouble.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Tue Nov 21 10:50:25 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 21 Nov 2006 02:50:25 -0700
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
Message-ID: <aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>

On 11/20/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 11/19/06, Guido van Rossum <guido at python.org> wrote:
> > A PEP may also discourage attempts to add more cruft,
>
> (1)  Is this an explicit rejection of a keyword for a format string, such as
>
>     print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")

Although odd, this appeals to me somehow.  It avoids the risk of
passing an untrusted string as the format (even if that's much less
dangerous than in C).


-- 
Adam Olsen, aka Rhamphoryncus

From fredrik at pythonware.com  Tue Nov 21 12:05:28 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 21 Nov 2006 12:05:28 +0100
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>	<ejqiqb$d38$1@sea.gmane.org>	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
	<aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>
Message-ID: <ejumhp$dm9$1@sea.gmane.org>

Adam Olsen wrote:

>> (1)  Is this an explicit rejection of a keyword for a format string, such as
>>
>>     print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
> 
> Although odd, this appeals to me somehow.  It avoids the risk of
> passing an untrusted string as the format (even if that's much less
> dangerous than in C).

it doesn't read well, though.

but if we take the print, printf, and str.format trojka, the rather 
common mistake you mention, and my preference for designing API:s that 
have room for performance optimizations, and refactor things a bit, we 
can do as follows:

1) move formatting to a separate object.

     str.format(args)

becomes one or more of

     format(str).format(args) # object style
     format(str)(args) # partial application style
     format(str) % (args) # old-style

this allows the formatter object to parse the formatting string (and 
possibly also cache it), and also opens up for alternative target API:s 
(e.g. tostringlist, etc).

and the format object should somewhat modular and fully subclassable, of 
course, so Barry can prototype all his ideas all on his own ;-)

2) for convenience, extend print to treat a format object as a format 
specifier for the following arguments:

     print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")

becomes

     print(format("My X: %s, Your Y: %s, His Z: %s"), x, y ,z)

3) get rid of printf.

4) go to Iceland and optimize the heck out of format/print.

</F>


From tomerfiliba at gmail.com  Tue Nov 21 12:59:16 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 21 Nov 2006 13:59:16 +0200
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
Message-ID: <1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>

http://sourceforge.net/tracker/index.php?func=detail&aid=1600346&group_id=5470&atid=305470

"""some notes:
* the latex docs have been updated
* the slot name was changed from nb_nonzero to nb_bool
* all XXX_nonzero methods where changed to XXX_bool
* all the comments relating to nb_nonzero have been updated
* stdlib was updated
* all the test suites were updated

otoh, i couldn't get it to compile (MSVC++2003), because of a strange
bug in ceval.c (none of my code). seems the GETLOCAL macro causes
syntactic problems, but i didn't have time to check it thoroughly."""


-tomer

On 11/21/06, Guido van Rossum <guido at python.org> wrote:
>
> Looks about right. Patch?
>
> On 11/20/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> > how about changing the name of __nonzero__ to __bool__?
> >
> > today it looks like:
> > * int(x) --> x.__int__()
> > * long(x) --> x.__long__()
> > * float(x) --> x.__float__()
> > * str(x) --> x.__str__()
> >
> > but for bool it's x.__nonzero__()
> >
> > i'd guess __nonzero__ existed even before the bool type was introduced
> > (2.2?),
> > but in the cleanup-old-design-issues spirit, perhaps it can be made
> > symmetrical
> > to the other builtin types.
> >
> >
> > -tomer
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-3000/guido%40python.org
> >
> >
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061121/aa6d3280/attachment.htm 

From tomerfiliba at gmail.com  Tue Nov 21 13:23:24 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 21 Nov 2006 14:23:24 +0200
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
Message-ID: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>

[GvR]
> I'd like to have a discussion of abilities and interfaces but right
> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
> mostly isomorphic to Zope's notion of interfaces, but the new
> terminology may make it possible to clarify certain distinctions
> better, e.g. the distinction between a class that provides an ability
> to its instances, and the abilities that an instance has.

[Antoine Pitrou]
> class Complex:
>    __implements__ = (Number,)
...
> So to sum it up, to know whether an object-or-class implements a given
> interface, you could call a standard function
> implements(obj, interface), which in turn would call
> interface.__implemented_by__(obj). It's up to each interface to decide
> what breadth of tests it wants to perform, although it would be
> practical to fallback on a default implementation (in the Interface base
> class common to all interfaces) which just checks for presence of the
> interface in obj.__implements__.
-------------------------------------------------------------------------------

it reminds me greatly of the old (rejected) mechanism suggested by
the adaptation PEP... only weaker.

http://www.python.org/dev/peps/pep-0246/

these approaches only *check* a type for conformance with some
interface, but the adaptation introduced a new way to look at
interfaces/protocol/conformance at all:

instead of asking "does object X supports interface Y?", adapt()
attempts to *give* you an object X that supports interface Y.

adaptation is much more capable ("don't complain, do something
about it") and reflective (both the protocol and the instance can
perform the adaptation) so you don't have to explicitly inherit
from many base classes or interfaces.

i'm very +1 on reviving adapt() and dropping all those "i want
strongly-duck-typed interfaces in python" talks.

give power to the duck -- adapt can do it all.


-tomer

From barry at python.org  Tue Nov 21 14:28:32 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 21 Nov 2006 08:28:32 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ca471dc20611201738o42ffc7f1v5d6fe7907b78382d@mail.gmail.com>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>
	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
	<ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>
	<46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>
	<ca471dc20611201736l47e3b1d8gf64c2bdf7706573d@mail.gmail.com>
	<ca471dc20611201738o42ffc7f1v5d6fe7907b78382d@mail.gmail.com>
Message-ID: <34A5E882-228F-4CB9-ACBB-278394EB4585@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 20, 2006, at 8:38 PM, Guido van Rossum wrote:

> I'm still confused. Are you proposing that in order to print to a
> different file we do
>
> save_file = print.file
> try:
>  print.file = open("log.txt", "a")
>  print(x, y, z)
> finally:
>  print.file = save_file

You could do that, but to make it more convenient I was thinking  
about adding class methods, such as the clone() I used earlier.   
Let's rename that method though so that it's hopefully clearer:

lprint = print.open('log.txt', 'a')
lprint(x, y, z)
lprint.close()

- -or-

lprint.file.close()

lprint is just another instance of whatever class 'print' is, except  
that its 'file' attribute is set to the opened log.txt file.

Regardless of whether you like this or not <wink>, does that make sense?

- -Barry



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRWL/AHEjvBPtnXfVAQJ1TQQAltC6uJEpzkYXyb4Zr6hRrWsPmYfOyJnp
EOLieDvn/pTOVgA+iAuccEtac+Yp5Kl8bR+5s4B8XSLoD8gbKt7aMwmwBdy7fIOc
2OvH+pb2rn8laMAzp/OTzg7OF+ptJFXKdfOx8+pZdy41htV+6U8rsHWUCo2F59Sj
q3XzVDbMF44=
=DycV
-----END PGP SIGNATURE-----

From barry at python.org  Tue Nov 21 14:37:49 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 21 Nov 2006 08:37:49 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ejumhp$dm9$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>	<ejqiqb$d38$1@sea.gmane.org>	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
	<aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>
	<ejumhp$dm9$1@sea.gmane.org>
Message-ID: <55DAAEFE-373C-4799-9C8E-5A48FABD63C1@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 21, 2006, at 6:05 AM, Fredrik Lundh wrote:

> 2) for convenience, extend print to treat a format object as a format
> specifier for the following arguments:
>
>      print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
>
> becomes
>
>      print(format("My X: %s, Your Y: %s, His Z: %s"), x, y ,z)
>
> 3) get rid of printf.

That's not bad.  Is the format object allowed in any positional  
argument, or just the first?

- -Barry


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRWMBLnEjvBPtnXfVAQIPkQQAo9LHQIzIJnQ1AK7b2OqWKSEfReDQbaEd
7rVVjaYQEb9yuMmAdBEH1HzgA/llSEt4hwY/A7QM079rXTvikcNDQZOc6ATzOGIw
mBkRDizn2Z7XKluODe+zr0NOk74Phiv03CYBvlEoYVLUQSNBKI4EFKBmclmx+YsC
LR2FUmV5enU=
=f6RC
-----END PGP SIGNATURE-----

From p.f.moore at gmail.com  Tue Nov 21 15:10:47 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 21 Nov 2006 14:10:47 +0000
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
References: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
Message-ID: <79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>

On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> adaptation is much more capable ("don't complain, do something
> about it") and reflective (both the protocol and the instance can
> perform the adaptation) so you don't have to explicitly inherit
> from many base classes or interfaces.
>
> i'm very +1 on reviving adapt() and dropping all those "i want
> strongly-duck-typed interfaces in python" talks.

To some extent I agree, but I also think that generic functions are
even more flexible. (As per the previous discussion which started
covering adaptation and moved on to generic functions).

I'd like to see some clear evaluation of the pros and cons of the 3 -
interfaces, adaptation, and generic functions. My feeling is that they
are to some extent "competing" proposals, and we need agreement (or
BDFL ruling!) on which is the way forward, before we start discussing
specifics.

Paul.

From gsakkis at rutgers.edu  Tue Nov 21 16:10:43 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Tue, 21 Nov 2006 10:10:43 -0500
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ejumhp$dm9$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
	<aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>
	<ejumhp$dm9$1@sea.gmane.org>
Message-ID: <91ad5bf80611210710m36a56959m552aea57a0c88354@mail.gmail.com>

On 11/21/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Adam Olsen wrote:
>
> >> (1)  Is this an explicit rejection of a keyword for a format string, such as
> >>
> >>     print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
> >
> > Although odd, this appeals to me somehow.  It avoids the risk of
> > passing an untrusted string as the format (even if that's much less
> > dangerous than in C).
>
> it doesn't read well, though.
>
> but if we take the print, printf, and str.format trojka, the rather
> common mistake you mention, and my preference for designing API:s that
> have room for performance optimizations, and refactor things a bit, we
> can do as follows:
>
> 1) move formatting to a separate object.
>
>      str.format(args)
>
> becomes one or more of
>
>      format(str).format(args) # object style
>      format(str)(args) # partial application style
>      format(str) % (args) # old-style
>
> this allows the formatter object to parse the formatting string (and
> possibly also cache it), and also opens up for alternative target API:s
> (e.g. tostringlist, etc).
>
> and the format object should somewhat modular and fully subclassable, of
> course, so Barry can prototype all his ideas all on his own ;-)
>
> 2) for convenience, extend print to treat a format object as a format
> specifier for the following arguments:
>
>      print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
>
> becomes
>
>      print(format("My X: %s, Your Y: %s, His Z: %s"), x, y ,z)
>
> 3) get rid of printf.
>
> 4) go to Iceland and optimize the heck out of format/print.

I like the idea of a format object and (1),(3) and (4), but (2)
doesn't feel "right":
a. Will print have to do an isinstance(args[0], format) to decide what
to do ? If so, don't the usual arguments for duck typing and against
type checking apply here ? Even if print uses duck typing (e.g. tries
to call a format() method), it still doesn't feel right (wouldn't like
to pass some HardDrive object as first argument <wink>).
b. print(a,b,c) is no longer obvious whether it prints 3 unformatted
objects or 2 formatted, if you don't know what a is.
b. What if for some reason one actually wants to print the format
object instead of interpreting as a format specifier ?

I think format should be either a positional argument at a fixed
position or a keyword argument. Between the two there is a tradeoff;
the former is more readable but requires a separate printf() function,
the latter is less readable but we can get away with just print(). I'm
not strongly inclined for either, but they both look better than an
optional positional format argument.

George

From ncoghlan at gmail.com  Tue Nov 21 16:25:19 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Nov 2006 01:25:19 +1000
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <4561857A.8080101@cenix-bioscience.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>	<455FEEBB.9010100@cs.byu.edu>	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>	<06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
	<4561857A.8080101@cenix-bioscience.com>
Message-ID: <45631A5F.5080501@gmail.com>

Aaron Bingham wrote:
> Bill Janssen wrote:
> 
>>> Java interfaces are very useful, however. Java programming seems to be 
>>> less and less about inheritance and more and more about implementing 
>>> interfaces; at least it does amongst Java programmers with taste :-)
>>>    
>> It seems to me that that's where Python has a real advantage.  With
>> real support for multiple inheritance, Python "interfaces" could be
>> real classes (either like real Java classes or Java abstract classes),
>> perhaps providing default implementations.  You get the goodness of
>> mix-ins, along with interface communication.
>>
> I agree.  In Java, interfaces are necessary because multiple inheritance 
> is not supported.  I see no good reason to add an additional language 
> mechanism for interfaces when multiple inheritance would do the job, AFAICT.

Just because multiple inheritance is *possible* in Python, don't make the 
mistake of thinking that it is straight forward. Aside from mixin classes that 
themselves inherit directly from object, multiple inheritance can get very 
messy outside of tightly controlled type hierarchies (cooperative calls in 
particular can become a nightmare). This flies in the face of Python's use as 
a glue language to tie multiple components together (particularly given the 
problem that some of the elements being integrated may come from environments 
that *don't* support multiple inheritance, like the JVM).

The idea of a formal interface mechanism is to have something which conveys 
the desired information *without* all of the additional issues that come with 
using multiple inheritance.

It's similar to the rationale behind adding class decorators, even though 
everything that a class decorator can do could be done with a metaclass 
instead: getting a metaclass right is *hard*, but getting a class decorator 
right is relatively easy and permits many of the things that programmers would 
currently have to use a metaclass for.

Similarly, it's obviously possible to do interfaces in current Python (Zope, 
PEAK and Twisted all use them, after all), but the question is whether or not 
it can be made *easier* (both to write, and, more importantly, to read).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From rrr at ronadam.com  Tue Nov 21 16:31:24 2006
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 21 Nov 2006 09:31:24 -0600
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <34A5E882-228F-4CB9-ACBB-278394EB4585@python.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>	<bbaeab100611191704n2c086324padb41652e0ca567b@mail.gmail.com>	<ca471dc20611191803o6889236t3518bdd3d19faacd@mail.gmail.com>	<041601c70cb4$5ff11470$cf09f01b@bagio>	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>	<ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>	<46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>	<ca471dc20611201736l47e3b1d8gf64c2bdf7706573d@mail.gmail.com>	<ca471dc20611201738o42ffc7f1v5d6fe7907b78382d@mail.gmail.com>
	<34A5E882-228F-4CB9-ACBB-278394EB4585@python.org>
Message-ID: <ejv6bj$6h5$1@sea.gmane.org>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Nov 20, 2006, at 8:38 PM, Guido van Rossum wrote:
> 
>> I'm still confused. Are you proposing that in order to print to a
>> different file we do
>>
>> save_file = print.file
>> try:
>>  print.file = open("log.txt", "a")
>>  print(x, y, z)
>> finally:
>>  print.file = save_file
> 
> You could do that, but to make it more convenient I was thinking  
> about adding class methods, such as the clone() I used earlier.   
> Let's rename that method though so that it's hopefully clearer:
> 
> lprint = print.open('log.txt', 'a')
> lprint(x, y, z)
> lprint.close()
> 
> - -or-
> 
> lprint.file.close()
> 
> lprint is just another instance of whatever class 'print' is, except  
> that its 'file' attribute is set to the opened log.txt file.
> 
> Regardless of whether you like this or not <wink>, does that make sense?


I'll add 2 cents here...  (or possibly less)

I agree with others here the print function probably should be as simple and 
easy to use as possible, basically nearly identical to the current print use, 
but as a function.  It's easy and generalized enough to work in most situations 
and it fits the use case of an easy convenient built-in way to output basic text.

I think maybe there could be more specialized print methods on possibly a 
console (and printer?) object in the library.  That would then match the file 
objects use of write and writeline.

So you then might have a console.print() and console.printline(), and 
printer.print() and printer.printline(),  along with the built in file.write() 
and file.writeline().  But I think having a generic print object somehow doesn't 
fit this pattern well.

And weather or not these types of objects ever get added to the library, I can't 
see how it would effect the print function use in any way.

Does this make more (or less) sense?

Cheers,
    Ron







From ncoghlan at gmail.com  Tue Nov 21 16:28:02 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Nov 2006 01:28:02 +1000
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
References: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
	<79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
Message-ID: <45631B02.8050005@gmail.com>

Paul Moore wrote:
> On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
>> adaptation is much more capable ("don't complain, do something
>> about it") and reflective (both the protocol and the instance can
>> perform the adaptation) so you don't have to explicitly inherit
>> from many base classes or interfaces.
>>
>> i'm very +1 on reviving adapt() and dropping all those "i want
>> strongly-duck-typed interfaces in python" talks.
> 
> To some extent I agree, but I also think that generic functions are
> even more flexible. (As per the previous discussion which started
> covering adaptation and moved on to generic functions).
> 
> I'd like to see some clear evaluation of the pros and cons of the 3 -
> interfaces, adaptation, and generic functions. My feeling is that they
> are to some extent "competing" proposals, and we need agreement (or
> BDFL ruling!) on which is the way forward, before we start discussing
> specifics.

The BDFL has favoured function overloading (aka generic functions) pretty 
heavily in previous rounds of this discussion, but more formal interfaces may 
still form a significant part of that as an efficient means of dispatching to 
the different overloaded operations.

Multi-argument dispatch plus interfaces is likely to be cleaner than trying to 
do everything based on concrete types. But this really needs more input from 
the enterprise framework folks (PEAK, Zope, etc), as that's the level this 
feature is really aimed at. Quick & dirty scripts are unlikely to need it any 
more than they do now :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Tue Nov 21 17:09:07 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 08:09:07 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
Message-ID: <ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>

Can anyone help out evaluating this patch? If it has to wait for me
it's gonna be a looooooong wait...

On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> http://sourceforge.net/tracker/index.php?func=detail&aid=1600346&group_id=5470&atid=305470
>
> """some notes:
>  * the latex docs have been updated
>  * the slot name was changed from nb_nonzero to nb_bool
>  * all XXX_nonzero methods where changed to XXX_bool
>  * all the comments relating to nb_nonzero have been updated
>  * stdlib was updated
>  * all the test suites were updated
>
>  otoh, i couldn't get it to compile (MSVC++2003), because of a strange
> bug in ceval.c (none of my code). seems the GETLOCAL macro causes
> syntactic problems, but i didn't have time to check it thoroughly."""
>
>
> -tomer
>
>
> On 11/21/06, Guido van Rossum < guido at python.org> wrote:
> > Looks about right. Patch?
> >
> > On 11/20/06, tomer filiba < tomerfiliba at gmail.com> wrote:
> > > how about changing the name of __nonzero__ to __bool__?
> > >
> > > today it looks like:
> > > * int(x) --> x.__int__()
> > > * long(x) --> x.__long__()
> > > * float(x) --> x.__float__()
> > > * str(x) --> x.__str__()
> > >
> > > but for bool it's x.__nonzero__()
> > >
> > > i'd guess __nonzero__ existed even before the bool type was introduced
> > > ( 2.2?),
> > > but in the cleanup-old-design-issues spirit, perhaps it can be made
> > > symmetrical
> > > to the other builtin types.
> > >
> > >
> > > -tomer
> > > _______________________________________________
> > > Python-3000 mailing list
> > > Python-3000 at python.org
> > > http://mail.python.org/mailman/listinfo/python-3000
> > > Unsubscribe:
> > >
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
> > >
> > >
> > >
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> >
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Tue Nov 21 17:12:19 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 21 Nov 2006 18:12:19 +0200
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
References: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
	<79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
Message-ID: <1d85506f0611210812i2fcb14b1h9440ae530c3d30c5@mail.gmail.com>

adapt() is a "casting" mechanism, whereas generic functions are
dispatch mechanism (multimethods)... i'd guess there are some
overlapping characteristics, but they are different by concept. [1]

we can say interfaces are a subset of adaption (adaption > interfaces):
* interfaces are mostly static, where as adaption is dynamic
* interfaces only indicate compliance, adaptation resolves mismatches
* you can implement interfaces using adaptation (__adapt__ just checks
  for isinstance or some mechanism)

but comparing adaption to generic functions is apples to oranges.

suppose i have a function that logs text to a file:
def log(text, fileobj = sys.stdout):
    fileobj.write(text + "\n")

and for reasons like computation with side-effects, i want *early* detection
of errors (before side effects occur): i want to make sure fileobj has is a
file-like object. [2]

interfaces and adaptation solve this issue, but generic functions
are not applicable really. i mean, if i had a version of log() for
files, and another for lists as the backing store, i'd say
generic functions are the solution.

but all i asked for is an object that has a write() method, and for that
i don't want to require any multiple dispatch mechanisms.

besides, the use case of multiple dispatch is by far less common
than "i need to make sure the object i'm dealing with is what i expect"

= = = = = = = = =

as a bonus, type annotations go very well with adaption:

@adapting
def log(text : str, fileobj : file = sys.stdout):
     fileobj.write(text)

where the `adapting` deco takes the function's __signature__,
adapts all the arguments accordingly, and invokes the original
function with the adapted arguments. this way, all mismatching
types are detected prior to executing any code with side effects.

adaptions also doesn't require any best-fit algorithms, and is
easily extensible, as every argument can theoretically provide its
own "how-to-use-me" adapter.

and last but not least -- adaption is well suited for object proxies,
which would make RPyC's job easier :)


-tomer

[1] i know this pep has been killed, but i wanted to point out the
differences. if it's meant to stay dead, i won't bring it up anymore.

[2] i may also define a WritableFileProtocol for adapt, that makes sure the
file's open-mode allows writing. i mean, the object's interface may provide
a write() method (hasattr(fileobj, "write")), but i want to make sure the file
is open properly, etc.




On 11/21/06, Paul Moore <p.f.moore at gmail.com> wrote:
> On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> > adaptation is much more capable ("don't complain, do something
> > about it") and reflective (both the protocol and the instance can
> > perform the adaptation) so you don't have to explicitly inherit
> > from many base classes or interfaces.
> >
> > i'm very +1 on reviving adapt() and dropping all those "i want
> > strongly-duck-typed interfaces in python" talks.
>
> To some extent I agree, but I also think that generic functions are
> even more flexible. (As per the previous discussion which started
> covering adaptation and moved on to generic functions).
>
> I'd like to see some clear evaluation of the pros and cons of the 3 -
> interfaces, adaptation, and generic functions. My feeling is that they
> are to some extent "competing" proposals, and we need agreement (or
> BDFL ruling!) on which is the way forward, before we start discussing
> specifics.
>
> Paul.
>

From guido at python.org  Tue Nov 21 17:14:21 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 08:14:21 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <34A5E882-228F-4CB9-ACBB-278394EB4585@python.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<041601c70cb4$5ff11470$cf09f01b@bagio>
	<FA736C7D-816F-4157-9D13-42383B2FF45F@python.org>
	<ca471dc20611201021w534f51cft157e5e6c8a1e308a@mail.gmail.com>
	<64AE2078-2ABF-4975-95B8-5698EED1EEFC@python.org>
	<ca471dc20611201152v626bc479wfae8ebfbc3486741@mail.gmail.com>
	<46A6517D-6EA1-4BD2-94F7-02D3C7A4AB64@python.org>
	<ca471dc20611201736l47e3b1d8gf64c2bdf7706573d@mail.gmail.com>
	<ca471dc20611201738o42ffc7f1v5d6fe7907b78382d@mail.gmail.com>
	<34A5E882-228F-4CB9-ACBB-278394EB4585@python.org>
Message-ID: <ca471dc20611210814j77065daas909ce14391a9c25e@mail.gmail.com>

OK. I think this proposal is dead, if only because I still don't
understand it (or at least I don't understand why anyone would propose
this particular API). Everything about it seems backwards. Even if you
could explain it, that doesn't bode well for how future generations of
Python users will struggle with it. So let's stop wasting our breath
and bury it.

Summary: -1.

--Guido

On 11/21/06, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 20, 2006, at 8:38 PM, Guido van Rossum wrote:
>
> > I'm still confused. Are you proposing that in order to print to a
> > different file we do
> >
> > save_file = print.file
> > try:
> >  print.file = open("log.txt", "a")
> >  print(x, y, z)
> > finally:
> >  print.file = save_file
>
> You could do that, but to make it more convenient I was thinking
> about adding class methods, such as the clone() I used earlier.
> Let's rename that method though so that it's hopefully clearer:
>
> lprint = print.open('log.txt', 'a')
> lprint(x, y, z)
> lprint.close()
>
> - -or-
>
> lprint.file.close()
>
> lprint is just another instance of whatever class 'print' is, except
> that its 'file' attribute is set to the opened log.txt file.
>
> Regardless of whether you like this or not <wink>, does that make sense?
>
> - -Barry
>
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (Darwin)
>
> iQCVAwUBRWL/AHEjvBPtnXfVAQJ1TQQAltC6uJEpzkYXyb4Zr6hRrWsPmYfOyJnp
> EOLieDvn/pTOVgA+iAuccEtac+Yp5Kl8bR+5s4B8XSLoD8gbKt7aMwmwBdy7fIOc
> 2OvH+pb2rn8laMAzp/OTzg7OF+ptJFXKdfOx8+pZdy41htV+6U8rsHWUCo2F59Sj
> q3XzVDbMF44=
> =DycV
> -----END PGP SIGNATURE-----
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 21 17:18:34 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 08:18:34 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ejumhp$dm9$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
	<aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>
	<ejumhp$dm9$1@sea.gmane.org>
Message-ID: <ca471dc20611210818x4683862dm9e95ca61a2868dbc@mail.gmail.com>

On 11/21/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Adam Olsen wrote:
>
> >> (1)  Is this an explicit rejection of a keyword for a format string, such as
> >>
> >>     print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
> >
> > Although odd, this appeals to me somehow.  It avoids the risk of
> > passing an untrusted string as the format (even if that's much less
> > dangerous than in C).
>
> it doesn't read well, though.
>
> but if we take the print, printf, and str.format trojka, the rather
> common mistake you mention, and my preference for designing API:s that
> have room for performance optimizations, and refactor things a bit, we
> can do as follows:
>
> 1) move formatting to a separate object.
>
>      str.format(args)
>
> becomes one or more of
>
>      format(str).format(args) # object style
>      format(str)(args) # partial application style
>      format(str) % (args) # old-style
>
> this allows the formatter object to parse the formatting string (and
> possibly also cache it), and also opens up for alternative target API:s
> (e.g. tostringlist, etc).
>
> and the format object should somewhat modular and fully subclassable, of
> course, so Barry can prototype all his ideas all on his own ;-)
>
> 2) for convenience, extend print to treat a format object as a format
> specifier for the following arguments:
>
>      print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
>
> becomes
>
>      print(format("My X: %s, Your Y: %s, His Z: %s"), x, y ,z)
>
> 3) get rid of printf.
>
> 4) go to Iceland and optimize the heck out of format/print.

Hm. While not as obviously from a different universe as Barry's
proposal, this still pretty weird, probably at least from a different
planet (much farther than Iceland anyway :-). Treating the first
argument differently based on its being of a specific type doesn't
sound right to me; what if you are handed an object x and you decide
to print it using print(x), but surreptitiously (or by mistake) they
hand you a format object?

I do support adding printf(), or printf()-like capabilities. But I
suggest to leave this for a separate PEP. After all, one of the
arguments PEP 3105 mentions for making print() a function is that it
opens the door for printf()...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 21 17:24:11 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 08:24:11 -0800
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <1d85506f0611210812i2fcb14b1h9440ae530c3d30c5@mail.gmail.com>
References: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
	<79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
	<1d85506f0611210812i2fcb14b1h9440ae530c3d30c5@mail.gmail.com>
Message-ID: <ca471dc20611210824l532d11dq1db1e317cb8abaf8@mail.gmail.com>

On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> adapt() is a "casting" mechanism, whereas generic functions are
> dispatch mechanism (multimethods)... i'd guess there are some
> overlapping characteristics, but they are different by concept. [1]

You're wasting everybody's time. We went over this last time. Please
read the archives.

Quick summary: using generic functions, you can trivially implement
adapt() or anything like it -- and lots of other things. Either way,
interfaces or abilities add hugely to the usability -- without them,
you're bound to dispatch only on concrete classes, which have severe
limitations for this purpose. There's a reason Zope has interfaces
*and* adapt -- they go together. So do generic functions and
interfaces.

References (these don't mention interfaces yet, but they explain how
having generic functions makes it unnecessary to have adapt as a
central mechanism):

http://www.artima.com/weblogs/viewpost.jsp?thread=155123
http://www.artima.com/weblogs/viewpost.jsp?thread=155514

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Tue Nov 21 17:47:09 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 21 Nov 2006 08:47:09 PST
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com> 
References: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
	<79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
Message-ID: <06Nov21.084719pst."58648"@synergy1.parc.xerox.com>

> To some extent I agree, but I also think that generic functions are
> even more flexible. (As per the previous discussion which started
> covering adaptation and moved on to generic functions).

While I like generic functions, they only solve part of the problem.
There are cases other than calling a function when a program needs to
be able to look at a type and understand what to do with it.  I'd
favor promoting an inheritance-based interface system for type
understanding, then move on to generic functions based on that
infrastructure.

Bill

From krstic at solarsail.hcs.harvard.edu  Tue Nov 21 17:49:38 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Tue, 21 Nov 2006 11:49:38 -0500
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
Message-ID: <45632E22.2030705@solarsail.hcs.harvard.edu>

Guido van Rossum wrote:
> Can anyone help out evaluating this patch? If it has to wait for me
> it's gonna be a looooooong wait...

Looks fine to me functionally, although it seems to gratuitously retab
some code that used to be aligned with tabstop 8 to a tabstop of 4.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From janssen at parc.com  Tue Nov 21 17:49:49 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 21 Nov 2006 08:49:49 PST
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <45631A5F.5080501@gmail.com> 
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu>
	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>
	<06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
	<4561857A.8080101@cenix-bioscience.com>
	<45631A5F.5080501@gmail.com>
Message-ID: <06Nov21.084954pst."58648"@synergy1.parc.xerox.com>

> The idea of a formal interface mechanism is to have something which conveys 
> the desired information *without* all of the additional issues that come with 
> using multiple inheritance.

I hear what you're saying here, and I'm sympathetic to this.  My
question is whether it's practically possible to add another "type"
system on top of the existing one.

Bill

From jimjjewett at gmail.com  Tue Nov 21 18:07:35 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 21 Nov 2006 12:07:35 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <45631A5F.5080501@gmail.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>
	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>
	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>
	<455FEEBB.9010100@cs.byu.edu>
	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>
	<4561857A.8080101@cenix-bioscience.com> <45631A5F.5080501@gmail.com>
Message-ID: <fb6fbf560611210907m42edd6fagbdd3a652c319e629@mail.gmail.com>

On 11/21/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Aaron Bingham wrote:

> > I agree.  In Java, interfaces are necessary because multiple inheritance
> > is not supported.  I see no good reason to add an additional language
> > mechanism for interfaces when multiple inheritance would do the job, AFAICT.

> Just because multiple inheritance is *possible* in Python, don't make the
> mistake of thinking that it is straight forward. Aside from mixin classes that
> themselves inherit directly from object, multiple inheritance can get very
> messy outside of tightly controlled type hierarchies

Interfaces have almost exactly the same limits.

A mixin class is basically the equivalent of

    (a)  A Java interface, plus
    (b)  A matching default (possibly abstract) base class, plus
    (c)  Automatic delegation (to the mixin), even for classes that
need to inherit from something else.

> (cooperative calls in particular can become a nightmare).

If the method name is reused with different meanings, you have a
nightmare no matter how you work it.

If the mixin knows that it is the the most base provider of a method,
you have no problem.  Normally, this can be arranged by putting mixins
at the end of the class statement.

If you have multiple mixins each of which may or may not implement
methods with the same name (such as "save") *and* that name won't be
found on object ... then it gets a bit ugly.  IMHO, it still isn't any
uglier than the Java equivalent of sequentially calling the method for
each interface separately.

-jJ

From exarkun at divmod.com  Tue Nov 21 18:22:31 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Tue, 21 Nov 2006 12:22:31 -0500
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <fb6fbf560611210907m42edd6fagbdd3a652c319e629@mail.gmail.com>
Message-ID: <20061121172231.20948.1479947618.divmod.quotient.37702@ohm>

On Tue, 21 Nov 2006 12:07:35 -0500, Jim Jewett <jimjjewett at gmail.com> wrote:
>On 11/21/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Aaron Bingham wrote:
>
>> > I agree.  In Java, interfaces are necessary because multiple inheritance
>> > is not supported.  I see no good reason to add an additional language
>> > mechanism for interfaces when multiple inheritance would do the job, AFAICT.
>
>> Just because multiple inheritance is *possible* in Python, don't make the
>> mistake of thinking that it is straight forward. Aside from mixin classes that
>> themselves inherit directly from object, multiple inheritance can get very
>> messy outside of tightly controlled type hierarchies
>
>Interfaces have almost exactly the same limits.

Having developed _extremely_ complex systems (often, the same system
repeatedly, sometimes using inheritance, sometimes using interface) with
each, I have to strongly disagree here.

>
>A mixin class is basically the equivalent of
>
>    (a)  A Java interface, plus
>    (b)  A matching default (possibly abstract) base class, plus
>    (c)  Automatic delegation (to the mixin), even for classes that
>need to inherit from something else.
>
>> (cooperative calls in particular can become a nightmare).
>
>If the method name is reused with different meanings, you have a
>nightmare no matter how you work it.

Not so.  Composition works much better than inheritance when this case
comes up, and composition and interfaces tend to go hand in hand.

>
>If the mixin knows that it is the the most base provider of a method,
>you have no problem.  Normally, this can be arranged by putting mixins
>at the end of the class statement.

It is often impossible to know this, except when every aspect of the
inheritance hierarchy is tightly controlled.  This is rarely, if ever,
the case for a class exposed by a library as part of its public API.

Jean-Paul

From guido at python.org  Tue Nov 21 18:35:02 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 09:35:02 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <45632E22.2030705@solarsail.hcs.harvard.edu>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
	<45632E22.2030705@solarsail.hcs.harvard.edu>
Message-ID: <ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>

On 11/21/06, Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> wrote:
> Guido van Rossum wrote:
> > Can anyone help out evaluating this patch? If it has to wait for me
> > it's gonna be a looooooong wait...
>
> Looks fine to me functionally, although it seems to gratuitously retab
> some code that used to be aligned with tabstop 8 to a tabstop of 4.

Thanks! I hope you meant "indentation level" instead of "tabstop".
Tomer, can you fix the indentation to be whatever's prevailing in the
file you're editing?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Tue Nov 21 18:42:25 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 21 Nov 2006 12:42:25 -0500
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
References: <1d85506f0611210423u7386f622ta8ace619509add30@mail.gmail.com>
	<79990c6b0611210610k57d5cda4n1a16fb53dfb3ae59@mail.gmail.com>
Message-ID: <fb6fbf560611210942g1577c3a3t909b748790ae2ca2@mail.gmail.com>

On 11/21/06, Paul Moore <p.f.moore at gmail.com> wrote:
> I'd like to see some clear evaluation of the pros and cons of the 3 -
> interfaces, adaptation, and generic functions. My feeling is that they
> are to some extent "competing" proposals, and we need agreement (or
> BDFL ruling!) on which is the way forward, before we start discussing
> specifics.

Interfaces
==========

An interface is the equivalent of

    >>> obj.magic_name = magic_value

Many function decorators do no more than this.  Function (or method,
or class) annotations can be very useful.  They do rely on everyone
following the same convention, so they end up with most of the same
drawbacks as an isinstance check.  (If you implement them by
inheriting from a specified otherwise empty class, then they are
equivalent to isinstance.)


Adaptations
===========

An adaptation is the equivalent of

    >>> obj = MagicAdapter(obj)

Adaptation is strictly more powerful than interfaces.  It may be
simpler to understand/write/use if it already has interfaces to build
on.  Think of it as dynamic typecasting.

Instead of saying "Is this an X?", you say "I want this to be an X".
If the object is already an X, then you lose only the time spent to
verify that (similar to an assert).  If it isn't an X, then the
adapter can turn it into an X, or return a new (equivalent) object
that is an X.  __index__ is an example -- if you have an int or a
long, you get it back.  Otherwise, you get the equivalent int or long.



Generic Functions
=================

Generic functions are the most powerful, but they are also the hardest
to get your head around.  They make metaclasses seem straightforward.
They are less difficult if you have Adaptation and Interfaces around
to build on.  I cannot come up with a one-line equivalent.

    defop foo(arg1:Int, arg2:Seq): ...

This changes what foo(1, []) will do, but not what foo(1.0, 7) would do.

-jJ

From pje at telecommunity.com  Tue Nov 21 18:43:27 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 21 Nov 2006 12:43:27 -0500
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <mailman.1352.1164126254.32030.python-3000@python.org>
Message-ID: <5.1.1.6.0.20061121120913.03fb6738@sparrow.telecommunity.com>

At 08:24 PM 11/21/2006 -0800, "Guido van Rossum" <guido at python.org> wrote:
>Quick summary: using generic functions, you can trivially implement
>adapt() or anything like it -- and lots of other things. Either way,
>interfaces or abilities add hugely to the usability -- without them,
>you're bound to dispatch only on concrete classes, which have severe
>limitations for this purpose. There's a reason Zope has interfaces
>*and* adapt -- they go together. So do generic functions and
>interfaces.

Let's ponder that a moment.  Do you really need interfaces?

Suppose that I want to write some code that behaves differently for 
string-like things and list-like things.

In the current case, I could write code that does 'if isinstance()' for the 
two behaviors, and anybody using my code is screwed.

In the generic function case, I write a generic function that subsumes the 
'if', and anybody can define their type as string-like or list-like by 
reusing my existing 'str' and 'list' methods in the generic 
function.  Using a hypothetical generic function API, let's suppose my 
generic function is "flatten", and I want to add UserList to it, I might do 
something like:

     flatten[UserList] = flatten[list]

And that's my "interface" declaration, at least for that one operation.

Okay, so maybe you're thinking, "well, we'd want Userlist to be usable for 
every operation I write that wants to accept 'list-like' objects."  Okay, 
so I write something like this to cover all the generic operations I'm 
providing in my package:

     def like(newtype, oldtype):
         flatten[newtype] = flatten[oldtype]
         foo[newtype] = foo[oldtype]
         ...

And then call 'like(UserList, list)'.  So now you ask, "well, I want it to 
be listlike *globally*", and I say, "Really?"  What does that mean, exactly?

Well, consider len().  Isn't it just another generic function?  I mean, if 
you write code that simply uses Python's builtin and stdlib generics, then 
it suffices to make your new type work with those generics in the first 
place.  If my default "flatten()" method simply uses those generic 
operations, then any type that isn't declared to do something different, 
will "just work" if it supports the operations, and won't otherwise.

We don't really want to encourage introspectable interfaces, because that 
leads to unnecessary LBYL-ing.  Generic functions provide a kind of 
"type-safe" duck typing, because they do not rely on similarity of method 
names, but rather on explicit registration with a known generic 
function.  Thus, explicit introspection just duplicates the work of the 
generic function *while making it impossible for anybody to override the 
behavior*.  (Because you've gone back to using an if-then, which can't be 
extended by outside code!)

Therefore, if we do have an interface mechanism for use with generic 
functions, it should merely be a convenient way of *copying methods*; a 
more formal way of doing the 'like()' operation I described above.  We 
should avoid providing direct introspection of interfaces, because it is 
always possible to introspect for the specific operations you need (e.g. 
iterability, length-ability) by querying the relevant generic functions.

For example, one could have a 'hasattr' analog 'hasop(f,t)', such that 
'hasop(iter,list)' returns True.  If hasop() is a generic function, then 
user-defined generic function types (e.g. RuleDispatch-based ones) are 
equally usable.

(Some of what I've described of course would need more fleshing out for 
multiple-dispatch use cases, but single-dispatch is by far and away the 
most common use case, as can be seen from Python's existing generics.)


From walter at livinglogic.de  Tue Nov 21 18:43:20 2006
From: walter at livinglogic.de (=?ISO-8859-2?Q?Walter_D=F6rwald?=)
Date: Tue, 21 Nov 2006 18:43:20 +0100
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>	<45632E22.2030705@solarsail.hcs.harvard.edu>
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
Message-ID: <45633AB8.9010306@livinglogic.de>

Guido van Rossum wrote:
> On 11/21/06, Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> wrote:
>> Guido van Rossum wrote:
>>> Can anyone help out evaluating this patch? If it has to wait for me
>>> it's gonna be a looooooong wait...
>> Looks fine to me functionally, although it seems to gratuitously retab
>> some code that used to be aligned with tabstop 8 to a tabstop of 4.
> 
> Thanks! I hope you meant "indentation level" instead of "tabstop".
> Tomer, can you fix the indentation to be whatever's prevailing in the
> file you're editing?

Another question is whether __bool__() should be allowed to return an
int (or any other object), or if it *must* be a bool. (The patch
currently allows ints). What if __bool__() returns subclasses of int,
that overwrite __bool__() themself?

Servus,
   Walter


From guido at python.org  Tue Nov 21 18:59:41 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 09:59:41 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <45633AB8.9010306@livinglogic.de>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
	<45632E22.2030705@solarsail.hcs.harvard.edu>
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
	<45633AB8.9010306@livinglogic.de>
Message-ID: <ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>

I think it would set a good example if __bool__ was *required* to
return exactly True or False and no other object, not even int
instances equal to 0 or 1. Surely that can't be much of a burden. Feel
free to upload an improved patch.

On 11/21/06, Walter D?rwald <walter at livinglogic.de> wrote:
> Guido van Rossum wrote:
> > On 11/21/06, Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> wrote:
> >> Guido van Rossum wrote:
> >>> Can anyone help out evaluating this patch? If it has to wait for me
> >>> it's gonna be a looooooong wait...
> >> Looks fine to me functionally, although it seems to gratuitously retab
> >> some code that used to be aligned with tabstop 8 to a tabstop of 4.
> >
> > Thanks! I hope you meant "indentation level" instead of "tabstop".
> > Tomer, can you fix the indentation to be whatever's prevailing in the
> > file you're editing?
>
> Another question is whether __bool__() should be allowed to return an
> int (or any other object), or if it *must* be a bool. (The patch
> currently allows ints). What if __bool__() returns subclasses of int,
> that overwrite __bool__() themself?
>
> Servus,
>    Walter
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Tue Nov 21 19:29:33 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 21 Nov 2006 20:29:33 +0200
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
	<45632E22.2030705@solarsail.hcs.harvard.edu>
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
	<45633AB8.9010306@livinglogic.de>
	<ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
Message-ID: <1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>

patch updated


-tomer

On 11/21/06, Guido van Rossum <guido at python.org> wrote:
>
> I think it would set a good example if __bool__ was *required* to
> return exactly True or False and no other object, not even int
> instances equal to 0 or 1. Surely that can't be much of a burden. Feel
> free to upload an improved patch.
>
> On 11/21/06, Walter D?rwald <walter at livinglogic.de> wrote:
> > Guido van Rossum wrote:
> > > On 11/21/06, Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> wrote:
> > >> Guido van Rossum wrote:
> > >>> Can anyone help out evaluating this patch? If it has to wait for me
> > >>> it's gonna be a looooooong wait...
> > >> Looks fine to me functionally, although it seems to gratuitously
> retab
> > >> some code that used to be aligned with tabstop 8 to a tabstop of 4.
> > >
> > > Thanks! I hope you meant "indentation level" instead of "tabstop".
> > > Tomer, can you fix the indentation to be whatever's prevailing in the
> > > file you're editing?
> >
> > Another question is whether __bool__() should be allowed to return an
> > int (or any other object), or if it *must* be a bool. (The patch
> > currently allows ints). What if __bool__() returns subclasses of int,
> > that overwrite __bool__() themself?
> >
> > Servus,
> >    Walter
> >
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061121/d5a1704a/attachment.html 

From g.brandl at gmx.net  Tue Nov 21 20:08:11 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 21 Nov 2006 20:08:11 +0100
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>	<45632E22.2030705@solarsail.hcs.harvard.edu>	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>	<45633AB8.9010306@livinglogic.de>	<ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
	<1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
Message-ID: <ejviqs$snd$1@sea.gmane.org>

I applied the new patch.
Aside from a syntax error on line 4241 in typeobject.c, it compiles.
However, test_decimal fails (before the make test run bails out at
test_hotshot, which is unrelated...)

Georg

tomer filiba schrieb:
> patch updated
> 
> 
> -tomer
> 
> On 11/21/06, *Guido van Rossum* <guido at python.org 
> <mailto:guido at python.org>> wrote:
> 
>     I think it would set a good example if __bool__ was *required* to
>     return exactly True or False and no other object, not even int
>     instances equal to 0 or 1. Surely that can't be much of a burden. Feel
>     free to upload an improved patch.
> 
>     On 11/21/06, Walter D?rwald <walter at livinglogic.de
>     <mailto:walter at livinglogic.de>> wrote:
>      > Guido van Rossum wrote:
>      > > On 11/21/06, Ivan Krsti? < krstic at solarsail.hcs.harvard.edu
>     <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
>      > >> Guido van Rossum wrote:
>      > >>> Can anyone help out evaluating this patch? If it has to wait
>     for me
>      > >>> it's gonna be a looooooong wait...
>      > >> Looks fine to me functionally, although it seems to
>     gratuitously retab
>      > >> some code that used to be aligned with tabstop 8 to a tabstop
>     of 4.
>      > >
>      > > Thanks! I hope you meant "indentation level" instead of "tabstop".
>      > > Tomer, can you fix the indentation to be whatever's prevailing
>     in the
>      > > file you're editing?
>      >
>      > Another question is whether __bool__() should be allowed to return an
>      > int (or any other object), or if it *must* be a bool. (The patch
>      > currently allows ints). What if __bool__() returns subclasses of int,
>      > that overwrite __bool__() themself?
>      >
>      > Servus,
>      >    Walter
>      >
>      >
> 
> 
>     --
>     --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> 


From guido at python.org  Tue Nov 21 20:16:20 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 11:16:20 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
Message-ID: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>

On 11/20/06, Guido van Rossum <guido at python.org> wrote:
> I'd like to have a discussion of abilities and interfaces but right
> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
> mostly isomorphic to Zope's notion of interfaces, but the new
> terminology may make it possible to clarify certain distinctions
> better, e.g. the distinction between a class that provides an ability
> to its instances, and the abilities that an instance has.
>
> Feel free to riff on this theme here. I'll check in within 24 hours or so.

OK, I have some more time for this now. I'm hoping Andrew, Bill and
Phillip will join the discussion here (instead of various other
threads).


A. Preliminaries.

There are some open issues that we should probably address before
diving too deep into the specification of abilities or interfaces.

1. Abstract Base Classes (ABCs)?

Are we sure that we have passed the station of ABCs, or should we
revisit why those aren't sufficient for our purposes?

Bill Janssen seems to be in favor of just using ABCs. Pro: less new
infrastructure. Con: hard to add a new ABC on the fly to an existing
3rd party class (mucking with __bases__ is too much of a hack to
seriously propose).

I'd like to explore this some more before committing to anything.

2. Do we need interfaces if we have generic functions?

Phillip Eby argues that we don't need interfaces, because the right
API for generic functions makes it easy to do what we do with
interfaces. Let me paraphrase his argument.

Suppose we have a generic function F with implementations F1, F2, ...
(generally: Fi for 1 <= i <= n) suitable for different kinds of
arguments. (Concrete example: flatten() implemented one way for
strings, another way for lists.) Suppose we have a class C and we want
to make it acceptable to F, using some implementation Fi. Using
interfaces, each Fi is declared to take an argument that has a
particular interface, and we bind C to a certain Fi by claiming that C
provides the interface that Fi takes. This also makes C instances
behave in a certain way for all other generic functions that mention
the same interface. Using Phillip's proposal, we select the right Fi
and bind it to the class C using the assignment F[C] = Fi. This
doesn't do anything for other generic functions that might also
conceivably work with C. Phillip addresses that by proposing help
functions that do a bunch of these bindings at once, and by noticing
that if you just use the standard library and generic functions, the
default implementation might just work.

Phillip then argues that he doesn't want to encourage introspectable
interfaces. I think others have use cases for those though. It seems
that Phillip's approach only hanges well together if everybody totally
adopts the generic functions religion; I think that's an unlikely
(since too radical) change of course for Python 3.0. It also doesn't
seem to work for abilities that are tied to an instance instead of to
a class as Zope allows (see below).

Assuming we go with abilities (of some sort), we have more concrete
things to argue over:


B. Developing a spec.

I see this as a number of steps, although there's feedback possible
between them. We can borrow heavily from Zope, and perhaps somewhat
from Java.

1. Naming and general ideas. Should we call these Abilities or
Interfaces? Abilities, the term proposed by Andrew Koenig, have less
baggage, but possibly they're just an isomorphic relabeling of
interfaces. From Zope comes the important question: Do we want to
discuss the interfaces/abilities of an instance separately from those
provided by its class? Zope's answer is Yes, reminds Jean-Paul
Calderone, and I tend to like this (having been exposed to it in the
past).

2. More precise concepts. I'd like to distinguish between checking for
abilities (which should be quick as it may be used many times during
execution, e.g. whenever a generic function is called) and verifying
an ability, which would entail checking the class to see whether it
actually implements the API (and possibly maintains the invariants, if
we go that route) stipulated by a claimed ability. It would be a
performance nightmare if during he normal course of an application we
would be *verifying* abilities over and over -- this is equivalent to
a structural type check and can be very expensive. (Even more so if we
have optional argument type declarations.)

3. API design -- how do we spell the various concepts? E.g.
has_ability(x, A) asks whether object x has the ability A;
provides_ability(C, A) asks whether class C provides the ability A to
its instances. We could state that provides_ability(C, A) and
isinstance(x, C) implies has_ability(x, A). (A corollary would be that
a subclass can't take away an ability provided by a base class!) I
think it would be handy to have APIs that ask a class or instance for
the set of abilities it provides or has, e.g. get_abilities(x) or
get_provisions(C). Should And probably also for APIs to add/remove
abilities dynamically (if not constrained by base classes). The
verification API is separate also.

Some other questions:

- We need to come up with syntax for the common case, declaring a
class that claims to provide some abilities. Some thoughts: putting
the interfaces in the list of base classes might work -- the metaclass
can easily tell whether a particular base is a real class or an
ability. Another idea might be to add keyword parameters to the class
declaration, so we could write e.g. class C(B1, B2, abilities=(A1,
A2)): ...

- Should has_ability(x, A) pass the responsibility of computing the
outcome to the object A, which could implement an arbitrarily complex
algorithm to decide? Antoine Pitrou proposes this. The downside is
that we won't able to implement get_abilities(x) properly, or
efficiently. I'm also not sure of the use case.

- If has_ability(x, A) is true, should isinstance(x, C) be true?

- How does this interact with generic functions? I think the generic
function mechanism (if we introduce one) will have to be able to use
either classes or abilities to trigger behavior; while abilities are
nice and general, one shouldn't be forced to declare a new ability
when all one cares about is one particular concrete class (this
happens often enough in applications I develop). Obviously this is an
area where Bill's ABC's score better.


C. A library of standard abilities and mapping of abilities to
standard objects (Sequence to list, etc.).

It's easy to come up with a strawman set of interfaces, e.g. Iterable,
Iterator, Collection, Set, Sequence, Mapping, String, MutableSet,
MutableSequence, MutableMapping, MutableString, and inheritance
relations between them, e.g. Iterable - Collection - Sequence -
MutableSequence - list.

But it's not so easy to define which APIs must be present to for
verification of a particular interface to pass. A Mapping must
implement __getitem__() in a certain way. But does it need to
implement __len__? get()? keys()? values() and items()? Is it okay for
a particular mapping implementation to allow only one iterator to
exist  (per instance) at any time, like dbm files do? Does a String
need to implement lower()? Must a Sequence implement __add__() as
concatenation?

Answering these and similar questions probably requires a
standardization committed. (Any volunteers?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 21 20:18:20 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 11:18:20 -0800
Subject: [Python-3000] Abilities / Interfaces: why not adapt()?
In-Reply-To: <5.1.1.6.0.20061121120913.03fb6738@sparrow.telecommunity.com>
References: <mailman.1352.1164126254.32030.python-3000@python.org>
	<5.1.1.6.0.20061121120913.03fb6738@sparrow.telecommunity.com>
Message-ID: <ca471dc20611211118n79c5575p2c1185173366f2ed@mail.gmail.com>

On 11/21/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> Let's ponder that a moment.  Do you really need interfaces?

I responded to this in the parent thread (Abilities / Interfaces).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From walter at livinglogic.de  Tue Nov 21 21:18:30 2006
From: walter at livinglogic.de (=?ISO-8859-2?Q?Walter_D=F6rwald?=)
Date: Tue, 21 Nov 2006 21:18:30 +0100
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>	
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>	
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>	
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>	
	<45632E22.2030705@solarsail.hcs.harvard.edu>	
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>	
	<45633AB8.9010306@livinglogic.de>	
	<ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
	<1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
Message-ID: <45635F16.3060404@livinglogic.de>

tomer filiba wrote:
> patch updated

You're fast! ;)

AFAICT now we have the following problem: If
Objects/typeobject.c::slot_nb_bool() falls back to using __len__(), the
return value of __len__() must be a bool.

Servus,
   Walter

> On 11/21/06, *Guido van Rossum* <guido at python.org
> <mailto:guido at python.org>> wrote:
> 
>     I think it would set a good example if __bool__ was *required* to
>     return exactly True or False and no other object, not even int
>     instances equal to 0 or 1. Surely that can't be much of a burden. Feel
>     free to upload an improved patch.
> 
>     On 11/21/06, Walter D?rwald <walter at livinglogic.de
>     <mailto:walter at livinglogic.de>> wrote:
>     > Guido van Rossum wrote:
>     > > On 11/21/06, Ivan Krsti? < krstic at solarsail.hcs.harvard.edu
>     <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
>     > >> Guido van Rossum wrote:
>     > >>> Can anyone help out evaluating this patch? If it has to wait
>     for me
>     > >>> it's gonna be a looooooong wait...
>     > >> Looks fine to me functionally, although it seems to
>     gratuitously retab
>     > >> some code that used to be aligned with tabstop 8 to a tabstop of 4.
>     > >
>     > > Thanks! I hope you meant "indentation level" instead of "tabstop".
>     > > Tomer, can you fix the indentation to be whatever's prevailing
>     in the
>     > > file you're editing?
>     >
>     > Another question is whether __bool__() should be allowed to return an
>     > int (or any other object), or if it *must* be a bool. (The patch
>     > currently allows ints). What if __bool__() returns subclasses of int,
>     > that overwrite __bool__() themself?
>     >
>     > Servus,
>     >    Walter
>     >
>     >
> 
> 
>     --
>     --Guido van Rossum (home page: http://www.python.org/~guido/)
> 
> 


From sluggoster at gmail.com  Tue Nov 21 21:15:37 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 21 Nov 2006 12:15:37 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <6e9196d20611211215j782fd682s64681e1e6357960@mail.gmail.com>

On 11/21/06, Guido van Rossum <guido at python.org> wrote:
> 2. More precise concepts. I'd like to distinguish between checking for
> abilities (which should be quick as it may be used many times during
> execution, e.g. whenever a generic function is called) and verifying
> an ability, which would entail checking the class to see whether it
> actually implements the API (and possibly maintains the invariants, if
> we go that route) stipulated by a claimed ability. It would be a
> performance nightmare if during he normal course of an application we
> would be *verifying* abilities over and over -- this is equivalent to
> a structural type check and can be very expensive. (Even more so if we
> have optional argument type declarations.)

If I understand correctly, checking for an ability means "Does class C
claim to have ability A?", analagous to isinstance(), and verifying an
ability means introspecting the class and inspecting its methods.

The distinction is similar to checking whether an XML document claims
to match a DTD (or just assuming it does), vs checking all the tags
and attributes against the DTD.  The latter is definitely a lot of
overhead you don't need... except when you do.

> A Mapping must
> implement __getitem__() in a certain way. But does it need to
> implement __len__? get()? keys()? values() and items()?

Yes.  And no.  In other words, there should be at least two choices.
When most people think of "mapping" they expect x[key] to work.  When
they think of "dict like" they may mean this, or they may want to also
call  x.keys/values/items, and less commonly x.update and x.popitem.
Having two levels (Mapping and DictLike) would solve 90% of the
problem.  Having three levels may be overkill.  Same for lists, etc.

I wouldn't expect len(x) to work on every mapping.  Maybe the mapping
models an external resource like a filesystem, and calculating the
length would be expensive.  Maybe the object is volatile and doesn't
allow ascertaining all keys but only lookup of a particular key.

An example would be Cheetah's NameMapper.  It looks up a variable
against a list of objects, examining each object's attributes and keys
at the current moment.  Although it would be theoretically possible to
list all keys and calculate their number, this is not a supported
operation, and when you're recursing through 'self' and '__builtins__'
the number would become very large.  The correct response to "Does
variable x exist?"  is not "Is it in the list of existing keys?" but
"Does the NameMapper succeed in finding it?"

(As an aside, I got a momentary power outage while typing this, which
made me restart my computer.  It reminded me that these things can
happen at any time.)

-- 
Mike Orr <sluggoster at gmail.com>

From pje at telecommunity.com  Tue Nov 21 21:26:50 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 21 Nov 2006 15:26:50 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.co
 m>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>

At 11:16 AM 11/21/2006 -0800, Guido van Rossum wrote:
>Phillip then argues that he doesn't want to encourage introspectable
>interfaces. I think others have use cases for those though.

Examples?


>  It seems
>that Phillip's approach only hanges well together if everybody totally
>adopts the generic functions religion; I think that's an unlikely
>(since too radical) change of course for Python 3.0.

Is it really radical?

Consider the fact that single-dispatch generic functions are ubiquitous in 
the language and the stdlib, and have been pretty much forever.  They just 
vary in how they're implemented, and there's no One Obvious Way for 
developers to implement new ones, nor is there a standard way to add a 
method to an existing one.

That's because some generic functions use if-isinstance checks (bad), while 
others use custom __special__ methods (not bad, but not great), registries 
(good), adaptation (okay), or generic function libraries.

So, all I am proposing is that we:

    * provide a standard GF implementation for the simple cases
    * ...that is extensible to allow others to handle the more complex 
cases, by having a generic API for manipulating generic functions

So that the "one obvious way" to create new generics is to use the standard 
GF implementation unless you need something else.  And, there would be a 
standard way to add new methods to existing generic functions, perhaps 
using an addmethod() builtin or decorator, and it could be designed so that 
e.g.:

     addmethod(iter, somefunc, sometype)

would actually work by doing 'sometype.__iter__ = somefunc' under the 
hood.  This allows us not to have to change any builtin generics that are 
based on special method names.  In other words, the mechanism isn't 
radical, nor is the idea of having generics.  It's just an improvement in 
ease-of-use: an HCI change, not a comp-sci change!

And, with this relatively simple mechanism, all the fancier forms of 
generic functions (or interfaces and adapters) can be implemented via user 
libraries.  (More on how, below.)


>  It also doesn't
>seem to work for abilities that are tied to an instance instead of to
>a class as Zope allows (see below).

Actually, it still allows for that, as long as the base generic function 
system is extensible.  For example, RuleDispatch generic functions can 
choose implementations based on conditions such as whether an object has a 
particular attribute.  Zope could easily add a generic function type that 
uses their instance-based interface stuff to do the same thing.

I am merely proposing that Python not provide this sort of 
introspection-oriented stuff in the core or stdlib, not that nobody should 
be *allowed* to have it.

Indeed, I explicitly want the Python generic function API (e.g. addmethod, 
hasmethod, or whatever we call them) to *itself* be generic, so that users 
can create their own types that work with the Python-provided operations 
for generic function manipulation.  That is, I should be able to call:

     addmethod(addmethod, adder_for_my_gf_type, my_gf_type)

So that others can then call:

     # this calls adder_for_my_gf_type under the hood:
     addmethod(some_gf, some_method, some_cls)

where 'some_gf' is an instance of 'my_gf_type'.  This allows us not to keep 
the core or stdlib implementation of generic functions quite simple to 
handle the 80% (single dispatch) or 90% (concrete type-based multiple 
dispatch) with ease.  The final 10% (predicate dispatch, fancy interfaces, 
etc.) can then be done by outside libraries, since the use cases in that 
final 10% are more likely to vary than the base 80-90%.

Using the 'simplegeneric' library from PyPI (easy_install simplegeneric):

     from simplegeneric import generic

     @generic
     def addmethod(gf, method, cls):
         """Add a method to a generic function"""
         raise TypeError("Unknown generic function type", gf)

     @addmethod.when_object(iter)
     def add_to_iter(gf, method, cls):
         # XXX should have more error checking here, e.g.
         #     check for a local __iter__ first
         cls.__iter__ = method

     # ... similar declarations for other builtin generics

     @addmethod.when_type(FunctionType)
     def add_to_function(gf, method, cls):
         if hasattr(gf, 'when_type'):
             gf.when_type(cls)(method)
         else:
             raise TypeError("Not a generic function", gf)

And there you are: an extensible way to add new extensible function types, 
while using the same API for all of them.

(If you need multiple or predicate dispatch or method combining, it's easy 
to have decorators that apply to the method itself, so that the same 
three-argument addmethod() can still be used.)


>3. API design -- how do we spell the various concepts? E.g.
>has_ability(x, A) asks whether object x has the ability A;
>provides_ability(C, A) asks whether class C provides the ability A to
>its instances. We could state that provides_ability(C, A) and
>isinstance(x, C) implies has_ability(x, A).

Even if we explicitly have some type of "ability" object, I'd like them to 
be defined in terms of generic functions, so that I could effectively say 
something like:

     sequence_like = (iter & len)

(or perhaps spelled out in some other fashion) to create an "ability" 
representing iter-ability and len-ability.

One of the biggest conceptual/modeling issues with Zope-style interfaces is 
that they don't allow this kind of fine-grained protocol combination.  See, 
for example, zope.interface's long history of trying to nail down various 
Python protocols through elaborate interface inheritance 
hierarchies.  Defining interfaces strictly in terms of operations 
eliminates this issue.


>- How does this interact with generic functions?

The advanced abilities (per-instance/on-the-fly and multi-operation tests) 
will likely affect performance and/or simplicity of the default 
implementation.  A purely type-based system can be implemented efficiently, 
because declaring an interface can simultaneously add a concrete type 
registration for all affected generics.  (And registering an 
interface-linked method in a generic function can pull in all the concrete 
classes.)

Per-instance tests, however, increase code complexity.  Generally speaking, 
RuleDispatch does per-instance tests after class tests, but the way it gets 
reasonable performance is by building decision trees beforehand to manage 
the possible tests and avoid overlap.  If it actually had to call 
individual methods, or call a method more than once (ob.has_ability(Foo), 
ob.has_ability(Bar), etc.)  it would be considerably slower to select an 
option.

Also -- and this is the real kicker -- what do you do if more than one 
ability applies?  Now we have to have precedence rules for what's more 
specific.  The advantage of defining an ability as a set of one or more 
applicable generic functions is that precedence is comparatively 
straightforward: supersets take precedence over subsets, and overlaps are 
ambiguous.  You also have some possibility of being able to implement this 
by registration-time checks, without needing to build a dispatch tree at all.

I don't know of anybody who really uses per-instance interface declarations 
except for Zope.  I used them for a little while with PEAK, and decided 
they were the wrong idea for me; it made more sense to adapt to an 
interface that provides the metadata, or to use a RuleDispatch function 
that just directly introspected for whatever was needed.  I can't think of 
anything in the core or stdlib that would even need to go that far.

Again, my argument is that anything other than isinstance-based single and 
multiple-dispatch is too framework-dependent to be part of the 
language.  Simple things should be simple, complex things should be possible.


>Answering these and similar questions probably requires a
>standardization committed. (Any volunteers?)

Note that Zope has already tried for many years to do exactly this, and 
there is no indication that they have actually succeeded.  When I first 
mentioned PyProtocols on Python-Dev many years ago, Samuele Pedroni argued 
that individual operations (and groups thereof) should be the currency of 
interfaces, and after playing with them a bit, I agreed.  PyProtocols 
defines such interfaces as e.g.:

     protocolForType(file, ['read', 'close'])

to mean "file-like object with 'read' and 'close' methods".  If I define a 
class as supporting that protocol, and somebody wants to adapt to 
'protocolForType(file, ["read"])', then my type will work.

See 
http://peak.telecommunity.com/protocol_ref/protocols-generated-type.html 
for a more complete explanation, including how to get the system to 
actually check for the method names (so as not to require explicit 
declarations).

I think Samuele's idea (i.e., this) works a lot better than trying to 
define a rigid interface hierarchy for builtin-protocols, although I'd just 
as soon be able to do something like:

     file.read & file.close

rather than having to use strings.  But all this assumes that we are 
providing a builtin way to spell abilities and introspect them, which I 
still think is overkill for what Python actually needs to provide as a base.

In short, I think it's adding interface introspection that's the radical 
move, where generic functions are just a bit of polish that brings a bit 
more order to what Python already has.  In contrast, interfaces are a 
foreign religion imported from other languages, while the roots of the 
generic function faith (len, iter, etc.) have already been in place since 
the dawn of time!  :)


From brett at python.org  Tue Nov 21 21:32:05 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 21 Nov 2006 12:32:05 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <45635F16.3060404@livinglogic.de>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com>
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
	<45632E22.2030705@solarsail.hcs.harvard.edu>
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
	<45633AB8.9010306@livinglogic.de>
	<ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
	<1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
	<45635F16.3060404@livinglogic.de>
Message-ID: <bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>

On 11/21/06, Walter D?rwald <walter at livinglogic.de> wrote:
>
> tomer filiba wrote:
> > patch updated
>
> You're fast! ;)
>
> AFAICT now we have the following problem: If
> Objects/typeobject.c::slot_nb_bool() falls back to using __len__(), the
> return value of __len__() must be a bool.


Why can't the fallback usage just pass the return value from __len__ to
bool() (forget the C function name) and return that result?  It's just like
doing::

  def bool(obj):
      try:
          return obj.__bool__()
      except AttributeError:
          return bool(len(obj))

-Brett

Servus,
>    Walter
>
> > On 11/21/06, *Guido van Rossum* <guido at python.org
> > <mailto:guido at python.org>> wrote:
> >
> >     I think it would set a good example if __bool__ was *required* to
> >     return exactly True or False and no other object, not even int
> >     instances equal to 0 or 1. Surely that can't be much of a burden.
> Feel
> >     free to upload an improved patch.
> >
> >     On 11/21/06, Walter D?rwald <walter at livinglogic.de
> >     <mailto:walter at livinglogic.de>> wrote:
> >     > Guido van Rossum wrote:
> >     > > On 11/21/06, Ivan Krsti? < krstic at solarsail.hcs.harvard.edu
> >     <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
> >     > >> Guido van Rossum wrote:
> >     > >>> Can anyone help out evaluating this patch? If it has to wait
> >     for me
> >     > >>> it's gonna be a looooooong wait...
> >     > >> Looks fine to me functionally, although it seems to
> >     gratuitously retab
> >     > >> some code that used to be aligned with tabstop 8 to a tabstop
> of 4.
> >     > >
> >     > > Thanks! I hope you meant "indentation level" instead of
> "tabstop".
> >     > > Tomer, can you fix the indentation to be whatever's prevailing
> >     in the
> >     > > file you're editing?
> >     >
> >     > Another question is whether __bool__() should be allowed to return
> an
> >     > int (or any other object), or if it *must* be a bool. (The patch
> >     > currently allows ints). What if __bool__() returns subclasses of
> int,
> >     > that overwrite __bool__() themself?
> >     >
> >     > Servus,
> >     >    Walter
> >     >
> >     >
> >
> >
> >     --
> >     --Guido van Rossum (home page: http://www.python.org/~guido/)
> >
> >
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061121/3e851bf4/attachment.html 

From fumanchu at amor.org  Tue Nov 21 21:50:47 2006
From: fumanchu at amor.org (Robert Brewer)
Date: Tue, 21 Nov 2006 12:50:47 -0800
Subject: [Python-3000] Abilities / Interfaces
Message-ID: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>

Guido van Rossum wrote:
> But it's not so easy to define which APIs must be present to for
> verification of a particular interface to pass. A Mapping must
> implement __getitem__() in a certain way.
> But does it need to implement __len__?

The Ref Manual seems to think so:
http://docs.python.org/ref/types.html#l2h-68:

"The subscript notation a[k] selects the item indexed by k from the
mapping a; this can be used in expressions and as the target of
assignments or del statements. The built-in function len() returns the
number of items in a mapping."

> get()? keys()? values() and items()?

http://docs.python.org/ref/sequence-types.html implies all other methods
are optional (even if heavily encouraged):

"It is also recommended that mappings provide the methods keys(),
values(), items(), has_key(), get(), clear(), setdefault(), iterkeys(),
itervalues(), iteritems(), pop(), popitem(), copy(), and update()
behaving similar to those for Python's standard dictionary objects."

I'd be happy starting with the requirements as stated in the ref
(__getitem__, __setitem__, __delitem__, and __len__, as I read it), and
make any additions or other changes to that contract a separate
discussion. (But then, I'm personally happy with having this API
specified in the ref and don't see a need to codify it with interfaces.)

> Is it okay for a particular mapping implementation to allow
> only one iterator to exist  (per instance) at any time,
> like dbm files do?

Sure, because the current spec doesn't deny it.

If we have to have interfaces, I think we'd be far better off
implementing the spec as-is first, and worry about changing the spec
later (even if if only seems to be an "implied spec" at the moment). I
didn't write the Ref Manual, of course, sir ;) so I can't assert its
intent unequivocally, but as a consumer of it, I expect "it is
recommended" to mean "not an interface requirement".


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org

From guido at python.org  Tue Nov 21 22:27:17 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 13:27:17 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>
References: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>
Message-ID: <ca471dc20611211327i3a5ce396q4a96f5f4d1c563ca@mail.gmail.com>

On 11/21/06, Robert Brewer <fumanchu at amor.org> wrote:
> Guido van Rossum wrote:
> > But it's not so easy to define which APIs must be present to for
> > verification of a particular interface to pass. A Mapping must
> > implement __getitem__() in a certain way.
> > But does it need to implement __len__?
>
> The Ref Manual seems to think so:
> http://docs.python.org/ref/types.html#l2h-68:
>
> "The subscript notation a[k] selects the item indexed by k from the
> mapping a; this can be used in expressions and as the target of
> assignments or del statements. The built-in function len() returns the
> number of items in a mapping."
>
> > get()? keys()? values() and items()?
>
> http://docs.python.org/ref/sequence-types.html implies all other methods
> are optional (even if heavily encouraged):
>
> "It is also recommended that mappings provide the methods keys(),
> values(), items(), has_key(), get(), clear(), setdefault(), iterkeys(),
> itervalues(), iteritems(), pop(), popitem(), copy(), and update()
> behaving similar to those for Python's standard dictionary objects."
>
> I'd be happy starting with the requirements as stated in the ref
> (__getitem__, __setitem__, __delitem__, and __len__, as I read it), and
> make any additions or other changes to that contract a separate
> discussion. (But then, I'm personally happy with having this API
> specified in the ref and don't see a need to codify it with interfaces.)

For Py3k we certainly have the freedom (not to mention the obligation
:-) to rewrite the reference manual, so this isn't dogma for me.

> > Is it okay for a particular mapping implementation to allow
> > only one iterator to exist  (per instance) at any time,
> > like dbm files do?
>
> Sure, because the current spec doesn't deny it.
>
> If we have to have interfaces, I think we'd be far better off
> implementing the spec as-is first, and worry about changing the spec
> later (even if if only seems to be an "implied spec" at the moment). I
> didn't write the Ref Manual, of course, sir ;) so I can't assert its
> intent unequivocally, but as a consumer of it, I expect "it is
> recommended" to mean "not an interface requirement".

IMO the situation with respect to the current "de-facto" interfaces
and standard types is sufficiently messy that we will be *forced* to
change the spec. In fact I'd be disappointed if some good deep
thinking about the ontology of interfaces wouldn't lead us to some
welcome changes to the specs.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Tue Nov 21 22:34:50 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Tue, 21 Nov 2006 23:34:50 +0200
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
	<45632E22.2030705@solarsail.hcs.harvard.edu>
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
	<45633AB8.9010306@livinglogic.de>
	<ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
	<1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
	<45635F16.3060404@livinglogic.de>
	<bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
Message-ID: <1d85506f0611211334h1182c73cj6aac9d9b40bef99b@mail.gmail.com>

patch updated (hopefully for the last time. it got me all bored :)


On 11/21/06, Brett Cannon <brett at python.org> wrote:
>
>
>
> On 11/21/06, Walter D?rwald <walter at livinglogic.de> wrote:
> >
> > tomer filiba wrote:
> > > patch updated
> >
> > You're fast! ;)
> >
> > AFAICT now we have the following problem: If
> > Objects/typeobject.c::slot_nb_bool() falls back to using __len__(), the
> > return value of __len__() must be a bool.
>
>
> Why can't the fallback usage just pass the return value from __len__ to
> bool() (forget the C function name) and return that result?  It's just like
> doing::
>
>   def bool(obj):
>       try:
>           return obj.__bool__()
>       except AttributeError:
>           return bool(len(obj))
>
> -Brett
>
> Servus,
> >    Walter
> >
> > > On 11/21/06, *Guido van Rossum* <guido at python.org
> > > <mailto:guido at python.org>> wrote:
> > >
> > >     I think it would set a good example if __bool__ was *required* to
> > >     return exactly True or False and no other object, not even int
> > >     instances equal to 0 or 1. Surely that can't be much of a burden.
> > Feel
> > >     free to upload an improved patch.
> > >
> > >     On 11/21/06, Walter D?rwald <walter at livinglogic.de
> > >     <mailto: walter at livinglogic.de>> wrote:
> > >     > Guido van Rossum wrote:
> > >     > > On 11/21/06, Ivan Krsti? < krstic at solarsail.hcs.harvard.edu
> > >     <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
> > >     > >> Guido van Rossum wrote:
> > >     > >>> Can anyone help out evaluating this patch? If it has to wait
> >
> > >     for me
> > >     > >>> it's gonna be a looooooong wait...
> > >     > >> Looks fine to me functionally, although it seems to
> > >     gratuitously retab
> > >     > >> some code that used to be aligned with tabstop 8 to a tabstop
> > of 4.
> > >     > >
> > >     > > Thanks! I hope you meant "indentation level" instead of
> > "tabstop".
> > >     > > Tomer, can you fix the indentation to be whatever's prevailing
> > >     in the
> > >     > > file you're editing?
> > >     >
> > >     > Another question is whether __bool__() should be allowed to
> > return an
> > >     > int (or any other object), or if it *must* be a bool. (The patch
> >
> > >     > currently allows ints). What if __bool__() returns subclasses of
> > int,
> > >     > that overwrite __bool__() themself?
> > >     >
> > >     > Servus,
> > >     >    Walter
> > >     >
> > >     >
> > >
> > >
> > >     --
> > >     --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> > )
> > >
> > >
> >
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-3000/brett%40python.org
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061121/374bc5be/attachment.html 

From sluggoster at gmail.com  Tue Nov 21 22:37:57 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Tue, 21 Nov 2006 13:37:57 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>
References: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>
Message-ID: <6e9196d20611211337k6bb50b77y30f1921a79131637@mail.gmail.com>

On 11/21/06, Robert Brewer <fumanchu at amor.org> wrote:
> If we have to have interfaces, I think we'd be far better off
> implementing the spec as-is first, and worry about changing the spec
> later (even if if only seems to be an "implied spec" at the moment).

Well, except that we have a lot of real-world experience since the
spec was written.  I'm not opposed to doing it this way, I'm just
afraid the first will happen and the second won't, and then the
argument will be, "We can't do it because it's against the spec."
We're doing something new here; we're adding something to Python, not
taking anything away.  So this is a perfect time to ponder whether the
spec was optimal in all its details; e.g., whether requiring all
mappings to have a length is necessary or desirable.  The real
question is, who will be using these interfaces, what groups of
capabilities will they most likely be looking for, and are these
adequately addressed in the spec?

-- 
Mike Orr <sluggoster at gmail.com>

From rrr at ronadam.com  Tue Nov 21 22:56:03 2006
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 21 Nov 2006 15:56:03 -0600
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <ejvssq$2fi$1@sea.gmane.org>

Guido van Rossum wrote:
> On 11/20/06, Guido van Rossum <guido at python.org> wrote:
>> I'd like to have a discussion of abilities and interfaces but right
>> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
>> mostly isomorphic to Zope's notion of interfaces, but the new
>> terminology may make it possible to clarify certain distinctions
>> better, e.g. the distinction between a class that provides an ability
>> to its instances, and the abilities that an instance has.
>>
>> Feel free to riff on this theme here. I'll check in within 24 hours or so.
> 
> OK, I have some more time for this now. I'm hoping Andrew, Bill and
> Phillip will join the discussion here (instead of various other
> threads).


> I'd like to explore this some more before committing to anything.


Here's a thought:

For identifying abilities, it almost seems like instead of hard coding a library 
of this is that relationships, which will always be incomplete and possibly easy 
to fool, an ability might be described as a small (minimal) test that a 
interface can pass.  Registering an ability relationship could require running 
it against an ability test prior to use.

It doesn't have to be done just prior to use.  It could be done at import time, 
or even earlier with the result restored at application run time so it can be a 
one time cost for most applications.

This may make it more open ended and add a level of actual testing that may be 
useful for debugging as well.

Ron



From guido at python.org  Tue Nov 21 23:05:07 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 14:05:07 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ejvssq$2fi$1@sea.gmane.org>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ejvssq$2fi$1@sea.gmane.org>
Message-ID: <ca471dc20611211405i1fab5c4i3ada14b9bf87683a@mail.gmail.com>

I thought I addressed that (search my mail for "Antoine Pitrou"); I
expect that this will be too slow for use by the dispatch mechanism
for generic functions (or adapt, or anything else that makes use of
interfaces assuming the declared and actual interface match).

On 11/21/06, Ron Adam <rrr at ronadam.com> wrote:
> Guido van Rossum wrote:
> > On 11/20/06, Guido van Rossum <guido at python.org> wrote:
> >> I'd like to have a discussion of abilities and interfaces but right
> >> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
> >> mostly isomorphic to Zope's notion of interfaces, but the new
> >> terminology may make it possible to clarify certain distinctions
> >> better, e.g. the distinction between a class that provides an ability
> >> to its instances, and the abilities that an instance has.
> >>
> >> Feel free to riff on this theme here. I'll check in within 24 hours or so.
> >
> > OK, I have some more time for this now. I'm hoping Andrew, Bill and
> > Phillip will join the discussion here (instead of various other
> > threads).
>
>
> > I'd like to explore this some more before committing to anything.
>
>
> Here's a thought:
>
> For identifying abilities, it almost seems like instead of hard coding a library
> of this is that relationships, which will always be incomplete and possibly easy
> to fool, an ability might be described as a small (minimal) test that a
> interface can pass.  Registering an ability relationship could require running
> it against an ability test prior to use.
>
> It doesn't have to be done just prior to use.  It could be done at import time,
> or even earlier with the result restored at application run time so it can be a
> one time cost for most applications.
>
> This may make it more open ended and add a level of actual testing that may be
> useful for debugging as well.
>
> Ron
>
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gsakkis at rutgers.edu  Tue Nov 21 23:18:31 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Tue, 21 Nov 2006 17:18:31 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>

On 11/21/06, Guido van Rossum <guido at python.org> wrote:

> A. Preliminaries.
>
> There are some open issues that we should probably address before
> diving too deep into the specification of abilities or interfaces.
>
> 1. Abstract Base Classes (ABCs)?
>
> Are we sure that we have passed the station of ABCs, or should we
> revisit why those aren't sufficient for our purposes?
>
> Bill Janssen seems to be in favor of just using ABCs. Pro: less new
> infrastructure. Con: hard to add a new ABC on the fly to an existing
> 3rd party class (mucking with __bases__ is too much of a hack to
> seriously propose).
>
> I'd like to explore this some more before committing to anything.

I'd like that too, with some concrete examples if possible.  I'm still
unsure what interfaces buy us that ABCs (are these the same/equivalent
to mixins?) don't, especially with an adaptation/generic function
mechanism in place.

George

From fdrake at acm.org  Tue Nov 21 23:25:06 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Nov 2006 17:25:06 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
Message-ID: <200611211725.06887.fdrake@acm.org>

On Tuesday 21 November 2006 17:18, George Sakkis wrote:
 > I'd like that too, with some concrete examples if possible.  I'm still
 > unsure what interfaces buy us that ABCs (are these the same/equivalent
 > to mixins?) don't, especially with an adaptation/generic function
 > mechanism in place.

One thing that mixins/ABCs can't support that we use quite frequently in Zope 
3 applications is stamping additional interfaces on instances.  This is often 
done to influence adaptation.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From ark-mlist at att.net  Tue Nov 21 23:42:32 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue, 21 Nov 2006 17:42:32 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211327i3a5ce396q4a96f5f4d1c563ca@mail.gmail.com>
Message-ID: <006301c70dbe$553962c0$6402a8c0@arkdesktop>

> In fact I'd be disappointed if some good deep
> thinking about the ontology of interfaces wouldn't lead us to some
> welcome changes to the specs.

I don't know where this idea falls on the continuum between deep and silly,
but...

Suppose we stick for the moment with the notion that types (and objects of
those types) can have abilities, and that those abilities are represented by
other objects.  So an object of type Foo might have ability Bar, and we
would like to have some way of testing whether Foo has that ability.  In
other words, we would like a way to determine whether the author of Foo
claims that Foo has ability Bar.

Nothing new so far.  But it just occurred to me that part of an ability
(i.e. one of the methods of an ability) might be code that performs
compliance tests on an object that claims to have that ability.  This idea
is very sketchy, but it suggests that perhaps abilities could be parking
places for generic unit tests for interfaces, rather than tests of specific
types or pieces of code.



From pje at telecommunity.com  Tue Nov 21 23:59:40 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 21 Nov 2006 17:59:40 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <200611211725.06887.fdrake@acm.org>
References: <91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061121174806.01f19c50@sparrow.telecommunity.com>

At 05:25 PM 11/21/2006 -0500, Fred L. Drake, Jr. wrote:
>On Tuesday 21 November 2006 17:18, George Sakkis wrote:
>  > I'd like that too, with some concrete examples if possible.  I'm still
>  > unsure what interfaces buy us that ABCs (are these the same/equivalent
>  > to mixins?) don't, especially with an adaptation/generic function
>  > mechanism in place.
>
>One thing that mixins/ABCs can't support that we use quite frequently in Zope
>3 applications is stamping additional interfaces on instances.  This is often
>done to influence adaptation.

I was under the impression that Zope's actual use cases for 
instance-specific interfaces have mainly to do with *view* lookups and 
other types of lookups that don't really come into the current discussion 
scope as I understand it.  That is, I thought that instance interfaces were 
used primarily for "n-tuple adaptation" (the adaptation equivalent of 
multiple and/or predicate dispatch)?

If that's the case, then ISTM that as long as we make the core mechanism 
extensible, Zope's use cases should be handle-able.  I certainly wouldn't 
be in favor of a proposal that would marginalize Zope's paradigm, any more 
than I'd be in favor of one that blessed it.  :)  I just want a minimal 
core that blesses what the language and stdlib *already* do (special 
methods and single-dispatch __mro__ lookups), while still allowing 
"advanced" paradigms (Zope, RuleDispatch, etc.) to "play on a level playing 
field" with each other.


From guido at python.org  Tue Nov 21 23:59:36 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 14:59:36 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <006301c70dbe$553962c0$6402a8c0@arkdesktop>
References: <ca471dc20611211327i3a5ce396q4a96f5f4d1c563ca@mail.gmail.com>
	<006301c70dbe$553962c0$6402a8c0@arkdesktop>
Message-ID: <ca471dc20611211459u39e83917peb2ae57cabefc68d@mail.gmail.com>

On 11/21/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > In fact I'd be disappointed if some good deep
> > thinking about the ontology of interfaces wouldn't lead us to some
> > welcome changes to the specs.
>
> I don't know where this idea falls on the continuum between deep and silly,
> but...
>
> Suppose we stick for the moment with the notion that types (and objects of
> those types) can have abilities, and that those abilities are represented by
> other objects.  So an object of type Foo might have ability Bar, and we
> would like to have some way of testing whether Foo has that ability.  In
> other words, we would like a way to determine whether the author of Foo
> claims that Foo has ability Bar.
>
> Nothing new so far.  But it just occurred to me that part of an ability
> (i.e. one of the methods of an ability) might be code that performs
> compliance tests on an object that claims to have that ability.  This idea
> is very sketchy, but it suggests that perhaps abilities could be parking
> places for generic unit tests for interfaces, rather than tests of specific
> types or pieces of code.

I believe Ka-Ping once proposed something similar. This also jives
nicely with the "verify" functionality that I mentioned. However, I
don't know how easy it will be to write such compliance tests given
that typically the constructor signature is not part of the ability.
It may be more pragmatic to put the tests in a separate unit testing
module that can be subclassed by unit testing code for classes that
want specific claims to be verified.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tomerfiliba at gmail.com  Wed Nov 22 00:07:06 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Wed, 22 Nov 2006 01:07:06 +0200
Subject: [Python-3000] LINQ
Message-ID: <1d85506f0611211507n54f22ed0l9654f0d02442221d@mail.gmail.com>

i read the references fredrik posted several days back, and it got
me thinking: why not add a "co_ast" attribute to code objects?

i.e., since (virtually) all code objects are constructed from source
code by compilation, the compiler could just emit the AST as an
attribute to the generated code object.

yes, there's an issue of memory consumption, but bear with me
for a moment. also, maybe co_ast could be a lazy attribute, reading
the ast from the pyc file on demand.

having the AST as part of the code object would allow third-party tools,
like a LINQ library, to take a normal code object and walk it's AST,
generating the relevant SQL text or what not.

for those who are not familiar with LINQ, it's an integrated query
mechanism, not unlike list comprehensions:
from X where Y select Z --> [Z for Z in X if Y]

but (and that's a big but) since it's integrated into the language
through some special interfaces, classes can implement custom
"iterables", i.e., a database.

instead of going over the DB in a O(N) fashion (as list-comp does),
the LINQ expression can generate SQL text, send it to the DB,
and yield the results one at a time.

if we combine co_ast with the frame info (or the function's globals),
we can do really impressive things:

db = sql.connection(...)
MIN_AGE = 17
rows = query(firstname for firstname, age in db if age > MIN_AGE)

instead of the cumbersome and error-prone version of

db.query("select ... from ... where age > %s" % (MIN_AGE,))

we can extract the SELECT and WHERE lines from co_ast,
and take the value of MIN_AGE from the frame's globals,
thus generating SQL text from an expression, just like LINQ.

my intuition tells me there are many possible paths to explore
using this method...

================================

[Fredrik Lundh]
> (And frankly, it would be *wonderful* if someone could come up with a
> new proposal that actually enabled Python programmers to do things they
> *cannot* do today, instead of just rehashing old "let's move the sofa
> over there" threads.  How about doing something along the lines of
> LINQ's "give me this expression as an AST so I can optimize it myself"
> model or looking at COmega's parallelization/synchronization stuff or
> digging deeper into how PJE's RuleDispatch could be fit into Python or
> stealing some cool idea from some functional research language that I
> haven't even heard of.  I want to get *new* things when I upgrade from
> 2.X to 3.0, not just silly syntax tweaks that would only give me the
> ability to change two lines of Python code to one line of code plus a
> comment that explains what the heck that line is doing.  Let's do some
> *hard* stuff, for a change.)

but then again, if we go this far, why not just introduce real, first class
code objects? first class code is something LISP has got right
from the very beginning (although i hate LISP :)

i know metasyntax is considered a taboo here, but i thought i'd share
this idea i had for an extensible language:
* the compiler's AST could be altered at runtime (a la boo, only better)
* bytecode instructions could be added at runtime

to show it off a little:

with_statement = Statement(
    Literal("with"),
    Expression,
    Literal("as"),
    Name,
    Suite
)
# also define the code generating visitors for this node ...

import compiler
compiler.add_statement(with_statement)

# now add a new instruction for the WITH statement
import sys

def enter_with(stack, obj):
    obj = stack.pop()
    stack.push( obj.__enter__() )

bytecode = sys.add_instruction(enter_with, 0)

---

this way, you could really do from "__future__ import X"...
the language can be self-extending, which means we can try new
features out before adding them to the C implementation
(kinda like PyPy)

but i guess it's too far fetched for py3k...
maybe i should aim it at py4k :)

g'night.


-tomer

From krstic at solarsail.hcs.harvard.edu  Wed Nov 22 00:11:40 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Tue, 21 Nov 2006 18:11:40 -0500
Subject: [Python-3000] LINQ
In-Reply-To: <1d85506f0611211507n54f22ed0l9654f0d02442221d@mail.gmail.com>
References: <1d85506f0611211507n54f22ed0l9654f0d02442221d@mail.gmail.com>
Message-ID: <456387AC.5060808@solarsail.hcs.harvard.edu>

tomer filiba wrote:
> but then again, if we go this far, why not just introduce real, first
> class code objects?

Because Guido has pronounced it's not happening.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From fumanchu at amor.org  Wed Nov 22 00:24:09 2006
From: fumanchu at amor.org (Robert Brewer)
Date: Tue, 21 Nov 2006 15:24:09 -0800
Subject: [Python-3000] LINQ
Message-ID: <435DF58A933BA74397B42CDEB8145A8606EDBC5E@ex9.hostedexchange.local>

tomer filiba wrote:
> i read the references fredrik posted several days back, and it got
> me thinking: why not add a "co_ast" attribute to code objects?
> 
> i.e., since (virtually) all code objects are constructed from source
> code by compilation, the compiler could just emit the AST as an
> attribute to the generated code object.
> 
> yes, there's an issue of memory consumption, but bear with me
> for a moment. also, maybe co_ast could be a lazy attribute, reading
> the ast from the pyc file on demand.
> 
> having the AST as part of the code object would allow 
> third-party tools,
> like a LINQ library, to take a normal code object and walk it's AST,
> generating the relevant SQL text or what not.
> 
> for those who are not familiar with LINQ, it's an integrated query
> mechanism, not unlike list comprehensions:
> from X where Y select Z --> [Z for Z in X if Y]
> 
> but (and that's a big but) since it's integrated into the language
> through some special interfaces, classes can implement custom
> "iterables", i.e., a database.
> 
> instead of going over the DB in a O(N) fashion (as list-comp does),
> the LINQ expression can generate SQL text, send it to the DB,
> and yield the results one at a time.
> 
> if we combine co_ast with the frame info (or the function's globals),
> we can do really impressive things:
> 
> db = sql.connection(...)
> MIN_AGE = 17
> rows = query(firstname for firstname, age in db if age > MIN_AGE)
> 
> instead of the cumbersome and error-prone version of
> 
> db.query("select ... from ... where age > %s" % (MIN_AGE,))
> 
> we can extract the SELECT and WHERE lines from co_ast,
> and take the value of MIN_AGE from the frame's globals,
> thus generating SQL text from an expression, just like LINQ.

This is exactly what I do in Dejavu [1], only using the co_code instead,
and first-class expression objects [2] instead of full-blown code
objects. If co_ast were present, I'd probably use it instead; the only
reason I use bytecode now (Python 2.4 and under) is because it's faster.

> my intuition tells me there are many possible paths to explore
> using this method...

No intuition necessary; I've already explored them in Dejavu. ;) Read
through the short Querying doc [3] for a summary.

I have a proposal in for PyCon 2007 to explain the first-class
expression objects in detail, and how they enable LINQ-style code.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org

[1] http://projects.amor.org/dejavu
[2] http://projects.amor.org/dejavu/browser/trunk/logic.py
[3] http://projects.amor.org/docs/dejavu/managing.html#Querying

From nestornissen at gmail.com  Wed Nov 22 00:59:00 2006
From: nestornissen at gmail.com (Nestor)
Date: Tue, 21 Nov 2006 18:59:00 -0500
Subject: [Python-3000] Abilities / Interfaces
Message-ID: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>

Short question.

Do these aproaches allow to do something like the
java.util.RandomAccess interface that is just used as a marker?

From rrr at ronadam.com  Wed Nov 22 01:02:28 2006
From: rrr at ronadam.com (Ron Adam)
Date: Tue, 21 Nov 2006 18:02:28 -0600
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211405i1fab5c4i3ada14b9bf87683a@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>	<ejvssq$2fi$1@sea.gmane.org>
	<ca471dc20611211405i1fab5c4i3ada14b9bf87683a@mail.gmail.com>
Message-ID: <ek049s$qjs$1@sea.gmane.org>

Guido van Rossum wrote:
> I thought I addressed that (search my mail for "Antoine Pitrou"); I
> expect that this will be too slow for use by the dispatch mechanism
> for generic functions (or adapt, or anything else that makes use of
> interfaces assuming the declared and actual interface match).

I wasn't able to find the referenced message in this news group or on gmane, 
but Andrew Koenig's recent post in this thread is along the lines of what I was 
thinking.


> On 11/21/06, Ron Adam <rrr at ronadam.com> wrote:
>> Guido van Rossum wrote:
>>> On 11/20/06, Guido van Rossum <guido at python.org> wrote:
>>>> I'd like to have a discussion of abilities and interfaces but right
>>>> now I don't have time -- maybe tomorrow. I see Andrew's proposal as
>>>> mostly isomorphic to Zope's notion of interfaces, but the new
>>>> terminology may make it possible to clarify certain distinctions
>>>> better, e.g. the distinction between a class that provides an ability
>>>> to its instances, and the abilities that an instance has.
>>>>
>>>> Feel free to riff on this theme here. I'll check in within 24 hours or so.
>>> OK, I have some more time for this now. I'm hoping Andrew, Bill and
>>> Phillip will join the discussion here (instead of various other
>>> threads).
>>
>>> I'd like to explore this some more before committing to anything.
>>
>> Here's a thought:
>>
>> For identifying abilities, it almost seems like instead of hard coding a library
>> of this is that relationships, which will always be incomplete and possibly easy
>> to fool, an ability might be described as a small (minimal) test that a
>> interface can pass.


>> Registering an ability relationship could require running
>> it against an ability test prior to use.

To be a bit clearer, I meant here, that running an ability test-unit could be 
part of the pre-registration (or other setup) process and needn't be done at a 
function's or method's call time.


>> It doesn't have to be done just prior to use.  It could be done at import time,
>> or even earlier with the result restored at application run time so it can be a
>> one time cost for most applications.
>>
>> This may make it more open ended and add a level of actual testing that may be
>> useful for debugging as well.
>>
>> Ron




From guido at python.org  Wed Nov 22 01:06:52 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 16:06:52 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
Message-ID: <ca471dc20611211606w6dcd870es8c783f6a328df8df@mail.gmail.com>

On 11/21/06, Nestor <nestornissen at gmail.com> wrote:
> Do these aproaches allow to do something like the
> java.util.RandomAccess interface that is just used as a marker?

Zope interfaces certainly do. I don't know about Phillip's
counter-proposal. With ABCs it would seem that one has to plan this
ahead of time (i.e., at class definition time), which isn't always
reasonable.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Nov 22 02:12:58 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 17:12:58 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <1d85506f0611211334h1182c73cj6aac9d9b40bef99b@mail.gmail.com>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com>
	<ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com>
	<45632E22.2030705@solarsail.hcs.harvard.edu>
	<ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com>
	<45633AB8.9010306@livinglogic.de>
	<ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com>
	<1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com>
	<45635F16.3060404@livinglogic.de>
	<bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
	<1d85506f0611211334h1182c73cj6aac9d9b40bef99b@mail.gmail.com>
Message-ID: <ca471dc20611211712h6cc51392m39554b6dae8654fc@mail.gmail.com>

Someone with commit privs please give this one more look over and submit...

On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> patch updated (hopefully for the last time. it got me all bored :)
>
>
>
> On 11/21/06, Brett Cannon <brett at python.org > wrote:
> >
> >
> >
> > On 11/21/06, Walter D?rwald <walter at livinglogic.de> wrote:
> > > tomer filiba wrote:
> > > > patch updated
> > >
> > > You're fast! ;)
> > >
> > > AFAICT now we have the following problem: If
> > > Objects/typeobject.c::slot_nb_bool() falls back to
> using __len__(), the
> > > return value of __len__() must be a bool.
> >
> >
> > Why can't the fallback usage just pass the return value from __len__ to
> bool() (forget the C function name) and return that result?  It's just like
> doing::
> >
> >   def bool(obj):
> >       try:
> >           return obj.__bool__()
> >       except AttributeError:
> >           return bool(len(obj))
> >
> > -Brett
> >
> >
> > >
> > > Servus,
> > >    Walter
> > >
> > > > On 11/21/06, *Guido van Rossum* <guido at python.org
> > > > <mailto: guido at python.org>> wrote:
> > > >
> > > >     I think it would set a good example if __bool__ was *required* to
> > > >     return exactly True or False and no other object, not even int
> > > >     instances equal to 0 or 1. Surely that can't be much of a burden.
> Feel
> > > >     free to upload an improved patch.
> > > >
> > > >     On 11/21/06, Walter D?rwald <walter at livinglogic.de
> > > >     <mailto: walter at livinglogic.de>> wrote:
> > > >     > Guido van Rossum wrote:
> > > >     > > On 11/21/06, Ivan Krsti? <
> krstic at solarsail.hcs.harvard.edu
> > > >     <mailto:krstic at solarsail.hcs.harvard.edu>> wrote:
> > > >     > >> Guido van Rossum wrote:
> > > >     > >>> Can anyone help out evaluating this patch? If it has to wait
> > > >     for me
> > > >     > >>> it's gonna be a looooooong wait...
> > > >     > >> Looks fine to me functionally, although it seems to
> > > >     gratuitously retab
> > > >     > >> some code that used to be aligned with tabstop 8 to a tabstop
> of 4.
> > > >     > >
> > > >     > > Thanks! I hope you meant "indentation level" instead of
> "tabstop".
> > > >     > > Tomer, can you fix the indentation to be whatever's prevailing
> > > >     in the
> > > >     > > file you're editing?
> > > >     >
> > > >     > Another question is whether __bool__() should be allowed to
> return an
> > > >     > int (or any other object), or if it *must* be a bool. (The patch
> > > >     > currently allows ints). What if __bool__() returns subclasses of
> int,
> > > >     > that overwrite __bool__() themself?
> > > >     >
> > > >     > Servus,
> > > >     >    Walter
> > > >     >
> > > >     >
> > > >
> > > >
> > > >     --
> > > >     --Guido van Rossum (home page: http://www.python.org/~guido/ )
> > > >
> > > >
> > >
> > > _______________________________________________
> > > Python-3000 mailing list
> > > Python-3000 at python.org
> > > http://mail.python.org/mailman/listinfo/python-3000
> > > Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/brett%40python.org
> > >
> >
> >
>
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe:
> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Nov 22 02:20:54 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 21 Nov 2006 20:20:54 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211606w6dcd870es8c783f6a328df8df@mail.gmail.co
 m>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>

At 04:06 PM 11/21/2006 -0800, Guido van Rossum wrote:
>On 11/21/06, Nestor <nestornissen at gmail.com> wrote:
> > Do these aproaches allow to do something like the
> > java.util.RandomAccess interface that is just used as a marker?
>
>Zope interfaces certainly do. I don't know about Phillip's
>counter-proposal.

If we're basing GF's on types, then certainly marker mixins are feasible to 
*use*.  I only contend that mixin markers (and other pure interfaces devoid 
of execution effect) are unnecessary and shouldn't be encouraged, not that 
they are or should be impossible.


>  With ABCs it would seem that one has to plan this
>ahead of time (i.e., at class definition time), which isn't always
>reasonable.

For this reason, I would be against allowing *only* definition-time 
interface declarations.  A major benefit of both adaptation and generic 
functions is that they allow retrofitting or gluing of "oblivious" code; 
i.e., code that wasn't originally designed to work with the thing you're 
gluing it to.

Having only interfaces and introspection would also just mean that we are 
adding Java-like declarations and LBYLisms, without any improvements in 
expressive power or code clarity.  Declaring an interface or adding a mixin 
should *do* something, instead of simply being a comment in the form of 
code, that other code is able to read.  :)


From tjreedy at udel.edu  Wed Nov 22 02:36:32 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 21 Nov 2006 20:36:32 -0500
Subject: [Python-3000] __nonzero__ vs. __bool__
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com><ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com><1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com><ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com><45632E22.2030705@solarsail.hcs.harvard.edu><ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com><45633AB8.9010306@livinglogic.de><ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com><1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com><45635F16.3060404@livinglogic.de>
	<bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
Message-ID: <ek09iv$850$1@sea.gmane.org>


"Brett Cannon" <brett at python.org> wrote in message 
news:bbaeab100611211232r53de3c33p915bee7b49dbdf9a at mail.gmail.com...

Why can't the fallback usage just pass the return value from __len__ to 
bool() (forget the C function name) and return that result?  It's just like 
doing::

  def bool(obj):
      try:
          return obj.__bool__()
      except AttributeError:
          return bool(len(obj))
------------

If an object without __bool__ returned itself as its length, this would be 
an infinite loop, at least in this Python version.  Do we worry about 
something so crazy?

tjr




From nestornissen at gmail.com  Wed Nov 22 02:38:11 2006
From: nestornissen at gmail.com (Nestor)
Date: Tue, 21 Nov 2006 20:38:11 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
Message-ID: <4b57b0700611211738o3e952f35pcc2ac91dcf1160c3@mail.gmail.com>

On 11/21/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 04:06 PM 11/21/2006 -0800, Guido van Rossum wrote:
> >On 11/21/06, Nestor <nestornissen at gmail.com> wrote:
> > > Do these aproaches allow to do something like the
> > > java.util.RandomAccess interface that is just used as a marker?
> >
> >Zope interfaces certainly do. I don't know about Phillip's
> >counter-proposal.
>
> If we're basing GF's on types, then certainly marker mixins are feasible to
> *use*.  I only contend that mixin markers (and other pure interfaces devoid
> of execution effect) are unnecessary and shouldn't be encouraged, not that
> they are or should be impossible.
>
>
> >  With ABCs it would seem that one has to plan this
> >ahead of time (i.e., at class definition time), which isn't always
> >reasonable.
>
> For this reason, I would be against allowing *only* definition-time
> interface declarations.  A major benefit of both adaptation and generic
> functions is that they allow retrofitting or gluing of "oblivious" code;
> i.e., code that wasn't originally designed to work with the thing you're
> gluing it to.
>
> Having only interfaces and introspection would also just mean that we are
> adding Java-like declarations and LBYLisms, without any improvements in
> expressive power or code clarity.  Declaring an interface or adding a mixin
> should *do* something, instead of simply being a comment in the form of
> code, that other code is able to read.  :)
>
>
I agree that interfaces instead of functions adds another level of
indirection for a mechanism that most of the time is used to signify
the presence of one or two methods. If the functionality of
dispatching on characteristics other than methods is worth adding to
Python I don't feel experienced enough to answer.

From tdelaney at avaya.com  Wed Nov 22 03:05:26 2006
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Wed, 22 Nov 2006 13:05:26 +1100
Subject: [Python-3000] __nonzero__ vs. __bool__
Message-ID: <2773CAC687FD5F4689F526998C7E4E5FF1EBD1@au3010avexu1.global.avaya.com>

Terry Reedy wrote:

> If an object without __bool__ returned itself as its length, this
> would be 
> an infinite loop, at least in this Python version.  Do we worry about
> something so crazy?

Doesn't len() have a requirement that __len__ return an integer? If so,
perhaps it would be better if this were:

  def bool(obj):
      try:
          return obj.__bool__()
      except AttributeError:
          return len(obj) > 0

If I'm wrong (don't have the source available right now) then perhaps
len() *should* have this requirement.

Tim Delaney

From guido at python.org  Wed Nov 22 03:07:56 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 18:07:56 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
Message-ID: <ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com>

On 11/21/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:16 AM 11/21/2006 -0800, Guido van Rossum wrote:
> >Phillip then argues that he doesn't want to encourage introspectable
> >interfaces. I think others have use cases for those though.
>
> Examples?

I'll wait for others to speak up, but I'd think that automatically
generating appropriate docs for interfaces qualifies. Python in
general has a very proactive approach to introspection -- everythign
is introspectable unless it would violate safety.

> >  It seems
> >that Phillip's approach only hanges well together if everybody totally
> >adopts the generic functions religion; I think that's an unlikely
> >(since too radical) change of course for Python 3.0.
>
> Is it really radical?

Given some of your examples, I'd say so.

Python users are used to providing functionality in classes by
defining special methods.

Java users (and users of many other languages) are used to interfaces
that describe contracts (even if they *appear* only to be syntactic)
and ways to claim that a class fulfills the contract, as part of the
class definition syntax. Optional argument type declarations in Python
would be happy to use such a mechanism too.

These are both quite differently than binding a class and a specific
operation (e.g. flatten) through some call to something named
addmethod(). While the approaches are probably provably equivalent in
expressive power, IMO doing common things is a lot more intuitive
using interfaces. (Always accepting that "intuitive" isn't -- it just
means "familiar from a prevous context".)

While we're discussing familiarity, if you want to continue the
discussion, may I remind you that most people (including myself)
aren't familiar with RuleDispatch, and examples like

  addmethod(addmethod, adder_for_my_gf_type, my_gf_type)

or

  @addmethod.when_object(iter)

are downright off-putting? What on earth is "when_object(iter)"
supposed to mean? HCI indeed! :-)

> Consider the fact that single-dispatch generic functions are ubiquitous in
> the language and the stdlib, and have been pretty much forever.  They just
> vary in how they're implemented, and there's no One Obvious Way for
> developers to implement new ones, nor is there a standard way to add a
> method to an existing one.

Actually the standard way is to have a __special__ method and define
that in your class. We've recently discussed doing that for dir(), for
example (on python-dev, as it can safely go into 2.6).

> That's because some generic functions use if-isinstance checks (bad), while
> others use custom __special__ methods (not bad, but not great), registries
> (good), adaptation (okay), or generic function libraries.

__special__ methods are great for many purposes because they are well
understood. Registries are mostly good when there's a need for an
independent third party to add an ability; IMO this isn't as common in
most places as you seem to think.

> So, all I am proposing is that we:
>
>     * provide a standard GF implementation for the simple cases
>     * ...that is extensible to allow others to handle the more complex
> cases, by having a generic API for manipulating generic functions

I'm all for this. I think it would be more successful if it had
support for interfaces.

> So that the "one obvious way" to create new generics is to use the standard
> GF implementation unless you need something else.

I'd rather not try to teach people that they can't define a __len__
method any more but must update half a dozen generic functions in
order to create a new sequence type. Overriding __special__ methods
works fine for the most part.

The one exception I've seen is binary operations where some types are
too aggressive in rejecting unacceptable 'other' operands, raising
TypeError where they should return NotImplemented. I believe datetime
does this, but I also believe this is mostly to work around issues
with comparisons in 2.x that have been fixed in 3.0 (and datetime has
been taught to be less rude).

Sure, I see a use case for defining an operation that the class
author(s) did *not* foresee, but that's the exception rather than the
rule.

> And, there would be a
> standard way to add new methods to existing generic functions, perhaps
> using an addmethod() builtin or decorator, and it could be designed so that
> e.g.:
>
>      addmethod(iter, somefunc, sometype)

I don't actually understand what you want this example to mean. What
is 'iter' supposed to be? A generic function?

> would actually work by doing 'sometype.__iter__ = somefunc' under the
> hood.

How would addmethod know this?

> This allows us not to have to change any builtin generics that are
> based on special method names.  In other words, the mechanism isn't
> radical, nor is the idea of having generics.  It's just an improvement in
> ease-of-use: an HCI change, not a comp-sci change!

You can repeat that as many times as you want but that doesn't make it so.

At least one poster has already remarked that the head-exploding
capacity of generic functions is greater than that of metaclasses. I
think that's a gross exaggeration, but calling it "simple" and "easy
to use" is an exaggeration too. It's quite deep (otherwise we'd seen
it in Java already ;-).

> And, with this relatively simple mechanism, all the fancier forms of
> generic functions (or interfaces and adapters) can be implemented via user
> libraries.  (More on how, below.)
>
>
> >  It also doesn't
> >seem to work for abilities that are tied to an instance instead of to
> >a class as Zope allows (see below).
>
> Actually, it still allows for that, as long as the base generic function
> system is extensible.

How should it be extensible? Please explain this without making any
use of examples from RuleDispatch.

> For example, RuleDispatch generic functions can
> choose implementations based on conditions such as whether an object has a
> particular attribute.  Zope could easily add a generic function type that
> uses their instance-based interface stuff to do the same thing.

What's a generic function type? How does one create one?

> I am merely proposing that Python not provide this sort of
> introspection-oriented stuff in the core or stdlib, not that nobody should
> be *allowed* to have it.

And I think introspection is too important to leave out of the core.
People will want to introspect and find a way to do it by referring to
implementation details -- just like there's currently code in
inspect.py to reconstruct a function's signature from random
attributes of the code object, including flag bits.

> Indeed, I explicitly want the Python generic function API (e.g. addmethod,
> hasmethod, or whatever we call them) to *itself* be generic, so that users
> can create their own types that work with the Python-provided operations
> for generic function manipulation.  That is, I should be able to call:
>
>      addmethod(addmethod, adder_for_my_gf_type, my_gf_type)

OK, I take it back. My brain just exploded. it *is* worse than
metaclasses. Please find enclosed the bits of brain that I scraped of
my monitors. :-)

> So that others can then call:
>
>      # this calls adder_for_my_gf_type under the hood:
>      addmethod(some_gf, some_method, some_cls)
>
> where 'some_gf' is an instance of 'my_gf_type'.  This allows us not to keep
> the core or stdlib implementation of generic functions quite simple to
> handle the 80% (single dispatch) or 90% (concrete type-based multiple
> dispatch) with ease.  The final 10% (predicate dispatch, fancy interfaces,
> etc.) can then be done by outside libraries, since the use cases in that
> final 10% are more likely to vary than the base 80-90%.
>
> Using the 'simplegeneric' library from PyPI (easy_install simplegeneric):

Hey, another example referring to technology so advanced that for me
and other readers it is indistinguishable from magic. ;-)

>      from simplegeneric import generic
>
>      @generic
>      def addmethod(gf, method, cls):
>          """Add a method to a generic function"""
>          raise TypeError("Unknown generic function type", gf)
>
>      @addmethod.when_object(iter)
>      def add_to_iter(gf, method, cls):
>          # XXX should have more error checking here, e.g.
>          #     check for a local __iter__ first
>          cls.__iter__ = method
>
>      # ... similar declarations for other builtin generics
>
>      @addmethod.when_type(FunctionType)
>      def add_to_function(gf, method, cls):
>          if hasattr(gf, 'when_type'):
>              gf.when_type(cls)(method)
>          else:
>              raise TypeError("Not a generic function", gf)
>
> And there you are: an extensible way to add new extensible function types,
> while using the same API for all of them.

Poof. You disappear in a cloud of orange smoke.

> (If you need multiple or predicate dispatch or method combining, it's easy
> to have decorators that apply to the method itself, so that the same
> three-argument addmethod() can still be used.)
>
>
> >3. API design -- how do we spell the various concepts? E.g.
> >has_ability(x, A) asks whether object x has the ability A;
> >provides_ability(C, A) asks whether class C provides the ability A to
> >its instances. We could state that provides_ability(C, A) and
> >isinstance(x, C) implies has_ability(x, A).
>
> Even if we explicitly have some type of "ability" object, I'd like them to
> be defined in terms of generic functions, so that I could effectively say
> something like:
>
>      sequence_like = (iter & len)
>
> (or perhaps spelled out in some other fashion) to create an "ability"
> representing iter-ability and len-ability.

That's an interesting thought but it works with or without generic
functions -- it's just "ability algebra". I think I've played with
"type algebra" using similar ideas in a blog entry once.

> One of the biggest conceptual/modeling issues with Zope-style interfaces is
> that they don't allow this kind of fine-grained protocol combination.  See,
> for example, zope.interface's long history of trying to nail down various
> Python protocols through elaborate interface inheritance
> hierarchies.  Defining interfaces strictly in terms of operations
> eliminates this issue.

Not really, IMO. The underlying problem is that standard type
hierarchy defies capturing it in interfaces *BECAUSE INTERFACES DIDN'T
EXIST WHEN THEY WERE CREATED*.

> >- How does this interact with generic functions?
>
> The advanced abilities (per-instance/on-the-fly and multi-operation tests)
> will likely affect performance and/or simplicity of the default
> implementation.  A purely type-based system can be implemented efficiently,
> because declaring an interface can simultaneously add a concrete type
> registration for all affected generics.  (And registering an
> interface-linked method in a generic function can pull in all the concrete
> classes.)

This seems to argue *for* interfaces?

> Per-instance tests, however, increase code complexity.  Generally speaking,
> RuleDispatch does per-instance tests after class tests, but the way it gets
> reasonable performance is by building decision trees beforehand to manage
> the possible tests and avoid overlap.  If it actually had to call
> individual methods, or call a method more than once (ob.has_ability(Foo),
> ob.has_ability(Bar), etc.)  it would be considerably slower to select an
> option.

That would make more sense as Foo.implemented_by(ob), right?

> Also -- and this is the real kicker -- what do you do if more than one
> ability applies?  Now we have to have precedence rules for what's more
> specific.  The advantage of defining an ability as a set of one or more
> applicable generic functions is that precedence is comparatively
> straightforward: supersets take precedence over subsets, and overlaps are
> ambiguous.  You also have some possibility of being able to implement this
> by registration-time checks, without needing to build a dispatch tree at all.

Couldn't ability inheritance serve the same purpose?

> I don't know of anybody who really uses per-instance interface declarations
> except for Zope.  I used them for a little while with PEAK, and decided
> they were the wrong idea for me; it made more sense to adapt to an
> interface that provides the metadata, or to use a RuleDispatch function
> that just directly introspected for whatever was needed.  I can't think of
> anything in the core or stdlib that would even need to go that far.
>
> Again, my argument is that anything other than isinstance-based single and
> multiple-dispatch is too framework-dependent to be part of the
> language.  Simple things should be simple, complex things should be possible.

Sure, I can dig that. I'd like to have separate APIs for querying
instances and classes, but I don't mind if the built-in facilities
only let you define abilities for classes. As long as it's possible to
add an ability to a class dynamically (to deal with the problem of
pre-existing 3rd party classes that implement de-facto interfaces but
don't declare them).

> >Answering these and similar questions probably requires a
> >standardization committed. (Any volunteers?)
>
> Note that Zope has already tried for many years to do exactly this, and
> there is no indication that they have actually succeeded.

I tend to blame that on their inability to change the language and the
standard library though. Py3k has that option, and I'm not afraid to
exercise it if it would help.

> When I first
> mentioned PyProtocols on Python-Dev many years ago, Samuele Pedroni argued
> that individual operations (and groups thereof) should be the currency of
> interfaces, and after playing with them a bit, I agreed.  PyProtocols
> defines such interfaces as e.g.:
>
>      protocolForType(file, ['read', 'close'])
>
> to mean "file-like object with 'read' and 'close' methods".  If I define a
> class as supporting that protocol, and somebody wants to adapt to
> 'protocolForType(file, ["read"])', then my type will work.
>
> See
> http://peak.telecommunity.com/protocol_ref/protocols-generated-type.html
> for a more complete explanation, including how to get the system to
> actually check for the method names (so as not to require explicit
> declarations).
>
> I think Samuele's idea (i.e., this) works a lot better than trying to
> define a rigid interface hierarchy for builtin-protocols, although I'd just
> as soon be able to do something like:
>
>      file.read & file.close
>
> rather than having to use strings.  But all this assumes that we are
> providing a builtin way to spell abilities and introspect them, which I
> still think is overkill for what Python actually needs to provide as a base.

Well, I think it would be useful and I'm sure it would be popular.

I'm asking for a committee to lay this to rest by defining the one
true hierarchy for containers. Other languages have succeeded in doing
so (even Java).

> In short, I think it's adding interface introspection that's the radical
> move, where generic functions are just a bit of polish that brings a bit
> more order to what Python already has.  In contrast, interfaces are a
> foreign religion imported from other languages, while the roots of the
> generic function faith (len, iter, etc.) have already been in place since
> the dawn of time!  :)

Python has a long tradition of borrowing ideas from other languages,
and mostly very successful.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Nov 22 04:53:29 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 21 Nov 2006 19:53:29 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com> 
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <06Nov21.195340pst."58648"@synergy1.parc.xerox.com>

> Bill Janssen seems to be in favor of just using ABCs. Pro: less new
> infrastructure. Con: hard to add a new ABC on the fly to an existing
> 3rd party class (mucking with __bases__ is too much of a hack to
> seriously propose).

Don't get me wrong, I think having optional interface-based method
dispatch would be an additional goodness, if only for the
documentation aspect of it.  But yes, I think adding ABC to define the
interfaces, and using inheritance for aggregation, is the way to go
about it.  I think an additional benefit would be an increased
emphasis on mixins as a simple way to add functionality.  About that
con -- perhaps a new bit of machinery could be added to muck with
__bases__ in a principled fashion?  But isn't this a bit like the
argument about anonymous code blocks?  If someone wants to add a new
ABC to an existing third-party class, can't they just write

  class NewClass(ExistingThirdParty, NewABC):
    pass

and use an instance of that new class?  Then we get into the argument
about factories that only produce instances of ExistingThirdParty, and
so on...

> 2. Do we need interfaces if we have generic functions?

I don't mind having both, but generic functions (interface-based
method dispatch) is really only one aspect of the issue.

Bill

From janssen at parc.com  Wed Nov 22 04:55:10 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 21 Nov 2006 19:55:10 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>
References: <435DF58A933BA74397B42CDEB8145A8606E5D65D@ex9.hostedexchange.local>
Message-ID: <06Nov21.195515pst."58648"@synergy1.parc.xerox.com>

> If we have to have interfaces, I think we'd be far better off
> implementing the spec as-is first, and worry about changing the spec
> later (even if if only seems to be an "implied spec" at the moment).

Sure.  But we could factor the present spec into pieces.

Bill

From janssen at parc.com  Wed Nov 22 04:58:33 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 21 Nov 2006 19:58:33 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121174806.01f19c50@sparrow.telecommunity.com> 
References: <91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
	<5.1.1.6.0.20061121174806.01f19c50@sparrow.telecommunity.com>
Message-ID: <06Nov21.195840pst."58648"@synergy1.parc.xerox.com>

> I just want a minimal 
> core that blesses what the language and stdlib *already* do (special 
> methods and single-dispatch __mro__ lookups)

Well, this *is* Py3K.  We can fix things that the language doesn't do
but should.

Bill

From janssen at parc.com  Wed Nov 22 05:06:23 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 21 Nov 2006 20:06:23 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com> 
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com>
Message-ID: <06Nov21.200625pst."58648"@synergy1.parc.xerox.com>

> I'm asking for a committee to lay this to rest by defining the one
> true hierarchy for containers. Other languages have succeeded in doing
> so (even Java).

Well, I don't think Java has really done this yet.  But it's a good
idea.  Someone should try to re-describe the standard set of Python
built-in types (list, tuple, file, string, dict, etc.) as a set of
ABCs (not sure they have to be all that abstract, either), using
inheritance to describe abilities.  If it can't be done, that would be
a strike against ABC, I'd think :-).

Bill

From pje at telecommunity.com  Wed Nov 22 05:07:13 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 21 Nov 2006 23:07:13 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.co
 m>
References: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>

Before delving into any specific points, let me attempt to explain my 
assumptions, briefly, because I suspect that you and I do not mean the same 
thing by the words "generic function", and thus you perceive me to be 
proposing to change things that I am in fact proposing we *not* change.

To me, Python already has generic functions, including len(), iter(), 
getattr(), hasattr(), operator.getitem(), etc. etc.  Anything based on 
__special__ methods is in my view, *already* a "generic function", because 
it can take various types, and new types can be created that have 
specialized behavior.  The implementation of 'len()' is *open to extension 
for new types*, and therefore it is "generic".

That being the case, my position is that we should provide language support 
for defining other kinds of generic functions that are on an equal footing 
with the builtins in terms of language support.  __special__-based generics 
are adequate for quite a lot of purposes, registry-based ones are useful 
for many others, and advanced special cases have their uses as well.

This doesn't seem particularly radical to me, no more so than say, 
type/class unification.  And it's far less ambitious than type/class 
unification was, from an implementation point of view, because I don't 
propose to change the implementation of anything that already exists!

*Nothing* in this position relies upon RuleDispatch or even the simple 
multiple-dispatch generic function implementation you prototyped earlier 
this year.  Nor is there anything head-exploding about __special__ methods, 
as far as I am aware.  :)

So I'm at a loss to see any radicalness here.

What I have said that seems to be controversial, is the idea that we could 
do without interfaces.  However, we actually *already have interfaces based 
on generics*.

Consider 'iter()', for example, which can be viewed as adapting an object 
to the "iteration interface" and returning an object supporting iteration.

This is an *example* of what I am proposing: i.e., using generic operations 
to define interfaces.  And again, it's already supported by Python!

So how is this radical?  Everything I've stated up to this point in this 
message, is *exactly what we have today*!

The only things I propose to *add*, would be:

1. functions like 'addmethod()' and 'hasmethod()', themselves to be 
generics in the style of iter() or len().  Whether they are implemented 
using registries or __special__ methods is of no consequence.

2. a 'defop' syntax as shorthand for 'addmethod()', such that e.g.:

     import operator

     class Foo:
         defop iter(self):
             ...
         defop len(self):
             ...
         defop operator.getitem(self, key):
             ...

would produce exactly the same results as the same code using __special__ 
methods, except that it could be expanded to add methods for arbitrary 
*new* operations not defined by the language.

With these two things, Python would have enough foundation to allow any 
number of interface, adaptation, or generic function-based frameworks to 
run on top of it.  And, instead of each framework having to teach people 
new __special__ methods or provide different decorators, they could all 
simply say, "here are the generic functions or interfaces we use".

I submit that this is more than sufficient to cover the 80-90% of common 
use cases that don't need abstract interfaces at all, or which can simply 
use a generic function to stand in as an interface, the way 'iter' can 
stand in for the "iteration interface".

And, in the same way that 'def __iter__(self): return self' means "I 
implement iteration", doing "defop IFoo(self): return self" would mean, "I 
implement IFoo".  (Note, by the way, that common usage of interface 
adaptation in Python is already to call IFoo(anObject), so this is *also* 
no change to current usage!)

So, from my POV, this is actually a very modest proposal thus far.

Now I'll answer those specific points you brought up that aren't covered in 
the above.


At 06:07 PM 11/21/2006 -0800, Guido van Rossum wrote:
>are downright off-putting? What on earth is "when_object(iter)"
>supposed to mean? HCI indeed! :-)

Simplegeneric allows you to define methods for either types or 
instances.  The doc is at http://cheeseshop.python.org/pypi/simplegeneric


> > So, all I am proposing is that we:
> >
> >     * provide a standard GF implementation for the simple cases
> >     * ...that is extensible to allow others to handle the more complex
> > cases, by having a generic API for manipulating generic functions
>
>I'm all for this. I think it would be more successful if it had
>support for interfaces.

Sure.  See my explanation above for why I think that this *does* support 
interfaces.


> > So that the "one obvious way" to create new generics is to use the standard
> > GF implementation unless you need something else.
>
>I'd rather not try to teach people that they can't define a __len__
>method any more but must update half a dozen generic functions in
>order to create a new sequence type.

Implementing __len__ (or invoking 'addmethod(len,...)', or 'defop 
len(self):...') will still be sufficient to make a type len-able.  So code 
that uses len() will "just work" in that case.  Same thing for 
operator.getitem() or its [] shorthand syntax.


>Overriding __special__ methods works fine for the most part.

We can certainly keep the implementation of built-in generics based on 
__special__ methods.  My example code was intended to demonstrate that it's 
possible even in today's Python to have a uniform interface to defining 
such methods -- not that it's necessarily a desirable way to do it, given 
the absence of dedicated syntax like 'defop' for operator (operation?) 
overloading.

Even *with* a 'defop' syntax, I would still not propose we eliminate 
__special__ methods, as they are very fast when implemented as C slots.  I 
would just say that, like 'apply()', they would no longer be the optimum 
way to do something for which there is a syntactical shortcut.  Note that 
__special__ methods can be misspelled, and the resulting error can be hard 
to find.  Misspell a 'defop', and the failure is immediate.  (Of course, 
you could still defop the *wrong* function, but it's still an improvement.)


>Sure, I see a use case for defining an operation that the class
>author(s) did *not* foresee, but that's the exception rather than the
>rule.

Right, at least in the sense that team development is exceptional for 
Python.  In "enterprisey" and other team development scenarios (e.g. Zope, 
Chandler, etc.), it's valuable to have separation between domain model code 
and presentation code.  IIRC, the driving use cases for adaptation in Zope 
3 were to achieve this separation.

In other words, even if you *can* define the methods, that doesn't mean 
it's a good idea to, if you are trying to maximize reuse.  But I freely 
admit that Zope and PEAK at least are exceptional situations: Zope Corp. 
(and my group at Verio during the time I initially created PEAK), were both 
effectively consulting organizations gaining cost-effectiveness through 
reuse of framework code.  This is not exactly the most common use of 
Python...  unless of course you're in an "enterprise" shop or consulting 
organization.


> >      addmethod(iter, somefunc, sometype)
>
>I don't actually understand what you want this example to mean. What
>is 'iter' supposed to be? A generic function?

The iter builtin, as an example of the uniform addmethod() being applicable 
to any generic function, including existing builtin ones.  Although I guess 
you already know that from what you read beyond that point.


> > would actually work by doing 'sometype.__iter__ = somefunc' under the
> > hood.
>
>How would addmethod know this?

By addmethod itself being a generic function.


>At least one poster has already remarked that the head-exploding
>capacity of generic functions is greater than that of metaclasses. I
>think that's a gross exaggeration, but calling it "simple" and "easy
>to use" is an exaggeration too. It's quite deep (otherwise we'd seen
>it in Java already ;-).

We've already seen it in Python, actually.  Generic operations of this 
nature are at the very soul of the language; we simply provide syntactic 
shorthand for many of them, and builtins for the rest.

So, adding an explicit GF API seems more like type/class unification to me 
(or adding the callable __class__ hook), than a sea change in the nature of 
Python itself.  That is, a nice improvement in uniformity and 
user-extensibility, rather than any actual change.


>How should it be extensible? Please explain this without making any
>use of examples from RuleDispatch.

Implement the API using generic functions.  If you have an addmethod(), 
make it a generic.  If you have a 'hasmethod()', make it generic.

(Please note that 'generic' here refers to the extensibility of the 
function, not to a requirement that it be implemented via a registry!  I 
don't care if we have to give generic functions __addmethod__ and 
__hasmethod__ specials.  The point is merely that there should be *some* 
way to extend the core API.)


>What's a generic function type? How does one create one?

Implement a callable object that can be passed to addmethod(), hasmethod(), 
etc.


>OK, I take it back. My brain just exploded. it *is* worse than
>metaclasses. Please find enclosed the bits of brain that I scraped of
>my monitors. :-)

Ironically, it's *you* who gave me this idea, although I don't think you 
realized it at the time.  It was back when you were prototyping the 
multiple-dispatch implementation using a tuple of types -- something you 
said made me realize that we could have a generic API for manipulating 
generics, and thus allow hypothetical Zope and PEAK generics to live 
alongside Python-provided generics as equal citizens.  I then went away and 
made a proof of concept, before coming back to explode your head.


>Poof. You disappear in a cloud of orange smoke.

I guess now it's my turn not to understand what something means.  :)

I was merely trying to show that my idea is trivially 
implementable.  simplegeneric is only about 100 lines of Python added to 
what I wrote.  Throw in a 'defop' syntax that calls addmethod(), and we're 
done.

What I suppose is not obvious to anyone else from what I've written, is 
that this would:

1. allow competing type systems to coexist and even interoperate with a 
minimum of fuss
2. provide a *definition-time checkable* alternative to __special__ method 
names
3. ...while still allowing __specials__ to be used for fast lookup under 
the hood
4. allow for a "blessed" interface mechanism to either be chosen based on 
actual uses, OR
5. avoid the need for having a "blessed" interface mechanism at all, if GF 
introspection or adaptation suffices


>That's an interesting thought but it works with or without generic
>functions -- it's just "ability algebra". I think I've played with
>"type algebra" using similar ideas in a blog entry once.

Sure -- the difference is that I'm suggesting using the operation objects 
themselves as the basic unit of that algebra.

>Not really, IMO. The underlying problem is that standard type
>hierarchy defies capturing it in interfaces *BECAUSE INTERFACES DIDN'T
>EXIST WHEN THEY WERE CREATED*.

That's a reasonable hypothesis.  You might find it works out okay for 
things with few operations (like sequences and "mappings") but not so well 
for things with lots of utility methods ("list-like", "file-like", 
"dict-like" etc.).  I guess we'll see how it works out in practice.


> > The advanced abilities (per-instance/on-the-fly and multi-operation tests)
> > will likely affect performance and/or simplicity of the default
> > implementation.  A purely type-based system can be implemented efficiently,
> > because declaring an interface can simultaneously add a concrete type
> > registration for all affected generics.  (And registering an
> > interface-linked method in a generic function can pull in all the concrete
> > classes.)
>
>This seems to argue *for* interfaces?

I was pointing out that if you really need interfaces, you can have them, 
without needing to create a distinct interface object.  As you yourself 
pointed out in your blog, an "interface" can just be a generic function 
that returns an object supporting that interface, for a given input 
object.  In this sense, "iter" is an interface, because when you call it on 
an object, you get back an object that supports the "iter" interface.

Thus, I don't see much need to create a special notion of interfaces as a 
first-class citizen.  In the common case, a generic "adapting" function 
(like iter) is sufficient, even if the "interface" includes multiple 
methods (like __iter__ and next()).


> > Per-instance tests, however, increase code complexity.  Generally speaking,
> > RuleDispatch does per-instance tests after class tests, but the way it gets
> > reasonable performance is by building decision trees beforehand to manage
> > the possible tests and avoid overlap.  If it actually had to call
> > individual methods, or call a method more than once (ob.has_ability(Foo),
> > ob.has_ability(Bar), etc.)  it would be considerably slower to select an
> > option.
>
>That would make more sense as Foo.implemented_by(ob), right?

Um, I guess.  I was emphasizing the dynamic/per-instance aspect of the 
issue.  Whichever way you do it, unless you have a well-defined implication 
or inheritance hierarchy between interfaces, you have the problem of 
ambiguity between opaque interfaces.


> > Also -- and this is the real kicker -- what do you do if more than one
> > ability applies?  Now we have to have precedence rules for what's more
> > specific.  The advantage of defining an ability as a set of one or more
> > applicable generic functions is that precedence is comparatively
> > straightforward: supersets take precedence over subsets, and overlaps are
> > ambiguous.  You also have some possibility of being able to implement this
> > by registration-time checks, without needing to build a dispatch tree 
> at all.
>
>Couldn't ability inheritance serve the same purpose?

Sure.  The above was part of my argument against *dynamically-determined* 
abilities as a core feature.


> > In short, I think it's adding interface introspection that's the radical
> > move, where generic functions are just a bit of polish that brings a bit
> > more order to what Python already has.  In contrast, interfaces are a
> > foreign religion imported from other languages, while the roots of the
> > generic function faith (len, iter, etc.) have already been in place since
> > the dawn of time!  :)
>
>Python has a long tradition of borrowing ideas from other languages,
>and mostly very successful.

Of course -- and generic functions have a successful tradition in many 
other languages too, from Common Lisp to Haskell, that IIUC goes back much 
further than "interfaces" as such do.  I was pointing out that you had 
*already* borrowed this tradition many years ago, whether you realized it 
or not.  You just "hid" it by using __special__ method names instead of 
registries.  ;-)


From guido at python.org  Wed Nov 22 05:57:53 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Nov 2006 20:57:53 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
Message-ID: <ca471dc20611212057w7c46ad33h2dd9a9fd0fb2f481@mail.gmail.com>

Phillip, please shorten your posts. You're hogging all the bandwidth I
have for thinking about this. Please!

On 11/21/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> The only things I propose to *add*, would be:
>
> 1. functions like 'addmethod()' and 'hasmethod()', themselves to be
> generics in the style of iter() or len().  Whether they are implemented
> using registries or __special__ methods is of no consequence.

Please write a spec for these rather than assuming the reader already
knows what they are supposed to do.

If you could present simplified versions of these that were *not*
generic themselves first that would be a huge boon -- the
self-referentiality is what's head-exploding.

> 2. a 'defop' syntax as shorthand for 'addmethod()', such that e.g.:
>
>      import operator
>
>      class Foo:
>          defop iter(self):
>              ...
>          defop len(self):
>              ...
>          defop operator.getitem(self, key):
>              ...
>
> would produce exactly the same results as the same code using __special__
> methods,

New syntax is radical. Especialy when it's such superficial syntactic sugar.

> except that it could be expanded to add methods for arbitrary
> *new* operations not defined by the language.

I fail to see how we need new syntax to define new __special__
operations.  There are plenty of new __special__ methods being
introduced (e.g. by Zope, or by some library modules, like pickle, at
least originally) without new syntax.

Please spec this without new syntax first. The lack of new syntax is
not what blocks the proposal's acceptance.

> With these two things, Python would have enough foundation to allow any
> number of interface, adaptation, or generic function-based frameworks to
> run on top of it.  And, instead of each framework having to teach people
> new __special__ methods or provide different decorators, they could all
> simply say, "here are the generic functions or interfaces we use".

Again, that sounds mostly like a very thin layer of syntactic sugar.
Let's pretend we don't need that sugar -- at least not in the first
round.

> I submit that this is more than sufficient to cover the 80-90% of common
> use cases that don't need abstract interfaces at all, or which can simply
> use a generic function to stand in as an interface, the way 'iter' can
> stand in for the "iteration interface".

I'm still at a conceptual loss about this particular example. I fail
to see how 'iter' means 'iteration interface'. iter is a callable that
can do two very different things based on how it's called. Pretending
that it means "an object that has an __iter__ method" is too far a
stretch for me, since there is no way to find out that that is its
meaning except by knowing it. This is the problem I have with
non-introspectable interfaces. I don't understand why you find that an
advantage.

> And, in the same way that 'def __iter__(self): return self' means "I
> implement iteration", doing "defop IFoo(self): return self" would mean, "I
> implement IFoo".

Why the 'return self'? Did you mean '...'? Or did you really mean that
the 'return self' is part of the statement "I implement XXX"?

> (Note, by the way, that common usage of interface
> adaptation in Python is already to call IFoo(anObject), so this is *also*
> no change to current usage!)

You're taking some liberties by calling that current usage. What does
"interface adaptation" even mean? It's not part of the current Python
vocabulary.

> So, from my POV, this is actually a very modest proposal thus far.
>
> Now I'll answer those specific points you brought up that aren't covered in
> the above.

Maybe I'll have time to read and respond to those tomorrow.
>
> At 06:07 PM 11/21/2006 -0800, Guido van Rossum wrote:
> >are downright off-putting? What on earth is "when_object(iter)"
> >supposed to mean? HCI indeed! :-)
>
> Simplegeneric allows you to define methods for either types or
> instances.  The doc is at http://cheeseshop.python.org/pypi/simplegeneric
>
>
> > > So, all I am proposing is that we:
> > >
> > >     * provide a standard GF implementation for the simple cases
> > >     * ...that is extensible to allow others to handle the more complex
> > > cases, by having a generic API for manipulating generic functions
> >
> >I'm all for this. I think it would be more successful if it had
> >support for interfaces.
>
> Sure.  See my explanation above for why I think that this *does* support
> interfaces.
>
>
> > > So that the "one obvious way" to create new generics is to use the standard
> > > GF implementation unless you need something else.
> >
> >I'd rather not try to teach people that they can't define a __len__
> >method any more but must update half a dozen generic functions in
> >order to create a new sequence type.
>
> Implementing __len__ (or invoking 'addmethod(len,...)', or 'defop
> len(self):...') will still be sufficient to make a type len-able.  So code
> that uses len() will "just work" in that case.  Same thing for
> operator.getitem() or its [] shorthand syntax.
>
>
> >Overriding __special__ methods works fine for the most part.
>
> We can certainly keep the implementation of built-in generics based on
> __special__ methods.  My example code was intended to demonstrate that it's
> possible even in today's Python to have a uniform interface to defining
> such methods -- not that it's necessarily a desirable way to do it, given
> the absence of dedicated syntax like 'defop' for operator (operation?)
> overloading.
>
> Even *with* a 'defop' syntax, I would still not propose we eliminate
> __special__ methods, as they are very fast when implemented as C slots.  I
> would just say that, like 'apply()', they would no longer be the optimum
> way to do something for which there is a syntactical shortcut.  Note that
> __special__ methods can be misspelled, and the resulting error can be hard
> to find.  Misspell a 'defop', and the failure is immediate.  (Of course,
> you could still defop the *wrong* function, but it's still an improvement.)
>
>
> >Sure, I see a use case for defining an operation that the class
> >author(s) did *not* foresee, but that's the exception rather than the
> >rule.
>
> Right, at least in the sense that team development is exceptional for
> Python.  In "enterprisey" and other team development scenarios (e.g. Zope,
> Chandler, etc.), it's valuable to have separation between domain model code
> and presentation code.  IIRC, the driving use cases for adaptation in Zope
> 3 were to achieve this separation.
>
> In other words, even if you *can* define the methods, that doesn't mean
> it's a good idea to, if you are trying to maximize reuse.  But I freely
> admit that Zope and PEAK at least are exceptional situations: Zope Corp.
> (and my group at Verio during the time I initially created PEAK), were both
> effectively consulting organizations gaining cost-effectiveness through
> reuse of framework code.  This is not exactly the most common use of
> Python...  unless of course you're in an "enterprise" shop or consulting
> organization.
>
>
> > >      addmethod(iter, somefunc, sometype)
> >
> >I don't actually understand what you want this example to mean. What
> >is 'iter' supposed to be? A generic function?
>
> The iter builtin, as an example of the uniform addmethod() being applicable
> to any generic function, including existing builtin ones.  Although I guess
> you already know that from what you read beyond that point.
>
>
> > > would actually work by doing 'sometype.__iter__ = somefunc' under the
> > > hood.
> >
> >How would addmethod know this?
>
> By addmethod itself being a generic function.
>
>
> >At least one poster has already remarked that the head-exploding
> >capacity of generic functions is greater than that of metaclasses. I
> >think that's a gross exaggeration, but calling it "simple" and "easy
> >to use" is an exaggeration too. It's quite deep (otherwise we'd seen
> >it in Java already ;-).
>
> We've already seen it in Python, actually.  Generic operations of this
> nature are at the very soul of the language; we simply provide syntactic
> shorthand for many of them, and builtins for the rest.
>
> So, adding an explicit GF API seems more like type/class unification to me
> (or adding the callable __class__ hook), than a sea change in the nature of
> Python itself.  That is, a nice improvement in uniformity and
> user-extensibility, rather than any actual change.
>
>
> >How should it be extensible? Please explain this without making any
> >use of examples from RuleDispatch.
>
> Implement the API using generic functions.  If you have an addmethod(),
> make it a generic.  If you have a 'hasmethod()', make it generic.
>
> (Please note that 'generic' here refers to the extensibility of the
> function, not to a requirement that it be implemented via a registry!  I
> don't care if we have to give generic functions __addmethod__ and
> __hasmethod__ specials.  The point is merely that there should be *some*
> way to extend the core API.)
>
>
> >What's a generic function type? How does one create one?
>
> Implement a callable object that can be passed to addmethod(), hasmethod(),
> etc.
>
>
> >OK, I take it back. My brain just exploded. it *is* worse than
> >metaclasses. Please find enclosed the bits of brain that I scraped of
> >my monitors. :-)
>
> Ironically, it's *you* who gave me this idea, although I don't think you
> realized it at the time.  It was back when you were prototyping the
> multiple-dispatch implementation using a tuple of types -- something you
> said made me realize that we could have a generic API for manipulating
> generics, and thus allow hypothetical Zope and PEAK generics to live
> alongside Python-provided generics as equal citizens.  I then went away and
> made a proof of concept, before coming back to explode your head.
>
>
> >Poof. You disappear in a cloud of orange smoke.
>
> I guess now it's my turn not to understand what something means.  :)
>
> I was merely trying to show that my idea is trivially
> implementable.  simplegeneric is only about 100 lines of Python added to
> what I wrote.  Throw in a 'defop' syntax that calls addmethod(), and we're
> done.
>
> What I suppose is not obvious to anyone else from what I've written, is
> that this would:
>
> 1. allow competing type systems to coexist and even interoperate with a
> minimum of fuss
> 2. provide a *definition-time checkable* alternative to __special__ method
> names
> 3. ...while still allowing __specials__ to be used for fast lookup under
> the hood
> 4. allow for a "blessed" interface mechanism to either be chosen based on
> actual uses, OR
> 5. avoid the need for having a "blessed" interface mechanism at all, if GF
> introspection or adaptation suffices
>
>
> >That's an interesting thought but it works with or without generic
> >functions -- it's just "ability algebra". I think I've played with
> >"type algebra" using similar ideas in a blog entry once.
>
> Sure -- the difference is that I'm suggesting using the operation objects
> themselves as the basic unit of that algebra.
>
> >Not really, IMO. The underlying problem is that standard type
> >hierarchy defies capturing it in interfaces *BECAUSE INTERFACES DIDN'T
> >EXIST WHEN THEY WERE CREATED*.
>
> That's a reasonable hypothesis.  You might find it works out okay for
> things with few operations (like sequences and "mappings") but not so well
> for things with lots of utility methods ("list-like", "file-like",
> "dict-like" etc.).  I guess we'll see how it works out in practice.
>
>
> > > The advanced abilities (per-instance/on-the-fly and multi-operation tests)
> > > will likely affect performance and/or simplicity of the default
> > > implementation.  A purely type-based system can be implemented efficiently,
> > > because declaring an interface can simultaneously add a concrete type
> > > registration for all affected generics.  (And registering an
> > > interface-linked method in a generic function can pull in all the concrete
> > > classes.)
> >
> >This seems to argue *for* interfaces?
>
> I was pointing out that if you really need interfaces, you can have them,
> without needing to create a distinct interface object.  As you yourself
> pointed out in your blog, an "interface" can just be a generic function
> that returns an object supporting that interface, for a given input
> object.  In this sense, "iter" is an interface, because when you call it on
> an object, you get back an object that supports the "iter" interface.
>
> Thus, I don't see much need to create a special notion of interfaces as a
> first-class citizen.  In the common case, a generic "adapting" function
> (like iter) is sufficient, even if the "interface" includes multiple
> methods (like __iter__ and next()).
>
>
> > > Per-instance tests, however, increase code complexity.  Generally speaking,
> > > RuleDispatch does per-instance tests after class tests, but the way it gets
> > > reasonable performance is by building decision trees beforehand to manage
> > > the possible tests and avoid overlap.  If it actually had to call
> > > individual methods, or call a method more than once (ob.has_ability(Foo),
> > > ob.has_ability(Bar), etc.)  it would be considerably slower to select an
> > > option.
> >
> >That would make more sense as Foo.implemented_by(ob), right?
>
> Um, I guess.  I was emphasizing the dynamic/per-instance aspect of the
> issue.  Whichever way you do it, unless you have a well-defined implication
> or inheritance hierarchy between interfaces, you have the problem of
> ambiguity between opaque interfaces.
>
>
> > > Also -- and this is the real kicker -- what do you do if more than one
> > > ability applies?  Now we have to have precedence rules for what's more
> > > specific.  The advantage of defining an ability as a set of one or more
> > > applicable generic functions is that precedence is comparatively
> > > straightforward: supersets take precedence over subsets, and overlaps are
> > > ambiguous.  You also have some possibility of being able to implement this
> > > by registration-time checks, without needing to build a dispatch tree
> > at all.
> >
> >Couldn't ability inheritance serve the same purpose?
>
> Sure.  The above was part of my argument against *dynamically-determined*
> abilities as a core feature.
>
>
> > > In short, I think it's adding interface introspection that's the radical
> > > move, where generic functions are just a bit of polish that brings a bit
> > > more order to what Python already has.  In contrast, interfaces are a
> > > foreign religion imported from other languages, while the roots of the
> > > generic function faith (len, iter, etc.) have already been in place since
> > > the dawn of time!  :)
> >
> >Python has a long tradition of borrowing ideas from other languages,
> >and mostly very successful.
>
> Of course -- and generic functions have a successful tradition in many
> other languages too, from Common Lisp to Haskell, that IIUC goes back much
> further than "interfaces" as such do.  I was pointing out that you had
> *already* borrowed this tradition many years ago, whether you realized it
> or not.  You just "hid" it by using __special__ method names instead of
> registries.  ;-)
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From talin at acm.org  Wed Nov 22 07:15:56 2006
From: talin at acm.org (Talin)
Date: Tue, 21 Nov 2006 22:15:56 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <4563EB1C.5040408@acm.org>

Guido van Rossum wrote:
> 1. Naming and general ideas. Should we call these Abilities or
> Interfaces? Abilities, the term proposed by Andrew Koenig, have less
> baggage, but possibly they're just an isomorphic relabeling of
> interfaces. From Zope comes the important question: Do we want to
> discuss the interfaces/abilities of an instance separately from those
> provided by its class? Zope's answer is Yes, reminds Jean-Paul
> Calderone, and I tend to like this (having been exposed to it in the
> past).

In C++ template-land these are called "concepts". Thus, a vector 
embodies the 'sequential container' concept.

Other terms that one can use are: features, affordances, 
characteristics, facet, quality, etc.

Both 'ability' and 'interface' imply (to me, anyway) that the class 
being inspected is an actor, that it 'does something' rather than being 
operated on.

I kind of like 'feature', although that too has baggage. (I'm thinking 
of the word 'feature' in the sense of having some property or 
characteristic, rather than the sense of having positive benefit.)

-- Talin


From pje at telecommunity.com  Wed Nov 22 08:24:35 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 02:24:35 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611212057w7c46ad33h2dd9a9fd0fb2f481@mail.gmail.co
 m>
References: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>

At 08:57 PM 11/21/2006 -0800, Guido van Rossum wrote:
>Phillip, please shorten your posts. You're hogging all the bandwidth I
>have for thinking about this. Please!

Er, that *was* the shortened version; I actually spent about an hour *just* 
on shortening it.

I tried to write a short reply to your message, but with little 
success.  So, I'll just rate limit myself by refraining from posting for a 
couple of days.  Will that work?


From pje at telecommunity.com  Wed Nov 22 08:26:43 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 02:26:43 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov21.195340pst."58648"@synergy1.parc.xerox.com>
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>

At 07:53 PM 11/21/2006 -0800, Bill Janssen wrote:
>I don't mind having both, but generic functions (interface-based
>method dispatch) is really only one aspect of the issue.

How so?


From g.brandl at gmx.net  Wed Nov 22 09:36:09 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 22 Nov 2006 09:36:09 +0100
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <ek09iv$850$1@sea.gmane.org>
References: <1d85506f0611201317m3f1697d7rd9d3e2fff1124209@mail.gmail.com><ca471dc20611201721t39a975a6k83a99793ccd73605@mail.gmail.com><1d85506f0611210359m7625c01ava3b2530465d4c8a4@mail.gmail.com><ca471dc20611210809w464d52bdhcf2d11b2e8ce86c3@mail.gmail.com><45632E22.2030705@solarsail.hcs.harvard.edu><ca471dc20611210935l7b1b4333s99c7cb55d3c694a2@mail.gmail.com><45633AB8.9010306@livinglogic.de><ca471dc20611210959n5057b444qb43e2f0b6b3131c4@mail.gmail.com><1d85506f0611211029i3969a91cw1ad621d15d4374e7@mail.gmail.com><45635F16.3060404@livinglogic.de>	<bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
	<ek09iv$850$1@sea.gmane.org>
Message-ID: <ek125p$36l$1@sea.gmane.org>

Terry Reedy schrieb:
> "Brett Cannon" <brett at python.org> wrote in message 
> news:bbaeab100611211232r53de3c33p915bee7b49dbdf9a at mail.gmail.com...
> 
> Why can't the fallback usage just pass the return value from __len__ to 
> bool() (forget the C function name) and return that result?  It's just like 
> doing::
> 
>   def bool(obj):
>       try:
>           return obj.__bool__()
>       except AttributeError:
>           return bool(len(obj))
> ------------
> 
> If an object without __bool__ returned itself as its length, this would be 
> an infinite loop, at least in this Python version.  Do we worry about 
> something so crazy?

The length would have to be an integer, and this would have to be checked.

Georg


From walter at livinglogic.de  Wed Nov 22 09:45:46 2006
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 22 Nov 2006 09:45:46 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com>
Message-ID: <45640E3A.7010307@livinglogic.de>

Guido van Rossum wrote:
> On 11/21/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> [...]
>> That's because some generic functions use if-isinstance checks (bad), while
>> others use custom __special__ methods (not bad, but not great), registries
>> (good), adaptation (okay), or generic function libraries.
> 
> __special__ methods are great for many purposes because they are well
> understood. Registries are mostly good when there's a need for an
> independent third party to add an ability; IMO this isn't as common in
> most places as you seem to think.

Another effect of __special__ methods is that they divide the class 
namespace into two parts: The __special__ names are those that have to 
be implemented to support core Python interfaces, and the methods 
without underscores are those that implement the "main aspect" of the 
class. However there might be other aspects worth supporting (fancy 
repr, pickling, GUI output, etc.). The methods needed for that support 
either have to share the special namespace or the main namespace.

In Java there are interfaces, but there's only one namespace in which 
support for an interface can be implemented, so method names have to be 
fairly long to minimize collisions.

With generic functions that problem goes away. It's no longer the name 
of the method that links the implementation to the protocol, but an 
explicit registration of the implementing function.

> [...]

Servus,
    Walter

From p.f.moore at gmail.com  Wed Nov 22 10:22:11 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 22 Nov 2006 09:22:11 +0000
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
Message-ID: <79990c6b0611220122y49d27d67hf719670119d715c8@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:57 PM 11/21/2006 -0800, Guido van Rossum wrote:
> >Phillip, please shorten your posts. You're hogging all the bandwidth I
> >have for thinking about this. Please!
>
> Er, that *was* the shortened version; I actually spent about an hour *just*
> on shortening it.
>
> I tried to write a short reply to your message, but with little
> success.  So, I'll just rate limit myself by refraining from posting for a
> couple of days.  Will that work?

I'm also getting overloaded by the flood of information in the longer
postings in this thread, compounded by the relatively abstract nature
of the discussion.

Phillip, as you seem to be of the view that your proposal is
relatively small in scope, but wider in implications, maybe you could
write a (pre-)PEP? Formally describing the problem, why the current
solution(s) aren't sufficient, the precise implementation details and
some simple but complete examples, may help people like me understand
the scope of what you're proposing. But *please* keep it concrete -
it's the abstract descriptions that are losing me!

Paul.

From gustavo at niemeyer.net  Wed Nov 22 14:16:56 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Wed, 22 Nov 2006 11:16:56 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
Message-ID: <20061122131656.GA8659@niemeyer.net>

Hello Phillip,

> If we're basing GF's on types, then certainly marker mixins are
> feasible to *use*.  I only contend that mixin markers (and other pure
> interfaces devoid of execution effect) are unnecessary and shouldn't
> be encouraged, not that they are or should be impossible.

I don't see a reason to discourage them.  It seems like a useful
pattern used in a few places in Zope 3, genuinely providing
additional value to the adaptation mechanism.

> For this reason, I would be against allowing *only* definition-time
> interface declarations.  A major benefit of both adaptation and
> generic functions is that they allow retrofitting or gluing of
> "oblivious" code; i.e., code that wasn't originally designed to work
> with the thing you're gluing it to.

Yes, late binding of interfaces to classes and instances is also of
real usage when using interface as markers, as it allows changing the
adaptation effect on uncontrolled classes and specific object instances.

> Having only interfaces and introspection would also just mean that we
> are adding Java-like declarations and LBYLisms, without any
> improvements in expressive power or code clarity.  Declaring an
> interface or adding a mixin should *do* something, instead of simply
> being a comment in the form of code, that other code is able to read.
> :)

I've lost you here.  As I understand interfaces, their most important
feature is indeed being a "comment in the form of code", and I hope
they continue to be that way.  What you do with them, being an
adaptation system like the Zope 3 component architecture, or even
generic functions with interface capabilities, is up to the framework
using them.  Limiting a standard interface package to a given behavior
without these introspection features will only ensure that more
variants continue to exist and multiply.  Having a standard interface
package will of course allow the interpreter itself to do decisions
about them ("doing something"), but it would be just one more user
of their intrinsic expressiveness.

-- 
Gustavo Niemeyer
http://niemeyer.net

From solipsis at pitrou.net  Wed Nov 22 14:23:39 2006
From: solipsis at pitrou.net (Antoine)
Date: Wed, 22 Nov 2006 14:23:39 +0100 (CET)
Subject: [Python-3000] defop ?
In-Reply-To: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
Message-ID: <55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>

> 2. a 'defop' syntax as shorthand for 'addmethod()', such that e.g.:
>
>      import operator
>
>      class Foo:
>          defop iter(self):
>              ...
>          defop len(self):
>              ...
>          defop operator.getitem(self, key):
>              ...

Is "defop" useful?

Why wouldn't it be possible to enable special method lookup with this kind
of syntax:

iter = generic_function()
iter.lookup_special_method("__iter__")

And then the classical:

class Foo:
    def __iter__(self):
        pass




From ncoghlan at gmail.com  Wed Nov 22 14:55:27 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Nov 2006 23:55:27 +1000
Subject: [Python-3000] defop ?
In-Reply-To: <55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
References: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
Message-ID: <456456CF.8000005@gmail.com>

Antoine wrote:
> Is "defop" useful?
> 
> Why wouldn't it be possible to enable special method lookup with this kind
> of syntax:
> 
> iter = generic_function()
> iter.lookup_special_method("__iter__")
> 
> And then the classical:
> 
> class Foo:
>     def __iter__(self):
>         pass

A major benefit of the defop syntax lies in the fact that the full Python 
module namespace is available to you for disambiguation of which operation you 
are referring to. So if module A defines an overloaded operation "snap" and 
module B define an overloaded operation "snap", that's OK, because "defop 
A.snap" and "defop B.snap" can be used to make it clear which one you mean.

The magic method namespace, on the other hand, is completely flat: there can 
be only one __snap__ magic method in a given Python application. If two 
different modules both want to use the same magic method name, you're out of 
luck because you can't use the module namespace to disambiguate which one you 
actually want to support (heck, you might even want to support both of them).

A second benefit of defop is that if you have a reference to a generic 
function then you can overload it. With magic methods, you can only overload 
it if you know the appropriate incantation (see the thread about correcting 
the fact that the incantation to overload bool in 2.x is __nonzero__ instead 
of __bool__ - with defop, it could be written simply as "defop bool(self):", 
and issue wouldn't need to rise).

Finally, the defop syntax means a typo will likely result in an exception when 
the class definition is executed instead of at some random later time. For 
example, the typo "def __getiten__(self, key):" will go completely undetected 
when a class definition is executed, but "defop operator.getiten(self, key):" 
would fail with an AttributeError.

That said, it doesn't *have* to be new syntax - an appropriate function 
decorator and a class decorator or metaclass to register the appropriate 
overloads also support this. defop just makes it easier to read and write.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From benji at benjiyork.com  Wed Nov 22 14:36:18 2006
From: benji at benjiyork.com (Benji York)
Date: Wed, 22 Nov 2006 08:36:18 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <45645252.6010101@benjiyork.com>

Guido van Rossum wrote:
> 1. Naming and general ideas. Should we call these Abilities or
> Interfaces?

I'll try to make an argument for "Interface" over "Ability" using 
examples from Zope.

It seems to me that not all interfaces coincide with something the 
object can _do_.  Some speak to what can be done _to_ an object:

class IAttributeAnnotatable(IAnnotatable):
     """Marker indicating that annotations can be stored on an attribute.

     This is a marker interface giving permission for an `IAnnotations`
     adapter to store data in an attribute named `__annotations__`.

     """

Some give describe how the object will _act_: Persistent.  Some are 
application-level statements about how what is allowed to be _done_with_ 
the object: ISharable (the user can "share" this object with other users 
in various ways; read, write, etc.).

These (especially the marker interfaces) don't seem to be statements 
about what the objects can do (abilities), but about other 
characteristics, therefore a more general name like "interface" seems 
appropriate.
--
Benji York

From ark-mlist at att.net  Wed Nov 22 15:51:27 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 09:51:27 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211459u39e83917peb2ae57cabefc68d@mail.gmail.com>
Message-ID: <002101c70e45$b0888de0$6402a8c0@arkdesktop>

> I believe Ka-Ping once proposed something similar. This also jives
> nicely with the "verify" functionality that I mentioned. However, I
> don't know how easy it will be to write such compliance tests given
> that typically the constructor signature is not part of the ability.
> It may be more pragmatic to put the tests in a separate unit testing
> module that can be subclassed by unit testing code for classes that
> want specific claims to be verified.

Well, suppose we invent an ability that indicates compliance with a
unit-test protocol.  Objects that have this ability have a method or methods
that can be used to test the object, to the extend that the author wants to
render it testable.

Then the verify functionality could, as a matter of convention, self-test
all objects that claim to be auditable, without the verifier having to know
the details.
 


From ark-mlist at att.net  Wed Nov 22 15:55:57 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 09:55:57 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <4563EB1C.5040408@acm.org>
Message-ID: <002201c70e46$5149f750$6402a8c0@arkdesktop>

> Both 'ability' and 'interface' imply (to me, anyway) that the class
> being inspected is an actor, that it 'does something' rather than being
> operated on.

I chose 'ability' because to me it doesn't require that the class being
inspected is active by itself.  For example, it feels natural to me to speak
of a class as "having the totally ordered ability".

The problem I have with 'feature' is that it's already in widespread use
without a formal meaning.



From ark-mlist at att.net  Wed Nov 22 15:59:14 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 09:59:14 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <45645252.6010101@benjiyork.com>
Message-ID: <002301c70e46$c7473260$6402a8c0@arkdesktop>

> I'll try to make an argument for "Interface" over "Ability" using
> examples from Zope.

> It seems to me that not all interfaces coincide with something the
> object can _do_.  Some speak to what can be done _to_ an object:

When I see the word "interface", I think of the collection of method calls
that a type supports, because the word is used that way in other well-known
programming languages.  I am suggesting the word "ability" because it covers
more than just method calls.



From guido at python.org  Wed Nov 22 16:01:42 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 07:01:42 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <002301c70e46$c7473260$6402a8c0@arkdesktop>
References: <45645252.6010101@benjiyork.com>
	<002301c70e46$c7473260$6402a8c0@arkdesktop>
Message-ID: <ca471dc20611220701u4f837661i65ed1a667de426a9@mail.gmail.com>

On 11/22/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > I'll try to make an argument for "Interface" over "Ability" using
> > examples from Zope.
>
> > It seems to me that not all interfaces coincide with something the
> > object can _do_.  Some speak to what can be done _to_ an object:
>
> When I see the word "interface", I think of the collection of method calls
> that a type supports, because the word is used that way in other well-known
> programming languages.  I am suggesting the word "ability" because it covers
> more than just method calls.

Hm, I would think it extends very naturally to instance variables, and
I see no reason why it's a particularly bad idea to also extend it
(mostly in our minds anyway, I expect) to describe certain behaviors
that cannot be expressed (easily) in code.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Nov 22 16:08:21 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 07:08:21 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <45640E3A.7010307@livinglogic.de>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com>
	<45640E3A.7010307@livinglogic.de>
Message-ID: <ca471dc20611220708v13f28179s7ded48a5308ab7cf@mail.gmail.com>

On 11/22/06, Walter D?rwald <walter at livinglogic.de> wrote:
> Another effect of __special__ methods is that they divide the class
> namespace into two parts: The __special__ names are those that have to
> be implemented to support core Python interfaces, and the methods
> without underscores are those that implement the "main aspect" of the
> class.

Well, there are plenty of examples of __special__ methods for other
purposes than core Python interfaces, and plenty of core Python
interfaces that don't use __special__ (dict.get(), list.append(),
iter.next(), str.lower(), you get the idea.)

The *original* idea was for __special__ names to apply to anything
that wasn't invoked using regular method notation: x+y, a[i],
o.foobar, f(args), len(a), that sort of thing. By extension or
expedience it also became used for meta-data (e.g. Zope's
__implements__) and for generic functions like pickling (__reduce__).

They do have the downside that there's only one __special__ namespace.
But that can be mitigated by using naming conventions. IMO it remains
to be seen whether there will be too many generic functions to make
this work sensibly.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gsakkis at rutgers.edu  Wed Nov 22 16:28:47 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 22 Nov 2006 10:28:47 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <002201c70e46$5149f750$6402a8c0@arkdesktop>
References: <4563EB1C.5040408@acm.org>
	<002201c70e46$5149f750$6402a8c0@arkdesktop>
Message-ID: <91ad5bf80611220728p55dc2220q8673cd6858fcf60e@mail.gmail.com>

On 11/22/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > Both 'ability' and 'interface' imply (to me, anyway) that the class
> > being inspected is an actor, that it 'does something' rather than being
> > operated on.
>
> I chose 'ability' because to me it doesn't require that the class being
> inspected is active by itself.  For example, it feels natural to me to speak
> of a class as "having the totally ordered ability".

>From a linguistic point of view, "having the totally ordered property"
seems more natural and, more generally, captures better both active
("does something") and passive ("is [a description of] something")
concepts, but...

> The problem I have with 'feature' is that it's already in widespread use
> without a formal meaning.

Even worse, "property" has already a (different) formal meaning in
python, so I guess that's not an option (unless someone can come up
with a better name for existing properties _and_ convince that the
backwards incompatible name change is not a show-stopper).

George

From ark-mlist at att.net  Wed Nov 22 16:29:22 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 10:29:22 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220701u4f837661i65ed1a667de426a9@mail.gmail.com>
Message-ID: <000501c70e4a$fcb29800$6402a8c0@arkdesktop>

> Hm, I would think it extends very naturally to instance variables, and
> I see no reason why it's a particularly bad idea to also extend it
> (mostly in our minds anyway, I expect) to describe certain behaviors
> that cannot be expressed (easily) in code.

I think of an ability as a property of an interface rather than an interface
itself.  For example, I can imagine a single interface having multiple
abilities.




From pje at telecommunity.com  Wed Nov 22 16:45:40 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 10:45:40 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061122131656.GA8659@niemeyer.net>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>

At 11:16 AM 11/22/2006 -0200, Gustavo Niemeyer wrote:
>I've lost you here.  As I understand interfaces, their most important
>feature is indeed being a "comment in the form of code", and I hope
>they continue to be that way.

What does it add to have to declare a class as being "Iterable", if it 
already implements __iter__?  What does the notion of "Iterable" add to the 
execution *or* understanding of the code?

If you say that it is because having "Iterable" is needed to distinguish 
different possible interpretations of the same method name(s), then why not 
just have method namespaces to begin with?

"defop" is one way to do that, of course, but there could easily be others, 
like a "namespace" statement inside of classes that would contain the 
method definitions.

I'd like to try to meet the needs, without assuming that "interfaces" are 
the right answer.  Indeed, if interfaces are considered the answer, I want 
to know what the question is.  :)  Many years ago, I helped to create 
Zope's interface adaptation system, and I built PyProtocols as well, using 
interfaces quite extensively in PEAK.  So, I am familiar with the presumed 
benefits of interfaces as well as their drawbacks -- especially their 
drawbacks.  And chief amongst those drawbacks are complexity and 
duplication.

In essence, interfaces turn libraries into "frameworks", but generic 
functions turn frameworks into libraries.  I didn't really discover this 
until January of last year, though, when I did some experiments in 
replacing various PEAK interfaces with generic functions:

http://dirtsimple.org/2005/01/like-acid-for-frameworks.html

The experience of refactoring was what made me realize that generic 
functions weren't just for obscure fancy use cases like security rules and 
binary math operators.  They *gutted* my interface-based code down to 
nothing -- hence my description of them as being "like acid for frameworks".


From guido at python.org  Wed Nov 22 16:53:08 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 07:53:08 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
Message-ID: <ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>

On 11/21/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:57 PM 11/21/2006 -0800, Guido van Rossum wrote:
> >Phillip, please shorten your posts. You're hogging all the bandwidth I
> >have for thinking about this. Please!
>
> Er, that *was* the shortened version; I actually spent about an hour *just*
> on shortening it.

Ouch.

> I tried to write a short reply to your message, but with little
> success.  So, I'll just rate limit myself by refraining from posting for a
> couple of days.  Will that work?

Sure. Or you could take up my suggestion and spec out addmethod etc.,
trying to do without defop for now. Or answer the email below in less
space than the original.

Here are a few more things to ponder.

1. While I write a lot of code, I don't write much code that could
benefit from interfaces or generic functions much. What I write is
typically a very specific application or a library that speaks a very
specific protocol, not a framework. There's one kind of interface that
could benefit me, and that's a quick way to distinguish between
certain standard types. Examples that keep recurring are things like
the smtplib.SMTP.sendmail() method, whose second argument is either a
string or a list of strings. The optimal way of writing this, IMO, is
a single method that typechecks its 2nd argument. It currently uses
"if isinstance(x, basestring)" but I'd be happier if this could be
written as some kind of check for implementing the String interface.
(If it's neither a string nor a list, I don't really care what
happens, it will just raise an exception a few lines later and that's
perfectly fine for diagnosing the problem). Rewriting this as a
generic method (how do I even create generic methods? I'm sure it's
easy) sounds like a total waste of time and code. Expecting that
authors of alternative String implementations would care enough about
this obscure and super-specialized method to add a binding for their
type (smtplib.SMTP.sendmail[MyStringType] =
smtplib.SMTP.sendmail[str]) would be naive. So every application that
wants to pass a MyStringType to sendmail() needs to do this assignment
itself -- a serious reduction in convenience. If there were an
interface that means "Behaves like a string" then those few brave
souls who write their own string implementation from scratch could
just claim to implement it and their string objects would be welcome
everywhere. (Sure, there might be a few places where they weren't
welcome due to very subtle differences from 'str' -- but I don't
expect those to be simply the lack of a certain method, rather some
invariant involving several objects.) Another example is the many
"open" functions (for opening e.g. tar or zip files, or compressed or
encrypted streams, or parsers) that take either a filename or a file
objects. It is much more convenient to write a single function that
checks if the argument is a string and in that case opens it than to
have to use a generic function. I'm foreseeing that you'll propose
having a standard generic function that takes a string or a stream and
returns a stream -- that is indeed a handy thing to have, except we
still have to know whether the argument was a string or not, because
if we open it, we want to close it, but if we were passed a stream, we
*don't* want to close it.

2. Numbers are another example. They are very common arguments to
functions and methods. Very few people write their own number
implementations, because the built-in ones are so good. But there
*are* people who do, for very specialized reasons (numpy has a whole
slew of new number types). I don't think it makes sense to turn
everything that takes a number as argument into a generic function.
Now, due to duck typing, if you just *use* numbers, and the numpy
folks have done their job, things will just work -- this is your
argument about using generic operations. But as soon as any type
testing happens (e.g. when writing an API that takes either a number
of a string or a list of numbers), I'd much rather have a few "big"
interfaces (Number, RealNumber, Integer) than lots of small ones
Pedroni-style (number-that-support-exponentiation,
number-that-supports-bitwise-and, number-that-supports-left-shift,
...?).

I guess I either don't understand or don't completely buy the
"Pedroni" argument that interfaces should focus on single methods. Or
perhaps numbers and strings (types with rich APIs that are only rarely
re-implemented from scratch) are special cases?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Nov 22 16:55:25 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 07:55:25 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <000501c70e4a$fcb29800$6402a8c0@arkdesktop>
References: <ca471dc20611220701u4f837661i65ed1a667de426a9@mail.gmail.com>
	<000501c70e4a$fcb29800$6402a8c0@arkdesktop>
Message-ID: <ca471dc20611220755s23b4af05l967e30cf8ca1d022@mail.gmail.com>

On 11/22/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > Hm, I would think it extends very naturally to instance variables, and
> > I see no reason why it's a particularly bad idea to also extend it
> > (mostly in our minds anyway, I expect) to describe certain behaviors
> > that cannot be expressed (easily) in code.
>
> I think of an ability as a property of an interface rather than an interface
> itself.  For example, I can imagine a single interface having multiple
> abilities.

Hm, that sounds like a case for inheritance. The abilities would be
the bases and the interface would extend all of them. They could all
be called abilities or they could all be called interfaces. The
linguistic metaphor only goes so far...

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Nov 22 17:04:31 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 11:04:31 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <45645252.6010101@benjiyork.com>
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>

At 08:36 AM 11/22/2006 -0500, Benji York wrote:
>It seems to me that not all interfaces coincide with something the object 
>can _do_.  Some speak to what can be done _to_ an object:
>
>class IAttributeAnnotatable(IAnnotatable):
>     """Marker indicating that annotations can be stored on an attribute.
>
>     This is a marker interface giving permission for an `IAnnotations`
>     adapter to store data in an attribute named `__annotations__`.
>
>     """

Notice that this can be defined in terms of a "get_annotation_map(ob)" 
generic function.  Classes registered for that function can simply return 
self.__annotations__ (after setting it to a dictionary, if not already 
initialized).  In fact, it seems to me that such would be the *default* 
implementation of such a function, eliminating the need to declare anything 
except for classes that need to implement annotation storage 
differently.  Or am I misunderstanding the intended use?


>Some give describe how the object will _act_: Persistent.

Which indicates what outside operations can be performed on the object, 
such as "load", "save", etc.


>   Some are application-level statements about how what is allowed to be 
> _done_with_ the object: ISharable (the user can "share" this object with 
> other users in various ways; read, write, etc.).

Aren't these also *operations* being performed?


>These (especially the marker interfaces) don't seem to be statements about 
>what the objects can do (abilities),

"Ability" doesn't imply that the object itself has to do the thing.


>  but about other characteristics,
>therefore a more general name like "interface" seems appropriate.

Actually "ability" is more general than "interface", because interface 
implies two things talking to one another.  Some Zope uses of "interfaces" 
don't include any actual "interfacing" at all, so to me your examples 
actually support calling them "abilities" rather than "interfaces".


From guido at python.org  Wed Nov 22 17:23:51 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 08:23:51 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<45645252.6010101@benjiyork.com>
	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
Message-ID: <ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> Actually "ability" is more general than "interface", because interface
> implies two things talking to one another.  Some Zope uses of "interfaces"
> don't include any actual "interfacing" at all, so to me your examples
> actually support calling them "abilities" rather than "interfaces".

I like this line of reasoning. It makes a lot of sense.

I'm still torn because I also like using familiar terminology or
syntax even if I have to give it new semantics; Python is all about
that! It can get tedious to have to explain to new users "well, an
assignment is known as PUT, and a variable is called a location, and
subroutines we call a HOW'TO, and strings we call texts" (as was the
case for ABC, Python's ill-fated predecessor which invented "optimal"
terminology in a vacuum).

But for the time being I'd like to try out this new word and see how it fits.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Wed Nov 22 17:41:10 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 11:41:10 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
Message-ID: <000801c70e55$043464a0$6402a8c0@arkdesktop>

> What does it add to have to declare a class as being "Iterable", if it
> already implements __iter__?  What does the notion of "Iterable" add to
> the execution *or* understanding of the code?

Let's suppose for the sake of argument that declaring a class as being
"Iterable" adds nothing.  What does that say about whether the ability to
declare other abilities is useful?  I think it doesn't say much.



From steven.bethard at gmail.com  Wed Nov 22 17:42:52 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 22 Nov 2006 09:42:52 -0700
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
Message-ID: <d11dcfba0611220842l5d83c319od0294f9e8e71cef6@mail.gmail.com>

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> Examples that keep recurring are things like the
> smtplib.SMTP.sendmail() method, whose second argument
> is either a string or a list of strings.
[snip]
> Rewriting this as a generic method
[snip]
> sounds like a total waste of time and code. Expecting that
> authors of alternative String implementations would care enough about
> this obscure and super-specialized method to add a binding for their
> type (smtplib.SMTP.sendmail[MyStringType] =
> smtplib.SMTP.sendmail[str]) would be naive.

I'm probably totally off base here, but I thought Phillip's suggestion
would be not for people to be registering their types for functions,
but to be registering their type's methods with the classes they're
supposed to be emulating.  So I would have expected something like:

    str.lower[MyStringType] = MyStringType.lower
    str.split[MyStringType] = MyStringType.split
    ...

And then they wouldn't register themselves for smtplib.SMTP.sendmail;
they'd just rely on that function asking for a ``str.lower &
str.split`` or whatever as its second argument.  (Does seem like there
needs to be some sort of shorthand though for "all the string
methods".)

> 2. Numbers are another example. They are very common arguments to
> functions and methods. Very few people write their own number
> implementations, because the built-in ones are so good. But there
> *are* people who do, for very specialized reasons (numpy has a whole
> slew of new number types). I don't think it makes sense to turn
> everything that takes a number as argument into a generic function.

Again, I may have misunderstood the proposal, but I didn't think you'd
be changing the functions that took numbers, I thought you'd be
registering the new number type's methods with the builtin methods
they were intended to emulate, e.g.::

    int.__add__[MyIntegerType] = MyIntegerType.__add__
    int.__mul__[MyIntegerType] = MyIntegerType.__mul__

(or however + and * are supposed to be registered).

> Now, due to duck typing, if you just *use* numbers, and the numpy
> folks have done their job, things will just work -- this is your
> argument about using generic operations. But as soon as any type
> testing happens (e.g. when writing an API that takes either a number
> of a string or a list of numbers), I'd much rather have a few "big"
> interfaces (Number, RealNumber, Integer)

Yeah, it certainly seems like in both this case and the string case,
there needs to be some shorthand for saying "implements all the basic
String/Number/whatever methods".


STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From pje at telecommunity.com  Wed Nov 22 18:12:16 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 12:12:16 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <000801c70e55$043464a0$6402a8c0@arkdesktop>
References: <5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122120602.041d2af0@sparrow.telecommunity.com>

At 11:41 AM 11/22/2006 -0500, Andrew Koenig wrote:
> > What does it add to have to declare a class as being "Iterable", if it
> > already implements __iter__?  What does the notion of "Iterable" add to
> > the execution *or* understanding of the code?
>
>Let's suppose for the sake of argument that declaring a class as being
>"Iterable" adds nothing.  What does that say about whether the ability to
>declare other abilities is useful?  I think it doesn't say much.

I submit that every other ability can be similarly defined, because the 
code to *implement* that ability must exist in one of three places:

1. a method of the object
2. a method of an adapter
3. a method registered somewhere, e.g. with a generic function

Neither #1 nor #3 require a separate interface declaration, and #2 can be 
subsumed into an adaptation generic function -- which is actually what 
iter() does, anyway.

Note that all three cases offer potential for directly verifying 
implementation of the ability, whereas *declaring* an ability merely adds a 
distinct *claim* that it has been implemented -- which then may need to be 
separately verified.


From pje at telecommunity.com  Wed Nov 22 18:02:18 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 12:02:18 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122111942.041e6c30@sparrow.telecommunity.com>

At 07:53 AM 11/22/2006 -0800, Guido van Rossum wrote:
>  If there were an
>interface that means "Behaves like a string" then those few brave
>souls who write their own string implementation from scratch could
>just claim to implement it and their string objects would be welcome
>everywhere.

Okay, make an 'as_string()' generic function that returns a string (or 
subclass thereof), or fails.  Then, allow the use of one generic function 
as a type specifier for another, so that if I want to make a 'flatten()' 
function, I can 'addmethod(flatten, lambda ob: ob, as_string)' to mean "the 
flatten() of something that can be passed to as_string is the object itself".


>having a standard generic function that takes a string or a stream and
>returns a stream -- that is indeed a handy thing to have, except we
>still have to know whether the argument was a string or not, because
>if we open it, we want to close it, but if we were passed a stream, we
>*don't* want to close it.

Actually, I'd say you'd have a generic function that returns either an 
adapter or a context manager.  The stream adapter would have a no-op 
close(), or the stream context manager would do nothing on exit.  (e.g. 
"with stream_for(string_or_stream) as f:".)


>Now, due to duck typing, if you just *use* numbers, and the numpy
>folks have done their job, things will just work -- this is your
>argument about using generic operations. But as soon as any type
>testing happens (e.g. when writing an API that takes either a number
>of a string or a list of numbers),

Isn't this basically just another flatten() example?  The only time you'd 
need special casing for numbers would be if you had an iterable number 
type; otherwise, why even bother checking if they're numbers?


>perhaps numbers and strings (types with rich APIs that are only rarely
>re-implemented from scratch) are special cases?

The Pedroni argument is oriented towards types like file, list, and dict, 
that have a rich API but are more frequently re-implemented from scratch 
and more likely to be missing methods.


From gustavo at niemeyer.net  Wed Nov 22 17:55:13 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Wed, 22 Nov 2006 14:55:13 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
Message-ID: <20061122165513.GA13795@niemeyer.net>

> What does it add to have to declare a class as being "Iterable", if it
> already implements __iter__?  What does the notion of "Iterable" add
> to the execution *or* understanding of the code?

If you take a standard method which has a well documented meaning,
it's of course non-useful.

Now, let's say that you create a library named libfruit, and that
it implements a very nice IFruit interface.  This library is so nice
and shiny that it becomes widely used, and thus many ways to handle
a fruit are developed by third parties.  Now, someone else has a
libapple library, which unfortunately is a bit outdated, and wasn't
designed to work with the shiny libfruit library.

With interfaces, all you need to do is define an adapter from Apple
to IFruit, and instantly all the widely available frameworks will
be handling apples just fine.

Even though silly, this example describes some of the issues that
are common when handling libraries from several vendors.  Can you
describe how that'd work under your ideas?


> If you say that it is because having "Iterable" is needed to
> distinguish different possible interpretations of the same method
> name(s), then why not just have method namespaces to begin with?
> 
> "defop" is one way to do that, of course, but there could easily be
> others, like a "namespace" statement inside of classes that would
> contain the method definitions.
(...)

I'm sure there are many ways to get similar semantics to that offered
by interfaces. I can argue that a namespace like this would just take
ambiguity to a different level since it needs a name, or that it'd be
worse in terms of validation, or even that it'd take away some of the
benefits of introspection, and so on. But I don't see value in moving
on that kind of discussion here comparing a well established practice
and implementation with an idea which isn't clearly defined.


> In essence, interfaces turn libraries into "frameworks", but generic 
> functions turn frameworks into libraries.  I didn't really discover this 
> until January of last year, though, when I did some experiments in 
> replacing various PEAK interfaces with generic functions:

I'm sure these sentences have a well defined meaning in your mind,
but they seem extremmely vague from my perspective.


> The experience of refactoring was what made me realize that generic
> functions weren't just for obscure fancy use cases like security rules
> and binary math operators.  They *gutted* my interface-based code down
> to nothing -- hence my description of them as being "like acid for
> frameworks".

You seem to compare interfaces to generic functions.  As you
must be aware of, they're really orthogonal concepts.  You can
do adaptation and generic function-based code with or without
interfaces.

-- 
Gustavo Niemeyer
http://niemeyer.net

From janssen at parc.com  Wed Nov 22 18:20:05 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 09:20:05 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <000501c70e4a$fcb29800$6402a8c0@arkdesktop> 
References: <000501c70e4a$fcb29800$6402a8c0@arkdesktop>
Message-ID: <06Nov22.092011pst."58648"@synergy1.parc.xerox.com>

Andrew Koenig writes:
> For example, I can imagine a single interface having multiple
> abilities.

Perhaps because it inherits from multiple sub-interfaces?

Bill

From janssen at parc.com  Wed Nov 22 18:24:26 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 09:24:26 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com> 
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
Message-ID: <06Nov22.092436pst."58648"@synergy1.parc.xerox.com>

> At 07:53 PM 11/21/2006 -0800, Bill Janssen wrote:
> >I don't mind having both, but generic functions (interface-based
> >method dispatch) is really only one aspect of the issue.
> 
> How so?
> 

I think Guido answered this better than I could in
<ca471dc20611220753q5be31ce0sc2e36906a782d585 at mail.gmail.com>.

I seem to spend a lot of time inside functions looking over values and
figuring out what to do do with them, actions which don't involve
invoking further functions, but rather operating directly on those
values in different ways.  That's probably why the lack of
interface-based design in the Python fundamentals seems such a problem
to me (and not to others).

Bill

From janssen at parc.com  Wed Nov 22 18:32:46 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 09:32:46 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com> 
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<45645252.6010101@benjiyork.com>
	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
	<ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com>
Message-ID: <06Nov22.093251pst."58648"@synergy1.parc.xerox.com>

Guido wrote:
> On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> > Actually "ability" is more general than "interface", because interface
> > implies two things talking to one another.  Some Zope uses of "interfaces"
> > don't include any actual "interfacing" at all, so to me your examples
> > actually support calling them "abilities" rather than "interfaces".
> 
> I like this line of reasoning. It makes a lot of sense.
> 
> I'm still torn because I also like using familiar terminology or
> syntax even if I have to give it new semantics; Python is all about
> that! It can get tedious to have to explain to new users "well, an
> assignment is known as PUT, and a variable is called a location, and
> subroutines we call a HOW'TO, and strings we call texts" (as was the
> case for ABC, Python's ill-fated predecessor which invented "optimal"
> terminology in a vacuum).
> 
> But for the time being I'd like to try out this new word and see how it fits.
> 
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

I see nothing wrong with defining "empty" ABCs to indicate abilities
rather than interfaces.

   class WellTested (Ability):
     """Indicates that the type has passed the QA process"""
     pass

   class TestedNumber (Number, WellTested):
     ...

The whole point of using classes instead of just looking at the
methods is to be able to indicate semantics that the programming
language can't express.  (Well, along with improved simplicity and
improved communication and faster type-checking :-).

Bill

From pje at telecommunity.com  Wed Nov 22 18:59:38 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 12:59:38 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <d11dcfba0611220842l5d83c319od0294f9e8e71cef6@mail.gmail.co
 m>
References: <ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>

At 09:42 AM 11/22/2006 -0700, Steven Bethard wrote:
>Yeah, it certainly seems like in both this case and the string case,
>there needs to be some shorthand for saying "implements all the basic
>String/Number/whatever methods".

But we are probably better off composing such a concept *from* the methods, 
rather than having the concept be merely a container *of* methods.

This is basically the difference between Java interfaces and Haskell's 
notion of "typeclasses".  You can define typeclasses like "ordered" or 
"four-function arithmetic" and it's okay if they overlap, because they're 
defined in terms of the operations you can perform.  So nothing prevents 
somebody else from making "six-function arithmetic", or "reversible 
strings" or whatever.  And nobody has to *re*define their type as 
supporting these new "interfaces", as long as they have the requisite 
operations.

So, if we have to have something like an interface, let's steal Haskell's 
version of the idea instead of Java's, because it involves a lot less 
typing.  :)


From brett at python.org  Wed Nov 22 19:05:36 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 22 Nov 2006 10:05:36 -0800
Subject: [Python-3000] LINQ
In-Reply-To: <1d85506f0611211507n54f22ed0l9654f0d02442221d@mail.gmail.com>
References: <1d85506f0611211507n54f22ed0l9654f0d02442221d@mail.gmail.com>
Message-ID: <bbaeab100611221005u12d5c93m38b297102b90ce30@mail.gmail.com>

On 11/21/06, tomer filiba <tomerfiliba at gmail.com> wrote:
>
> i read the references fredrik posted several days back, and it got
> me thinking: why not add a "co_ast" attribute to code objects?
>
> i.e., since (virtually) all code objects are constructed from source
> code by compilation, the compiler could just emit the AST as an
> attribute to the generated code object.
>
> yes, there's an issue of memory consumption, but bear with me
> for a moment. also, maybe co_ast could be a lazy attribute, reading
> the ast from the pyc file on demand.


Because the .pyc file does not store enough information for an easy
bytecode->AST transformation.  You would need to read from the source code
and and reconstruct the AST.

This is why, as of now, you need to go through compile() to get an AST.  I
personally would rather wait to see how much the AST is actually used before
trying to give it an attribute on code objects.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061122/4734ad25/attachment.html 

From steven.bethard at gmail.com  Wed Nov 22 19:08:20 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 22 Nov 2006 11:08:20 -0700
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>
Message-ID: <d11dcfba0611221008u7c8a7b49u3ae749df97c0b95c@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:42 AM 11/22/2006 -0700, Steven Bethard wrote:
> >Yeah, it certainly seems like in both this case and the string case,
> >there needs to be some shorthand for saying "implements all the basic
> >String/Number/whatever methods".
>
> But we are probably better off composing such a concept *from* the methods,
> rather than having the concept be merely a container *of* methods.

Yeah, I kinda assumed that the shorthand would be to define something like:

    IString = str.lower & str.upper & str.split & ...

But I'm still a little vague on how this supposed to be implemented.
This should definitely be elaborated in the PEP.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From pje at telecommunity.com  Wed Nov 22 19:23:42 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 13:23:42 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov22.092436pst."58648"@synergy1.parc.xerox.com>
References: <5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>

At 09:24 AM 11/22/2006 -0800, Bill Janssen wrote:
>I seem to spend a lot of time inside functions looking over values and
>figuring out what to do do with them, actions which don't involve
>invoking further functions, but rather operating directly on those
>values in different ways.

Whereas to me, that's an antipattern because that makes your code closed to 
extension.    What if somebody has a new kind of object that you didn't 
anticipate?  Interfaces don't solve the problem, they just kick it up to 
another level of abstraction, like hiding the dirty laundry under the rug.

The real problem here is that type inspection violates encapsulation and 
the whole point of being "object-oriented" in the first place.  The point 
of OO is that it lets you *hide* the implementation.  If you put the 
implementation in an if-then branch, you've just lost your OO-ness.

The reason it's *tempting* to do this in Python, is because we have so many 
convenient built-in data types that don't have application-specific 
methods.  In Java, where you have to create new types every time you turn 
around, and have method overloading anyway, you don't *do* this kind of 
type inspection.  It gets handled by type dispatching -- a primitive sort 
of compile-time only generic function!  It's considered bad style to write 
a huge block of 'ob instanceof IFoo' tests in Java, when you can just write 
overloaded methods.

In Java, SMTP.sendmail would be something like this (using Python-like 
syntax 'cause my Java is rusty):

     def sendmail(self, from, to_addrs:str, msg, ...):
         return self.sendmail(from_addr, [to_addrs], msg, ...)

     def sendmail(self, from, to_addrs:list[str], msg, ...):
         # main implementation

So, what I've been trying to say is that I don't think it's a good idea to 
copy Java interfaces without also copying the machinery that allows you to 
avoid 'instanceof' spaghetti.


From ark-mlist at att.net  Wed Nov 22 19:27:07 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 13:27:07 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov22.092011pst."58648"@synergy1.parc.xerox.com>
Message-ID: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>

> Andrew Koenig writes:
> > For example, I can imagine a single interface having multiple
> > abilities.

> Perhaps because it inherits from multiple sub-interfaces?

Or perhaps because after the interface was defined, someone noticed that it
happened to have those abilities and wanted to be able to say to after the
fact.



From guido at python.org  Wed Nov 22 19:31:38 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 10:31:38 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221031h322102d6x7d7501972ca118be@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:42 AM 11/22/2006 -0700, Steven Bethard wrote:
> >Yeah, it certainly seems like in both this case and the string case,
> >there needs to be some shorthand for saying "implements all the basic
> >String/Number/whatever methods".
>
> But we are probably better off composing such a concept *from* the methods,
> rather than having the concept be merely a container *of* methods.
>
> This is basically the difference between Java interfaces and Haskell's
> notion of "typeclasses".  You can define typeclasses like "ordered" or
> "four-function arithmetic" and it's okay if they overlap, because they're
> defined in terms of the operations you can perform.  So nothing prevents
> somebody else from making "six-function arithmetic", or "reversible
> strings" or whatever.  And nobody has to *re*define their type as
> supporting these new "interfaces", as long as they have the requisite
> operations.
>
> So, if we have to have something like an interface, let's steal Haskell's
> version of the idea instead of Java's, because it involves a lot less
> typing.  :)

But the main thing that stands in the way of doing the right thing in
Java is that all interfaces have to be declared when the class is
defined. In *my* notion of abilities, this part being inspired by Zope
interfaces, you can state after the fact that a particular class
(which you didn't write) provides a given ability (which you didn't
write either). I think it would also be great if we had "ability
algebra" whereby you could state that a given ability A is composed of
existing abilities A1 and A2, and every object or class that already
has A1 and A2 will automatically be considered to have A.

However, I do *not* want to go the way of duck typing here -- if an
ability A claims that it is implemented by method foo(), and an object
x has a foo() method but doesn't explicitly claim to have ability A
(and neither does its class), then testing x for A should return
false. I think this deviates from Haskell, which seems to be big on
structural type equivalence (?).

So I think Python needs to do its own thing here, to some extent.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jackdied at jackdied.com  Wed Nov 22 18:04:32 2006
From: jackdied at jackdied.com (Jack Diederich)
Date: Wed, 22 Nov 2006 12:04:32 -0500
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <ek125p$36l$1@sea.gmane.org>
References: <bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
	<ek09iv$850$1@sea.gmane.org> <ek125p$36l$1@sea.gmane.org>
Message-ID: <20061122170432.GA21751@performancedrivers.com>

On Wed, Nov 22, 2006 at 09:36:09AM +0100, Georg Brandl wrote:
> Terry Reedy schrieb:
> > "Brett Cannon" <brett at python.org> wrote in message 
> > news:bbaeab100611211232r53de3c33p915bee7b49dbdf9a at mail.gmail.com...
> > 
> > Why can't the fallback usage just pass the return value from __len__ to 
> > bool() (forget the C function name) and return that result?  It's just like 
> > doing::
> > 
> >   def bool(obj):
> >       try:
> >           return obj.__bool__()
> >       except AttributeError:
> >           return bool(len(obj))
> > ------------
> > 
> > If an object without __bool__ returned itself as its length, this would be 
> > an infinite loop, at least in this Python version.  Do we worry about 
> > something so crazy?
> 
> The length would have to be an integer, and this would have to be checked.
> 

It looks like the regular checks are happening on __len__ methods
anyway so the explicit int check in slot_nb_bool is redundant.
This is the first time I've looked at the new slots in py3k so
feel free to correct. (using bool4.patch)

sprat:~/src/py3k-rw> ./python
Python 3.0x (p3yk:52823M, Nov 22 2006, 11:57:34) 
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> class A(object):
  def __len__(self):
    return -1

a = A()
print bool(a)
  ... ... ... >>> >>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: __len__() should return >= 0
>>> 

-Jack

From ark-mlist at att.net  Wed Nov 22 19:37:06 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 13:37:06 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221031h322102d6x7d7501972ca118be@mail.gmail.com>
Message-ID: <001901c70e65$3a40a7b0$6402a8c0@arkdesktop>

> I think it would also be great if we had "ability
> algebra" whereby you could state that a given ability A is composed of
> existing abilities A1 and A2, and every object or class that already
> has A1 and A2 will automatically be considered to have A.

Yes!

> However, I do *not* want to go the way of duck typing here -- if an
> ability A claims that it is implemented by method foo(), and an object
> x has a foo() method but doesn't explicitly claim to have ability A
> (and neither does its class), then testing x for A should return
> false.

Also yes.

> I think this deviates from Haskell, which seems to be big on
> structural type equivalence (?).

How so?

> So I think Python needs to do its own thing here, to some extent.

Yup.  That's part of why I like the term "ability" -- one isn't tempted to
think it means the same thing as the corresponding term in another language.



From guido at python.org  Wed Nov 22 19:43:41 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 10:43:41 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221043q5ab8c55vdb63ed41e1bf7d5f@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 09:24 AM 11/22/2006 -0800, Bill Janssen wrote:
> >I seem to spend a lot of time inside functions looking over values and
> >figuring out what to do do with them, actions which don't involve
> >invoking further functions, but rather operating directly on those
> >values in different ways.
>
> Whereas to me, that's an antipattern because that makes your code closed to
> extension.    What if somebody has a new kind of object that you didn't
> anticipate?

I'm sorry Phillip, but not all code needs to be open to extension.
There's an awful lot of bread-and-butter coding (including some of my
own :-) that really doesn't benefit from being open to extension.

> Interfaces don't solve the problem, they just kick it up to
> another level of abstraction, like hiding the dirty laundry under the rug.

There is a dangerous amount of content-free rhetoric in this thread.
(And I apogize from my own contributions.)

> The real problem here is that type inspection violates encapsulation and
> the whole point of being "object-oriented" in the first place.  The point
> of OO is that it lets you *hide* the implementation.  If you put the
> implementation in an if-then branch, you've just lost your OO-ness.

OO is not religion, it's a tool, and it is not the only hammer we
have. As I've tried to mention through examples, there are plenty of
situations where a carefully placed if-then is a lot more practical
than adding a new method to some remote class (or several).

> The reason it's *tempting* to do this in Python, is because we have so many
> convenient built-in data types that don't have application-specific
> methods.  In Java, where you have to create new types every time you turn
> around, and have method overloading anyway, you don't *do* this kind of
> type inspection.  It gets handled by type dispatching -- a primitive sort
> of compile-time only generic function!  It's considered bad style to write
> a huge block of 'ob instanceof IFoo' tests in Java, when you can just write
> overloaded methods.

True, some type testing code in Python would be done using overloading
in Java. But it doesn't always apply, for example when you're
examining objects that weren't passed as arguments but were maybe
gotten by accessing an argument's attribute, or pulled out of a
container, or perhaps produced by some factory function.

> In Java, SMTP.sendmail would be something like this (using Python-like
> syntax 'cause my Java is rusty):
>
>      def sendmail(self, from, to_addrs:str, msg, ...):
>          return self.sendmail(from_addr, [to_addrs], msg, ...)
>
>      def sendmail(self, from, to_addrs:list[str], msg, ...):
>          # main implementation

Right. If this syntax was possible in Python lots of people would be
very happy. But even the best generic function API I've seen is a lot
more verbose than this -- there seems to be a separate set-up
involved. A single if-then test does the same job with less fuss, and
when you need to support more types, it's easy to refactor to use the
GF approach.

> So, what I've been trying to say is that I don't think it's a good idea to
> copy Java interfaces without also copying the machinery that allows you to
> avoid 'instanceof' spaghetti.

It's not always automatically spaghetti. Often a single carefully
placed test is all you need. sendmail() grew that test long ago when
it was noticed that passing a string instead of a list was a common
mistake (a certain Googler with a single-letter username gets a lot of
junk mail occasionally :-). Nobody has felt the need to add more
branches to it in many years.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Nov 22 19:45:21 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 10:45:21 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <001801c70e63$d172eeb0$6402a8c0@arkdesktop> 
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
Message-ID: <06Nov22.104526pst."58648"@synergy1.parc.xerox.com>

> > Andrew Koenig writes:
> > > For example, I can imagine a single interface having multiple
> > > abilities.
> 
> > Perhaps because it inherits from multiple sub-interfaces?
> 
> Or perhaps because after the interface was defined, someone noticed that it
> happened to have those abilities and wanted to be able to say to after the
> fact.

More of a tagging approach, then?

Something like

  class MyNewClass (ExistingClass, OtherInterfaceIJustNoticedExistingClassImplements):
    pass

?

Bill

From guido at python.org  Wed Nov 22 19:46:42 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 10:46:42 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <001901c70e65$3a40a7b0$6402a8c0@arkdesktop>
References: <ca471dc20611221031h322102d6x7d7501972ca118be@mail.gmail.com>
	<001901c70e65$3a40a7b0$6402a8c0@arkdesktop>
Message-ID: <ca471dc20611221046q948b402qec5dc07fc3b1026@mail.gmail.com>

On 11/22/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > I think this deviates from Haskell, which seems to be big on
> > structural type equivalence (?).

I may be mistaken, but I thought that in Haskell, if there's a
typeclass composed of methods X and Y (with appropriate signatures),
every object that implements X and Y (with those signatures) is
*automatically* assumed to be a member of that typeclass? Since I've
never used Haskell and all my knowledge of it is from reading about it
in Phillip's posts (I've never even followed any of the links he gave)
I may well be mistaken. Anyway, I don't care so much about Haskell or
whether we copy it; I care about the specific rules/ideas I posted.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Wed Nov 22 19:48:24 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed, 22 Nov 2006 13:48:24 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov22.104526pst."58648"@synergy1.parc.xerox.com>
Message-ID: <001b01c70e66$ca43ea10$6402a8c0@arkdesktop>

> More of a tagging approach, then?
> 
> Something like
> 
>   class MyNewClass (ExistingClass,
> OtherInterfaceIJustNoticedExistingClassImplements):
>     pass
> 
> ?

Maybe, maybe not.  I'm not sure.  I'm thinking that it may be useful to be
able somehow to assert that pre-existing class C has pre-existing ability A,
because the authors of C and A weren't talking to each other at the time
they wrote them.



From guido at python.org  Wed Nov 22 19:49:24 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 10:49:24 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <3218934778639261534@unknownmsgid>
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
	<3218934778639261534@unknownmsgid>
Message-ID: <ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com>

On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> Something like
>
>   class MyNewClass (ExistingClass, OtherInterfaceIJustNoticedExistingClassImplements):
>     pass

No, unless you can also patch up all code that creates ExistingClass
instances. You alluded to this before and it sounded like you didn't
think it was an issue. Why do you think that? I see it as a huge
problem when retrofitting pre-existing third-party code.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Nov 22 19:55:28 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 10:55:28 PST
Subject: [Python-3000] Special methods and interface-based type system
Message-ID: <06Nov22.105531pst."58648"@synergy1.parc.xerox.com>

Talking about the Abilities/Interfaces made me think about some of our
"rogue" special method names.  In the Language Reference, it says, "A
class can implement certain operations that are invoked by special
syntax (such as arithmetic operations or subscripting and slicing) by
defining methods with special names."  But there are all these methods
with special names like __len__ or __unicode__ which seem to be
provided for the benefit of built-in functions, rather than for
support of syntax.  Presumably in an interface-based Python, these
methods would turn into regularly-named methods on an ABC, so that
__len__ would become

  class container:
    ...
    def len(self):
      raise NotImplemented

Though, thinking about it some more, I don't see why *all* syntactic
operations wouldn't just invoke the appropriate normally-named method
on a specific ABC.  "<", for instance, would presumably invoke
"object.lessthan" (or perhaps "comparable.lessthan").  So another
benefit would be the ability to wean Python away from this
mangled-name oddness, which seems to me an HCI improvement.

Bill

From walter at livinglogic.de  Wed Nov 22 19:56:08 2006
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 22 Nov 2006 19:56:08 +0100
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <20061122170432.GA21751@performancedrivers.com>
References: <bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>	<ek09iv$850$1@sea.gmane.org>
	<ek125p$36l$1@sea.gmane.org>
	<20061122170432.GA21751@performancedrivers.com>
Message-ID: <45649D48.3090703@livinglogic.de>

Jack Diederich wrote:

> On Wed, Nov 22, 2006 at 09:36:09AM +0100, Georg Brandl wrote:
>> Terry Reedy schrieb:
>>> "Brett Cannon" <brett at python.org> wrote in message 
>>> news:bbaeab100611211232r53de3c33p915bee7b49dbdf9a at mail.gmail.com...
>>>
>>> Why can't the fallback usage just pass the return value from __len__ to 
>>> bool() (forget the C function name) and return that result?  It's just like 
>>> doing::
>>>
>>>   def bool(obj):
>>>       try:
>>>           return obj.__bool__()
>>>       except AttributeError:
>>>           return bool(len(obj))
>>> ------------
>>>
>>> If an object without __bool__ returned itself as its length, this would be 
>>> an infinite loop, at least in this Python version.  Do we worry about 
>>> something so crazy?
>> The length would have to be an integer, and this would have to be checked.
>>
> 
> It looks like the regular checks are happening on __len__ methods
> anyway so the explicit int check in slot_nb_bool is redundant.
> This is the first time I've looked at the new slots in py3k so
> feel free to correct. (using bool4.patch)
> 
> sprat:~/src/py3k-rw> ./python
> Python 3.0x (p3yk:52823M, Nov 22 2006, 11:57:34) 
> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> class A(object):
>   def __len__(self):
>     return -1
> 
> a = A()
> print bool(a)
>   ... ... ... >>> >>> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ValueError: __len__() should return >= 0

OK, I've updated the patch: bugs.python.org/1600346

Servus,
   Walter

From fumanchu at amor.org  Wed Nov 22 20:16:02 2006
From: fumanchu at amor.org (Robert Brewer)
Date: Wed, 22 Nov 2006 11:16:02 -0800
Subject: [Python-3000] Abilities / Interfaces
Message-ID: <435DF58A933BA74397B42CDEB8145A8606F87058@ex9.hostedexchange.local>

Phillip J. Eby wrote:
> The real problem here is that type inspection violates 
> encapsulation and the whole point of being "object-oriented"
> in the first place.  The point of OO is that it lets you
> *hide* the implementation.  If you put the implementation
> in an if-then branch, you've just lost your OO-ness.
> 
> The reason it's *tempting* to do this in Python, is because 
> we have so many convenient built-in data types that don't
> have application-specific methods.

Grrmbl. It's such a strong temptation, I'm not sure it's worth fighting
in the interests of OO purity. I find myself constantly pushing
teammates to favor builtins over custom types for exactly this reason;
it reduces the overall complexity budget of any application, thus
freeing brain cells to add value (rather than "waste" them striving for
pure OO). And given the ease of subclassing builtin types, I'd rather
standardize on built-in methods than "application-specific" methods;
they meet 99+% of my needs.

> In Java, SMTP.sendmail would be something like this (using 
> Python-like syntax 'cause my Java is rusty):
> 
>      def sendmail(self, from, to_addrs:str, msg, ...):
>          return self.sendmail(from_addr, [to_addrs], msg, ...)
> 
>      def sendmail(self, from, to_addrs:list[str], msg, ...):
>          # main implementation

I have a feeling that the current Python-function-call performance
overhead would outweigh the benefits of this approach in my own code
(which rarely uses more than one isinstance check now).


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org

From pje at telecommunity.com  Wed Nov 22 20:32:58 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 14:32:58 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221031h322102d6x7d7501972ca118be@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<5.1.1.6.0.20061122123304.03a6af48@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122141542.03790d10@sparrow.telecommunity.com>

At 10:31 AM 11/22/2006 -0800, Guido van Rossum wrote:
>In *my* notion of abilities, this part being inspired by Zope
>interfaces, you can state after the fact that a particular class
>(which you didn't write) provides a given ability (which you didn't
>write either). I think it would also be great if we had "ability
>algebra" whereby you could state that a given ability A is composed of
>existing abilities A1 and A2, and every object or class that already
>has A1 and A2 will automatically be considered to have A.
>
>However, I do *not* want to go the way of duck typing here -- if an
>ability A claims that it is implemented by method foo(), and an object
>x has a foo() method but doesn't explicitly claim to have ability A
>(and neither does its class), then testing x for A should return
>false. I think this deviates from Haskell, which seems to be big on
>structural type equivalence (?).
>
>So I think Python needs to do its own thing here, to some extent.

Actually, you just described Haskell typeclasses.  :)  That is, they are:

1. Able to be specified after the fact
2. Are algebraically defined as sets of operations
3. Strongly typed based on functions

With respect to #3, I think you are confusing Haskell function names with 
Python method names.  Typeclasses also have one more ability that you 
didn't include above:

4. Able to provide implementations after the fact, not merely declarations

This is basically the same concept as adaptation and generic functions, so 
that you can not only *declare* that a type meets an interface, but also 
provide a way to make it work.

So, we are in agreement on some basics, I think.  I am mainly concerned 
that we do not treat interfaces as collections of method names, rather than 
collections of operations.  (Attributes can be considered operations to 
read, write, or delete them, and in many cases you will want to specify 
those operations individually in an interface/typeclass).

In Haskell, however, it is not that common for ordinary code to define or 
use typeclasses, since direct use of the operations (e.g. len()/iter() 
equivalents) normally suffices to imply the typeclass(es) involved.  Also, 
like Java, Haskell selects overloaded implementations based on their 
argument type, so you don't write "if isinstance" tests either.

Generics reduce the need for type testing and explicit interfaces, since 
you can just use an operation rather than needing a method (as in Haskell) 
or you can define overloads to select an implementation (as in Java).

In Python, however, interface libraries have historically treated their 
operations as simply a bag of method names and call signatures, which 
doesn't allow algebraic operations and introduces the need for explicit 
adapter types.  In contrast, an operation-based approach (where interfaces 
are collections of first-class "operations" of some kind, that can be 
recombined to create new interfaces) allows both interface algebra and 
*automatic* adapter generation (as described in my "monkey typing" proposal 
early last year).


From pje at telecommunity.com  Wed Nov 22 20:50:00 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 14:50:00 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <435DF58A933BA74397B42CDEB8145A8606F87058@ex9.hostedexchang
	e.local>
Message-ID: <5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>

At 11:16 AM 11/22/2006 -0800, Robert Brewer wrote:
>Phillip J. Eby wrote:
> > The real problem here is that type inspection violates
> > encapsulation and the whole point of being "object-oriented"
> > in the first place.  The point of OO is that it lets you
> > *hide* the implementation.  If you put the implementation
> > in an if-then branch, you've just lost your OO-ness.
> >
> > The reason it's *tempting* to do this in Python, is because
> > we have so many convenient built-in data types that don't
> > have application-specific methods.
>
>Grrmbl. It's such a strong temptation, I'm not sure it's worth fighting
>in the interests of OO purity. I find myself constantly pushing
>teammates to favor builtins over custom types for exactly this reason;

I wasn't saying that doing it was a *bad* thing, I suggested we need things 
to help us do it *better* as a matter of course.  For me, if I'm not 
writing "throwaway" code, I pull type-testing operations out into separate 
functions, because more often than not, the *same* type testing operation 
will be used many times in the overall body of code.  This is just ordinary 
DRY to me, not any special "OO purity".

To be clear: OO purity isn't my goal.  Maintainability, reusability, and 
incremental implementation are the point.

Generic operations win out over interfaces as a fundamental concept because 
if you need the interface you can always make one *from* a set of 
operations.  And a lot of times, you only care about the one operation anyway.

Anyway, I'm beginning to see that the real problem in this discussion is 
that I tend to phrase "analog" ideas in a way that make other people think 
they're "digital" rules.  I've been mostly trying to say that the path of 
implementation (and perhaps teaching) should be:

     generics -> interfaces/typeclasses -> inspection ability

In other words, my references to "necessary" and "unnecessary" are not 
categorical declarations about the nature of programming, but rather are 
references to how "fundamental" or "incremental" the concept is in a 
constructive sense.  You can achieve all the use cases people have 
presented using generics only, without any formal notion of interfaces or 
the ability to do inspection.  (As Guido points out, this may not always be 
*convenient*, but it can be done.)

But the reverse isn't true: you can't really do what generics do, using 
only interfaces.  The generics are a more general solution, with interfaces 
and inspection being merely *convenience* features layered atop the more 
fundamental base.  And since generics are useful to have in their own 
right, it makes more sense (I argue) to start there.

In other words, this isn't some sort of moral judgment regarding the 
"purity" of interfaces or inspection, it's merely a comment about how best 
to *construct* them.  Building interfaces in the Java/Zope style (as mere 
collections of method names) loses you things that you can't just put back 
in later.


From guido at python.org  Wed Nov 22 20:52:16 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 11:52:16 -0800
Subject: [Python-3000] defop ?
In-Reply-To: <456456CF.8000005@gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
Message-ID: <ca471dc20611221152i1744e88bl3818dc01b2752cb0@mail.gmail.com>

[Side remark: I'm sure we would reach understanding and agreement much
quicker if we were all in the same room for a day. Maybe we could try
to have a day at the next PyCon where we hash this out?]

One of the problems that I still have with defop is that it is
presented as pure magic.

On the one hand, it looks like

  defop len(self):
    return 42

is just syntactic sugar for

  def __len__(self):
    return 42

OTOH it seems that it is expected to do something completely different
when someone write

  defop foobar(self):
    ...

where foobar is some user-defined generic function. Can someone who
understands this provide exact syntax and semantics?

Do I understand correctly that the syntax is more like

  'defop' <expr> '(' <args> ')' ':' <suite>

than like

  'defop' NAME '(' <args> ')' ':' <suite>

? (That would need some kind of limitation on the syntax for <expr> so
that it can't contain a call, or else it would be syntactically
ambiguous.)

What kind of object should the expression produce? What methods on it
are we going to call, with what arguments? I'd love to see an
explanation that showed how instead of defop we could also use a
certain decorator. Perhaps

  defop <expr> ( <args> ) : <suite>

is equivalent to

  @defop(<expr>)
  def _ ( <args> ) : <suite>  # The method name is immaterial here

? Then what goes into the defop decorator? And how does this jive with
it being inside a class? (And what would it do if it wasn't?)

I'm assuming that "defop foo.bar (...)" only works if foo is imported
or defined as a global and has a bar attribute.

I'm grasping for understanding here. I don't think Phillip explained
this part of his proposal -- all I can find is examples using built-in
operations (e.g. "defop len" or "defop operator.getitem") and the
handwaving gesture "it could be expanded to add methods for arbitrary
*new* operations not defined by the language". My lack of
understanding is directly related to this support for new operations.

Suppose I wanted to create a flatten operation. What do I put in the
module that defines the 'flatten' generic function? Perhaps


# This is flattening.py
from <somewhere> import generic

@generic
def flatten(obj):
  "Generator that flattens a possibly nested sequence."
  # Default implementation assumes the argument is atomic.
  yield obj

@flatten.when(list)  # Oh how I hate "when" but that's not the point here
def flatten_list(obj):
  for value in obj:
    for v in flatten(value):
      yield v

@flatten[tuple] = flatten_list   # or flatten[list]


Now I have another module where I define a class implementing a binary
tree. I want it to be flattened as the flattening of its items in
in-order.


import flattening

class BinaryTree:

  def __init__(self, value, left=None, right=None):
    # Let's pretend the labels must always be strings
    assert isinstance(value, str)
    # I really want this to be homogeneous, i.e. all nodes are
BinaryTrees or None
    assert left is None or isinstance(left, BinaryTree)
    assert right is None or isinstance(right, BinaryTree)
    self.value = value
    self.left = left
    self.right = right

  def __iter__(self):
    # Recursive in-order iterator
    if self.left is not None:
      for value in self.left: yield value
    yield self.value
    if self.right is not None:
      for value in self.right: yield value

  defop flattening.flatten(self):
    for value in self:
      yield value


I realize that I could also have written

  flattening.flatten[BinaryTree] = flattening.flatten[list]

but I'd like to assume for the sake of argument that the BinaryTree
class wants to define its own flattening code -- for example, since we
know its values are always strings (see the first assert), we don't
need the recursive call to flatten() that's present in flatten_list,
and we really care about performance. Anyway, the point is to show a
class that has its own implementation of some user-defined generic
function.

So I'm still at a loss about how defop works here. I could understand
a different way of defining the flattening of a binary tree:


class BinaryTree:
  ... # as before, but without the defop

# now, outside the class:

import flattening

@flattening.flatten.when(BinaryTree)
def flatten_binary_tree(bt):
  for value in bt:
    yield value


but how on earth is the defop syntax of the @defop decorator going to
generate this?  I can't see any way to implement it without having
access to the class object, but that doesn't exist yet at the time
defop or @defop executes. The best I can think of is for @defop to use
sys._getframe() to access the dict in which the class body is being
evaluated, find or create a list __defop_deferred__ there, and append
the tuple(<expr>, <function>) to it. Then the metaclass (i.e., type)
should look for this list, and if it exists, do the registrations. I'm
guessing you had something like that in mind. I find it pretty fragile
though -- what if defop is used in a different context (not directly
inside a class)? Is that a syntactic error or does it just modify a
random frame's globals, effectively becoming an expensive no-op?

A different line of questioning would be to try to understand how
Phillip's addmethod and hasmethod are supposed to work. I still hope
someone will explain it to me.

--Guido

On 11/22/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Antoine wrote:
> > Is "defop" useful?
> >
> > Why wouldn't it be possible to enable special method lookup with this kind
> > of syntax:
> >
> > iter = generic_function()
> > iter.lookup_special_method("__iter__")
> >
> > And then the classical:
> >
> > class Foo:
> >     def __iter__(self):
> >         pass
>
> A major benefit of the defop syntax lies in the fact that the full Python
> module namespace is available to you for disambiguation of which operation you
> are referring to. So if module A defines an overloaded operation "snap" and
> module B define an overloaded operation "snap", that's OK, because "defop
> A.snap" and "defop B.snap" can be used to make it clear which one you mean.
>
> The magic method namespace, on the other hand, is completely flat: there can
> be only one __snap__ magic method in a given Python application. If two
> different modules both want to use the same magic method name, you're out of
> luck because you can't use the module namespace to disambiguate which one you
> actually want to support (heck, you might even want to support both of them).
>
> A second benefit of defop is that if you have a reference to a generic
> function then you can overload it. With magic methods, you can only overload
> it if you know the appropriate incantation (see the thread about correcting
> the fact that the incantation to overload bool in 2.x is __nonzero__ instead
> of __bool__ - with defop, it could be written simply as "defop bool(self):",
> and issue wouldn't need to rise).
>
> Finally, the defop syntax means a typo will likely result in an exception when
> the class definition is executed instead of at some random later time. For
> example, the typo "def __getiten__(self, key):" will go completely undetected
> when a class definition is executed, but "defop operator.getiten(self, key):"
> would fail with an AttributeError.
>
> That said, it doesn't *have* to be new syntax - an appropriate function
> decorator and a class decorator or metaclass to register the appropriate
> overloads also support this. defop just makes it easier to read and write.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From krstic at solarsail.hcs.harvard.edu  Wed Nov 22 21:07:50 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Wed, 22 Nov 2006 15:07:50 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov22.105531pst."58648"@synergy1.parc.xerox.com>
References: <06Nov22.105531pst."58648"@synergy1.parc.xerox.com>
Message-ID: <4564AE16.5040409@solarsail.hcs.harvard.edu>

Bill Janssen wrote:
> Though, thinking about it some more, I don't see why *all* syntactic
> operations wouldn't just invoke the appropriate normally-named method
> on a specific ABC.  "<", for instance, would presumably invoke
> "object.lessthan" (or perhaps "comparable.lessthan").  

Because it's odd to have these "plumbing" methods in the same namespace
as the "porcelain" methods (the ones that implement the exposed
functionality of the class.)

The desire to keep this separation of namespaces is one of the reasons I
understand Phillip is championing `defop`, though other approaches would
for example include explicit method namespaces.

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From guido at python.org  Wed Nov 22 21:09:31 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 12:09:31 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <-6683921950610559197@unknownmsgid>
References: <-6683921950610559197@unknownmsgid>
Message-ID: <ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>

On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> Talking about the Abilities/Interfaces made me think about some of our
> "rogue" special method names.  In the Language Reference, it says, "A
> class can implement certain operations that are invoked by special
> syntax (such as arithmetic operations or subscripting and slicing) by
> defining methods with special names."  But there are all these methods
> with special names like __len__ or __unicode__ which seem to be
> provided for the benefit of built-in functions, rather than for
> support of syntax.  Presumably in an interface-based Python, these
> methods would turn into regularly-named methods on an ABC, so that
> __len__ would become
>
>   class container:
>     ...
>     def len(self):
>       raise NotImplemented
>
> Though, thinking about it some more, I don't see why *all* syntactic
> operations wouldn't just invoke the appropriate normally-named method
> on a specific ABC.  "<", for instance, would presumably invoke
> "object.lessthan" (or perhaps "comparable.lessthan").  So another
> benefit would be the ability to wean Python away from this
> mangled-name oddness, which seems to me an HCI improvement.

Hm. I'm not sure I agree (figure that :-).

There are two bits of "Python rationale" that I'd like to explain first.

First of all, I chose len(x) over x.len() for HCI reasons (def
__len__() came much later). There are two intertwined reasons
actually, both HCI:

(a) For some operations, prefix notation just reads better than
postfix -- prefix (and infix!) operations have a long tradition in
mathematics which likes notations where the visuals help the
mathematician thinking about a problem. Compare the easy with which we
rewrite a formula like x*(a+b) into x*a + x*b to the clumsiness of
doing the same thing using a raw OO notation.

(b) When I read code that says len(x) I *know* that it is asking for
the length of something. This tells me two things: the result is an
integer, and the argument is some kind of container. To the contrary,
when I read x.len(), I have to already know that x is some kind of
container implementing an interface or inheriting from a class that
has a standard len(). Witness the confusion we occasionally have when
a class that is not implementing a mapping has a get() or keys()
method, or something that isn't a file has a write() method.

Saying the same thing in another way, I see 'len' as a built-in
*operation*. I'd hate to lose that. I can't say for sure whether you
meant that or not, but 'def len(self): ...' certainly sounds like you
want to demote it to an ordinary method. I'm strongly -1 on that.

The second bit of Python rationale I promised to explain is the reason
why I chose special methods to look __special__ and not merely
special. I was anticipating lots of operations that classes might want
to override, some standard (e.g. __add__ or __getitem__), some not so
standard (e.g. pickle's __reduce__ for a long time had no support in C
code at all). I didn't want these special operations to use ordinary
method names, because then pre-existing classes, or classes written by
users without an encyclopedic memory for all the special methods,
would be liable to accidentally define operations they didn't mean to
implement, with possibly disastrous consequences. Ivan Krsti?
explained this more concise in his message, which arrived after I'd
written all this up.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Nov 22 21:16:49 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 12:16:49 -0800
Subject: [Python-3000] __nonzero__ vs. __bool__
In-Reply-To: <45649D48.3090703@livinglogic.de>
References: <bbaeab100611211232r53de3c33p915bee7b49dbdf9a@mail.gmail.com>
	<ek09iv$850$1@sea.gmane.org> <ek125p$36l$1@sea.gmane.org>
	<20061122170432.GA21751@performancedrivers.com>
	<45649D48.3090703@livinglogic.de>
Message-ID: <ca471dc20611221216l369e81c1h1523528919c54192@mail.gmail.com>

Once all who care agree on the patch, feel free to check it in without
my review.

On 11/22/06, Walter D?rwald <walter at livinglogic.de> wrote:
> Jack Diederich wrote:
>
> > On Wed, Nov 22, 2006 at 09:36:09AM +0100, Georg Brandl wrote:
> >> Terry Reedy schrieb:
> >>> "Brett Cannon" <brett at python.org> wrote in message
> >>> news:bbaeab100611211232r53de3c33p915bee7b49dbdf9a at mail.gmail.com...
> >>>
> >>> Why can't the fallback usage just pass the return value from __len__ to
> >>> bool() (forget the C function name) and return that result?  It's just like
> >>> doing::
> >>>
> >>>   def bool(obj):
> >>>       try:
> >>>           return obj.__bool__()
> >>>       except AttributeError:
> >>>           return bool(len(obj))
> >>> ------------
> >>>
> >>> If an object without __bool__ returned itself as its length, this would be
> >>> an infinite loop, at least in this Python version.  Do we worry about
> >>> something so crazy?
> >> The length would have to be an integer, and this would have to be checked.
> >>
> >
> > It looks like the regular checks are happening on __len__ methods
> > anyway so the explicit int check in slot_nb_bool is redundant.
> > This is the first time I've looked at the new slots in py3k so
> > feel free to correct. (using bool4.patch)
> >
> > sprat:~/src/py3k-rw> ./python
> > Python 3.0x (p3yk:52823M, Nov 22 2006, 11:57:34)
> > [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> class A(object):
> >   def __len__(self):
> >     return -1
> >
> > a = A()
> > print bool(a)
> >   ... ... ... >>> >>> Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> > ValueError: __len__() should return >= 0
>
> OK, I've updated the patch: bugs.python.org/1600346
>
> Servus,
>    Walter
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Nov 22 21:28:36 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 12:28:36 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221228h2579d281u38d939fc33ab8fd4@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> In other words, this isn't some sort of moral judgment regarding the
> "purity" of interfaces or inspection, it's merely a comment about how best
> to *construct* them.  Building interfaces in the Java/Zope style (as mere
> collections of method names) loses you things that you can't just put back
> in later.

I'm not sure that it's right to describe Java/Zope interfaces as "mere
collections of method names". For example, Java's collections
framework (http://java.sun.com/j2se/1.4.2/docs/guide/collections/)
ascribes a lot of semantics to its interfaces that aren't expressible
by method signatures alone, and strongly urges implementations to
implement the right semantics (of course some folks cheat, but that
doesn't invalidate the point).

So maybe there are even less differences between the different POVs.

I'm not arguing that generics aren't strictly more powerful than
interfaces -- I just think that interfaces don't necessarily deserve
the bad rep you're creating for them, and I'd like to have interfaces
prominently in Py3k's toolbox (as well as generics) rather than having
to hide them in the closet.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gustavo at niemeyer.net  Wed Nov 22 21:29:24 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Wed, 22 Nov 2006 18:29:24 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <20061122202924.GA21361@niemeyer.net>

> 2. Do we need interfaces if we have generic functions?
> 
> Phillip Eby argues that we don't need interfaces, because the right
> API for generic functions makes it easy to do what we do with
> interfaces. Let me paraphrase his argument.

I agree that we don't *need* interfaces to have generic functions,
just like we don't need them to have adaptation.  Using your
example, consider Flatten as a base class, and F as an adapter
registry containing adapters Fi from C to Flatten.  It's feasible.

Interfaces provide greater value for both approaches.  A short
usage case, shortened from the other thread.

"""
Let's say that you create a library named libfruit, and that it
implements a very nice IFruit interface.  This library is so nice
and shiny that it becomes widely used, and thus many ways to handle
a fruit are developed by third parties.  Now, someone else has a
libapple library, which unfortunately is a bit outdated, and wasn't
designed to work with the shiny libfruit library.

With interfaces, all you need to do is define an adapter from Apple
to IFruit, and instantly all the widely available frameworks will
be handling apples just fine.
"""

Without interfaces, we won't be able to say "in all places, use the
generic method registered for an IFruit whenever an Apple is seen".

Using generic functions, the same is true.  The only difference is
that with generic functions each "adapter registry" is capable of
adapting only to a single "interface", thus removing the need to
explicitly declare it.


> 3. API design -- how do we spell the various concepts? E.g.
> has_ability(x, A) asks whether object x has the ability A;
> provides_ability(C, A) asks whether class C provides the ability A to
> its instances.

I'm sure you're aware about it, but in Zope 3 terminology, these
are 'provide' and 'implement', respectively.


> - We need to come up with syntax for the common case, declaring a
> class that claims to provide some abilities. Some thoughts: putting
(...)

Have you considered class decorators?

  @implements(FooBar)
  class Baz:
     ...

-- 
Gustavo Niemeyer
http://niemeyer.net

From fredrik at pythonware.com  Wed Nov 22 21:38:00 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 22 Nov 2006 21:38:00 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
Message-ID: <ek2cf8$su5$1@sea.gmane.org>

Guido van Rossum wrote:

 > Ivan Krsti? explained this more concise in his message, which arrived
 > after I'd written all this up.

which is a good thing, because your post is a much better version of
the corresponding FAQ entry than what we came up with when this was last 
discussed in this thread (last week, I think?).  I hope you don't mind 
me stealing it ;-)

cheers /F


From guido at python.org  Wed Nov 22 21:38:45 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 12:38:45 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061122202924.GA21361@niemeyer.net>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<20061122202924.GA21361@niemeyer.net>
Message-ID: <ca471dc20611221238o60c1a4b3rca74b07036d361fa@mail.gmail.com>

On 11/22/06, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
> > 2. Do we need interfaces if we have generic functions?
> >
> > Phillip Eby argues that we don't need interfaces, because the right
> > API for generic functions makes it easy to do what we do with
> > interfaces. Let me paraphrase his argument.
>
> I agree that we don't *need* interfaces to have generic functions,
> just like we don't need them to have adaptation.  Using your
> example, consider Flatten as a base class, and F as an adapter
> registry containing adapters Fi from C to Flatten.  It's feasible.
>
> Interfaces provide greater value for both approaches.  A short
> usage case, shortened from the other thread.

Right.

> """
> Let's say that you create a library named libfruit, and that it
> implements a very nice IFruit interface.  This library is so nice
> and shiny that it becomes widely used, and thus many ways to handle
> a fruit are developed by third parties.  Now, someone else has a
> libapple library, which unfortunately is a bit outdated, and wasn't
> designed to work with the shiny libfruit library.
>
> With interfaces, all you need to do is define an adapter from Apple
> to IFruit, and instantly all the widely available frameworks will
> be handling apples just fine.
> """
>
> Without interfaces, we won't be able to say "in all places, use the
> generic method registered for an IFruit whenever an Apple is seen".

Well, this assumes automatic adaptation (which is off the table), *or*
everything that uses IFruits must always explicitly invoke some
adapter, rather than just assuming that it is passed an IFruit. Or it
only works for generic functions that are declared as taking IFruit --
there are many other situations where an IFruit could pop up without
directly being the argument of some (generic) function.

Consider container types, which haven't figured prominently in this
thread yet. Suppose I have a function that expects a sequence of
IFruits. Now I have a sequence of Apples. I don't think there's a
reasonable mechanism that lets me pass this to that function; I will
need to map it to a sequence of IFruits explicitly.

> Using generic functions, the same is true.  The only difference is
> that with generic functions each "adapter registry" is capable of
> adapting only to a single "interface", thus removing the need to
> explicitly declare it.

You've lost me here. A less abstract example might work better.

> > 3. API design -- how do we spell the various concepts? E.g.
> > has_ability(x, A) asks whether object x has the ability A;
> > provides_ability(C, A) asks whether class C provides the ability A to
> > its instances.
>
> I'm sure you're aware about it, but in Zope 3 terminology, these
> are 'provide' and 'implement', respectively.

I like provide, but I'm not so sure about implement, since it is
awfully ambiguous -- most of the time it is the class that does the
implementing. That's why I settled for "has".

> > - We need to come up with syntax for the common case, declaring a
> > class that claims to provide some abilities. Some thoughts: putting
> (...)
>
> Have you considered class decorators?
>
>   @implements(FooBar)
>   class Baz:
>      ...

No, I'm still waiting for someone to write a PEP for those. But as far
as ideas go it's a fine idea.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From solipsis at pitrou.net  Wed Nov 22 21:40:53 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 22 Nov 2006 21:40:53 +0100
Subject: [Python-3000] defop ?
In-Reply-To: <ca471dc20611221152i1744e88bl3818dc01b2752cb0@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<ca471dc20611221152i1744e88bl3818dc01b2752cb0@mail.gmail.com>
Message-ID: <1164228053.4674.54.camel@fsol>

Le mercredi 22 novembre 2006 ? 11:52 -0800, Guido van Rossum a ?crit :
> but how on earth is the defop syntax of the @defop decorator going to
> generate this?  I can't see any way to implement it without having
> access to the class object, but that doesn't exist yet at the time
> defop or @defop executes. The best I can think of is for @defop to use
> sys._getframe() to access the dict in which the class body is being
> evaluated, find or create a list __defop_deferred__ there, and append
> the tuple(<expr>, <function>) to it. Then the metaclass (i.e., type)
> should look for this list, and if it exists, do the registrations.

Let's say @defop is implemented as follows:

class DefOpDescriptor:
    def __init__(self, genfunc, implfunc):
        self.genfunc = genfunc
        self.implfunc = implfunc
    def __call__(self, *args, **kargs):
        # Directly call the implementation (why not)
        self.implfunc(*args, **kargs)
    def __with_class__(self, cls):
        """ Register self.implfunc as an implementation
        of generic function self.genfunc for class cls. """
        self.genfunc.when(cls)(self.implfunc)

def defop(genfunc):
    def decorate(implfunc):
        return DefOpDescriptor(genfunc, implfunc)
    return decorate


And then in the metaclass (i.e., type), if an attribute "attr" has a
__with_class__ method, execute attr.__with_class__(cls).

(this could perhaps be used for other things than defop)



From guido at python.org  Wed Nov 22 21:42:53 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 12:42:53 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ek2cf8$su5$1@sea.gmane.org>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
Message-ID: <ca471dc20611221242m7094ab82t179a544f2d5fd959@mail.gmail.com>

You're welcome.

It did occur to me that writing up that particular piece of rationale
would be of value outside this thread -- I had no idea it would be
picked up so quickly. :-)

On 11/22/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
>  > Ivan Krsti? explained this more concise in his message, which arrived
>  > after I'd written all this up.
>
> which is a good thing, because your post is a much better version of
> the corresponding FAQ entry than what we came up with when this was last
> discussed in this thread (last week, I think?).  I hope you don't mind
> me stealing it ;-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Nov 22 21:47:29 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 12:47:29 -0800
Subject: [Python-3000] defop ?
In-Reply-To: <1164228053.4674.54.camel@fsol>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<ca471dc20611221152i1744e88bl3818dc01b2752cb0@mail.gmail.com>
	<1164228053.4674.54.camel@fsol>
Message-ID: <ca471dc20611221247r42ec30eo4a6d3ab36c12fd72@mail.gmail.com>

On 11/22/06, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le mercredi 22 novembre 2006 ? 11:52 -0800, Guido van Rossum a ?crit :
> > but how on earth is the defop syntax of the @defop decorator going to
> > generate this?  I can't see any way to implement it without having
> > access to the class object, but that doesn't exist yet at the time
> > defop or @defop executes. The best I can think of is for @defop to use
> > sys._getframe() to access the dict in which the class body is being
> > evaluated, find or create a list __defop_deferred__ there, and append
> > the tuple(<expr>, <function>) to it. Then the metaclass (i.e., type)
> > should look for this list, and if it exists, do the registrations.
>
> Let's say @defop is implemented as follows:
>
> class DefOpDescriptor:
>     def __init__(self, genfunc, implfunc):
>         self.genfunc = genfunc
>         self.implfunc = implfunc
>     def __call__(self, *args, **kargs):
>         # Directly call the implementation (why not)
>         self.implfunc(*args, **kargs)

I'm not sure how/when __call__ would be used.

Shouldn't you implement a __get__ descriptor instead?

>     def __with_class__(self, cls):
>         """ Register self.implfunc as an implementation
>         of generic function self.genfunc for class cls. """
>         self.genfunc.when(cls)(self.implfunc)
>
> def defop(genfunc):
>     def decorate(implfunc):
>         return DefOpDescriptor(genfunc, implfunc)
>     return decorate
>
>
> And then in the metaclass (i.e., type), if an attribute "attr" has a
> __with_class__ method, execute attr.__with_class__(cls).
>
> (this could perhaps be used for other things than defop)

Aha, cool. You forgot to add that now the method name chosen must be
unique. This works fine to describe @defop, and I like it better than
using sys._getframe(). But it doesn't really work for the defop
syntax, unless that makes up a unique (random?) name for each
occurrence.

I guess there's also the issue of invoking it in the wrong context.
Maybe that should be a pychecker issue.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Nov 22 21:15:11 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 15:15:11 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221043q5ab8c55vdb63ed41e1bf7d5f@mail.gmail.com
 >
References: <5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>

At 10:43 AM 11/22/2006 -0800, Guido van Rossum wrote:
>There is a dangerous amount of content-free rhetoric in this thread.

Actually, I think that some of my statements are being misunderstood as 
such, when I'm intending only to describe *how* interfaces (or inspection) 
can be *derived* from generic functions.  When I say that these other 
concepts are "unnecessary", I mean that they can be derived from generics, 
and that the reverse is not true.

This isn't rhetoric, IMO, and it's not about religion or purity, but about 
practicalities.  Often, an interface consists of just one operation, so 
treating interfaces as fundamental introduces additional "unnecessary" 
complexity in those cases, whereas being able to treat that one-operation 
interface as a single generic function removes the need for interfaces -- 
even as a *concept* to be understood, let alone implemented.  That's the 
power of Python's existing generics: they don't rely on you "implementing 
the iterable interface", but just on defining the iter() *operation*.


>As I've tried to mention through examples, there are plenty of
>situations where a carefully placed if-then is a lot more practical
>than adding a new method to some remote class (or several).

Absolutely.  I am saying only that the inspection can be implemented *in 
terms of* generic functions.


>But even the best generic function API I've seen is a lot
>more verbose than this -- there seems to be a separate set-up
>involved.

Actually, my 'defop' proposal would allow you to do this:

     def sendmail(...):
         # default case

     defop sendmail(...):
         # special case, w/additional type info in arg signature

The first statement creates a function called 'sendmail', and the 
subsequent statement overloads it.  Of course, I suppose 'defop' could also 
be spelled differently to emphasize the difference, e.g.:

     def sendmail(...):
         # default case

     overload sendmail(...):
         # special case, w/additional type info in arg signature


From pje at telecommunity.com  Wed Nov 22 21:52:21 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 15:52:21 -0500
Subject: [Python-3000] defop ?
In-Reply-To: <ca471dc20611221152i1744e88bl3818dc01b2752cb0@mail.gmail.co
 m>
References: <456456CF.8000005@gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
Message-ID: <5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>

At 11:52 AM 11/22/2006 -0800, Guido van Rossum wrote:
>Do I understand correctly that the syntax is more like
>
>   'defop' <expr> '(' <args> ')' ':' <suite>
>
>than like
>
>   'defop' NAME '(' <args> ')' ':' <suite>
>
>? (That would need some kind of limitation on the syntax for <expr> so
>that it can't contain a call, or else it would be syntactically
>ambiguous.)

Yes - it should be the same as for decorators or imports, i.e. a possibly 
dotted name.


>What kind of object should the expression produce? What methods on it
>are we going to call, with what arguments?

The defop statement (and its decorators if any) would translate to this code:

     @<decorators>
     def <anonfunc>(<args>):
         <suite>

     addmethod(<expr>, <anonfunc>, enclosing_class_or_None)

In other words, addmethod() is invoked following completion of the class 
suite (if there is one), and passed the decorated anonymous function object.

Now, we could say that addmethod() just calls expr.__addmethod__.  This 
would work as long as each builtin had its own __addmethod__() that did the 
'cls.__special__ = method' work.

However, I personally find it easier to just think of addmethod() as a 
generic function, so that you can define methods for it that recognize the 
builtin generics and do the right thing.  This also doesn't require the 
builtins to grow new methods, and thus can be implemented in today's Python 
using only Python code.  (Like the example that you described as 
brain-exploding orange smoke, which was an actual implementation of 
addmethod that would work in today's Python 2.x)

But if it's easier for you to conceptualize, let's just say it expects expr 
to have an __addmethod__ method.


>  I'd love to see an
>explanation that showed how instead of defop we could also use a
>certain decorator. Perhaps
>
>   defop <expr> ( <args> ) : <suite>
>
>is equivalent to
>
>   @defop(<expr>)
>   def _ ( <args> ) : <suite>  # The method name is immaterial here
>
>? Then what goes into the defop decorator? And how does this jive with
>it being inside a class? (And what would it do if it wasn't?)

An implementation for today's Python:

     from peak.util.decorators import decorate_class

     def defop(expr):
         def decorate(func):
             def do_add(cls):
                 addmethod(expr, func, cls)
                 return cls
             try:
                 decorate_class(do_add)
             except SyntaxError:   # not in a class
                 do_add(None)      # handle top-level function
         return decorate

(The 'decorate_class' function applies its argument as a class decorator 
for the class its caller is invoked from.  This is the same mechanism (and 
actual code!) that Zope uses to do in-body class decorators like 
'implements()'.)  decorate_class raises a SyntaxError if its caller is not 
being run directly inside a class body.

And yes, this uses frame magic, and no, it's not fragile, as it knows the 
difference between class and non-class frames.  The implementation has been 
in Zope for a good few years now, likewise Twisted and PEAK.  It's also 
available as a standalone package from:

     http://cheeseshop.python.org/pypi/DecoratorTools


>but how on earth is the defop syntax of the @defop decorator going to
>generate this?

     @defop(flattening.flatten)
     def flatten_btree(bt:BinaryTree):
         # ... do stuff to bt

Of course, this assumes that flattening.flatten.__addmethod__() is smart 
enough to pull the type information off the first argument, if the third 
addmethod() argument is None.


>A different line of questioning would be to try to understand how
>Phillip's addmethod and hasmethod are supposed to work. I still hope
>someone will explain it to me.

addmethod registers a method with a generic function.  The only reason we 
even need it as a separate function (as opposed to just assuming a 
__hasmethod__) is to allow retrofitting of existing Python objects (such as 
iter/len/etc.) as generic functions via __special__ method setting.  The 
same applies to hasmethod vs. just using __contains__ or something of that 
sort.

If you want to treat it as a Py3K only thing and have __hasmethod__ and 
__addmethod__ slots for builtin functions, then there's no need to have 
actual addmethod/hasmethod functions.  But that's sort of like saying we 
don't need getattr() any more because we can just use 
type(ob).__getattribute__() now!  :)


From pje at telecommunity.com  Wed Nov 22 21:54:20 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 15:54:20 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221228h2579d281u38d939fc33ab8fd4@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122154127.03f77708@sparrow.telecommunity.com>

At 12:28 PM 11/22/2006 -0800, Guido van Rossum wrote:
>So maybe there are even less differences between the different POVs.

Yep.


>I'm not arguing that generics aren't strictly more powerful than
>interfaces -- I just think that interfaces don't necessarily deserve
>the bad rep you're creating for them, and I'd like to have interfaces
>prominently in Py3k's toolbox (as well as generics) rather than having
>to hide them in the closet.

My point is that I'm not trying to create a "bad rep", I'm just emphasizing 
that it's easier/more useful to define interfaces as collections of generic 
operations, because it makes the simple cases simple and powerful, and the 
complex ones possible.  Doing it the other way around makes all of the 
cases more complex, and loses other capabilities.

My use of the word "unnecessary" isn't a moral judgment of interfaces, it's 
an argument about how best to layer the various concepts we want to 
have.  Interfaces aren't necessary for simple things, and it's better (IMO) 
if we define them *in terms of* simpler things, because then people don't 
have to learn about them unless they need something more complex.

And if they need something more complex, it is also more likely (I believe) 
that they will have more specialized needs.  So, I intended to suggest that 
if we just do the basics (operations) and allow others to define interface 
frameworks atop that, then we can avoid overgeneralizing from insufficient 
experience.  However, in the expanded discussion it seems to me that you 
have goals for additional abilities you'd like to see sooner rather than later.

What worries me about Java-style interfaces is that you can't split and 
recombine them in arbitrary ways, without effectively decomposing them into 
single-operation interfaces using multiple inheritance.  At which point, 
you just might as well have treated them as operations to start 
with!  Thus, taking an operation-based approach means there need not be a 
concept of the "one true sequence interface", just the *default* sequence 
interface.  If my needs are more or less expansive than the default 
"sequence" interface, I should be able to do something like (sequence.len, 
sequence.iter) and recombine a new interface.

This would also make the interface committee's job at least somewhat 
easier.  :)


From pje at telecommunity.com  Wed Nov 22 21:59:32 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 15:59:32 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061122202924.GA21361@niemeyer.net>
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061122155546.03f6c7c0@sparrow.telecommunity.com>

At 06:29 PM 11/22/2006 -0200, Gustavo Niemeyer wrote:
>Without interfaces, we won't be able to say "in all places, use the
>generic method registered for an IFruit whenever an Apple is seen".

...for some definition of "interface".  :)

Dylan has "protocols", Haskell has "typeclasses", and they do what you're 
describing.  The term "interface" in this discussion has been heavily 
linked to Java/Zope-style interfaces, however, as opposed to these other 
styles of "interface".  To be clear, I'm opposing a particular style most 
commonly associated with the word "interface", *not* to the idea of having 
some way to describe a multitude of types.


From guido at python.org  Wed Nov 22 22:00:52 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 13:00:52 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221300p129551fft51fc742a115a1099@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:43 AM 11/22/2006 -0800, Guido van Rossum wrote:
> >There is a dangerous amount of content-free rhetoric in this thread.
>
> Actually, I think that some of my statements are being misunderstood as
> such, when I'm intending only to describe *how* interfaces (or inspection)
> can be *derived* from generic functions.  When I say that these other
> concepts are "unnecessary", I mean that they can be derived from generics,
> and that the reverse is not true.
>
> This isn't rhetoric, IMO, and it's not about religion or purity, but about
> practicalities.  Often, an interface consists of just one operation, so
> treating interfaces as fundamental introduces additional "unnecessary"
> complexity in those cases, whereas being able to treat that one-operation
> interface as a single generic function removes the need for interfaces --
> even as a *concept* to be understood, let alone implemented.  That's the
> power of Python's existing generics: they don't rely on you "implementing
> the iterable interface", but just on defining the iter() *operation*.
>
>
> >As I've tried to mention through examples, there are plenty of
> >situations where a carefully placed if-then is a lot more practical
> >than adding a new method to some remote class (or several).
>
> Absolutely.  I am saying only that the inspection can be implemented *in
> terms of* generic functions.
>
>
> >But even the best generic function API I've seen is a lot
> >more verbose than this -- there seems to be a separate set-up
> >involved.
>
> Actually, my 'defop' proposal would allow you to do this:
>
>      def sendmail(...):
>          # default case
>
>      defop sendmail(...):
>          # special case, w/additional type info in arg signature
>
> The first statement creates a function called 'sendmail', and the
> subsequent statement overloads it.  Of course, I suppose 'defop' could also
> be spelled differently to emphasize the difference, e.g.:
>
>      def sendmail(...):
>          # default case
>
>      overload sendmail(...):
>          # special case, w/additional type info in arg signature

It's important for my understanding to show what's on the dots and
also how defop is implemented. See another thread. Right now, while
not rhetoric, the above examples are indistinguishable from magic to
me (I hope you recognize the quote).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jan.grant at bristol.ac.uk  Wed Nov 22 22:14:42 2006
From: jan.grant at bristol.ac.uk (Jan Grant)
Date: Wed, 22 Nov 2006 21:14:42 +0000 (GMT)
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220755s23b4af05l967e30cf8ca1d022@mail.gmail.com>
References: <ca471dc20611220701u4f837661i65ed1a667de426a9@mail.gmail.com>
	<000501c70e4a$fcb29800$6402a8c0@arkdesktop>
	<ca471dc20611220755s23b4af05l967e30cf8ca1d022@mail.gmail.com>
Message-ID: <20061122210226.O12950@tribble.ilrt.bris.ac.uk>

On Wed, 22 Nov 2006, Guido van Rossum wrote:

> On 11/22/06, Andrew Koenig <ark-mlist at att.net> wrote:
> > > Hm, I would think it extends very naturally to instance variables, and
> > > I see no reason why it's a particularly bad idea to also extend it
> > > (mostly in our minds anyway, I expect) to describe certain behaviors
> > > that cannot be expressed (easily) in code.
> >
> > I think of an ability as a property of an interface rather than an interface
> > itself.  For example, I can imagine a single interface having multiple
> > abilities.
> 
> Hm, that sounds like a case for inheritance. The abilities would be
> the bases and the interface would extend all of them. They could all
> be called abilities or they could all be called interfaces. The
> linguistic metaphor only goes so far...

I don't think it's quite inheritance (except in the set-theoretic 
sense), unless you can define new superclasses to existing interfaces.

To illustrate, let's say that python can take "typed" arguments. You 
write a function, foo, that has one argument. That argument must have 
two separate abilities: let's say (for argument's sake) that we want to 
iterate over its contents (bad choice, perhaps, but bear with me) and 
also persist it. How do we declare the function?

	interface interable_and_serialisable(iterable, serialisable):
		pass

	def foo(x : iterable_and_serialisable):
		etc.

That's great, but if we want to call foo(x) for an x that comes from a 
third-party library? Perhaps the library owner even was kind enough to 
declare that x implemented "iterable" and "serialisable". We've invented 
"iterable_and_serialisable" after the library was written. Unless we can 
go back and declare that the class of x implements that interface, 
there's an impasse. Fine-grained "abilities" lead to some kind of simple 
ability algebra or other hoop-jumping if you want to do typing of 
arguments.

Of course, without typed arguments we can simply check that x implements 
iterable and serialisable inside the body of "foo".

(Perhaps this is a non-problem: it certainly doesn't appear to crop up 
much in the Java world.)

jan

-- 
jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/
Tel +44 (0)117 3317661   http://ioctl.org/jan/
Hang on, wasn't he holding a wooden parrot? No! It was a porcelain owl.

From pje at telecommunity.com  Wed Nov 22 22:25:03 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 16:25:03 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221300p129551fft51fc742a115a1099@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>

At 01:00 PM 11/22/2006 -0800, Guido van Rossum wrote:
>On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
>>The first statement creates a function called 'sendmail', and the
>>subsequent statement overloads it.  Of course, I suppose 'defop' could also
>>be spelled differently to emphasize the difference, e.g.:
>>
>>      def sendmail(...):
>>          # default case
>>
>>      overload sendmail(...):
>>          # special case, w/additional type info in arg signature
>
>It's important for my understanding to show what's on the dots and
>also how defop is implemented. See another thread.

The defop questions are now answered in that thread.  Regarding the dots, I 
meant them to refer to the Java-style overload example that came 
previously, assuming the use of a multi-dispatch overload implementation 
like your type-tuple prototype of earlier this year, using the optional 
type declaration syntax to describe the overloaded properties.  IOW:

     def sendmail(self, from_addr, to_addrs, msg, mail_options=[],
                  rcpt_options=[]):
         # default implementation

     overload sendmail(self, from_addr, to_addrs:str, msg,
                       mail_options=[], rcpt_options=[]):
         self.sendmail(from_addr, [to_addrs], msg, mail_options, rcpt_options)

While this is more verbose than sticking an 'if' into the original 
sendmail(), note that it can also be added by a third party, via:

     overload smtpblib.SMTP.sendmail(
         self, from_addr, to_addrs:str, msg,
         mail_options=[], rcpt_options=[]
     ):
         self.sendmail(from_addr, [to_addrs], msg, mail_options, rcpt_options)

So, someone who didn't want to wait for the patch to add your "if" could 
just add their own fix on the fly.  :)


>  Right now, while
>not rhetoric, the above examples are indistinguishable from magic to
>me (I hope you recognize the quote).

The problem I have sometimes in making my proposals clear to you is that, 
at the level of abstraction I think on, there are often multiple ways to 
carry out a useful/usable implementation, but you seem to want to 
understand one *particular* implementation, even though I'm trying to 
propose an approach to the solution, rather than a particular solution.

However, when I present you with a particular implementation, you then pick 
holes in the implementation itself, or the tradeoffs made to render the 
example simple.  If I knew what tradeoffs you would prefer ahead of time, 
then I could make my examples match the desired tradeoffs, but in the 
absence of that information I can only fall back to describing a general 
concept.  So we tend to go back and forth between abstractions rejected for 
lack of specificity, and specific things rejected for their particular 
nature.  :)  Perhaps I should try giving implementations explicitly 
labelled as prototypes and stating that there are other ways to do the same 
thing?


From gsakkis at rutgers.edu  Wed Nov 22 22:48:11 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 22 Nov 2006 16:48:11 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ek2cf8$su5$1@sea.gmane.org>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
Message-ID: <91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>

On 11/22/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
>  > Ivan Krsti? explained this more concise in his message, which arrived
>  > after I'd written all this up.
>
> which is a good thing, because your post is a much better version of
> the corresponding FAQ entry than what we came up with when this was last
> discussed in this thread (last week, I think?).  I hope you don't mind
> me stealing it ;-)
>
> cheers /F


Although I don't necessarily agree with the arguments from a puristic
point of view ("then why not make keys(), values(), items(), read(),
write(), ... builtin GFs ?"), they are indeed a better answer to the
FAQ entry. After all, HCI-based arguments are usually a fancier way of
saying "it's a matter of taste" and arguing against them is like
arguing that green is a better colour than red.

George

From janssen at parc.com  Wed Nov 22 23:05:09 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 14:05:09 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
Message-ID: <06Nov22.140512pst."58648"@synergy1.parc.xerox.com>

> Saying the same thing in another way, I see 'len' as a built-in
> *operation*. I'd hate to lose that. I can't say for sure whether you
> meant that or not, but 'def len(self): ...' certainly sounds like you
> want to demote it to an ordinary method. I'm strongly -1 on that.

OK, I can see that.  So the built-in function len() and its cousins
should be thought of as special Python syntax.  I see no real reason
to remove that; if "<" just invokes a standard method
"orderable.lessthan", the built-in len can just invoke
"container.len".  But foo.len() would also be available for use.

> I didn't want these special operations to use ordinary
> method names, because then pre-existing classes, or classes written by
> users without an encyclopedic memory for all the special methods,
> would be liable to accidentally define operations they didn't mean to
> implement, with possibly disastrous consequences.

Hmmm.  This seems kind of a weak rationale to me, but OK.

Bill

From guido at python.org  Wed Nov 22 23:06:33 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 14:06:33 -0800
Subject: [Python-3000] defop ?
In-Reply-To: <5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221406o562f9f4du700992125bfab8c2@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> The defop statement (and its decorators if any) would translate to this code:
>
>      @<decorators>
>      def <anonfunc>(<args>):
>          <suite>
>
>      addmethod(<expr>, <anonfunc>, enclosing_class_or_None)

Where is it gonna get the enclosing class? That object hasn't been
created yet at the time defop executes.

> In other words, addmethod() is invoked following completion of the class
> suite (if there is one), and passed the decorated anonymous function object.

> Now, we could say that addmethod() just calls expr.__addmethod__.  This
> would work as long as each builtin had its own __addmethod__() that did the
> 'cls.__special__ = method' work.

OK, this I can understand. Thanks for being concrete.

> However, I personally find it easier to just think of addmethod() as a
> generic function,

yes, but that's you've had generic functions on your brain for a year
or more now. For the rest of us, saying "it calls expr.__addmethod__"
is a lot easier to process. And you've already shown earlier that
these two are pretty much equivalent. :-)

> so that you can define methods for it that recognize the
> builtin generics and do the right thing.  This also doesn't require the
> builtins to grow new methods, and thus can be implemented in today's Python
> using only Python code.  (Like the example that you described as
> brain-exploding orange smoke, which was an actual implementation of
> addmethod that would work in today's Python 2.x)

Working code can explode heads too. :-)

> But if it's easier for you to conceptualize, let's just say it expects expr
> to have an __addmethod__ method.

Yes, thank you, this helps.

> An implementation for today's Python:
>
>      from peak.util.decorators import decorate_class
>
>      def defop(expr):
>          def decorate(func):
>              def do_add(cls):
>                  addmethod(expr, func, cls)
>                  return cls
>              try:
>                  decorate_class(do_add)
>              except SyntaxError:   # not in a class
>                  do_add(None)      # handle top-level function
>          return decorate

Alas, this is all smoke and mirrors again for me, since I don't know
what decorate_class does, or why on earth it would raise SyntaxError.

How does this address the problem that the class doesn't yet exist?

> (The 'decorate_class' function applies its argument as a class decorator
> for the class its caller is invoked from.  This is the same mechanism (and
> actual code!) that Zope uses to do in-body class decorators like
> 'implements()'.)  decorate_class raises a SyntaxError if its caller is not
> being run directly inside a class body.

That's a bizarre use of that exception.

I'm guessing that decorate_class squirrels away some info for later?
Everything here seems designed to hide the implementation details,
which is great for actual use, but kills understanding of the
mechanism. :-(

> And yes, this uses frame magic, and no, it's not fragile, as it knows the
> difference between class and non-class frames.

Wow. How can you tell? Does it work with Jython, IronPython and PyPy?

I still find it fragile somehow -- or at least severely hackish. I
like the other proposal (where it puts a special attribute on the
function object which is then found by the type) better, it doesn't
require sys._getframe().

> The implementation has been
> in Zope for a good few years now, likewise Twisted and PEAK.  It's also
> available as a standalone package from:
>
>      http://cheeseshop.python.org/pypi/DecoratorTools
>
>
> >but how on earth is the defop syntax of the @defop decorator going to
> >generate this?
>
>      @defop(flattening.flatten)
>      def flatten_btree(bt:BinaryTree):
>          # ... do stuff to bt

I'm assuming that's outside the context of the BinaryTree class --
since inside it, Binarytree is undefined.

> Of course, this assumes that flattening.flatten.__addmethod__() is smart
> enough to pull the type information off the first argument, if the third
> addmethod() argument is None.

Assuming we have optional argument annotations that would be nice.

> >A different line of questioning would be to try to understand how
> >Phillip's addmethod and hasmethod are supposed to work. I still hope
> >someone will explain it to me.
>
> addmethod registers a method with a generic function.  The only reason we
> even need it as a separate function (as opposed to just assuming a
> __hasmethod__) is to allow retrofitting of existing Python objects (such as
> iter/len/etc.) as generic functions via __special__ method setting.  The
> same applies to hasmethod vs. just using __contains__ or something of that
> sort.

Hm, I think this is the first time in this thread that you meant to
use 'iter', 'len' etc. as "tokens". I think I'd rather change them a
bit so that they can be objects that actually understand __addmethod__
and __hasmethod__.

> If you want to treat it as a Py3K only thing

I do, that's why we're discussing it here.

> and have __hasmethod__ and
> __addmethod__ slots for builtin functions,

I have no problem with making those builtins that are generic
functions into instances of some other class. I don't think all 50+
builtins should become generic methods.

> then there's no need to have
> actual addmethod/hasmethod functions.  But that's sort of like saying we
> don't need getattr() any more because we can just use
> type(ob).__getattribute__() now!  :)

But I think you're really proposing an addmethod builtin that calls an
__addmethod__ method, right? That sounds fine to me if we can work out
the rest of the details.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Wed Nov 22 23:22:08 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 22 Nov 2006 23:22:08 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov22.140512pst."58648"@synergy1.parc.xerox.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<06Nov22.140512pst."58648"@synergy1.parc.xerox.com>
Message-ID: <ek2iig$gni$1@sea.gmane.org>

Bill Janssen wrote:

 > This seems kind of a weak rationale to me, but OK.

yeah, namespaces are such a useless thing.  let's put everything in one 
big flat namespace, so we don't have to wonder where things are.

</F>


From fredrik at pythonware.com  Wed Nov 22 23:26:33 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 22 Nov 2006 23:26:33 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
Message-ID: <ek2iqp$hsi$1@sea.gmane.org>

George Sakkis wrote:

> After all, HCI-based arguments are usually a fancier way of
> saying "it's a matter of taste"

It would be easier to take you seriously if you gave up that "I don't 
understand their arguments; therefore I'm right and they're stupid" 
style of argumentation.  Mike and others said *exactly* the same thing 
as Guido, but that time, you just didn't want to listen.

</F>


From gustavo at niemeyer.net  Wed Nov 22 23:33:56 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Wed, 22 Nov 2006 20:33:56 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221238o60c1a4b3rca74b07036d361fa@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<20061122202924.GA21361@niemeyer.net>
	<ca471dc20611221238o60c1a4b3rca74b07036d361fa@mail.gmail.com>
Message-ID: <20061122223356.GB22089@niemeyer.net>

> Well, this assumes automatic adaptation (which is off the table), *or*
> everything that uses IFruits must always explicitly invoke some
(...)
> You've lost me here. A less abstract example might work better.

Sorry.. I should have provided a good concrete example in
the first place.  Here it goes.

implements(Apple, IFruit)
implements(Orange, IFruit)

With adapters:

  registry = AdapterRegistry()
  registry.register(IFruit, IJuice, fruit_to_juice)
  registry.register(IFruit, IJelly, fruit_to_jelly)
  apple_juice = registry.query(apple, IJuice)
  orange_jelly = registry.query(orange, IJelly)

With generic functions:

  @generic
  juice(obj): return None
  juice.register(IFruit)(fruit_to_juice)
  @generic
  jelly(obj): return None
  jelly.register(IFruit)(fruit_to_jelly)
  apple_juice = juice(apple)
  orange_jelly = jelly(orange)

Notice the subtle difference between them.  The "target" interface
is gone with generic functions.  Think of how both would be handled
without interfaces.  It can be done for sure, but interfaces do
provide additional value.
 
-- 
Gustavo Niemeyer
http://niemeyer.net

From fredrik at pythonware.com  Wed Nov 22 23:50:57 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 22 Nov 2006 23:50:57 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>	<5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
	<ca471dc20611221300p129551fft51fc742a115a1099@mail.gmail.co m>
	<5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
Message-ID: <ek2k8h$m81$1@sea.gmane.org>

Phillip J. Eby wrote:

 > Perhaps I should try giving implementations explicitly labelled as
 > prototypes

tends to work.

> and stating that there are other ways to do the same thing?

that's kind of implied in any proposal posted to this list, isn't it?

</F>


From janssen at parc.com  Wed Nov 22 23:51:37 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 14:51:37 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ek2iig$gni$1@sea.gmane.org> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<06Nov22.140512pst."58648"@synergy1.parc.xerox.com>
	<ek2iig$gni$1@sea.gmane.org>
Message-ID: <06Nov22.145147pst."58648"@synergy1.parc.xerox.com>

> Bill Janssen wrote:
> 
>  > This seems kind of a weak rationale to me, but OK.
> 
> yeah, namespaces are such a useless thing.  let's put everything in one 
> big flat namespace, so we don't have to wonder where things are.
> 
> </F>

Pardon me?  That's what we've got now, isn't it?  That's why we have
to do that ugly and hard-to-explain name mangling for "special"
methods.

Here's why I think it's weak:

1)  Mainly, all methods are special to someone.  Having only one
hard-to-differentiate syntactic mechanism to identify them really
doesn't work very well.

2)  But also, if Python was built with interfaces, "special" methods
would each be in their own namespace.  That is, a user wouldn't have
to worry about accidentally overriding a method; they'd have to
explicitly override the "len" method inherited from the "container"
interface; just defining a method called "len" in their subclass
wouldn't do it.  CLOS is rather elegant in this respect.

Bill

From solipsis at pitrou.net  Wed Nov 22 23:56:03 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 22 Nov 2006 23:56:03 +0100
Subject: [Python-3000] defop ?
In-Reply-To: <ca471dc20611221247r42ec30eo4a6d3ab36c12fd72@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<ca471dc20611221152i1744e88bl3818dc01b2752cb0@mail.gmail.com>
	<1164228053.4674.54.camel@fsol>
	<ca471dc20611221247r42ec30eo4a6d3ab36c12fd72@mail.gmail.com>
Message-ID: <1164236163.4674.125.camel@fsol>

Le mercredi 22 novembre 2006 ? 12:47 -0800, Guido van Rossum a ?crit :
> > class DefOpDescriptor:
> >     def __init__(self, genfunc, implfunc):
> >         self.genfunc = genfunc
> >         self.implfunc = implfunc
> >     def __call__(self, *args, **kargs):
> >         # Directly call the implementation (why not)
> >         self.implfunc(*args, **kargs)
> 
> I'm not sure how/when __call__ would be used.
> 
> Shouldn't you implement a __get__ descriptor instead?

Hmmm yes, thanks for pointing that out :-o. It will (or should) work
better like this:

    def __get__(self, obj, objtype):
        return types.MethodType(self.implfunc, obj, objtype)

Besides, if we don't want to allow direct calls of the implementation
without going through generic method dispath (see below), we don't need
__get__ at all.

> Aha, cool. You forgot to add that now the method name chosen must be
> unique. This works fine to describe @defop, and I like it better than
> using sys._getframe(). But it doesn't really work for the defop
> syntax, unless that makes up a unique (random?) name for each
> occurrence.

Indeed it could mangle a unique name, like __private does.

Also, DefOpDescriptors could be removed from the class dictionary at
class construction, and appended to a __generic_overloads__ attribute.
This would keep the class dict clean, and produce better documentation
by separating generic function implementations from other methods in the
documentation.

(it probably still sounds a bit hackish)




From fredrik at pythonware.com  Wed Nov 22 23:55:45 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 22 Nov 2006 23:55:45 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>	<45645252.6010101@benjiyork.com>	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
	<ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com>
Message-ID: <ek2khl$n4k$1@sea.gmane.org>

Guido van Rossum wrote:

 > as was the case for ABC, Python's ill-fated predecessor which invented
 > "optimal" terminology in a vacuum

optimal?  I've always thought it was just odd translations from dutch?

(but please, cannot we get the trains back in Python 3.0?  my son would 
*love* that ;-)

</F>


From guido at python.org  Thu Nov 23 00:06:26 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:06:26 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ek2khl$n4k$1@sea.gmane.org>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<45645252.6010101@benjiyork.com>
	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
	<ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com>
	<ek2khl$n4k$1@sea.gmane.org>
Message-ID: <ca471dc20611221506h2a002280i3ce5b2155d5c9b33@mail.gmail.com>

On 11/22/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
>  > as was the case for ABC, Python's ill-fated predecessor which invented
>  > "optimal" terminology in a vacuum
>
> optimal?  I've always thought it was just odd translations from dutch?

No, dutch was not involved. Lambert's English is impeccable as far as
I can recall and Steven Pemberton's British (although he tries to pass
for a typical Amsterdammer :-).

> (but please, cannot we get the trains back in Python 3.0?  my son would
> *love* that ;-)

Mine would love it too, as long as the trains were called Thomas. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Thu Nov 23 00:05:49 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 00:05:49 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov22.145147pst."58648"@synergy1.parc.xerox.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<06Nov22.140512pst."58648"@synergy1.parc.xerox.com>	<ek2iig$gni$1@sea.gmane.org>
	<06Nov22.145147pst."58648"@synergy1.parc.xerox.com>
Message-ID: <ek2l4f$onm$1@sea.gmane.org>

Bill Janssen wrote:

> Pardon me?  That's what we've got now, isn't it? 

no.  names of the form "__name__" are distinctly different from names of 
the form "name".

</F>


From fredrik at pythonware.com  Thu Nov 23 00:09:41 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 00:09:41 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
Message-ID: <ek2lbr$pme$1@sea.gmane.org>

Guido van Rossum wrote:

> Examples that keep recurring are things like
> the smtplib.SMTP.sendmail() method, whose second argument is either a
> string or a list of strings. The optimal way of writing this, IMO, is
> a single method that typechecks its 2nd argument. It currently uses
> "if isinstance(x, basestring)" but I'd be happier if this could be
> written as some kind of check for implementing the String interface.
> (If it's neither a string nor a list, I don't really care what
> happens, it will just raise an exception a few lines later and that's
> perfectly fine for diagnosing the problem). Rewriting this as a
> generic method (how do I even create generic methods? I'm sure it's
> easy) sounds like a total waste of time and code.

I'm not sure I see how changing

     def sendmail(self, from_addr, to_addr, ...):
         if isinstance(to_addr, basestring):
              to_addr = [to_addr]
	...

to, say

     def sendmail(self, from_addr, to_addr is a string, ...): # specific
         self.sendmail(from_addr, [to_addr], ...)

     def sendmail(self, from_addr, to_addr, ...): # general
         ...

would be a *total* waste of time.  but maybe I've just done too much C# 
programming ;-)

</F>


From baptiste13 at altern.org  Thu Nov 23 00:15:18 2006
From: baptiste13 at altern.org (Baptiste Carvello)
Date: Thu, 23 Nov 2006 00:15:18 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov22.093251pst."58648"@synergy1.parc.xerox.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>	<45645252.6010101@benjiyork.com>	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>	<ca471dc20611220823o411633a1k416525f464664def@mail.gmail.com>
	<06Nov22.093251pst."58648"@synergy1.parc.xerox.com>
Message-ID: <ek2lmo$qjg$1@sea.gmane.org>

Bill Janssen a ?crit :
> 
> I see nothing wrong with defining "empty" ABCs to indicate abilities
> rather than interfaces.
> 
>    class WellTested (Ability):
>      """Indicates that the type has passed the QA process"""
>      pass
> 
>    class TestedNumber (Number, WellTested):
>      ...
> 
And then if people want to add some tests to check if TestedNumber really 
implements the Ability, they can do it using docstring syntax.


class WellTested (Ability):
   """Indicates that the type has passed the QA process

   >>> myinstance=get_test_object()
   >>> hasattr(myinstance, 'attribute')
   True
   >>> myinstance.method(1,2)
   3

   """

class TestedNumber (Number, WellTested):
   ...

check_ability(TestedNumber, WellTested)


The check_ability function would just call the doctest with the appropriate test 
object (here a TestedNumber instance).

pros:

- mixin class has no behavior, hence no multiple inheritance problems

- doctest syntax allows to describe complex semantic properties without 
reinventing an ad-hoc language like Zope interfaces do

- could be done already in python 2.X

cons:

- we need another mechanism for tagging instances. Perhaps a WeakKeyDictionary 
mapping the instances to their added abilities can do.

Cheers,
BC


From guido at python.org  Thu Nov 23 00:19:26 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:19:26 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ek2lbr$pme$1@sea.gmane.org>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<ek2lbr$pme$1@sea.gmane.org>
Message-ID: <ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>

I consider it a waste of time since the if-version works and there's
no reason for it to be extensible. There's only so much time we can
spend on each program before we move on; perfection isn't always
desirable, just working well enough.

Now if we had the nice syntax you propose I probably would have used
it. But I don't think Phillips' proposal is quite that smooth just
yet.

On 11/22/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> > Examples that keep recurring are things like
> > the smtplib.SMTP.sendmail() method, whose second argument is either a
> > string or a list of strings. The optimal way of writing this, IMO, is
> > a single method that typechecks its 2nd argument. It currently uses
> > "if isinstance(x, basestring)" but I'd be happier if this could be
> > written as some kind of check for implementing the String interface.
> > (If it's neither a string nor a list, I don't really care what
> > happens, it will just raise an exception a few lines later and that's
> > perfectly fine for diagnosing the problem). Rewriting this as a
> > generic method (how do I even create generic methods? I'm sure it's
> > easy) sounds like a total waste of time and code.
>
> I'm not sure I see how changing
>
>      def sendmail(self, from_addr, to_addr, ...):
>          if isinstance(to_addr, basestring):
>               to_addr = [to_addr]
>         ...
>
> to, say
>
>      def sendmail(self, from_addr, to_addr is a string, ...): # specific
>          self.sendmail(from_addr, [to_addr], ...)
>
>      def sendmail(self, from_addr, to_addr, ...): # general
>          ...
>
> would be a *total* waste of time.  but maybe I've just done too much C#
> programming ;-)
>
> </F>
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Nov 23 00:21:03 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 15:21:03 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com> 
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
	<3218934778639261534@unknownmsgid>
	<ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com>
Message-ID: <06Nov22.152112pst."58648"@synergy1.parc.xerox.com>

> On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> > Something like
> >
> >   class MyNewClass (ExistingClass, OtherInterfaceIJustNoticedExistingClassImplements):
> >     pass
> 
> No, unless you can also patch up all code that creates ExistingClass
> instances. You alluded to this before and it sounded like you didn't
> think it was an issue. Why do you think that? I see it as a huge
> problem when retrofitting pre-existing third-party code.

I don't think the specific issue raised above is very important (wow,
I just noticed that type A also implements the equivalent of interface
B, so I wish I could treat A as a B, but darn it, I can't since it
didn't inherit from B).  I think that ad-hoc adapter classes work OK
for handling that rare and incidental case, in the extremely rare case
that it *has* to be handled.

Not to say that there aren't interesting problems, though I'm not sure
just how "big" they are.  The biggest issue I see is the factory
problem.  The other is the existing value problem, which is a variant
on either the above or the factory problem.

Factory problem: This is a known OO problem.  You're given a factory,
which produces instances of type A.  You've got B, which is a subtype
of A, and you wish the factory would return that, since nobody who
knows anything uses A instead of B anymore.  You've got a number of
options: (1) change the factory to produce B instead of A; (2) wrap
each instance of A in a B which forwards A's methods to the original
instance; (3) somehow change the instance of A into an instance of B.

(1) is often practically not possible, though it's probably the right
thing to do.  In fact, "factories" should always be built to be
extensible (that is, they should be built so that user code can
register a new factory if desired).  (2) is always possible, but takes
some extra coding.  It would be nifty if Python could have a standard
way of doing (3).  That's what I'd aim for.  That would solve all
three of these related problems.  Notice, however, that (2) and (3)
would probably look pretty much the same for the practicing
programmer:

    myB = coerce_to_B(factory.produce_A());

(3) is practically speaking only possible for types which have
relatively independent initializers.

Is munging __bases__ so terrible?  Isn't it just:

  def ensure_base (i, clss):
    if not isinstance(i, clss):
        class X (i.__class__, clss):
            pass
        i.__class__ = X
        clss.__init__(i)
    return i

Would be nice to have anonymous classes :-).


Bill


From guido at python.org  Thu Nov 23 00:30:33 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:30:33 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221530y72f9dcf8s25e461608bfcc2e3@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> The problem I have sometimes in making my proposals clear to you is that,
> at the level of abstraction I think on, there are often multiple ways to
> carry out a useful/usable implementation, but you seem to want to
> understand one *particular* implementation, even though I'm trying to
> propose an approach to the solution, rather than a particular solution.

Oh, it would all be so much easier if we could all be in the same room
for a day...

The problem often seems to be that I don't have the imagination to
come up with a suitable implementation for the abstraction you are
describing, and at that point it's no longer an abstraction but an
arbitrary string of words to me. Those words may brim over with
meaning, but there are so many choices that it's hard to guess which
one is meant.

[[As an analogy, someone with a lot of HCI experience once pointed out
to me that when you show a code fragment to someone who isn't used to
programming, they tend to only see the words and lose the punctuation,
so for example they would see "self.__getattr__(x)+y" as "self getattr
x y" which loses a lot of meaning. Maybe I have the same problem with
some of your examples.]]

I guess an abstraction ideally needs to be described by formal
semantics, not by a hypothetical usage example. Since formal semantics
are often hard to come by, I'll settle for an implementation sketch.

For example, I really couldn't fathom what you meant for
addmethod(iter, ...) to do. It suddenly became completely clear once
you said "addmethod(x, ...) calls x.__addmethod__(...)".

> However, when I present you with a particular implementation, you then pick
> holes in the implementation itself, or the tradeoffs made to render the
> example simple.

That's unfortunate but can't always be avoided. I'm thinking that I
criticize the implementation in order to understand the abstraction
better. Since the implementation is often the only clue I have for
understanding the abstraction, if the implementation oversimplifies
things or has an obvious hole in it this distracts from the
understanding.

> If I knew what tradeoffs you would prefer ahead of time,
> then I could make my examples match the desired tradeoffs, but in the
> absence of that information I can only fall back to describing a general
> concept.

That's why I wish we had more direct face-to-face interaction.

> So we tend to go back and forth between abstractions rejected for
> lack of specificity, and specific things rejected for their particular
> nature.  :)  Perhaps I should try giving implementations explicitly
> labelled as prototypes and stating that there are other ways to do the same
> thing?

I don't know. In general, I find implementations that draw too
strongly on PEAK or RuleDispatch distracting because I'm not familiar
with their semantics or terminology and I'm in the "self getattr x y"
situation again.

At this point I think I understand the defop proposal and I understand
addmethod. But I think I could use some help with hasmethod. Suppose I
wanted to write the sendmail example using hasmethod. (I know it can
be written differently, but other examples may need hasmethod, and I'm
familiar with the sendmail example.) Could you should (a) what the
example would look like and (b) what would / could go on behind the
scenes to make it work? Bonus points for an attempt to show that it
can be implemented efficiently.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Nov 23 00:32:38 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 18:32:38 -0500
Subject: [Python-3000] defop ?
In-Reply-To: <ca471dc20611221406o562f9f4du700992125bfab8c2@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>

At 02:06 PM 11/22/2006 -0800, Guido van Rossum wrote:
> > (The 'decorate_class' function applies its argument as a class decorator
> > for the class its caller is invoked from.  This is the same mechanism (and
> > actual code!) that Zope uses to do in-body class decorators like
> > 'implements()'.)  decorate_class raises a SyntaxError if its caller is not
> > being run directly inside a class body.
>
>That's a bizarre use of that exception.

Well, the idea was that a class decorator can only be invoked inside a 
class, so it's a syntax error to invoke it elsewhere.  And, there's little 
chance of confusing the SyntaxError with some error generated by *running* 
the decorator itself, as opposed to merely its positioning.

Anyway, it seemed like a good idea at the time.  :)


>I'm guessing that decorate_class squirrels away some info for later?

It introduces a stealth metaclass to the class frame, that will call the 
decorator.  If there's an existing metaclass in the frame, it invokes it 
first to create the class.  The devil is in the details, of course, and it 
does require that you define any explicit metaclass *before* using any 
class decorators, but this is a documented limitation of the DecoratorTools 
library.


>Everything here seems designed to hide the implementation details,
>which is great for actual use, but kills understanding of the
>mechanism. :-(
>
> > And yes, this uses frame magic, and no, it's not fragile, as it knows the
> > difference between class and non-class frames.
>
>Wow. How can you tell?

Basically, if f_locals contains a '__module__' key that's equal to the 
'__name__' in f_globals, although there are some interesting corner cases 
involving 'exec' -- the details are in the implementation.  The relevant 
function is the 'frameinfo(frame)' function which tells you what "kind" of 
frame you have.


>  Does it work with Jython, IronPython and PyPy?

PyPy should be fine, since it uses the same code objects and 
bytecode.  IIRC, the others emulate enough of the frame protocol that the 
'__module__' and '__name__' checks should be no problem.  However, I have 
not tried it in any of these.  Since this hack requires metaclass support, 
there was no point to attempting it in Jython.  And when the code was 
written, neither IronPython nor PyPy existed yet.


>I still find it fragile somehow -- or at least severely hackish. I
>like the other proposal (where it puts a special attribute on the
>function object which is then found by the type) better, it doesn't
>require sys._getframe().

Sure, there are certainly lots of other ways to implement it!  What I 
showed was just to demonstrate the *possibility*.  If there were defop 
syntax, then of course the compiler could simply generate the appropriate 
bytecode following the MAKE_CLASS operation.  Or MAKE_CLASS could look for 
some data inserted into the source dictionary.

decorate_class() is just what I had handy to answer your question - it 
should not be confused with being the One Obvious Way to do it is.  It's 
just proof of possibility (and practical workability, in that it has been 
long and widely used successfully with at least CPython).

But, there are still other ways.  The compiler could stick 
"__owning_class__" attributes on any function defined in a class, and 
__addmethod__ could defer actually registering the functions until the 
generic function was invoked, thus ensuring that __owning_class__ would be 
set first.

In general, if we're allowed to change the implementation of Python, it's 
easy to find some way to do it.  But you asked for a way to do it in (I 
thought) today's Python, so that's what I gave you.


>I'm assuming that's outside the context of the BinaryTree class --
>since inside it, Binarytree is undefined.

Correct.


>Hm, I think this is the first time in this thread that you meant to
>use 'iter', 'len' etc. as "tokens".

I don't understand what you mean.  They're objects, and so usable as 
tokens.  That's why I've also been saying they can also represent 
"abilities", with defop being a way to unify their use as generic operation 
(e.g. iter()), implementation (i.e. defop iter), and "interface" or ability 
(e.g. 'hasmethod(iter,ob)' or something of that sort).


>I think I'd rather change them a
>bit so that they can be objects that actually understand __addmethod__
>and __hasmethod__.

Okay.  That's an example of the kind of preference I have no way to divine 
ahead of time.  If I knew you preferred function attributes for deferred 
registration, and to make these objects of a special type, I'd have 
proposed it that way.  :)  My assumption was that changing the type of 
Python builtin functions would be considered more "radical" than the idea 
of defop, since it requires changing something, rather than just adding.


> > then there's no need to have
> > actual addmethod/hasmethod functions.  But that's sort of like saying we
> > don't need getattr() any more because we can just use
> > type(ob).__getattribute__() now!  :)
>
>But I think you're really proposing an addmethod builtin that calls an
>__addmethod__ method, right?

Yes, assuming that we can call do this with regular functions too.  That 
is, as long as there's some way to retroactively overload a regular 
function.  Other than that, I'm not worried about the need to retrofit any 
other existing generic function types; their authors will have to change 
them for Py3K anyway.  :)


From fredrik at pythonware.com  Thu Nov 23 00:36:11 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 00:36:11 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>	<ek2lbr$pme$1@sea.gmane.org>
	<ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>
Message-ID: <ek2mto$tp7$1@sea.gmane.org>

Guido van Rossum wrote:

> Now if we had the nice syntax you propose I probably would have used
> it.

well, if it looks nice, what's keeping us from having that? ;-)

(but I'm suspect that my view of "generics" is somewhat different from 
Phillips, so let's focus on fully understanding his proposal first)

</F>


From pje at telecommunity.com  Thu Nov 23 00:37:59 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 18:37:59 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.co
 m>
References: <ek2lbr$pme$1@sea.gmane.org>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<ek2lbr$pme$1@sea.gmane.org>
Message-ID: <5.1.1.6.0.20061122183441.0403dbf8@sparrow.telecommunity.com>

At 03:19 PM 11/22/2006 -0800, Guido van Rossum wrote:
>I consider it a waste of time since the if-version works and there's
>no reason for it to be extensible.

But if that's true, then there's no reason for it to use interfaces, 
either, is there?  :)  It seems like a double standard to say, "well, 
sometimes you have quick-and-dirty code that doesn't need extensibility", 
and then later complain that it should be able to check for an interface, 
instead of using either a concrete type or an overload or generic.

That is, I don't think you can use the quick'n'dirty case simultaneously as 
an argument against overloads and *for* interfaces.  :)


From guido at python.org  Thu Nov 23 00:38:19 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:38:19 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <-3135130819962384877@unknownmsgid>
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
	<3218934778639261534@unknownmsgid>
	<ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com>
	<-3135130819962384877@unknownmsgid>
Message-ID: <ca471dc20611221538lfb33072wfd61abbbdac59195@mail.gmail.com>

I'm concerned about the additional wrinkle where there is a library
that calls the factory to produce A's and then turns around and passes
them on to *another* library. Now the other can deal with A's but has
recently been upgraded so that it works much better with B's; but it
doesn't know that A's are fine B's already. ideally, there'd be an
indirection in there somewhere that would let us insert a conversion
from A to B; but if there isn't, it would be nice to be able to poke
into A a marker indicating that it also supports B.

Somehow that looks nicer with interfaces than with ABCs.

On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> > On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> > > Something like
> > >
> > >   class MyNewClass (ExistingClass, OtherInterfaceIJustNoticedExistingClassImplements):
> > >     pass
> >
> > No, unless you can also patch up all code that creates ExistingClass
> > instances. You alluded to this before and it sounded like you didn't
> > think it was an issue. Why do you think that? I see it as a huge
> > problem when retrofitting pre-existing third-party code.
>
> I don't think the specific issue raised above is very important (wow,
> I just noticed that type A also implements the equivalent of interface
> B, so I wish I could treat A as a B, but darn it, I can't since it
> didn't inherit from B).  I think that ad-hoc adapter classes work OK
> for handling that rare and incidental case, in the extremely rare case
> that it *has* to be handled.
>
> Not to say that there aren't interesting problems, though I'm not sure
> just how "big" they are.  The biggest issue I see is the factory
> problem.  The other is the existing value problem, which is a variant
> on either the above or the factory problem.
>
> Factory problem: This is a known OO problem.  You're given a factory,
> which produces instances of type A.  You've got B, which is a subtype
> of A, and you wish the factory would return that, since nobody who
> knows anything uses A instead of B anymore.  You've got a number of
> options: (1) change the factory to produce B instead of A; (2) wrap
> each instance of A in a B which forwards A's methods to the original
> instance; (3) somehow change the instance of A into an instance of B.
>
> (1) is often practically not possible, though it's probably the right
> thing to do.  In fact, "factories" should always be built to be
> extensible (that is, they should be built so that user code can
> register a new factory if desired).  (2) is always possible, but takes
> some extra coding.  It would be nifty if Python could have a standard
> way of doing (3).  That's what I'd aim for.  That would solve all
> three of these related problems.  Notice, however, that (2) and (3)
> would probably look pretty much the same for the practicing
> programmer:
>
>     myB = coerce_to_B(factory.produce_A());
>
> (3) is practically speaking only possible for types which have
> relatively independent initializers.
>
> Is munging __bases__ so terrible?  Isn't it just:
>
>   def ensure_base (i, clss):
>     if not isinstance(i, clss):
>         class X (i.__class__, clss):
>             pass
>         i.__class__ = X
>         clss.__init__(i)
>     return i
>
> Would be nice to have anonymous classes :-).
>
>
> Bill
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Nov 23 00:44:23 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:44:23 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122183441.0403dbf8@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<ek2lbr$pme$1@sea.gmane.org>
	<5.1.1.6.0.20061122183441.0403dbf8@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221544m64354e09r2316605aea2dc663@mail.gmail.com>

Touche.

I think what I was trying to argue is that I probably wouldn't have
written it using function overloading, but I would have liked to write
the if-test using has_ability(to_addrs, String) instead of
isinstance(to_addrs, basestring). (The fact that I even wrote
basestring suggests that I was feeling a *little* guilty about the
isinstance() call. :-)

I also believe that a has_ability() test applies to places where
function overloading doesn't, e.g. when the object that could be one
of two kinds is not directly an argument but retrieved from a
container or returned by some function. Having to introduce an
overloaded helper just to avoid a single has_ability() call is a lot
of distraction.

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 03:19 PM 11/22/2006 -0800, Guido van Rossum wrote:
> >I consider it a waste of time since the if-version works and there's
> >no reason for it to be extensible.
>
> But if that's true, then there's no reason for it to use interfaces,
> either, is there?  :)  It seems like a double standard to say, "well,
> sometimes you have quick-and-dirty code that doesn't need extensibility",
> and then later complain that it should be able to check for an interface,
> instead of using either a concrete type or an overload or generic.
>
> That is, I don't think you can use the quick'n'dirty case simultaneously as
> an argument against overloads and *for* interfaces.  :)
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Nov 23 00:45:29 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 18:45:29 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ek2mto$tp7$1@sea.gmane.org>
References: <ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<ek2lbr$pme$1@sea.gmane.org>
	<ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061122184343.02871500@sparrow.telecommunity.com>

At 12:36 AM 11/23/2006 +0100, Fredrik Lundh wrote:
>well, if it looks nice, what's keeping us from having that? ;-)

Hear hear.  ;)


>(but I'm suspect that my view of "generics" is somewhat different from
>Phillips, so let's focus on fully understanding his proposal first)

So now I'm curious.  What *is* your view?  Maybe it's better.


From guido at python.org  Thu Nov 23 00:48:53 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:48:53 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
Message-ID: <ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>

On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> Although I don't necessarily agree with the arguments from a puristic
> point of view ("then why not make keys(), values(), items(), read(),
> write(), ... builtin GFs ?"), they are indeed a better answer to the
> FAQ entry. After all, HCI-based arguments are usually a fancier way of
> saying "it's a matter of taste" and arguing against them is like
> arguing that green is a better colour than red.

I'm so glad you're not trying to hide that you're a troll.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Nov 23 00:54:28 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 15:54:28 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <8853879015409334245@unknownmsgid>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
Message-ID: <ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>

On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> 2)  But also, if Python was built with interfaces, "special" methods
> would each be in their own namespace.  That is, a user wouldn't have
> to worry about accidentally overriding a method; they'd have to
> explicitly override the "len" method inherited from the "container"
> interface; just defining a method called "len" in their subclass
> wouldn't do it.  CLOS is rather elegant in this respect.

Ow, that's the first time you describe this particular wrinkle. I
don't know anything by CLOS. Python has a rather much simpler approach
to attribute namespaces for class instances; you can have only one
attribute named "len" (since it's contained within a class, most folks
consider this sufficient localization to not have to worry about
conflicts). I don't think that anyone else here thought of ABCs or
interfaces introducing separate namespaces yet. The __special__
namespace is effectively a second namespace, orthogonal to the regular
namespace, because there's an explicit convention that you shouldn't
define __special__ attributes unless you are implementing whatever
semantics a particular __special__ name has.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Nov 23 00:59:55 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 15:59:55 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221538lfb33072wfd61abbbdac59195@mail.gmail.com> 
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
	<3218934778639261534@unknownmsgid>
	<ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com>
	<-3135130819962384877@unknownmsgid>
	<ca471dc20611221538lfb33072wfd61abbbdac59195@mail.gmail.com>
Message-ID: <06Nov22.160002pst."58648"@synergy1.parc.xerox.com>

> I'm concerned about the additional wrinkle where there is a library
> that calls the factory to produce A's and then turns around and passes
> them on to *another* library. Now the other can deal with A's but has
> recently been upgraded so that it works much better with B's; but it
> doesn't know that A's are fine B's already. ideally, there'd be an
> indirection in there somewhere that would let us insert a conversion
> from A to B; but if there isn't, it would be nice to be able to poke
> into A a marker indicating that it also supports B.

There's a dangling "it" in there which I'm not sure I have the right
antecedent for.  Let's see if I've got this straight :-).

     class B (A):
        ...

     def library_1_code():
        ...
        value_A = factory_which_really_produces_Bs()
        library_2_which_works_much_better_with_Bs(value_A)
        ...

Isn't the marker already implicit?  Does it matter that the first
library only thinks of the A interface for the value?  Or are you just
looking for a new built-in like

   def markas (i, interface_class):
     if not isinstance(i, interface_class):
         class X (i.__class__, interface_class):
             pass
         i.__class__ = X
         interface_class.__init__(i)
     return i

?

> Somehow that looks nicer with interfaces than with ABCs.

I think I've lost track of the difference.

Bill

From guido at python.org  Thu Nov 23 01:14:16 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 16:14:16 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122154127.03f77708@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122154127.03f77708@sparrow.telecommunity.com>
Message-ID: <ca471dc20611221614h6614fb95t235845816f528df6@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> My point is that I'm not trying to create a "bad rep", I'm just emphasizing
> that it's easier/more useful to define interfaces as collections of generic
> operations, because it makes the simple cases simple and powerful, and the
> complex ones possible.  Doing it the other way around makes all of the
> cases more complex, and loses other capabilities.
>
> My use of the word "unnecessary" isn't a moral judgment of interfaces, it's
> an argument about how best to layer the various concepts we want to
> have.  Interfaces aren't necessary for simple things, and it's better (IMO)
> if we define them *in terms of* simpler things, because then people don't
> have to learn about them unless they need something more complex.
>
> And if they need something more complex, it is also more likely (I believe)
> that they will have more specialized needs.  So, I intended to suggest that
> if we just do the basics (operations) and allow others to define interface
> frameworks atop that, then we can avoid overgeneralizing from insufficient
> experience.  However, in the expanded discussion it seems to me that you
> have goals for additional abilities you'd like to see sooner rather than later.
>
> What worries me about Java-style interfaces is that you can't split and
> recombine them in arbitrary ways, without effectively decomposing them into
> single-operation interfaces using multiple inheritance.

But that's a limitation of Java interfaces (and perhaps of Zope
interfaces); it doesn't have to be a limitation of Python 3000
abilities.

> At which point,
> you just might as well have treated them as operations to start
> with!  Thus, taking an operation-based approach means there need not be a
> concept of the "one true sequence interface", just the *default* sequence
> interface.  If my needs are more or less expansive than the default
> "sequence" interface, I should be able to do something like (sequence.len,
> sequence.iter) and recombine a new interface.

I'm not sure I understand the significance of prefixing len and iter
with "sequence." in your example. After all len and iter are globals.
And a default interface will surely tend to become a de-facto
standard.

What's your view on the interfaces for numbers and strings? I really
don't think it's very useful to care much about objects that implement
+ and * but not / or ** -- while mathematicians have tons of use cases
for those, programmers rarely do. As long as those mathematicians can
create their own interfaces and apply them to the standard types they
should be happy; the rest of us will be served well by a simple
hierarchy e.g. Number, Complex, Real, Integer. Even more so for
strings; I expect that a single String interface would cover most
uses. Well, perhaps there ought to be something more basic
representing the shared operations of strings and bytes. But that's
about it.

Container and file types are a different story; there are many partial
reimplementations of the basic patterns and it is useful to be able to
describe fairly minimal custom interfaces composed of a small number
of operations.

User-defined types are different again. I've recently had the
opportunity to think a bit about refactoring a library that works with
Perforce changelists (CLs; Perforce is a commercial source control
system) to be generalized into supporting Subversion revisions. There
are a lot of operations one can do on a Perforce CL objects. There is
a lot of code that receives, produces or passes around Perforce CL
objects. Some of this generalizes to Subversion; some of it doesn't.
But I don't expect I'll be needing more than three interfaces to
describe the eventual refactoring: an abstract source control revision
class as the base interface, and two subclasses specializing for
Subversion and Perforce. Sure, eventually someone will want to support
CVS, or RCS, or (more likely) git or Mercurial or what have you. The
code will continue to evolve and be refactored. But I don't think at
any point I'll need to break my interfaces up into individual
operations.

> This would also make the interface committee's job at least somewhat
> easier.  :)

Anything to make their job easier. I guess we'll have to lock them
(us? :-) up in a room for several days before/after PyCon.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From krstic at solarsail.hcs.harvard.edu  Thu Nov 23 01:19:03 2006
From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?B?SXZhbiBLcnN0acSH?=)
Date: Wed, 22 Nov 2006 19:19:03 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov22.145147pst."58648"@synergy1.parc.xerox.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<06Nov22.140512pst."58648"@synergy1.parc.xerox.com>	<ek2iig$gni$1@sea.gmane.org>
	<06Nov22.145147pst."58648"@synergy1.parc.xerox.com>
Message-ID: <4564E8F7.6060605@solarsail.hcs.harvard.edu>

Bill Janssen wrote:
> 1)  Mainly, all methods are special to someone.  

But some, such as those that have special meaning to the language
itself, are more special than others.

__methods__ are not just plain old methods with funny names. You write
regular methods to express functionality you wish your class to have,
but you write __methods__ to provide low-level glue tying your class in
with the interpreter. This lets the interpreter maintain a kind of
pleasant fiction in which user code can depend on system methods, like
len() or attribute access, returning sensible results and performing
sensible actions even for objects the user code knows nothing about.

This difference is substantial enough that a different terminology has
been quietly tossed around: the name `defop` implicitly proposes that
__methods__ are really "operations". I'm not sure if I like the term
yet, but in the meantime, thinking about these methods as system glue,
staples, paperclips, or a different fastener term of your choice, might
illuminate the difference further. :)

-- 
Ivan Krsti? <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D

From pje at telecommunity.com  Thu Nov 23 01:20:22 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 19:20:22 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221530y72f9dcf8s25e461608bfcc2e3@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122185122.03e49500@sparrow.telecommunity.com>

At 03:30 PM 11/22/2006 -0800, Guido van Rossum wrote:
>Oh, it would all be so much easier if we could all be in the same room
>for a day...

Well, there's always Skype/Gtalk et al...  :)

>The problem often seems to be that I don't have the imagination to
>come up with a suitable implementation for the abstraction you are
>describing, and at that point it's no longer an abstraction but an
>arbitrary string of words to me.

This, and the rest of your message helps a lot, at least in terms of me 
getting past the "what the heck does he *want* from me?" question.  :)


>I criticize the implementation in order to understand the abstraction
>better.  Since the implementation is often the only clue I have for
>understanding the abstraction, if the implementation oversimplifies
>things or has an obvious hole in it this distracts from the
>understanding.

Ah.  Here, I usually resist working out all the details of a 
production-quality implementation, because then it seems to make it harder 
to talk about the *ideas* and select a *good* implementation approach.  But 
I will do my best to accommodate this now that I understand more 
specifically what you need.


>I don't know. In general, I find implementations that draw too
>strongly on PEAK or RuleDispatch distracting because I'm not familiar
>with their semantics or terminology and I'm in the "self getattr x y"
>situation again.

Fair enough, except that I haven't drawn on them as much as you think I 
have; in this discussion I've only referred to the 'simplegeneric' and 
'DecoratorTools' libraries, both of which have usage examples on their 
CheeseShop pages, and both of which are only a few hundred lines of code 
(~100 for simplegeneric, ~350 for all of DecoratorTools including many 
things besides the comparatively brief decorate_class feature).

What has happened instead, is that when I have introduced hypothetical 
constructs like 'addmethod', you've assumed that I was referring to 
something that existed elsewhere and were then saying "but I don't know 
what that is".  But I was saying, "here is a thing I'm *proposing*, and 
here is how it would work".


>But I think I could use some help with hasmethod. Suppose I
>wanted to write the sendmail example using hasmethod. (I know it can
>be written differently, but other examples may need hasmethod, and I'm
>familiar with the sendmail example.) Could you should (a) what the
>example would look like and

Assuming the existence of an 'as_string()' generic function, either 
built-in or in the stdlib, one could write it as:

     if hasmethod(as_string, to_addrs):
         # it's a string-like thing
         to_addrs = [to_addrs]

The assumption here is that we've chosen to have an 'as_string' generic to 
denote a general notion of "stringness".  The semantics of 'as_string' is 
that it returns a string of the same "value" as its input, or raises an 
error if its input isn't "really" a string.

In other words, this function would be to strings what __index__ is to 
integers, if that makes sense.  It doesn't convert non-strings to strings, 
it only converts conceptual strings to concrete strings, the way __index__ 
converts conceptual integers to concrete integers.  (Assuming I understand 
__index__ correctly, of course, because I tuned out after some of the 
earliest discussions about what it should do.)

So, the generic function 'as_string' here functions as an "interface" or 
"ability".  Of course, instead of being called 'as_string', it could live 
as an attribute of the 'str' type, e.g. 'str.cast':

     if hasmethod(str.cast, to_addrs):
         # it's a string-like thing
         to_addrs = [to_addrs]

I make the assumption, by the way, that once the string-like thing is 
passed to something that really needs a concrete, C-compatible string (like 
open() or socket.send()), that it will actually *call* 'str.cast' to get 
that string.


>(b) what would / could go on behind the
>scenes to make it work?

The generic function's __hasmethod__ checks for an implementation.  This 
can happen via __special__ attribute check (for built-in generics like 
len/iter/etc.) or via registry lookup (other kinds of generics).


>Bonus points for an attempt to show that it
>can be implemented efficiently.

Registry lookup is a dict lookup against MRO -- i.e., it's like an 
attribute access check.  Slot checks are O(1) if there's a C-level slot, or 
equivalent to an attribute access check otherwise.


From guido at python.org  Thu Nov 23 01:32:51 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 16:32:51 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <6961093367361956666@unknownmsgid>
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
	<3218934778639261534@unknownmsgid>
	<ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com>
	<-3135130819962384877@unknownmsgid>
	<ca471dc20611221538lfb33072wfd61abbbdac59195@mail.gmail.com>
	<6961093367361956666@unknownmsgid>
Message-ID: <ca471dc20611221632w1effe31cxe3107b45d90c5ab9@mail.gmail.com>

I was trying to make it so that the factory is part of library 1 and
really returns As, which however can be turned into Bs without adding
new functionality, merely by marking them as such -- but the
assumption is that we don't have a convenient way to modify the
factory so that it marks the objects it returns as Bs.

IMO the difference between ABCs and interfaces/abilities is that
inserting a new ABC into an existing class hierarchy without modifying
its source code is considered a hack, and must be done by a helper
routine that *actually* mucks with __bases__; while (Zope-style)
interfaces use an external registry or a separate class attribute
(__implements__) that is intended to be mucked with.

So perhaps it really just boils down to mucking with __bases__ or with
__implements__. I happen to know that there are cases where Python
prevents you from modifying __bases__ in certain ways because the C
part of the implementation's safety depends on what's in __bases__.
Also modifying __bases__ is expensive, involving a walk of all
subclasses. The code that implements safely setting __bases__ is a
single long function at lines 184-322 of Objects/typeobject.c in
Python (svn head). OTOH, since __implements__ has no special meaning
to Python, setting or modifying __implements__ is a simple class
attribute assignment.

On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> > I'm concerned about the additional wrinkle where there is a library
> > that calls the factory to produce A's and then turns around and passes
> > them on to *another* library. Now the other can deal with A's but has
> > recently been upgraded so that it works much better with B's; but it
> > doesn't know that A's are fine B's already. ideally, there'd be an
> > indirection in there somewhere that would let us insert a conversion
> > from A to B; but if there isn't, it would be nice to be able to poke
> > into A a marker indicating that it also supports B.
>
> There's a dangling "it" in there which I'm not sure I have the right
> antecedent for.  Let's see if I've got this straight :-).
>
>      class B (A):
>         ...
>
>      def library_1_code():
>         ...
>         value_A = factory_which_really_produces_Bs()
>         library_2_which_works_much_better_with_Bs(value_A)
>         ...
>
> Isn't the marker already implicit?  Does it matter that the first
> library only thinks of the A interface for the value?  Or are you just
> looking for a new built-in like
>
>    def markas (i, interface_class):
>      if not isinstance(i, interface_class):
>          class X (i.__class__, interface_class):
>              pass
>          i.__class__ = X
>          interface_class.__init__(i)
>      return i
>
> ?
>
> > Somehow that looks nicer with interfaces than with ABCs.
>
> I think I've lost track of the difference.
>
> Bill
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Nov 23 01:37:58 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 19:37:58 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221614h6614fb95t235845816f528df6@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122154127.03f77708@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122143522.03be6308@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122154127.03f77708@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122192155.03e3a780@sparrow.telecommunity.com>

At 04:14 PM 11/22/2006 -0800, Guido van Rossum wrote:
>But that's a limitation of Java interfaces (and perhaps of Zope
>interfaces); it doesn't have to be a limitation of Python 3000
>abilities.

Agreed -- but some people made statements that implied this was their 
prototypical vision of "interface", and I was concerned that you considered 
it the canonical model as well, since you referenced both Java and Zope.


> > At which point,
> > you just might as well have treated them as operations to start
> > with!  Thus, taking an operation-based approach means there need not be a
> > concept of the "one true sequence interface", just the *default* sequence
> > interface.  If my needs are more or less expansive than the default
> > "sequence" interface, I should be able to do something like (sequence.len,
> > sequence.iter) and recombine a new interface.
>
>I'm not sure I understand the significance of prefixing len and iter
>with "sequence." in your example. After all len and iter are globals.

There was no significance, it was because I had the Zope/Java interface 
model in my head at the time and I spoke through that paradigm.  IOW, just 
a brain fart.  :)


>And a default interface will surely tend to become a de-facto
>standard.

Sure...  except that as long as the operations define compliance, then the 
"interface" is just an explanation of which operations are required.  You 
can always require a lesser interface (such as iter) rather than a more 
complex one.


>What's your view on the interfaces for numbers and strings? I really
>don't think it's very useful to care much about objects that implement
>+ and * but not / or ** -- while mathematicians have tons of use cases
>for those, programmers rarely do. As long as those mathematicians can
>create their own interfaces and apply them to the standard types they
>should be happy; the rest of us will be served well by a simple
>hierarchy e.g. Number, Complex, Real, Integer. Even more so for
>strings; I expect that a single String interface would cover most
>uses. Well, perhaps there ought to be something more basic
>representing the shared operations of strings and bytes. But that's
>about it.

Yep, pretty much.  However, note that all of these "interfaces" are 
actually more like *data types*.  For immutable data types, I would argue 
that what you care about is the ability to treat an item "as a" string or 
whatever.  That is, "can I cast this thing to a string without data loss or 
noise?"  "Can I cast to some concrete integral type without loss of 
precision?"  Note too that these casts are *operations* -- perfect for 
representation as generic operations like int.cast and str.cast.  Number, 
Complex and Real are a little harder to place, since there are no concrete 
types for those, but there's no reason not to just have the three casting 
generic functions, and boom, you're done.

Now, when you move back up to "protocols" like sequence or arithmetic, 
we're again talking about single operations (or sets thereof), because 
rarely does any one piece of code require *all* sequence or arithmetic 
operations.  Ergo, little reason to have a single fixed "interface" 
spelling out every aspect of the protocol, versus using generics like 
operator.getitem et al to denote the required capabilities.

(Note that in a 'defop' world, it may make sense to put 'operator' in the 
builtin namespace rather than requiring it to be imported.  Or maybe it 
doesn't.  I haven't thought about that one deeply yet.)


>Container and file types are a different story; there are many partial
>reimplementations of the basic patterns and it is useful to be able to
>describe fairly minimal custom interfaces composed of a small number
>of operations.

Yep.


>The code will continue to evolve and be refactored. But I don't think at
>any point I'll need to break my interfaces up into individual
>operations.

True enough, however if the concept of "interface" is implemented in terms 
of "set of operations", this allows for such things as automatically 
generated adapters, and partial implementations ala some of Ping and Alex's 
past proposals, where UserList and UserDict would actually be "interfaces" 
rather than concrete objects.



From greg.ewing at canterbury.ac.nz  Thu Nov 23 01:50:38 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 13:50:38 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <002201c70e46$5149f750$6402a8c0@arkdesktop>
References: <002201c70e46$5149f750$6402a8c0@arkdesktop>
Message-ID: <4564F05E.7090504@canterbury.ac.nz>

Andrew Koenig wrote:
> it feels natural to me to speak
> of a class as "having the totally ordered ability".

That doesn't sound natural to me at all.

If you were speaking about a person with brown
eyes, would you say that they "had the brown
eyes ability"?

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 23 02:26:15 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 14:26:15 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061122165513.GA13795@niemeyer.net>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
Message-ID: <4564F8B7.5030007@canterbury.ac.nz>

Gustavo Niemeyer wrote:

> With interfaces, all you need to do is define an adapter from Apple
> to IFruit, and instantly all the widely available frameworks will
> be handling apples just fine.

I don't quite see how that works. Where exactly is
the adaptation supposed to be done? It can't be done
in libapple, because libfruit didn't exist when it
was written. Is libfruit supposed to adapt everything
it's given to IFruit just in case it's necessary?
That seems like a lot of extra work to go to, both
at coding time and run time, for no immediate benefit.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 23 02:34:49 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 14:34:49 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov22.092436pst.58648@synergy1.parc.xerox.com>
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<06Nov22.092436pst.58648@synergy1.parc.xerox.com>
Message-ID: <4564FAB9.1010902@canterbury.ac.nz>

Bill Janssen wrote:

> I seem to spend a lot of time inside functions looking over values and
> figuring out what to do do with them,

If you're really spending that much time doing
things like that, I'd strongly suggest you take
a step back and re-think your APIs.

--
Greg

From gustavo at niemeyer.net  Thu Nov 23 03:00:54 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 00:00:54 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <4564F8B7.5030007@canterbury.ac.nz>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
Message-ID: <20061123020054.GA31618@niemeyer.net>

> I don't quite see how that works. Where exactly is
> the adaptation supposed to be done? It can't be done
> in libapple, because libfruit didn't exist when it
> was written. Is libfruit supposed to adapt everything
> it's given to IFruit just in case it's necessary?
> That seems like a lot of extra work to go to, both
> at coding time and run time, for no immediate benefit.

The user of both libraries may define how Apple adapts to
IFruit.  Being able to define adapters for things that weren't
previously prepared for it is one of the usage cases for the
component architecture in Zope 3.  The mentioned example is a
bad one for generic functions, specifically.

Look at the more recent mail for a more concrete example than
this one, which validates the same point of view for both
generic functions and adaptation.

-- 
Gustavo Niemeyer
http://niemeyer.net

From janssen at parc.com  Thu Nov 23 03:14:37 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 18:14:37 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
Message-ID: <06Nov22.181438pst."58648"@synergy1.parc.xerox.com>

> Ow, that's the first time you describe this particular wrinkle.

Yep.  I hesitated to bring it up, but if there's a need for separate
namespaces for method names, why not do it right?

Bill

From janssen at parc.com  Thu Nov 23 03:20:18 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 18:20:18 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <4564FAB9.1010902@canterbury.ac.nz> 
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<06Nov22.092436pst.58648@synergy1.parc.xerox.com>
	<4564FAB9.1010902@canterbury.ac.nz>
Message-ID: <06Nov22.182023pst."58648"@synergy1.parc.xerox.com>

> Bill Janssen wrote:
> 
> > I seem to spend a lot of time inside functions looking over values and
> > figuring out what to do do with them,
> 
> If you're really spending that much time doing
> things like that, I'd strongly suggest you take
> a step back and re-think your APIs.
> 
> --
> Greg

Typically, they're values that I've gotten back from other people's
APIs (another reason not generic functions alone don't solve enough of
the problem).  I seem to like to do things that they hadn't thought
of :-).

Bill

From jonathan-lists at cleverdevil.org  Thu Nov 23 03:30:01 2006
From: jonathan-lists at cleverdevil.org (Jonathan LaCour)
Date: Wed, 22 Nov 2006 21:30:01 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
Message-ID: <02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>

Phillip J. Eby wrote:

> In essence, interfaces turn libraries into "frameworks", but generic
> functions turn frameworks into libraries.  I didn't really discover
> this until January of last year, though, when I did some experiments
> in replacing various PEAK interfaces with generic functions.

This is easily the most important part of the conversation, IMO, and I
would hate for it to be missed. "Interfaces" in the Java universe are a
hard contract that everyone has to agree upon up-front, and if you don't
agree, you have to create your own framework that does the very same
things in a different way. What you end up with is a world of many
competing frameworks, none of which interoperate.

The alternative may not be as *familiar* but it certainly paints a much
brighter picture. If you have two different frameworks that take
slightly different approaches (or even _radically_ different approaches)
to the same problems, you can make them work together as one, rather
than having to pick one or the other. Its like framework detente!

Adopting Java-like interfaces just because they are "familiar" would be
a huge mistake, as it would condemn Python to the limitations of such an
implementation unnecessarily. This is Python 3000, not Java, and while I
certainly understand the argument of familiarity, I won't let something
that seems a tad magical at first scare me into a comfortable decision.

I have actually used generic functions (using RuleDispatch) in practice
and find them to not only be hugely powerful, but much more pragmatic.
Am I the only one out there who sees it this way?

--
Jonathan LaCour
http://cleverdevil.org




From janssen at parc.com  Thu Nov 23 03:30:32 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 18:30:32 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <4564E8F7.6060605@solarsail.hcs.harvard.edu> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<06Nov22.140512pst."58648"@synergy1.parc.xerox.com>
	<ek2iig$gni$1@sea.gmane.org>
	<06Nov22.145147pst."58648"@synergy1.parc.xerox.com>
	<4564E8F7.6060605@solarsail.hcs.harvard.edu>
Message-ID: <06Nov22.183038pst."58648"@synergy1.parc.xerox.com>

> > 1)  Mainly, all methods are special to someone.  
> 
> But some, such as those that have special meaning to the language
> itself, are more special than others.
> 
> __methods__ are not just plain old methods with funny names. You write
> regular methods to express functionality you wish your class to have,
> but you write __methods__ to provide low-level glue tying your class in
> with the interpreter.

That's a somewhat different rationale than Guido provided, thanks.  Or
maybe I just understand it better.  Still seems an odd way to
differentiate a namespace.

Bill

From janssen at parc.com  Thu Nov 23 03:31:32 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 22 Nov 2006 18:31:32 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
Message-ID: <06Nov22.183133pst."58648"@synergy1.parc.xerox.com>

> I don't know anything by CLOS.

I'll drop off a copy of the book on Monday.  Lord knows we have
zillions of extra copies floating around PARC.

Bill

From guido at python.org  Thu Nov 23 04:03:57 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 19:03:57 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>
Message-ID: <ca471dc20611221903nf9a6c1am1d70a5cba088c38d@mail.gmail.com>

On 11/22/06, Jonathan LaCour <jonathan-lists at cleverdevil.org> wrote:
> Phillip J. Eby wrote:
>
> > In essence, interfaces turn libraries into "frameworks", but generic
> > functions turn frameworks into libraries.  I didn't really discover
> > this until January of last year, though, when I did some experiments
> > in replacing various PEAK interfaces with generic functions.
>
> This is easily the most important part of the conversation, IMO, and I
> would hate for it to be missed. "Interfaces" in the Java universe are a
> hard contract that everyone has to agree upon up-front, and if you don't
> agree, you have to create your own framework that does the very same
> things in a different way. What you end up with is a world of many
> competing frameworks, none of which interoperate.
>
> The alternative may not be as *familiar* but it certainly paints a much
> brighter picture. If you have two different frameworks that take
> slightly different approaches (or even _radically_ different approaches)
> to the same problems, you can make them work together as one, rather
> than having to pick one or the other. Its like framework detente!
>
> Adopting Java-like interfaces just because they are "familiar" would be
> a huge mistake, as it would condemn Python to the limitations of such an
> implementation unnecessarily. This is Python 3000, not Java, and while I
> certainly understand the argument of familiarity, I won't let something
> that seems a tad magical at first scare me into a comfortable decision.
>
> I have actually used generic functions (using RuleDispatch) in practice
> and find them to not only be hugely powerful, but much more pragmatic.
> Am I the only one out there who sees it this way?

Sorry, but that was about as informative as a "+1" vote.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gsakkis at rutgers.edu  Thu Nov 23 04:08:47 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 22 Nov 2006 22:08:47 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ek2iqp$hsi$1@sea.gmane.org>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ek2iqp$hsi$1@sea.gmane.org>
Message-ID: <91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>

On 11/22/06, Fredrik Lundh <fredrik at pythonware.com> wrote:

> George Sakkis wrote:
>
> > After all, HCI-based arguments are usually a fancier way of
> > saying "it's a matter of taste"
>
> It would be easier to take you seriously if you gave up that "I don't
> understand their arguments; therefore I'm right and they're stupid"
> style of argumentation.  Mike and others said *exactly* the same thing
> as Guido, but that time, you just didn't want to listen.

First off, I never implied someone's stupid just because we don't
happen to agree on everything. As for Mike's answer, I went back and
read it again; please do the same. He doesn't address HCI reasons at
all, it's a purely technical argument by pointing out an inconsistency
of Java (which kinda distracts from the main point but that's ok).

George

From greg.ewing at canterbury.ac.nz  Thu Nov 23 04:12:28 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 16:12:28 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <06Nov22.182023pst.58648@synergy1.parc.xerox.com>
References: <ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<06Nov22.092436pst.58648@synergy1.parc.xerox.com>
	<4564FAB9.1010902@canterbury.ac.nz>
	<06Nov22.182023pst.58648@synergy1.parc.xerox.com>
Message-ID: <4565119C.6060700@canterbury.ac.nz>

Bill Janssen wrote:

> Typically, they're values that I've gotten back from other people's
> APIs

Can you give an example? I'm having a hard time
trying to think of an API that I've used which
returns unpredictable types.

Maybe it's those other APIs that need to be
redesigned?

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 23 04:19:49 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 16:19:49 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061123020054.GA31618@niemeyer.net>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
Message-ID: <45651355.7090605@canterbury.ac.nz>

Gustavo Niemeyer wrote:

> The user of both libraries may define how Apple adapts to
> IFruit.

Sure, but then there's no need for a formal framwork
for defining interfaces and adaptation -- by making
use of duck typing, the joint user of the two libraries
can wrap things as needed to make it all work.

The proponents of adaptation seem to be making a
stronger claim, however -- that somehow you can just
register an adaptor and have all users of IFruits
automatically understand Apples. That's the part that
I fail to understand.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 23 04:19:53 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 16:19:53 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <d11dcfba0611220842l5d83c319od0294f9e8e71cef6@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<d11dcfba0611220842l5d83c319od0294f9e8e71cef6@mail.gmail.com>
Message-ID: <45651359.8020800@canterbury.ac.nz>

Steven Bethard wrote:
> So I would have expected something like:
> 
>     str.lower[MyStringType] = MyStringType.lower
>     str.split[MyStringType] = MyStringType.split

But that would only work if everyone switched to
writing str.lower(s) everywhere instead of s.lower(),
etc. In other words, abolish method call notation
completely and use generic function calls instead.
I would say that *is* a radical proposal...

And it still wouldn't help SMTP.sendmail to tell
whether it was dealing with a string or a list
of strings.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 23 04:23:55 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 16:23:55 +1300
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
Message-ID: <4565144B.5060109@canterbury.ac.nz>

Guido van Rossum wrote:
> I chose len(x) over x.len() for HCI reasons
> 
> (a) For some operations, prefix notation just reads better than
> postfix
> (b) When I read code that says len(x) I *know* that it is asking for
> the length of something.

Just as a matter of interest, how far do you
think these principles apply or don't apply
to more recent things like iter.next() that
don't follow this pattern?

--
Greg

From gsakkis at rutgers.edu  Thu Nov 23 04:29:15 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 22 Nov 2006 22:29:15 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
Message-ID: <91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > Although I don't necessarily agree with the arguments from a puristic
> > point of view ("then why not make keys(), values(), items(), read(),
> > write(), ... builtin GFs ?"), they are indeed a better answer to the
> > FAQ entry. After all, HCI-based arguments are usually a fancier way of
> > saying "it's a matter of taste" and arguing against them is like
> > arguing that green is a better colour than red.
>
> I'm so glad you're not trying to hide that you're a troll.

I see how my reply could be misinterpreted as sarcastic but that was
not at all my intention. If I were a troll,that would be pretty
obvious after three years in c.l.py. I've supported the vast majority
of language changes since 2.2-2.3 that I first met Python (even
controversial ones such as decorators and conditional expressions) as
well as pretty much all goals listed in PEP 3100. Still, we don't have
to agree on everything, do we ? After all, what I said essentially was
that beauty is in the eye of the beholder. That hardly justifies the
troll labeling.

George

From greg.ewing at canterbury.ac.nz  Thu Nov 23 04:33:34 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Nov 2006 16:33:34 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
Message-ID: <4565168E.8000404@canterbury.ac.nz>

Phillip J. Eby wrote:

> Consider 'iter()', for example, which can be viewed as adapting an object 
> to the "iteration interface" and returning an object supporting iteration.

An architecture astronaut might view it that way, but
I don't. To my way of thinking, iter(x) creates a new
object that iterates over x. Calling it a form of
adaptation just muddies things with uneccessary words.

(The fact that iter(x) returns x when it's already an
iterator is a distraction. It's really just a kludge
so that the same for-statement syntax can be used on
both iterators and iterables.)

--
Greg

From guido at python.org  Thu Nov 23 04:56:23 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 19:56:23 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
	<91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
Message-ID: <ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>

Your presence here has been entirely a distraction. That's a troll to me.

On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > Although I don't necessarily agree with the arguments from a puristic
> > > point of view ("then why not make keys(), values(), items(), read(),
> > > write(), ... builtin GFs ?"), they are indeed a better answer to the
> > > FAQ entry. After all, HCI-based arguments are usually a fancier way of
> > > saying "it's a matter of taste" and arguing against them is like
> > > arguing that green is a better colour than red.
> >
> > I'm so glad you're not trying to hide that you're a troll.
>
> I see how my reply could be misinterpreted as sarcastic but that was
> not at all my intention. If I were a troll,that would be pretty
> obvious after three years in c.l.py. I've supported the vast majority
> of language changes since 2.2-2.3 that I first met Python (even
> controversial ones such as decorators and conditional expressions) as
> well as pretty much all goals listed in PEP 3100. Still, we don't have
> to agree on everything, do we ? After all, what I said essentially was
> that beauty is in the eye of the beholder. That hardly justifies the
> troll labeling.
>
> George
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ironfroggy at gmail.com  Thu Nov 23 04:59:15 2006
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Wed, 22 Nov 2006 22:59:15 -0500
Subject: [Python-3000] defop ?
In-Reply-To: <5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
Message-ID: <76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>

This whole thing seems a bit off from start to finish. A seperate
definition syntax with a special name/expression weirdo thingy, etc.
Anyway, we need something that handles this but isnt specific to this.
There are other situations it may be useful. Essentially we want to
have names that are known to be in some namespace, even within
another. Does that make sense?

Perhaps we could allow a special type of key in __dict__'s (and
__slots__'s ?) that was more than a simple string name but included a
namespace or context in which the name was to be understood. Even a
simple 2-tuple would be enough. (operator, 'len') might be the entry
for a method that the len builtin called, for example. A simple syntax
would preceed the . operator to create names with this convention,
maybe even using the familiar :: from other languages.

class myclass(object):
    def operator::len(self):
        return self.length
    ...

Very simple, can have more uses than the single, limited use case
being discussed.

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 02:06 PM 11/22/2006 -0800, Guido van Rossum wrote:
> > > (The 'decorate_class' function applies its argument as a class decorator
> > > for the class its caller is invoked from.  This is the same mechanism (and
> > > actual code!) that Zope uses to do in-body class decorators like
> > > 'implements()'.)  decorate_class raises a SyntaxError if its caller is not
> > > being run directly inside a class body.
> >
> >That's a bizarre use of that exception.
>
> Well, the idea was that a class decorator can only be invoked inside a
> class, so it's a syntax error to invoke it elsewhere.  And, there's little
> chance of confusing the SyntaxError with some error generated by *running*
> the decorator itself, as opposed to merely its positioning.
>
> Anyway, it seemed like a good idea at the time.  :)
>
>
> >I'm guessing that decorate_class squirrels away some info for later?
>
> It introduces a stealth metaclass to the class frame, that will call the
> decorator.  If there's an existing metaclass in the frame, it invokes it
> first to create the class.  The devil is in the details, of course, and it
> does require that you define any explicit metaclass *before* using any
> class decorators, but this is a documented limitation of the DecoratorTools
> library.
>
>
> >Everything here seems designed to hide the implementation details,
> >which is great for actual use, but kills understanding of the
> >mechanism. :-(
> >
> > > And yes, this uses frame magic, and no, it's not fragile, as it knows the
> > > difference between class and non-class frames.
> >
> >Wow. How can you tell?
>
> Basically, if f_locals contains a '__module__' key that's equal to the
> '__name__' in f_globals, although there are some interesting corner cases
> involving 'exec' -- the details are in the implementation.  The relevant
> function is the 'frameinfo(frame)' function which tells you what "kind" of
> frame you have.
>
>
> >  Does it work with Jython, IronPython and PyPy?
>
> PyPy should be fine, since it uses the same code objects and
> bytecode.  IIRC, the others emulate enough of the frame protocol that the
> '__module__' and '__name__' checks should be no problem.  However, I have
> not tried it in any of these.  Since this hack requires metaclass support,
> there was no point to attempting it in Jython.  And when the code was
> written, neither IronPython nor PyPy existed yet.
>
>
> >I still find it fragile somehow -- or at least severely hackish. I
> >like the other proposal (where it puts a special attribute on the
> >function object which is then found by the type) better, it doesn't
> >require sys._getframe().
>
> Sure, there are certainly lots of other ways to implement it!  What I
> showed was just to demonstrate the *possibility*.  If there were defop
> syntax, then of course the compiler could simply generate the appropriate
> bytecode following the MAKE_CLASS operation.  Or MAKE_CLASS could look for
> some data inserted into the source dictionary.
>
> decorate_class() is just what I had handy to answer your question - it
> should not be confused with being the One Obvious Way to do it is.  It's
> just proof of possibility (and practical workability, in that it has been
> long and widely used successfully with at least CPython).
>
> But, there are still other ways.  The compiler could stick
> "__owning_class__" attributes on any function defined in a class, and
> __addmethod__ could defer actually registering the functions until the
> generic function was invoked, thus ensuring that __owning_class__ would be
> set first.
>
> In general, if we're allowed to change the implementation of Python, it's
> easy to find some way to do it.  But you asked for a way to do it in (I
> thought) today's Python, so that's what I gave you.
>
>
> >I'm assuming that's outside the context of the BinaryTree class --
> >since inside it, Binarytree is undefined.
>
> Correct.
>
>
> >Hm, I think this is the first time in this thread that you meant to
> >use 'iter', 'len' etc. as "tokens".
>
> I don't understand what you mean.  They're objects, and so usable as
> tokens.  That's why I've also been saying they can also represent
> "abilities", with defop being a way to unify their use as generic operation
> (e.g. iter()), implementation (i.e. defop iter), and "interface" or ability
> (e.g. 'hasmethod(iter,ob)' or something of that sort).
>
>
> >I think I'd rather change them a
> >bit so that they can be objects that actually understand __addmethod__
> >and __hasmethod__.
>
> Okay.  That's an example of the kind of preference I have no way to divine
> ahead of time.  If I knew you preferred function attributes for deferred
> registration, and to make these objects of a special type, I'd have
> proposed it that way.  :)  My assumption was that changing the type of
> Python builtin functions would be considered more "radical" than the idea
> of defop, since it requires changing something, rather than just adding.
>
>
> > > then there's no need to have
> > > actual addmethod/hasmethod functions.  But that's sort of like saying we
> > > don't need getattr() any more because we can just use
> > > type(ob).__getattribute__() now!  :)
> >
> >But I think you're really proposing an addmethod builtin that calls an
> >__addmethod__ method, right?
>
> Yes, assuming that we can call do this with regular functions too.  That
> is, as long as there's some way to retroactively overload a regular
> function.  Other than that, I'm not worried about the need to retrofit any
> other existing generic function types; their authors will have to change
> them for Py3K anyway.  :)
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com
>


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From gsakkis at rutgers.edu  Thu Nov 23 05:11:30 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 22 Nov 2006 23:11:30 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
	<91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
	<ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
Message-ID: <91ad5bf80611222011j1ec3a872v478ccf6d1923f706@mail.gmail.com>

If you consider distraction a thread that drew more than 100 replies,
many of them very insightful (including Andrew Koenig's about
abilities that has spawned 3-4 new threads), please allow me to
disagree.

On 11/22/06, Guido van Rossum <guido at python.org> wrote:

> Your presence here has been entirely a distraction. That's a troll to me.
>
> On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > > On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > > Although I don't necessarily agree with the arguments from a puristic
> > > > point of view ("then why not make keys(), values(), items(), read(),
> > > > write(), ... builtin GFs ?"), they are indeed a better answer to the
> > > > FAQ entry. After all, HCI-based arguments are usually a fancier way of
> > > > saying "it's a matter of taste" and arguing against them is like
> > > > arguing that green is a better colour than red.
> > >
> > > I'm so glad you're not trying to hide that you're a troll.
> >
> > I see how my reply could be misinterpreted as sarcastic but that was
> > not at all my intention. If I were a troll,that would be pretty
> > obvious after three years in c.l.py. I've supported the vast majority
> > of language changes since 2.2-2.3 that I first met Python (even
> > controversial ones such as decorators and conditional expressions) as
> > well as pretty much all goals listed in PEP 3100. Still, we don't have
> > to agree on everything, do we ? After all, what I said essentially was
> > that beauty is in the eye of the beholder. That hardly justifies the
> > troll labeling.
> >
> > George
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/george.sakkis%40gmail.com
>

From guido at python.org  Thu Nov 23 05:12:33 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 20:12:33 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <91ad5bf80611222011j1ec3a872v478ccf6d1923f706@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
	<91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
	<ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
	<91ad5bf80611222011j1ec3a872v478ccf6d1923f706@mail.gmail.com>
Message-ID: <ca471dc20611222012x4f85abefidc5173e4b9497108@mail.gmail.com>

It could have been a much shorter thread without you.

On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> If you consider distraction a thread that drew more than 100 replies,
> many of them very insightful (including Andrew Koenig's about
> abilities that has spawned 3-4 new threads), please allow me to
> disagree.
>
> On 11/22/06, Guido van Rossum <guido at python.org> wrote:
>
> > Your presence here has been entirely a distraction. That's a troll to me.
> >
> > On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > > > On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > > > Although I don't necessarily agree with the arguments from a puristic
> > > > > point of view ("then why not make keys(), values(), items(), read(),
> > > > > write(), ... builtin GFs ?"), they are indeed a better answer to the
> > > > > FAQ entry. After all, HCI-based arguments are usually a fancier way of
> > > > > saying "it's a matter of taste" and arguing against them is like
> > > > > arguing that green is a better colour than red.
> > > >
> > > > I'm so glad you're not trying to hide that you're a troll.
> > >
> > > I see how my reply could be misinterpreted as sarcastic but that was
> > > not at all my intention. If I were a troll,that would be pretty
> > > obvious after three years in c.l.py. I've supported the vast majority
> > > of language changes since 2.2-2.3 that I first met Python (even
> > > controversial ones such as decorators and conditional expressions) as
> > > well as pretty much all goals listed in PEP 3100. Still, we don't have
> > > to agree on everything, do we ? After all, what I said essentially was
> > > that beauty is in the eye of the beholder. That hardly justifies the
> > > troll labeling.
> > >
> > > George
> > >
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/george.sakkis%40gmail.com
> >
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gsakkis at rutgers.edu  Thu Nov 23 05:15:44 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Wed, 22 Nov 2006 23:15:44 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611222012x4f85abefidc5173e4b9497108@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
	<91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
	<ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
	<91ad5bf80611222011j1ec3a872v478ccf6d1923f706@mail.gmail.com>
	<ca471dc20611222012x4f85abefidc5173e4b9497108@mail.gmail.com>
Message-ID: <91ad5bf80611222015s6d41c612x7035169154283558@mail.gmail.com>

Feel free to ignore it if you find it trivial/stupid/irrelevant; many
other apparently didn't.

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> It could have been a much shorter thread without you.
>
> On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > If you consider distraction a thread that drew more than 100 replies,
> > many of them very insightful (including Andrew Koenig's about
> > abilities that has spawned 3-4 new threads), please allow me to
> > disagree.
> >
> > On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> >
> > > Your presence here has been entirely a distraction. That's a troll to me.
> > >
> > > On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > > On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > > > > On 11/22/06, George Sakkis <gsakkis at rutgers.edu> wrote:
> > > > > > Although I don't necessarily agree with the arguments from a puristic
> > > > > > point of view ("then why not make keys(), values(), items(), read(),
> > > > > > write(), ... builtin GFs ?"), they are indeed a better answer to the
> > > > > > FAQ entry. After all, HCI-based arguments are usually a fancier way of
> > > > > > saying "it's a matter of taste" and arguing against them is like
> > > > > > arguing that green is a better colour than red.
> > > > >
> > > > > I'm so glad you're not trying to hide that you're a troll.
> > > >
> > > > I see how my reply could be misinterpreted as sarcastic but that was
> > > > not at all my intention. If I were a troll,that would be pretty
> > > > obvious after three years in c.l.py. I've supported the vast majority
> > > > of language changes since 2.2-2.3 that I first met Python (even
> > > > controversial ones such as decorators and conditional expressions) as
> > > > well as pretty much all goals listed in PEP 3100. Still, we don't have
> > > > to agree on everything, do we ? After all, what I said essentially was
> > > > that beauty is in the eye of the beholder. That hardly justifies the
> > > > troll labeling.
> > > >
> > > > George
> > > >
> > >
> > >
> > > --
> > > --Guido van Rossum (home page: http://www.python.org/~guido/)
> > > _______________________________________________
> > > Python-3000 mailing list
> > > Python-3000 at python.org
> > > http://mail.python.org/mailman/listinfo/python-3000
> > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/george.sakkis%40gmail.com
> > >
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/george.sakkis%40gmail.com
>

From guido at python.org  Thu Nov 23 05:29:54 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 20:29:54 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <4565144B.5060109@canterbury.ac.nz>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<4565144B.5060109@canterbury.ac.nz>
Message-ID: <ca471dc20611222029mb7cfa7cl9436db31e4b0957a@mail.gmail.com>

On 11/22/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
> > I chose len(x) over x.len() for HCI reasons
> >
> > (a) For some operations, prefix notation just reads better than
> > postfix
> > (b) When I read code that says len(x) I *know* that it is asking for
> > the length of something.
>
> Just as a matter of interest, how far do you
> think these principles apply or don't apply
> to more recent things like iter.next() that
> don't follow this pattern?

it.next() isn't something one does commonly; it's mostly implied by
the for-loop. iter(seq) which was invented at the same time *is* a
prefix op.

There's probably a somewhat better rule hiding in all this; something
that explains why keys() is a method. Perhaps it depends on the
generality of the operation -- keys() is pretty unique to mappings,
while len() and iter() apply to all containers. Similarly, next() only
applies to iterators (even if there are about as many iterators as
their are containers, they all have the same API). I guess the chances
that something becomes an "operation" are better if it applies to a
wide variety of types (that have otherwise different interfaces,
unlike iterators which all have the same interface).

Interestingly, keys() being a method makes it pretty much impossible
to turn it into a generic function. This doesn't seem a big loss, it's
unlikely one could implement it easily for a mapping type that doesn't
provide keys() natively. OTOH, values() and items() could be
implemented easily for anything that has keys() and __getitem__() (or
iterkeys() and __getitem__()). But it's also easy to write an app
without using values() and items() (using keys() and __getitem__()) so
this is still not a big deal.

One thing that rubs me the wrong way about generic functions is that
it appears to go against OO. Now I'm not someone to take OO as
religion, but there's something uncomfortable (for me) about how, in
Phillip's world, many things become functions instead of methods,
which brings along concerns about the global namespace filling up, and
also about functionality being spread randomly across too many
modules. I fear I will miss the class as a convenient focus for
related functionality.

While [@]defop moves the operation definitions back into the class,
the invocations still takes the form of a function call. We could end
up with two different mainstream notations: foo(x) and x.foo(), and
which one is used for a particular operation is the result of a very
subtle decision process that takes into account different evolution
and/or usage patterns for the classes involved (as is shown by the
difficulty we've had explaining why len() and iter() are functions but
next() and keys() are methods, for example). Of course Python always
had functions and methods, but the choice used to be so much simpler
before generic functions: a function has a single implementation, a
method can be overridden. I will miss that simplicity.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Nov 23 05:34:25 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 23:34:25 -0500
Subject: [Python-3000] defop ?
In-Reply-To: <76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122233205.01f446a8@sparrow.telecommunity.com>

At 10:59 PM 11/22/2006 -0500, Calvin Spealman wrote:
>Perhaps we could allow a special type of key in __dict__'s (and
>__slots__'s ?) that was more than a simple string name but included a
>namespace or context in which the name was to be understood. Even a
>simple 2-tuple would be enough. (operator, 'len') might be the entry
>for a method that the len builtin called, for example. A simple syntax
>would preceed the . operator to create names with this convention,
>maybe even using the familiar :: from other languages.
>
>class myclass(object):
>     def operator::len(self):
>         return self.length
>     ...
>
>Very simple, can have more uses than the single, limited use case
>being discussed.

So, how does that work for adding methods after the fact, *outside* of a 
class?  (See Guido's BinaryTree flattening example.)


From steven.bethard at gmail.com  Thu Nov 23 05:38:09 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 22 Nov 2006 21:38:09 -0700
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <45651359.8020800@canterbury.ac.nz>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
	<d11dcfba0611220842l5d83c319od0294f9e8e71cef6@mail.gmail.com>
	<45651359.8020800@canterbury.ac.nz>
Message-ID: <d11dcfba0611222038q5de8a174j3be5f6d40f68b17@mail.gmail.com>

On 11/22/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Steven Bethard wrote:
> > So I would have expected something like:
> >
> >     str.lower[MyStringType] = MyStringType.lower
> >     str.split[MyStringType] = MyStringType.split
>
> But that would only work if everyone switched to
> writing str.lower(s) everywhere instead of s.lower(),

I don't think so, but perhaps I misunderstood the proposal.  I thought
it would just rely on someone defining a String Interface as the union
of ``str.lower``, ``str.split``, etc.  Then to figure out what to do
with your ``string_or_list`` argument, you'd test it against this
String Interface. You could then freely make calls like ``s.lower()``,
knowing that they were calling the equivalent of ``str.lower()``.

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Thu Nov 23 05:42:34 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 22 Nov 2006 20:42:34 -0800
Subject: [Python-3000] defop ?
In-Reply-To: <76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
Message-ID: <ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>

On 11/22/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> This whole thing seems a bit off from start to finish. A seperate
> definition syntax with a special name/expression weirdo thingy, etc.

I have the same gut feelings but find it hard to explain why.

But I've learned to trust my gut -- eventually it will come to me.

Perhaps it's okay for the operation definitions to be outside the
class for which they apply? That at least clarifies that their
*invocation* also doesn't involve method notation.

> Anyway, we need something that handles this but isnt specific to this.
> There are other situations it may be useful. Essentially we want to
> have names that are known to be in some namespace, even within
> another. Does that make sense?
>
> Perhaps we could allow a special type of key in __dict__'s (and
> __slots__'s ?) that was more than a simple string name but included a
> namespace or context in which the name was to be understood. Even a
> simple 2-tuple would be enough. (operator, 'len') might be the entry
> for a method that the len builtin called, for example. A simple syntax
> would preceed the . operator to create names with this convention,
> maybe even using the familiar :: from other languages.
>
> class myclass(object):
>     def operator::len(self):
>         return self.length
>     ...
>
> Very simple, can have more uses than the single, limited use case
> being discussed.

Not sure I like this better. My spelling of "operator::XXX" is
"__XXX__" and I like that just fine -- no new syntax needed.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Nov 23 05:47:12 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 22 Nov 2006 23:47:12 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <4565168E.8000404@canterbury.ac.nz>
References: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061122233433.028ea230@sparrow.telecommunity.com>

At 04:33 PM 11/23/2006 +1300, Greg Ewing wrote:
>Phillip J. Eby wrote:
>
> > Consider 'iter()', for example, which can be viewed as adapting an object
> > to the "iteration interface" and returning an object supporting iteration.
>
>An architecture astronaut might view it that way, but

Calling people names isn't particularly conducive to a technical 
discussion...  or any other kind of discussion, for that matter.  :)


>I don't. To my way of thinking, iter(x) creates a new
>object that iterates over x. Calling it a form of
>adaptation just muddies things with uneccessary words.

You're right.  We should never reuse words to describe similar things, 
because then people would be confused.  We should only ever invent *new* 
words, because everything is different from everything else.  ;-)

More seriously, the purpose of descriptive words like "adapting" is to 
allow comparison and the discovery of patterns.  If iter() were unique in 
all the world, it would make no sense to have a special word that applied 
only to it.  But it's far from unique...


>(The fact that iter(x) returns x when it's already an
>iterator is a distraction. It's really just a kludge
>so that the same for-statement syntax can be used on
>both iterators and iterables.)

That kludge, as you call it, is something called idempo...  whoops, I 
almost used an architecture word there!  Whew, that was close!  :)  Suffice 
to say, however, that it is a property shared by other builtins such as 
str, int, tuple, and bool, and that many perverted Python developers 
actually believe this unspeakable idea is a highly desirable property in a 
builtin function or type!  Let's not tell them it's really a "kludge", 
though, 'cause it might hurt their feelings.  :)

crudely-attempting-Tim-Peters-style-humor'ly yours,

--PJE


From pje at telecommunity.com  Thu Nov 23 07:01:18 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 01:01:18 -0500
Subject: [Python-3000] Special methods and interface-based type  system
In-Reply-To: <ca471dc20611222029mb7cfa7cl9436db31e4b0957a@mail.gmail.com
 >
References: <4565144B.5060109@canterbury.ac.nz>
	<-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<4565144B.5060109@canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20061122234926.02d249e0@sparrow.telecommunity.com>

At 08:29 PM 11/22/2006 -0800, Guido van Rossum wrote:
>One thing that rubs me the wrong way about generic functions is that
>it appears to go against OO. Now I'm not someone to take OO as
>religion, but there's something uncomfortable (for me) about how, in
>Phillip's world, many things become functions instead of methods,
>which brings along concerns about the global namespace filling up, and
>also about functionality being spread randomly across too many
>modules. I fear I will miss the class as a convenient focus for
>related functionality.

I originally proposed a solution for this back in January '05, but it was 
too premature.  But since you have now stated the problem that the proposal 
was intended to solve, perhaps the solution has a chance now.  :)

I will try to be as concrete as possible.  Let's start with an actual, 
hopefully non-exploding 'Interface' implementation, based on an assumption 
that we have generic functions available:

     class InterfaceClass(type):
         def __init__(cls, name, bases, cdict):
             for k,v in cdict.items():
                 # XXX this should probably skip at least __slots__,
                 #     __metaclass__, and __module__, but oh well
                 cdict[k] = AdaptingDescriptor(v)

     class Interface:
         __metaclass__ = InterfaceClass
         __slots__ = '__self__'

         def __init__(self, subject):
             # this isinstance() check should be replaced by an
             # 'unwrap()' generic function, so other adapter types
             # will work, but this is just an example, so...
             if isinstance(subject, Interface):
                 subject = subject.__self__
             self.__self__ = subject

     class AdaptingDescriptor:
         def __init__(self, descriptor):
             self.wrapped = descriptor
         def __get__(self, ob, typ=None):
             if ob is None:
                 return self
             return self.wrapped.__get__(ob.__self__, typ)

Now, using this new "interface framework", let's implement a small 
"mapping" typeclas... er, interface.

     class Mapping(Interface):
         def keys(self):
             return [k for k,v in self.items()]
         def items(self):
             return [k,self[k] for k in self.keys()]
         # ... other self-recursive definitions

What does this do?  Well, we can now call Mapping(foo) to turn an arbitrary 
object into something that has Mapping's generic functions as its methods, 
and invokes them on foo!  (I am assuming here that normal functions are 
implicitly overloadable, even if that means they change type at runtime to 
do so.)  We could even use interfaces for argument type declarations, to 
automatically put things in the "right namespace" for what the code expects 
to use.  That is, if you declare an argument to be a Mapping, then that's 
what you get.  If you call .keys() on the resulting adapted object and the 
type doesn't support the operation, you get an error.

Too late a form of error checking you say?  Well, make a more sophisticated 
factory mechanism in Interface.__new__ that actually creates (and caches) 
different adapter types based on the type of object being adapted, so that 
hasattr() tests will work on the wrapped type, or so that you can get an 
early error if none of the wrapped generic functions has a method defined 
for the target type.

A few important points here:

1. A basic interface mechanism is extemely simple to implement, given 
generic functions

2. It is highly customizable with respect to error checking and other 
features, even on a per-user basis, because there doesn't have to be only 
one "true" Interface type to rule them all (or one true generic function 
type either, but that's a separate discussion).

3. It allows interfaces to include partial implementations, ala Ping and 
Alex's past proposals, thus allowing you to implement partial mapping or 
"file" objects and have the rest of the interface's implementation filled 
in for you

4. It allows you to hide the very existence of the notion of a "generic 
function", if you prefer not to think about such things

5. It even supports interface inheritance and interface algebra: 
subclassing an interface allows adding new operations, and simple 
assignment suffices to compose new interfaces, e.g.:

     class MappingItems(Interface):
         items = Mapping.items

Notice that nothing special is required, this "just works" as a natural 
consequence of the rest of the implementation shown.

Okay, so now you want to know how to *implement* a "Mapping".  Well, 
simplest but most tedious, you can just register operations directly, e.g.:

      class MyMapping:
          def __init__(self, data):
              self.data = dict(data)
          defop operator.getitem(self, key):
              return self.data[key]
          defop Mapping.items(self):
              return self.data.items()

But as you can imagine, this would probably get a bit tedious if you're 
implementing lots of methods.  So, we can add metaclasses or class 
decorators here to say, "I implement these interfaces, so any methods I 
have whose names match the method names in the interfaces, please hook 'em 
up for me."  I'm going to leave out the implementation, as it should be a 
straightforward exercise for the reader to come up with many ways by which 
it can be accomplished.  The spelling might be something like:

     class MyMapping:
         implements(Mapping)

         def items(self):
             ...

         #etc.

At which point, we have now come full circle to being able to provide all 
of the features of interfaces, adaptation, and generic functions, without 
forcing anyone to give up the tasty OO flavor of method calls.  Heck, they 
can even keep the way they spell existing adaptation calls (e.g. IFoo(bar) 
to adapt bar to IFoo) in PEAK, Twisted, and Zope!

And finally, note that if you only want to perform one method call on a 
given object, you can also use the generics directly, e.g. 
Mapping.items(foo) instead of Mapping(foo).items().

Voila -- generic goodness and classic OO method-calling simplicity, all in 
one simple to implement package.  It should now be apparent why I said that 
interfaces are trivial to implement if you define them as namespaces for 
generic functions, rather than as namespaces for methods.

There are many spinoffs possible, too.  For example, you could have a 
factory function that turns an existing class's public operations into an 
interface object.  There are also probably also some dark corners of the 
idea that haven't been explored, because when I first proposed basically 
this idea in '05, nobody was ready for it.  Now maybe we can actually talk 
about the implications.


From ironfroggy at gmail.com  Thu Nov 23 08:10:57 2006
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Thu, 23 Nov 2006 02:10:57 -0500
Subject: [Python-3000] Fwd:  defop ?
In-Reply-To: <76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
Message-ID: <76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>

I need to move away from the GMail client for lists or they need to
fix it. Sorry Guido, forwarding to the list as GMail should be doing
in the first place!

---------- Forwarded message ----------
From: Calvin Spealman <ironfroggy at gmail.com>
Date: Nov 23, 2006 2:10 AM
Subject: Re: [Python-3000] defop ?
To: Guido van Rossum <guido at python.org>


On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> Not sure I like this better. My spelling of "operator::XXX" is
> "__XXX__" and I like that just fine -- no new syntax needed.

Well, that might be a spelling of `operator::XXX` but what about other
use cases like `third_party_interface_system::adapt` or
`anything_at_all::XXX`? Thats what I mean with not being too limiting
and solving the problem in a way that opens up solutions to other
problems. I get the opposition to it, but it seems reasonable,
nonetheless.

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> So, how does that work for adding methods after the fact, *outside* of a
> class?  (See Guido's BinaryTree flattening example.)

class MyClass(object):
  pass
MyClass.operator::iter = lambda self: iter((self.a, self.b, self.c))

Or, possibly:

def MyClass.operator::iter(self):
  yield a
  yield b
  yield c

But, I understand if that would get even more opposition, so I won't
push it, but its on the table.

Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From fredrik at pythonware.com  Thu Nov 23 08:23:40 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 08:23:40 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2cf8$su5$1@sea.gmane.org>	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>	<ek2iqp$hsi$1@sea.gmane.org>
	<91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>
Message-ID: <ek3i9t$qe8$1@sea.gmane.org>

George Sakkis wrote:

> First off, I never implied someone's stupid just because we don't
> happen to agree on everything.

brushing off a carefully thought out design and the process that led
up to it with "it's just like picking between two random colors" is a 
pretty direct way of saying that you consider yourself so much smarter 
than the people involved that you don't even care about what they're doing.

> As for Mike's answer, I went back and
> read it again; please do the same. He doesn't address HCI reasons at
> all, it's a purely technical argument by pointing out an inconsistency
> of Java (which kinda distracts from the main point but that's ok).

HCI is all about picking the *right* technical solutions, though. 
Python's designed for humans, and humans aren't robots; the solutions 
you choose and the way you integrate them affect how people understands 
them and use them, on many different levels.  what Mike wrote about was 
one such case (which follows from 1b in GvR's post), where Python's way 
of doing things helped people avoid a certain real-life problem, without 
even noticing.

seriously, if you think that HCI (or for that matter, design) is only 
about what color you use for a technical solution designed by real 
engineers, you've pretty much missed the point of the whole field.  and 
you've pretty much missed the point of Python's design.

</F>


From talin at acm.org  Thu Nov 23 08:44:02 2006
From: talin at acm.org (Talin)
Date: Wed, 22 Nov 2006 23:44:02 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ek3i9t$qe8$1@sea.gmane.org>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2cf8$su5$1@sea.gmane.org>	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>	<ek2iqp$hsi$1@sea.gmane.org>	<91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>
	<ek3i9t$qe8$1@sea.gmane.org>
Message-ID: <45655142.7050800@acm.org>

Fredrik Lundh wrote:
> George Sakkis wrote:
> 
>> First off, I never implied someone's stupid just because we don't
>> happen to agree on everything.
> 
> brushing off a carefully thought out design and the process that led
> up to it with "it's just like picking between two random colors" is a 
> pretty direct way of saying that you consider yourself so much smarter 
> than the people involved that you don't even care about what they're doing.

That's not what he said though. If you go back and read his original 
posting, he was commenting on the quality of *most arguments* about HCI, 
not HCI itself, or all HCI-related discussions.

While I wouldn't say it quite so strongly, I do think that there is a 
great deal of subjectivity in HCI discussions. One person's 'intuitive' 
may be another person's 'orange smoke', as we've seen in the current 
discussion.

I'm not saying that such discussion don't have merit, but we must be 
careful not to take our own sense of 'obviousness' too seriously. And 
that's what I think George was trying to say.

-- Talin

From fredrik at pythonware.com  Thu Nov 23 08:52:04 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 08:52:04 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122233433.028ea230@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<4565168E.8000404@canterbury.ac.nz>
	<5.1.1.6.0.20061122233433.028ea230@sparrow.telecommunity.com>
Message-ID: <ek3jv5$ukb$1@sea.gmane.org>

Phillip J. Eby wrote:

> That kludge, as you call it, is something called idempo...  whoops, I 
> almost used an architecture word there! 

No, that's a mathematical term.  And while Greg wasn't using that word, 
he was actually talking about its CS form, where it means something 
similar, but not entirely equivalent.

In math, something is idempotent if you can change f(x) to f(f(x)) and 
still get the same result; in CS, something is idempotent if you can do 
f(x) twice on the same x, and get the same result both times.

Python's len() function is idempotent in the CS sense, but not in the 
math sense.  And Python's iter() is not necessarily idempotent in the CS 
sense when used on an *iterable*, but it is, in both senses, when used 
on an iterator.  Which was Greg's point, I think.

 > We should never reuse words to describe similar things, because then
 > people would be confused.

Well, I'd say you just illustrated the danger of using the same word to 
described two similar but yet distinctly different concepts.

</F>


From bingham at cenix-bioscience.com  Thu Nov 23 08:58:38 2006
From: bingham at cenix-bioscience.com (Aaron Bingham)
Date: Thu, 23 Nov 2006 08:58:38 +0100
Subject: [Python-3000] Builtin iterator type
In-Reply-To: <45631A5F.5080501@gmail.com>
References: <91ad5bf80611152232m2e375225vcc5dea6665effb1@mail.gmail.com>	<002901c70b54$9b1dd350$6402a8c0@arkdesktop>	<ca471dc20611182028q195cebcex23135f29ec0beb9d@mail.gmail.com>	<455FEEBB.9010100@cs.byu.edu>	<20061119115009.V14417@tribble.ilrt.bris.ac.uk>	<06Nov19.170737pst."58648"@synergy1.parc.xerox.com>
	<4561857A.8080101@cenix-bioscience.com>
	<45631A5F.5080501@gmail.com>
Message-ID: <456554AE.1050803@cenix-bioscience.com>

Nick Coghlan wrote:

> Aaron Bingham wrote:
>
>> Bill Janssen wrote:
>>
>>>> Java interfaces are very useful, however. Java programming seems to 
>>>> be less and less about inheritance and more and more about 
>>>> implementing interfaces; at least it does amongst Java programmers 
>>>> with taste :-)
>>>>    
>>>
>>> It seems to me that that's where Python has a real advantage.  With
>>> real support for multiple inheritance, Python "interfaces" could be
>>> real classes (either like real Java classes or Java abstract classes),
>>> perhaps providing default implementations.  You get the goodness of
>>> mix-ins, along with interface communication.
>>>
>> I agree.  In Java, interfaces are necessary because multiple 
>> inheritance is not supported.  I see no good reason to add an 
>> additional language mechanism for interfaces when multiple 
>> inheritance would do the job, AFAICT.
>
>
> Just because multiple inheritance is *possible* in Python, don't make 
> the mistake of thinking that it is straight forward. Aside from mixin 
> classes that themselves inherit directly from object, multiple 
> inheritance can get very messy outside of tightly controlled type 
> hierarchies (cooperative calls in particular can become a nightmare). 
> This flies in the face of Python's use as a glue language to tie 
> multiple components together (particularly given the problem that some 
> of the elements being integrated may come from environments that 
> *don't* support multiple inheritance, like the JVM).

It's true that multiple inheritance in Python can get quite messy.  
Python lacks two mechanisms to control this complexity that are, AFAIK, 
only available in Eiffel: renaming and hiding of superclass attributes 
and methods.  Implementing these mechanisms in a dynamically typed 
languages is AFAICT impossible.  I won't go into a detailed explanation 
here unless someone requests it.

Regards,

-- 
--------------------------------------------------------------------
Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH
--------------------------------------------------------------------


From fredrik at pythonware.com  Thu Nov 23 08:59:39 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 08:59:39 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <45655142.7050800@acm.org>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2cf8$su5$1@sea.gmane.org>	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>	<ek2iqp$hsi$1@sea.gmane.org>	<91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>	<ek3i9t$qe8$1@sea.gmane.org>
	<45655142.7050800@acm.org>
Message-ID: <ek3kdc$vup$1@sea.gmane.org>

Talin wrote:

>> brushing off a carefully thought out design and the process that led
>> up to it with "it's just like picking between two random colors" is a 
>> pretty direct way of saying that you consider yourself so much smarter 
>> than the people involved that you don't even care about what they're doing.
> 
> That's not what he said though. If you go back and read his original 
> posting, he was commenting on the quality of *most arguments* about HCI, 
> not HCI itself, or all HCI-related discussions.

same thing.  he didn't get the argument, therefore it didn't have
the right quality, and therefore it's not very important, and both
the argument and the person making it can be ignored.  it's a circular 
argument.  and the only way to snap out of it is to start asking 
yourself "I don't get his argument, but he definitely has a track
record; maybe that guy knows something that I don't".

</F>


From fredrik at pythonware.com  Thu Nov 23 09:04:12 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 09:04:12 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122184343.02871500@sparrow.telecommunity.com>
References: <ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>	<ek2lbr$pme$1@sea.gmane.org>	<ca471dc20611221519l6be3b322i14f32a387db61aaa@mail.gmail.com>
	<ek2mto$tp7$1@sea.gmane.org>
	<5.1.1.6.0.20061122184343.02871500@sparrow.telecommunity.com>
Message-ID: <ek3klt$19m$1@sea.gmane.org>

Phillip J. Eby wrote:

>> (but I'm suspect that my view of "generics" is somewhat different from
>> Phillips, so let's focus on fully understanding his proposal first)
> 
> So now I'm curious.

Me too.  Let me see if I can figure out what it is, well enough to write 
it down.

 > Maybe it's better.

I'm pretty sure we're close from a *technological* perspective (at least 
on the "internal mechanism" level.  But I'm not sure we're close from a 
*conceptual* perspective.  Or something.  I'll think about it.

</F>


From thomas at thomas-lotze.de  Thu Nov 23 08:47:35 2006
From: thomas at thomas-lotze.de (Thomas Lotze)
Date: Thu, 23 Nov 2006 08:47:35 +0100
Subject: [Python-3000] Abilities / Interfaces
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>	<20061122202924.GA21361@niemeyer.net>	<ca471dc20611221238o60c1a4b3rca74b07036d361fa@mail.gmail.com>
Message-ID: <pan.2006.11.23.07.24.19.642912@ID-174572.user.uni-berlin.de>

Guido van Rossum wrote:

>> I'm sure you're aware about it, but in Zope 3 terminology, these are
>> 'provide' and 'implement', respectively.
> 
> I like provide, but I'm not so sure about implement, since it is awfully
> ambiguous -- most of the time it is the class that does the implementing.
> That's why I settled for "has".

JFTR: In Zope3, classes "implement" interfaces while instances "provide"
them.

-- 
Thomas




From fredrik at pythonware.com  Thu Nov 23 09:05:59 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 09:05:59 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov22.181438pst."58648"@synergy1.parc.xerox.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2iig$gni$1@sea.gmane.org>
	<8853879015409334245@unknownmsgid>	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
	<06Nov22.181438pst."58648"@synergy1.parc.xerox.com>
Message-ID: <ek3kp8$19m$2@sea.gmane.org>

Bill Janssen wrote:

>> Ow, that's the first time you describe this particular wrinkle.
> 
> Yep.  I hesitated to bring it up, but if there's a need for separate
> namespaces for method names, why not do it right?

what makes you so sure that Python's current design isn't "right" ?

</F>


From fredrik at pythonware.com  Thu Nov 23 09:09:06 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 09:09:06 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <4565144B.5060109@canterbury.ac.nz>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<4565144B.5060109@canterbury.ac.nz>
Message-ID: <ek3kv2$19m$3@sea.gmane.org>

Greg Ewing wrote:

> Just as a matter of interest, how far do you
> think these principles apply or don't apply
> to more recent things like iter.next() that
> don't follow this pattern?

this has been discussed before, I think. googling for __next__ might 
bring up some python-dev discussions.

</F>


From talin at acm.org  Thu Nov 23 09:35:02 2006
From: talin at acm.org (Talin)
Date: Thu, 23 Nov 2006 00:35:02 -0800
Subject: [Python-3000] defop ?
In-Reply-To: <ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>	<456456CF.8000005@gmail.com>	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
Message-ID: <45655D36.9060809@acm.org>

Guido van Rossum wrote:
> On 11/22/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
>> This whole thing seems a bit off from start to finish. A seperate
>> definition syntax with a special name/expression weirdo thingy, etc.
> 
> I have the same gut feelings but find it hard to explain why.
> 
> But I've learned to trust my gut -- eventually it will come to me.

Here's my understanding of the rationale for defop:

 From an implementation standpoint, everything that defop does can be 
emulated by an appropriate set of function decorators. However, when we 
start getting into things like namespaces and operators, some of those 
decorators start to look pretty ugly and have unwanted side-effects.

If all we wanted to do was to be able to create generic functions in the 
global namespace, the decorator syntax could be fairly simple:

    @overload
    def len( x : list ):
       ...

The decorator can inspect the function object to discover the name of 
the function - so there's no need to pass a separate argument to the 
decorator telling it what function to overload:

    @overload( len )
    def len( x : list ):
       ...

This syntax is less desirable since it violates DRY.

In either case, what the decorator returns is not the reference to the 
function object, but rather the reference to the generic dispatcher 
object, 'len'. The result of the decorator will then be bound to the 
name 'len' in that scope.

(It means that for every overload, the name 'len' gets rebound to the 
same dispatcher object over and over.)

Now, all that's fine and dandy as long as we limit ourselves to only 
creating global functions with names that conform to the syntax of 
Python identifiers.

But what if we don't want to clutter up the global namespace? Suppose we 
want to create a generic function in some other namespace, such as an 
attribute of an object. Now in order to define the overload, we have to 
do something like this:

    @overload( sequtils.len )
    def len( x : list ):
       ...

The problem here is that the function object is going to get bound to 
the name 'len' no matter what - the decorator can't control that. Even 
if we go ahead and add the function object as a method of 
'sequtils.len', we still, as a side-effect, end up binding the result of 
the decorator to 'len', which will at best create a spurious symbol, and 
at worse overwrite something we actually wanted to keep.

Taking this one step further, suppose we wanted to get rid of the 
__special__ names for overloaded operators, and instead be able to use 
the actual symbol for the operator. Suppose there was a built-in 
dictionary of operators, where the keys of this dictionary was the 
actual operators themselves. So 'infix['+']' would be the expression for 
the generic infix addition operator.

You could easily create a decorator that takes the operator name as an 
argumemt, similar to @overload above:

    @overload_infix('+')
    def add_bool_to_bool( a : bool, b : bool ):
       ...

But again, you have to deal with the unwanted function name. And you 
can't embed the '+' symbol in the function name itself, since '+' isn't 
allowed in Python identifiers.

(Maybe not the best of examples, but it illustrates the idea that 
perhaps not all generic functions will have simple names.)

Now, if we had anonymous functions (hint, hint :) ) we could say 
something along the lines of:

    infix( '+' ) += def( a : bool, b: bool ):
       ...

(Not the prettiest syntax I know...but at least it gets around the 
'tyranny of naming')

I suppose you could do that with lambda now, if your implementation was 
limited to a single expression with no statements (although that's a 
question - will lambda support decorated arguments?)

So my understanding is that 'defop' is a way to get around all of these 
issues. 'defop' is nothing more than 'def', except that it has a 
different rule as to how the function object is bound to a name.

(PJE, do I have this right?)

All that being said - I'm not sure that 'defop' is the right name for 
it, or that it's even the right way to solve it. But I hope that my 
explanation (if it's correct) may help clear up the discussion a bit.

-- Talin

From p.f.moore at gmail.com  Thu Nov 23 09:45:14 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 23 Nov 2006 08:45:14 +0000
Subject: [Python-3000] defop ?
In-Reply-To: <ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
Message-ID: <79990c6b0611230045w5c2c95e6l97e51cb7ae1c230@mail.gmail.com>

On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> On 11/22/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > This whole thing seems a bit off from start to finish. A seperate
> > definition syntax with a special name/expression weirdo thingy, etc.
>
> I have the same gut feelings but find it hard to explain why.
>
> But I've learned to trust my gut -- eventually it will come to me.

I agree entirely. The whole defop thing seems odd (and I *hate* the
name "defop"). I fail to see what it gains over a decorator syntax
such as is already available in the 3 generic function packages
available (RuleDispatch/PEAK.Rules, simplegeneric, and Guido's version
in the sandbox).

Without defop, the proposal seems to boil down to sanctioning a
standard library addition which provides a generic function package -
*possibly* with some modification of existing standard library and
builtin functions to become generics.

I'm sure I'm missing something, because the proposal feels alternately
overcomplex and trivial to me...

Paul.

From p.f.moore at gmail.com  Thu Nov 23 09:56:22 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 23 Nov 2006 08:56:22 +0000
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122185122.03e49500@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122145257.03be18b8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122160753.03cd51d0@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122185122.03e49500@sparrow.telecommunity.com>
Message-ID: <79990c6b0611230056i5fcf0ac3td46e8df7f2d73f12@mail.gmail.com>

On 11/23/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> So, the generic function 'as_string' here functions as an "interface" or
> "ability".  Of course, instead of being called 'as_string', it could live
> as an attribute of the 'str' type, e.g. 'str.cast':

OK, I think I just "clicked" with what defop is for. It's to allow you
to define additional behaviour for methods, etc, where the normal def
syntax won't allow something like str.cast? Sorry if that description
is a bit vague - it reflects a feeling of "why bother?"

Is

    defop str.cast(s: MyClass):
        ...

that much better than

    @str.cast.when(MyClass)
    def _(s):
        ...

given that Guido doesn't like "when", so a better name may end up
being used, and the slight namespace pollution of having to come up
with a name to use for the function (I used _ here, maybe there's a
case for allowing def without a name which passes the anonymous
function to the decorator but does no assignment to names...) I could
quite happily live with the decorator form as it stands though - new
syntax seems like overkill.

Paul

From gsakkis at rutgers.edu  Thu Nov 23 11:38:53 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Thu, 23 Nov 2006 05:38:53 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <45655142.7050800@acm.org>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ek2iqp$hsi$1@sea.gmane.org>
	<91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>
	<ek3i9t$qe8$1@sea.gmane.org> <45655142.7050800@acm.org>
Message-ID: <91ad5bf80611230238u50c770a1kaba0f2431318f941@mail.gmail.com>

On 11/23/06, Talin <talin at acm.org> wrote:

> Fredrik Lundh wrote:
> > George Sakkis wrote:
> >
> >> First off, I never implied someone's stupid just because we don't
> >> happen to agree on everything.
> >
> > brushing off a carefully thought out design and the process that led
> > up to it with "it's just like picking between two random colors" is a
> > pretty direct way of saying that you consider yourself so much smarter
> > than the people involved that you don't even care about what they're doing.
>
> That's not what he said though. If you go back and read his original
> posting, he was commenting on the quality of *most arguments* about HCI,
> not HCI itself, or all HCI-related discussions.
>
> While I wouldn't say it quite so strongly, I do think that there is a
> great deal of subjectivity in HCI discussions. One person's 'intuitive'
> may be another person's 'orange smoke', as we've seen in the current
> discussion.
>
> I'm not saying that such discussion don't have merit, but we must be
> careful not to take our own sense of 'obviousness' too seriously. And
> that's what I think George was trying to say.

Exactly. My point was not against HCI in general, or Python's in
specific; most probably it wouldn't be my primary language for the
last three years if it didn't "feel right" in so many ways. What I'm
saying is that I can't claim it *is* right because it *feels* right to
*me*.

And to come back to len(), many people value uniformity and
consistency in a language. While I am not an absolute purist, I don't
see the gain in practicality and/or elegance of having len() a
function instead of method to justify the special case (at least in
today's python; generic functions may change this). Guido's point
about the generality of the operation (keys() applies to mappings,
len() and iter() apply to containers) raises the question of where to
set the "generality threshold" between functions and methods. Is it
absolutely wrong to set it higher than the container level (e.g. at
the object level) or lower (e.g. at sequence or mapping) ? I don't
think it's either wrong or right; it's subjective. That's all I am
saying.

George

From gustavo at niemeyer.net  Thu Nov 23 13:14:17 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 10:14:17 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <45651355.7090605@canterbury.ac.nz>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
	<45651355.7090605@canterbury.ac.nz>
Message-ID: <20061123121416.GA6072@niemeyer.net>

> Sure, but then there's no need for a formal framwork
> for defining interfaces and adaptation -- by making
> use of duck typing, the joint user of the two libraries
> can wrap things as needed to make it all work.

You can always do everything in another way, just like
you don't need OO to be able to develop something.

> The proponents of adaptation seem to be making a
> stronger claim, however -- that somehow you can just
> register an adaptor and have all users of IFruits
> automatically understand Apples. That's the part that
> I fail to understand.

If you have an adapter from Apple to IFruit, any user
requiring an IFruit interface for an object will be
able to obtain it from Apple instances.  The "automatically"
part is not quite true, as adaptation must be explicitly
requested.

This is true for generic functions as well, except that
instead of using an IFruit interface, you have a fruit()
generic function which handles the adaptation.  Unfortunately,
this model doesn't work with more complex hierachies.

For instance:

  >>> from zope import interface, component
  >>> class Apple(object): pass
  >>> class IFruit(Interface): pass
  >>> class ITastyFruit(IFruit): pass
  >>> def to_tasty_fruit(obj): return "I'm actually a tasty fruit"
  >>> component.provideAdapter(to_tasty_fruit, (Apple,), ITastyFruit)
  >>> IFruit(Apple())
  "I'm actually a tasty fruit"

-- 
Gustavo Niemeyer
http://niemeyer.net

From gustavo at niemeyer.net  Thu Nov 23 13:45:48 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 10:45:48 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <4565168E.8000404@canterbury.ac.nz>
References: <5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<4565168E.8000404@canterbury.ac.nz>
Message-ID: <20061123124548.GB6072@niemeyer.net>

> > Consider 'iter()', for example, which can be viewed as adapting an
> > object to the "iteration interface" and returning an object
> > supporting iteration.
> 
> An architecture astronaut might view it that way, but
> I don't. To my way of thinking, iter(x) creates a new
> object that iterates over x. Calling it a form of
> adaptation just muddies things with uneccessary words.

FWIW, Phillip was pointing out the following similarity:

  >>> class IIterable(Interface): pass
  ... 
  >>> class IIter(Interface): pass
  ... 
  >>> class C(object):
  ...   interface.implements(IIterable)
  ...   def __iter__(self): return iter("foo")
  ... 
  >>> def adapt_to_iter(obj):
  ...     return obj.__iter__()
  ... 
  >>> component.provideAdapter(adapt_to_iter, (IIterable,), IIter)
  >>> 
  >>> iter(C()).next()
  'f'
  >>> IIter(C()).next()
  'f'

As mentioned before, the similarity doesn't hold for more
complex interface hierarchies.

-- 
Gustavo Niemeyer
http://niemeyer.net

From gustavo at niemeyer.net  Thu Nov 23 14:02:43 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 11:02:43 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061121174806.01f19c50@sparrow.telecommunity.com>
References: <91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<91ad5bf80611211418o76941e13l6864e97e2df23154@mail.gmail.com>
	<5.1.1.6.0.20061121174806.01f19c50@sparrow.telecommunity.com>
Message-ID: <20061123130243.GC6072@niemeyer.net>

> I was under the impression that Zope's actual use cases for
> instance-specific interfaces have mainly to do with *view* lookups and
> other types of lookups that don't really come into the current
> discussion scope as I understand it.  That is, I thought that instance
> interfaces were used primarily for "n-tuple adaptation" (the
> adaptation equivalent of multiple and/or predicate dispatch)?

While both, instance-specific interfaces and multiple adaptation,
are used in Zope 3 and considered very useful, their usefulness is
not dependent on each other.  The whole concept surely becomes
more interesting with both in place, of course.

(...)
> :)  I just want a minimal core that blesses what the language and
> stdlib *already* do (special methods and single-dispatch __mro__
> lookups), while still allowing "advanced" paradigms (Zope,
> RuleDispatch, etc.) to "play on a level playing field" with each
> other.

What the standard library is already doing doesn't need blessing,
and neither is blessing needed for allowing advanced paradigms
elsewhere, since they're already in place right now.

-- 
Gustavo Niemeyer
http://niemeyer.net

From python3000 at davious.org  Thu Nov 23 13:14:59 2006
From: python3000 at davious.org (Dave Anderson)
Date: Thu, 23 Nov 2006 07:14:59 -0500
Subject: [Python-3000] Abilities / Interfaces
Message-ID: <456590C3.4040801@davious.org>

Proposal Summary
================

Add 2 new keywords: implements, implement
(alternatively, implements and implement can be implemented as object 
level functions)
Add new built-in function: interface
Add new object level function: does_implement


Interfaces
==========

There is a fight between using classes/types for dispatch verses class 
methods for dispatch.  There is a fight between declaring class 
compatibility and declaring method compatibility.

We see this explicit declaration in Phillip's
     defop iter(self):

versus method checking

We should unify these concepts by allowing them to mix in an interface 
definition:

Interfaces, here, are a set of class types, class methods/attributes, 
and strings

* Class/Type: Can be built-in or user-defined
   when Class2 declares it implements Class1, we will presume
   the class will support anything Class1 will support, but will only 
crash on a call to a method Class1 has, but Class2 does not
* Class Method/Attribute: Can be from a built-in object or be 
user-defined class
   when Class2 declares it implements a Class1.amethod, we will
   presume the class as a method amethod, but will only crash on a call 
to Class2.amethod if Class2.amethod is not defined.
* String: Indicates a method or attribute with that name, introspection 
is used to see if a Class implements this

iter_interface = interface(iter)  # class-based interface
file_open_close_inter = interface(file.open, file.close) # method interface
# here is an interface looking for both a class and some methods to be 
'implemented'
iter_file_open_close_inter = interface(iter, file.open, file.close)

Implementation of Interfaces (Implements and Implement)
=======================================================

In the common case, a class already implements another class behavior 
and we just need to say that.
Other times, we want to adapt or use a special method in a certain context.

Implement and Implement:
* Implements: Declares pre-existing computability
   It is a list of classes/types and class methods/attributes
   when Class2 declares it implements Class1, we will presume
   the class will support anything Class1 will support, but will
   only crash on a call to a method Class1 has, but Class2 does not
* Implement: Indicates special method-help via adaptation or a different 
method.
   The keyword is used to define a function that acts as a class/type 
adapter or special function.
   when Class2 declares it implements a Class1.amethod, we will
   presume the class as a method amethod, but will only crash
   on a call to Class2.amethod if Class2.amethod is not defined.

Class's declared to be implemented are presumed to have the full method 
set of the class, those reducing the need for extensive method checking 
at the adult-allowable risk of someone not completely implementing the 
classes methods.  (Of course, we now have implementation declaration 
inspection of a sort.)

Dispatching by Interface and Declared Implementation
=====================================================

the example Bar class (following these dispatching examples) should be 
able to be dispatched by
* methods looking for a iter, Foo, SomeOtherClass, YetAnotherClass instances
   example: def funct(iter_var: iter):

* methods looking for a method of those classes
   example: def funct(var_that_implements_foo_or_declares_a_foo_method: 
Foo.method):

* methods looking for methods file.open, file.close, AnotherClass.close
   example: def funct(var_with_a_method_like_file_open: file.open):

* methods looking for just method of with a particular name
   example: def funct(var_with_a_method_called_open: "open"):

the dispatch need not be based on a singular type or method
  (see below for those multi-condition interface examples)

also, these dispatch functions can be class functions
  example def funct(self, var_with_a_file_open_meth: file.open):


Example of Using Implements and Implement
=========================================

class Bar(Foo):
"""based on the declarations that follow class is taken as iter, Foo, 
and SomeOtherClass without method introspection (return of self for 
these is implicit) methods open and close are understood to be file like 
methods. An adaption functionality is available in the declaration 
implement AnotherClass and special method as a particular class is 
available in the declaration implement
YetAnotherClass.method."""
     # new keyword implements
     implements iter, SomeOtherClass, file.open, file.close

     # new keyword implement: for class adaptation
     implement AnotherClass(self):
         return transform_to_yac(self)

     implement YetAnotherClass.open(self, var):
	return different_open_function(self, var)

     # presumed for file.open
     def open(self):

     # presumed for file.close
     def close(self):

     [implementing iter functions not shown]


Interface objects, including multiple object apis
===================================================
* use of new standard interface function
* allows multiple interface apis to be combined

foo_interface = interface(Foo)
iter_interface = interface(iter)
file_open_close_interface = interface(file.open, file.close)
any_open_close_interface = interface("open", "close")
iter_file_open_close_inter = interface(iter, file_open_close_interface)

def funct(foo_var: Foo):
.. equivalent to ..
def funct(foo_var: foo_interface):

def funct(iter_var: iter):
.. equivalent to ..
def funct(iter_var: iter_interface):

def funct(file_like_var: file.open, file.close):
.. equivalent to ..
def funct(file_like_var: file_open_close_interface):

def funct(method_abilities_var: "open", "close"):
.. equivalent to ..
def funct(method_abilities_var: just_open_close_methods_interface):

def funct(method_abilities_var: iter, file.open, file.close):
.. equivalent to ..
def funct(method_abilities_var: iter_file_open_close_inter):


does_implement
==============

* does_implement: a boolean function that at the object level that
   returns true if the object has declared that it implements an 
interface object

if obj.does_implement(iter):

if obj.does_implement(Class1):

if obj.does_implement(interface_obj):

Implementing Interfaces without the keywords Implements, Implement
==================================================================
Class2.implements(iter)

Class2.implements(Class1)

Class2.implements(interface_obj)

Class2.implements(file.open)

Class2.implements(iter, Class1, interface_obj, file.open)

Class2.implement(Class1, transform_function_def)

Class2.implement(Class1.method, function_def)

Dispatching Issues
==================

There will have to be some way to decide which declarations are chosen
ahead of others, but I haven't thought about that too much.. we'd
probably look at the order of the objects in the implements statement.

The last issue I have in my head is Guido's issue
 > This doesn't do anything for other generic functions that might also
 > conceivably work with C. Phillip addresses that by proposing help
 > functions that do a bunch of these bindings at once, and by noticing
 > that if you just use the standard library and generic functions, the
 > default implementation might just work.

But, it think those bindings could be generated with some *implements*
declarations inside of standard objects like Iterable - Collection -
Sequence - MutableSequence - list to each other.  These declarations 
will be inherited by those object which declare they implement them.

With this syntax, dispatch functions would seamlessly integrate with the 
current python language, the absence of any interface in a function def 
would be the python language as it works now, by defaulting to the 
lowest and most-likely only "dispatching" function.

From python3000 at davious.org  Thu Nov 23 13:46:34 2006
From: python3000 at davious.org (Dave Anderson)
Date: Thu, 23 Nov 2006 07:46:34 -0500
Subject: [Python-3000] Abilities / Interfaces
Message-ID: <4565982A.9030803@davious.org>

Proposal Summary
================

Add 2 new keywords: implements, implement
(alternatively, implements and implement can be implemented as object 
level functions)
Add new built-in function: interface
Add new object level function: does_implement

(with minor corrections in Implementation of Interfaces (Implements and 
Implement) section)

Interfaces
==========

There is a fight between using classes/types for dispatch verses class 
methods for dispatch.  There is a fight between declaring class 
compatibility and declaring method compatibility.

We see this explicit declaration in Phillip's
     defop iter(self):

versus method checking

We should unify these concepts by allowing them to mix in an interface 
definition:

Interfaces, here, are a set of class types, class methods/attributes, 
and strings

* Class/Type: Can be built-in or user-defined
   when Class2 declares it implements Class1, we will presume
   the class will support anything Class1 will support, but will only 
crash on a call to a method Class1 has, but Class2 does not
* Class Method/Attribute: Can be from a built-in object or be 
user-defined class
   when Class2 declares it implements a Class1.amethod, we will
   presume the class as a method amethod, but will only crash on a call 
to Class2.amethod if Class2.amethod is not defined.
* String: Indicates a method or attribute with that name, introspection 
is used to see if a Class implements this

iter_interface = interface(iter)  # class-based interface
file_open_close_inter = interface(file.open, file.close) # method interface
# here is an interface looking for both a class and some methods to be 
'implemented'
iter_file_open_close_inter = interface(iter, file.open, file.close)

Implementation of Interfaces (Implements and Implement)
=======================================================

In the common case, a class already implements another class behavior 
and we just need to say that.
Other times, we want to adapt or use a special method in a certain context.

Implement and Implement:
* Implements: Declares pre-existing compatability
   It is a list of classes/types and class methods/attributes

   when Class2 declares it implements Class1, we will presume
   the class will support anything Class1 will support, but will
   only crash on a call to a method Class1 has, but Class2 does not

   when Class2 declares it implements a Class1.amethod, we will
   presume the class as a method amethod, but will only crash
   on a call to Class2.amethod if Class2.amethod is not defined.

* Implement: Indicates special method-help via adaptation or a different 
method.
   The keyword is used to define a function that acts as a class/type 
adapter or special function.

   When Class2 uses an implement to associate a class/type or
   method/attribute with a function definition, that function should
   called to which effectively adapts the Class2 obj when the context is
   'implement Class' or runs a specially defined method when the context
    is 'implement method'.

Class's declared to be implemented are presumed to have the full method 
set of the class, those reducing the need for extensive method checking 
at the adult-allowable risk of someone not completely implementing the 
classes methods.  (Of course, we now have implementation declaration 
inspection of a sort.)

Dispatching by Interface and Declared Implementation
=====================================================

the example Bar class (following these dispatching examples) should be 
able to be dispatched by
* methods looking for a iter, Foo, SomeOtherClass, YetAnotherClass instances
   example: def funct(iter_var: iter):

* methods looking for a method of those classes
   example: def funct(var_that_implements_foo_or_declares_a_foo_method: 
Foo.method):

* methods looking for methods file.open, file.close, AnotherClass.close
   example: def funct(var_with_a_method_like_file_open: file.open):

* methods looking for just method of with a particular name
   example: def funct(var_with_a_method_called_open: "open"):

the dispatch need not be based on a singular type or method
  (see below for those multi-condition interface examples)

also, these dispatch functions can be class functions
  example def funct(self, var_with_a_file_open_meth: file.open):


Example of Using Implements and Implement
=========================================

class Bar(Foo):
"""based on the declarations that follow class is taken as iter, Foo, 
and SomeOtherClass without method introspection (return of self for 
these is implicit) methods open and close are understood to be file like 
methods. An adaption functionality is available in the declaration 
implement AnotherClass and special method as a particular class is 
available in the declaration implement
YetAnotherClass.method."""
     # new keyword implements
     implements iter, SomeOtherClass, file.open, file.close

     # new keyword implement: for class adaptation
     implement AnotherClass(self):
         return transform_to_yac(self)

     implement YetAnotherClass.open(self, var):
		return different_open_function(self, var)

     # presumed for file.open
     def open(self):

     # presumed for file.close
     def close(self):

     [implementing iter functions not shown]


Interface objects, including multiple object apis
===================================================
* use of new standard interface function
* allows multiple interface apis to be combined

foo_interface = interface(Foo)
iter_interface = interface(iter)
file_open_close_interface = interface(file.open, file.close)
any_open_close_interface = interface("open", "close")
iter_file_open_close_inter = interface(iter, file_open_close_interface)

def funct(foo_var: Foo):
.. equivalent to ..
def funct(foo_var: foo_interface):

def funct(iter_var: iter):
.. equivalent to ..
def funct(iter_var: iter_interface):

def funct(file_like_var: file.open, file.close):
.. equivalent to ..
def funct(file_like_var: file_open_close_interface):

def funct(method_abilities_var: "open", "close"):
.. equivalent to ..
def funct(method_abilities_var: just_open_close_methods_interface):

def funct(method_abilities_var: iter, file.open, file.close):
.. equivalent to ..
def funct(method_abilities_var: iter_file_open_close_inter):


does_implement
==============

* does_implement: a boolean function that at the object level that
   returns true if the object has declared that it implements an 
interface object

if obj.does_implement(iter):

if obj.does_implement(Class1):

if obj.does_implement(interface_obj):

Implementing Interfaces without the keywords Implements, Implement
==================================================================
Class2.implements(iter)

Class2.implements(Class1)

Class2.implements(interface_obj)

Class2.implements(file.open)

Class2.implements(iter, Class1, interface_obj, file.open)

Class2.implement(Class1, transform_function_def)

Class2.implement(Class1.method, function_def)

Dispatching Issues
==================

There will have to be some way to decide which declarations are chosen
ahead of others, but I haven't thought about that too much.. we'd
probably look at the order of the objects in the implements statement.

The last issue I have in my head is Guido's issue
 > This doesn't do anything for other generic functions that might also
 > conceivably work with C. Phillip addresses that by proposing help
 > functions that do a bunch of these bindings at once, and by noticing
 > that if you just use the standard library and generic functions, the
 > default implementation might just work.

But, it think those bindings could be generated with some *implements*
declarations inside of standard objects like Iterable - Collection -
Sequence - MutableSequence - list to each other.  These declarations 
will be inherited by those object which declare they implement them.

With this syntax, dispatch functions would seamlessly integrate with the 
current python language, the absence of any interface in a function def 
would be the python language as it works now, by defaulting to the 
lowest and most-likely only "dispatching" function.

From gustavo at niemeyer.net  Thu Nov 23 14:20:22 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 11:20:22 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221632w1effe31cxe3107b45d90c5ab9@mail.gmail.com>
References: <001801c70e63$d172eeb0$6402a8c0@arkdesktop>
	<3218934778639261534@unknownmsgid>
	<ca471dc20611221049r3a11944fq4f20d8c3b86d1d11@mail.gmail.com>
	<-3135130819962384877@unknownmsgid>
	<ca471dc20611221538lfb33072wfd61abbbdac59195@mail.gmail.com>
	<6961093367361956666@unknownmsgid>
	<ca471dc20611221632w1effe31cxe3107b45d90c5ab9@mail.gmail.com>
Message-ID: <20061123132022.GD6072@niemeyer.net>

> IMO the difference between ABCs and interfaces/abilities is that
> inserting a new ABC into an existing class hierarchy without modifying
> its source code is considered a hack, and must be done by a helper
> routine that *actually* mucks with __bases__; while (Zope-style)
> interfaces use an external registry or a separate class attribute
> (__implements__) that is intended to be mucked with.

Right.  That and also being able to assign interfaces to individual
instances, being able to split and merge interfaces in a level that
doesn't need changing the implementation itself, and so on.

-- 
Gustavo Niemeyer
http://niemeyer.net

From fredrik at pythonware.com  Thu Nov 23 14:53:33 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 14:53:33 +0100
Subject: [Python-3000] print() parameters in py3k
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>	<ejqiqb$d38$1@sea.gmane.org>	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com><aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com><ejumhp$dm9$1@sea.gmane.org>
	<55DAAEFE-373C-4799-9C8E-5A48FABD63C1@python.org>
Message-ID: <ek494t$bbq$1@sea.gmane.org>

Barry Warsaw wrote:

>> 2) for convenience, extend print to treat a format object as a format
>> specifier for the following arguments:
>>
>>      print (x, y, z, fmt="My X: %s, Your Y: %s, His Z: %s")
>>
>> becomes
>>
>>      print(format("My X: %s, Your Y: %s, His Z: %s"), x, y ,z)
>>
>> 3) get rid of printf.
>
> That's not bad.  Is the format object allowed in any positional
> argument, or just the first?

any, I think.  you can either just let print collect all none-format objects and
pass it to the preceeding format object, or (better) ask the format object for
how many arguments it expects, and use a default formatter (i.e. str) on the
rest.

</F> 




From fredrik at pythonware.com  Thu Nov 23 15:10:27 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 15:10:27 +0100
Subject: [Python-3000] print() parameters in py3k
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com><ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com><ejqiqb$d38$1@sea.gmane.org><ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com><fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com><aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com><ejumhp$dm9$1@sea.gmane.org>
	<ca471dc20611210818x4683862dm9e95ca61a2868dbc@mail.gmail.com>
Message-ID: <ek4a4k$eun$1@sea.gmane.org>

Guido van Rossum wrote:

> Hm. While not as obviously from a different universe as Barry's
> proposal, this still pretty weird, probably at least from a different
> planet (much farther than Iceland anyway :-)

it's inspired by

    http://online.effbot.org/2006_11_01_archive.htm#et-builder

which owes a lot to

    http://www.tothink.com/python/ElementBuilder/

which is inspired by

    http://effbot.org/zone/idea-xml-literal.htm

which you found "interesting in a futuristic kind of way" at the time ;-),
and which is probably influenced by

    http://okmij.org/ftp/Scheme/SXML.html

whether or not I was aware of that at the time, and google gives me
around 150,000 hits for "John McCarthy ufo", so I guess that settles
it.

> Treating the first argument differently based on its being of a specific
> type doesn't sound right to me; what if you are handed an object x
> and you decide to print it using print(x), but surreptitiously (or by
> mistake) they hand you a format object?

it'll fail in some way (exactly how remains to be defined), but at least it'll fail
more consistently and reliably than if you accidentally do

    printf(value)

on a string value (see Adam's post at the top of this subthread).  a failure caused
by the wrong type is easier to handle than a failure caused by using the right type,
but the wrong value.

(functions that needs to print arbitrary objects in a safe fashion cannot really use
a bare "print" anyway, of course.)

</F> 




From fredrik at pythonware.com  Thu Nov 23 15:13:43 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 23 Nov 2006 15:13:43 +0100
Subject: [Python-3000] print() parameters in py3k
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com><ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com><ejqiqb$d38$1@sea.gmane.org><ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com><fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com><aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com><ejumhp$dm9$1@sea.gmane.org>
	<91ad5bf80611210710m36a56959m552aea57a0c88354@mail.gmail.com>
Message-ID: <ek4aan$fiv$1@sea.gmane.org>

George Sakkis wrote:

> a. Will print have to do an isinstance(args[0], format) to decide what
> to do ? If so, don't the usual arguments for duck typing and against
> type checking apply here ?

isinstance (or an equivalent mechanism) would be more robust, in this
specific case.

> b. print(a,b,c) is no longer obvious whether it prints 3 unformatted
> objects or 2 formatted, if you don't know what a is.

nope, but on the other hand, you don't know what it prints today either
(see my other post).

> b. What if for some reason one actually wants to print the format
> object instead of interpreting as a format specifier ?

repr() is your friend.

</F> 




From aahz at pythoncraft.com  Thu Nov 23 16:52:32 2006
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 23 Nov 2006 07:52:32 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <45655142.7050800@acm.org>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ek2iqp$hsi$1@sea.gmane.org>
	<91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>
	<ek3i9t$qe8$1@sea.gmane.org> <45655142.7050800@acm.org>
Message-ID: <20061123155232.GA13376@panix.com>

On Wed, Nov 22, 2006, Talin wrote:
>
> While I wouldn't say it quite so strongly, I do think that there is a 
> great deal of subjectivity in HCI discussions. One person's 'intuitive' 
> may be another person's 'orange smoke', as we've seen in the current 
> discussion.

But the whole point of making Guido BDFL was to codify his intuition WRT
language decisions.  There's a reason why we here are using Python
instead of other languages, and as Guido himself has noted, often he
knows what the right decisions is without being able to explain it.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"In many ways, it's a dull language, borrowing solid old concepts from
many other languages & styles:  boring syntax, unsurprising semantics,
few automatic coercions, etc etc.  But that's one of the things I like
about it."  --Tim Peters on Python, 16 Sep 1993

From gsakkis at rutgers.edu  Thu Nov 23 17:26:36 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Thu, 23 Nov 2006 11:26:36 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <20061123155232.GA13376@panix.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ek2iqp$hsi$1@sea.gmane.org>
	<91ad5bf80611221908t4e1ed686yb94d700d1bbc0cb5@mail.gmail.com>
	<ek3i9t$qe8$1@sea.gmane.org> <45655142.7050800@acm.org>
	<20061123155232.GA13376@panix.com>
Message-ID: <91ad5bf80611230826g37a29e0fx46414562bb537fe1@mail.gmail.com>

On 11/23/06, Aahz <aahz at pythoncraft.com> wrote:

> On Wed, Nov 22, 2006, Talin wrote:
> >
> > While I wouldn't say it quite so strongly, I do think that there is a
> > great deal of subjectivity in HCI discussions. One person's 'intuitive'
> > may be another person's 'orange smoke', as we've seen in the current
> > discussion.
>
> But the whole point of making Guido BDFL was to codify his intuition WRT
> language decisions.  There's a reason why we here are using Python
> instead of other languages, and as Guido himself has noted, often he
> knows what the right decisions is without being able to explain it.

No, the reason we're here is that we share the same notion of
rightness in most, or at least the most important, things. It's not an
all-or-nothing choice though. Making general claims about rightness is
some absolute way (and indirectly implying that Rubyists, Lispers and
all other folks are wrong) should better remain a priviledge of
religion, not language design.

George

From guido at python.org  Thu Nov 23 17:38:46 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 08:38:46 -0800
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
Message-ID: <ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>

On 11/22/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > Not sure I like this better. My spelling of "operator::XXX" is
> > "__XXX__" and I like that just fine -- no new syntax needed.
>
> Well, that might be a spelling of `operator::XXX` but what about other
> use cases like `third_party_interface_system::adapt` or
> `anything_at_all::XXX`? Thats what I mean with not being too limiting
> and solving the problem in a way that opens up solutions to other
> problems. I get the opposition to it, but it seems reasonable,
> nonetheless.

How was I to generalize from a single example what you meant? I
thought you were using 'operator' as a fixed keyword like it is in
C++, and I was wondering what you meant by the double colon.

I still believe it is easy enough to solve this using a naming convention.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Nov 23 17:50:08 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 08:50:08 -0800
Subject: [Python-3000] print() parameters in py3k
In-Reply-To: <ek4a4k$eun$1@sea.gmane.org>
References: <8f01efd00611191013k5fa1118br2daa99d8817d975d@mail.gmail.com>
	<ca471dc20611191333u3b28a07dh7849fed189c0b515@mail.gmail.com>
	<ejqiqb$d38$1@sea.gmane.org>
	<ca471dc20611191355s20fee30agd3e4070175c1f30@mail.gmail.com>
	<fb6fbf560611201105p3f1d399fic77ff9add2b7c18f@mail.gmail.com>
	<aac2c7cb0611210150y5275dc5dv844274789f3d1b42@mail.gmail.com>
	<ejumhp$dm9$1@sea.gmane.org>
	<ca471dc20611210818x4683862dm9e95ca61a2868dbc@mail.gmail.com>
	<ek4a4k$eun$1@sea.gmane.org>
Message-ID: <ca471dc20611230850u1112bbe6kafb7af4bd2f61c59@mail.gmail.com>

Let me just say -1 on this idea so we can close the thread. None of
that convinces me. I don't think that print(a, b) failing when b has a
certain unexpected type can be considered consistent. The
ElementBuilder example doesn't apply because  it takes very limited
argument types (string, dict, E; everything else raises an exception).
print is carefully designed to be able to print *anything*. Not being
able to print a format object strikes me as wrong. Apart from that,
I'm not sure that having formats be a separate type is a good idea
either; but even if it were a good idea, the rest of the reason for
the -1 still stands.

--Guido

On 11/23/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> > Hm. While not as obviously from a different universe as Barry's
> > proposal, this still pretty weird, probably at least from a different
> > planet (much farther than Iceland anyway :-)
>
> it's inspired by
>
>     http://online.effbot.org/2006_11_01_archive.htm#et-builder
>
> which owes a lot to
>
>     http://www.tothink.com/python/ElementBuilder/
>
> which is inspired by
>
>     http://effbot.org/zone/idea-xml-literal.htm
>
> which you found "interesting in a futuristic kind of way" at the time ;-),
> and which is probably influenced by
>
>     http://okmij.org/ftp/Scheme/SXML.html
>
> whether or not I was aware of that at the time, and google gives me
> around 150,000 hits for "John McCarthy ufo", so I guess that settles
> it.
>
> > Treating the first argument differently based on its being of a specific
> > type doesn't sound right to me; what if you are handed an object x
> > and you decide to print it using print(x), but surreptitiously (or by
> > mistake) they hand you a format object?
>
> it'll fail in some way (exactly how remains to be defined), but at least it'll fail
> more consistently and reliably than if you accidentally do
>
>     printf(value)
>
> on a string value (see Adam's post at the top of this subthread).  a failure caused
> by the wrong type is easier to handle than a failure caused by using the right type,
> but the wrong value.
>
> (functions that needs to print arbitrary objects in a safe fashion cannot really use
> a bare "print" anyway, of course.)
>
> </F>
>
>
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Nov 23 18:00:54 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 09:00:54 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <456590C3.4040801@davious.org>
References: <456590C3.4040801@davious.org>
Message-ID: <ca471dc20611230900x54653301q86a101bf1fd1aa88@mail.gmail.com>

Smells a bit like the rejected PEP 245 doesn't it? You should at least
explain how it differs in a summary or rationale.

Anyway, it's Thanksgiving day here, and I'm unlikely to be able to
spend much time on this collection of threads until Monday or so...
And even then I'll have other work to do -- it was a rare opportunity
that I could spend pretty much the entire day reading, thinking and
writing about one topic (if perhaps a little too quick of the
trigger). If you want to save me time, a personal email from Phillip,
Fredrik, Bill or Andrew with a pointer to a proto-PEP that summarizes
a spec that might emerge from the discussion would carry a lot of
weight in my inbox (even if there were 4 competing proto-PEPs :-).
(I'm not suggesting they are the only ones who can write the PEP, but
without an endorsement from a heavyweight I would treat it as just
another post.)

--Guido

On 11/23/06, Dave Anderson <python3000 at davious.org> wrote:
> Proposal Summary
> ================
>
> Add 2 new keywords: implements, implement
> (alternatively, implements and implement can be implemented as object
> level functions)
> Add new built-in function: interface
> Add new object level function: does_implement
>
>
> Interfaces
> ==========
>
> There is a fight between using classes/types for dispatch verses class
> methods for dispatch.  There is a fight between declaring class
> compatibility and declaring method compatibility.
>
> We see this explicit declaration in Phillip's
>      defop iter(self):
>
> versus method checking
>
> We should unify these concepts by allowing them to mix in an interface
> definition:
>
> Interfaces, here, are a set of class types, class methods/attributes,
> and strings
>
> * Class/Type: Can be built-in or user-defined
>    when Class2 declares it implements Class1, we will presume
>    the class will support anything Class1 will support, but will only
> crash on a call to a method Class1 has, but Class2 does not
> * Class Method/Attribute: Can be from a built-in object or be
> user-defined class
>    when Class2 declares it implements a Class1.amethod, we will
>    presume the class as a method amethod, but will only crash on a call
> to Class2.amethod if Class2.amethod is not defined.
> * String: Indicates a method or attribute with that name, introspection
> is used to see if a Class implements this
>
> iter_interface = interface(iter)  # class-based interface
> file_open_close_inter = interface(file.open, file.close) # method interface
> # here is an interface looking for both a class and some methods to be
> 'implemented'
> iter_file_open_close_inter = interface(iter, file.open, file.close)
>
> Implementation of Interfaces (Implements and Implement)
> =======================================================
>
> In the common case, a class already implements another class behavior
> and we just need to say that.
> Other times, we want to adapt or use a special method in a certain context.
>
> Implement and Implement:
> * Implements: Declares pre-existing computability
>    It is a list of classes/types and class methods/attributes
>    when Class2 declares it implements Class1, we will presume
>    the class will support anything Class1 will support, but will
>    only crash on a call to a method Class1 has, but Class2 does not
> * Implement: Indicates special method-help via adaptation or a different
> method.
>    The keyword is used to define a function that acts as a class/type
> adapter or special function.
>    when Class2 declares it implements a Class1.amethod, we will
>    presume the class as a method amethod, but will only crash
>    on a call to Class2.amethod if Class2.amethod is not defined.
>
> Class's declared to be implemented are presumed to have the full method
> set of the class, those reducing the need for extensive method checking
> at the adult-allowable risk of someone not completely implementing the
> classes methods.  (Of course, we now have implementation declaration
> inspection of a sort.)
>
> Dispatching by Interface and Declared Implementation
> =====================================================
>
> the example Bar class (following these dispatching examples) should be
> able to be dispatched by
> * methods looking for a iter, Foo, SomeOtherClass, YetAnotherClass instances
>    example: def funct(iter_var: iter):
>
> * methods looking for a method of those classes
>    example: def funct(var_that_implements_foo_or_declares_a_foo_method:
> Foo.method):
>
> * methods looking for methods file.open, file.close, AnotherClass.close
>    example: def funct(var_with_a_method_like_file_open: file.open):
>
> * methods looking for just method of with a particular name
>    example: def funct(var_with_a_method_called_open: "open"):
>
> the dispatch need not be based on a singular type or method
>   (see below for those multi-condition interface examples)
>
> also, these dispatch functions can be class functions
>   example def funct(self, var_with_a_file_open_meth: file.open):
>
>
> Example of Using Implements and Implement
> =========================================
>
> class Bar(Foo):
> """based on the declarations that follow class is taken as iter, Foo,
> and SomeOtherClass without method introspection (return of self for
> these is implicit) methods open and close are understood to be file like
> methods. An adaption functionality is available in the declaration
> implement AnotherClass and special method as a particular class is
> available in the declaration implement
> YetAnotherClass.method."""
>      # new keyword implements
>      implements iter, SomeOtherClass, file.open, file.close
>
>      # new keyword implement: for class adaptation
>      implement AnotherClass(self):
>          return transform_to_yac(self)
>
>      implement YetAnotherClass.open(self, var):
>         return different_open_function(self, var)
>
>      # presumed for file.open
>      def open(self):
>
>      # presumed for file.close
>      def close(self):
>
>      [implementing iter functions not shown]
>
>
> Interface objects, including multiple object apis
> ===================================================
> * use of new standard interface function
> * allows multiple interface apis to be combined
>
> foo_interface = interface(Foo)
> iter_interface = interface(iter)
> file_open_close_interface = interface(file.open, file.close)
> any_open_close_interface = interface("open", "close")
> iter_file_open_close_inter = interface(iter, file_open_close_interface)
>
> def funct(foo_var: Foo):
> .. equivalent to ..
> def funct(foo_var: foo_interface):
>
> def funct(iter_var: iter):
> .. equivalent to ..
> def funct(iter_var: iter_interface):
>
> def funct(file_like_var: file.open, file.close):
> .. equivalent to ..
> def funct(file_like_var: file_open_close_interface):
>
> def funct(method_abilities_var: "open", "close"):
> .. equivalent to ..
> def funct(method_abilities_var: just_open_close_methods_interface):
>
> def funct(method_abilities_var: iter, file.open, file.close):
> .. equivalent to ..
> def funct(method_abilities_var: iter_file_open_close_inter):
>
>
> does_implement
> ==============
>
> * does_implement: a boolean function that at the object level that
>    returns true if the object has declared that it implements an
> interface object
>
> if obj.does_implement(iter):
>
> if obj.does_implement(Class1):
>
> if obj.does_implement(interface_obj):
>
> Implementing Interfaces without the keywords Implements, Implement
> ==================================================================
> Class2.implements(iter)
>
> Class2.implements(Class1)
>
> Class2.implements(interface_obj)
>
> Class2.implements(file.open)
>
> Class2.implements(iter, Class1, interface_obj, file.open)
>
> Class2.implement(Class1, transform_function_def)
>
> Class2.implement(Class1.method, function_def)
>
> Dispatching Issues
> ==================
>
> There will have to be some way to decide which declarations are chosen
> ahead of others, but I haven't thought about that too much.. we'd
> probably look at the order of the objects in the implements statement.
>
> The last issue I have in my head is Guido's issue
>  > This doesn't do anything for other generic functions that might also
>  > conceivably work with C. Phillip addresses that by proposing help
>  > functions that do a bunch of these bindings at once, and by noticing
>  > that if you just use the standard library and generic functions, the
>  > default implementation might just work.
>
> But, it think those bindings could be generated with some *implements*
> declarations inside of standard objects like Iterable - Collection -
> Sequence - MutableSequence - list to each other.  These declarations
> will be inherited by those object which declare they implement them.
>
> With this syntax, dispatch functions would seamlessly integrate with the
> current python language, the absence of any interface in a function def
> would be the python language as it works now, by defaulting to the
> lowest and most-likely only "dispatching" function.
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Nov 23 18:37:07 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 09:37:07 -0800
Subject: [Python-3000] Generic functions vs. OO
Message-ID: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>

I'm reposting this under a different subject because the other subject
seems to have gone off on a tangent. The rest are Phillip's words.

--Guido

---------- Forwarded message ----------
From: Phillip J. Eby <pje at telecommunity.com>
Date: Nov 22, 2006 10:01 PM
Subject: Re: [Python-3000] Special methods and interface-based type system
To: Guido van Rossum <guido at python.org>
Cc: Python 3000 <python-3000 at python.org>


At 08:29 PM 11/22/2006 -0800, Guido van Rossum wrote:
>One thing that rubs me the wrong way about generic functions is that
>it appears to go against OO. Now I'm not someone to take OO as
>religion, but there's something uncomfortable (for me) about how, in
>Phillip's world, many things become functions instead of methods,
>which brings along concerns about the global namespace filling up, and
>also about functionality being spread randomly across too many
>modules. I fear I will miss the class as a convenient focus for
>related functionality.

I originally proposed a solution for this back in January '05, but it was
too premature.  But since you have now stated the problem that the proposal
was intended to solve, perhaps the solution has a chance now.  :)

I will try to be as concrete as possible.  Let's start with an actual,
hopefully non-exploding 'Interface' implementation, based on an assumption
that we have generic functions available:

     class InterfaceClass(type):
         def __init__(cls, name, bases, cdict):
             for k,v in cdict.items():
                 # XXX this should probably skip at least __slots__,
                 #     __metaclass__, and __module__, but oh well
                 cdict[k] = AdaptingDescriptor(v)

     class Interface:
         __metaclass__ = InterfaceClass
         __slots__ = '__self__'

         def __init__(self, subject):
             # this isinstance() check should be replaced by an
             # 'unwrap()' generic function, so other adapter types
             # will work, but this is just an example, so...
             if isinstance(subject, Interface):
                 subject = subject.__self__
             self.__self__ = subject

     class AdaptingDescriptor:
         def __init__(self, descriptor):
             self.wrapped = descriptor
         def __get__(self, ob, typ=None):
             if ob is None:
                 return self
             return self.wrapped.__get__(ob.__self__, typ)

Now, using this new "interface framework", let's implement a small
"mapping" typeclas... er, interface.

     class Mapping(Interface):
         def keys(self):
             return [k for k,v in self.items()]
         def items(self):
             return [k,self[k] for k in self.keys()]
         # ... other self-recursive definitions

What does this do?  Well, we can now call Mapping(foo) to turn an arbitrary
object into something that has Mapping's generic functions as its methods,
and invokes them on foo!  (I am assuming here that normal functions are
implicitly overloadable, even if that means they change type at runtime to
do so.)  We could even use interfaces for argument type declarations, to
automatically put things in the "right namespace" for what the code expects
to use.  That is, if you declare an argument to be a Mapping, then that's
what you get.  If you call .keys() on the resulting adapted object and the
type doesn't support the operation, you get an error.

Too late a form of error checking you say?  Well, make a more sophisticated
factory mechanism in Interface.__new__ that actually creates (and caches)
different adapter types based on the type of object being adapted, so that
hasattr() tests will work on the wrapped type, or so that you can get an
early error if none of the wrapped generic functions has a method defined
for the target type.

A few important points here:

1. A basic interface mechanism is extemely simple to implement, given
generic functions

2. It is highly customizable with respect to error checking and other
features, even on a per-user basis, because there doesn't have to be only
one "true" Interface type to rule them all (or one true generic function
type either, but that's a separate discussion).

3. It allows interfaces to include partial implementations, ala Ping and
Alex's past proposals, thus allowing you to implement partial mapping or
"file" objects and have the rest of the interface's implementation filled
in for you

4. It allows you to hide the very existence of the notion of a "generic
function", if you prefer not to think about such things

5. It even supports interface inheritance and interface algebra:
subclassing an interface allows adding new operations, and simple
assignment suffices to compose new interfaces, e.g.:

     class MappingItems(Interface):
         items = Mapping.items

Notice that nothing special is required, this "just works" as a natural
consequence of the rest of the implementation shown.

Okay, so now you want to know how to *implement* a "Mapping".  Well,
simplest but most tedious, you can just register operations directly, e.g.:

      class MyMapping:
          def __init__(self, data):
              self.data = dict(data)
          defop operator.getitem(self, key):
              return self.data[key]
          defop Mapping.items(self):
              return self.data.items()

But as you can imagine, this would probably get a bit tedious if you're
implementing lots of methods.  So, we can add metaclasses or class
decorators here to say, "I implement these interfaces, so any methods I
have whose names match the method names in the interfaces, please hook 'em
up for me."  I'm going to leave out the implementation, as it should be a
straightforward exercise for the reader to come up with many ways by which
it can be accomplished.  The spelling might be something like:

     class MyMapping:
         implements(Mapping)

         def items(self):
             ...

         #etc.

At which point, we have now come full circle to being able to provide all
of the features of interfaces, adaptation, and generic functions, without
forcing anyone to give up the tasty OO flavor of method calls.  Heck, they
can even keep the way they spell existing adaptation calls (e.g. IFoo(bar)
to adapt bar to IFoo) in PEAK, Twisted, and Zope!

And finally, note that if you only want to perform one method call on a
given object, you can also use the generics directly, e.g.
Mapping.items(foo) instead of Mapping(foo).items().

Voila -- generic goodness and classic OO method-calling simplicity, all in
one simple to implement package.  It should now be apparent why I said that
interfaces are trivial to implement if you define them as namespaces for
generic functions, rather than as namespaces for methods.

There are many spinoffs possible, too.  For example, you could have a
factory function that turns an existing class's public operations into an
interface object.  There are also probably also some dark corners of the
idea that haven't been explored, because when I first proposed basically
this idea in '05, nobody was ready for it.  Now maybe we can actually talk
about the implications.

From p.f.moore at gmail.com  Thu Nov 23 18:55:20 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 23 Nov 2006 17:55:20 +0000
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <79990c6b0611230955j57bd3c34y60a35d2b95ea1dfa@mail.gmail.com>

On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> I'm reposting this under a different subject because the other subject
> seems to have gone off on a tangent. The rest are Phillip's words.
[...]

OK, I've read and tried to digest this. It looks good. The one thing
I'm still not getting, at a very concrete level, is precisely what
changes are required to Python to make it work. The code is clear
enough, but it's subtly not Python... For example, the comment "(I am
assuming here that normal functions are implicitly overloadable, even
if that means they change type at runtime to do so.)" - I don't
understand what, if anything this implies about the semantics of the
"def" statement (assuming that's the statement involved).

If there's a pointer to a previous posting that would clarify, I'd
appreciate it - although maybe it's another thing that would benefit
from being restated in the light of the new context.

Thanks, and apologies if I'm being dumb.

Paul.

From janssen at parc.com  Thu Nov 23 19:17:11 2006
From: janssen at parc.com (Bill Janssen)
Date: Thu, 23 Nov 2006 10:17:11 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov22.183133pst."58648"@synergy1.parc.xerox.com> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
	<06Nov22.183133pst."58648"@synergy1.parc.xerox.com>
Message-ID: <06Nov23.101719pst."58648"@synergy1.parc.xerox.com>

> > I don't know anything by CLOS.
> 
> I'll drop off a copy of the book on Monday.  Lord knows we have
> zillions of extra copies floating around PARC.

I'll still drop off a copy (of Common Lisp the Language, version 2),
but there's no need to wait.  It's on the Web at
http://www.supelec.fr/docs/cltl/clm/node260.html.  See in particular
the "change-class" operation at
http://www.supelec.fr/docs/cltl/clm/node305.html.

I think I'm still confused (happens a lot :-) about our method
namespace discussion.  It seems to me that Python's method namespaces
work pretty much the same way that CLOS's do, already.  That is, you
don't "clobber" an existing method in a base class when you define a
new method by the same name in a derived class; you just mask it.  The
base class' method is still there, and can still be called explicitly.

Bill

From pje at telecommunity.com  Thu Nov 23 19:19:07 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 13:19:07 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <79990c6b0611230955j57bd3c34y60a35d2b95ea1dfa@mail.gmail.co
 m>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>

At 05:55 PM 11/23/2006 +0000, Paul Moore wrote:
>On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> > I'm reposting this under a different subject because the other subject
> > seems to have gone off on a tangent. The rest are Phillip's words.
>[...]
>
>OK, I've read and tried to digest this. It looks good. The one thing
>I'm still not getting, at a very concrete level, is precisely what
>changes are required to Python to make it work.

No changes as such, just additions:

1. defop, addmethod/__addmethod__, maybe hasmethod/__hasmethod__
2. some generic function implementation that can be applied to "normal" 
Python functions


>  The code is clear
>enough, but it's subtly not Python... For example, the comment "(I am
>assuming here that normal functions are implicitly overloadable, even
>if that means they change type at runtime to do so.)" - I don't
>understand what, if anything this implies about the semantics of the
>"def" statement (assuming that's the statement involved).

No changes in semantics to "def".  I'm just saying that you have to be able 
to call:

     addmethod(a_function, methodfunc, a_class)

And have the existing simple Python function 'a_function' be overloaded as 
a result.  RuleDispatch and PEAK-Rules do this by replacing the function's 
func_code with some newly-generated dispatching code, but if this is to 
become a feature of the Python language, it might be better to simply allow 
subclassing FunctionType, and let the addmethod operation change the 
function's __class__ on the fly.  This would allow generic function 
implementations to be written in C without additional calling overhead, but 
still just modify the original object in-place instead of having to replace it.

(You can't just replace the function with a *different* object, because by 
the time you overload it, there may be lots of references to it already.)

It may be that if a_function doesn't have an __addmethod__, then 
addmethod() should try to call some special method (__overload__?) on 
methodfunc (in case it's been decorated with something that knows how to 
turn a_function into a generic function.  Finally, if neither side knows 
what to do, addmethod() should default to using some built-in or stdlib 
generic function implementation.


From pje at telecommunity.com  Thu Nov 23 19:31:56 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 13:31:56 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061123121416.GA6072@niemeyer.net>
References: <45651355.7090605@canterbury.ac.nz>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
	<45651355.7090605@canterbury.ac.nz>
Message-ID: <5.1.1.6.0.20061123132551.04454ae8@sparrow.telecommunity.com>

At 10:14 AM 11/23/2006 -0200, Gustavo Niemeyer wrote:
>This is true for generic functions as well, except that
>instead of using an IFruit interface, you have a fruit()
>generic function which handles the adaptation.  Unfortunately,
>this model doesn't work with more complex hierachies.

You've got that backwards, actually; generic functions can work with 
"recombinant" hierarchies, where traditional interfaces can't.  See Guido's 
repost of my explanation under the "Generic functions vs OO" thread.  In 
it, I present a short "Interface" implementation based on generic functions 
that provides universal adaptation for arbitrary interface hierarchies, 
including subset interfaces -- *without having to write adapter classes*:

http://mail.python.org/pipermail/python-3000/2006-November/004746.html


From janssen at parc.com  Thu Nov 23 19:39:24 2006
From: janssen at parc.com (Bill Janssen)
Date: Thu, 23 Nov 2006 10:39:24 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ek3kp8$19m$2@sea.gmane.org> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
	<06Nov22.181438pst."58648"@synergy1.parc.xerox.com>
	<ek3kp8$19m$2@sea.gmane.org>
Message-ID: <06Nov23.103924pst."58648"@synergy1.parc.xerox.com>

> Bill Janssen wrote:
> 
> >> Ow, that's the first time you describe this particular wrinkle.
> > 
> > Yep.  I hesitated to bring it up, but if there's a need for separate
> > namespaces for method names, why not do it right?
> 
> what makes you so sure that Python's current design isn't "right" ?

Well, "right" is a highly subjective term, and I think it's a bit
foolish to say that a language as widely used, and widely-useful, as
Python, isn't "right".  But in Py3K, we're talking about what could be
improved, and there are *facets* of Python which could better support
its use.  I think method namespaces are already OK; the question is
whether to move these specially-named (for whatever reason) methods to
a real namespace (a dict somewhere), rather than using this
name-mangling kludge to introduce one.  If they're special methods for
the VM, as Ivan implied, let's put them in "PythonVM".  If they're
what operations and built-in functions call, let's use the "operation"
namespace suggested by Calvin Speakman.

Bill

From guido at python.org  Thu Nov 23 19:43:43 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 10:43:43 -0800
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
Message-ID: <ca471dc20611231043l4e339fd8oc909c717ef687db5@mail.gmail.com>

On 11/23/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 05:55 PM 11/23/2006 +0000, Paul Moore wrote:
> >On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> > > I'm reposting this under a different subject because the other subject
> > > seems to have gone off on a tangent. The rest are Phillip's words.
> >[...]
> >
> >OK, I've read and tried to digest this. It looks good. The one thing
> >I'm still not getting, at a very concrete level, is precisely what
> >changes are required to Python to make it work.
>
> No changes as such, just additions:
>
> 1. defop, addmethod/__addmethod__, maybe hasmethod/__hasmethod__
> 2. some generic function implementation that can be applied to "normal"
> Python functions
>
>
> >  The code is clear
> >enough, but it's subtly not Python... For example, the comment "(I am
> >assuming here that normal functions are implicitly overloadable, even
> >if that means they change type at runtime to do so.)" - I don't
> >understand what, if anything this implies about the semantics of the
> >"def" statement (assuming that's the statement involved).
>
> No changes in semantics to "def".  I'm just saying that you have to be able
> to call:
>
>      addmethod(a_function, methodfunc, a_class)
>
> And have the existing simple Python function 'a_function' be overloaded as
> a result.  RuleDispatch and PEAK-Rules do this by replacing the function's
> func_code with some newly-generated dispatching code, but if this is to
> become a feature of the Python language, it might be better to simply allow
> subclassing FunctionType, and let the addmethod operation change the
> function's __class__ on the fly.  This would allow generic function
> implementations to be written in C without additional calling overhead, but
> still just modify the original object in-place instead of having to replace it.
>
> (You can't just replace the function with a *different* object, because by
> the time you overload it, there may be lots of references to it already.)
>
> It may be that if a_function doesn't have an __addmethod__, then
> addmethod() should try to call some special method (__overload__?) on
> methodfunc (in case it's been decorated with something that knows how to
> turn a_function into a generic function.  Finally, if neither side knows
> what to do, addmethod() should default to using some built-in or stdlib
> generic function implementation.

Changing the class can't add fields to the C struct that represent the
object (since that would mean realloc()'ing the object and hence
potentially changing its address). So why not make this a capability
of the base function type, rather than messing with __class__?

(This is not an endorsement of the whole thing, which I haven't read yet.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tony at PageDNA.com  Thu Nov 23 19:00:01 2006
From: tony at PageDNA.com (Tony Lownds)
Date: Thu, 23 Nov 2006 10:00:01 -0800
Subject: [Python-3000] optional argument annotations
Message-ID: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>

I have a working optional argument syntax implementation, I'm hoping  
to get some direction on
the implementation decisions so far.... please see below.

Python 3.0x (p3yk:52824M, Nov 23 2006, 09:22:23)
[GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> def f()-> 1: pass
...
 >>> f.func_returns
1
 >>> def f(): pass
...
 >>> f.func_annotations
 >>> f.func_returns
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
AttributeError: 'function' object has no attribute 'func_returns'
 >>> def f(x:1): pass
...
 >>> f.func_annotations
{'x': 1}
 >>>
 >>> import dis
 >>> dis.dis(compile("def f(x:1, y:2=3, z=4)->5:pass", "<str>", "exec"))
   1           0 LOAD_CONST               0 (3)         # default for y
               3 LOAD_CONST               1 (4)          # default for z
               6 LOAD_CONST               2 (1)          # annotation  
for x
               9 LOAD_CONST               3 (2)          # annotation  
for y
              12 LOAD_CONST               4 (6)         # bitmask for  
annotations... 0b110
              15 LOAD_CONST               5 (5)         # func_returns
              18 LOAD_CONST               6 (<code object f at  
0xb7f08848, file "<str>", line 1>)
              21 MAKE_FUNCTION        49154      # numdefaults(2) +  
kwdefaults(0) << 8 + has_annotations(1) << 14 + has_returns(1) << 15
              24 STORE_NAME               0 (f)
              27 LOAD_CONST               7 (None)
              30 RETURN_VALUE

Index: Parser/Python.asdl
===================================================================
--- Parser/Python.asdl  (revision 52824)
+++ Parser/Python.asdl  (working copy)
@@ -9,8 +9,8 @@
             -- not really an actual node but useful in Jython's  
typesystem.
             | Suite(stmt* body)
-       stmt = FunctionDef(identifier name, arguments args,
-                           stmt* body, expr* decorators)
+       stmt = FunctionDef(identifier name, typedarguments args,
+                           stmt* body, expr* decorators, expr? returns)
               | ClassDef(identifier name, expr* bases, stmt* body)
               | Return(expr? value)
@@ -102,6 +102,9 @@
         arguments = (expr* args, identifier? vararg, expr* kwonlyargs,
                          identifier? kwarg, expr* defaults, expr*  
kw_defaults)
+       typedarguments = (annotatedarg* args, identifier? vararg,  
expr* kwonlyargs,
+                        identifier? kwarg, expr* defaults, expr*  
kw_defaults)
+       annotatedarg = (expr arg, expr? annotation)
          -- keyword arguments supplied to call
          keyword = (identifier arg, expr value)



From guido at python.org  Thu Nov 23 19:53:19 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 10:53:19 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <pan.2006.11.23.07.24.19.642912@ID-174572.user.uni-berlin.de>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<20061122202924.GA21361@niemeyer.net>
	<ca471dc20611221238o60c1a4b3rca74b07036d361fa@mail.gmail.com>
	<pan.2006.11.23.07.24.19.642912@ID-174572.user.uni-berlin.de>
Message-ID: <ca471dc20611231053u7121c048paebc3eca5d0744e6@mail.gmail.com>

On 11/22/06, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> Guido van Rossum wrote:
>
> >> I'm sure you're aware about it, but in Zope 3 terminology, these are
> >> 'provide' and 'implement', respectively.
> >
> > I like provide, but I'm not so sure about implement, since it is awfully
> > ambiguous -- most of the time it is the class that does the implementing.
> > That's why I settled for "has".
>
> JFTR: In Zope3, classes "implement" interfaces while instances "provide"
> them.

Well, that pretty much proves their terminology is confusing. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Nov 23 20:12:50 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 14:12:50 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611231053u7121c048paebc3eca5d0744e6@mail.gmail.co
 m>
References: <pan.2006.11.23.07.24.19.642912@ID-174572.user.uni-berlin.de>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<20061122202924.GA21361@niemeyer.net>
	<ca471dc20611221238o60c1a4b3rca74b07036d361fa@mail.gmail.com>
	<pan.2006.11.23.07.24.19.642912@ID-174572.user.uni-berlin.de>
Message-ID: <5.1.1.6.0.20061123140510.044eb4e8@sparrow.telecommunity.com>

At 10:53 AM 11/23/2006 -0800, Guido van Rossum wrote:
>On 11/22/06, Thomas Lotze <thomas at thomas-lotze.de> wrote:
> > Guido van Rossum wrote:
> >
> > >> I'm sure you're aware about it, but in Zope 3 terminology, these are
> > >> 'provide' and 'implement', respectively.
> > >
> > > I like provide, but I'm not so sure about implement, since it is awfully
> > > ambiguous -- most of the time it is the class that does the implementing.
> > > That's why I settled for "has".
> >
> > JFTR: In Zope3, classes "implement" interfaces while instances "provide"
> > them.
>
>Well, that pretty much proves their terminology is confusing. :-)

You can actually blame that one on me; I'm the one who proposed it.  :)

At the time, mailing list discussions were hard to follow as to whether we 
were talking about instances or classes when we were talking about 
"implements".  So I suggested that we add "provide" to mean that you could 
actually *use* the interface on the object, and leave "implements" to mean 
that your *instances* provide the interface.

The distinction helped the immediate discussion, but later, when I went on 
to create PyProtocols, I realized it wasn't as clear outside the original 
context, so I stopped talking about 'implements' entirely and just always 
used 'provides' instead.  So, instead of saying that a class "implements" 
X, PyProtocols says that a class' *instances* provide X, and you declare 
this with 'instancesProvide'.  If the class *itself* provides the interface 
(e.g. via classmethods), you declare it with 'classProvides'.

So, I think the "provide" term was a good idea, it's just that having a 
different word to mean "my instances provide", didn't work out for me.


From pje at telecommunity.com  Thu Nov 23 20:21:23 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 14:21:23 -0500
Subject: [Python-3000] optional argument annotations
In-Reply-To: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
Message-ID: <5.1.1.6.0.20061123141550.04c672e8@sparrow.telecommunity.com>

At 10:00 AM 11/23/2006 -0800, Tony Lownds wrote:
>I have a working optional argument syntax implementation, I'm hoping
>to get some direction on
>the implementation decisions so far.... please see below.

Why not just have the generated bytecode construct the annotation 
dictionary directly and assign it to the returned function's 
func_annotations, and the same for func_return if needed?  Then there'd be 
no need to change the MAKE_FUNCTION opcode.  Mightn't that make it easier 
to e.g. backport to the 2.x line, since it'd only be a compiler change, not 
an interpreter or function type change?


From pje at telecommunity.com  Thu Nov 23 20:24:04 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 14:24:04 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611231043l4e339fd8oc909c717ef687db5@mail.gmail.co
 m>
References: <5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061123135021.04c2ecf8@sparrow.telecommunity.com>

At 10:43 AM 11/23/2006 -0800, Guido van Rossum wrote:
>Changing the class can't add fields to the C struct that represent the
>object (since that would mean realloc()'ing the object and hence
>potentially changing its address). So why not make this a capability
>of the base function type, rather than messing with __class__?

Because it would allow generic function implementations written in Python 
(i.e. using __dict__ for any additional data) to do the same thing without 
needing to much with func_code, the way some of my implementations do now.

The use case for changing the type is allowing functions to only become 
generic once they're *actually* overloaded, so you don't have to go around 
decorating every function as generic "just in case" you want to overload it 
later.  Meanwhile, it would interfere with the "fast track" calling path in 
the interpreter to have generic functions be the same type as regular 
functions.  We'd have to add another check there, like a flag or something, 
so swapping out the type would make it just work with the existing eval 
loop.  (If we later want to have a fast-track for generics, we can always 
check for that new type, too.)

Anyway, all of this implementation discussion is probably premature 
optimization.  My main purpose in allowing delayed overloading is to 
support CLOS-style method combination (or other specialized generic 
function features) on normal functions, without having to predefine the 
original function as being generic.  Of course, changing func_code will 
presumably still work for that, but it precludes the possibility of using a 
C implementation in that case.

But of course all of this is moot unless/until we're actually discussing a 
"real" implementation instead of the proof-of-concept.


From gustavo at niemeyer.net  Thu Nov 23 21:02:11 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 18:02:11 -0200
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <20061123200211.GA22490@niemeyer.net>

> At 10:14 AM 11/23/2006 -0200, Gustavo Niemeyer wrote:
> >This is true for generic functions as well, except that
> >instead of using an IFruit interface, you have a fruit()
> >generic function which handles the adaptation.  Unfortunately,
> >this model doesn't work with more complex hierachies.
> 
> You've got that backwards, actually; generic functions can work with
> "recombinant" hierarchies, where traditional interfaces can't.  See
> Guido's repost of my explanation under the "Generic functions vs OO"
> thread.  In it, I present a short "Interface" implementation based on

Nonsense.  Coming up with an interface implementation that
includes generic functions makes the point that interfaces are
important stronger, and thus reinforce what I just said.  I never
said generic functions are bad or non-useful, I said that they're
limited and won't work in more complex cases without additional
infrastructure.

-- 
Gustavo Niemeyer
http://niemeyer.net

From gustavo at niemeyer.net  Thu Nov 23 21:14:13 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 18:14:13 -0200
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <20061123201413.GB22490@niemeyer.net>

(...)
> A few important points here:
> 
> 1. A basic interface mechanism is extemely simple to implement, given
> generic functions
(...)

All of these points are completely true for interfaces implemented
without generic functions.

In fact, I fail to recognize in your mail a reason why generic
functions improve the developer's experience at all.  Would you be
able to expose a concrete case where this system would present
advantages?

-- 
Gustavo Niemeyer
http://niemeyer.net

From tony at PageDNA.com  Thu Nov 23 21:27:48 2006
From: tony at PageDNA.com (Tony Lownds)
Date: Thu, 23 Nov 2006 12:27:48 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <5.1.1.6.0.20061123141550.04c672e8@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061123141550.04c672e8@sparrow.telecommunity.com>
Message-ID: <73F95868-C85C-4AD7-ACA7-E969D641BB67@PageDNA.com>


On Nov 23, 2006, at 11:21 AM, Phillip J. Eby wrote:

> Why not just have the generated bytecode construct the annotation  
> dictionary directly and assign it to the returned function's  
> func_annotations, and the same for func_return if needed?  Then  
> there'd be no need to change the MAKE_FUNCTION opcode.  Mightn't  
> that make it easier to e.g. backport to the 2.x line, since it'd  
> only be a compiler change, not an interpreter or function type change?
>
>

It's more trips around the ceval loop and more constants on
the code object with the MAKE_FUNCTION opcode.

OTOH the mask stuff is pretty ugly. As a bonus, solving this problem  
is easier:

 >>> def f((x,y):1): pass
...
 >>> f.func_annotations
{'.0': 1}

I'm not sure how backwards compatibility is affected. MAKE_FUNCTION  
has already changed in p3yk.

It's worth a try though, thanks!

-Tony


From greg.ewing at canterbury.ac.nz  Thu Nov 23 21:27:48 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Nov 2006 09:27:48 +1300
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
	<91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
	<ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
Message-ID: <45660444.4090204@canterbury.ac.nz>

Guido van Rossum wrote:
> Your presence here has been entirely a distraction. That's a troll to me.

Steady on, Guido. Trolling is making deliberately
inflammatory statements with the intention of
stirring up controversy. That's a pretty heavy
accusation to make. You're effectively saying
he's setting out to be obnoxious, and I don't
see that in any of his postings.

--
Greg

From pje at telecommunity.com  Thu Nov 23 21:54:23 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 15:54:23 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <20061123201413.GB22490@niemeyer.net>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061123153733.01f418e8@sparrow.telecommunity.com>

At 06:14 PM 11/23/2006 -0200, Gustavo Niemeyer wrote:
>(...)
> > A few important points here:
> >
> > 1. A basic interface mechanism is extemely simple to implement, given
> > generic functions
>(...)
>
>All of these points are completely true for interfaces implemented
>without generic functions.

Let's see your implementation.  Zope's interface system fails on at least 
points 3 and 5.  And regarding #1, Zope's implementation of the basic 
features I described is an order of magnitude more complex, even if you 
compare *just* its code that implements adaptation and interface 
inheritance, with *all* of my code plus Guido's generic function 
implementation.

Indeed, even PyProtocols' implementation of similar features is an order of 
magnitude more complex than this.  PyProtocols at least has a kludgy way of 
doing #5, but it fails #3 along with Zope.

My point is that no interface system for Python that I'm aware of can do 
anywhere *near* as much with so little code or complexity.


>In fact, I fail to recognize in your mail a reason why generic
>functions improve the developer's experience at all.  Would you be
>able to expose a concrete case where this system would present
>advantages?

In no particular order:

1. "Recombinant" interfaces

2. Absence of adapter classes for "stateless" adapters

3. No need to adapt for single-operation interfaces (or to define 
interfaces at all in that case)

4. No need to understand interface or adaptation theory to be able to use it

5. Generic functions are easily grasped as a dynamic version of overloading 
as seen in other languages (including C#, Java, C++, etc., where there's no 
built-in notion of "adaptation")


From pje at telecommunity.com  Thu Nov 23 22:05:15 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 16:05:15 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <5.1.1.6.0.20061123153733.01f418e8@sparrow.telecommunity.co
 m>
References: <20061123201413.GB22490@niemeyer.net>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061123155720.01f40858@sparrow.telecommunity.com>

At 03:54 PM 11/23/2006 -0500, Phillip J. Eby wrote:
>5. Generic functions are easily grasped as a dynamic version of overloading
>as seen in other languages (including C#, Java, C++, etc., where there's no
>built-in notion of "adaptation")

Oh, and I forgot: they're also easily understood as generic or polymorphic 
functions by people who use them in languages like CLOS, Dylan, Haskell, 
etc.  Not that that's necessarily a big population compared to the above 
languages, but my point here is that even these less-popular languages 
don't really have adaptation as such.  (Haskell's typeclasses are also very 
close in spirit to the interface approach I propose, but that's neither an 
advantage nor disadvantage, just a data point.)

Also, COM and .NET *do* have a notion of interfaces, but it's actually more 
similar to what I'm proposing than to what Zope or PyProtocols interfaces 
do.  They are like typeclasses in the sense that an interface on a COM 
object is normally just a stateless adapter providing an effective "method 
namespace" on the object.

Anyway, the point was just that one advantage of my proposal is that it's 
easier to teach, because it has many things it can be compared to, 
depending on the audience: dynamic overloading, generic functions, 
typeclasses, COM "query-interface", etc.  It can even be compared to 
existing Python interface implementations, in terms of what things it can 
do that they can't, as well as the things they have in common.  :)


From p.f.moore at gmail.com  Thu Nov 23 22:53:57 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 23 Nov 2006 21:53:57 +0000
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
Message-ID: <79990c6b0611231353o70059496w8e2e04aea2786421@mail.gmail.com>

On 11/23/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 05:55 PM 11/23/2006 +0000, Paul Moore wrote:
> >On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> > > I'm reposting this under a different subject because the other subject
> > > seems to have gone off on a tangent. The rest are Phillip's words.
> >[...]
> >
> >OK, I've read and tried to digest this. It looks good. The one thing
> >I'm still not getting, at a very concrete level, is precisely what
> >changes are required to Python to make it work.
>
> No changes as such, just additions:

Hmm, I'm not getting my question across (but that's OK, you answered
anyway). From my POV, additions are changes - if you like, I'm trying
to imagine what a diff between current Python and your proposal would
look like.

> 1. defop, addmethod/__addmethod__, maybe hasmethod/__hasmethod__
> 2. some generic function implementation that can be applied to "normal"
> Python functions

Um, isn't (1) a generic function implementation that can be applied to
Python functions?

> No changes in semantics to "def".  I'm just saying that you have to be able
> to call:
>
>      addmethod(a_function, methodfunc, a_class)

OK. So under your proposal, all Python functions are (potentially)
generic, and addmethod is how you add extra overloads? (And hasmethod
is pretty obvious, and defop AFAICT is a syntax for addmethod, is that
right?)

Did I get that right?
Paul

From greg.ewing at canterbury.ac.nz  Thu Nov 23 23:03:15 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Nov 2006 11:03:15 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061123121416.GA6072@niemeyer.net>
References: <5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
	<45651355.7090605@canterbury.ac.nz>
	<20061123121416.GA6072@niemeyer.net>
Message-ID: <45661AA3.9070803@canterbury.ac.nz>

Gustavo Niemeyer wrote:

> If you have an adapter from Apple to IFruit, any user
> requiring an IFruit interface for an object will be
> able to obtain it from Apple instances.  The "automatically"
> part is not quite true, as adaptation must be explicitly
> requested.

Yes, and that's where it falls down, as far as I can
see. It's not just that it's "not quite" automatic,
it's almost completely *non*-automatic. Given that,
I see little benefit in having a formal mechanism
for it.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From pje at telecommunity.com  Thu Nov 23 23:27:05 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 23 Nov 2006 17:27:05 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <79990c6b0611231353o70059496w8e2e04aea2786421@mail.gmail.co
 m>
References: <5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<5.1.1.6.0.20061123130727.01f42838@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061123172607.01f443f0@sparrow.telecommunity.com>

At 09:53 PM 11/23/2006 +0000, Paul Moore wrote:
>OK. So under your proposal, all Python functions are (potentially)
>generic, and addmethod is how you add extra overloads? (And hasmethod
>is pretty obvious, and defop AFAICT is a syntax for addmethod, is that
>right?)
>
>Did I get that right?

Yes, that's exactly it.  Thanks for the succinct summary of the entire 
thing (minus the interface example).  As you may have noticed, I have a 
very hard time doing "succinct".  :)


From gustavo at niemeyer.net  Thu Nov 23 23:47:59 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 23 Nov 2006 20:47:59 -0200
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <45661AA3.9070803@canterbury.ac.nz>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
	<45651355.7090605@canterbury.ac.nz>
	<20061123121416.GA6072@niemeyer.net>
	<45661AA3.9070803@canterbury.ac.nz>
Message-ID: <20061123224759.GA27828@niemeyer.net>

> Yes, and that's where it falls down, as far as I can
> see. It's not just that it's "not quite" automatic,
> it's almost completely *non*-automatic. Given that,
> I see little benefit in having a formal mechanism
> for it.

I don't see your point here.  Are you arguing that
pretty much every approach being discussed is bad and
we should instead implement something magically
transforming types?  I'm strongly -1 if that's the case.

All proposals so far do some kind of conditional jumping
only in specific entry points depending on features (class,
interface, ability, whatever) declared for the object.

-- 
Gustavo Niemeyer
http://niemeyer.net

From greg.ewing at canterbury.ac.nz  Fri Nov 24 00:41:35 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Nov 2006 12:41:35 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <20061123224759.GA27828@niemeyer.net>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
	<45651355.7090605@canterbury.ac.nz>
	<20061123121416.GA6072@niemeyer.net>
	<45661AA3.9070803@canterbury.ac.nz>
	<20061123224759.GA27828@niemeyer.net>
Message-ID: <456631AF.1020208@canterbury.ac.nz>

Gustavo Niemeyer wrote:
> Are you arguing that
> pretty much every approach being discussed is bad and
> we should instead implement something magically
> transforming types?

No, I'm questioning the need to do anything at all.
I don't see a big problem that merits a heavyweight
solution like a formal interface or adaptation
framework.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov 24 01:02:17 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Nov 2006 13:02:17 +1300
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611222029mb7cfa7cl9436db31e4b0957a@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<4565144B.5060109@canterbury.ac.nz>
	<ca471dc20611222029mb7cfa7cl9436db31e4b0957a@mail.gmail.com>
Message-ID: <45663689.5030006@canterbury.ac.nz>

Guido van Rossum wrote:
> I fear I will miss the class as a convenient focus for
> related functionality.

I have the same feeling. I get the impression that
generic functions work okay in a language designed
around them from the beginning. But adding them
belatedly to a language built around classes and
methods would give Too Many Ways To Do It.

Another thing I don't like about generic functions
is their comefromish nature. You know that an
implementation of a method for some given type
is going to be defined in one of a few easily-found
places, but an implementation of a generic function
for that type could be anywhere in the program.

Or maybe that's just another way of stating your
"classes as a focus of functionality" argument. In
any case, from a software engineering viewpoint it
seems bad, for the same reasons that monkeypatching
is generally considered undesirable.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov 24 01:09:44 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Nov 2006 13:09:44 +1300
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122233433.028ea230@sparrow.telecommunity.com>
References: <5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122233433.028ea230@sparrow.telecommunity.com>
Message-ID: <45663848.1000508@canterbury.ac.nz>

Phillip J. Eby wrote:

> Calling people names isn't particularly conducive to a technical 
> discussion...

Sorry, I didn't mean it to sound that way. I was just
making reference to a concept elucidated elsewhere --
invoking an "argument pattern", so to speak.-)

What I'm saying is that, because the object returned
by iter() not only has a different interface but a
different *function*, it doesn't seem to fit the notion
implied to me by "adaptation". And that to stretch the
definition of adaptation to include it would be to
make the term so broad it would lose much of its
usefulness.

--
Greg

From guido at python.org  Fri Nov 24 03:55:33 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 18:55:33 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
Message-ID: <ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>

On 11/23/06, Tony Lownds <tony at pagedna.com> wrote:
> I have a working optional argument syntax implementation, I'm hoping
> to get some direction on
> the implementation decisions so far.... please see below.

Wow!

> Python 3.0x (p3yk:52824M, Nov 23 2006, 09:22:23)
> [GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> def f()-> 1: pass
> ...
>  >>> f.func_returns
> 1
>  >>> def f(): pass
> ...
>  >>> f.func_annotations

It would be ok if this returned {} too. (But None is fine too I think.)

>  >>> f.func_returns
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> AttributeError: 'function' object has no attribute 'func_returns'

I would prefer this to be None. Attributes that don't always exist are
a pain to use.

>  >>> def f(x:1): pass
> ...
>  >>> f.func_annotations
> {'x': 1}

Very cool! I agree that Phillip's idea for simplification is worth a
try. We're generally not too concerned over the cost of function
declarations since they typically execute only once per program. As
long as it's really cheap when no annotations are present (which would
suggest that func_annotations should be None in that case since an
empty dict is kind of expensive, at least in memory).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Nov 24 05:10:41 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 20:10:41 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <-4940442411288794438@unknownmsgid>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
	<-4940442411288794438@unknownmsgid>
Message-ID: <ca471dc20611232010i182761cdr54f808cb60938231@mail.gmail.com>

On 11/23/06, Bill Janssen <janssen at parc.com> wrote:
> > > I don't know anything by CLOS.
> >
> > I'll drop off a copy of the book on Monday.  Lord knows we have
> > zillions of extra copies floating around PARC.
>
> I'll still drop off a copy (of Common Lisp the Language, version 2),
> but there's no need to wait.  It's on the Web at
> http://www.supelec.fr/docs/cltl/clm/node260.html.  See in particular
> the "change-class" operation at
> http://www.supelec.fr/docs/cltl/clm/node305.html.
>
> I think I'm still confused (happens a lot :-) about our method
> namespace discussion.  It seems to me that Python's method namespaces
> work pretty much the same way that CLOS's do, already.  That is, you
> don't "clobber" an existing method in a base class when you define a
> new method by the same name in a derived class; you just mask it.  The
> base class' method is still there, and can still be called explicitly.

OK, maybe I  misunderstood what you wrote. I thought I heard you say
that "len" isn't just "len" -- it's the "len" defined by some
interface (and presumably implemented in a base class), and if one
defined a new "len" it wouldn't override the "len" defined by that
interface (unless one explicitly stated that it did), it would just
add a different method named "len". That would fly in the face of
Python's lookup algorithm for methods (where the first "len" you find
is the one you get). If that's not what you meant, all is probably
well.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Nov 24 05:13:38 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 20:13:38 -0800
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <45660444.4090204@canterbury.ac.nz>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2cf8$su5$1@sea.gmane.org>
	<91ad5bf80611221348n59ecbfb6w930046b1378d49ca@mail.gmail.com>
	<ca471dc20611221548j4b3d49a8n1e03475de2df86c1@mail.gmail.com>
	<91ad5bf80611221929o4230fccfh663bf7a5bc5564d@mail.gmail.com>
	<ca471dc20611221956h474548em243745d73bc3ec9f@mail.gmail.com>
	<45660444.4090204@canterbury.ac.nz>
Message-ID: <ca471dc20611232013w6fce0bacy9dbf4ed5e89cee4a@mail.gmail.com>

I'll give you that it may not have been intentional. But the effect is
the same. And I felt rather offended by his offhand rejection of "most
HCI arguments" -- in my view sarcasm is quite a bit more acceptable
than insult. I stick to my point that he is lacking essential social
skills for effectively participating in an on-line discussion.

On 11/23/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
> > Your presence here has been entirely a distraction. That's a troll to me.
>
> Steady on, Guido. Trolling is making deliberately
> inflammatory statements with the intention of
> stirring up controversy. That's a pretty heavy
> accusation to make. You're effectively saying
> he's setting out to be obnoxious, and I don't
> see that in any of his postings.
>
> --
> Greg
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Nov 24 06:34:49 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 23 Nov 2006 21:34:49 -0800
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
Message-ID: <ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>

Phillip J. Eby wrote:
> At 08:29 PM 11/22/2006 -0800, Guido van Rossum wrote:
> >One thing that rubs me the wrong way about generic functions is that
> >it appears to go against OO. Now I'm not someone to take OO as
> >religion, but there's something uncomfortable (for me) about how, in
> >Phillip's world, many things become functions instead of methods,
> >which brings along concerns about the global namespace filling up, and
> >also about functionality being spread randomly across too many
> >modules. I fear I will miss the class as a convenient focus for
> >related functionality.
>
> I originally proposed a solution for this back in January '05, but it was
> too premature.  But since you have now stated the problem that the proposal
> was intended to solve, perhaps the solution has a chance now.  :)
>
> I will try to be as concrete as possible.  Let's start with an actual,
> hopefully non-exploding 'Interface' implementation, based on an assumption
> that we have generic functions available:

Could you point out where the example uses generic functions?

>      class InterfaceClass(type):
>          def __init__(cls, name, bases, cdict):
>              for k,v in cdict.items():
>                  # XXX this should probably skip at least __slots__,
>                  #     __metaclass__, and __module__, but oh well
>                  cdict[k] = AdaptingDescriptor(v)
>
>      class Interface:
>          __metaclass__ = InterfaceClass
>          __slots__ = '__self__'
>
>          def __init__(self, subject):
>              # this isinstance() check should be replaced by an
>              # 'unwrap()' generic function, so other adapter types
>              # will work, but this is just an example, so...
>              if isinstance(subject, Interface):
>                  subject = subject.__self__
>              self.__self__ = subject
>
>      class AdaptingDescriptor:
>          def __init__(self, descriptor):
>              self.wrapped = descriptor
>          def __get__(self, ob, typ=None):
>              if ob is None:
>                  return self
>              return self.wrapped.__get__(ob.__self__, typ)

OK, that didn't explode my head, but it went straight over it. It may
only be 20 or so lines of code, but it's so concentrated that I can't
really understand it. All I get is that there's a class named
Interface that does some magic. (I'm sure I could understand it, but
it would take me a whiteboard and half an hour close reading of the
code, and I just don't want to spend that time right now.)

> Now, using this new "interface framework", let's implement a small
> "mapping" typeclas... er, interface.
>
>      class Mapping(Interface):
>          def keys(self):
>              return [k for k,v in self.items()]
>          def items(self):
>              return [k,self[k] for k in self.keys()]
>          # ... other self-recursive definitions
>
> What does this do?  Well, we can now call Mapping(foo) to turn an arbitrary
> object into something that has Mapping's generic functions as its methods,

What are Mapping's generic functions? keys and items? How can I tell
they are generic? What made them generic?

> and invokes them on foo!  (I am assuming here that normal functions are
> implicitly overloadable, even if that means they change type at runtime to
> do so.)

I read your explanation of that assumption before but I still don't
like it. I'd much rather have a slot where the overloading gets stored
that's NULL if there's no overloading. We can optimize the snot out of
it some other way. (I guess my problem with objects that change type
is that it's not a common thing to happen in Python, even though it's
explicitly possible, and I expect that it will cause all sorts of
bizarre surprises for code that thinks it understands Python's object
model. Anyway, it's a distraction. Maybe we can put the class-changing
idea aside as a premature optimization.)

> We could even use interfaces for argument type declarations, to
> automatically put things in the "right namespace" for what the code expects
> to use.

What do you mean by "the right namespace"? What things are put there?

> That is, if you declare an argument to be a Mapping, then that's
> what you get.

Sounds like implied adaptation to me, which was already explicitly
rejected long ago.

> If you call .keys() on the resulting adapted object and the
> type doesn't support the operation, you get an error.

How could it not support it if Mapping(x) succeeded?

How has keys() suddenly turned from a method into an operation
(assuming "operation" is a technical term and not just a different
word for method)?

> Too late a form of error checking you say?  Well, make a more sophisticated
> factory mechanism in Interface.__new__ that actually creates (and caches)
> different adapter types based on the type of object being adapted, so that
> hasattr() tests will work on the wrapped type, or so that you can get an
> early error if none of the wrapped generic functions has a method defined
> for the target type.
>
> A few important points here:
>
> 1. A basic interface mechanism is extemely simple to implement, given
> generic functions

Given my inability to understand it I can't yet agree with this claim.

> 2. It is highly customizable with respect to error checking and other
> features, even on a per-user basis, because there doesn't have to be only
> one "true" Interface type to rule them all (or one true generic function
> type either, but that's a separate discussion).

I would think there are few approaches (in Python) that would require
"one true interface type". Maybe Zope/Twisted do, but probably more
for expedience than because it's the only way it could be done.

> 3. It allows interfaces to include partial implementations, ala Ping and
> Alex's past proposals, thus allowing you to implement partial mapping or
> "file" objects and have the rest of the interface's implementation filled
> in for you

I'm not even sure I like that. It also seems to hinge on adaptation.
I'd rather *not* make adaptation a key concept; to me adaptation and
interfaces are completely orthogonal.

I think of interfaces as ways to *talk* about sets of methods or
operations or abilities, and I like that interfaces are objects so you
can inspect what you're talking about, but I don't like the interface
to play a role in the actual *use* of an object. Perhaps as analogy,
the use of hasattr in current Python will help: I can test whether an
object has an attribute, and if it does, use the attribute. But the
attribute test is not used as an intermediary for using the attribute.
Similar with isinstance -- I can test whether an object is a
basestring, and if it is, use its lower() method. But the basestring
object isn't involved in using the lower() method. This is somewhat in
contrast to Java and other statically typed languages with dynamic
typecasts -- there you might have an Object x that you believe is
really a String, so you write "String s = (String)x" (I believe that's
the syntax -- I'm a bit rusty) and then you can use String methods on
s -- but not on x. I find that aspect unPythonic.

> 4. It allows you to hide the very existence of the notion of a "generic
> function", if you prefer not to think about such things

You've hidden them so well that I can't find them in your example. :-)

> 5. It even supports interface inheritance and interface algebra:
> subclassing an interface allows adding new operations, and simple
> assignment suffices to compose new interfaces, e.g.:
>
>      class MappingItems(Interface):
>          items = Mapping.items
>
> Notice that nothing special is required, this "just works" as a natural
> consequence of the rest of the implementation shown.

This I believe I actually understand.

> Okay, so now you want to know how to *implement* a "Mapping".  Well,
> simplest but most tedious, you can just register operations directly, e.g.:
>
>       class MyMapping:
>           def __init__(self, data):
>               self.data = dict(data)
>           defop operator.getitem(self, key):
>               return self.data[key]
>           defop Mapping.items(self):
>               return self.data.items()

Looks like you're being inconsistent with your example above, which
has keys() and items() but not __getitem__(). This confuses me.

I'm also concerned that this looks to me like a big step back from
simply writing

class MyMapping:
  def __init__(self, data): self.data = dict(data)
  def __getitem__(self, key): return self.data[key]
  def items(self): return self.data.items()

and then (either inside the class or external to it) adding some
statement that claims that MyMapping implements Mapping.

> But as you can imagine, this would probably get a bit tedious if you're
> implementing lots of methods.  So, we can add metaclasses or class
> decorators here to say, "I implement these interfaces, so any methods I
> have whose names match the method names in the interfaces, please hook 'em
> up for me."  I'm going to leave out the implementation, as it should be a
> straightforward exercise for the reader to come up with many ways by which
> it can be accomplished.  The spelling might be something like:
>
>      class MyMapping:
>          implements(Mapping)
>
>          def items(self):
>              ...
>
>          #etc.

Sure, that's what I was after above. But how would you do this after
the fact, when you find that some 3rd party module already has the
right methods but didn't bother to add the correct implements()
clause?

> At which point, we have now come full circle to being able to provide all
> of the features of interfaces, adaptation, and generic functions, without
> forcing anyone to give up the tasty OO flavor of method calls.  Heck, they
> can even keep the way they spell existing adaptation calls (e.g. IFoo(bar)
> to adapt bar to IFoo) in PEAK, Twisted, and Zope!
>
> And finally, note that if you only want to perform one method call on a
> given object, you can also use the generics directly, e.g.
> Mapping.items(foo) instead of Mapping(foo).items().

But I want to write foo.items()!!!

> Voila -- generic goodness and classic OO method-calling simplicity, all in
> one simple to implement package.  It should now be apparent why I said that
> interfaces are trivial to implement if you define them as namespaces for
> generic functions, rather than as namespaces for methods.
>
> There are many spinoffs possible, too.  For example, you could have a
> factory function that turns an existing class's public operations into an
> interface object.  There are also probably also some dark corners of the
> idea that haven't been explored, because when I first proposed basically
> this idea in '05, nobody was ready for it.  Now maybe we can actually talk
> about the implications.

Well, okay, I have to admit that I only half "get" what that was all about.

But let me end on a more positive note. I find the idea very
attractive that many builtin operations (from len to iter to + to <=
to __getitem__) can be considered to be generic functions. In my head
I worked out how you could take the built-in set type and the
(ancient) set-of-integers implementation hiding in mhlib.py
(mhlib.IntSet), and add the various set-theoretical operations (union,
intersection, differences, and containment relationships) on mixed
operands without modifying the code of the IntSet class. This is
pretty neat. You could even add those operations for two IntSet
instances without modifying that class. I am envisioning that most of
these operations would have a default implementation that does what
those operations currently do; e.g. the generic function __add__ would
be something like

@generic
def __add__(a, b):  # default implementation
  if hasattr(a, "__add__"):
    r = a.__add__(b)
    if r is not NotImplemented: return r
  if hasattr(b, "__radd__"):
    r = b.__radd__(a)
    if r is not NotImplemented: return r
  raise TypeError(...)

I also like the idea that we can associate an "atomic" interface with
a generic function, so that an object for which that GF has an
implementation is automatically said to implement that interface. And
I like the idea that we can compose new interfaces out of such atomic
interfaces, such that testing for whether an object implements such a
composite interface is automatically turned into testing whether it
implements each of the atomic interfaces. And I also like being able
to take arbitrary subsets of composite interfaces as the logical
consequence of this.

(I don't think the name of the interface should be the same as the
name of the generic function. This has caused me enough confusion
already. I'd rather be able to say e.g. that there's an interface
"Sizeable" which means that the operation "len" is provided.)

I think there are still some important holes in this idea (at least in
my understanding). For example, if all generic functions have a
default implementation, then essentially all generic functions are
implemented for all objects, and all objects have all interfaces!
That's not right. Of course, the default implementation may well raise
TypeError for some arguments (like my __add__ example). But that's not
helpful in deciding whether an object has an interface -- invoking the
operation is way too heavy-handed. Maybe we should provide a separate
testing function. (Although that doesn't seem quite sufficient for the
__add__ example, since it really *does* need to execute the operation
in order to determine whether it is implemented. I'll have to come
back to this problem later.)

Another issue is GFs taking multiple arguments. At least for binary
operators (like __add__ above) it may make sense to say that a class C
implements the operator's interface (e.g. Addable in this case) if the
operator is defined for operands that are both C instances. Or maybe
that interface should be called SelfAddable, and we should reserve
Addable for anything that can occur on the lhs of a + operator? I
think we can probably come up with a reasonable rule for this; right
now I'm too tired to think of one.

But the biggest hole (to me) is being able to talk about regular
methods in an interface. I would like to be able to say that e.g. the
interface StandardMapping implements the operations getitem, len, iter
and contains, and the methods get, keys, values and items. I have no
problem with the operations, but I don't quite get how we can add the
methods to the interface. I understand that your explanation above
provides ways to talk about those, I just don't quite see the light
yet.

I would like to end up in a world where, in order to claim that a
class is a standard mapping class, the class definition would
(normally) explicitly claim to implement StandardMapping (using as yet
unspecified syntax) and that there would be a way (again I'm not yet
specifying syntax for this) to efficiently test any given class or
instance for that interface, returning True or False. The standard
dict implementation would claim to implement StandardMapping (and
probably also something that we could call StandardMutableMapping).
Someone could define an interface MinimalMapping as the interface
formed of StandardMapping.getitem, StandardMapping.contains and
StandardMapping.keys, and testing a dict for MinimalMapping would
return True. But if someone created ExtendedMapping as StandardMapping
plus the copy method and the eq operation, testing a dict for
ExtendedMapping would return False; however there should be something
one could execute to explicitly mark dict as implementing
ExtendedMapping, using some kind of registry.

Reiterating, I wonder if perhaps a major disconnect has to do with the
difference between "testing" for an interface vs. "casting" to an
interface. I don't like casting, since it smells like adaptation.

(There's also an extension of these ideas I'd like to explore where
full method signatures are possible in an interface, in an
introspectable way. This would probably require parameterizable types,
so that one could talk about StandardMapping[int, str] (a standard
mapping from ints to strings), and one could derive that the argument
to getitem is an int and its return value is a str, and so on; keys()
would return an Iterable of ints (or an IterableSet of ints, or
whatever), for example. But that's also for a later post.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Fri Nov 24 07:49:03 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 24 Nov 2006 07:49:03 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov23.103924pst."58648"@synergy1.parc.xerox.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2iig$gni$1@sea.gmane.org>
	<8853879015409334245@unknownmsgid>	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>	<06Nov22.181438pst."58648"@synergy1.parc.xerox.com>	<ek3kp8$19m$2@sea.gmane.org>
	<06Nov23.103924pst."58648"@synergy1.parc.xerox.com>
Message-ID: <ek64l3$10a$1@sea.gmane.org>

Bill Janssen wrote:

 > I think method namespaces are already OK; the question is
> whether to move these specially-named (for whatever reason)
 > methods to a real namespace (a dict somewhere)

where "somewhere" is exactly where?  how do I add things to
that namespace?  how do I specify what namespace to look in
when accessing methods or attributes?

</F>


From rrr at ronadam.com  Fri Nov 24 08:00:05 2006
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 24 Nov 2006 01:00:05 -0600
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <456631AF.1020208@canterbury.ac.nz>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>	<20061122165513.GA13795@niemeyer.net>	<4564F8B7.5030007@canterbury.ac.nz>	<20061123020054.GA31618@niemeyer.net>	<45651355.7090605@canterbury.ac.nz>	<20061123121416.GA6072@niemeyer.net>	<45661AA3.9070803@canterbury.ac.nz>	<20061123224759.GA27828@niemeyer.net>
	<456631AF.1020208@canterbury.ac.nz>
Message-ID: <ek65gu$2qg$1@sea.gmane.org>

Greg Ewing wrote:
> Gustavo Niemeyer wrote:
>> Are you arguing that
>> pretty much every approach being discussed is bad and
>> we should instead implement something magically
>> transforming types?
> 
> No, I'm questioning the need to do anything at all.
> I don't see a big problem that merits a heavyweight
> solution like a formal interface or adaptation
> framework.
> 
> --
> Greg

I think I agree with you Greg,  although it is still early and something simpler 
and possibly more direct may yet come from the discussion.  It does look like it 
is starting to be a bit more focused.

It might be nice to have some selected examples of current library code that may 
benefit from interfaces/abilities, with a brief comment of where it currently 
falls short for each example.  These may be useful later in a PEP.


Earlier messages here had sometimes vague suggestions of what interfaces would 
do, but it seems different people have different ideas of that.  Some of those 
implied ideas are:

[A probably incomplete list with letter for referencing]

     (A.) Better introspection to use in situations where type(), isinstance(), 
or hasattribute(), is perceived as being less than optimal.

     (B.) Adding an additional or alternate method of auto-dispatching either 
through generic functions or though the class structure.

     (C.) Selecting an Adapter, or using adapters.

     (D.) Verifying or Testing a function/object/or method to determine it's 
ability or signature. (Or some other variation of pragmatically determining a 
signature.)

I believe Guido already set aside discussion of "C" and "D", as they are 
implementation details that can be examined separately and added later if it 
they seem to be promising. (Guido may have other reasons.)

So that leaves "A" and "B". Most of the discussion is focused on "B" at the 
moment. The optional signatures proposed in a different thread may serve for 
"A".  I don't know how much "A" and "B" overlap, possibly only a little or not 
at all depending on how "B" is implemented, or it may be that a good solution to 
"A" is needed in order for "B" to work well enough to bother with.


It may also be that only a good unified solution to "A" is needed in order to 
allow library and user code to be written "easier" to address "B","C" and "D". 
It may even be just doing "A" is a good enough 95% solution (works well enough 
for 95% of actual use cases) and nothing more needs to be done. <It's too soon 
to tell>

There has also been put forth various, (also sometimes vague), suggestion of 
just what item "A" is.  Does "A" return information of "is_a", "has_a", or 
"does_a" relationships?  A "does_a" relationship may be difficult to define. 
Mixing these in a signature may not be as straight forward as it may seem.  The 
optional signatures addresses "is_a" relationships fairly well, but not "has_a" 
or "does_a" relationships.

Anyway, I'm waiting on a PEP and hoping I will be able to understand it.  A 
difficult to understand and use 99% solution is probably not better than what we 
currently have I think.

Cheers,
    Ron


From jimjjewett at gmail.com  Fri Nov 24 09:11:50 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 24 Nov 2006 03:11:50 -0500
Subject: [Python-3000] optional argument annotations
In-Reply-To: <ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
Message-ID: <fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>

On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> On 11/23/06, Tony Lownds <tony at pagedna.com> wrote:
> > I have a working optional argument syntax implementation, I'm hoping
> > to get some direction on
> > the implementation decisions so far.... please see below.

I would rather see it integrated into a Signature object (Brett had an
implementation), instead of showing up as two separate attributes.

> >  >>> f.func_returns
> > Traceback (most recent call last):
> >    File "<stdin>", line 1, in <module>
> > AttributeError: 'function' object has no attribute 'func_returns'

> I would prefer this to be None. Attributes that don't always exist are
> a pain to use.

I suspect he was trying to distinguish "returns None" from "Doesn't
say what it returns".  There is no good way to do this, but a separate
flag on a Signature object seems the least bad to me.

-jJ

From gustavo at niemeyer.net  Fri Nov 24 15:00:37 2006
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Fri, 24 Nov 2006 12:00:37 -0200
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <5.1.1.6.0.20061123153733.01f418e8@sparrow.telecommunity.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<5.1.1.6.0.20061123153733.01f418e8@sparrow.telecommunity.com>
Message-ID: <20061124140037.GA10468@niemeyer.net>

> Let's see your implementation.  Zope's interface system fails on at
> least points 3 and 5.  And regarding #1, Zope's implementation of the
> basic features I described is an order of magnitude more complex, even
> if you compare *just* its code that implements adaptation and
> interface inheritance, with *all* of my code plus Guido's generic
> function implementation.

Comparing complexity of implementations which offer completely
different end results isn't of great value.  Copying methods
from the interface to the class is trivial, as you're aware
of (it doesn't mean it's a good idea).  Support for partial
implementations may also be easily achieved, but its drawbacks
must be considered as well (how to deal with marker
interfaces, etc).

In any case, here is a silly implementation showing that it's
easily feasible without generic functions.  It's slow, since it
does linear searching, but there are several paths for speed
improvements.

    class InterfaceClass(type):
        def __init__(cls, name, bases, attrs):
            signature = set()
            for base in bases:
                signature.update(base.__signature__)
            for name, attr in attrs.items():
                if name not in ("__module__", "__metaclass__"):
                    if isinstance(attr, UnboundMethodType):
                        attr = attr.im_func
                    signature.add((name, attr))
            cls.__signature__ = frozenset(signature)
    
    class Interface:
        __metaclass__ = InterfaceClass
    
    class AdapterRegistry(object):
        def __init__(self):
            self._adapters = []
    
        def register(self, adapter, iorigin, itarget):
            self._adapters.append((adapter, iorigin.__signature__,
                                   itarget.__signature__))
    
        def adapt(self, obj, iface):
            origin = set()
            for obj_iface in obj.__interfaces__:
                origin.update(obj_iface.__signature__)
            target = iface.__signature__
            if origin.issuperset(target):
                return obj
            for adapter, aorigin, atarget in self._adapters:
                if origin.issuperset(aorigin) and target.issubset(atarget):
                    return adapter(obj)
            return None

That's it.

Here is a tested example:

    class C(Interface):
        def m1(): pass
    class D(Interface):
        m1 = C.m1
        def m2(): pass
    class E(Interface):
        def m3(): pass
    class F(E):
        def m4(): pass

    registry = AdapterRegistry()

    class Foo(object):
        __interfaces__ = [D]

    registry.register(lambda x: "Adapted!", C, E)
	
    foo = Foo()
    print registry.adapt(foo, E) # => "Adapted!"
    print registry.adapt(foo, D) # => foo
    print registry.adapt(foo, F) # => None


> 4. No need to understand interface or adaptation theory to be able
> to use it

Sure.. no need to understand concepts you're not using.  You have
to understand what you *are* using instead.

-- 
Gustavo Niemeyer
http://niemeyer.net

From python3000 at davious.org  Fri Nov 24 15:58:50 2006
From: python3000 at davious.org (Dave Anderson)
Date: Fri, 24 Nov 2006 09:58:50 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
Message-ID: <456708AA.7040907@davious.org>


on 11/24/2006 12:34 AM Guido van Rossum wrote:

> I would like to end up in a world where, in order to claim that a
> class is a standard mapping class, the class definition would
> (normally) explicitly claim to implement StandardMapping (using as yet
> unspecified syntax)

Guido,
In my world
http://mail.python.org/pipermail/python-3000/2006-November/004736.html
I feel have a usable syntax for you

class StandardMapping:
	...

class Foo:
	implements StandardMapping
	...

> and that there would be a way (again I'm not yet
> specifying syntax for this) to efficiently test any given class or
> instance for that interface, returning True or False.

Foo.does_implement(StandardMapping)
-> True
# based solely on the declaration of Foo implements StandardMapping

> The standard
> dict implementation would claim to implement StandardMapping (and
> probably also something that we could call StandardMutableMapping).

class StandardMutableMapping:
	implements StandardMapping
	...

dict.implements(StandardMutableMapping)

dict.does_implement(StandardMapping)
-> True
# based on dict's declaration of implementing StandardMutableMapping
# and StandardMutableMapping's declaration of implementing
# StandardMapping

> Someone could define an interface MinimalMapping as the interface
> formed of StandardMapping.getitem, StandardMapping.contains and
> StandardMapping.keys, 

MinimalMapping = interface(StandardMapping.getitem, 
StandardMapping.contains, StandardMapping.keys)

> and testing a dict for MinimalMapping would
> return True. 

dict.does_implement(MinimalMapping)
-> True

# Since dict has declared that it implements StandardMutableMapping
# and StandardMutableMapping has declared it implements StandardMapping
# so, introspection would stop there, since a class implementation
# declaration is pledging that all StandardMapping's methods are
# implemented

> But if someone created ExtendedMapping as StandardMapping
> plus the copy method and the eq operation, testing a dict for
> ExtendedMapping would return False; 

ExtendedMapping = interface(StandardMapping, "copy", "__eq__")

dict.does_implement(ExtendedMapping)
-> True
# According to the current implementation of dict
# "copy", "__eq__" are implemented by dict, looking at dir(dict)
# As for StandardMapping, the above declaration chain of dict implements
# StandardMutableMapping and then StandardMutableMapping implements
# StandardMapping then
# dict implements all of ExtendedMapping

> however there should be something
> one could execute to explicitly mark dict as implementing
> ExtendedMapping, using some kind of registry.

dict.implements(ExtendedMapping)

dict.does_implement(ExtendedMapping)
-> True
# Now, since I've explicitly claimed that dict implements
# ExtendedMapping, my introspection stops there.
# the previous introspection described just above has been made
# unnecessary






From tomerfiliba at gmail.com  Fri Nov 24 16:18:24 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Fri, 24 Nov 2006 17:18:24 +0200
Subject: [Python-3000] iostack and Oh Oh
Message-ID: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>

recap: we agreed py3k needs a unified, better organized and unbuffered new
io stack. i began working on the subject several months back, and we had
long discussions on what it's design.

you can see where we got at the following page (although not up to date):
http://sebulba.wikispaces.com/project+iostack+v2

i mostly neglected it in the past months, and got back to only now.
the problem is, i got confused with all the current OO/generics/interfaces
talks.

back in the day, we said all stream objects would support read() and
write(), and streams with limited functionality (readonly, writeonly) would
be "queriable" using the following properties:
* writable
* readable
* seekable

i.e.,
>>> f = FileStream("somefile.txt", "r")
>>> f.readable
True
>>> f.writable
False

but in light of the current discussion, maybe we'd prefer better a
java-like approach, separating the stream into readers and writers
over some fundamental "source".

i.e., a source would be a socket, file, StringIO, whatever, over which
we'll put a reader/writer layer. it may work well for uni-directional stream,
but for sockets it causes some overhead:
>>> s = socket.socket(...)
>>> sr = SocketReader(s)
>>> sw = SocketWriter(s)
>>> sr.read(5)
>>> sw.write("hello")

you have to construct both a writer an a reader to work properly with
a socket, and use a different object for different operations...
doesn't feel right to me.

also, it would be a huge headache to keep in sync the buffers of
both reader/writer, when a source is opened for both read and write...

- - - - - - - - -

we had another problem, iirc, namely "seeking-decoding problem":
what is the meaning of seeking in a character-stream over a file-stream?
the underlying file-stream works with bytes, while the text stream works
with characters of mixed widths...

* how do we handle unaligned-seeking?
* should tell() return "opaque cookies", instead of ints, so we could
  seek only to locations that tell() returned?
* still, if we modify the stream, these cookies need to be invalidated...

here's an example:
>>> f = TextAdapter(FileStream("somefile.txt", "w+"), "utf16")
>>> p0 = f.tell() # filepos = 0
>>> f.write("ABC")
>>> p1 = f.tell() # filepos = 3
>>> f.seek(p0)
>>> f.write(u"\uffff") # takes 4 bytes
>>> f.seek(p1)
>>> f.read(1) # in the middle of a character now...

there was a suggestion back in the day, to make streams sequential
by definition (i.e., no seek/tell) -- mostly like java. a StreamReader can
skip() a number of characters (essentially equivalent to reading and
discarding the result)

instead of seeking at the stream level, the "sources" would grow
indexing (getitem/setitem) for that. a seekable source (only files actually)
could be accessed directly:

>>> f = FileSource("somefile.txt", "r")
>>> f[10:20] = "hello worl"

i.e., the source acts like a sequence of bytes.

but of course sources should be unbuffered, which may prove problematic
when we have buffering added by streams on top of the source...  bladt

- - - - - - - - -

any suggestions? i'd guess the interfaces-supporters (Bill?) would prefer
a java-like stack (where subclasses of StreamReader provide reading
and subclasses of StreamWriter provide writing); duck-typing-supporters
and conservatives would probably prefer the succinct queiable approach
(streams support both read and write, but can be checked using the
relevant properties)

we ought to agree on the design here, if this is supposed to replace
the entire io stack... it may also be a good starting place for the supporters
of each camp to demonstrate how their approach "does it better".


tata,
- tomer

From guido at python.org  Fri Nov 24 17:30:12 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 24 Nov 2006 08:30:12 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
Message-ID: <ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>

On 11/24/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> > On 11/23/06, Tony Lownds <tony at pagedna.com> wrote:
> > > I have a working optional argument syntax implementation, I'm hoping
> > > to get some direction on
> > > the implementation decisions so far.... please see below.
>
> I would rather see it integrated into a Signature object (Brett had an
> implementation), instead of showing up as two separate attributes.

Yes, you're right; I forgot about that.

> > >  >>> f.func_returns
> > > Traceback (most recent call last):
> > >    File "<stdin>", line 1, in <module>
> > > AttributeError: 'function' object has no attribute 'func_returns'
>
> > I would prefer this to be None. Attributes that don't always exist are
> > a pain to use.
>
> I suspect he was trying to distinguish "returns None" from "Doesn't
> say what it returns".  There is no good way to do this, but a separate
> flag on a Signature object seems the least bad to me.

Hm, I think it would be fine if there *was* no distinction. IOW if

  def foo(a: None) -> None: pass

was indistinguishable from

  def foo(a): pass

In fact I think I'd prefer it that way. Having an explicit way to say
"no type here, move along" sounds fine. It's different from defaults,
whose absence translates in different behavior (i.e., requiring that a
value be passed in).

(It does imply that we can't use "-> None" to indicate "returns no
value". We'd have to spell that as "-> type(None)" or find some other
way. I think I don't care. Do others? Collin?)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Nov 24 18:19:16 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 24 Nov 2006 09:19:16 -0800
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
Message-ID: <ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>

I'd like to take the opportunity to present another example of what
I'd like to be able to do with abilities and generic functions. It's
short on implementation and I don't want to emphasize syntax, but it's
high on concepts. The baseline assumption is that most built-in
operations, especially binary operators, are implemented as generic
functions.

Let's suppose we didn't have a set type yet, and we wanted to introduce it.

I'd start by defining three abilities: MinimalSet which only provides
the 'in' operator; IterableSet which also provides iteration (but not
len(); it may aply to infinite sets); and StandardSet which adds most
standard operations on sets, but still not len(). The operations on
standard sets are: & (intersection), | (union), ^ (symmetric
difference), - (asymmetric difference), <= (is subset), >= (is
superset), < (strict subset), and > (strict superset). Sets also
implement == and !=, as these are inherited from an even more basic
ability (say, Object). Mutable operations are not part of this
ability.

One reason why I'm saying ability instead of interface is that not
every object that implements the above operations should automatically
be considered a set! If a class claims to be a set, it should conform
to a much richer contract, which guarantees the basic properties of
mathematical sets. For example, a&b == b&a, a&a == a, and so on. The
contract implies the existence of an empty set, and all empty sets
should compare equal (even if they have different underlying
implementations).

So StandardSet implies more than a bunch of operations! In particular,
subclassing int and adding implementations for __contains__ and
__iter__ should *not* result in that subclass having the StandardSet
ability (unless I explicitly declare it). Note: while this appears
incompatible with Phillip's proposal "interfaces are just collections
of operations", I think this can be consolidated easily by allowing
some kind of "marker" operation.

Now I'd like to introduce a standard set implementation. It implements
all those operations. But I'd like it to automatically accept operands
of other set implementations, as long as they claim the relevant
ability. For example, in Python 2.4, the built-in set implementation's
__and__ is defined roughly like this (ignoring some optimizations,
like that it's written in C :-):

# This a method of class 'set', a concrete set implementation

def __and__(self, other):
  if not isinstance(other, baseset):
    return NotImplemented
  result = set()
  for elem in self:
    if elem in other:
      result.add(elem)
  return result

But I'd like to be able to write it like this instead (still inside
the 'set' class):

def __and__(self, other: MinimalSet):
  result = set()
  for elem in self:
    if elem in other:
      result.add(elem)
  return result

which means that it automatically interoperates with other set implementations.

The header "def __and__(self, other: MinimalSet)" is meant to be
enough syntax to declare an implementation of the pre-existing __and__
operations for the pair (set, MinimalSet) -- the first being a
concrete type (class), the second being an ability.

Actually, I might want to define three implementations of __and__: one
for the pair (set, set), which could be optimized based on knowledge
of the internals; one for (set, MinimalSet); and one for (MinimalSet,
set). The exact syntax for all this is not that important; the latter
one may have to be defined outside the class as a true generic
function; or defining a method __radd__ could do the trick; or even
aliasing __radd__ to __add__ (like we would do today).

I believe this example captures an important requirement that has been
expressed both by Bill Janssen and by Andrew Koenig (needless to say I
agree): abilities should be able to imply contracts that go beyond a
collection of methods (without necessarily being able to express those
contracts in code).

The example doesn't show a way to test whether an object has an
ability, but I certainly want to make that possible and even
efficient; maybe isinstance(x, A) could be made to work (it's already
pretty flexible under the hood, and could easily be turned into a GF
itself), or maybe you'll have to write has_ability(x, A).

Note that I haven't shown syntax for declaring the various abilities.
That is because I don't have a concrete proposal yet. However, I would
prefer syntax that looks declarative (similar to a class declaration)
for the normal case. Syntax that looks procedural (e.g. "IterableSet =
MinimalSet + iter") might be available as an alternative way to create
abilities more dynamically, just like in a pinch we can create a new
class today by calling type(name, bases, classdict).

I still don't have new ideas for how to add regular-looking methods
(like keys()) to an ability.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Fri Nov 24 18:53:07 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri, 24 Nov 2006 12:53:07 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
Message-ID: <001601c70ff1$663f54f0$6402a8c0@arkdesktop>

> I believe this example captures an important requirement that has been
> expressed both by Bill Janssen and by Andrew Koenig (needless to say I
> agree): abilities should be able to imply contracts that go beyond a
> collection of methods (without necessarily being able to express those
> contracts in code).

Yes.  For that matter, I would like an ability to be able to say things
about a + b regardless of whether it happens to be implemented through
__add__ or __radd__ -- which is yet another reason why abilities need to be
more than just collections of methods.

> Note that I haven't shown syntax for declaring the various abilities.
> That is because I don't have a concrete proposal yet. However, I would
> prefer syntax that looks declarative (similar to a class declaration)
> for the normal case.

A thought borrowed from ML:  You wrote

	def __and__(self, other: MinimalSet):

and that reminds me of ML's syntax:

	fun f(x, y: MininmalSet) = (* whatever *)

In the ML version, we're saying that f's first argument can be of any type
(that meets whatever other type constraints are implied by f's body, and the
second argument must be of type MinimalSet in addition to those constraints.

Seeing this analogy, I am tempted to carry it further.  In ML, I can write

	val foo = bar: MinimalSet

which binds foo to the value of bar, and also implies a (compile-time) type
check that bar has type MinimalSet.  So consider extending that analogy to
Python:

	foo = bar: MinimalSet

which would verify (at run time) that bar has the MinimalSet ability, and
then bind foo to the same object as bar.  If bar doesn't have the MinimalSet
ability, this would do whatever would have happened if you tried to call the
__and__ function above -- probably raise an exception.



From tony at pagedna.com  Fri Nov 24 18:59:03 2006
From: tony at pagedna.com (Tony Lownds)
Date: Fri, 24 Nov 2006 09:59:03 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
Message-ID: <99B2A3E4-1226-45D5-BC0D-DF72890A8D54@pagedna.com>


On Nov 24, 2006, at 8:30 AM, Guido van Rossum wrote:

>> I would rather see it integrated into a Signature object (Brett  
>> had an
>> implementation), instead of showing up as two separate attributes.
>
> Yes, you're right; I forgot about that.
>

I'd imagine that the Signature object would live on top of this change.

>>>>>>> f.func_returns
>>>> Traceback (most recent call last):
>>>>    File "<stdin>", line 1, in <module>
>>>> AttributeError: 'function' object has no attribute 'func_returns'
>>
>>> I would prefer this to be None. Attributes that don't always  
>>> exist are
>>> a pain to use.
>>
>> I suspect he was trying to distinguish "returns None" from "Doesn't
>> say what it returns".  There is no good way to do this, but a  
>> separate
>> flag on a Signature object seems the least bad to me.
>
> Hm, I think it would be fine if there *was* no distinction. IOW if
>
>   def foo(a: None) -> None: pass
>
> was indistinguishable from
>
>   def foo(a): pass
>
> In fact I think I'd prefer it that way. Having an explicit way to say
> "no type here, move along" sounds fine. It's different from defaults,
> whose absence translates in different behavior (i.e., requiring that a
> value be passed in).
>
> (It does imply that we can't use "-> None" to indicate "returns no
> value". We'd have to spell that as "-> type(None)" or find some other
> way. I think I don't care. Do others? Collin?)
>

That's fine with me, but just to toss out another idea: we could put the
return annotation in func_annotations['return'].

-Tony



From tony at pagedna.com  Fri Nov 24 19:00:48 2006
From: tony at pagedna.com (Tony Lownds)
Date: Fri, 24 Nov 2006 10:00:48 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
Message-ID: <523FAB43-7859-437B-9A80-167F14A5291D@pagedna.com>

>>>>> f.func_annotations
>> {'x': 1}
>
> Very cool! I agree that Phillip's idea for simplification is worth a
> try. We're generally not too concerned over the cost of function
> declarations since they typically execute only once per program. As
> long as it's really cheap when no annotations are present (which would
> suggest that func_annotations should be None in that case since an
> empty dict is kind of expensive, at least in memory).
>
>

func_annotations could create the empty dict just in time.

Building the annotations dict in bytecode actually takes slightly  
more C code
and quite a bit more bytecode. Maybe using dict(zip(<tuple from  
co_consts>, <values from stack>)) would
be more efficient. I'll probably keep both the MAKE_FUNCTION change  
and the bytecode version for a while.

The next step is changing the syntax for nested parameters, eg

def f((x:1, y:2))

 >>> dis.dis(compile("def f(x:1, y:2=3, z=4)->5:pass", "<str>", "exec"))
   1           0 LOAD_CONST               0 (3)
               3 LOAD_CONST               1 (4)
               6 LOAD_CONST               2 (<code object f at  
0xb7ec70b0, file "<str>", line 1>)
               9 MAKE_FUNCTION            2
              12 DUP_TOP
              13 BUILD_MAP                0
              16 DUP_TOP
              17 LOAD_CONST               3 (1)
              20 ROT_TWO
              21 LOAD_CONST               4 ('x')
              24 STORE_SUBSCR
              25 DUP_TOP
              26 LOAD_CONST               5 (2)
              29 ROT_TWO
              30 LOAD_CONST               6 ('y')
              33 STORE_SUBSCR
              34 ROT_TWO
              35 STORE_ATTR               0 (func_annotations)
              38 DUP_TOP
              39 LOAD_CONST               7 (5)
              42 ROT_TWO
              43 STORE_ATTR               1 (func_returns)
              46 STORE_NAME               2 (f)
              49 LOAD_CONST               8 (None)
              52 RETURN_VALUE

>>> dis.dis(compile("def f(x:1, y:2=3, z=4)->5:pass", "<str>", "exec"))
>>>
    1           0 LOAD_CONST               0 (3)         # default for y
                3 LOAD_CONST               1 (4)          # default  
for z
                6 LOAD_CONST               2 (1)          # annotation
for x
                9 LOAD_CONST               3 (2)          # annotation
for y
               12 LOAD_CONST               4 (6)         # bitmask for
annotations... 0b110
               15 LOAD_CONST               5 (5)         # func_returns
               18 LOAD_CONST               6 (<code object f at
0xb7f08848, file "<str>", line 1>)
               21 MAKE_FUNCTION        49154      # numdefaults(2) +
kwdefaults(0) << 8 + has_annotations(1) << 14 + has_returns(1) << 15
               24 STORE_NAME               0 (f)
               27 LOAD_CONST               7 (None)
               30 RETURN_VALUE




From brett at python.org  Fri Nov 24 20:20:39 2006
From: brett at python.org (Brett Cannon)
Date: Fri, 24 Nov 2006 11:20:39 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
Message-ID: <bbaeab100611241120j6b8ea9aka67b28b7ed4e63cc@mail.gmail.com>

On 11/24/06, Guido van Rossum <guido at python.org> wrote:
> On 11/24/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> > > On 11/23/06, Tony Lownds <tony at pagedna.com> wrote:
> > > > I have a working optional argument syntax implementation, I'm hoping
> > > > to get some direction on
> > > > the implementation decisions so far.... please see below.
> >
> > I would rather see it integrated into a Signature object (Brett had an
> > implementation), instead of showing up as two separate attributes.
>
> Yes, you're right; I forgot about that.
>

Guess I need to make sure to bug you about pronouncing on the PEP once
I get keyword-only arguments support added before you forget again.
=)

Obviously signature objects would grow support for annotations, but I
still need the information to be carried on the code object to
incorporate into signature objects.

Another option for how to handle annotations is to take the route of
having a tuple listing all of the strings representing the
annotations, using None for arguments that have no annotation.  That
way there is no issue distinguishing whether an annotation exists or
not plus it is a cheap operation since you are then just storing
strings.  This also allows the last value in the tuple to imply the
return annotation and thus only have a single attribute for
annotations.  Then you can have signature objects worry about pulling
out the info to tie into specific parameters to work with the
annotations.

> > > >  >>> f.func_returns
> > > > Traceback (most recent call last):
> > > >    File "<stdin>", line 1, in <module>
> > > > AttributeError: 'function' object has no attribute 'func_returns'
> >
> > > I would prefer this to be None. Attributes that don't always exist are
> > > a pain to use.
> >
> > I suspect he was trying to distinguish "returns None" from "Doesn't
> > say what it returns".  There is no good way to do this, but a separate
> > flag on a Signature object seems the least bad to me.
>
> Hm, I think it would be fine if there *was* no distinction. IOW if
>
>   def foo(a: None) -> None: pass
>
> was indistinguishable from
>
>   def foo(a): pass
>
> In fact I think I'd prefer it that way. Having an explicit way to say
> "no type here, move along" sounds fine. It's different from defaults,
> whose absence translates in different behavior (i.e., requiring that a
> value be passed in).
>

This also mirrors the default way that functions work since they
return None if you don't bother to specify somethng.

> (It does imply that we can't use "-> None" to indicate "returns no
> value". We'd have to spell that as "-> type(None)" or find some other
> way. I think I don't care. Do others? Collin?)
>

I always figured we would return type.

-Brett

From ark-mlist at att.net  Fri Nov 24 20:42:31 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Fri, 24 Nov 2006 14:42:31 -0500
Subject: [Python-3000] optional argument annotations
In-Reply-To: <99B2A3E4-1226-45D5-BC0D-DF72890A8D54@pagedna.com>
Message-ID: <000a01c71000$ae9a7130$6402a8c0@arkdesktop>

> Hm, I think it would be fine if there *was* no distinction. IOW if
>
>   def foo(a: None) -> None: pass
>
> was indistinguishable from
>
>   def foo(a): pass
>
> In fact I think I'd prefer it that way. Having an explicit way to say
> "no type here, move along" sounds fine.

I'd like to urge against making None magical -- I don't see any reason for
it.

Instead, consider that if I say

	def foo(a: T1) -> T2:

I am presumably going to accept an argument of type T1 or a type derived
from T1, and return a result of type T2 or a type derived from T2.  In which
case,

	def foo(a): pass

should be equivalent to

	def foo(a: object) -> object: pass

and we don't need to recycle None for the purpose -- especially as
ininstance(None,object) yields True and I presume that isn't going to
change.



From python3000 at davious.org  Fri Nov 24 22:15:24 2006
From: python3000 at davious.org (Dave Anderson)
Date: Fri, 24 Nov 2006 16:15:24 -0500
Subject: [Python-3000] My poor-man's non-oo implementation of my
 proposed interface and implements/implement syntax
In-Reply-To: <ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
Message-ID: <456760EC.6070503@davious.org>

class Interface(object):
	""" A hashable, type-identifiable 'list'
	
	The list should contain only other Interface objects
	, Classes/Types, or Class Methods/Attributes, and strings

	Interface objects can be used in specifying dispatch rules
	and function parameter "types"
	"""
	def __init__(self, *args):
		self.interfacelist = list(args)
	
class Implementation(dict):
	"""This is used just to indicate that implementation sets are
	dicts

	Reasoning is to allow for adaptation on the class or method 	
	level by a class reference pointing to a transform method
	or a class method reference pointing to a method definition
	(this is not implemented, though, and would be dependent on
	smart dispatching)
	"""
	pass

def implements(class_, *interface_objects):
	"""Declares that a class implements one or more interface
	objects
	
	See Interface class's description for allowed items
	"""
	for interface in interface_objects:
	
		# the implementation slot in a real implementation
		# would allows be an object attribute
		if not class_.__dict__.has_key('implementation'):
			class_.implementation = Implementation()
			
		# the class vouches to implement an Interface list			
		if isinstance(interface, Interface):
			class_.implementation[interface] = class_			
			
		# the class vouches to implement a class/type			
		if isinstance(interface, type):
			class_.implementation[interface] = class_
			
		# the class vouchers to implement a
		# non-class-specific method/attribute 			
		elif isinstance(interface, str):
			class_.implementation[interface] = interface
			
		# the class vouchers to implement a
		# class-specific method/attribute
		elif "__name__" in dir(interface):
			# this is a class-specific method:
			class_.implementation[interface] = \
			interface.__name__
			
		else:
			raise TypeError, interface, \
			 "must be an Interface, Class/Type, Class Method/Attribute, or string"

		
def implement(class_, obj, adaptor_or_method):
	"""a special adaption case for declaring an implementation
	
	   upon dispatch the adaptor_or_method value
	   would be used depending on context
	   class context:
	    apply adaptor_or_method to class instance to adapt
	   method context: use adaptor_or_method in the call
	"""
	if not class_.__dict__.has_key('implementation'):
		class_.implementation = Implementation()
	class_.implementation[obj] = adaptor_or_method


def does_implement(class_or_instance, *interface_objects):
	"""returns True or False depending on whether the class
	   implements

	   the list of interface objects
	"""

	# interface_objects can be an Interface object, a class, or
	# a class method/attribute
	
	class_ = class_or_instance
	if not isinstance(class_, type) and \
            not isinstance(class_, Interface):
		class_ = class_.__class__
	
	for interface in interface_objects:
	# interface is either a class/type or class method/attribute
	# or a string representation some name of a method or attribute
		
		# self or inherited classes are implemented
		if isinstance(interface, type) and \
                    isinstance(class_, interface):
			continue

		# interface is declared to be implemented
		# (interface is either a class or class
		# method/attribute)
		elif class_.__dict__.has_key('implementation') \
		 and interface in class_.implementation:
			continue

		# if this is a real interface
		# check each of the interfaces specified in it		
		elif isinstance(interface, Interface):
			# check the items within the interface
			if does_implement(class_,
                                           *interface.interfacelist):
				continue				

		# check non-class-specific method/attribute interfaces
		elif isinstance(interface, str) and \
                      interface in dir(class_):
			continue
				
		# check to see if the class is implemented
		# if a class specific the methods has not been declared
		elif 'im_class' in dir(interface) \
		 and does_implement(class_, interface.im_class):
			continue
			
		# recursive base implementations check
		elif class_.__dict__.has_key('implementation'):
			base_implementation = True
			for implementation in class_.implementation:
				if does_implement(implementation,
							interface):
					break
			else:	
				base_implementation = False
			
			if base_implementation:
				continue			

		return False

	return True


def test_prototype():
	"""Tests implementation of Guido's desired goals per bottom of
	http://mail.python.org/pipermail/python-3000/2006-November/004774.html
	"""
	class StandardMapping(object):
		def __getitem__(self):
			pass
		def __contains__(self):
			pass
		def keys(self):
			pass

	class Foo(object):
		pass
		
	implements(Foo, StandardMapping)
	
	print "does_implement(Foo, StandardMapping):",
	print does_implement(Foo, StandardMapping)
	print
	
	class StandardMutableMapping(object):
		pass
		
	implements(StandardMutableMapping, StandardMapping)	

	class dict_(dict):
		pass
	
	implements(dict_, StandardMutableMapping)
		
	print "does_implement(dict_, StandardMapping):",
	print does_implement(dict_, StandardMapping)
	print
	
	MinimalMapping = Interface(StandardMapping.__getitem__,
StandardMapping.__contains__, StandardMapping.keys)

	print "does_implement(dict_, MinimalMapping):",
	print does_implement(dict_, MinimalMapping)
	print
	
	ExtendedMapping = Interface(StandardMapping, "copy", "__eq__")	
	
	print "does_implement(dict_, ExtendedMapping):",
	print does_implement(dict_, ExtendedMapping)
	print
	
	implements(dict_, ExtendedMapping)	

	print "does_implement(dict_, ExtendedMapping):",
	print does_implement(dict_, ExtendedMapping)	
	print
	

From greg.ewing at canterbury.ac.nz  Fri Nov 24 23:47:46 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 25 Nov 2006 11:47:46 +1300
Subject: [Python-3000] optional argument annotations
In-Reply-To: <000a01c71000$ae9a7130$6402a8c0@arkdesktop>
References: <000a01c71000$ae9a7130$6402a8c0@arkdesktop>
Message-ID: <45677692.103@canterbury.ac.nz>

Andrew Koenig wrote:
> 	def foo(a: T1) -> T2:
> 
> I am presumably going to accept an argument of type T1 or a type derived
> from T1,

You're making assumptions about the semantics of the
annotations here, which I thought wasn't going to be
done.

--
Greg

From greg.ewing at canterbury.ac.nz  Fri Nov 24 23:48:54 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 25 Nov 2006 11:48:54 +1300
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
Message-ID: <456776D6.8020909@canterbury.ac.nz>

tomer filiba wrote:

> back in the day, we said all stream objects would support read() and
> write(), and streams with limited functionality (readonly, writeonly) would
> be "queriable" using the following properties:
> * writable
> * readable
> * seekable

Is that really necessary? I can't remember ever writing
code that didn't already know what kind of stream it
was expecting.

--
Greg

From guido at python.org  Fri Nov 24 23:55:31 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 24 Nov 2006 14:55:31 -0800
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <456776D6.8020909@canterbury.ac.nz>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
Message-ID: <ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>

On 11/24/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> tomer filiba wrote:
>
> > back in the day, we said all stream objects would support read() and
> > write(), and streams with limited functionality (readonly, writeonly) would
> > be "queriable" using the following properties:
> > * writable
> > * readable
> > * seekable
>
> Is that really necessary? I can't remember ever writing
> code that didn't already know what kind of stream it
> was expecting.

Agreed that for the distinction between readable/writable it's pretty
silly, and probably just encourages LBYL code ("if not f.readable:
raise IOError;; f.read(...)" :-). But I've written plenty of code that
could make good use of seekability but was able to do without it if
necessary, and I suspect I'm not the only one.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Nov 25 00:18:27 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 24 Nov 2006 15:18:27 -0800
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
	<ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
Message-ID: <ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>

[With apologies for having a monologue with myself here.]

On 11/24/06, I wrote:
> I'd like to take the opportunity to present another example of what
> I'd like to be able to do with abilities and generic functions.
[...]

Thinking about my own example some more, I think it would actually
work just as well using ABCs instead of abilities. (The GFs are still
needed.) It seems the use case for making abilities/interfaces
separate from ABCs hinges on the assumption that there are classes
that implement certain protocols without realizing it, IOW emergent
protocols, plus that it's not practical to change __bases__. This
seems to be exacerbated by there not being any standard
abilities/interfaces defined, while some of the standard types are
just *begging* to be classified by a container hierarchy. Plus, the
standard types won't let you alter their __bases__ at all.

I wonder if a bunch of well thought-out standard ABCs, applied to the
standard data types, and perhaps more support for setting __bases__,
wouldn't address most concerns.

Note: I still like GFs for operations, especially binary operators,
because they solve the problem of two-sided dispatch much better than
__add__ and __radd__ ever can. (For unary operators I'm not sure it
buys us that much; I'm pretty happy with __len__ and __iter__.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tony at pagedna.com  Sat Nov 25 03:19:40 2006
From: tony at pagedna.com (Tony Lownds)
Date: Fri, 24 Nov 2006 18:19:40 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <bbaeab100611241120j6b8ea9aka67b28b7ed4e63cc@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
	<bbaeab100611241120j6b8ea9aka67b28b7ed4e63cc@mail.gmail.com>
Message-ID: <C0D2AD09-FC51-432C-87EE-BC91E8FE99D8@pagedna.com>

> Obviously signature objects would grow support for annotations, but I
> still need the information to be carried on the code object to
> incorporate into signature objects.
>

Signature objects still need a way to know the nested parameters, right?
How about a co_argnames attribute? eg for

def f((x, y), z): pass

f.func_code.co_argnames would be (('x', 'y'), 'z')

I need to implement something like this to properly build  
func_annotations
inside MAKE_FUNCTION.

-Tony




From ironfroggy at gmail.com  Sat Nov 25 03:35:24 2006
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Fri, 24 Nov 2006 21:35:24 -0500
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<55071.62.39.9.251.1164201819.squirrel@webmail.nerim.net>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
	<ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
Message-ID: <76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>

On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> On 11/22/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > > Not sure I like this better. My spelling of "operator::XXX" is
> > > "__XXX__" and I like that just fine -- no new syntax needed.
> >
> > Well, that might be a spelling of `operator::XXX` but what about other
> > use cases like `third_party_interface_system::adapt` or
> > `anything_at_all::XXX`? Thats what I mean with not being too limiting
> > and solving the problem in a way that opens up solutions to other
> > problems. I get the opposition to it, but it seems reasonable,
> > nonetheless.
>
> How was I to generalize from a single example what you meant? I
> thought you were using 'operator' as a fixed keyword like it is in
> C++, and I was wondering what you meant by the double colon.
>
> I still believe it is easy enough to solve this using a naming convention.

Sorry, I meant it in the context of this thread, where the previous
examples where using the operator module (which, by virtue of
existing, should suggest that I wasn't implying its name for use in
some other way).

Naming conventions will only take us so far before we run into
conflicts more often than we can offord. We only have one convention
for special names, really, and it will run up its invitation with or
without a suitable solution around its limitations. I understand the
associtiation of this with "Too Much Magic" or simply the idea that
its more than is needed, overkill, etc. and I share that warriness.
But its something to consider, I suppose. I guess the thing to
consider is simply, "What's in a name?"

Another suggestion for this proposal:

class MyList(object):
    def __builtins__::len(self):
        return 0

This would basically say that MyList has a method identified by the
len attribute of __builtins__. Does that make sense? Maybe the syntax
isn't enjoyable but the effect is more the value I see, so are there
other ways it could be expressed?

From collinw at gmail.com  Sat Nov 25 04:12:35 2006
From: collinw at gmail.com (Collin Winter)
Date: Fri, 24 Nov 2006 21:12:35 -0600
Subject: [Python-3000] optional argument annotations
In-Reply-To: <99B2A3E4-1226-45D5-BC0D-DF72890A8D54@pagedna.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
	<99B2A3E4-1226-45D5-BC0D-DF72890A8D54@pagedna.com>
Message-ID: <43aa6ff70611241912j2afa8894g1d583aaca287f75b@mail.gmail.com>

On 11/24/06, Tony Lownds <tony at pagedna.com> wrote:
> On Nov 24, 2006, at 8:30 AM, Guido van Rossum wrote:
> > (It does imply that we can't use "-> None" to indicate "returns no
> > value". We'd have to spell that as "-> type(None)" or find some other
> > way. I think I don't care. Do others? Collin?)
>
> That's fine with me, but just to toss out another idea: we could put the
> return annotation in func_annotations['return'].

This is the solution I had initially envisioned, for exactly the
reason Guido outlines.

Collin Winter

From guido at python.org  Sat Nov 25 06:29:49 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 24 Nov 2006 21:29:49 -0800
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<456456CF.8000005@gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
	<ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
	<76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>
Message-ID: <ca471dc20611242129na1f48d5o2efb1983139c1e23@mail.gmail.com>

Hm. The double colon rubs me the wrong way (Perl and/or C++). But
apart from that, if this is the solution, I'm not sure the problem
you're trying to solve is really worth solving. I just don't expect
there will be all that many generic operations that need to be stuffed
into arbitrary classes. Maybe I'll just take back what I said about
wanting all such operations to be inside the class. Or maybe I'd be
happier if there was a decorator indicating the name of the operation.

Also, I think you're overloading 'def' in a non-obvious way.
Currently, "def foo..." means an assignment to the local variable foo.
I would expect that if we extend the syntax for the thing between
'def' and the argument list to be more than just a name, it should
still be considered an assignment target. But that's clearly not what
you're after. From that POV, I find defop (while still unpleasant for
other reasons) more "honest" than your overloading of def -- at least
defop says upfront that it's not just an assignment. (Although the
similarity with def is still confusing IMO.)

Still not convinced? Focus on other problems first. This isn't the
most important problem we're trying to solve.

PS, It's __builtin__, not __builtins__ -- the latter is an
unfortunately named but ultimately unimportant implementation detail
(unimportant unless you're implemented restricted python, that is);
the former is the module that is implied at the end of every free name
search.


--Guido

On 11/24/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> On 11/23/06, Guido van Rossum <guido at python.org> wrote:
> > On 11/22/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > > On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> > > > Not sure I like this better. My spelling of "operator::XXX" is
> > > > "__XXX__" and I like that just fine -- no new syntax needed.
> > >
> > > Well, that might be a spelling of `operator::XXX` but what about other
> > > use cases like `third_party_interface_system::adapt` or
> > > `anything_at_all::XXX`? Thats what I mean with not being too limiting
> > > and solving the problem in a way that opens up solutions to other
> > > problems. I get the opposition to it, but it seems reasonable,
> > > nonetheless.
> >
> > How was I to generalize from a single example what you meant? I
> > thought you were using 'operator' as a fixed keyword like it is in
> > C++, and I was wondering what you meant by the double colon.
> >
> > I still believe it is easy enough to solve this using a naming convention.
>
> Sorry, I meant it in the context of this thread, where the previous
> examples where using the operator module (which, by virtue of
> existing, should suggest that I wasn't implying its name for use in
> some other way).
>
> Naming conventions will only take us so far before we run into
> conflicts more often than we can offord. We only have one convention
> for special names, really, and it will run up its invitation with or
> without a suitable solution around its limitations. I understand the
> associtiation of this with "Too Much Magic" or simply the idea that
> its more than is needed, overkill, etc. and I share that warriness.
> But its something to consider, I suppose. I guess the thing to
> consider is simply, "What's in a name?"
>
> Another suggestion for this proposal:
>
> class MyList(object):
>     def __builtins__::len(self):
>         return 0
>
> This would basically say that MyList has a method identified by the
> len attribute of __builtins__. Does that make sense? Maybe the syntax
> isn't enjoyable but the effect is more the value I see, so are there
> other ways it could be expressed?
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From kay.schluehr at gmx.net  Sat Nov 25 19:56:31 2006
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sat, 25 Nov 2006 19:56:31 +0100
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <456631AF.1020208@canterbury.ac.nz>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>	<20061122165513.GA13795@niemeyer.net>	<4564F8B7.5030007@canterbury.ac.nz>	<20061123020054.GA31618@niemeyer.net>	<45651355.7090605@canterbury.ac.nz>	<20061123121416.GA6072@niemeyer.net>	<45661AA3.9070803@canterbury.ac.nz>	<20061123224759.GA27828@niemeyer.net>
	<456631AF.1020208@canterbury.ac.nz>
Message-ID: <456891DF.3060609@gmx.net>

Greg Ewing schrieb:

>Gustavo Niemeyer wrote:
>  
>
>>Are you arguing that
>>pretty much every approach being discussed is bad and
>>we should instead implement something magically
>>transforming types?
>>    
>>
>
>No, I'm questioning the need to do anything at all.
>I don't see a big problem that merits a heavyweight
>solution like a formal interface or adaptation
>framework.
>
>--
>Greg
>_______________________________________________
>Python-3000 mailing list
>Python-3000 at python.org
>http://mail.python.org/mailman/listinfo/python-3000
>Unsubscribe: http://mail.python.org/mailman/options/python-3000/kay.schluehr%40gmx.net
>  
>
Seems like we need more formal procedures for writing test than 
application code. This also solves the problem of expense of 
"verification" of typeclasses/concepts/abilities that need to be done 
just once for a class with a fixed interface and not each time when the 
class gets loaded from a module. The value of "test time" is increased 
to a higher status not unlike the role of "compile time" for static 
analysis in languages with default type systems. But maybe my head is 
just too far in the clouds when I'm thinking about the end of the 
static/dynamic language distinction using type reconstruction at "test 
time". Heavyweight frameworks naturally emerge also in Python but being 
latent. Reconstructing them from runtime information is a challenge for 
dynamically typed languages. A good enough approximation can be turned 
into a set of prescriptions. This is not much different from the way 
natural scientists frame reality according to Karl Poppers "logic of 
progress" ( hypothetical deductive method ). Regular enough data are 
used to form a hypothesis about the future behaviour of a system. This 
hypothesis is used to frame reality in a mathematical description - just 
like type systems. Making new experiments / measurents can falsify a yet 
established deductive system which leads to improved frames ( reframing 
) and progressive understanding of systems properties. The role of tests 
in this approach is twofold: they check application code and keep 
information about the "true nature" of the programs behaviour using 
measurements which leads to automatical generation of a formal 
hypothesis of a systems behaviour that may become prescriptive i.e. 
forms a type system:

Note that I've not yet strong indications that a Popperian methodology 
really works for any kind of language including Python i.e. I have 
neither worked out a formal proof nor do I have a working system ( not 
to mention treatments of "singularities" ) But I believe enough in this 
idea for questioning the value of making "Python more Java like" which 
seems to be the main intention of the interface/ability/optional type 
declaration proposals discussed here.

Kay




From tomerfiliba at gmail.com  Sat Nov 25 20:32:44 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 25 Nov 2006 21:32:44 +0200
Subject: [Python-3000] two things
Message-ID: <1d85506f0611251132q5325b36cn4570f5bbe403484@mail.gmail.com>

i'd suggest using the term "contract" instead of abilities or interfaces.
they way BDFL described it [1], abilities specify more than mere method
signatures -- they go as deep as how the implementation should implement
the desired behavior.

http://mail.python.org/pipermail/python-3000/2006-November/004782.html
> I'd start by defining three abilities: MinimalSet which only provides
> the 'in' operator; IterableSet which also provides iteration (but not
> len(); it may aply to infinite sets); and StandardSet which adds most
> standard operations on sets
[...]
> If a class claims to be a set, it should conform
> to a much richer contract, which guarantees the basic properties of
> mathematical sets. For example, a&b == b&a, a&a == a, and so on.

"contract" is a better term, IMO, since it's already used in CS (as in Eiffel),
and describes the situation more correctly: *behavior* rather than *signature*.
"ability" just doesn't seem right to me: my class is not *able* to be a set,
it *behaves* like a set. it follows the set contract, not "ability"

- - - - -

instead of the suggested /defop/ keyword, which seems very awkward to me,
i'd prefer an extension to the current function syntax:

def __contains__(self) of MinimalSet:
    pass

if we don't want to make "of" a keyword, we can use "in",
with a similar meaning.

this syntax is inspired by (god forbid) visual basic .net:
http://msdn2.microsoft.com/en-us/library/sect4ck6.aspx

Class Spam
    Implements IFooable

    Sub Foo(ByVal x as Integer) As String Implements IFooable.SomeMethod
        ...
    End Sub
End Class

the suggested syntax of
defop MinimalSet::__contains___(self, other):
    pass

starts to look like C++ to me (yuch), and it's not necessarily an
"operator", as *defop* implies.


-tomer

From solipsis at pitrou.net  Sat Nov 25 20:56:32 2006
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 25 Nov 2006 20:56:32 +0100
Subject: [Python-3000] two things
In-Reply-To: <1d85506f0611251132q5325b36cn4570f5bbe403484@mail.gmail.com>
References: <1d85506f0611251132q5325b36cn4570f5bbe403484@mail.gmail.com>
Message-ID: <1164484592.5176.27.camel@fsol>


Le samedi 25 novembre 2006 ? 21:32 +0200, tomer filiba a ?crit :
> "contract" is a better term, IMO, since it's already used in CS (as in Eiffel),
> and describes the situation more correctly: *behavior* rather than *signature*.
> "ability" just doesn't seem right to me: my class is not *able* to be a set,
> it *behaves* like a set.

You can simply call it a behaviour rather than a contract.

Contract-based programming is usually defined as involving pre- and
post-conditions on method calls, to check that the method behaves as
expected. The underlying idea is that the terms of the contract are
programmatically enforced rather than simply declared.

Wikipedia mentions those attempts at contract-based programming with
Python:
http://www.nongnu.org/pydbc/
http://www.wayforward.net/pycontract/
http://www.targeted.org/python/recipes/ipdbc.py

> instead of the suggested /defop/ keyword, which seems very awkward to me,
> i'd prefer an extension to the current function syntax:
> 
> def __contains__(self) of MinimalSet:
>     pass

Other suggestion:

from MinimalSet def __contains__(self):
    pass

The "origin" of the method is then clearly visible instead of being
relegated at the end of the declaration (it works better with multi-line
declarations too). Also the class declaration look nicer when you have
several of these methods:

class BazContainer:
    # Implement methods from MinimalSet

    from MinimalSet def __contains__(self):
        pass

    from MinimalSet def add(self):
        pass

    from MinimalSet def remove(self):
        pass

    # Other methods

    def to_bicycle(self):
        """ Turn this container into a bicycle """

    (etc.)




From tjreedy at udel.edu  Sat Nov 25 20:59:10 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 25 Nov 2006 14:59:10 -0500
Subject: [Python-3000] Abilities / Interfaces
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>	<20061122165513.GA13795@niemeyer.net>	<4564F8B7.5030007@canterbury.ac.nz>	<20061123020054.GA31618@niemeyer.net>	<45651355.7090605@canterbury.ac.nz>	<20061123121416.GA6072@niemeyer.net>	<45661AA3.9070803@canterbury.ac.nz>	<20061123224759.GA27828@niemeyer.net><456631AF.1020208@canterbury.ac.nz>
	<456891DF.3060609@gmx.net>
Message-ID: <eka7ae$rbt$1@sea.gmane.org>


"Kay Schluehr" <kay.schluehr at gmx.net> wrote in message 
news:456891DF.3060609 at gmx.net...
> Seems like we need more formal procedures for writing test than
> application code. This also solves the problem of expense of
> "verification" of typeclasses/concepts/abilities that need to be done
> just once for a class with a fixed interface and not each time when the
> class gets loaded from a module. The value of "test time" is increased
> to a higher status not unlike the role of "compile time" for static
> analysis in languages with default type systems. But maybe my head is
> just too far in the clouds when I'm thinking about the end of the
> static/dynamic language distinction using type reconstruction at "test
> time". Heavyweight frameworks naturally emerge also in Python but being
> latent. Reconstructing them from runtime information is a challenge for
> dynamically typed languages. A good enough approximation can be turned
> into a set of prescriptions. This is not much different from the way
> natural scientists frame reality according to Karl Poppers "logic of
> progress" ( hypothetical deductive method ). Regular enough data are
> used to form a hypothesis about the future behaviour of a system. This
> hypothesis is used to frame reality in a mathematical description - just
> like type systems. Making new experiments / measurents can falsify a yet
> established deductive system which leads to improved frames ( reframing
> ) and progressive understanding of systems properties. The role of tests
> in this approach is twofold: they check application code and keep
> information about the "true nature" of the programs behaviour using
> measurements which leads to automatical generation of a formal
> hypothesis of a systems behaviour that may become prescriptive i.e.
> forms a type system:

To me, run-on paragraphs (and the above seems to me to be at least three) 
have the same comprehension problems as run-on sentences ;-)

"In the clouds" indeed.  Maybe try again?

Terry Jan Reedy 




From ark-mlist at att.net  Sat Nov 25 21:17:58 2006
From: ark-mlist at att.net (Andrew Koenig)
Date: Sat, 25 Nov 2006 15:17:58 -0500
Subject: [Python-3000] two things
In-Reply-To: <1d85506f0611251132q5325b36cn4570f5bbe403484@mail.gmail.com>
Message-ID: <001c01c710ce$d3000610$6402a8c0@arkdesktop>

> "contract" is a better term, IMO, since it's already used in CS (as in
> Eiffel), and describes the situation more correctly: *behavior* rather
> than *signature*.
> "ability" just doesn't seem right to me: my class is not *able* to be a
> set,
> it *behaves* like a set. it follows the set contract, not "ability"

I avoided suggesting the word "contract" instead of "ability" because I do
not want people to think that the notion is somehow connected to Eiffel
contracts.  As Guido (I think) pointed out, it's closer to Haskell
typeclasses.



From jimjjewett at gmail.com  Sun Nov 26 01:03:16 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 25 Nov 2006 19:03:16 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <3497511880590744734@unknownmsgid>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
Message-ID: <fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>

On 11/22/06, Bill Janssen <janssen at parc.com> wrote:
> > yeah, namespaces are such a useless thing.  let's put everything in one
> > big flat namespace, so we don't have to wonder where things are.

> Pardon me?  That's what we've got now, isn't it?  That's why we have
> to do that ugly and hard-to-explain name mangling for "special"
> methods.

Not quite.

The documentation (http://docs.python.org/ref/id-classes.html) states
that __special__ methods are reserved to the language.  This isn't
strictly enforced, but it does add a second namespace.

There isn't usually any need for a third namespace, unless you are
using another huge framework, such as Zope or Peak.  If so, the
frameworks have their own conventions.  I have no opinion on whether
their conventions happen to be good, safe from false alarms, or even
frequently used; I am stating only that the frameworks themselves are
responsible for carving out their pseudo-namespaces.

That said, I think the situation would change (for the worse) if we
started to see heavy use of generic functions or interfaces.

These should not use the __special__ convention (they aren't part of
the system, and won't all even be widespread).

Using a designated name in the main namespace will lead to clashes.
(Not every "next" attribute is an iterator-related function).

I had thought that you (Bill), (along with PJE) were assuming that the
generic function or interface itself would serve to distinguish the
namespace.  Instead of

    class A(object):
        def magic_meth_foo(self, ...):

I would write

    class A(object):
        defop foo.magic_meth(self, ...)

This does work, but ... code gets just a little bit longer, with the
extra going to dots and boilerplate.  The boilerplate won't hide the
way "__" does.  It probably shouldn't always hide, since it won't even
be boilerplate in some cases.  But it will be boilerplate so often
that readers will start to skim, even when they shouldn't.

In the end, that extra almost-boilerplate strikes me as a very high
price to pay, and I'm not sure it can be avoided unless Interfaces and
Generic Functions stay relatively rare.

-jJ

From greg.ewing at canterbury.ac.nz  Sun Nov 26 01:26:45 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 26 Nov 2006 13:26:45 +1300
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
	<fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>
Message-ID: <4568DF45.1050200@canterbury.ac.nz>

Jim Jewett wrote:

> There isn't usually any need for a third namespace, unless you are
> using another huge framework, such as Zope or Peak.

Even then, there isn't a problem unless you want to
multiply inherit from two classes that clash. I can't
remember ever encountering such a problem with
__special__ methods.

--
Greg

From pje at telecommunity.com  Sun Nov 26 02:52:06 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 25 Nov 2006 20:52:06 -0500
Subject: [Python-3000] Special methods and interface-based type  system
In-Reply-To: <fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.co
 m>
References: <3497511880590744734@unknownmsgid>
	<-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
Message-ID: <5.1.1.6.0.20061125204355.027f7be8@sparrow.telecommunity.com>

At 07:03 PM 11/25/2006 -0500, Jim Jewett wrote:
>I had thought that you (Bill), (along with PJE) were assuming that the
>generic function or interface itself would serve to distinguish the
>namespace.  Instead of
>
>     class A(object):
>         def magic_meth_foo(self, ...):
>
>I would write
>
>     class A(object):
>         defop foo.magic_meth(self, ...)
>
>This does work, but ... code gets just a little bit longer, with the
>extra going to dots and boilerplate.  The boilerplate won't hide the
>way "__" does.  It probably shouldn't always hide, since it won't even
>be boilerplate in some cases.  But it will be boilerplate so often
>that readers will start to skim, even when they shouldn't.
>
>In the end, that extra almost-boilerplate strikes me as a very high
>price to pay, and I'm not sure it can be avoided unless Interfaces and
>Generic Functions stay relatively rare.

This isn't quite correct.  First, in the interface API I proposed, you 
could use regular methods, if you were putting them directly in the object 
class.

However, in the normal use cases for generic functions, you don't often put 
the methods directly in the classes!  As Guido mentions for e.g. __add__ 
and __radd__, you are probably defining them in some kind of glue code that 
is not of either type.

Second, for "visitor" patterns, you aren't going to be modifying the base 
types to add new methods to generic functions.  Consider, for example, the 
compiler.ast package, that has a variety of objects that represent AST 
nodes.  If you are creating a new compiler or refactoring tool or "lint" 
checker with generic functions, you'd write the functions in a new module 
for the function to be performed, and then define methods for the function 
by AST class.  You would not modify the AST classes to add "lint_me" methods!

And so it is for any other case where you are retrofitting or extending the 
behavior of existing types - making it on the whole relatively infrequent 
that you'll want to put most methods inside class bodies.  This is 
especially true in cases where you want to support your operation on 
built-in types: there's no place for you to add the code in that case.

Finally, in situations where you *do* put the methods directly on the 
class, chances are good that you'll either be adding a single operation or 
will want to just have a generic factory function to just produce an object 
that has all the desired methods in the first place.  In none of these 
cases will you have many generic function methods in a single class body.

The only case left, really, where you'd put many generic function methods 
in a class body is when they're builtins like len(), iter() etc...  which 
don't have the "namespace noise" you're worried about.


From jimjjewett at gmail.com  Sun Nov 26 03:17:36 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 25 Nov 2006 21:17:36 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <456590C3.4040801@davious.org>
References: <456590C3.4040801@davious.org>
Message-ID: <fb6fbf560611251817o534085na31319b0a7d1c3ae@mail.gmail.com>

On 11/23/06, Dave Anderson <python3000 at davious.org> wrote:
> Interfaces
> ==========
> iter_interface = interface(iter)  # class-based interface
> file_open_close_inter = interface(file.open, file.close) # method interface

[ strings were also treated differently ]

Are you keying off the dot in the name, or the type of the named object?

SomeClass.name is typically a method, but may be a string, or even
another class.

-jJ

From jimjjewett at gmail.com  Sun Nov 26 03:56:06 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sat, 25 Nov 2006 21:56:06 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221903nf9a6c1am1d70a5cba088c38d@mail.gmail.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>
	<ca471dc20611221903nf9a6c1am1d70a5cba088c38d@mail.gmail.com>
Message-ID: <fb6fbf560611251856h6a49c11ek373e8ff25c5c908@mail.gmail.com>

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> On 11/22/06, Jonathan LaCour <jonathan-lists at cleverdevil.org> wrote:
> > Phillip J. Eby wrote:

> > > In essence, interfaces turn libraries into "frameworks", but generic
> > > functions turn frameworks into libraries.  I didn't really discover
> > > this until January of last year ...

> > This is easily the most important part of the conversation, IMO, and I
> > would hate for it to be missed. "Interfaces" in the Java universe are a
> > hard contract that everyone has to agree upon up-front, ... you end up
> > with is a world of many competing frameworks, none of which interoperate.

> > ... something that seems a tad magical at first
> > I have actually used generic functions (using RuleDispatch) in practice
> > and find them to not only be hugely powerful, but much more pragmatic.

> Sorry, but that was about as informative as a "+1" vote.

I found it a bit more specific on two points:

What I (mis?)read was:

(1)
PJE:  Interfaces rely on buyin; they end up accidentally creating
walled gardens.  Generic Functions make fewer demands, so they start
to work sooner, and integrate better.  [The down side is that finding
the relevant code -- or even being sure that it *is* is the relevant
code, and isn't overridden elsewhere -- gets more difficult. -jJ]

Jonathan:  Phillip has written a lot of long messages; this one point
is particularly important for the summary.  It captures a (the?) key
advantage of generic functions.

(2)
Generic Functions are harder to get your head around, but once you do,
they're worth it.  Phillip has been saying this already, but it
matters that someone else has gotten to the point where they are
useful and seem simple.

Personally, I stand by my assessment that they can make metaclasses
look straightforward -- but that is partly because metaclasses have
had their roughest edges worn off already.  (For example, they no
longer require compiling a C class so odd that the pattern gets named
after someone.)  If GF explanations start to come from several
different people, the same process should eventually simplify them.
If that happened quickly enough, it would change the tradeoffs.

-jJ

From ironfroggy at gmail.com  Sun Nov 26 05:28:47 2006
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sat, 25 Nov 2006 23:28:47 -0500
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <ca471dc20611242129na1f48d5o2efb1983139c1e23@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061122151242.03f68a88@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
	<ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
	<76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>
	<ca471dc20611242129na1f48d5o2efb1983139c1e23@mail.gmail.com>
Message-ID: <76fd5acf0611252028i496ef6d7r9b23674bcb38046e@mail.gmail.com>

I will make only one more comment and then ill drop my comments
without direct questions.

On 11/25/06, Guido van Rossum <guido at python.org> wrote:
> Hm. The double colon rubs me the wrong way (Perl and/or C++). But
> apart from that, if this is the solution, I'm not sure the problem
> you're trying to solve is really worth solving. I just don't expect
> there will be all that many generic operations that need to be stuffed
> into arbitrary classes. Maybe I'll just take back what I said about
> wanting all such operations to be inside the class. Or maybe I'd be
> happier if there was a decorator indicating the name of the operation.

I don't care about the syntax. anything that can denote an expression
(the left of the ::) and a name (the right of the ::) is OK and I dont
care how its denoted.

> Also, I think you're overloading 'def' in a non-obvious way.
> Currently, "def foo..." means an assignment to the local variable foo.
> I would expect that if we extend the syntax for the thing between
> 'def' and the argument list to be more than just a name, it should
> still be considered an assignment target. But that's clearly not what
> you're after. From that POV, I find defop (while still unpleasant for
> other reasons) more "honest" than your overloading of def -- at least
> defop says upfront that it's not just an assignment. (Although the
> similarity with def is still confusing IMO.)

You must be misunderstanding me. I am not saying that its not an
assignment. It would not change what def really means. operator::len
would be the actual name of the function to be created and the name of
the global, local, or class attribute it is bound to. I am saying
operator::len would become something like
MyClass.__dict__[operator::len] and what operator::len evaluates to, i
dont know. something that represents what it is. Maybe just a tuple. I
would expect it also exist for any assignment. special casing being
bad and all.

> Still not convinced? Focus on other problems first. This isn't the
> most important problem we're trying to solve.
>
> PS, It's __builtin__, not __builtins__ -- the latter is an
> unfortunately named but ultimately unimportant implementation detail
> (unimportant unless you're implemented restricted python, that is);
> the former is the module that is implied at the end of every free name
> search.
>
>
> --Guido

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From jonathan-lists at cleverdevil.org  Sun Nov 26 05:54:36 2006
From: jonathan-lists at cleverdevil.org (Jonathan LaCour)
Date: Sat, 25 Nov 2006 23:54:36 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <fb6fbf560611251856h6a49c11ek373e8ff25c5c908@mail.gmail.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>
	<ca471dc20611221903nf9a6c1am1d70a5cba088c38d@mail.gmail.com>
	<fb6fbf560611251856h6a49c11ek373e8ff25c5c908@mail.gmail.com>
Message-ID: <E1ED9CFA-9E56-47DE-BBCD-B15D8ACF28A8@cleverdevil.org>

Jim Jewett wrote:

>> Sorry, but that was about as informative as a "+1" vote.
>
> I found it a bit more specific on two points:
>
> What I (mis?)read was:
>
> (1)
> PJE:  Interfaces rely on buyin; they end up accidentally creating
> walled gardens.  Generic Functions make fewer demands, so they start
> to work sooner, and integrate better.  [The down side is that finding
> the relevant code -- or even being sure that it *is* is the relevant
> code, and isn't overridden elsewhere -- gets more difficult. -jJ]
>
> Jonathan:  Phillip has written a lot of long messages; this one point
> is particularly important for the summary.  It captures a (the?) key
> advantage of generic functions.

I am glad that *someone* got what I was trying to say.  Philip indeed
writes long, informative messages.  Its easy for little gems like this
one go by unnoticed.  I had not seen this relative merit of generic
functions (and disadvantage of interfaces) discussed, so I responded
with my agreement that this was very true for me in actual practice
(not just in theory).

> (2)
> Generic Functions are harder to get your head around, but once you do,
> they're worth it.  Phillip has been saying this already, but it
> matters that someone else has gotten to the point where they are
> useful and seem simple.

In fact, I used Philip's wonderful RuleDispatch module again just today
to create something quite useful.  I personally found generic functions
to be quite easy to understand, but didn't truly begin to appreciate
them until I started to use them.

So, sure, it might have been a +1 vote, but it comes with the context
that I have actually *used* both generic functions and interfaces in
practice, and have found Philip's statement above to ring very true.

--
Jonathan LaCour
http://cleverdevil.org




From sluggoster at gmail.com  Sun Nov 26 10:55:44 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Sun, 26 Nov 2006 01:55:44 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <87ac2x7wz0.fsf@qrnik.zagroda>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	<87ac2x7wz0.fsf@qrnik.zagroda>
Message-ID: <6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>

Status update and questions about root splitting.

I've got a path module written (called unipath) and am in the middle
of the unit tests.   When they're done I'll release it for review.
It's been slow coz I was setting up two computers at the same time.

I tried to make a separate PathAlgebra class and FSPath class, but it
got so unweildly to use I made the latter a subclass.  They're now
called PathName and Path.  The main issues are:

    - Should FSPath.cwd(), .walk(), .rel_path_to(), etc return a
PathAlgebraObject or
      an FSPath?  The use cases seem evenly divided.

    - It gets ugly when you have to deconstruct an FSPath to modify
it, then call
      the constructor again to create a new FSPath to use it.  Users
would complain
      at:
             my_directory = FSPath(PathName("...")).mkdir()
      or:
             my_directory = FSPath("...")
             FSPath(my_directory.path, "subdir").mkdir()

Since I really don't think I want to use the non-subclass version in
my own applications and thus have little incentive to work on it,
maybe someone else who really wants that model for the stdlib can take
over implementing that part.

* * *
But the main question I wanted to ask about was root splitting.  I'm
going for the following API:

       p = PathName(*joinable_components)
       p.parent
       p.name
       p.ext
       p.name_without_ext
       p.drive
       p.unc
       p.split_root()  ->  (root, rest)
       p.components()  ->  [root, component1, component2, ...]

The idea being that you'll often want to make a path relative and then
attach it to a new parent, add/remove components, etc.

My idea, based on Noam's proposal, was to consider the root:
    /
    \
    c:\
    C:
    \\unc_share\
or "" for a relative path.  That way, root + rest would always be
concatenation without a separator.  But os.path.split* don't work like
this.

 >>> ntpath.splitdrive(r"C:\dir\subdir")
('C:', '\\dir\\subdir')
>>> ntpath.splitunc(r"\\unc_share\foo\bar\baz")
('\\\\unc_share\\foo', '\\bar\\baz')

There is no method to split a slash root (/ or \) so I have to do that manually.

To realize my original idea, in the drive and UNC cases I'd have to
move the intervening slash from the remainder to the root.  But this
would make the root different that what p.drive and p.unc return -- if
that matters.  Do we even need p.drive and p.unc if we have
p.split_root() that handles all kinds of roots?

Do we need the .split* methods if the properties/methods above handle
all the use cases?  It would be cleaner not to include them but
somebody will whine "where is my method?"  It's also slightly less
efficient to do (p.parent, p.name) or (p.name_without_ext, p.ext)
because you're implicitly doing the same split twice -- if that
matters.

On the other hand, os.path.join() handles all this, adding a separator
when necessary, and even handling "" as the first argument.  So maybe
I should just let os.path.join() do its magic and be happy.

Noam distinguished between a drive-relative (r"C:foo") and
drive-absolute (r"C:\foo").  I'm treating the former as absolute
because I don't think it matters in this context.  os.path.join() does
not add a slash with drive letters:

>>> ntpath.join(r"C:", "foo")
'C:foo'
>>> ntpath.join(r"C:\ "[:-1], "foo")
'C:\\foo'

(The latter funny syntax is to get around the fact that you can't have
a backslash at the end of a raw string literal.)

So with drive letters it's vital not to add a backslash, and to trust
the root portion ends with a backslash if necessary.

* * *
I've put some work into converting paths:
NTPath(PosixPath("foo/bar.py"), using the .components() format as a
universal intermediary, and refusing to convert absolute paths.  This
has taken some work to implement and several paragraphs to document,
and I still haven't tested to see if it's correct.  I'm wondering if
it's becoming overkill for the original goal and anticipated use
cases.  We thought, "Yeah, it would be a good idea to support
non-native paths and to convert paths."  Then Talin, who wanted it,
said he was mainly concerned about putting Posix paths in config files
and having them work on other platforms (Windows on embedded systems
that don't have a current directory), without having to manually
convert them.  But os.path.normpath() and os.path.normcase() already
convert Posix slashes to Windows backslashes.  Maybe that's enough?
Is there a need to go the other way or do other conversions?

* * *
Path APIs are really like web frameworks in Python: there's no way
everybody's going to agree on one, and it's so easy to write a
different one.  So I'm factoring the more complex code into a function
library that can be used as a back end to other path APIs.

-- 
Mike Orr <sluggoster at gmail.com>

From jimjjewett at gmail.com  Sun Nov 26 18:25:17 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 26 Nov 2006 12:25:17 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <5.1.1.6.0.20061125204355.027f7be8@sparrow.telecommunity.com>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
	<5.1.1.6.0.20061125204355.027f7be8@sparrow.telecommunity.com>
Message-ID: <fb6fbf560611260925o762b9ffdq370190cac4953a52@mail.gmail.com>

On 11/25/06, Phillip J. Eby <pje at telecommunity.com> wrote:

> However, in the normal use cases for generic functions,
> you don't often put the methods directly in the classes!

I think this gets to the core of my biggest worry.

When I'm doing sysadmin-type maintenance (which at my employer does
not mean python, but rather nested *sh scripts, some sourced and some
called), my biggest problem is figuring out where things *really* came
from.

Ahh ... it comes from this variable, which was set ... where?  And
reset where?  Oh, at that point the path was this, so it was really
using *that* version.

I have resisted using twisted because (at least at first glance) it
seems to factor classes so small that I end up with the same feeling
of magic -- I couldn't easily find where things were defined, or get
the full API and documents in one place.

I fear that generic functions would encourage this sort of action at a
distance.  I would probably use it myself, because I know it can be
very helpful when I'm initially writing something.  But it can also
make life terribly hard for the next person.  One reason I like python
is that it doesn't usually encourage that sort of tradeoff.

-jJ

From jimjjewett at gmail.com  Sun Nov 26 18:54:53 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 26 Nov 2006 12:54:53 -0500
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	<87ac2x7wz0.fsf@qrnik.zagroda>
	<6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
Message-ID: <fb6fbf560611260954r119b2d7cl950595efba9b7f1b@mail.gmail.com>

On 11/26/06, Mike Orr <sluggoster at gmail.com> wrote:
> I tried to make a separate PathAlgebra class and FSPath class, but it
> got so unweildly to use I made the latter a subclass.  They're now
> called PathName and Path.

This makes sense to me.  An FSPath without path algebra is basically a
"directory listing"  That might be a useful abstraction on its own, or
it might be too much purity.
> My idea, based on Noam's proposal, was to consider the root:
>     /
>     \
>     c:\
>     C:
>     \\unc_share\
> or "" for a relative path.  That way, root + rest would always be
> concatenation without a separator.  But os.path.split* don't work like
> this.

[ They put the C: or such as the root, and the first \ in the path]

> Do we even need p.drive and p.unc if we have
> p.split_root() that handles all kinds of roots?

I don't think so.  Maybe on a WindowsFSPath subclass, for
compatibility.  But taking them out of the generic path is a good
thing.  Most people don't need to worry about them, and shouldn't need
to wonder whether or not they do.

> Noam distinguished between a drive-relative (r"C:foo") and
> drive-absolute (r"C:\foo").  I'm treating the former as absolute
> because I don't think it matters in this context.

I think you are correct, though I'm not sure I would have thought of
it.  C: without a slash is effectively a mount point into the current
directory.  This particular mount point might shift around
unexpectedly, but ... someone could rewrite the regular files on you
too.

It does make a leading ".." meaningful though; I'm not sure if that matters.

> >>> ntpath.join(r"C:\ "[:-1], "foo")
> 'C:\\foo'

> (The latter funny syntax is to get around the fact that you can't have
> a backslash at the end of a raw string literal.)

You have to double it.  One way to make lemonade of this lemon is to
add a class attribute P.sep equivalent to os.path.sep.  (And,
realistically, the separator is more tied to the File System than it
is to the Operating System.)  If might encourage people to use
path.sep instead of just assuming the locally correct character.

-jJ

From sluggoster at gmail.com  Sun Nov 26 19:42:43 2006
From: sluggoster at gmail.com (Mike Orr)
Date: Sun, 26 Nov 2006 10:42:43 -0800
Subject: [Python-3000] Mini Path object
In-Reply-To: <fb6fbf560611260954r119b2d7cl950595efba9b7f1b@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	<87ac2x7wz0.fsf@qrnik.zagroda>
	<6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
	<fb6fbf560611260954r119b2d7cl950595efba9b7f1b@mail.gmail.com>
Message-ID: <6e9196d20611261042y561a9a92s83f71291e5ef87b9@mail.gmail.com>

On 11/26/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 11/26/06, Mike Orr <sluggoster at gmail.com> wrote:
> > I tried to make a separate PathAlgebra class and FSPath class, but it
> > got so unweildly to use I made the latter a subclass.  They're now
> > called PathName and Path.
>
> This makes sense to me.  An FSPath without path algebra is basically a
> "directory listing"  That might be a useful abstraction on its own, or
> it might be too much purity.

It's a single path, actually, upon which you can call filesystem operations.
    p.mkdir(), p.copytree(), p.read_file(), p.readlink(), p.listdir()

What you *can't* do is extract/alter the pathname itself using FSPath
operations.
    p.ext, p + ".gz", p.expand_user()    # Error!  Use p.path.ext instead.

Nobody has specifically asked for a FSPath class.  But some people are
adamant they want a PathAlgebra class, a clear separation between
pathname vs filesystem operations. Otherwise we'd just use the earlier
implementations which combine both in one class.  One proposal is to
put PathAlgebra in os.path and not have a FSPath class in the stdlib
at all.  That's not my ideal but it may be small enough and
non-controversial enough to get into Python 2.6, and then we can build
a more elaborate FSPath on top of it for 3.0 (or not).

> C: without a slash is effectively a mount point into the current
> directory.

That's what I always thought "C:foo" is.  But Glyph said it's more
complicated than that:

'"C:blah" does not mean what you think it means on Windows.
Regardless of what you think it means, it is not that.  I thought I
understood it once as the current process having a current directory
on every mapped drive, but then I had to learn about UNC paths of
network mapped drives and it stopped making sense again.'

[python-dev message dated 2006-Nov-1, subject: Path object design,
from glyph at divmod.com]

-- 
Mike Orr <sluggoster at gmail.com>

From jimjjewett at gmail.com  Sun Nov 26 20:13:57 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 26 Nov 2006 14:13:57 -0500
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611261042y561a9a92s83f71291e5ef87b9@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	<87ac2x7wz0.fsf@qrnik.zagroda>
	<6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
	<fb6fbf560611260954r119b2d7cl950595efba9b7f1b@mail.gmail.com>
	<6e9196d20611261042y561a9a92s83f71291e5ef87b9@mail.gmail.com>
Message-ID: <fb6fbf560611261113l5f96b2b3jc9e3eb7d320e92c@mail.gmail.com>

On 11/26/06, Mike Orr <sluggoster at gmail.com> wrote:
> On 11/26/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> > On 11/26/06, Mike Orr <sluggoster at gmail.com> wrote:
> > > I tried to make a separate PathAlgebra class and
> > > FSPath class, but it got so unweildly to use I made
> > > the latter a subclass.  They're now called
> > > PathName and Path.

> > This makes sense to me.  An FSPath without path
> > algebra is basically a "directory listing"

> It's a single path, actually,

Fair enough.

> upon which you can call filesystem operations.
>     p.mkdir(), p.copytree(), p.read_file(), p.readlink(), p.listdir()

So if it doesn't happen to point to a directory, you can either ask it
for a different Path (the result of readlink) or open it
(read_file)...

> What you *can't* do is extract/alter the pathname
> itself using FSPath operations.
>     p.ext, p + ".gz", p.expand_user()    # Error!  Use p.path.ext instead.

I had thought that Path objects were immutable, and so these would
produce new isntances.

> > C: without a slash is effectively a mount point into
> >  the current directory.

> That's what I always thought "C:foo" is.  But Glyph
> said it's more complicated than that:

Most of the exceptions were things like the CON special file.  These
make the windows (or DOS) filesystem wierd, but they aren't really
affected by C: vs C:\

CON, C:\dir1\CON, C:CON, and .\con all refer to the same special file.

-jJ

From p.f.moore at gmail.com  Sun Nov 26 20:48:55 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 26 Nov 2006 19:48:55 +0000
Subject: [Python-3000] Mini Path object
In-Reply-To: <fb6fbf560611261113l5f96b2b3jc9e3eb7d320e92c@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	<87ac2x7wz0.fsf@qrnik.zagroda>
	<6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
	<fb6fbf560611260954r119b2d7cl950595efba9b7f1b@mail.gmail.com>
	<6e9196d20611261042y561a9a92s83f71291e5ef87b9@mail.gmail.com>
	<fb6fbf560611261113l5f96b2b3jc9e3eb7d320e92c@mail.gmail.com>
Message-ID: <79990c6b0611261148v5d95dabfnecf00d5dc73b8c62@mail.gmail.com>

On 11/26/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 11/26/06, Mike Orr <sluggoster at gmail.com> wrote:
> > On 11/26/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> > > C: without a slash is effectively a mount point into
> > >  the current directory.
>
> > That's what I always thought "C:foo" is.  But Glyph
> > said it's more complicated than that:
>
> Most of the exceptions were things like the CON special file.  These
> make the windows (or DOS) filesystem wierd, but they aren't really
> affected by C: vs C:\
>
> CON, C:\dir1\CON, C:CON, and .\con all refer to the same special file.

Technically, every drive has a "current path". You can see this by
doing the following on a Windows system (here, the prompt shows the
current path)

C:\Data>cd F:\Software

*** Note from the following prompt that this does *not* change the
current directory - because the current drive is C:, and we changed
the directory on F:

C:\Data>dir F:4nt.zip
 Volume in drive F is Data
 Volume Serial Number is 7043-3187

 Directory of F:\Software

[...]

*** But note that F:4nt.zip is located in F:\Software, which is where
we made the current directory on F:

C:\Data>f:

F:\Software>

*** And if we go to the F: drive (unspecified directory) we get to
\Software, the current directory on F:.

Basically, Windows maintains a full set of current directories, one
for each drive (actually stored as hidden environment variables named
something like "=C:", "=D:" etc.) and D:foo refers to the file foo in
the current directory on the D: drive.

This is obscure to the point where I doubt anyone relies on it, but
it's the sort of edge case that you need to get right or someone will
complain in the end... :-)

Nitpicking-ly y'rs

Paul.

From brett at python.org  Sun Nov 26 22:10:24 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 26 Nov 2006 13:10:24 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <C0D2AD09-FC51-432C-87EE-BC91E8FE99D8@pagedna.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
	<bbaeab100611241120j6b8ea9aka67b28b7ed4e63cc@mail.gmail.com>
	<C0D2AD09-FC51-432C-87EE-BC91E8FE99D8@pagedna.com>
Message-ID: <bbaeab100611261310x3bdaa5aan6534dde3b1f5b7e5@mail.gmail.com>

On 11/24/06, Tony Lownds <tony at pagedna.com> wrote:
>
> > Obviously signature objects would grow support for annotations, but I
> > still need the information to be carried on the code object to
> > incorporate into signature objects.
> >
>
> Signature objects still need a way to know the nested parameters, right?


They already handle them.

How about a co_argnames attribute? eg for
>
> def f((x, y), z): pass
>
> f.func_code.co_argnames would be (('x', 'y'), 'z')


That's fine, but the compiler would need to change if it were to use this.
Plus I am still hoping to make nested parameters disappear in Py3K (I won't
be pushing for it any sooner than PyCon, though).

I need to implement something like this to properly build
> func_annotations
> inside MAKE_FUNCTION.



I don't quite follow.  Don't you already have support in MAKE_FUNCTION when
the object is created?

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061126/fc3c1711/attachment.htm 

From greg.ewing at canterbury.ac.nz  Sun Nov 26 21:57:50 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 27 Nov 2006 09:57:50 +1300
Subject: [Python-3000] Mini Path object
In-Reply-To: <6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
References: <45536FF3.4070105@acm.org>
	<79990c6b0611091245r2d0251f0q813951d8b94dca1e@mail.gmail.com>
	<87ac2x7wz0.fsf@qrnik.zagroda>
	<6e9196d20611260155h3e476474oe9e0beff8c9bbb0d@mail.gmail.com>
Message-ID: <4569FFCE.9050507@canterbury.ac.nz>

Mike Orr wrote:
> They're now called PathName and Path.

Not a sane choice of names, since in this context,
"path" and "pathname" are synonymous in ordinary
language.

--
Greg

From jimjjewett at gmail.com  Sun Nov 26 22:47:02 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 26 Nov 2006 16:47:02 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220708v13f28179s7ded48a5308ab7cf@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<ca471dc20611211807s396e8a4aj2d4316ce6ef3329e@mail.gmail.com>
	<45640E3A.7010307@livinglogic.de>
	<ca471dc20611220708v13f28179s7ded48a5308ab7cf@mail.gmail.com>
Message-ID: <fb6fbf560611261347v1abbc565u8e8b8f1af7f839d2@mail.gmail.com>

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> On 11/22/06, Walter D?rwald <walter at livinglogic.de> wrote:
> > Another effect of __special__ methods is that they divide
> > the class namespace into two parts: The __special__ names
> > are those that have to be implemented to support core
> > Python interfaces, and the methods without underscores
> > are those that implement the "main aspect" of the
> > class.

> Well, there are plenty of examples of __special__ methods
> for other purposes than core Python interfaces,

The documentation explicitly reserves __*__ to the language.  You can
argue that some uses aren't core, but that still means __*__ isn't
appropriate for a random user-supplied aspect.  (And if it were, that
would lead to conflicts over attractive names like __log__.
__package_module__name__ might work, but ... starts to get hackish.)

> and plenty of core Python
> interfaces that don't use __special__ (dict.get(), list.append(),
> iter.next(), str.lower(), you get the idea.)

ermm... I'm not seeing any counterexamples just yet.  get, append, and
lower apply only to specific classes, and from there they get
generalized to the informal interfaces that those classes prototype.

Less obviously, this is true even of next.  __iter__ is the special
method, and after you have called it, you are operating on the
iterable it returns (which may be self).  next isn't generic to all
objects; only to those which implement iteration (result from a
previous call to __iter__).

> The *original* idea was for __special__ names to apply to
> anything that wasn't invoked using regular method notation:

By this original intent, next would indeed be special, as would
implementation of any generic functions -- but if 3rd-party
generic/extensible functions become common they probably will need a
way to carve out their own namespaces.

-jJ

From jimjjewett at gmail.com  Sun Nov 26 22:52:55 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 26 Nov 2006 16:52:55 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<45645252.6010101@benjiyork.com>
	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
Message-ID: <fb6fbf560611261352w2870812as6e3e0411e5cf4dbd@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 08:36 AM 11/22/2006 -0500, Benji York wrote:
> >It seems to me that not all interfaces coincide with
> >something the object can _do_.  ...

> Aren't these also *operations* being performed?

ISafeForThreading may boil down to certain operations not being
dangerous, but I don't think the operations are the primary meaning.

-jJ

From pje at telecommunity.com  Sun Nov 26 23:01:48 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 26 Nov 2006 17:01:48 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <fb6fbf560611261352w2870812as6e3e0411e5cf4dbd@mail.gmail.co
 m>
References: <5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
	<ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<45645252.6010101@benjiyork.com>
	<5.1.1.6.0.20061122104602.02e32618@sparrow.telecommunity.com>
Message-ID: <5.1.1.6.0.20061126165856.03ef19b0@sparrow.telecommunity.com>

At 04:52 PM 11/26/2006 -0500, Jim Jewett wrote:
>On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 08:36 AM 11/22/2006 -0500, Benji York wrote:
> > >It seems to me that not all interfaces coincide with
> > >something the object can _do_.  ...
>
> > Aren't these also *operations* being performed?
>
>ISafeForThreading may boil down to certain operations not being
>dangerous, but I don't think the operations are the primary meaning.

I don't see how you can specify that something is thread-safe without 
explicit reference to what *operations* are thread-safe, and in what 
way.  Indeed, it's the *operations* that one wishes to confirm are 
thread-safe; a data structure in and of itself has no "thread safety" per se.


From tony at pagedna.com  Sun Nov 26 23:14:25 2006
From: tony at pagedna.com (Tony Lownds)
Date: Sun, 26 Nov 2006 14:14:25 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <bbaeab100611261310x3bdaa5aan6534dde3b1f5b7e5@mail.gmail.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
	<bbaeab100611241120j6b8ea9aka67b28b7ed4e63cc@mail.gmail.com>
	<C0D2AD09-FC51-432C-87EE-BC91E8FE99D8@pagedna.com>
	<bbaeab100611261310x3bdaa5aan6534dde3b1f5b7e5@mail.gmail.com>
Message-ID: <EAEDCB4B-DC66-435A-AC5C-D111167FC020@pagedna.com>


On Nov 26, 2006, at 1:10 PM, Brett Cannon wrote:

> On 11/24/06, Tony Lownds <tony at pagedna.com> wrote:
> > Obviously signature objects would grow support for annotations,  
> but I
> > still need the information to be carried on the code object to
> > incorporate into signature objects.
> >
>
> Signature objects still need a way to know the nested parameters,  
> right?
>
> They already handle them.
>

I see, through the bytecode inspection that inspect.py does.

> How about a co_argnames attribute? eg for
>
> def f((x, y), z): pass
>
> f.func_code.co_argnames would be (('x', 'y'), 'z')
>
> That's fine, but the compiler would need to change if it were to  
> use this.  Plus I am still hoping to make nested parameters  
> disappear in Py3K (I won't be pushing for it any sooner than PyCon,  
> though).
>

Yes, it is the compiler that would implement this. I have it  
implemented as follows.

 >>> def f(a, (b, c), d=1, *e, f, g=1, **h): pass
...
 >>> f.func_code.co_argnames
('a', ('b', 'c'), 'd', '*e', 'f', 'g', '**h')

However since inspect.py doesn't need this and neither does my code,  
I'll drop it.

> I need to implement something like this to properly build
> func_annotations
> inside MAKE_FUNCTION.
>
>
> I don't quite follow.  Don't you already have support in  
> MAKE_FUNCTION when the object is created?
>

The support available is the code object and anything pushed onto the  
stack. Just looking at the code object,
the information is simply not available outside of reverse- 
engineering co_code (as inspect does).
I ended up pushing a tuple of argument names onto the stack.

eg, for

def f((x:1, y))->3: pass

the bytecode is...

   1           0 LOAD_CONST               0 (1)
               3 LOAD_CONST               1 (3)
               6 LOAD_CONST               2 (('x', 'return'))
               9 LOAD_CONST               3 (<code object f at  
0xb7ecd848, file "<str>", line 1>)
              12 EXTENDED_ARG             3
              15 MAKE_FUNCTION        196608
              18 STORE_NAME               0 (f)
              21 LOAD_CONST               4 (None)
              24 RETURN_VALUE

(the argument is so big because I need to pass the # of annotations  
in the argument of MAKE_FUNCTION so
that the stack effect can be easily calculated)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061126/cd7596f8/attachment.html 

From jimjjewett at gmail.com  Mon Nov 27 00:27:15 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Sun, 26 Nov 2006 18:27:15 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061121142441.03fb64e8@sparrow.telecommunity.com>
	<5.1.1.6.0.20061121211338.01f35718@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122022306.03828420@sparrow.telecommunity.com>
	<ca471dc20611220753q5be31ce0sc2e36906a782d585@mail.gmail.com>
Message-ID: <fb6fbf560611261527t57d6879eg972fe55e20d41912@mail.gmail.com>

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> I'd much rather have a few "big" interfaces (Number,
> RealNumber, Integer) than lots of small ones
> Pedroni-style (number-that-support-exponentiation,
> number-that-supports-bitwise-and,
> number-that-supports-left-shift, ...?).

My understanding is that the Pedroni-style interfaces are big; they
are just decomposable.

File would be a (large) interface, but you could also specify that you
only cared about (or only implemented) the subset consisting of
File.close and File.write, in case you wanted something sort-of-like
an existing interface.

-jJ

From guido at python.org  Mon Nov 27 02:36:53 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 26 Nov 2006 17:36:53 -0800
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <76fd5acf0611252028i496ef6d7r9b23674bcb38046e@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<5.1.1.6.0.20061122180342.03ce3008@sparrow.telecommunity.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
	<ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
	<76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>
	<ca471dc20611242129na1f48d5o2efb1983139c1e23@mail.gmail.com>
	<76fd5acf0611252028i496ef6d7r9b23674bcb38046e@mail.gmail.com>
Message-ID: <ca471dc20611261736y5944e669t8c6e489376caa65a@mail.gmail.com>

On 11/25/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> I will make only one more comment and then ill drop my comments
> without direct questions.
>
> On 11/25/06, Guido van Rossum <guido at python.org> wrote:
> > Hm. The double colon rubs me the wrong way (Perl and/or C++). But
> > apart from that, if this is the solution, I'm not sure the problem
> > you're trying to solve is really worth solving. I just don't expect
> > there will be all that many generic operations that need to be stuffed
> > into arbitrary classes. Maybe I'll just take back what I said about
> > wanting all such operations to be inside the class. Or maybe I'd be
> > happier if there was a decorator indicating the name of the operation.
>
> I don't care about the syntax. anything that can denote an expression
> (the left of the ::) and a name (the right of the ::) is OK and I dont
> care how its denoted.
>
> > Also, I think you're overloading 'def' in a non-obvious way.
> > Currently, "def foo..." means an assignment to the local variable foo.
> > I would expect that if we extend the syntax for the thing between
> > 'def' and the argument list to be more than just a name, it should
> > still be considered an assignment target. But that's clearly not what
> > you're after. From that POV, I find defop (while still unpleasant for
> > other reasons) more "honest" than your overloading of def -- at least
> > defop says upfront that it's not just an assignment. (Although the
> > similarity with def is still confusing IMO.)
>
> You must be misunderstanding me. I am not saying that its not an
> assignment. It would not change what def really means. operator::len
> would be the actual name of the function to be created and the name of
> the global, local, or class attribute it is bound to. I am saying
> operator::len would become something like
> MyClass.__dict__[operator::len] and what operator::len evaluates to, i
> dont know. something that represents what it is. Maybe just a tuple. I
> would expect it also exist for any assignment. special casing being
> bad and all.

Do you realize that expr[expr::expr] already has a meaning?

You seem to be doing an exceptionally poor job explaining your
proposal; you are introducing a new token "::" without ever explaining
what you propose it should mean. How does operator::len differ from
operator.len?

> > Still not convinced? Focus on other problems first. This isn't the
> > most important problem we're trying to solve.
> >
> > PS, It's __builtin__, not __builtins__ -- the latter is an
> > unfortunately named but ultimately unimportant implementation detail
> > (unimportant unless you're implemented restricted python, that is);
> > the former is the module that is implied at the end of every free name
> > search.
> >
> >
> > --Guido
>
> --
> Read my blog! I depend on your acceptance of my opinion! I am interesting!
> http://ironfroggy-code.blogspot.com/
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Nov 27 02:40:03 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 26 Nov 2006 17:40:03 -0800
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <fb6fbf560611251856h6a49c11ek373e8ff25c5c908@mail.gmail.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>
	<ca471dc20611221903nf9a6c1am1d70a5cba088c38d@mail.gmail.com>
	<fb6fbf560611251856h6a49c11ek373e8ff25c5c908@mail.gmail.com>
Message-ID: <ca471dc20611261740m111f537es66c54d94b73c016@mail.gmail.com>

On 11/25/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> Personally, I stand by my assessment that they can make metaclasses
> look straightforward -- but that is partly because metaclasses have
> had their roughest edges worn off already.  (For example, they no
> longer require compiling a C class so odd that the pattern gets named
> after someone.)  If GF explanations start to come from several
> different people, the same process should eventually simplify them.
> If that happened quickly enough, it would change the tradeoffs.

Did you read my blog post where I give my own interpretation and implementation?

http://www.artima.com/weblogs/viewpost.jsp?thread=155514

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Mon Nov 27 02:54:10 2006
From: janssen at parc.com (Bill Janssen)
Date: Sun, 26 Nov 2006 17:54:10 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <ca471dc20611232010i182761cdr54f808cb60938231@mail.gmail.com> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <8853879015409334245@unknownmsgid>
	<ca471dc20611221554q7874f210vfc5a7a5d83a9ad06@mail.gmail.com>
	<-4940442411288794438@unknownmsgid>
	<ca471dc20611232010i182761cdr54f808cb60938231@mail.gmail.com>
Message-ID: <06Nov26.175416pst."58648"@synergy1.parc.xerox.com>

> > I'll still drop off a copy (of Common Lisp the Language, version 2),
> > but there's no need to wait.  It's on the Web at
> > http://www.supelec.fr/docs/cltl/clm/node260.html.  See in particular
> > the "change-class" operation at
> > http://www.supelec.fr/docs/cltl/clm/node305.html.
> >
> > I think I'm still confused (happens a lot :-) about our method
> > namespace discussion.  It seems to me that Python's method namespaces
> > work pretty much the same way that CLOS's do, already.  That is, you
> > don't "clobber" an existing method in a base class when you define a
> > new method by the same name in a derived class; you just mask it.  The
> > base class' method is still there, and can still be called explicitly.
> 
> OK, maybe I  misunderstood what you wrote. I thought I heard you say
> that "len" isn't just "len" -- it's the "len" defined by some
> interface (and presumably implemented in a base class), and if one
> defined a new "len" it wouldn't override the "len" defined by that
> interface (unless one explicitly stated that it did), it would just
> add a different method named "len". That would fly in the face of
> Python's lookup algorithm for methods (where the first "len" you find
> is the one you get). If that's not what you meant, all is probably
> well.

Actually, Python does both:  it overrides the base class' "len", and
adds a new "len" of its own.

class Base:

   def len(self):
      ...

class Derived(Base):

   def len(self):
      ...

Given an instance of Derived, I can make a call on Derived.len,
Base.len, or just let the default method lookup work, in which case I
get Derived.len().  So even if Derived.len() is overridden, Base.len()
is available.  The question then becomes more subtle: is the data
model maintained by the derived class consistent with that of the base
class?  But this is essentially off-topic:  I still think the concern
about "accidentally" overriding methods inherited from some base class
is misplaced, but still minor compared to the other issue of not being
able to figure out whether a value has a particular type.

By the way, it's interesting to look at the distinction made in Guy
Steele's (and Sun's) new research language, Fortress, between "traits"
(base classes), and "objects" (final classes).  Values are instances
of "object" types, but "object" types may inherit code and interfaces
from traits.  Only "object" types can define instance variables, which
means that much of the code in non-abstract "trait" methods is
eventually based on abstract method calls (and property getter/setters).

Bill

From janssen at parc.com  Mon Nov 27 02:59:35 2006
From: janssen at parc.com (Bill Janssen)
Date: Sun, 26 Nov 2006 17:59:35 PST
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ek65gu$2qg$1@sea.gmane.org> 
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<20061122165513.GA13795@niemeyer.net>
	<4564F8B7.5030007@canterbury.ac.nz>
	<20061123020054.GA31618@niemeyer.net>
	<45651355.7090605@canterbury.ac.nz>
	<20061123121416.GA6072@niemeyer.net>
	<45661AA3.9070803@canterbury.ac.nz>
	<20061123224759.GA27828@niemeyer.net>
	<456631AF.1020208@canterbury.ac.nz> <ek65gu$2qg$1@sea.gmane.org>
Message-ID: <06Nov26.175941pst."58648"@synergy1.parc.xerox.com>

> It may also be that only a good unified solution to "A" is needed in order to 
> allow library and user code to be written "easier" to address "B","C" and "D". 
> It may even be just doing "A" is a good enough 95% solution (works well enough 
> for 95% of actual use cases) and nothing more needs to be done. <It's too soon 
> to tell>

I guess this is sort of what I think.  And I hate to add yet another
framework/system to the Python system, thus my proposal to handle it
all using the existing type system, by re-factoring the various base
classes (file, sequence, number, list, mapping, string, etc.) into a
set of interface types, and re-defining the base types as various
combinations of these interface types (which may be abstract, or
concrete -- we haven't really talked enough about specifics to know
yet).

This would give us a decent base to use later (or now) for adding a
multi-method dispatching system, as well.  And optional type
annotations, etc.

Bill

From janssen at parc.com  Mon Nov 27 03:16:10 2006
From: janssen at parc.com (Bill Janssen)
Date: Sun, 26 Nov 2006 18:16:10 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
	<fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>
Message-ID: <06Nov26.181616pst."58648"@synergy1.parc.xerox.com>

> I had thought that you (Bill), (along with PJE) were assuming that the
> generic function or interface itself would serve to distinguish the
> namespace.  Instead of
> 
>     class A(object):
>         def magic_meth_foo(self, ...):
> 
> I would write
> 
>     class A(object):
>         defop foo.magic_meth(self, ...)
> 

There seem to be two justifications for "special" methods that have
been adduced (though I see no real justification for the name-mangling
scheme).  One is that they serve as a way of specifying which code
operators and their cousins, the "built-in" generic functions, call.
The other is that they somehow serve as connections to the underlying
VM implementation.  Now, I don't see why the connection to the
underlying VM implementation needs name-mangling -- it would be
reasonable just to define a distinguished module (call it, say, for
tradition's sake, "__VM__" :-), and put them in that module.

But there does need to be some way to connect operator syntax to code,
and we could talk about how to do that.  Defop is one solution; you
could also imagine an ABC for each operator, and classes which wish to
use that operator, or overload it, would inherit from that ABC and
re-define the methods.  The question is, would those methods, bound to
specific operators, be so pervasive and so common that their logical
method names would, if not mangled, occupy too much of the available
naming space?  Maybe.  I think it's a judgement call.

Bill

From ross at sourcelabs.com  Mon Nov 27 06:09:29 2006
From: ross at sourcelabs.com (Ross Jekel)
Date: Sun, 26 Nov 2006 21:09:29 -0800 (PST)
Subject: [Python-3000] Generic function queries
In-Reply-To: <238202539.33881164602206006.JavaMail.root@mail-2.colo.sourcelabs.com>
Message-ID: <1017234058.33901164604169480.JavaMail.root@mail-2.colo.sourcelabs.com>

Hi,

I've been reading the discussions on interfaces and generic functions with much interest. Some people have expressed concerns that readability and code discovery (where is the code that this call is going to run) may suffer. Could someone please address some simple questions for me about this as I'm not familiar with using generics in large projects?

For a reusable library of code that includes generics, does the library writer typically try to limit exception types that can pass out of the generic function? In other words, I guess what I'm asking is if there is way a for a library writer to register an "around" rule that would always be outermost so that one could wrap everything including later overrides in a try: except: or a transaction or something like that. If this is a *common* pattern would it be better handled by registering one or more context managers for the generic function? I haven't fully thought this out yet, but it seems like if context managers were useful for a suite of code using the "with" statement, that they could be just as applicable for managing the context of the suites of code reachable through the generic function dispatch. I leave syntax proposals for later if this is a useful idea, but some obvious ones come to mind.

I understand addmethod and hasmethod as have been proposed, but is there a need (for unit tests or other situations) to have a function that would return the concrete function object or list of function objects (if there is a chain) for a particular set of arguments without actually calling them (a getmethod or querymethod operation)?

If I had a generic function f, could I interactively do:

dir(f)

to begin discovering concrete functions associated with the generic, or there be a separate inspection mechanism?

What does help(f) do? I think Phillip said something about this, but I don't understand it from the usability aspect (i.e. Does help merge all docstrings from known implementations, provide the docstring of the default?)

Ross



From ironfroggy at gmail.com  Mon Nov 27 06:51:39 2006
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Mon, 27 Nov 2006 00:51:39 -0500
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <ca471dc20611261736y5944e669t8c6e489376caa65a@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<76fd5acf0611221959g74e8d7a9o36723f63814aa950@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
	<ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
	<76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>
	<ca471dc20611242129na1f48d5o2efb1983139c1e23@mail.gmail.com>
	<76fd5acf0611252028i496ef6d7r9b23674bcb38046e@mail.gmail.com>
	<ca471dc20611261736y5944e669t8c6e489376caa65a@mail.gmail.com>
Message-ID: <76fd5acf0611262151l7abc50bekd5bef41ca9311d0e@mail.gmail.com>

On 11/26/06, Guido van Rossum <guido at python.org> wrote:
> On 11/25/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> > I will make only one more comment and then ill drop my comments
> > without direct questions.
> >
> > On 11/25/06, Guido van Rossum <guido at python.org> wrote:
> > > Hm. The double colon rubs me the wrong way (Perl and/or C++). But
> > > apart from that, if this is the solution, I'm not sure the problem
> > > you're trying to solve is really worth solving. I just don't expect
> > > there will be all that many generic operations that need to be stuffed
> > > into arbitrary classes. Maybe I'll just take back what I said about
> > > wanting all such operations to be inside the class. Or maybe I'd be
> > > happier if there was a decorator indicating the name of the operation.
> >
> > I don't care about the syntax. anything that can denote an expression
> > (the left of the ::) and a name (the right of the ::) is OK and I dont
> > care how its denoted.
> >
> > > Also, I think you're overloading 'def' in a non-obvious way.
> > > Currently, "def foo..." means an assignment to the local variable foo.
> > > I would expect that if we extend the syntax for the thing between
> > > 'def' and the argument list to be more than just a name, it should
> > > still be considered an assignment target. But that's clearly not what
> > > you're after. From that POV, I find defop (while still unpleasant for
> > > other reasons) more "honest" than your overloading of def -- at least
> > > defop says upfront that it's not just an assignment. (Although the
> > > similarity with def is still confusing IMO.)
> >
> > You must be misunderstanding me. I am not saying that its not an
> > assignment. It would not change what def really means. operator::len
> > would be the actual name of the function to be created and the name of
> > the global, local, or class attribute it is bound to. I am saying
> > operator::len would become something like
> > MyClass.__dict__[operator::len] and what operator::len evaluates to, i
> > dont know. something that represents what it is. Maybe just a tuple. I
> > would expect it also exist for any assignment. special casing being
> > bad and all.
>
> Do you realize that expr[expr::expr] already has a meaning?

Yes. As I've stated repeatedly, I don't care for or encourage the a::b
syntax itself, but am using it as a representation of whatever might
become used. I don't know what, at this point.

> You seem to be doing an exceptionally poor job explaining your
> proposal; you are introducing a new token "::" without ever explaining
> what you propose it should mean. How does operator::len differ from
> operator.len?

A.b evaluates to the value of attribute 'b' of some object bound to
name 'A', of course.
A::b (or whatever syntax might be used) would evaluate to some value
that actually represents "The attribute 'b' of this object". This
"attribute object" (poor term, yes) could have other uses, but thats
not completely relevent now. For the case at hand, basically we would
allow the syntax for these objects to be used as legal names (keys in
the __dict__ or elements of the __slots__). Thus, the way in
`A.__len__` the string '__len__' is created and looked up in the
dictionary, with `A.operator::len` (or whatever syntax is used)
<ObjectAttribute operator module, attribute 'len'> would be the key in
the __dict__, the actual name of the attribute of the A class.

Basically, it allows for using any object in combination with a name
to have identifiers that do not conflict. Thus we can have all the
operator methods we want and never conflict with names others might
use. The interface libraries can operate without worry. Lots of
problems solved.

I am sorry if my thoughts are not conveyed as clear as they sound in my mind.

> > > Still not convinced? Focus on other problems first. This isn't the
> > > most important problem we're trying to solve.
> > >
> > > PS, It's __builtin__, not __builtins__ -- the latter is an
> > > unfortunately named but ultimately unimportant implementation detail
> > > (unimportant unless you're implemented restricted python, that is);
> > > the former is the module that is implied at the end of every free name
> > > search.
> > >
> > >
> > > --Guido
> >
> > --
> > Read my blog! I depend on your acceptance of my opinion! I am interesting!
> > http://ironfroggy-code.blogspot.com/
> > _______________________________________________
> > Python-3000 mailing list
> > Python-3000 at python.org
> > http://mail.python.org/mailman/listinfo/python-3000
> > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/

From fredrik at pythonware.com  Mon Nov 27 08:01:26 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 27 Nov 2006 08:01:26 +0100
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <06Nov26.181616pst."58648"@synergy1.parc.xerox.com>
References: <-6683921950610559197@unknownmsgid>	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>	<ek2iig$gni$1@sea.gmane.org>
	<3497511880590744734@unknownmsgid>	<fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>
	<06Nov26.181616pst."58648"@synergy1.parc.xerox.com>
Message-ID: <eke2g6$i9m$1@sea.gmane.org>

Bill Janssen wrote:

> Now, I don't see why the connection to the
> underlying VM implementation needs name-mangling -- it would be
> reasonable just to define a distinguished module (call it, say, for
> tradition's sake, "__VM__" :-), and put them in that module.

putting methods in a module?  how would that work?

</F>


From jimjjewett at gmail.com  Mon Nov 27 14:30:07 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 27 Nov 2006 08:30:07 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611261740m111f537es66c54d94b73c016@mail.gmail.com>
References: <4b57b0700611211559y20e84b5fp2198a15d9f75e59f@mail.gmail.com>
	<5.1.1.6.0.20061121201100.038cf008@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122102152.01f2f350@sparrow.telecommunity.com>
	<02AFCDEB-25D2-4D10-9BE4-A2773CCED65C@cleverdevil.org>
	<ca471dc20611221903nf9a6c1am1d70a5cba088c38d@mail.gmail.com>
	<fb6fbf560611251856h6a49c11ek373e8ff25c5c908@mail.gmail.com>
	<ca471dc20611261740m111f537es66c54d94b73c016@mail.gmail.com>
Message-ID: <fb6fbf560611270530q26b2b96dkfcad7ed5cffea32e@mail.gmail.com>

On 11/26/06, Guido van Rossum <guido at python.org> wrote:
> On 11/25/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> > Personally, I stand by my assessment that they can make metaclasses
> > look straightforward -- but that is partly because metaclasses have
> > had their roughest edges worn off already.  (For example, they no
> > longer require compiling a C class so odd that the pattern gets named
> > after someone.)  If GF explanations start to come from several
> > different people, the same process should eventually simplify them.
> > If that happened quickly enough, it would change the tradeoffs.
>
> Did you read my blog post where I give my own interpretation and implementation?
>
> http://www.artima.com/weblogs/viewpost.jsp?thread=155514


Yes, and also the comments.

If overloaded functions were proposed as an optional 3rd party module
(a possibility you did mention), they would be interesting.

When they are proposed as a fundamental part of the language itself,
it is crucial that there be a quick, easy explanation available as
soon as people need to work with them.  For metaclasses, people don't
need to work with them for quite a while.  For something like print or
len ... they need it sooner.  (And 3rd party registration means that
someone else's misunderstanding can cause bugs in your code.)

I felt that there was still too much confusion for something that is
proposed as a fundamental building block.

I expect that the confusion would get worse once you have to start
debugging multiple 3rd-party overloads.   The "TypeError: hex()
argument can't be converted to hex" is a hint at the problems, but I
expect them to be worse when the chosen function is a match -- just
not the one you expected.

-jJ

From gsakkis at rutgers.edu  Mon Nov 27 15:21:29 2006
From: gsakkis at rutgers.edu (George Sakkis)
Date: Mon, 27 Nov 2006 09:21:29 -0500
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <eke2g6$i9m$1@sea.gmane.org>
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
	<fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>
	<eke2g6$i9m$1@sea.gmane.org>
Message-ID: <91ad5bf80611270621v193cc81cv86b4b5577eef6bfc@mail.gmail.com>

On 11/27/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Bill Janssen wrote:
>
> > Now, I don't see why the connection to the
> > underlying VM implementation needs name-mangling -- it would be
> > reasonable just to define a distinguished module (call it, say, for
> > tradition's sake, "__VM__" :-), and put them in that module.
>
> putting methods in a module?  how would that work?
>
> </F>

How about a nested class, like Meta in Django ?

George

From janssen at parc.com  Mon Nov 27 18:07:37 2006
From: janssen at parc.com (Bill Janssen)
Date: Mon, 27 Nov 2006 09:07:37 PST
Subject: [Python-3000] Special methods and interface-based type system
In-Reply-To: <eke2g6$i9m$1@sea.gmane.org> 
References: <-6683921950610559197@unknownmsgid>
	<ca471dc20611221209q4bcd9f8exe9cf13bab15adff3@mail.gmail.com>
	<ek2iig$gni$1@sea.gmane.org> <3497511880590744734@unknownmsgid>
	<fb6fbf560611251603p71c1c7f6h48eace7ca19e1a91@mail.gmail.com>
	<06Nov26.181616pst."58648"@synergy1.parc.xerox.com>
	<eke2g6$i9m$1@sea.gmane.org>
Message-ID: <06Nov27.090742pst."58648"@synergy1.parc.xerox.com>

> putting methods in a module?  how would that work?

You'd define classes and functions intended for VM coupling in that
module's namespace.  They might (or might not, depending on the VM)
actually be implemented by the VM, but to Python code they would
appear in that namespace.

Bill

From guido at python.org  Mon Nov 27 18:41:26 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 27 Nov 2006 09:41:26 -0800
Subject: [Python-3000] Fwd: defop ?
In-Reply-To: <76fd5acf0611262151l7abc50bekd5bef41ca9311d0e@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611222042i36041c43p1b374068fc19a083@mail.gmail.com>
	<76fd5acf0611222310x64f03aa3i3d516c30452b1d79@mail.gmail.com>
	<76fd5acf0611222310h1f6e4137r1dee4afa7a747ef0@mail.gmail.com>
	<ca471dc20611230838g13672decueed84d58fae037f1@mail.gmail.com>
	<76fd5acf0611241835m73186e80h7eb0e9eb3fa878ff@mail.gmail.com>
	<ca471dc20611242129na1f48d5o2efb1983139c1e23@mail.gmail.com>
	<76fd5acf0611252028i496ef6d7r9b23674bcb38046e@mail.gmail.com>
	<ca471dc20611261736y5944e669t8c6e489376caa65a@mail.gmail.com>
	<76fd5acf0611262151l7abc50bekd5bef41ca9311d0e@mail.gmail.com>
Message-ID: <ca471dc20611270941x16b04efcxb48cf3dc0f7b0894@mail.gmail.com>

On 11/26/06, Calvin Spealman <ironfroggy at gmail.com> wrote:
> A.b evaluates to the value of attribute 'b' of some object bound to
> name 'A', of course.
> A::b (or whatever syntax might be used) would evaluate to some value
> that actually represents "The attribute 'b' of this object". This
> "attribute object" (poor term, yes) could have other uses, but thats
> not completely relevent now. For the case at hand, basically we would
> allow the syntax for these objects to be used as legal names (keys in
> the __dict__ or elements of the __slots__). Thus, the way in
> `A.__len__` the string '__len__' is created and looked up in the
> dictionary, with `A.operator::len` (or whatever syntax is used)
> <ObjectAttribute operator module, attribute 'len'> would be the key in
> the __dict__, the actual name of the attribute of the A class.
>
> Basically, it allows for using any object in combination with a name
> to have identifiers that do not conflict. Thus we can have all the
> operator methods we want and never conflict with names others might
> use. The interface libraries can operate without worry. Lots of
> problems solved.
>
> I am sorry if my thoughts are not conveyed as clear as they sound in my mind.

No worry. It is now completely clear. The idea is interesting and it's
a useful one to have handy, even without a concrete syntax proposal.
Thanks for hanging in there!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jimjjewett at gmail.com  Mon Nov 27 20:01:55 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 27 Nov 2006 14:01:55 -0500
Subject: [Python-3000] Abilities / Interfaces
In-Reply-To: <ca471dc20611221043q5ab8c55vdb63ed41e1bf7d5f@mail.gmail.com>
References: <ca471dc20611201027q2f0d8ec9u5ed0c60a92ef6535@mail.gmail.com>
	<ca471dc20611211116v42d2a3b4p81a6768cdfcb2ac1@mail.gmail.com>
	<5.1.1.6.0.20061122022615.037de350@sparrow.telecommunity.com>
	<5.1.1.6.0.20061122130031.041f4960@sparrow.telecommunity.com>
	<ca471dc20611221043q5ab8c55vdb63ed41e1bf7d5f@mail.gmail.com>
Message-ID: <fb6fbf560611271101tddfde5dte73472cc50d8af43@mail.gmail.com>

On 11/22/06, Guido van Rossum <guido at python.org> wrote:
> On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 09:24 AM 11/22/2006 -0800, Bill Janssen wrote:
> > In Java, SMTP.sendmail would be something like this (using Python-like
> > syntax 'cause my Java is rusty):

> >      def sendmail(self, from, to_addrs:str, msg, ...):
> >          return self.sendmail(from_addr, [to_addrs], msg, ...)

> >      def sendmail(self, from, to_addrs:list[str], msg, ...):
> >          # main implementation

> Right. If this syntax was possible in Python lots of people would be
> very happy. But even the best generic function API I've seen is a lot
> more verbose than this -- there seems to be a separate set-up
> involved.

3rd-party registration is fundamental to extensible functions.  The
best you can do is to minimize it.


    def sendmail(self, from, to_addrs, msg, ...):
        raise TypeError(to_addrs must be a string or list)

    overload sendmail(self, from, to_addrs:str, msg, ...):
        return self.sendmail(from_addr, [to_addrs], msg, ...)

    overload sendmail(self, from, to_addrs:list[str], msg, ...):
        # main implementation

And then someone else could write

overload smtplib.SMTP.sendmail(self, from, to_addrs:Seq[my_str], msg, ...):
    to_addrs = [str(addr) for addr in to_addrs]
    return smtplib.SMTP.sendmail(self, from, to_addrs:list[str], msg, ...)

From jimjjewett at gmail.com  Mon Nov 27 20:19:22 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 27 Nov 2006 14:19:22 -0500
Subject: [Python-3000] Abilities / Interfaces and security
Message-ID: <fb6fbf560611271119o61279f79ha6ca726d2e0ff038@mail.gmail.com>

On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:

> While this is more verbose than sticking an 'if' into the original
> sendmail(), note that it can also be added by a third party, via:

>      overload smtpblib.SMTP.sendmail(
>          self, from_addr, to_addrs:str, msg,
>          mail_options=[], rcpt_options=[]
>      ):
>          self.sendmail(from_addr, [to_addrs], msg, mail_options, rcpt_options)

> So, someone who didn't want to wait for the patch to add your "if" could
> just add their own fix on the fly.  :)

Hey, cool, can I also do

    overload smtplib.SMTP.sendmail(self, from_addr:str, to_addrs:str, msg, ...)
        if stalkee == from_addr:
            archive_and_randomly_rewrite_before_sending(...)

Since this is more specific on the from_addr, it should always be called.

Also, can I

    overload __builtins__.str(arg:object)

to effectively extend (or replace) the builtin type?

There is certainly value in open classes, and it can make writing code
easier.  But it can also make maintaining code much harder.  Today,
almost all classes are open, but those coded in C are typically closed
-- and Brett is relying on this for security.
How-it-happened-to-be-implemented may not be the best way to decide
open/closed, but I think there needs to be *some* way of keeping a
class closed.

-jJ

From jimjjewett at gmail.com  Mon Nov 27 20:34:33 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Mon, 27 Nov 2006 14:34:33 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
	<ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
	<ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>
Message-ID: <fb6fbf560611271134h15797a4fhb24d4c4f8b63a9e6@mail.gmail.com>

On 11/24/06, Guido van Rossum <guido at python.org> wrote:
> Thinking about my own example some more, I think it would actually
> work just as well using ABCs instead of abilities. ... It seems the use
> case for making abilities/interfaces separate from ABCs hinges on
> the assumption that there are classes that implement certain
> protocols without realizing it, IOW emergent
> protocols, plus that it's not practical to change __bases__.

In some cases, the interface might require slight adaptation, such as
aliasing a function.  But that is something which can be handled
better by adding a new base class.

> I wonder if a bunch of well thought-out standard ABCs, applied to the
> standard data types, and perhaps more support for setting __bases__,
> wouldn't address most concerns.

I think so.  A couple thoughts:

The problem with setting bases (other than extension classes) is
largely a style issue -- it can be done now.  The code for setting
bases even has comments suggesting how to speed it up, but saying that
it isn't worth it because bases shouldn't change much.

It might be worth adding an __interfaces__ that gets searched after
__bases__.  The distinction would indicate that an interface is
lightweight, may change, and (for security and efficiency) cannot
override existing methods.  I couldn't change the builtin
str.__contains__, but I could add myproto to str.__interfaces__, and
then I could make calls to "asdf".__interfaces__.myproto.__contains__
for arbitrary strings.

I'm not sure that adding it to the builtin (but in a new name) would
really buy much over an adapter (x in MyProto("asdf") ), but the
interface fans may feel differently.

-jJ

From tim.hochberg at ieee.org  Mon Nov 27 20:53:29 2006
From: tim.hochberg at ieee.org (Tim Hochberg)
Date: Mon, 27 Nov 2006 12:53:29 -0700
Subject: [Python-3000] Abilities / Interfaces and security
In-Reply-To: <fb6fbf560611271119o61279f79ha6ca726d2e0ff038@mail.gmail.com>
References: <fb6fbf560611271119o61279f79ha6ca726d2e0ff038@mail.gmail.com>
Message-ID: <ekffo4$iak$1@sea.gmane.org>

Jim Jewett wrote:
> On 11/22/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> 
>> While this is more verbose than sticking an 'if' into the original
>> sendmail(), note that it can also be added by a third party, via:
> 
>>      overload smtpblib.SMTP.sendmail(
>>          self, from_addr, to_addrs:str, msg,
>>          mail_options=[], rcpt_options=[]
>>      ):
>>          self.sendmail(from_addr, [to_addrs], msg, mail_options, rcpt_options)
> 
>> So, someone who didn't want to wait for the patch to add your "if" could
>> just add their own fix on the fly.  :)
> 
> Hey, cool, can I also do
> 
>     overload smtplib.SMTP.sendmail(self, from_addr:str, to_addrs:str, msg, ...)
>         if stalkee == from_addr:
>             archive_and_randomly_rewrite_before_sending(...)
> 
> Since this is more specific on the from_addr, it should always be called.

I suppose this issue may be real, but this example seems like FUD; for 
most classes I can completely override the behavior today simply by 
monkey patching the method and this isn't typeically a problem that we see:

	smtplib.SMTP.sendmail = my_bogus_sendmail_function

> 
> Also, can I
> 
>     overload __builtins__.str(arg:object)
> 
> to effectively extend (or replace) the builtin type?
> 
> There is certainly value in open classes, and it can make writing code
> easier. 

This isn't about open/closed classes except incidentally is it? It's 
really about open/closed functions and methods.

  But it can also make maintaining code much harder.  Today,
> almost all classes are open, but those coded in C are typically closed
> -- and Brett is relying on this for security.
> How-it-happened-to-be-implemented may not be the best way to decide
> open/closed, but I think there needs to be *some* way of keeping a
> class closed.

I would expect that either functions would default to closed and you 
would get a generic function with an appropriate decorator, or functions 
would default to open and you could instead close them, again with an 
appropriate decorator. To be concrete, either:

     def my_closed_function(arg):
         #....

     @overloadable
     def my_open_function(arg):
         #....
     overload my_open_function(arg:str):
        #....

or:

     @closed  #final? nonoverloadable?
     def my_closed_function(arg):
         #....

     def my_open_function(arg):
         #....
     overload my_open_function(arg:str):
         #....

Either way, this doesn't look like a big deal. I'm more interested in 
having, for debugging purposes, a way to do some introspection on 
generic functions. For instance, finding out what function would be 
called for a given set of arguments and perhaps some information on why 
that match was chosen. Also, what overloads are set up and with what 
signatures.

-tim




From pje at telecommunity.com  Mon Nov 27 21:25:14 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 27 Nov 2006 15:25:14 -0500
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <fb6fbf560611271134h15797a4fhb24d4c4f8b63a9e6@mail.gmail.co
 m>
References: <ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>
	<ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
	<ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
	<ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>
Message-ID: <5.1.1.6.0.20061127151739.028664a8@sparrow.telecommunity.com>

At 02:34 PM 11/27/2006 -0500, Jim Jewett wrote:
>I'm not sure that adding it to the builtin (but in a new name) would
>really buy much over an adapter (x in MyProto("asdf") ), but the
>interface fans may feel differently.

For what it's worth, in Chandler there is a system called Annotations that 
allows third-party packages to add private attributes to existing 
(persistent) data types.  It's used to implement "stamping", which is a way 
that you can (for example) dynamically change a note or photo into a task 
or a calendar event.

So, if I want to be able to add calendar information to "stampable" 
objects, I can do something like:

     class CalendarEvent(Stamp):
         # field definitions here
         # method definitions

And then I can access fields on stamped items by using e.g. 
CalendarEvent(someItem).startTime.  So, effectively CalendarEvent is a kind 
of adapter class, but we don't use the word adapter, simply because it 
isn't relevant to the application domain.  It's simply a way of having an 
open system for adding additional data to existing object types.  There are 
no interfaces or generic functions involved, either, it's just a way of 
separating namespaces belonging to independent developers.

I just mention it because the reaction to it among developers with no 
previous exposure to Python adaptation has been fairly positive.


From brett at python.org  Mon Nov 27 21:35:20 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 27 Nov 2006 12:35:20 -0800
Subject: [Python-3000] optional argument annotations
In-Reply-To: <EAEDCB4B-DC66-435A-AC5C-D111167FC020@pagedna.com>
References: <33D8CBFC-9B3B-4FDE-927F-4B6D6B11C434@PageDNA.com>
	<ca471dc20611231855sfe2df48ld63bbcef2492dbaf@mail.gmail.com>
	<fb6fbf560611240011m38217255hee9c31030a04308f@mail.gmail.com>
	<ca471dc20611240830x6bb8846cob2ac4f736527258d@mail.gmail.com>
	<bbaeab100611241120j6b8ea9aka67b28b7ed4e63cc@mail.gmail.com>
	<C0D2AD09-FC51-432C-87EE-BC91E8FE99D8@pagedna.com>
	<bbaeab100611261310x3bdaa5aan6534dde3b1f5b7e5@mail.gmail.com>
	<EAEDCB4B-DC66-435A-AC5C-D111167FC020@pagedna.com>
Message-ID: <bbaeab100611271235j2ab03d2i1f09286954612e4@mail.gmail.com>

On 11/26/06, Tony Lownds <tony at pagedna.com> wrote:
>
>
> On Nov 26, 2006, at 1:10 PM, Brett Cannon wrote:
>
> On 11/24/06, Tony Lownds <tony at pagedna.com> wrote:
> >
> > > Obviously signature objects would grow support for annotations, but I
> > > still need the information to be carried on the code object to
> > > incorporate into signature objects.
> > >
> >
> > Signature objects still need a way to know the nested parameters, right?
> >
>
>
> They already handle them.
>
>
> I see, through the bytecode inspection that inspect.py does.
>

Yep.

How about a co_argnames attribute? eg for
> >
> > def f((x, y), z): pass
> >
> > f.func_code.co_argnames would be (('x', 'y'), 'z')
>
>
> That's fine, but the compiler would need to change if it were to use
> this.  Plus I am still hoping to make nested parameters disappear in Py3K (I
> won't be pushing for it any sooner than PyCon, though).
>
>
> Yes, it is the compiler that would implement this. I have it implemented
> as follows.
>
> >>> def f(a, (b, c), d=1, *e, f, g=1, **h): pass
> ...
> >>> f.func_code.co_argnames
> ('a', ('b', 'c'), 'd', '*e', 'f', 'g', '**h')
>
> However since inspect.py doesn't need this and neither does my code, I'll
> drop it.
>

OK.

I need to implement something like this to properly build
> > func_annotations
> > inside MAKE_FUNCTION.
>
>
>
> I don't quite follow.  Don't you already have support in MAKE_FUNCTION
> when the object is created?
>
>
> The support available is the code object and anything pushed onto the
> stack. Just looking at the code object,
> the information is simply not available outside of reverse-engineering
> co_code (as inspect does).
> I ended up pushing a tuple of argument names onto the stack.
>
> eg, for
>
> def f((x:1, y))->3: pass
>
> the bytecode is...
>
>   1           0 LOAD_CONST               0 (1)
>               3 LOAD_CONST               1 (3)
>               6 LOAD_CONST               2 (('x', 'return'))
>               9 LOAD_CONST               3 (<code object f at 0xb7ecd848,
> file "<str>", line 1>)
>              12 EXTENDED_ARG             3
>              15 MAKE_FUNCTION        196608
>              18 STORE_NAME               0 (f)
>              21 LOAD_CONST               4 (None)
>              24 RETURN_VALUE
>
> (the argument is so big because I need to pass the # of annotations in the
> argument of MAKE_FUNCTION so
> that the stack effect can be easily calculated)
>

Ah, I see how you are doing it.  Well, as I said, as long as the info is
available in some way that can be used to infer what the type annotations
are per argument then I should be fine.  It doesn't have to be overly clean
just for signature objects.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061127/eb33f641/attachment.htm 

From talin at acm.org  Tue Nov 28 10:22:07 2006
From: talin at acm.org (Talin)
Date: Tue, 28 Nov 2006 01:22:07 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
Message-ID: <456BFFBF.9000208@acm.org>

More on the 'cruft removal' topic:

I notice that a number of fields in PyTypeObject are deprecated; the 
question is, how to get rid of them without gratuitously breaking 
everything? Even if there's zero code out there that still uses 
tp_getattr instead of tp_getattro, you can't simply remove the field 
because it takes up a placeholder slot in the C initializer list.

Has anyone proposed the notion of getting away from C-style initializer 
lists, at least for the case of PyTypeObject?

For example, consider the case for tp_init and tp_new. Currently these 
are filled in with the C initializer list for PyTypeObject. Because 
these are the 38th and 40th fields in PyTypeObject, you have to put 37 
initializer values before them, whether you use them or not.

Instead of doing that, however, you could put them in the method table, 
tp_methods:

    static PyMethodDef Noddy_methods[] = {
        { SPECMETH_NEW, (PyCFunction)Noddy_new, METH_VARARGS, "..." },
        { SPECMETH_INIT, (PyCFunction)Noddy_init, METH_VARARGS, "..." },
        { NULL }  /* Sentinel */
    };

'SPECMETH_NEW' and 'SPECMETH_INIT' are sentinel values indicating that 
these are 'special' methods, which are ignored during the normal 
scanning of this table. Instead, when the type is initialized, the 
method table is scanned for these special methods, and the corresponding 
field in PyTypeObject is filled in with the method pointer. The same 
holds true for all of the other special method pointers.

Now, you still have the problem that 'tp_methods' itself is something 
like the 28th field in the struct, which means you still have all of 
those empty fields to consider. So instead, make a new version of 
PyType_Ready which takes an optional parameter of the method table for 
the type, which is filled in as part of the initialization.

You could add another parameter for filling of data fields, which would 
point to a similar table that would hold values for non-function 
initializers.

What you end up with is code that looks like this:

PyTypeObject myType = {
     PyObject_HEAD_INIT(NULL)
     0,
     "myType",
     sizeof(myInstance)
}

void init() {
     if (PyType_ReadyInit( &myType, myTypeMethods, myTypeData ) < 0)
	return;
}

Note that this requires no change to the PyTypeObject struct itself, or 
of any code that currently uses it - the only thing that changes is the 
way that the struct's data fields are filled in.

What this would give you is a way to construct types that would be 
immune to future revisions of the PyTypeObject struct; You could add, 
remove, rename, or re-order the fields in the struct without breaking 
existing code.

(If there's an existing proposal for this, feel free to post a 
reference. I did look :)

-- Talin

From fredrik at pythonware.com  Tue Nov 28 10:35:48 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 28 Nov 2006 10:35:48 +0100
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456BFFBF.9000208@acm.org>
References: <456BFFBF.9000208@acm.org>
Message-ID: <ekgvtj$dr9$1@sea.gmane.org>

Talin wrote:

> Has anyone proposed the notion of getting away from C-style initializer 
> lists, at least for the case of PyTypeObject?

yes, and it was mentioned in the python-dev summaries that were mailed 
out a couple of days ago:

     http://effbot.org/zone/idea-register-type.htm

Larry Hastings has prepared a patch for this; check the patch tracker or 
the python-dev archives for details.

</F>


From greg.ewing at canterbury.ac.nz  Tue Nov 28 11:42:04 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Nov 2006 23:42:04 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456BFFBF.9000208@acm.org>
References: <456BFFBF.9000208@acm.org>
Message-ID: <456C127C.8080502@canterbury.ac.nz>

Talin wrote:

> What you end up with is code that looks like this:
> 
> PyTypeObject myType = {
>      PyObject_HEAD_INIT(NULL)
>      0,
>      "myType",
>      sizeof(myInstance)
> }
> 
> void init() {
>      if (PyType_ReadyInit( &myType, myTypeMethods, myTypeData ) < 0)
> 	return;
> }

If you're going that far, why not go a step further and do
away with the statically-declared type object altogether?

   PyTypeObject *myType;

   myType = PyType_Create(sizeof(myInstance), myTypeMethods, myTypeData);

--
Greg

From talin at acm.org  Tue Nov 28 11:53:51 2006
From: talin at acm.org (Talin)
Date: Tue, 28 Nov 2006 02:53:51 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C127C.8080502@canterbury.ac.nz>
References: <456BFFBF.9000208@acm.org> <456C127C.8080502@canterbury.ac.nz>
Message-ID: <456C153F.4070108@acm.org>

Greg Ewing wrote:
> Talin wrote:
> 
>> What you end up with is code that looks like this:
>>
>> PyTypeObject myType = {
>>      PyObject_HEAD_INIT(NULL)
>>      0,
>>      "myType",
>>      sizeof(myInstance)
>> }
>>
>> void init() {
>>      if (PyType_ReadyInit( &myType, myTypeMethods, myTypeData ) < 0)
>>     return;
>> }
> 
> If you're going that far, why not go a step further and do
> away with the statically-declared type object altogether?
> 
>   PyTypeObject *myType;
> 
>   myType = PyType_Create(sizeof(myInstance), myTypeMethods, myTypeData);

That makes sense - I was trying to avoid allocations (my day-job habits 
leaking through again), but this is Python after all, and avoiding a 
single memory allocation per type is silly.

> -- 
> Greg
> 

From guido at python.org  Tue Nov 28 16:39:11 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2006 07:39:11 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ekgvtj$dr9$1@sea.gmane.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
Message-ID: <ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>

Some comments:

- I'm all for trying something new here; the existing approach is
definitely aged.

- Fredrik's solution makes one call per registered method. (I don't
know if the patch he refers to follows that model.) That seems a fair
amount of code for an average type -- I'm wondering if it's too early
to worry about code bloat (I don't think the speed is going to
matter).

- Talin's solution seems to require the definition of an awful lot of
new constants -- one per slot. And a lot of special-casing in the type
initialization code to handle them because there are so many different
signatures.

- Both solutions proposed require rewriting *all* type initialization.
This is likely to require a tool that can do 99% of the work
automatically (or else extension writers will truly hate us). But then
maybe we could write a tool instead that can automatically rewrite a
type struct declaration to follow the new lay-out. Since almost all
type declarations are cloned from the examples in early Python, they
are very regular -- basically it's an optional case, a function name
or zero, and a comment with the member name.

- Can't we require a C99 compiler and use C99 struct initialization?
Then the table lines could look like

  tp_new = Noddy_new,
  tp_init = Noddy_init,

This probably means the conversion tool would be even simpler (a
couple of lines of sed would do). It has my vote if C99 is available
on Windows (GCC covers all other platforms -- vendors that don't have
a C99 compiler yet lose, it's 2006 now for X sake).

--Guido

On 11/28/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Talin wrote:
>
> > Has anyone proposed the notion of getting away from C-style initializer
> > lists, at least for the case of PyTypeObject?
>
> yes, and it was mentioned in the python-dev summaries that were mailed
> out a couple of days ago:
>
>      http://effbot.org/zone/idea-register-type.htm
>
> Larry Hastings has prepared a patch for this; check the patch tracker or
> the python-dev archives for details.
>
> </F>
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Tue Nov 28 17:19:54 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 28 Nov 2006 17:19:54 +0100
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
Message-ID: <ekhnja$3q5$1@sea.gmane.org>

Guido van Rossum wrote:

> - Fredrik's solution makes one call per registered method. (I don't
> know if the patch he refers to follows that model.) That seems a fair
> amount of code for an average type -- I'm wondering if it's too early
> to worry about code bloat (I don't think the speed is going to
> matter).

too early, I think.

and memory is relatively cheap, compare to the costs of upgrade pain,
programmer time, and lack of optimization opportunities due to "bare
data structures".

> - Both solutions proposed require rewriting *all* type initialization.
> This is likely to require a tool that can do 99% of the work
> automatically (or else extension writers will truly hate us).

yup.  I think a tool that generates cut-that-and-paste-this instructions
for the developer should be good enough, though, and fairly easy to
write, for the reasons you give.

> Can't we require a C99 compiler and use C99 struct initialization?

that won't address the binary compatibility and optimization issues that
are the main rationales for my proposal, though.

</F>


From Brent.Benson at oracle.com  Tue Nov 28 17:10:19 2006
From: Brent.Benson at oracle.com (Brent Benson)
Date: Tue, 28 Nov 2006 11:10:19 -0500
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
Message-ID: <456C5F6B.9090009@oracle.com>

Guido van Rossum wrote:
> - Can't we require a C99 compiler and use C99 struct initialization?
> Then the table lines could look like
>
>   tp_new = Noddy_new,
>   tp_init = Noddy_init,
>
> This probably means the conversion tool would be even simpler (a
> couple of lines of sed would do). It has my vote if C99 is available
> on Windows (GCC covers all other platforms -- vendors that don't have
> a C99 compiler yet lose, it's 2006 now for X sake).
My reading of the Microsoft Visual C++ documentation indicates that they 
do not support or plan to support C99.  On the other hand, it looks like 
the Intel V8 compiler for Windows supports most of C99 including the new 
style struct initialization.

-Brent



From jimjjewett at gmail.com  Tue Nov 28 17:59:56 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 28 Nov 2006 11:59:56 -0500
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456BFFBF.9000208@acm.org>
References: <456BFFBF.9000208@acm.org>
Message-ID: <fb6fbf560611280859m4ce7f9a6l4e53cc8233a9e091@mail.gmail.com>

On 11/28/06, Talin <talin at acm.org> wrote:
> For example, consider the case for tp_init and tp_new. ... you have
> to put 37 initializer values before them, whether you use them or not.

> Now, you still have the problem that 'tp_methods' itself is something
> like the 28th field in the struct, which means you still have all of
> those empty fields to consider.

> Note that this requires no change to the PyTypeObject struct itself, or
> of any code that currently uses it - the only thing that changes is the
> way that the struct's data fields are filled in.

> What this would give you is a way to construct types that would be
> immune to future revisions of the PyTypeObject struct; You could add,
> remove, rename, or re-order the fields in the struct without breaking
> existing code.

Am I understanding this correctly?

(1)  The change is that instead of a header structure filled in with
zeros, you have a collection of function calls.  (And this secretly
makes that zero-filled structure for you.)  Since they're each a
function call, it may take just as much space to write out, but there
won't be any (source code) place to read the physical layout, and it
won't be as obvious that you've left a method out.

With C99 initializers, you at least get named (rather than
comment-named) fields, but I don't see any advantage to using function
calls if the type slots don't change.

(2)  In theory, it would be easier to add/remove/reorder slots between
major versions, but you wouldn't actually do this because of backwards
compatibility.

So then why make a change at all?

-jJ

From guido at python.org  Tue Nov 28 18:34:33 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2006 09:34:33 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ekhnja$3q5$1@sea.gmane.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<ekhnja$3q5$1@sea.gmane.org>
Message-ID: <ca471dc20611280934v1abf9055m5274bc92a3cc5c0f@mail.gmail.com>

On 11/28/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> > - Fredrik's solution makes one call per registered method. (I don't
> > know if the patch he refers to follows that model.) That seems a fair
> > amount of code for an average type -- I'm wondering if it's too early
> > to worry about code bloat (I don't think the speed is going to
> > matter).
>
> too early, I think.
>
> and memory is relatively cheap, compare to the costs of upgrade pain,
> programmer time, and lack of optimization opportunities due to "bare
> data structures".
>
> > - Both solutions proposed require rewriting *all* type initialization.
> > This is likely to require a tool that can do 99% of the work
> > automatically (or else extension writers will truly hate us).
>
> yup.  I think a tool that generates cut-that-and-paste-this instructions
> for the developer should be good enough, though, and fairly easy to
> write, for the reasons you give.
>
> > Can't we require a C99 compiler and use C99 struct initialization?
>
> that won't address the binary compatibility and optimization issues that
> are the main rationales for my proposal, though.

Why not? This is the Py3k list -- there is no hope for binary
compatibility with 2.x. AFAIU the C99 approach can be easily binary
compatible between 3.0, 3.1 and beyond -- please explain if this is
not so.

And what's the optimization issue? Do you mean the ccasional NULL
poitner check? Once we make PyType_Ready() obligatory, it can do the
same thing. Or whatever else you are thinking of.
-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Tue Nov 28 18:48:01 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 28 Nov 2006 18:48:01 +0100
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611280934v1abf9055m5274bc92a3cc5c0f@mail.gmail.com>
References: <456BFFBF.9000208@acm.org>
	<ekgvtj$dr9$1@sea.gmane.org>	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>	<ekhnja$3q5$1@sea.gmane.org>
	<ca471dc20611280934v1abf9055m5274bc92a3cc5c0f@mail.gmail.com>
Message-ID: <ekhsoh$nvt$1@sea.gmane.org>

Guido van Rossum wrote:

>>> Can't we require a C99 compiler and use C99 struct initialization?
>> that won't address the binary compatibility and optimization issues that
>> are the main rationales for my proposal, though.
> 
> Why not? This is the Py3k list -- there is no hope for binary
> compatibility with 2.x. AFAIU the C99 approach can be easily binary
> compatible between 3.0, 3.1 and beyond -- please explain if this is
> not so.

how can it be?  isn't the C99 syntax just a way to avoid filling all the 
zeros?  you'll still end up with a fixed-size structure in memory, with 
no way to move things around, and nowhere to put any auxiliary data.

> And what's the optimization issue? Do you mean the ccasional NULL
> poitner check? Once we make PyType_Ready() obligatory, it can do the
> same thing. Or whatever else you are thinking of.

null pointer checks, using different structures depending on object 
types, different ways to represent method lists (linear lists for short 
method lists, custom dictionaries for longer lists), and what else we 
might come with during our next trip to Reykjavik.

"naked structures" are a lot more limiting than data structures hidden 
behind simple API:s.

</F>


From talin at acm.org  Tue Nov 28 18:58:06 2006
From: talin at acm.org (Talin)
Date: Tue, 28 Nov 2006 09:58:06 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ekhnja$3q5$1@sea.gmane.org>
References: <456BFFBF.9000208@acm.org>
	<ekgvtj$dr9$1@sea.gmane.org>	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<ekhnja$3q5$1@sea.gmane.org>
Message-ID: <456C78AE.1080601@acm.org>

Fredrik Lundh wrote:
> Guido van Rossum wrote:
> 
>> - Fredrik's solution makes one call per registered method. (I don't
>> know if the patch he refers to follows that model.) That seems a fair
>> amount of code for an average type -- I'm wondering if it's too early
>> to worry about code bloat (I don't think the speed is going to
>> matter).
> 
> too early, I think.

I agree with this, especially given that I think that this should be 
*immediately* backported to 2.6, so that developers can gradually 
transition over; So that by the time Py3K comes out, there will be fewer 
  existing libraries to convert.

> and memory is relatively cheap, compare to the costs of upgrade pain,
> programmer time, and lack of optimization opportunities due to "bare
> data structures".
> 
>> - Both solutions proposed require rewriting *all* type initialization.
>> This is likely to require a tool that can do 99% of the work
>> automatically (or else extension writers will truly hate us).
> 
> yup.  I think a tool that generates cut-that-and-paste-this instructions
> for the developer should be good enough, though, and fairly easy to
> write, for the reasons you give.
> 
>> Can't we require a C99 compiler and use C99 struct initialization?
> 
> that won't address the binary compatibility and optimization issues that
> are the main rationales for my proposal, though.
> 
> </F>
> 
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org
> 

From janssen at parc.com  Tue Nov 28 19:03:24 2006
From: janssen at parc.com (Bill Janssen)
Date: Tue, 28 Nov 2006 10:03:24 PST
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com> 
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
	<ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
	<ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>
Message-ID: <06Nov28.100326pst."58648"@synergy1.parc.xerox.com>

Guido van Rossum wrote:
> I wonder if a bunch of well thought-out standard ABCs, applied to the
> standard data types, and perhaps more support for setting __bases__,
> wouldn't address most concerns.

I think it would, but I don't know if we're going to get anywhere
without getting more concrete.  Let's tap "the wisdom of crowds".  I've
set up a Wiki page at http://wiki.python.org/moin/AbstractBaseClasses,
and spent a little time defining some ABCs and concrete classes.  I'll
work on it more over the week, but feel free to pitch in and expand it.
Or create alternative versions (create a new Wiki page, and put a link
to it at the top of the existing page).

Bill

From guido at python.org  Tue Nov 28 19:12:51 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2006 10:12:51 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ekhsoh$nvt$1@sea.gmane.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<ekhnja$3q5$1@sea.gmane.org>
	<ca471dc20611280934v1abf9055m5274bc92a3cc5c0f@mail.gmail.com>
	<ekhsoh$nvt$1@sea.gmane.org>
Message-ID: <ca471dc20611281012w1431cd39le1cec23a5f13b8a1@mail.gmail.com>

On 11/28/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> >>> Can't we require a C99 compiler and use C99 struct initialization?
> >> that won't address the binary compatibility and optimization issues that
> >> are the main rationales for my proposal, though.
> >
> > Why not? This is the Py3k list -- there is no hope for binary
> > compatibility with 2.x. AFAIU the C99 approach can be easily binary
> > compatible between 3.0, 3.1 and beyond -- please explain if this is
> > not so.
>
> how can it be?  isn't the C99 syntax just a way to avoid filling all the
> zeros?  you'll still end up with a fixed-size structure in memory, with
> no way to move things around, and nowhere to put any auxiliary data.

We can use the same approach to extending the structure in a binary
compatible way as today. You don't need to move things around since
the initialization order doesn't need to match the order of the
members in the struct, so strictly adding to the end isn't a problem.

> > And what's the optimization issue? Do you mean the ccasional NULL
> > poitner check? Once we make PyType_Ready() obligatory, it can do the
> > same thing. Or whatever else you are thinking of.
>
> null pointer checks, using different structures depending on object
> types, different ways to represent method lists (linear lists for short
> method lists, custom dictionaries for longer lists), and what else we
> might come with during our next trip to Reykjavik.

That could all be handled by PyType_Ready(), once we allow for some
moderate extensibility in the 3.0 base struct.

> "naked structures" are a lot more limiting than data structures hidden
> behind simple API:s.

OTOH I like a data-driven approach for initialization better than a
code-driven approach. Perhaps Talin's way is a reasonable compromise?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Nov 28 19:14:43 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2006 10:14:43 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C78AE.1080601@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<ekhnja$3q5$1@sea.gmane.org> <456C78AE.1080601@acm.org>
Message-ID: <ca471dc20611281014m3b178000jf60c11a05348758f@mail.gmail.com>

On 11/28/06, Talin <talin at acm.org> wrote:
> Fredrik Lundh wrote:
> > Guido van Rossum wrote:
> >
> >> - Fredrik's solution makes one call per registered method. (I don't
> >> know if the patch he refers to follows that model.) That seems a fair
> >> amount of code for an average type -- I'm wondering if it's too early
> >> to worry about code bloat (I don't think the speed is going to
> >> matter).
> >
> > too early, I think.
>
> I agree with this,

Sure, forget I mentioned the efficiency thing.

> especially given that I think that this should be
> *immediately* backported to 2.6, so that developers can gradually
> transition over; So that by the time Py3K comes out, there will be fewer
> existing libraries to convert.

Not *immediately* -- only after a lot of people agree and a working
patch and conversion strategy has been defined.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From talin at acm.org  Tue Nov 28 19:21:47 2006
From: talin at acm.org (Talin)
Date: Tue, 28 Nov 2006 10:21:47 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
Message-ID: <456C7E3B.7090601@acm.org>

Guido van Rossum wrote:
> Some comments:
> 
> - Talin's solution seems to require the definition of an awful lot of
> new constants -- one per slot. And a lot of special-casing in the type
> initialization code to handle them because there are so many different
> signatures.

Actually, I was thinking more on this, and I have a couple ideas on how 
to avoid this, at least the part about the number of constants.

The first idea is that instead of having a constant per special method, 
simply use the regular name of the method, but have a method flag that 
indicates that the method is "special":

     static PyMethodDef Noddy_methods[] = {
         { "new", (PyCFunction)Noddy_new,
            METH_VARARGS | METH_SPECIAL, "..." },
         { "__init__", (PyCFunction)Noddy_init,
            METH_VARARGS | METH_SPECIAL, "..." },
         { NULL }  /* Sentinel */
     };

One drawback here is that not all of the fields in PyTypeObject have 
equivalent names; But there's no reason why we couldn't give them names, 
especially if the names were not valid Python identifiers.

The other drawback is that there's a greater chance of a misspelling, 
but I don't see that as a serious problem - even the most cursory 
testing of the class should discover this, it's not like the resulting 
bugs will be subtle and hard to find, IMHO.

You could go even further, and drop the "special" flag entirely, and 
there's a compelling reason why you might want to do this: It means that 
now the VM gets to decide what methods are special and what methods 
aren't. So if, for example, we decide in the future that '__unicode__' 
needs to be sped up by putting it directly in the PyTypeObject, we can 
add a slot for it, and have all existing code magically work.

It does mean that the interpretation of the method table is a bit more 
complex, but as long as you have a fast way of looking up the method 
names, and determining whether to put each method into the class dict or 
to fill in a PyTypeObject slot, I don't think it would be too bad.

This is what Fredrik is saying about the advantage over C99 
initialization, which only allows us to change the order of 
initializers, not their names or meanings.

Now, I tend to prefer using a static table vs. the 
function-call-per-method simply because of my habits, which tend to be 
overly parsimonious with code size and such (It's a side-effect of 
working on embedded systems, which is what game consoles effectively 
are.) I wouldn't have strong objections to doing it the other way.

-- Talin

From talin at acm.org  Tue Nov 28 19:54:33 2006
From: talin at acm.org (Talin)
Date: Tue, 28 Nov 2006 10:54:33 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
Message-ID: <456C85E9.8000905@acm.org>

Guido van Rossum wrote:
> Some comments:
> 
> - Fredrik's solution makes one call per registered method. (I don't
> know if the patch he refers to follows that model.) That seems a fair
> amount of code for an average type -- I'm wondering if it's too early
> to worry about code bloat (I don't think the speed is going to
> matter).

One other thought: The special constants could themselves be nothing 
more than the offset into the PyTypeObject struct, i.e.:

    #define SPECMETHOD_NEW ((const char*)offsetof(PyTypeObject,tp_new))

In the PyType_Ready code, you would see if the method name had a value 
of less than sizeof(PyTypeObject); If so, then it's a special method 
name, and you fill in the struct at the specified offset.

So the interpretation of the table could be very simple and fast. It has 
a slight disadvantage from the approach of using actual string names for 
special methods, in that it doesn't allow the VM to silently 
promote/demote methods to 'special' status.

-- Talin

From brett at python.org  Tue Nov 28 21:19:59 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 28 Nov 2006 12:19:59 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C7E3B.7090601@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org>
Message-ID: <bbaeab100611281219r764dfay99243c63f335ab2@mail.gmail.com>

On 11/28/06, Talin <talin at acm.org> wrote:
>
> Guido van Rossum wrote:
> > Some comments:
> >
> > - Talin's solution seems to require the definition of an awful lot of
> > new constants -- one per slot. And a lot of special-casing in the type
> > initialization code to handle them because there are so many different
> > signatures.
>
> Actually, I was thinking more on this, and I have a couple ideas on how
> to avoid this, at least the part about the number of constants.
>
> The first idea is that instead of having a constant per special method,
> simply use the regular name of the method, but have a method flag that
> indicates that the method is "special":
>
>      static PyMethodDef Noddy_methods[] = {
>          { "new", (PyCFunction)Noddy_new,
>             METH_VARARGS | METH_SPECIAL, "..." },
>          { "__init__", (PyCFunction)Noddy_init,
>             METH_VARARGS | METH_SPECIAL, "..." },
>          { NULL }  /* Sentinel */
>      };
>
> One drawback here is that not all of the fields in PyTypeObject have
> equivalent names; But there's no reason why we couldn't give them names,
> especially if the names were not valid Python identifiers.


Coming up with equivalent names should definitely not be difficult.  I
honestly can't think of any that don't have a matching name off the top of
my head short of tp_free/tp_dealloc.

One issue with this, though, will be the shifting of list and dict
interfaces.  They both have a __getitem__ method, but one expects only
integers and one expects anything.  Are you not going to optimize for that
and just force lists to handle the type check and cast for every call?

The other drawback is that there's a greater chance of a misspelling,
> but I don't see that as a serious problem - even the most cursory
> testing of the class should discover this, it's not like the resulting
> bugs will be subtle and hard to find, IMHO.
>
> You could go even further, and drop the "special" flag entirely, and
> there's a compelling reason why you might want to do this: It means that
> now the VM gets to decide what methods are special and what methods
> aren't. So if, for example, we decide in the future that '__unicode__'
> needs to be sped up by putting it directly in the PyTypeObject, we can
> add a slot for it, and have all existing code magically work.


That does help minimize backwards-compatibility issues.  But it does make
things implicit.

It does mean that the interpretation of the method table is a bit more
> complex, but as long as you have a fast way of looking up the method
> names, and determining whether to put each method into the class dict or
> to fill in a PyTypeObject slot, I don't think it would be too bad.
>
> This is what Fredrik is saying about the advantage over C99
> initialization, which only allows us to change the order of
> initializers, not their names or meanings.


Ah, that is what you guys are getting at !  That makes sense and is a
compelling argument.

I think with this cleanup and a standardization on calling conventions
(caller or callee checks for NULL arguments, who handles conversion to
specific  types, error return values, specifying ref stealing, etc.) this
should go a long way to make extension modules much easier to work with.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061128/a63e0793/attachment.htm 

From guido at python.org  Tue Nov 28 22:09:03 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Nov 2006 13:09:03 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C85E9.8000905@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
Message-ID: <ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>

On 11/28/06, Talin <talin at acm.org> wrote:
> Guido van Rossum wrote:
> > Some comments:
> >
> > - Fredrik's solution makes one call per registered method. (I don't
> > know if the patch he refers to follows that model.) That seems a fair
> > amount of code for an average type -- I'm wondering if it's too early
> > to worry about code bloat (I don't think the speed is going to
> > matter).
>
> One other thought: The special constants could themselves be nothing
> more than the offset into the PyTypeObject struct, i.e.:
>
>     #define SPECMETHOD_NEW ((const char*)offsetof(PyTypeObject,tp_new))

I think this would cause too many issues with backwards compatibility.

I like the idea much better to use special names (e.g. starting with a ".").

> In the PyType_Ready code, you would see if the method name had a value
> of less than sizeof(PyTypeObject); If so, then it's a special method
> name, and you fill in the struct at the specified offset.
>
> So the interpretation of the table could be very simple and fast. It has
> a slight disadvantage from the approach of using actual string names for
> special methods, in that it doesn't allow the VM to silently
> promote/demote methods to 'special' status.

I think the interpretation will be fast enough (or else what you said
about premature optimization earlier wouldn't be correct. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Wed Nov 29 01:04:10 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 29 Nov 2006 13:04:10 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C7E3B.7090601@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org>
Message-ID: <456CCE7A.6020109@canterbury.ac.nz>

Talin wrote:

> The other drawback is that there's a greater chance of a misspelling,

I don't think there is, really. It wouldn't be caught
at compile time, but it would be caught very quickly
at run time if you tried to initialise a type with a
method flagged as "special" whose name wasn't one of
the valid special method names.

> You could go even further, and drop the "special" flag entirely, and 
> there's a compelling reason why you might want to do this: It means that 
> now the VM gets to decide what methods are special and what methods 
> aren't.

No, that would be a bad idea. It would mean, for
example, that *any* method called "next" would have
to be assumed to potentially be the next() method of
an iterator and forced to have the corresponding
signature.

(This is why I changed its name to __next__ in Pyrex.
I suppose the same solution could be used here, if you
were willing to accept the name that you put in the method
table not necessarily being the name you use from Python.)

  --
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 29 01:10:41 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 29 Nov 2006 13:10:41 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C7E3B.7090601@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org>
Message-ID: <456CD001.4000907@canterbury.ac.nz>

Talin wrote:
> It means that 
> now the VM gets to decide what methods are special and what methods 
> aren't.

Further to that, the VM still gets to decide whether any
given special method gets a type slot. But the SPECIAL
flag is needed to say that you intend the method to be
*used* as a special method, which is something you can't
tell just from the name (as long as there exist special
methods like next() that don't use __xxx__ names).

BTW, another advantage of all this is that it provides a lot
more flexibility in the overall approach to implementing
the type object. For example, we might decide to move the
type slots into a separate memory block, so that the
type struct could be extended by subclasses in the same
way as other types.

--
Greg

From ronaldoussoren at mac.com  Wed Nov 29 06:55:55 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 29 Nov 2006 06:55:55 +0100
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456CD001.4000907@canterbury.ac.nz>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org> <456CD001.4000907@canterbury.ac.nz>
Message-ID: <D861E24D-86D3-4FF0-8ADA-DE75487827DD@mac.com>


On  29 Nov 2006, at 1:10 AM, Greg Ewing wrote:


>
> BTW, another advantage of all this is that it provides a lot
> more flexibility in the overall approach to implementing
> the type object. For example, we might decide to move the
> type slots into a separate memory block, so that the
> type struct could be extended by subclasses in the same
> way as other types.

You underestimate the amount of effort that was spend on the 2.x  
PyTypeObject implementation :-). It is already possibly to extend the  
type struct in Python 2.3 and later (at least of heap allocated  
types). One example use of this is PyObjCClassObject in PyObjC.

This won't work in Python 2.2, the necessary machinery was added to  
the type implementation in 2.3.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061129/c0638310/attachment.bin 

From talin at acm.org  Wed Nov 29 08:23:38 2006
From: talin at acm.org (Talin)
Date: Tue, 28 Nov 2006 23:23:38 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>	
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>	
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
Message-ID: <456D357A.8030707@acm.org>

Guido van Rossum wrote:
> On 11/28/06, Talin <talin at acm.org> wrote:
>> Guido van Rossum wrote:
>> > Some comments:
>> >
>> > - Fredrik's solution makes one call per registered method. (I don't
>> > know if the patch he refers to follows that model.) That seems a fair
>> > amount of code for an average type -- I'm wondering if it's too early
>> > to worry about code bloat (I don't think the speed is going to
>> > matter).
>>
>> One other thought: The special constants could themselves be nothing
>> more than the offset into the PyTypeObject struct, i.e.:
>>
>>     #define SPECMETHOD_NEW ((const char*)offsetof(PyTypeObject,tp_new))
> 
> I think this would cause too many issues with backwards compatibility.
> 
> I like the idea much better to use special names (e.g. starting with a 
> ".").
> 
>> In the PyType_Ready code, you would see if the method name had a value
>> of less than sizeof(PyTypeObject); If so, then it's a special method
>> name, and you fill in the struct at the specified offset.
>>
>> So the interpretation of the table could be very simple and fast. It has
>> a slight disadvantage from the approach of using actual string names for
>> special methods, in that it doesn't allow the VM to silently
>> promote/demote methods to 'special' status.
> 
> I think the interpretation will be fast enough (or else what you said
> about premature optimization earlier wouldn't be correct. :-)

OK, based on these comments and the other feedback from this thread, 
here's a more concrete proposal:

== Method Table ==

Method definitions are stored in a static table, identical in format to 
the existing PyMethodDef table.

For non-method initializers, the most commonly-used ones will be passed 
in as parameters to the type creation function. Those that are less 
commonly used can be written in as a secondary step after the type has 
been created, or in some cases represented in the tp_members table.

== Method Names ==

As suggested by Guido, we use a naming convention to determine how a 
method in the method table is handled. I propose that methods be divided 
into three categories, which are "Normal", "Special", and "Internal" 
methods, and which are interpreted slightly differently at type 
initialization time.

* Internal methods are those that have no equivalent Python name, such 
as tp_free/tp_alloc. Internal methods names start with a dot ("."), so 
tp_alloc would be represented by the string ".tp_alloc".

Internal methods are always stored into a slot in the PyTypeObject. If 
there is no corresponding slot for a given name, that is a runtime error.

* Special methods have the double-underscore (__special__) naming 
convention. A special method may or may not have a slot definition in 
PyTypeObject. If there is such a slot, the method pointer will be stored 
into it; If there is no such slot, then the method pointer is stored 
into the class dict just like a normal method.

Because the decision whether to put the method into a slot is made by 
the VM, the set of available slots can be modified in future Python 
releases without breaking existing code.

* Normal methods are any methods that are neither special or internal. 
They are not placed in a slot, but are simply stored in the class dict.

Brett Cannon brought up the point about __getitem__ being ambiguous, 
since there are two slots, one for lists and one for mappings. This is 
handled as follows:

The "mapping" version of __getitem__ is a special method, named 
"__getitem__". The "list" version, however, is considered an internal 
method (since it's more specialized), and has the name ".tp_getitem".

Greg Ewing's point about "next" is handled as follows: A function named 
"next" will never be treated as a special method name, since it does not 
follow the naming convention of either internal or special names. 
However, if you want to fill in the "tp_next" slot of the PyTypeObject, 
you can use the string ".tp_next" rather than "next".

== Type Creation ==

For backwards compatibility, the existing PyType_Ready function will 
continue to work on statically-declared PyTypeObject structures. A new 
function, 'PyType_Create' will be added that creates a new type from the 
input parameters and the method initialization tables as described 
previously. The actual type record may be allocated dynamically, as 
suggested by Greg Ewing.

Structures such as tp_as_sequence which extend the PyTypeObject will be 
created as needed, if there are any methods that require those extension 
structures.

== Backwards compatibility ==

The existing PyType_Ready and C-style static initialization mechanism 
will continue to work - the new method for type creation will coexist 
alongside the old.

It is an open question as to whether PyType_Ready should attempt to 
interpret the special method names and fill in the PyTypeObject slots. 
If it does, then PyType_Create can all PyType_Ready as a subroutine 
during the type creation process.

Otherwise, the only modifications to the interpreter will be the 
creation of the new PyType_Create function and any required subroutines. 
Existing code should be unaffected.

-- Talin

From guido at python.org  Wed Nov 29 16:52:03 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2006 07:52:03 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456D357A.8030707@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
Message-ID: <ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>

Have you thought much about the issue of different signature? The
regular method table only has functions taking one or more objects and
returning an object. (There are a few flags to indicate variations on
the call args.) The special slots have all sorts of other signatures,
some returning int, some returning void, some taking ints or C
strings, etc. (And changing this would be a huge inefficiency in some
cases.)

Not using some variant of C (or C99) struct initialization means more
chance for undetected errors since you will necessarily have to cast
everything to a standard type and then the compiler can't type-check
that the function signature matches the slot's needs.

Nevertheless, I kind of like this, and would like someone to produce a
patch for this approach. It is my opinion that without working code it
is too easy to overlook design flaws. Since you are keen to have this
in 2.6, the solution would have to be backwards compatible with 2.5,
so the patch wouldn't have to change every type initializer in the
system -- it would only have to provide the complete machinery and
convert a few example types to prove it works.

Any volunteers? Without code this proposal is dead in the water.

--Guido

On 11/28/06, Talin <talin at acm.org> wrote:
> Guido van Rossum wrote:
> > On 11/28/06, Talin <talin at acm.org> wrote:
> >> Guido van Rossum wrote:
> >> > Some comments:
> >> >
> >> > - Fredrik's solution makes one call per registered method. (I don't
> >> > know if the patch he refers to follows that model.) That seems a fair
> >> > amount of code for an average type -- I'm wondering if it's too early
> >> > to worry about code bloat (I don't think the speed is going to
> >> > matter).
> >>
> >> One other thought: The special constants could themselves be nothing
> >> more than the offset into the PyTypeObject struct, i.e.:
> >>
> >>     #define SPECMETHOD_NEW ((const char*)offsetof(PyTypeObject,tp_new))
> >
> > I think this would cause too many issues with backwards compatibility.
> >
> > I like the idea much better to use special names (e.g. starting with a
> > ".").
> >
> >> In the PyType_Ready code, you would see if the method name had a value
> >> of less than sizeof(PyTypeObject); If so, then it's a special method
> >> name, and you fill in the struct at the specified offset.
> >>
> >> So the interpretation of the table could be very simple and fast. It has
> >> a slight disadvantage from the approach of using actual string names for
> >> special methods, in that it doesn't allow the VM to silently
> >> promote/demote methods to 'special' status.
> >
> > I think the interpretation will be fast enough (or else what you said
> > about premature optimization earlier wouldn't be correct. :-)
>
> OK, based on these comments and the other feedback from this thread,
> here's a more concrete proposal:
>
> == Method Table ==
>
> Method definitions are stored in a static table, identical in format to
> the existing PyMethodDef table.
>
> For non-method initializers, the most commonly-used ones will be passed
> in as parameters to the type creation function. Those that are less
> commonly used can be written in as a secondary step after the type has
> been created, or in some cases represented in the tp_members table.
>
> == Method Names ==
>
> As suggested by Guido, we use a naming convention to determine how a
> method in the method table is handled. I propose that methods be divided
> into three categories, which are "Normal", "Special", and "Internal"
> methods, and which are interpreted slightly differently at type
> initialization time.
>
> * Internal methods are those that have no equivalent Python name, such
> as tp_free/tp_alloc. Internal methods names start with a dot ("."), so
> tp_alloc would be represented by the string ".tp_alloc".
>
> Internal methods are always stored into a slot in the PyTypeObject. If
> there is no corresponding slot for a given name, that is a runtime error.
>
> * Special methods have the double-underscore (__special__) naming
> convention. A special method may or may not have a slot definition in
> PyTypeObject. If there is such a slot, the method pointer will be stored
> into it; If there is no such slot, then the method pointer is stored
> into the class dict just like a normal method.
>
> Because the decision whether to put the method into a slot is made by
> the VM, the set of available slots can be modified in future Python
> releases without breaking existing code.
>
> * Normal methods are any methods that are neither special or internal.
> They are not placed in a slot, but are simply stored in the class dict.
>
> Brett Cannon brought up the point about __getitem__ being ambiguous,
> since there are two slots, one for lists and one for mappings. This is
> handled as follows:
>
> The "mapping" version of __getitem__ is a special method, named
> "__getitem__". The "list" version, however, is considered an internal
> method (since it's more specialized), and has the name ".tp_getitem".
>
> Greg Ewing's point about "next" is handled as follows: A function named
> "next" will never be treated as a special method name, since it does not
> follow the naming convention of either internal or special names.
> However, if you want to fill in the "tp_next" slot of the PyTypeObject,
> you can use the string ".tp_next" rather than "next".
>
> == Type Creation ==
>
> For backwards compatibility, the existing PyType_Ready function will
> continue to work on statically-declared PyTypeObject structures. A new
> function, 'PyType_Create' will be added that creates a new type from the
> input parameters and the method initialization tables as described
> previously. The actual type record may be allocated dynamically, as
> suggested by Greg Ewing.
>
> Structures such as tp_as_sequence which extend the PyTypeObject will be
> created as needed, if there are any methods that require those extension
> structures.
>
> == Backwards compatibility ==
>
> The existing PyType_Ready and C-style static initialization mechanism
> will continue to work - the new method for type creation will coexist
> alongside the old.
>
> It is an open question as to whether PyType_Ready should attempt to
> interpret the special method names and fill in the PyTypeObject slots.
> If it does, then PyType_Create can all PyType_Ready as a subroutine
> during the type creation process.
>
> Otherwise, the only modifications to the interpreter will be the
> creation of the new PyType_Create function and any required subroutines.
> Existing code should be unaffected.
>
> -- Talin
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Wed Nov 29 18:07:08 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 29 Nov 2006 09:07:08 PST
Subject: [Python-3000] Generic functions vs. OO
In-Reply-To: <06Nov28.100326pst."58648"@synergy1.parc.xerox.com> 
References: <ca471dc20611230937v73f3a754m22115cf50c1221ef@mail.gmail.com>
	<ca471dc20611232134p14dc95d0g64a0bdbce99f06f9@mail.gmail.com>
	<ca471dc20611240919l22d93715m6a6940defb92c602@mail.gmail.com>
	<ca471dc20611241518s6fc8e680m5fb6b5ea16def5ae@mail.gmail.com>
	<06Nov28.100326pst."58648"@synergy1.parc.xerox.com>
Message-ID: <06Nov29.090713pst."58648"@synergy1.parc.xerox.com>

I've updated the page to answer a few questions Jim Jewett posed, and
to expand a few more interfaces.  I haven't tried to explain the
implementation types, like Code or Frame.  I'm hoping someone else
will do that.  I've broken up the File type into a set of types which
attempt to capture (1) the difference between an open file-system
file, and a file-like object, and (2) the difference between files
opened in binary mode and files opened in text mode.

Going through the type system again is interesting, and raises a few
questions about some of the interfaces.  For instance, should files
opened in binary mode support iteration?  If so, what's the unit of
iteration?

See http://wiki.python.org/moin/AbstractBaseClasses for more.

Bill

> Guido van Rossum wrote:
> > I wonder if a bunch of well thought-out standard ABCs, applied to the
> > standard data types, and perhaps more support for setting __bases__,
> > wouldn't address most concerns.
> 
> I think it would, but I don't know if we're going to get anywhere
> without getting more concrete.  Let's tap "the wisdom of crowds".  I've
> set up a Wiki page at http://wiki.python.org/moin/AbstractBaseClasses,
> and spent a little time defining some ABCs and concrete classes.  I'll
> work on it more over the week, but feel free to pitch in and expand it.
> Or create alternative versions (create a new Wiki page, and put a link
> to it at the top of the existing page).
> 
> Bill

From tomerfiliba at gmail.com  Wed Nov 29 18:41:19 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Wed, 29 Nov 2006 19:41:19 +0200
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
Message-ID: <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>

Guido:
> Agreed that for the distinction between readable/writable it's pretty
> silly, and probably just encourages LBYL code

no, the point was -- should we use separate StreamReader/StreamWriter
classes, we wouldn't need this querying, the object will fail with
AttributeError/TypeError if we attempt to perform an invalid operation.

for example:
f = file("...", "w")
f.read() # IOError

fr = FileWriter(file("...", "w"))
fr.read() # AttributeError

with all that stricter-interfaces talk, which is likely to enter py3k,
the question we should ask is do we want the new IO stack to make
use of those (or at least be designed with that in mind)?

if we do want stricter interfaces in the IO stack too, then we
should follow the java-style path (separate readers and writers,
etc.).

for example, do you think users of the IO stack would want multiple
dispatch based on different streams? i'm well aware that GFs
are not yet finalized material, but do we want to take them into
account? or keep the plain old duck-typed io stack of today?


-tomer

From talin at acm.org  Wed Nov 29 19:19:29 2006
From: talin at acm.org (Talin)
Date: Wed, 29 Nov 2006 10:19:29 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>	
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>	
	<456C85E9.8000905@acm.org>	
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>	
	<456D357A.8030707@acm.org>
	<ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>
Message-ID: <456DCF31.50301@acm.org>

Guido van Rossum wrote:
> Have you thought much about the issue of different signature? The
> regular method table only has functions taking one or more objects and
> returning an object. (There are a few flags to indicate variations on
> the call args.) The special slots have all sorts of other signatures,
> some returning int, some returning void, some taking ints or C
> strings, etc. (And changing this would be a huge inefficiency in some
> cases.)
> 
> Not using some variant of C (or C99) struct initialization means more
> chance for undetected errors since you will necessarily have to cast
> everything to a standard type and then the compiler can't type-check
> that the function signature matches the slot's needs.

Yes, there will be some loss of type safety. I'm not sure what to say 
about that. (Although, there is one benefit that you will at least be 
able to declare the "self" argument as its concrete type, rather than as 
PyObject *.)

> Nevertheless, I kind of like this, and would like someone to produce a
> patch for this approach. It is my opinion that without working code it
> is too easy to overlook design flaws. Since you are keen to have this
> in 2.6, the solution would have to be backwards compatible with 2.5,
> so the patch wouldn't have to change every type initializer in the
> system -- it would only have to provide the complete machinery and
> convert a few example types to prove it works.
> 
> Any volunteers? Without code this proposal is dead in the water.

A patch won't be difficult to produce; It all depends on the 
availability of round 'tuits. Probably not this week at the very least.

I'm curious as to what folks would recommend as the best way to do the 
mapping of slot names to slot addresses as a static table in the VM. I'm 
assuming that the result of the mapping would be a "slot descriptor" 
containing the offset of the pointer, and a code indicating whether it 
goes into the main type object or one of the extension structures.

(Oh, and off topic, I have a procedural question about the list: Is it 
considered good etiquette to only reply to the list, and not the 
individuals whom you are replying to? If you "reply all", then folks 
might get two copies of the message, but if you reply to only the list, 
then non-subscribers posting via gmane might not get replies.)

> --Guido
> 
> On 11/28/06, Talin <talin at acm.org> wrote:
>> Guido van Rossum wrote:
>> > On 11/28/06, Talin <talin at acm.org> wrote:
>> >> Guido van Rossum wrote:
>> >> > Some comments:
>> >> >
>> >> > - Fredrik's solution makes one call per registered method. (I don't
>> >> > know if the patch he refers to follows that model.) That seems a 
>> fair
>> >> > amount of code for an average type -- I'm wondering if it's too 
>> early
>> >> > to worry about code bloat (I don't think the speed is going to
>> >> > matter).
>> >>
>> >> One other thought: The special constants could themselves be nothing
>> >> more than the offset into the PyTypeObject struct, i.e.:
>> >>
>> >>     #define SPECMETHOD_NEW ((const 
>> char*)offsetof(PyTypeObject,tp_new))
>> >
>> > I think this would cause too many issues with backwards compatibility.
>> >
>> > I like the idea much better to use special names (e.g. starting with a
>> > ".").
>> >
>> >> In the PyType_Ready code, you would see if the method name had a value
>> >> of less than sizeof(PyTypeObject); If so, then it's a special method
>> >> name, and you fill in the struct at the specified offset.
>> >>
>> >> So the interpretation of the table could be very simple and fast. 
>> It has
>> >> a slight disadvantage from the approach of using actual string 
>> names for
>> >> special methods, in that it doesn't allow the VM to silently
>> >> promote/demote methods to 'special' status.
>> >
>> > I think the interpretation will be fast enough (or else what you said
>> > about premature optimization earlier wouldn't be correct. :-)
>>
>> OK, based on these comments and the other feedback from this thread,
>> here's a more concrete proposal:
>>
>> == Method Table ==
>>
>> Method definitions are stored in a static table, identical in format to
>> the existing PyMethodDef table.
>>
>> For non-method initializers, the most commonly-used ones will be passed
>> in as parameters to the type creation function. Those that are less
>> commonly used can be written in as a secondary step after the type has
>> been created, or in some cases represented in the tp_members table.
>>
>> == Method Names ==
>>
>> As suggested by Guido, we use a naming convention to determine how a
>> method in the method table is handled. I propose that methods be divided
>> into three categories, which are "Normal", "Special", and "Internal"
>> methods, and which are interpreted slightly differently at type
>> initialization time.
>>
>> * Internal methods are those that have no equivalent Python name, such
>> as tp_free/tp_alloc. Internal methods names start with a dot ("."), so
>> tp_alloc would be represented by the string ".tp_alloc".
>>
>> Internal methods are always stored into a slot in the PyTypeObject. If
>> there is no corresponding slot for a given name, that is a runtime error.
>>
>> * Special methods have the double-underscore (__special__) naming
>> convention. A special method may or may not have a slot definition in
>> PyTypeObject. If there is such a slot, the method pointer will be stored
>> into it; If there is no such slot, then the method pointer is stored
>> into the class dict just like a normal method.
>>
>> Because the decision whether to put the method into a slot is made by
>> the VM, the set of available slots can be modified in future Python
>> releases without breaking existing code.
>>
>> * Normal methods are any methods that are neither special or internal.
>> They are not placed in a slot, but are simply stored in the class dict.
>>
>> Brett Cannon brought up the point about __getitem__ being ambiguous,
>> since there are two slots, one for lists and one for mappings. This is
>> handled as follows:
>>
>> The "mapping" version of __getitem__ is a special method, named
>> "__getitem__". The "list" version, however, is considered an internal
>> method (since it's more specialized), and has the name ".tp_getitem".
>>
>> Greg Ewing's point about "next" is handled as follows: A function named
>> "next" will never be treated as a special method name, since it does not
>> follow the naming convention of either internal or special names.
>> However, if you want to fill in the "tp_next" slot of the PyTypeObject,
>> you can use the string ".tp_next" rather than "next".
>>
>> == Type Creation ==
>>
>> For backwards compatibility, the existing PyType_Ready function will
>> continue to work on statically-declared PyTypeObject structures. A new
>> function, 'PyType_Create' will be added that creates a new type from the
>> input parameters and the method initialization tables as described
>> previously. The actual type record may be allocated dynamically, as
>> suggested by Greg Ewing.
>>
>> Structures such as tp_as_sequence which extend the PyTypeObject will be
>> created as needed, if there are any methods that require those extension
>> structures.
>>
>> == Backwards compatibility ==
>>
>> The existing PyType_Ready and C-style static initialization mechanism
>> will continue to work - the new method for type creation will coexist
>> alongside the old.
>>
>> It is an open question as to whether PyType_Ready should attempt to
>> interpret the special method names and fill in the PyTypeObject slots.
>> If it does, then PyType_Create can all PyType_Ready as a subroutine
>> during the type creation process.
>>
>> Otherwise, the only modifications to the interpreter will be the
>> creation of the new PyType_Create function and any required subroutines.
>> Existing code should be unaffected.
>>
>> -- Talin
>> _______________________________________________
>> Python-3000 mailing list
>> Python-3000 at python.org
>> http://mail.python.org/mailman/listinfo/python-3000
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>>
> 
> 

From guido at python.org  Wed Nov 29 19:27:06 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2006 10:27:06 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456DCF31.50301@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
	<ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>
	<456DCF31.50301@acm.org>
Message-ID: <ca471dc20611291027g4267dde4j2ceb89ff151a6662@mail.gmail.com>

On 11/29/06, Talin <talin at acm.org> wrote:
> (Oh, and off topic, I have a procedural question about the list: Is it
> considered good etiquette to only reply to the list, and not the
> individuals whom you are replying to? If you "reply all", then folks
> might get two copies of the message, but if you reply to only the list,
> then non-subscribers posting via gmane might not get replies.)

By all means use reply-all which CC's the author (and others). This
seems to be what most people do, so folks who aren't using gmail yet
have presumably developed other strategies to cope.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From benji at benjiyork.com  Wed Nov 29 19:44:56 2006
From: benji at benjiyork.com (Benji York)
Date: Wed, 29 Nov 2006 13:44:56 -0500
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <ca471dc20611291027g4267dde4j2ceb89ff151a6662@mail.gmail.com>
References: <456BFFBF.9000208@acm.org>
	<ekgvtj$dr9$1@sea.gmane.org>	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>	<456C85E9.8000905@acm.org>	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>	<456D357A.8030707@acm.org>	<ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>	<456DCF31.50301@acm.org>
	<ca471dc20611291027g4267dde4j2ceb89ff151a6662@mail.gmail.com>
Message-ID: <456DD528.2090802@benjiyork.com>

Guido van Rossum wrote:
> By all means use reply-all which CC's the author (and others). This
> seems to be what most people do, so folks who aren't using gmail yet
> have presumably developed other strategies to cope.

One such coping mechanism is to configure mailman to not send you copies 
of messages you were sent directly via the "Avoid duplicate copies of 
messages?" option.
--
Benji York

From brett at python.org  Wed Nov 29 20:18:35 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 29 Nov 2006 11:18:35 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456DCF31.50301@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
	<ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>
	<456DCF31.50301@acm.org>
Message-ID: <bbaeab100611291118r6dcc26b6n18bb55228f6a7c6a@mail.gmail.com>

On 11/29/06, Talin <talin at acm.org> wrote:
>
> Guido van Rossum wrote:
> > Have you thought much about the issue of different signature? The
> > regular method table only has functions taking one or more objects and
> > returning an object. (There are a few flags to indicate variations on
> > the call args.) The special slots have all sorts of other signatures,
> > some returning int, some returning void, some taking ints or C
> > strings, etc. (And changing this would be a huge inefficiency in some
> > cases.)
> >
> > Not using some variant of C (or C99) struct initialization means more
> > chance for undetected errors since you will necessarily have to cast
> > everything to a standard type and then the compiler can't type-check
> > that the function signature matches the slot's needs.
>
> Yes, there will be some loss of type safety. I'm not sure what to say
> about that. (Although, there is one benefit that you will at least be
> able to declare the "self" argument as its concrete type, rather than as
> PyObject *.)


I must be missing something, why is this any different than what you can do
now?  You either define the function with the exact call signature as typed
for the struct field, or you define how you want the arguments to be treated
and you cast to the call signature that is expected for the struct field.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061129/b7064aad/attachment.htm 

From brett at python.org  Wed Nov 29 20:35:17 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 29 Nov 2006 11:35:17 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456D357A.8030707@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
Message-ID: <bbaeab100611291135r6e7c7836nbe1ccd554dd76d9b@mail.gmail.com>

On 11/28/06, Talin <talin at acm.org> wrote:
>
> Guido van Rossum wrote:
> > On 11/28/06, Talin <talin at acm.org> wrote:
> >> Guido van Rossum wrote:
> >> > Some comments:
> >> >
> >> > - Fredrik's solution makes one call per registered method. (I don't
> >> > know if the patch he refers to follows that model.) That seems a fair
> >> > amount of code for an average type -- I'm wondering if it's too early
> >> > to worry about code bloat (I don't think the speed is going to
> >> > matter).
> >>
> >> One other thought: The special constants could themselves be nothing
> >> more than the offset into the PyTypeObject struct, i.e.:
> >>
> >>     #define SPECMETHOD_NEW ((const char*)offsetof(PyTypeObject,tp_new))
> >
> > I think this would cause too many issues with backwards compatibility.
> >
> > I like the idea much better to use special names (e.g. starting with a
> > ".").
> >
> >> In the PyType_Ready code, you would see if the method name had a value
> >> of less than sizeof(PyTypeObject); If so, then it's a special method
> >> name, and you fill in the struct at the specified offset.
> >>
> >> So the interpretation of the table could be very simple and fast. It
> has
> >> a slight disadvantage from the approach of using actual string names
> for
> >> special methods, in that it doesn't allow the VM to silently
> >> promote/demote methods to 'special' status.
> >
> > I think the interpretation will be fast enough (or else what you said
> > about premature optimization earlier wouldn't be correct. :-)
>
> OK, based on these comments and the other feedback from this thread,
> here's a more concrete proposal:
>
> == Method Table ==
>
> Method definitions are stored in a static table, identical in format to
> the existing PyMethodDef table.
>
> For non-method initializers, the most commonly-used ones will be passed
> in as parameters to the type creation function. Those that are less
> commonly used can be written in as a secondary step after the type has
> been created, or in some cases represented in the tp_members table.
>
> == Method Names ==
>
> As suggested by Guido, we use a naming convention to determine how a
> method in the method table is handled. I propose that methods be divided
> into three categories, which are "Normal", "Special", and "Internal"
> methods, and which are interpreted slightly differently at type
> initialization time.
>
> * Internal methods are those that have no equivalent Python name, such
> as tp_free/tp_alloc. Internal methods names start with a dot ("."), so
> tp_alloc would be represented by the string ".tp_alloc".


Haven't we had various arguments about how it's bad to use a leading dot to
have a special meaning?  I understand why we need some way to flag internal
methods on a type and I support going with an explicit way of specifying,
but is a dot really the best solution?  I mean something like INTERNAL_METH
"tp_alloc" would even work using C's automatic string concatentation and
doing::

  #define INTERNAL_METH "."

or whatever string we wanted that was not valid in a method name.  I don't
think this would lead us down the road of tons of macros and it makes things
very visible.

Internal methods are always stored into a slot in the PyTypeObject. If
> there is no corresponding slot for a given name, that is a runtime error.
>
> * Special methods have the double-underscore (__special__) naming
> convention. A special method may or may not have a slot definition in
> PyTypeObject. If there is such a slot, the method pointer will be stored
> into it; If there is no such slot, then the method pointer is stored
> into the class dict just like a normal method.
>
> Because the decision whether to put the method into a slot is made by
> the VM, the set of available slots can be modified in future Python
> releases without breaking existing code.
>
> * Normal methods are any methods that are neither special or internal.
> They are not placed in a slot, but are simply stored in the class dict.
>
> Brett Cannon brought up the point about __getitem__ being ambiguous,
> since there are two slots, one for lists and one for mappings. This is
> handled as follows:
>
> The "mapping" version of __getitem__ is a special method, named
> "__getitem__". The "list" version, however, is considered an internal
> method (since it's more specialized), and has the name ".tp_getitem".


Or the other option is that in the future we just don't have the distinction
and make sure that the __getitem__ methods do the requisite type checks.
The type check is done at some point in the C code anyway so it isn't like
there is a performance reason for the different slots.  And as for providing
a C-level function that provides a __getitem__ that takes Py_ssize_t, that
can still be provided, it just isn't what is put into the struct.

The one problem this does cause is testing for the interface support at the
C level.  But that could be a C function that looks for specific defined
functions.  Plus this would help make the C code less distinct from the way
things expose themselves at the Python level (which I personally think is a
good thing).

Greg Ewing's point about "next" is handled as follows: A function named
> "next" will never be treated as a special method name, since it does not
> follow the naming convention of either internal or special names.
> However, if you want to fill in the "tp_next" slot of the PyTypeObject,
> you can use the string ".tp_next" rather than "next".
>
> == Type Creation ==
>
> For backwards compatibility, the existing PyType_Ready function will
> continue to work on statically-declared PyTypeObject structures. A new
> function, 'PyType_Create' will be added that creates a new type from the
> input parameters and the method initialization tables as described
> previously. The actual type record may be allocated dynamically, as
> suggested by Greg Ewing.
>
> Structures such as tp_as_sequence which extend the PyTypeObject will be
> created as needed, if there are any methods that require those extension
> structures.
>
> == Backwards compatibility ==
>
> The existing PyType_Ready and C-style static initialization mechanism
> will continue to work - the new method for type creation will coexist
> alongside the old.
>
> It is an open question as to whether PyType_Ready should attempt to
> interpret the special method names and fill in the PyTypeObject slots.
> If it does, then PyType_Create can all PyType_Ready as a subroutine
> during the type creation process.
>
> Otherwise, the only modifications to the interpreter will be the
> creation of the new PyType_Create function and any required subroutines.
> Existing code should be unaffected.


Overall sounds good to me!

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-3000/attachments/20061129/7e47b095/attachment.html 

From fredrik at pythonware.com  Wed Nov 29 21:32:17 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 29 Nov 2006 21:32:17 +0100
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456C7E3B.7090601@acm.org>
References: <456BFFBF.9000208@acm.org>
	<ekgvtj$dr9$1@sea.gmane.org>	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org>
Message-ID: <ekkqoh$4g9$1@sea.gmane.org>

Talin wrote:

> Now, I tend to prefer using a static table vs. the 
> function-call-per-method simply because of my habits, which tend to be 
> overly parsimonious with code size and such (It's a side-effect of 
> working on embedded systems, which is what game consoles effectively 
> are.) I wouldn't have strong objections to doing it the other way.

Umm.  AFAICT, a call with one variable and two constant arguments 
typically needs 16-20 bytes on x86; a four-item struct with a pointer to 
a string literal needs 16 bytes plus the string literal, which means 
that you're restricted to 3-character names if you don't want to waste 
memory...

...and the decoder will of course be a lot larger, uglier, and slower
if it has to test for a whole bunch of strings constants instead just 
switching on an integer value.  Bloat alert.

</F>


From guido at python.org  Wed Nov 29 22:45:05 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2006 13:45:05 -0800
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
Message-ID: <ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>

I would not count on GFs, but I would suggest to count on ABCs and a
better hierarchy.

E.g. look at what Bill Janssen came up with (clearly incomplete):
http://wiki.python.org/moin/AbstractBaseClasses

I think that seekability should be a dynamically determined property though.

On 11/29/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> Guido:
> > Agreed that for the distinction between readable/writable it's pretty
> > silly, and probably just encourages LBYL code
>
> no, the point was -- should we use separate StreamReader/StreamWriter
> classes, we wouldn't need this querying, the object will fail with
> AttributeError/TypeError if we attempt to perform an invalid operation.
>
> for example:
> f = file("...", "w")
> f.read() # IOError
>
> fr = FileWriter(file("...", "w"))
> fr.read() # AttributeError
>
> with all that stricter-interfaces talk, which is likely to enter py3k,
> the question we should ask is do we want the new IO stack to make
> use of those (or at least be designed with that in mind)?
>
> if we do want stricter interfaces in the IO stack too, then we
> should follow the java-style path (separate readers and writers,
> etc.).
>
> for example, do you think users of the IO stack would want multiple
> dispatch based on different streams? i'm well aware that GFs
> are not yet finalized material, but do we want to take them into
> account? or keep the plain old duck-typed io stack of today?
>
>
> -tomer
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Thu Nov 30 00:45:35 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 29 Nov 2006 15:45:35 PST
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com> 
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
Message-ID: <06Nov29.154543pst."58648"@synergy1.parc.xerox.com>

Though I'm not happy with the current factoring of the "file" type at
http://wiki.python.org/moin/AbstractBaseClasses.  I'm thinking this
afternoon that there should be an InputStream type and an OutputStream
type, and that the particular mix of interfaces you get back from
"open" should depend on what kind of file it is (tty or disk, etc.)
and the mode that you used to open it.  The actual file classes might
be hidden; you would look at which interfaces you got to see what you
could do with it.  Of course, you should get back the right mix for
the specified mode and file kind!

Incidentally, what kind of iteration should apply to files opened in
"binary" mode (which I continue to believe should be the default for
Python)?

  for line in open("foo.jpg", "rb"):
    ...

just seems wrong to me.  Should be an iteration over bytes of the
file, or over blocks.  How about 80-byte "records" :-?

Bill

> I would not count on GFs, but I would suggest to count on ABCs and a
> better hierarchy.
> 
> E.g. look at what Bill Janssen came up with (clearly incomplete):
> http://wiki.python.org/moin/AbstractBaseClasses
> 
> I think that seekability should be a dynamically determined property though.
> 
> On 11/29/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> > Guido:
> > > Agreed that for the distinction between readable/writable it's pretty
> > > silly, and probably just encourages LBYL code
> >
> > no, the point was -- should we use separate StreamReader/StreamWriter
> > classes, we wouldn't need this querying, the object will fail with
> > AttributeError/TypeError if we attempt to perform an invalid operation.
> >
> > for example:
> > f = file("...", "w")
> > f.read() # IOError
> >
> > fr = FileWriter(file("...", "w"))
> > fr.read() # AttributeError
> >
> > with all that stricter-interfaces talk, which is likely to enter py3k,
> > the question we should ask is do we want the new IO stack to make
> > use of those (or at least be designed with that in mind)?
> >
> > if we do want stricter interfaces in the IO stack too, then we
> > should follow the java-style path (separate readers and writers,
> > etc.).
> >
> > for example, do you think users of the IO stack would want multiple
> > dispatch based on different streams? i'm well aware that GFs
> > are not yet finalized material, but do we want to take them into
> > account? or keep the plain old duck-typed io stack of today?


From guido at python.org  Thu Nov 30 01:30:21 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2006 16:30:21 -0800
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <3152438538160113168@unknownmsgid>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
	<3152438538160113168@unknownmsgid>
Message-ID: <ca471dc20611291630r300bf23cg7d002eddff315d38@mail.gmail.com>

First choice: binary files should not be iterable at all. I've never
felt the need for this feature.

Second choice: binary files should iterate given the buffer size
(explicit or default) passed to open(). For unbuffered files they
should iterate over bytes.

On 11/29/06, Bill Janssen <janssen at parc.com> wrote:
> Though I'm not happy with the current factoring of the "file" type at
> http://wiki.python.org/moin/AbstractBaseClasses.  I'm thinking this
> afternoon that there should be an InputStream type and an OutputStream
> type, and that the particular mix of interfaces you get back from
> "open" should depend on what kind of file it is (tty or disk, etc.)
> and the mode that you used to open it.  The actual file classes might
> be hidden; you would look at which interfaces you got to see what you
> could do with it.  Of course, you should get back the right mix for
> the specified mode and file kind!
>
> Incidentally, what kind of iteration should apply to files opened in
> "binary" mode (which I continue to believe should be the default for
> Python)?
>
>   for line in open("foo.jpg", "rb"):
>     ...
>
> just seems wrong to me.  Should be an iteration over bytes of the
> file, or over blocks.  How about 80-byte "records" :-?
>
> Bill
>
> > I would not count on GFs, but I would suggest to count on ABCs and a
> > better hierarchy.
> >
> > E.g. look at what Bill Janssen came up with (clearly incomplete):
> > http://wiki.python.org/moin/AbstractBaseClasses
> >
> > I think that seekability should be a dynamically determined property though.
> >
> > On 11/29/06, tomer filiba <tomerfiliba at gmail.com> wrote:
> > > Guido:
> > > > Agreed that for the distinction between readable/writable it's pretty
> > > > silly, and probably just encourages LBYL code
> > >
> > > no, the point was -- should we use separate StreamReader/StreamWriter
> > > classes, we wouldn't need this querying, the object will fail with
> > > AttributeError/TypeError if we attempt to perform an invalid operation.
> > >
> > > for example:
> > > f = file("...", "w")
> > > f.read() # IOError
> > >
> > > fr = FileWriter(file("...", "w"))
> > > fr.read() # AttributeError
> > >
> > > with all that stricter-interfaces talk, which is likely to enter py3k,
> > > the question we should ask is do we want the new IO stack to make
> > > use of those (or at least be designed with that in mind)?
> > >
> > > if we do want stricter interfaces in the IO stack too, then we
> > > should follow the java-style path (separate readers and writers,
> > > etc.).
> > >
> > > for example, do you think users of the IO stack would want multiple
> > > dispatch based on different streams? i'm well aware that GFs
> > > are not yet finalized material, but do we want to take them into
> > > account? or keep the plain old duck-typed io stack of today?
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Thu Nov 30 01:34:16 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 30 Nov 2006 13:34:16 +1300
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <06Nov29.154543pst.58648@synergy1.parc.xerox.com>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
	<06Nov29.154543pst.58648@synergy1.parc.xerox.com>
Message-ID: <456E2708.6070205@canterbury.ac.nz>

Bill Janssen wrote:

> Incidentally, what kind of iteration should apply to files opened in
> "binary" mode 

I don't think binary files should directly support
iteration at all. Wrap it in an object that iterates
the way you want.

   for b in readbytes(f):
     ...

   for blk in readblocks(f, 1024):
     ...

I'm inclined to think that text files shouldn't be
directly iterable either, and you should use

   for line in readlines(f):
     ...

I can appreciate that some people might consider
practicality to beat purity here, though.

 > How about 80-byte "records" :-?

No problem:

   for card in readblocks(f, 80):
     fortran_compiler.eat(card)

:-)

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 30 01:34:37 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 30 Nov 2006 13:34:37 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456D357A.8030707@acm.org>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
Message-ID: <456E271D.4070701@canterbury.ac.nz>

Talin wrote:
> The "list" version, however, is considered an internal 
> method (since it's more specialized), and has the name ".tp_getitem".

Is there really a need for both the dot and the tp_
prefix? Just ".getitem" etc. ought to be sufficient.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 30 01:34:44 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 30 Nov 2006 13:34:44 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <D861E24D-86D3-4FF0-8ADA-DE75487827DD@mac.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org> <456CD001.4000907@canterbury.ac.nz>
	<D861E24D-86D3-4FF0-8ADA-DE75487827DD@mac.com>
Message-ID: <456E2724.6090702@canterbury.ac.nz>

Ronald Oussoren wrote:
> It is already possibly to extend the  
> type struct in Python 2.3 and later

 From Python?

Python 2.3 (#1, Aug  5 2003, 15:52:30)
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> class C(type):
...   __slots__ = ['foo']
...
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
TypeError: nonempty __slots__ not supported for subtype of 'type'

That looks to me like the type object is not
a full member of the Python community just
yet.

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 30 01:38:53 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 30 Nov 2006 13:38:53 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <bbaeab100611291135r6e7c7836nbe1ccd554dd76d9b@mail.gmail.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
	<bbaeab100611291135r6e7c7836nbe1ccd554dd76d9b@mail.gmail.com>
Message-ID: <456E281D.3090606@canterbury.ac.nz>

Brett Cannon wrote:

> Or the other option is that in the future we just don't have the 
> distinction and make sure that the __getitem__ methods do the requisite 
> type checks.  The type check is done at some point in the C code anyway 
> so it isn't like there is a performance reason for the different slots.

The performance reason is that C code can call the
sq_getitem slot with a C integer directly, without
having to convert to a Python integer and back
again.

Has anyone ever tested whether this is an important
enough gain to be worth keeping the dual slots?

--
Greg

From greg.ewing at canterbury.ac.nz  Thu Nov 30 01:47:44 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 30 Nov 2006 13:47:44 +1300
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456DD528.2090802@benjiyork.com>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
	<ca471dc20611290752p686b24acv64a24163501dbfd8@mail.gmail.com>
	<456DCF31.50301@acm.org>
	<ca471dc20611291027g4267dde4j2ceb89ff151a6662@mail.gmail.com>
	<456DD528.2090802@benjiyork.com>
Message-ID: <456E2A30.5080003@canterbury.ac.nz>

Benji York wrote:

> One such coping mechanism is to configure mailman to not send you copies 
> of messages you were sent directly via the "Avoid duplicate copies of 
> messages?" option.

That's what I do, and it seems to work well enough.

When replying, sometimes I'll trim the address list down
to just the group, or just the group plus the person
I'm immediately replying to, depending on whether I'm
making comments to a particular participant or about
the discussion generally. Sometimes I don't bother.

I'm not all that worried about what happens for people
not following the list. If they want to participate in
a discussion taking place on the list, reading the list
doesn't seem like an unreasonable requirement.

Hopefully one day the writers of popular email software
will do the screamingly obvious thing and add a "Reply
to List" button...

--
Greg

From rasky at develer.com  Thu Nov 30 01:51:56 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Thu, 30 Nov 2006 01:51:56 +0100
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
Message-ID: <ekl9v5$lvn$1@sea.gmane.org>

Hello,

it's a minor issue, but I was wondering if the interpreter prompt could be 
changed in Py3K. The current prompt is ">>>" which happens to match the 
standard character used for quotes in e-mails. As a result, intepreter 
sessions might look funky when copy & pasted inside mails.

Given that showing interactive sessions in e-mails is *very* common and 
encouraged, it might be worthy to change the prompt to avoid this conflict. 
Something like "-->" or "---" or "%%%" or really *anything* else but sequences 
of ">" and "|" would do.

Giovanni Bajo


From guido at python.org  Thu Nov 30 02:45:16 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2006 17:45:16 -0800
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ekl9v5$lvn$1@sea.gmane.org>
References: <ekl9v5$lvn$1@sea.gmane.org>
Message-ID: <ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>

But the >>> prompt is Python's trademark! I always get a warm fuzzy
feeling when I see it, e.g. in a corner of a slide in a NASA
presentation.

Just set sys.ps1 = "-->" if you want something different. You can even
set PYTHONSTARTUP=<some_file_that_sets_sys.ps1> in your environment so
you won't hve to think about it.

--Guido

On 11/29/06, Giovanni Bajo <rasky at develer.com> wrote:
> Hello,
>
> it's a minor issue, but I was wondering if the interpreter prompt could be
> changed in Py3K. The current prompt is ">>>" which happens to match the
> standard character used for quotes in e-mails. As a result, intepreter
> sessions might look funky when copy & pasted inside mails.
>
> Given that showing interactive sessions in e-mails is *very* common and
> encouraged, it might be worthy to change the prompt to avoid this conflict.
> Something like "-->" or "---" or "%%%" or really *anything* else but sequences
> of ">" and "|" would do.
>
> Giovanni Bajo
>
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Nov 30 02:51:58 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 29 Nov 2006 17:51:58 -0800
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456E281D.3090606@canterbury.ac.nz>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C85E9.8000905@acm.org>
	<ca471dc20611281309i52cde6c8ie631e7922f870ab8@mail.gmail.com>
	<456D357A.8030707@acm.org>
	<bbaeab100611291135r6e7c7836nbe1ccd554dd76d9b@mail.gmail.com>
	<456E281D.3090606@canterbury.ac.nz>
Message-ID: <ca471dc20611291751y6587f345m5aed54def0afce63@mail.gmail.com>

I don't know about this one. But I'm sure that the CVS history of
typeobject.c (in the 2.2a through 2.3 era, probably) would show that
serious hacks were necessary to make certain slots perform well
enough. I remember doing quite a bit of performance tuning with some
incredible results -- all to make it competitive with classic classes.

On 11/29/06, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Brett Cannon wrote:
>
> > Or the other option is that in the future we just don't have the
> > distinction and make sure that the __getitem__ methods do the requisite
> > type checks.  The type check is done at some point in the C code anyway
> > so it isn't like there is a performance reason for the different slots.
>
> The performance reason is that C code can call the
> sq_getitem slot with a C integer directly, without
> having to convert to a Python integer and back
> again.
>
> Has anyone ever tested whether this is an important
> enough gain to be worth keeping the dual slots?
>
> --
> Greg
> _______________________________________________
> Python-3000 mailing list
> Python-3000 at python.org
> http://mail.python.org/mailman/listinfo/python-3000
> Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Thu Nov 30 02:56:56 2006
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 29 Nov 2006 17:56:56 -0800
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ekl9v5$lvn$1@sea.gmane.org>
References: <ekl9v5$lvn$1@sea.gmane.org>
Message-ID: <20061130015656.GB18780@panix.com>

On Thu, Nov 30, 2006, Giovanni Bajo wrote:
>
> it's a minor issue, but I was wondering if the interpreter prompt could be 
> changed in Py3K. The current prompt is ">>>" which happens to match the 
> standard character used for quotes in e-mails. As a result, intepreter 
> sessions might look funky when copy & pasted inside mails.

While I feel your pain, I loathe the idea of updating all my doctests...
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

Usenet is not a democracy.  It is a weird cross between an anarchy and a
dictatorship.

From janssen at parc.com  Thu Nov 30 03:38:18 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 29 Nov 2006 18:38:18 PST
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <456E2708.6070205@canterbury.ac.nz> 
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
	<06Nov29.154543pst.58648@synergy1.parc.xerox.com>
	<456E2708.6070205@canterbury.ac.nz>
Message-ID: <06Nov29.183824pst."58648"@synergy1.parc.xerox.com>

Greg Ewing wrote:

> Bill Janssen wrote:
> 
> > Incidentally, what kind of iteration should apply to files opened in
> > "binary" mode 
> 
> I don't think binary files should directly support
> iteration at all. Wrap it in an object that iterates
> the way you want.
> 
>    for b in readbytes(f):
>      ...
> 
>    for blk in readblocks(f, 1024):
>      ...
> 
> I'm inclined to think that text files shouldn't be
> directly iterable either, and you should use
> 
>    for line in readlines(f):
>      ...

I like both these ideas.  So file.readlines() would return an
iterable, then?

Bill

From janssen at parc.com  Thu Nov 30 04:21:10 2006
From: janssen at parc.com (Bill Janssen)
Date: Wed, 29 Nov 2006 19:21:10 PST
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com> 
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
Message-ID: <06Nov29.192119pst."58648"@synergy1.parc.xerox.com>

I've re-factored the file interfaces to reflect the current discussion.

I've also redone the MutableSequence and Mapping interfaces to use default
parameters, which makes them much more comprehensible.

See what you think.

> http://wiki.python.org/moin/AbstractBaseClasses

Bill

From barry at python.org  Thu Nov 30 05:39:02 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 29 Nov 2006 23:39:02 -0500
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ekl9v5$lvn$1@sea.gmane.org>
References: <ekl9v5$lvn$1@sea.gmane.org>
Message-ID: <DF6AAB88-8B2E-40B2-84AD-60C426DD0351@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 29, 2006, at 7:51 PM, Giovanni Bajo wrote:

> it's a minor issue, but I was wondering if the interpreter prompt  
> could be
> changed in Py3K. The current prompt is ">>>" which happens to match  
> the
> standard character used for quotes in e-mails. As a result, intepreter
> sessions might look funky when copy & pasted inside mails.

"Low hanging" as in peaches rotting on the ground? :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iQCVAwUBRW5gZnEjvBPtnXfVAQKDIgP/UgIVI94cW8cQuvcU/cbT3L1zTsbW8r0Q
EPWsJkDSxmV/H+9WDCNhIN6RfwozPtOSln2zL6K8BIZzBuZJ8qM7GyxHG+6knGr8
l61X7IXBOKboQSnN9+Mhhx7TKb9wnm4ME08GtAX5BqwaJ8KjuVj/+sNkMcIQSTPP
xdAhVFIylhY=
=flic
-----END PGP SIGNATURE-----

From ronaldoussoren at mac.com  Thu Nov 30 07:19:43 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 30 Nov 2006 07:19:43 +0100
Subject: [Python-3000] A better way to initialize PyTypeObject
In-Reply-To: <456E2724.6090702@canterbury.ac.nz>
References: <456BFFBF.9000208@acm.org> <ekgvtj$dr9$1@sea.gmane.org>
	<ca471dc20611280739w7eeeda80pd024182f0e8dad01@mail.gmail.com>
	<456C7E3B.7090601@acm.org> <456CD001.4000907@canterbury.ac.nz>
	<D861E24D-86D3-4FF0-8ADA-DE75487827DD@mac.com>
	<456E2724.6090702@canterbury.ac.nz>
Message-ID: <0106D5AC-C38D-4BF6-AD6D-73CE8C1258C0@mac.com>


On  30 Nov 2006, at 1:34 AM, Greg Ewing wrote:

> Ronald Oussoren wrote:
>> It is already possibly to extend the  type struct in Python 2.3  
>> and later
>
> From Python?

No, from C.

>
> Python 2.3 (#1, Aug  5 2003, 15:52:30)
> [GCC 3.1 20020420 (prerelease)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> class C(type):
> ...   __slots__ = ['foo']
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: nonempty __slots__ not supported for subtype of 'type'

<words type="famous,last">
It should be easy enough to add support for this in Python 2.6, all  
required machinery already seems to be present.
</words>

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3562 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-3000/attachments/20061130/ba65482c/attachment.bin 

From rasky at develer.com  Thu Nov 30 09:11:44 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Thu, 30 Nov 2006 09:11:44 +0100
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
References: <ekl9v5$lvn$1@sea.gmane.org>
	<ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
Message-ID: <456E9240.5010302@develer.com>

Guido van Rossum wrote:

> But the >>> prompt is Python's trademark! I always get a warm fuzzy
> feeling when I see it, e.g. in a corner of a slide in a NASA
> presentation.

In fact, there would be another reason to change the prompt in Py3k: to 
identify Py3k snippets and tell them from Python 2.x snippets. When people 
will really start using Py3k, and will start sending mails with interpreter 
sessions in them, or publish blog entries or whatnot, I think it would be 
useful if you could immediately, no-brainy tell whether it's Python 2.x or Py3k.

> Just set sys.ps1 = "-->" if you want something different. You can even
> set PYTHONSTARTUP=<some_file_that_sets_sys.ps1> in your environment so
> you won't hve to think about it.

The problem is not *my* environment, but when I read mails posted by others.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com


From greg.ewing at canterbury.ac.nz  Thu Nov 30 10:13:02 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 30 Nov 2006 22:13:02 +1300
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
References: <ekl9v5$lvn$1@sea.gmane.org>
	<ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
Message-ID: <456EA09E.8030908@canterbury.ac.nz>

Guido van Rossum wrote:
> But the >>> prompt is Python's trademark!

Maybe you could change it to something like

   _/\_/\_P

(That's meant to be an ascii-art python.)

--
Greg

From rasky at develer.com  Thu Nov 30 10:16:18 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Thu, 30 Nov 2006 10:16:18 +0100
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <20061130015656.GB18780@panix.com>
References: <ekl9v5$lvn$1@sea.gmane.org> <20061130015656.GB18780@panix.com>
Message-ID: <ekm7gq$t6a$1@sea.gmane.org>

Aahz wrote:

>> it's a minor issue, but I was wondering if the interpreter prompt could be 
>> changed in Py3K. The current prompt is ">>>" which happens to match the 
>> standard character used for quotes in e-mails. As a result, intepreter 
>> sessions might look funky when copy & pasted inside mails.
> 
> While I feel your pain, I loathe the idea of updating all my doctests...

doctest could easily be updated in the Py3k so to support also the old prompt 
syntax to ease the transition (either by default, or with 
doctest.testmod(old_syntax=True)).

Anyway, you could search & replace and... uhm... run the tests to see if you 
missed something :)
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com


From steven.bethard at gmail.com  Thu Nov 30 15:46:26 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Thu, 30 Nov 2006 07:46:26 -0700
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
References: <ekl9v5$lvn$1@sea.gmane.org>
	<ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
Message-ID: <d11dcfba0611300646tb9a179bnad9583b302fcfe85@mail.gmail.com>

On 11/29/06, Guido van Rossum <guido at python.org> wrote:
> But the >>> prompt is Python's trademark! I always get a warm fuzzy
> feeling when I see it, e.g. in a corner of a slide in a NASA
> presentation.

I was using ``py>`` for a while.  That still gave me a warm fuzzy
feeling ;-) and seemed to work with most newsreaders.

STeVe
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Thu Nov 30 16:43:07 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 30 Nov 2006 07:43:07 -0800
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <d11dcfba0611300646tb9a179bnad9583b302fcfe85@mail.gmail.com>
References: <ekl9v5$lvn$1@sea.gmane.org>
	<ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
	<d11dcfba0611300646tb9a179bnad9583b302fcfe85@mail.gmail.com>
Message-ID: <ca471dc20611300743m1cbed0d0o9dbdb28a3d52a740@mail.gmail.com>

OK, let's stop this. It's not going to change. The deficiencies of
email shouldn't be dictating the language. If I hear more pleading
I'll have to add it to PEP 3099.

Please read http://yellow.bikeshed.com.

--Guido

On 11/30/06, Steven Bethard <steven.bethard at gmail.com> wrote:
> On 11/29/06, Guido van Rossum <guido at python.org> wrote:
> > But the >>> prompt is Python's trademark! I always get a warm fuzzy
> > feeling when I see it, e.g. in a corner of a slide in a NASA
> > presentation.
>
> I was using ``py>`` for a while.  That still gave me a warm fuzzy
> feeling ;-) and seemed to work with most newsreaders.
>
> STeVe
> --
> I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
> tiny blip on the distant coast of sanity.
>         --- Bucky Katt, Get Fuzzy
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Thu Nov 30 17:37:05 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 30 Nov 2006 17:37:05 +0100
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ca471dc20611300743m1cbed0d0o9dbdb28a3d52a740@mail.gmail.com>
References: <ekl9v5$lvn$1@sea.gmane.org>	<ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>	<d11dcfba0611300646tb9a179bnad9583b302fcfe85@mail.gmail.com>
	<ca471dc20611300743m1cbed0d0o9dbdb28a3d52a740@mail.gmail.com>
Message-ID: <ekn1bg$pgf$1@sea.gmane.org>

Guido van Rossum wrote:

> Please read http://yellow.bikeshed.com.

make that

     http://gold.bikeshed.com/

</F>


From ldlandis at gmail.com  Thu Nov 30 18:28:36 2006
From: ldlandis at gmail.com (LD 'Gus' Landis)
Date: Thu, 30 Nov 2006 11:28:36 -0600
Subject: [Python-3000] Low-hanging fruit: change interpreter prompt?
In-Reply-To: <ekn1bg$pgf$1@sea.gmane.org>
References: <ekl9v5$lvn$1@sea.gmane.org>
	<ca471dc20611291745x2c246858m88dd504d80547895@mail.gmail.com>
	<d11dcfba0611300646tb9a179bnad9583b302fcfe85@mail.gmail.com>
	<ca471dc20611300743m1cbed0d0o9dbdb28a3d52a740@mail.gmail.com>
	<ekn1bg$pgf$1@sea.gmane.org>
Message-ID: <a1ddf57e0611300928v58f9909djcb4853fb4d173d58@mail.gmail.com>

NEITHER, both hard to read due to color intensity...
http://white.bikeshed.com/  ;-O see
>Fredrik Lundh <fredrik at pythonware.com> wrote:http://gold.bikeshed.com/
>Guido van Rossum wrote: http://yellow.bikeshed.com/

From tomerfiliba at gmail.com  Thu Nov 30 21:45:03 2006
From: tomerfiliba at gmail.com (tomer filiba)
Date: Thu, 30 Nov 2006 22:45:03 +0200
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <8979205067466383560@unknownmsgid>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
	<8979205067466383560@unknownmsgid>
Message-ID: <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com>

> http://wiki.python.org/moin/AbstractBaseClasses

NOOOOOOOO!  so many baseclasses, who can remember all that?!

now whenever i wish to implement, say, MyFunnyDict, i'll have to
go over the manual, see all the standard interfaces (which may
reach hundreds of classes), and think which interfaces i desire
for my class to "adhere to". that's very bad programming.
they have a name for that in CS: java.

you have to manually declare things, before doing them.
with these ABCs, instead of a simple "def __something__",
you first have to declare everything you'll possibly want to do
(either by explicitly inheriting from, or another syntax such as
__implements__ = ...).

and with all that reliance on the *type* of objects, how would proxies
work?

for example, in RPyC i use local proxy objects that refer to remote
objects (in another process/machine). just like weakref.proxy, you get
another object that behaves exactly the same as the real object,
but your code needn't be aware of that:
if the object's has all the attributes, it's a duck.

now, the proxy object will have to "inherit" all of the interfaces the
real object has, in order for dispatching/isinstance to work.
and that's bad. for weakrefs that may be simple, but in RPyC
we are talking about objects from different processes, which
is not simple at all (types compare by object id, for once).

to sum it up -- if it looks like java and limps like java, it's java!


-tomer

P.S, i took a look over my post, and it's seems rather angry. i'm not.
sorry if it does, but i'm too tired to rewrite it. it's been a long day
and i'm a moment from bed, so no offense in advance.

From janssen at parc.com  Thu Nov 30 22:47:25 2006
From: janssen at parc.com (Bill Janssen)
Date: Thu, 30 Nov 2006 13:47:25 PST
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> 
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
	<8979205067466383560@unknownmsgid>
	<1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com>
Message-ID: <06Nov30.134726pst."58648"@synergy1.parc.xerox.com>

> so no offense in advance.

Sure, no offense taken.  I've seen comments like this before on this
list (recently :-).  I think both approaches (interface types and duck
typing) are complicated in different ways.  But interface types seem
less so, because they provide a place where the user can stop thinking
about what's behind a function or method, instead of having to
understand the code "all the way down".  And it doesn't preclude the
use of "just implement the method" for code that you understand "all
the way down" already, where you know that the only use of a
particular value is to call the "foo" method on it.  But it also
ensures that when someone gets a Mapping, they can call "values" on it
(to list one mis-example of partial implementation that I've tripped
over in the past).

Bill

From greg.ewing at canterbury.ac.nz  Thu Nov 30 23:42:23 2006
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 01 Dec 2006 11:42:23 +1300
Subject: [Python-3000] iostack and Oh Oh
In-Reply-To: <06Nov29.183824pst.58648@synergy1.parc.xerox.com>
References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com>
	<456776D6.8020909@canterbury.ac.nz>
	<ca471dc20611241455j31c4f4edrc7a13cb1cf794573@mail.gmail.com>
	<1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com>
	<ca471dc20611291345q43205f6bqed83e8a8d79e63e6@mail.gmail.com>
	<06Nov29.154543pst.58648@synergy1.parc.xerox.com>
	<456E2708.6070205@canterbury.ac.nz>
	<06Nov29.183824pst.58648@synergy1.parc.xerox.com>
Message-ID: <456F5E4F.6090705@canterbury.ac.nz>

Bill Janssen wrote:
> So file.readlines() would return an iterable, then?

I wasn't thinking of having a file.readlines(), just
a stand-alone function. I'd like to see the file
object's interface made quite minimal in Py3k.

--
Greg