From daniel at stutzbachenterprises.com Thu Mar 1 00:12:07 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 28 Feb 2007 17:12:07 -0600 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: I just uploaded Patch 1671314 to SourceForge with a C implementation of a Raw File I/O type, along with unit tests. It still needs work (especially for supporting very large files and non-unixy systems), but should serve as a very good starting point. On 2/26/07, Mike Verdone wrote: > Hi all, > > Daniel Stutzbach and I have prepared a draft PEP for the new IO system > for Python 3000. This document is, hopefully, true to the info that > Guido wrote on the whiteboards here at PyCon. This is still a draft > and there's quite a few decisions that need to be made. Feedback is > welcomed. > > We've published it on Google Docs here: > http://docs.google.com/Doc?id=dfksfvqd_1cn5g5m > > What follows is a plaintext version. > > Thanks, > > Mike. > > > PEP: XXX > Title: New IO > Version: > Last-Modified: > Authors: Daniel Stutzbach, Mike Verdone > Status: Draft > Type: > Created: 26-Feb-2007 > > Rationale and Goals > Python allows for a variety of file-like objects that can be worked > with via bare read() and write() calls using duck typing. Anything > that provides read() and write() is stream-like. However, more exotic > and extremely useful functions like readline() or seek() may or may > not be available on a file-like object. Python needs a specification > for basic byte-based IO streams to which we can add buffering and > text-handling features. > > Once we have a defined raw byte-based IO interface, we can add > buffering and text-handling layers on top of any byte-based IO class. > The same buffering and text handling logic can be used for files, > sockets, byte arrays, or custom IO classes developed by Python > programmers. Developing a standard definition of a stream lets us > separate stream-based operations like read() and write() from > implementation specific operations like fileno() and isatty(). It > encourages programmers to write code that uses streams as streams and > not require that all streams support file-specific or socket-specific > operations. > > The new IO spec is intended to be similar to the Java IO libraries, > but generally less confusing. Programmers who don't want to muck about > in the new IO world can expect that the open() factory method will > produce an object backwards-compatible with old-style file objects. > Specification > The Python I/O Library will consist of three layers: a raw I/O layer, > a buffer I/O layer, and a text I/O layer. Each layer is defined by an > abstract base class, which may have multiple implementations. The raw > I/O and buffer I/O layers deal with units of bytes, while the text I/O > layer deals with units of characters. > Raw I/O > The abstract base class for raw I/O is RawIOBase. It has several > methods which are wrappers around the appropriate operating system > call. If one of these functions would not make sense on the object, > the implementation must raise an IOError exception. For example, if a > file is opened read-only, the .write() method will raise an IOError. > As another example, if the object represents a socket, then .seek(), > .tell(), and .truncate() will raise an IOError. > > .read() > .write() > .seek() > .tell() > .truncate() > .close() > > Additionally, it defines a few other methods: > > (should these "is_" functions be attributes instead? > "file.readable == True") > > .is_readable() > > Returns True if the object was opened for reading, False > otherwise. If False, .read() will raise an IOError if called. > > .is_writable() > > Returns True if the object was opened write writing, False > otherwise. If False, .write() and .truncate() will raise an IOError > if called. > > .is_seekable() (Should this be called .is_random()? or > .is_sequential() with opposite return values?) > > Returns True if the object supports random-access (such as disk > files), or False if the object only supports sequential access (such > as sockets, pipes, and ttys). If False, .seek(), .tell(), and > .truncate() will raise an IOError if called. > > Iff a RawIOBase implementation operates on an underlying file > descriptor, it must additionally provide a .fileno() member function. > This could be defined specifically by the implementation, or a mix-in > class could be used (Need to decide about this). > > .fileno() > > Returns the underlying file descriptor (an integer) > > Initially, three implementations will be provided that implement the > RawIOBase interface: FileIO, SocketIO, and ByteIO (also MMapIO?). > Each implementation must determine whether the object supports random > access as the information provided by the user may not be sufficient > (consider open("/dev/tty", "rw") or open("/tmp/named-pipe", "rw"). As > an example, FileIO can determine this by calling the seek() system > call; if it returns an error, the object does not support random > access. Each implementation may provided additional methods > appropriate to its type. The ByteIO object is analogous to Python 2's > cStringIO library, but operating on the new bytes type instead of > strings. > Buffered I/O > The next layer is the Buffer I/O layer which provides more efficient > access to file-like objects. The abstract base class for all Buffered > I/O implementations is BufferedIOBase, which provides similar methods > to RawIOBase: > > .read() > .write() > .seek() > .tell() > .truncate() > .close() > .is_readable() > .is_writable() > .is_seekable() > > Additionally, the abstract base class provides one member variable: > > .raw > > Provides a reference to the underling RawIOBase object. > > The BufferIOBase methods' syntax is identical to that of RawIOBase, > but may have different semantics. In particular, BufferIOBase > implementations may read more data than requested or delay writing > data using buffers. For the most part, this will be transparent to > the user (unless, for example, they open the same file through a > different descriptor). > > There are four implementations of the BufferIOBase abstract base > class, described below. > BufferedReader > The BufferedReader implementation is for sequential-access read-only > objects. It does not provide a .flush() method, since there is no > sensible circumstance where the user would want to discard the read > buffer. > BufferedWriter > The BufferedWriter implementation is for sequential-access write-only > objects. It provides a .flush() method, which forces all cached data > to be written to the underlying RawIOBase object. > BufferedRWPair > The BufferRWPair implementation is for sequential-access read-write > objects such as sockets and ttys. As the read and write streams of > these objects are completely independent, it could be implemented by > simply incorporating a BufferedReader and BufferedWriter instance. It > provides a .flush() method that has the same semantics as a > BufferWriter's .flush() method. > BufferedRandom > The BufferRandom implementation is for all random-access objects, > whether they are read-only, write-only, or read-write. Compared to > the previous classes that operate on sequential-access objects, the > BufferedRandom class must contend with the user calling .seek() to > reposition the stream. Therefore, an instance of BufferRandom must > keep track of both the logical and true position within the object. > It provides a .flush() method that forces all cached write data to be > written to the underlying RawIOBase object and all cached read data to > be forgotten (so that future reads are forced to go back to the disk). > > Q: Do we want to mandate in the specification that switching between > reading to writing on a read-write object implies a .flush()? Or is > that an implementation convenience that users should not rely on? > > For a read-only BufferRandom object, .is_writable() returns False and > the .write() and .truncate() methods throw IOError. > > For a write-only BufferRandom object, .is_readable() returns False and > the .read() method throws IOError. > Text I/O > The text I/O layer provides functions to read and write strings from > streams. Some new features include universal newlines and character > set encoding and decoding. The Text I/O layer is defined by a > TextIOBase abstract base class. It provides several methods that are > similar to the BufferIOBase methods, but operate on a per-character > basis instead of a per-byte basis. These methods are: > > .read() > .write() > .seek() > .tell() > .truncate() > > TextIOBase implementations also provide several methods that are > pass-throughs to the underlaying BufferIOBase objects: > > .close() > .is_readable() > .is_writable() > .is_seekable() > > TextIOBase class implementations additionally provide the following methods: > > .readline(self) > > Read until newline or EOF and return the line. > > .readlinesiter() > > Returns an iterator that returns lines from the file (which > happens to be 'self'). > > .next() > > Same as readline() > > .__iter__() > > Same as readlinesiter() > > .__enter__() > > Context management protocol. Returns self. > > .__exit__() > > Context management protocol. No-op. > > Two implementations will be provided by the Python library. The > primary implementation, TextIOWrapper, wraps a Buffered I/O object. > Each TextIOWrapper object has a property name ".buffer" that provides > a reference to the underlying BufferIOBase object. It's initializer > has the following signature: > > .__init__(self, buffer, encoding=None, universal_newlines=True, crlf=None) > > Buffer is a reference to the BufferIOBase object to be wrapped > with the TextIOWrapper. "Encoding" refers to an encoding to be used > for translating between the byte-representation and > character-representation. If "None", then the system's locale setting > will be used as the default. If "universal_newlines" is true, then > the TextIOWrapper will automatically translate the bytes "\r\n" into a > single newline character during reads. If "crlf" is False, then a > newline will be written as "\r\n". If "crlf" is True, then a newline > will be written as "\n". If "crlf" is None, then a system-specific > default will be used. > > Another way to do it is as follows (we should pick one or the other): > > .__init__(self, buffer, encoding=None, newline=None) > > Same as above but if newline is not None use that as the > newline pattern (for reading and writing), and if newline is not set > attempt to find the newline pattern from the file and if we can't for > some reason use the system default newline pattern. > > Another implementation, StringIO, creates a file-like TextIO > implementation without an underlying Buffer I/O object. While similar > functionality could be provided by wrapping a BytesIO object in a > Buffered I/O object in a TextIOWrapper, the String I/O object allows > for much greater efficiency as it does not need to actually performing > encoding and decoding. A String I/O object can just store the encoded > string as-is. The String I/O object's __init__ signature is similar > to the TextIOWrapper, but without the "buffer" parameter. > > END OF PEP > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/daniel%40stutzbachenterprises.com > -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From greg.ewing at canterbury.ac.nz Thu Mar 1 00:57:05 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 01 Mar 2007 12:57:05 +1300 Subject: [Python-3000] PEP Draft: Enhancing the buffer protcol In-Reply-To: References: <45E502F7.4070603@canterbury.ac.nz> Message-ID: <45E616D1.4030905@canterbury.ac.nz> Travis E. Oliphant wrote: > Yes, this was the reason for my dtype object. But, I think that folks > felt it was too much, especially since the struct-style syntax is > already there in Python. Making it a full-blown Python object would be too much for this application. But it could be something like an array of structs containing a type code and a size. > Besides not allowing for the request of a "contiguous" buffer Add an argument as appropriate. > you are also not describing how allocation for > this array of structs will be handled. That's up to the base object. Something with a fixed number of dimensions (as I expect most objects implementing this protocol will) can store it in the object itself. If the number of dimensions can vary, it might have to malloc/realloc. But that doesn't seem like a difficult thing to do. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 1 00:57:11 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 01 Mar 2007 12:57:11 +1300 Subject: [Python-3000] PEP Draft: Enhancing the buffer protcol In-Reply-To: References: Message-ID: <45E616D7.1060301@canterbury.ac.nz> Daniel Stutzbach wrote: > On 2/27/07, Travis E. Oliphant wrote: >>But I've heard that the reference counts on >>Python objects can be larger than 1 in some cases (even though there >>isn't really anything "viewing" the memory). > > Is that true? The interpreter machinery sometimes keeps temporary references internally, e.g. when passing args to a function, so sometimes the refcount is a bit higher than you would expect from thinking about it at the Python level. -- Greg From oliphant.travis at ieee.org Thu Mar 1 01:22:36 2007 From: oliphant.travis at ieee.org (Travis Oliphant) Date: Wed, 28 Feb 2007 17:22:36 -0700 Subject: [Python-3000] PEP Draft: Enhancing the buffer protcol In-Reply-To: <45E616D1.4030905@canterbury.ac.nz> References: <45E502F7.4070603@canterbury.ac.nz> <45E616D1.4030905@canterbury.ac.nz> Message-ID: > Travis E. Oliphant wrote: > > >>Yes, this was the reason for my dtype object. But, I think that folks >>felt it was too much, especially since the struct-style syntax is >>already there in Python. > > > Making it a full-blown Python object would be too much > for this application. But it could be something like an > array of structs containing a type code and a size. > Another thing going in the struct-syntax's favor is that it seems to be accepted by a wide number of Python devs. But, I'm open to anything that contains the same level of expressiveness. >>you are also not describing how allocation for >>this array of structs will be handled. > > > That's up to the base object. Something with a fixed > number of dimensions (as I expect most objects implementing > this protocol will) can store it in the object itself. > If the number of dimensions can vary, it might have to > malloc/realloc. But that doesn't seem like a difficult > thing to do. I see better now. The objects themselves would store the dimensions (either directly as part of the object), or in some extension-module-maintained memory (that is used as a buffer to copy into everytime it is needed). That is fine by me. I suppose the format-string could be managed the same way --- it's up to the type object implementing the interface to manage it. -Travis From greg.ewing at canterbury.ac.nz Thu Mar 1 03:02:29 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 01 Mar 2007 15:02:29 +1300 Subject: [Python-3000] unit test for advanced formatting In-Reply-To: <45E5E7F9.2060004@acm.org> References: <45E5E7F9.2060004@acm.org> Message-ID: <45E63435.8060504@canterbury.ac.nz> Talin wrote: > A typical example of what I am talking about here is something like a > web application server, where you have a "development" mode and a > "production" mode. In the development mode, you want to find errors as > quickly as possible, so you enable strict formatting. In production, > however, you want the server to be as fault-tolerant as possible, so you > would enable lenient mode. This seems misguided. There's a difference between being fault-tolerant and being bug-tolerant. I don't think that ignoring symptoms of bugs and going on to produce incorrect results is any more acceptable in a production web server as it would be in any other environment. -- Greg From talin at acm.org Thu Mar 1 03:58:31 2007 From: talin at acm.org (Talin) Date: Wed, 28 Feb 2007 18:58:31 -0800 Subject: [Python-3000] unit test for advanced formatting In-Reply-To: <45E63435.8060504@canterbury.ac.nz> References: <45E5E7F9.2060004@acm.org> <45E63435.8060504@canterbury.ac.nz> Message-ID: <45E64157.3030509@acm.org> Greg Ewing wrote: > Talin wrote: > >> A typical example of what I am talking about here is something like a >> web application server, where you have a "development" mode and a >> "production" mode. In the development mode, you want to find errors as >> quickly as possible, so you enable strict formatting. In production, >> however, you want the server to be as fault-tolerant as possible, so you >> would enable lenient mode. > > This seems misguided. There's a difference between being > fault-tolerant and being bug-tolerant. I don't think that > ignoring symptoms of bugs and going on to produce incorrect > results is any more acceptable in a production web server > as it would be in any other environment. It depends on the kind of web service you are running. For example, if your web site is a bank, then a crash may be preferable to an incorrect total; However if your web site is a blog, then something like "Last posted on ?MissingArgument?" might be preferable to a self-imposed denial of service attack, i.e. a server that produces no pages at all, or pages that say only "An error occurred while generating this page". In any case, this issue doesn't really matter to me. -- Talin From raymond.hettinger at verizon.net Thu Mar 1 07:11:10 2007 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed, 28 Feb 2007 22:11:10 -0800 Subject: [Python-3000] Stars Message-ID: <00dc01c75bc8$6974fd50$7600a8c0@RaymondLaptop1> What we have now: def f(a, b, *args, **dict(c=1)): # Yuck! What we really need: def f(a, b, *args, **, c=1): # Two stars spell dictionary. What I heard was planned instead: def f(a, b, *args, *, c=1): # One star spells iterable. Two stars make a nice visual separator. One star following *args just looks funny, Raymond P.S. I would actually like to see this in Py2.6. It doesn't conflict with anything else and solves an annoying problem. From thomas at python.org Thu Mar 1 08:45:07 2007 From: thomas at python.org (Thomas Wouters) Date: Thu, 1 Mar 2007 08:45:07 +0100 Subject: [Python-3000] Stars In-Reply-To: <00dc01c75bc8$6974fd50$7600a8c0@RaymondLaptop1> References: <00dc01c75bc8$6974fd50$7600a8c0@RaymondLaptop1> Message-ID: <9e804ac0702282345y2d060eeet49778e8bd9ae4539@mail.gmail.com> On 3/1/07, Raymond Hettinger wrote: > > What we have now: > def f(a, b, *args, **dict(c=1)): # Yuck! > > What we really need: > def f(a, b, *args, **, c=1): # Two stars spell dictionary. > > What I heard was planned instead: > def f(a, b, *args, *, c=1): # One star spells iterable. Nope. You can only have one one-star and one two-star. If you want keyword-only arguments *and* arbitrary positional arguments, you can just do it (in Py3k, and it will probably be backported to 2.6.) >>> def foo(a, b, *args, c=1): return a, b, args, c ... >>> foo(1, 2) (1, 2, (), 1) >>> foo(1, 2, 3, 4, c=5) (1, 2, (3, 4), 5) The one-star-only syntax is only if you *don't* want arbitrary positional arguments: >>> def foo(a, b, *, c=1): return a, b, c ... >>> foo(1, 2) (1, 2, 1) >>> foo(1, 2, 5) Traceback (most recent call last): File "", line 1, in TypeError: foo() takes exactly 2 positional arguments (3 given) >>> foo(1, 2, c=5) (1, 2, 5) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070301/c07eb8e1/attachment.html From jimjjewett at gmail.com Thu Mar 1 15:28:58 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 1 Mar 2007 09:28:58 -0500 Subject: [Python-3000] PEP Draft: Enhancing the buffer protcol In-Reply-To: References: Message-ID: On 2/27/07, Travis E. Oliphant wrote: > Rationale (This was a good section; thank you.) > 5. There is no shape information provided for the memory. But, > several array-like Python types could make use of a standard > way to describe the shape-interpretation of the memory > (!wxPython, GTK, pyQT, CVXOPT, !PyVox, Audio and Video > Libraries, ctypes, !NumPy, data-base interfaces, etc.) Are the "!" characters meaningful? > This view object should be used in the other API calls and > does not need to be decref'd. It should be "released" if the > interface exporter provides the bf_releasebuffer function. Pity you can't use decref instead of release ... could the description object be treated as a real object, and decrefed to indicate release? (Or are there lots of tiny buffers with identical large descriptions?) > Additions to the struct string-syntax > > The struct string-syntax is missing some characters to fully > implement data-format descriptions already available elsewhere (in > ctypes and NumPy for example). Here are the proposed additions: > > Character Description > ================================== > '1' bit (number before states how many bits) How do you distinguish the "1" for bit from a trailing 1 in a number? It sounds like "21" means 2 bits "2g" means 2 long doubles "21g" means either 2 bits and a long double, or 21 long doubles. > Code should also be able to request contiguous memory if needed and > objects exporting the buffer interface should be able to handle > that either by raising an error Do you want to specify the error? -jJ From rhamph at gmail.com Thu Mar 1 15:50:12 2007 From: rhamph at gmail.com (Adam Olsen) Date: Thu, 1 Mar 2007 07:50:12 -0700 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: On 2/28/07, Daniel Stutzbach wrote: > What should Buffered I/O .write() do for a non-blocking object? > > It seems like the .write() should write as much as it can to the Raw > I/O object and buffer the rest, but then how do we tell the Buffered > I/O object to "write more data from the buffer but still don't block"? > > Along the same lines, for a non-blocking Buffer I/O object, how do we > specify "Okay, I know I've been writing only one byte a time so you > probably haven't bothered writing it to the raw object. Write as much > data as you can now, but don't block". > > Option #1: On a non-blocking object, .flush() writes as much as it > can, but won't block. It would need a return value then, to indicate > whether the flush completed or not. > > Option #2: Calling .write() with no arguments causes the Buffer I/O > object to flush as much write data to the raw object, but won't block. > (For a blocking object, it would block until all data is written to > the raw object). > > I prefer option #2 because a .flush() that doesn't flush is more surprising. > > The goal of supporting non-blocking file-like objects is to be able to > use select() with buffered I/O objects (and other things like a > compressed socket stream). Why do non-blocking operations need to use the same methods when they're clearly not the same semantics? Although long, .nonblockflush() would be explicit and allow .flush() to still block. I'm especially wary of infinite buffers. They allow a malicious peer to consume all your memory, DoSing the process or even the whole box if Linux's OOM killer doesn't kick in fast enough. -- Adam Olsen, aka Rhamphoryncus From mike.verdone at gmail.com Thu Mar 1 17:07:18 2007 From: mike.verdone at gmail.com (Mike Verdone) Date: Thu, 1 Mar 2007 10:07:18 -0600 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> The current algorithm for non-blocking writes is as follows: When write is called check if the buffer is bigger than buffer_size. If it is attempt to pre-flush the buffer. If we can't pre-flush, raise BlockingIO error. Else, copy the new write data into the buffer (flushed or not) and if we're bigger than buffer_size try to flush again. If we couldn't flush the buffer, check that the remaining buffer is smaller than max_buffer_size. If the buffer is less than max_buffer_size return, if the buffer is greater than max_buffer_size, truncate the buffer and throw BlockingIO error, informing the user how many bytes were written/accepted into the buffer. This code is in the patch I sent to Guido for review. It hasn't been checked in yet. I'm thinking it might be better to have only one buffer size. We want to avoid partial writes as much as possible, but the algorithm seems overly complicated. I may take a look at this again this weekend. Mike. On 3/1/07, Adam Olsen wrote: > On 2/28/07, Daniel Stutzbach wrote: > > What should Buffered I/O .write() do for a non-blocking object? > > > > It seems like the .write() should write as much as it can to the Raw > > I/O object and buffer the rest, but then how do we tell the Buffered > > I/O object to "write more data from the buffer but still don't block"? > > > > Along the same lines, for a non-blocking Buffer I/O object, how do we > > specify "Okay, I know I've been writing only one byte a time so you > > probably haven't bothered writing it to the raw object. Write as much > > data as you can now, but don't block". > > > > Option #1: On a non-blocking object, .flush() writes as much as it > > can, but won't block. It would need a return value then, to indicate > > whether the flush completed or not. > > > > Option #2: Calling .write() with no arguments causes the Buffer I/O > > object to flush as much write data to the raw object, but won't block. > > (For a blocking object, it would block until all data is written to > > the raw object). > > > > I prefer option #2 because a .flush() that doesn't flush is more surprising. > > > > The goal of supporting non-blocking file-like objects is to be able to > > use select() with buffered I/O objects (and other things like a > > compressed socket stream). > > Why do non-blocking operations need to use the same methods when > they're clearly not the same semantics? Although long, > .nonblockflush() would be explicit and allow .flush() to still block. > > I'm especially wary of infinite buffers. They allow a malicious peer > to consume all your memory, DoSing the process or even the whole box > if Linux's OOM killer doesn't kick in fast enough. > > -- > Adam Olsen, aka Rhamphoryncus > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/mike.verdone%40gmail.com > From brett at python.org Thu Mar 1 18:09:26 2007 From: brett at python.org (Brett Cannon) Date: Thu, 1 Mar 2007 11:09:26 -0600 Subject: [Python-3000] How far to go with cleaning up exceptions Message-ID: I spent my sprint time cleaning up exceptions for Py3K and Guido suggested I run some things by the group to make sure they don't make transitioning from 2.6 too difficult. After adding the proper restrictions in terms of what can and cannot be raised or caught, I began cleaning up BaseException's API per PEP 352. First thing I did was remove slicing. That wasn't too bad since I just got exceptions about the lack of __getitem__ and went to the affected line and tossed in '.args' between the index and the exception. Not sure if 2to3 can handle this as it would need to realize when an exception is within scope and then insert '.args' when it is be indexed on. The next thing I did was strip out the use of '.args' and make BaseException take a single argument that gets assigned to 'message'. The fruits of this labour are in the p3yk_no_args_on_exc branch. This one turned out to be a pain in the rear. Probably the biggest headache were the built-in exceptions themselves and various exceptions specified by extension modules. Many of the built-in modules assumed 'args' would be there and so would pull from them as needed instead of assigning None to their various attributes. It also required implementing several __reduce__ methods as BaseException's just return 'args' which kept things simple. Plus their constructors had to be made to use BaseException's more restrictive constructor signature. For the extension modules, though, the biggest pain was the fact that most of them use PyErr_NewException() to create their exception objects. That provides no easy way of dealing with the more restrictive constructor on BaseException. For bsddb.DBError I just made it take one argument and forced all calls to pass in a tuple. For socket.error I defined the exception in Python and used a trick _bsddb.c uses so that the __init__ for socket.error stuck any second argument on to the 'errno' attribute. In other words the more restrictive construtor is going to be what causes the most pain. Exceptions that expect more than one argument are going to need to be rewritten so that if they get more than one argument they call BaseException.__init__ with only one and store away the rest in some other attribute. This kind of sucks at the C level, especially when most defined there just use PyErr_NewException(). Then the various accesses that would have gone to 'args' would need to be changed to access the specific attribute. For solving these issues, there a couple possibilities. For those exceptions that use PyErr_NewException() there could be a PyErr_NewExceptionEx() that takes an init function for thew new exception. As for transforming the 'args' accesses to something else, I guess 2to3 could automatically do 'args[0]' accesses to 'message', but anything else will need to be changed by hand. Now, for the question: is all of this worth it? If this happens exceptions will have a much nicer interface. By default you will have just 'message'. it will also force exceptions to explicitly set attributes instead of using the position within 'args' to get to a specific piece of information. But obviously there are transition pains in doing this. If you have an opinion on this please speak up. -Brett From jcarlson at uci.edu Thu Mar 1 18:26:19 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Thu, 01 Mar 2007 09:26:19 -0800 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: <20070301091455.AE8B.JCARLSON@uci.edu> "Daniel Stutzbach" wrote: [snip] > The goal of supporting non-blocking file-like objects is to be able to > use select() with buffered I/O objects (and other things like a > compressed socket stream). You can already pass sockets, files, and pipes to select (on linux, bsd, etc.). Being able to pass files and pipes to select on Windows, seems to me, a "doomed to fail" design goal - Windows select doesn't support them. Windows IOCP, or even a WaitForMultipleObjects call *may*, but then it's not select (unless the plan was to rewrite the select module for Windows). And honestly, when I'm dealing with non-blocking objects, I much prefer to deal with buffers myself, *especially* if I'm going to be putting them in a select loop or equivalent. Building-in buffering to non-blocking sockets, files, and pipes, seems to be more than a little bit of over-engineering (then again, the whole new IO system seems to be over-engineered, I haven't had any issues with the *current* system, and the new one with pluggable layers feels a bit like using Twisted for an echo server - no offense to the Twisted folks). - Josiah From jimjjewett at gmail.com Thu Mar 1 19:41:56 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 1 Mar 2007 13:41:56 -0500 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: On 2/28/07, Daniel Stutzbach wrote: > What should Buffered I/O .write() do for a non-blocking object? > It seems like the .write() should write as much as it can to the Raw > I/O object and buffer the rest, but then how do we tell the Buffered > I/O object to "write more data from the buffer but still don't block"? Why bother? The buffer layer should ensure that it gets written eventually, if only at buffer finalization. A (blocking) flush says "commit whatever you have *right now*". Needing anything in between is probably rare enough that it makes sense to let concrete objects handle it on their own, without a standardized method. This semi-flush seems far less important than a non-blocking read, which Guido has already said doesn't need standardization. > Option #2: Calling .write() with no arguments causes the Buffer I/O > object to flush as much write data to the raw object, but won't block. Writing a zero-length string should probably give it another chance to process the buffer if it wants to, but ... I don't think it should be *forced* to try, if it doesn't think there is enough data yet. That is what flush is for. -jJ From avassalotti at acm.org Fri Mar 2 05:08:39 2007 From: avassalotti at acm.org (Alexandre Vassalotti) Date: Thu, 1 Mar 2007 23:08:39 -0500 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> <457F3FA1.7010105@canterbury.ac.nz> Message-ID: On 12/18/06, Guido van Rossum wrote: > I think a reasonable solution here is to make the C version an > optional implementation detail of the Python version, such as was done > for the heapq module already (the C version is _heapq and > automatically imported by heapq.py if it exists). If this requires > that some of the C modules need to be upgraded to be more feature > compatible with the Python versions, so be it -- that would be a nice > summer-of-code project for someone. Also, the specific problem with > StringIO (that the .py version supports Unicode while the C version > doesn't) will hopefully go away with the string unification. > I think I will jump on this opportunity, if nobody else has proposed himself. I am not sure yet, how much coding this project will involve, but it is certainly an interesting Summer of Code project. I think I will try to write a PEP about the required changes, before submitting my application. So this way, I will have a clear understanding of the project. Also, I would appreciate it if someone could tell me what are the skills, which the students are expected to have, for Python core development. Kind Regards, -- Alexandre From guido at python.org Fri Mar 2 06:03:20 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 1 Mar 2007 21:03:20 -0800 Subject: [Python-3000] Work overload causing review delays Message-ID: All, I have to request your continued patience. Several of you have sent me code or ideas to review. I *will* get to it, probably early next week; but recovery from PyCon, first physical, then from the backlog at work, has made it necessary to prioritize things. While I have minutes here and there to respond to scattered email threads, the time needed to concentrate on deeper issues and code reviews seems lacking for now. It'll get better. Sorry! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Fri Mar 2 22:35:38 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 2 Mar 2007 16:35:38 -0500 Subject: [Python-3000] remove tuple exceptions? Message-ID: What is the reasoning behind allowing the raise of a tuple -- but really only raising its (recursively) first element? It seems to have been added (with different spelling) in 1991 (rev 2625) as the only alternative to string exceptions. (You couldn't raise a class or instance.) I assume it was kept for backwards compatibility. Were there other reasons, or should this be removed in python 3? Looking at ceval.c /* We support the following forms of raise: ... raise , None ... In addition, raise , is the same as raising the tuple's first item (and it better have one!); this rule is applied recursively. ... /* Next, repeatedly, replace a tuple exception with its first item */ while (PyTuple_Check(type) && PyTuple_Size(type) > 0) { PyObject *tmp = type; type = PyTuple_GET_ITEM(type, 0); Py_INCREF(type); Py_DECREF(tmp); } From exarkun at divmod.com Fri Mar 2 23:44:22 2007 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 2 Mar 2007 17:44:22 -0500 Subject: [Python-3000] remove tuple exceptions? In-Reply-To: Message-ID: <20070302224422.17852.1426065531.divmod.quotient.2052@ohm> On Fri, 2 Mar 2007 16:35:38 -0500, Jim Jewett wrote: >What is the reasoning behind allowing the raise of a tuple -- but >really only raising its (recursively) first element? > This, basically: SomeCondition = (FooException, BarException) AnotherCondition = (SomeCondition, BazException) try: ... except SomeCondition, e: ... raise AnotherCondition Jean-Paul From collinw at gmail.com Sat Mar 3 00:16:45 2007 From: collinw at gmail.com (Collin Winter) Date: Fri, 2 Mar 2007 17:16:45 -0600 Subject: [Python-3000] remove tuple exceptions? In-Reply-To: References: Message-ID: <43aa6ff70703021516m1efe5199v1361509991629e6@mail.gmail.com> On 3/2/07, Jim Jewett wrote: > What is the reasoning behind allowing the raise of a tuple -- but > really only raising its (recursively) first element? > > It seems to have been added (with different spelling) in 1991 (rev > 2625) as the only alternative to string exceptions. (You couldn't > raise a class or instance.) I assume it was kept for backwards > compatibility. > > Were there other reasons, or should this be removed in python 3? PEP 3109 already talks about removing this in Python 3. This will bring "raise" into line with generator.throw(), which already disallows this. Collin Winter From greg.ewing at canterbury.ac.nz Sat Mar 3 00:42:45 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 03 Mar 2007 12:42:45 +1300 Subject: [Python-3000] remove tuple exceptions? In-Reply-To: <20070302224422.17852.1426065531.divmod.quotient.2052@ohm> References: <20070302224422.17852.1426065531.divmod.quotient.2052@ohm> Message-ID: <45E8B675.5080602@canterbury.ac.nz> Jean-Paul Calderone wrote: > On Fri, 2 Mar 2007 16:35:38 -0500, Jim Jewett wrote: > >>What is the reasoning behind allowing the raise of a tuple > > SomeCondition = (FooException, BarException) > AnotherCondition = (SomeCondition, BazException) > > except SomeCondition, e: > ... > raise AnotherCondition So it was some sort of workaround for exceptions not being classes? If so, it sounds like we don't need it any more. -- Greg From guido at python.org Sat Mar 3 01:23:41 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 2 Mar 2007 16:23:41 -0800 Subject: [Python-3000] remove tuple exceptions? In-Reply-To: <45E8B675.5080602@canterbury.ac.nz> References: <20070302224422.17852.1426065531.divmod.quotient.2052@ohm> <45E8B675.5080602@canterbury.ac.nz> Message-ID: On 3/2/07, Greg Ewing wrote: > So it was some sort of workaround for exceptions not > being classes? If so, it sounds like we don't need > it any more. Right. I added it to be able to do something like create classes of exceptions when all we had was string exceptions. Definitely an early mistake I want to correct in Py3k! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Sat Mar 3 01:39:51 2007 From: collinw at gmail.com (Collin Winter) Date: Fri, 2 Mar 2007 18:39:51 -0600 Subject: [Python-3000] How far to go with cleaning up exceptions In-Reply-To: References: Message-ID: <43aa6ff70703021639p84beae1j883d44385d8046dc@mail.gmail.com> On 3/1/07, Brett Cannon wrote: > I spent my sprint time cleaning up exceptions for Py3K and Guido > suggested I run some things by the group to make sure they don't make > transitioning from 2.6 too difficult. After adding the proper > restrictions in terms of what can and cannot be raised or caught, I > began cleaning up BaseException's API per PEP 352. > > First thing I did was remove slicing. That wasn't too bad since I > just got exceptions about the lack of __getitem__ and went to the > affected line and tossed in '.args' between the index and the > exception. Not sure if 2to3 can handle this as it would need to > realize when an exception is within scope and then insert '.args' when > it is be indexed on. 2to3 currently cannot handle this. 2.6's "py3k-compat" mode could handle it easily, though. > The next thing I did was strip out the use of '.args' and make > BaseException take a single argument that gets assigned to 'message'. > The fruits of this labour are in the p3yk_no_args_on_exc branch. This > one turned out to be a pain in the rear. [snip] I took at stab at this same conversion a few weeks ago and had the exact same experience. I ended up resolving my frustration by applying "svn revert" and working on something else : ) I think this will shape up to be one of the harder parts of porting from 2.x to 3.0. > As for transforming the 'args' accesses to something else, > I guess 2to3 could automatically do 'args[0]' accesses to 'message', > but anything else will need to be changed by hand. There's very little 2to3 can do on this front. I think 2.6's py3k-compat flag is going to be the best bet for this particular conversion. Collin Winter From brett at python.org Sat Mar 3 01:56:59 2007 From: brett at python.org (Brett Cannon) Date: Fri, 2 Mar 2007 16:56:59 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) Message-ID: Those of you who attended PyCon 2007 probably know what PEP 3113 is all about: the removal of the automatic unpacking of sequence arguments when a tuple parameter is present in a function's signature (e.g., the ``(b, c)`` in ``def fxn(a, (b, c), d): pass``). Thanks to everyone who helped provide arguments for as to why they need to go (and even Ping who said he likes these things =). The PEP has been committed so those of you who prefer reading PEPs through a browser should be able to in about an hour. ------------------------------------------------- Abstract ======== Tuple parameter unpacking is the use of a tuple as a parameter in a function signature so as to have a sequence argument automatically unpacked. An example is:: def fxn(a, (b, c), d): pass The use of ``(b, c)`` in the signature requires that the second argument to the function be a sequence of length two (e.g., ``[42, -13]``). When such a sequence is passed it is unpacked and has its values assigned to the parameters, just as if the statement ``b, c = [42, -13]`` had been executed in the parameter. Unfortunately this feature of Python's rich function signature abilities, while handy in some situations, causes more issues than they are worth. Thus this PEP proposes their removal from the language in Python 3.0. Why They Should Go ================== Introspection Issues -------------------- Python has very powerful introspection capabilities. These extend to function signatures. There are no hidden details as to what a function's call signature is. In general it is fairly easy to figure out various details about a function's signature by viewing the function object and various attributes on it (including the function's ``func_code`` attribute). But there is great difficulty when it comes to tuple parameters. The existence of a tuple parameter is denoted by it name being made of a ``.`` and a number in the ``co_varnames`` attribute of the function's code object. This allows the tuple argument to be bound to a name that only the bytecode is aware of and cannot be typed in Python source. But this does not specify the format of the tuple: its length, whether there are nested tuples, etc. In order to get all of the details about the tuple from the function one must analyse the bytecode of the function. This is because the first bytecode in the function literally translates into the tuple argument being unpacked. Assuming the tuple parameter is named ``.1`` and is expected to unpack to variables ``spam`` and ``monty`` (meaning it is the tuple ``(spam, monty)``), the first bytecode in the function will be for the statement ``spam, monty = .1``. This means that to know all of the details of the tuple parameter one must look at the initial bytecode of the function to detect tuple unpacking for parameters formatted as ``\.\d+`` and deduce any and all information about the expected argument. Bytecode analysis is how the ``inspect.getargspec`` function is able to provide information on tuple parameters. This is not easy to do and is burdensome on introspection tools as they must know how Python bytecode works (an otherwise unneeded burden as all other types of parameters do not require knowledge of Python bytecode). The difficulty of analysing bytecode not withstanding, there is another issue with the dependency on using Python bytecode. IronPython [#ironpython]_ does not use Python's bytecode. Because it is based on the .NET framework it instead stores MSIL [#MSIL]_ in ``func_code.co_code`` attribute of the function. This fact prevents the ``inspect.getargspec`` function from working when run under IronPython. It is unknown whether other Python implementations are affected but is reasonable to assume if the implementation is not just a re-implementation of the Python virtual machine. No Loss of Abilities If Removed ------------------------------- As mentioned in `Introspection Issues`_, to handle tuple parameters the function's bytecode starts with the bytecode required to unpack the argument into the proper parameter names. This means that their is no special support required to implement tuple parameters and thus there is no loss of abilities if they were to be removed, only a possible convenience (which is addressed in `Why They Should (Supposedly) Stay`_). The example function at the beginning of this PEP could easily be rewritten as:: def fxn(a, b_c, d): b, c = b_c pass and in no way lose functionality. Exception To The Rule --------------------- When looking at the various types of parameters that a Python function can have, one will notice that tuple parameters tend to be an exception rather than the rule. Consider PEP 3102 (keyword-only arguments) and PEP 3107 (function annotations) [#pep-3102]_ [#pep-3107]_. Both PEPs have been accepted and introduce new functionality within a function's signature. And yet for both PEPs the new feature cannot be applied to tuple parameters. This leads to tuple parameters not being on the same footing as regular parameters that do not attempt to unpack their arguments. The existence of tuple parameters also places sequence objects separately from mapping objects in a function signature. There is no way to pass in a mapping object (e.g., a dict) as a parameter and have it unpack in the same fashion as a sequence does into a tuple parameter. Uninformative Error Messages ---------------------------- Consider the following function:: def fxn((a, b), (c, d)): pass If called as ``fxn(1, (2, 3))`` one is given the error message ``TypeError: unpack non-sequence``. This error message in no way tells you which tuple was not unpacked properly. There is also no indication that this was a result that occurred because of the arguments. Other error messages regarding arguments to functions explicitly state its relation to the signature: ``TypeError: fxn() takes exactly 2 arguments (0 given)``, etc. Little Usage ------------ While an informal poll of the handful of Python programmers I know personally and from the PyCon 2007 sprint indicates a huge majority of people do not know of this feature and the rest just do not use it, some hard numbers is needed to back up the claim that the feature is not heavily used. Iterating over every line in Python's code repository in the ``Lib/`` directory using the regular expression ``^\s*def\s*\w+\s*\(`` to detect function and method definitions there were 22,252 matches in the trunk. Tacking on ``.*,\s*\(`` to find ``def`` statements that contained a tuple parameter, only 41 matches were found. This means that for ``def`` statements, only 0.18% of them seem to use a tuple parameter. Why They Should (Supposedly) Stay ================================= Practical Use ------------- In certain instances tuple parameters can be useful. A common example is code that expect a two-item tuple that reperesents a Cartesian point. While true it is nice to be able to have the unpacking of the x and y coordinates for you, the argument is that this small amount of practical usefulness is heavily outweighed by other issues pertaining to tuple parameters. And as shown in `No Loss Of Abilities If Removed`_, their use is purely practical and in no way provide a unique ability that cannot be handled in other ways very easily. Self-Documentation For Parameters --------------------------------- It has been argued that tuple parameters provide a way of self-documentation for parameters that are expected to be of a certain sequence format. Using our Cartesian point example from `Practical Use`_, seeing ``(x, y)`` as a parameter in a function makes it obvious that a tuple of length two is expected as an argument for that parameter. But Python provides several other ways to document what parameters are for. Documentation strings are meant to provide enough information needed to explain what arguments are expected. Tuple parameters might tell you the expected length of a sequence argument, it does not tell you what that data will be used for. One must also read the docstring to know what other arguments are expected if not all parameters are tuple parameters. Function annotations (which do not work with tuple parameters) can also supply documentation. Because annotations can be of any form, what was once a tuple parameter can be a single argument parameter with an annotation of ``tuple``, ``tuple(2)``, ``Cartesian point``, ``(x, y)``, etc. Annotations provide great flexibility for documenting what an argument is expected to be for a parameter, including being a sequence of a certain length. Transition Plan =============== To transition Python 2.x code to 3.x where tuple parameters are removed, two steps are suggested. First, the proper warning is to be emitted when Python's compiler comes across a tuple parameter in Python 2.6. This will be treated like any other syntactic change that is to occur in Python 3.0 compared to Python 2.6. Second, the 2to3 refactoring tool [#2to3]_ will gain a rule for translating tuple parameters to being a single parameter this is unpacked as the first statement in the function. The name of the new parameter will be a mangling of tuple parameter's names by joining them with underscores. The new parameter will then be unpacked into the names originally used in the tuple parameter. This means that the following function:: def fxn((a, (b, c))): pass will be translated into:: def fxn(a_b_c): (a, (b, c)) = a_b_c pass References ========== .. [#2to3] 2to3 refactoring tool (http://svn.python.org/view/sandbox/trunk/2to3/) .. [#ironpython] IronPython (http://www.codeplex.com/Wiki/View.aspx?ProjectName=IronPython) .. [#MSIL] Microsoft Intermediate Language (http://msdn.microsoft.com/library/en-us/cpguide/html/cpconmicrosoftintermediatelanguagemsil.asp?frame=true) .. [#pep-3102] PEP 3102 (Keyword-Only Arguments) (http://www.python.org/dev/peps/pep-3102/) .. [#pep-3107] PEP 3107 (Function Annotations) (http://www.python.org/dev/peps/pep-3107/) From brett at python.org Sat Mar 3 02:00:17 2007 From: brett at python.org (Brett Cannon) Date: Fri, 2 Mar 2007 17:00:17 -0800 Subject: [Python-3000] How far to go with cleaning up exceptions In-Reply-To: <43aa6ff70703021639p84beae1j883d44385d8046dc@mail.gmail.com> References: <43aa6ff70703021639p84beae1j883d44385d8046dc@mail.gmail.com> Message-ID: On 3/2/07, Collin Winter wrote: > On 3/1/07, Brett Cannon wrote: > > I spent my sprint time cleaning up exceptions for Py3K and Guido > > suggested I run some things by the group to make sure they don't make > > transitioning from 2.6 too difficult. After adding the proper > > restrictions in terms of what can and cannot be raised or caught, I > > began cleaning up BaseException's API per PEP 352. > > > > First thing I did was remove slicing. That wasn't too bad since I > > just got exceptions about the lack of __getitem__ and went to the > > affected line and tossed in '.args' between the index and the > > exception. Not sure if 2to3 can handle this as it would need to > > realize when an exception is within scope and then insert '.args' when > > it is be indexed on. > > 2to3 currently cannot handle this. 2.6's "py3k-compat" mode could > handle it easily, though. > Yep. I exchanged some emails with Guido privately and this was the conclusion reached. > > The next thing I did was strip out the use of '.args' and make > > BaseException take a single argument that gets assigned to 'message'. > > The fruits of this labour are in the p3yk_no_args_on_exc branch. This > > one turned out to be a pain in the rear. > [snip] > > I took at stab at this same conversion a few weeks ago and had the > exact same experience. I ended up resolving my frustration by applying > "svn revert" and working on something else : ) > =) The people at the sprint (especially days 2 and 3) got to hear my frustrations constantly. > I think this will shape up to be one of the harder parts of porting > from 2.x to 3.0. > I agree, which is why I am asking if people are still supportive of this? Are you, Collin? > > As for transforming the 'args' accesses to something else, > > I guess 2to3 could automatically do 'args[0]' accesses to 'message', > > but anything else will need to be changed by hand. > > There's very little 2to3 can do on this front. I think 2.6's > py3k-compat flag is going to be the best bet for this particular > conversion. I think the slice removal is easy. But getting rid of 'args' and making BaseException's constructor only take a single argument is not painless. But a warning should help get this done in a reasonable fashion. -Brett From guido at python.org Sat Mar 3 02:16:19 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 2 Mar 2007 17:16:19 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: I am all for this; I don't think the advantage it has for the small minority of fans is all that big. Has anyone tried to create a 2to3 transformer for this? I think it should be possible, although there are a few warts (like inserting the new code after the docstring, and how to come up with names for the anonymous tuple(s), and having to do it for potentially any number of arguments. Still, it would require no dataflow analysis, and the except transformer already has a similar statement insertion. So I think it's doable. On 3/2/07, Brett Cannon wrote: > Those of you who attended PyCon 2007 probably know what PEP 3113 is > all about: the removal of the automatic unpacking of sequence > arguments when a tuple parameter is present in a function's signature > (e.g., the ``(b, c)`` in ``def fxn(a, (b, c), d): pass``). Thanks to > everyone who helped provide arguments for as to why they need to go > (and even Ping who said he likes these things =). > > The PEP has been committed so those of you who prefer reading PEPs > through a browser should be able to in about an hour. > > ------------------------------------------------- > > Abstract > ======== > > Tuple parameter unpacking is the use of a tuple as a parameter in a > function signature so as to have a sequence argument automatically > unpacked. An example is:: > > def fxn(a, (b, c), d): > pass > > The use of ``(b, c)`` in the signature requires that the second > argument to the function be a sequence of length two (e.g., > ``[42, -13]``). When such a sequence is passed it is unpacked and > has its values assigned to the parameters, just as if the statement > ``b, c = [42, -13]`` had been executed in the parameter. > > Unfortunately this feature of Python's rich function signature > abilities, while handy in some situations, causes more issues than > they are worth. Thus this PEP proposes their removal from the > language in Python 3.0. > > > Why They Should Go > ================== > > Introspection Issues > -------------------- > > Python has very powerful introspection capabilities. These extend to > function signatures. There are no hidden details as to what a > function's call signature is. In general it is fairly easy to figure > out various details about a function's signature by viewing the > function object and various attributes on it (including the function's > ``func_code`` attribute). > > But there is great difficulty when it comes to tuple parameters. The > existence of a tuple parameter is denoted by it name being made of a > ``.`` and a number in the ``co_varnames`` attribute of the function's > code object. This allows the tuple argument to be bound to a name > that only the bytecode is aware of and cannot be typed in Python > source. But this does not specify the format of the tuple: its > length, whether there are nested tuples, etc. > > In order to get all of the details about the tuple from the function > one must analyse the bytecode of the function. This is because the > first bytecode in the function literally translates into the tuple > argument being unpacked. Assuming the tuple parameter is > named ``.1`` and is expected to unpack to variables ``spam`` and > ``monty`` (meaning it is the tuple ``(spam, monty)``), the first > bytecode in the function will be for the statement > ``spam, monty = .1``. This means that to know all of the details of > the tuple parameter one must look at the initial bytecode of the > function to detect tuple unpacking for parameters formatted as > ``\.\d+`` and deduce any and all information about the expected > argument. Bytecode analysis is how the ``inspect.getargspec`` > function is able to provide information on tuple parameters. This is > not easy to do and is burdensome on introspection tools as they must > know how Python bytecode works (an otherwise unneeded burden as all > other types of parameters do not require knowledge of Python > bytecode). > > The difficulty of analysing bytecode not withstanding, there is > another issue with the dependency on using Python bytecode. > IronPython [#ironpython]_ does not use Python's bytecode. Because it > is based on the .NET framework it instead stores MSIL [#MSIL]_ in > ``func_code.co_code`` attribute of the function. This fact prevents > the ``inspect.getargspec`` function from working when run under > IronPython. It is unknown whether other Python implementations are > affected but is reasonable to assume if the implementation is not just > a re-implementation of the Python virtual machine. > > > No Loss of Abilities If Removed > ------------------------------- > > As mentioned in `Introspection Issues`_, to handle tuple parameters > the function's bytecode starts with the bytecode required to unpack > the argument into the proper parameter names. This means that their > is no special support required to implement tuple parameters and thus > there is no loss of abilities if they were to be removed, only a > possible convenience (which is addressed in > `Why They Should (Supposedly) Stay`_). > > The example function at the beginning of this PEP could easily be > rewritten as:: > > def fxn(a, b_c, d): > b, c = b_c > pass > > and in no way lose functionality. > > > Exception To The Rule > --------------------- > > When looking at the various types of parameters that a Python function > can have, one will notice that tuple parameters tend to be an > exception rather than the rule. > > Consider PEP 3102 (keyword-only arguments) and PEP 3107 (function > annotations) [#pep-3102]_ [#pep-3107]_. Both PEPs have been accepted and > introduce new functionality within a function's signature. And yet > for both PEPs the new feature cannot be applied to tuple parameters. > This leads to tuple parameters not being on the same footing as > regular parameters that do not attempt to unpack their arguments. > > The existence of tuple parameters also places sequence objects > separately from mapping objects in a function signature. There is no > way to pass in a mapping object (e.g., a dict) as a parameter and have > it unpack in the same fashion as a sequence does into a tuple > parameter. > > > Uninformative Error Messages > ---------------------------- > > Consider the following function:: > > def fxn((a, b), (c, d)): > pass > > If called as ``fxn(1, (2, 3))`` one is given the error message > ``TypeError: unpack non-sequence``. This error message in no way > tells you which tuple was not unpacked properly. There is also no > indication that this was a result that occurred because of the > arguments. Other error messages regarding arguments to functions > explicitly state its relation to the signature: > ``TypeError: fxn() takes exactly 2 arguments (0 given)``, etc. > > > Little Usage > ------------ > > While an informal poll of the handful of Python programmers I know > personally and from the PyCon 2007 sprint indicates a huge majority of > people do not know of this feature and the rest just do not use it, > some hard numbers is needed to back up the claim that the feature is > not heavily used. > > Iterating over every line in Python's code repository in the ``Lib/`` > directory using the regular expression ``^\s*def\s*\w+\s*\(`` to > detect function and method definitions there were 22,252 matches in > the trunk. > > Tacking on ``.*,\s*\(`` to find ``def`` statements that contained a > tuple parameter, only 41 matches were found. This means that for > ``def`` statements, only 0.18% of them seem to use a tuple parameter. > > > Why They Should (Supposedly) Stay > ================================= > > Practical Use > ------------- > > In certain instances tuple parameters can be useful. A common example > is code that expect a two-item tuple that reperesents a Cartesian > point. While true it is nice to be able to have the unpacking of the > x and y coordinates for you, the argument is that this small amount of > practical usefulness is heavily outweighed by other issues pertaining > to tuple parameters. And as shown in > `No Loss Of Abilities If Removed`_, their use is purely practical and > in no way provide a unique ability that cannot be handled in other > ways very easily. > > > Self-Documentation For Parameters > --------------------------------- > > It has been argued that tuple parameters provide a way of > self-documentation for parameters that are expected to be of a certain > sequence format. Using our Cartesian point example from > `Practical Use`_, seeing ``(x, y)`` as a parameter in a function makes > it obvious that a tuple of length two is expected as an argument for > that parameter. > > But Python provides several other ways to document what parameters are > for. Documentation strings are meant to provide enough information > needed to explain what arguments are expected. Tuple parameters might > tell you the expected length of a sequence argument, it does not tell > you what that data will be used for. One must also read the docstring > to know what other arguments are expected if not all parameters are > tuple parameters. > > Function annotations (which do not work with tuple parameters) can > also supply documentation. Because annotations can be of any form, > what was once a tuple parameter can be a single argument parameter > with an annotation of ``tuple``, ``tuple(2)``, ``Cartesian point``, > ``(x, y)``, etc. Annotations provide great flexibility for > documenting what an argument is expected to be for a parameter, > including being a sequence of a certain length. > > > Transition Plan > =============== > > To transition Python 2.x code to 3.x where tuple parameters are > removed, two steps are suggested. First, the proper warning is to be > emitted when Python's compiler comes across a tuple parameter in > Python 2.6. This will be treated like any other syntactic change that > is to occur in Python 3.0 compared to Python 2.6. > > Second, the 2to3 refactoring tool [#2to3]_ will gain a rule for > translating tuple parameters to being a single parameter this is > unpacked as the first statement in the function. The name of the new > parameter will be a mangling of tuple parameter's names by joining > them with underscores. The new parameter will then be unpacked into > the names originally used in the tuple parameter. This means that the > following function:: > > def fxn((a, (b, c))): > pass > > will be translated into:: > > def fxn(a_b_c): > (a, (b, c)) = a_b_c > pass > > > References > ========== > > .. [#2to3] 2to3 refactoring tool > (http://svn.python.org/view/sandbox/trunk/2to3/) > > .. [#ironpython] IronPython > (http://www.codeplex.com/Wiki/View.aspx?ProjectName=IronPython) > > .. [#MSIL] Microsoft Intermediate Language > (http://msdn.microsoft.com/library/en-us/cpguide/html/cpconmicrosoftintermediatelanguagemsil.asp?frame=true) > > .. [#pep-3102] PEP 3102 (Keyword-Only Arguments) > (http://www.python.org/dev/peps/pep-3102/) > > .. [#pep-3107] PEP 3107 (Function Annotations) > (http://www.python.org/dev/peps/pep-3107/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Sat Mar 3 02:22:01 2007 From: collinw at gmail.com (Collin Winter) Date: Fri, 2 Mar 2007 19:22:01 -0600 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: <43aa6ff70703021722j240c4e33x723e81f0c02ae7cc@mail.gmail.com> On 3/2/07, Guido van Rossum wrote: > Has anyone tried to create a 2to3 transformer for this? I think it > should be possible, although there are a few warts (like inserting the > new code after the docstring, and how to come up with names for the > anonymous tuple(s), and having to do it for potentially any number of > arguments. Still, it would require no dataflow analysis, and the > except transformer already has a similar statement insertion. So I > think it's doable. I can take a crack at this. I had to do similar things for the raise fixer before switching it over to use with_traceback(). Collin Winter From guido at python.org Sat Mar 3 02:28:40 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 2 Mar 2007 17:28:40 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: <43aa6ff70703021722j240c4e33x723e81f0c02ae7cc@mail.gmail.com> References: <43aa6ff70703021722j240c4e33x723e81f0c02ae7cc@mail.gmail.com> Message-ID: On 3/2/07, Collin Winter wrote: > On 3/2/07, Guido van Rossum wrote: > > Has anyone tried to create a 2to3 transformer for this? I think it > > should be possible, although there are a few warts (like inserting the > > new code after the docstring, and how to come up with names for the > > anonymous tuple(s), and having to do it for potentially any number of > > arguments. Still, it would require no dataflow analysis, and the > > except transformer already has a similar statement insertion. So I > > think it's doable. > > I can take a crack at this. I had to do similar things for the raise > fixer before switching it over to use with_traceback(). That would be great! You may end up adding some generally useful code for inserting statements into a block... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sat Mar 3 02:31:38 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 03 Mar 2007 11:31:38 +1000 Subject: [Python-3000] How far to go with cleaning up exceptions In-Reply-To: References: <43aa6ff70703021639p84beae1j883d44385d8046dc@mail.gmail.com> Message-ID: <45E8CFFA.7000605@gmail.com> Brett Cannon wrote: > I think the slice removal is easy. But getting rid of 'args' and > making BaseException's constructor only take a single argument is not > painless. But a warning should help get this done in a reasonable > fashion. The exception slicing is the only thing I ever thought was particularly odd - the rest of the current exception API has never really bothered me. All I would suggest doing is to move the BaseException constructor closer to PEP 352 by always setting the message attribute to str(args[0]) rather than only doing it when the constructor is called with a single argument. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From brett at python.org Sat Mar 3 03:22:08 2007 From: brett at python.org (Brett Cannon) Date: Fri, 2 Mar 2007 18:22:08 -0800 Subject: [Python-3000] How far to go with cleaning up exceptions In-Reply-To: <45E8CFFA.7000605@gmail.com> References: <43aa6ff70703021639p84beae1j883d44385d8046dc@mail.gmail.com> <45E8CFFA.7000605@gmail.com> Message-ID: On 3/2/07, Nick Coghlan wrote: > Brett Cannon wrote: > > I think the slice removal is easy. But getting rid of 'args' and > > making BaseException's constructor only take a single argument is not > > painless. But a warning should help get this done in a reasonable > > fashion. > > The exception slicing is the only thing I ever thought was particularly > odd - the rest of the current exception API has never really bothered me. > > All I would suggest doing is to move the BaseException constructor > closer to PEP 352 by always setting the message attribute to > str(args[0]) rather than only doing it when the constructor is called > with a single argument. > If 'args' is not removed that will definitely happen as that came out to be an odd special-casing. -Brett From collinw at gmail.com Sat Mar 3 05:25:07 2007 From: collinw at gmail.com (Collin Winter) Date: Fri, 2 Mar 2007 22:25:07 -0600 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: <43aa6ff70703021722j240c4e33x723e81f0c02ae7cc@mail.gmail.com> Message-ID: <43aa6ff70703022025y4072af3fuaba917f0cf18da27@mail.gmail.com> On 3/2/07, Guido van Rossum wrote: > On 3/2/07, Collin Winter wrote: > > On 3/2/07, Guido van Rossum wrote: > > > Has anyone tried to create a 2to3 transformer for this? I think it > > > should be possible, although there are a few warts (like inserting the > > > new code after the docstring, and how to come up with names for the > > > anonymous tuple(s), and having to do it for potentially any number of > > > arguments. Still, it would require no dataflow analysis, and the > > > except transformer already has a similar statement insertion. So I > > > think it's doable. > > > > I can take a crack at this. I had to do similar things for the raise > > fixer before switching it over to use with_traceback(). > > That would be great! You may end up adding some generally useful code > for inserting statements into a block... I just checked in fixes/fix_tuple_params.py (and associated tests). I still want to work on the code that generates the new names (so that they'll be more appropriate), but the functionality is there. I definitely want to sit down and abstract out the "insert these stmts at the beginning of a suite" logic. fix_tuple_params includes support for "def foo(): x = 5; y = 7"-style functions, which leads to some less-than-attractive code. Collin Winter From guido at python.org Sat Mar 3 06:00:43 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 2 Mar 2007 21:00:43 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: <43aa6ff70703022025y4072af3fuaba917f0cf18da27@mail.gmail.com> References: <43aa6ff70703021722j240c4e33x723e81f0c02ae7cc@mail.gmail.com> <43aa6ff70703022025y4072af3fuaba917f0cf18da27@mail.gmail.com> Message-ID: Very cool. This should make the PEP a complete success! On 3/2/07, Collin Winter wrote: > On 3/2/07, Guido van Rossum wrote: > > On 3/2/07, Collin Winter wrote: > > > On 3/2/07, Guido van Rossum wrote: > > > > Has anyone tried to create a 2to3 transformer for this? I think it > > > > should be possible, although there are a few warts (like inserting the > > > > new code after the docstring, and how to come up with names for the > > > > anonymous tuple(s), and having to do it for potentially any number of > > > > arguments. Still, it would require no dataflow analysis, and the > > > > except transformer already has a similar statement insertion. So I > > > > think it's doable. > > > > > > I can take a crack at this. I had to do similar things for the raise > > > fixer before switching it over to use with_traceback(). > > > > That would be great! You may end up adding some generally useful code > > for inserting statements into a block... > > I just checked in fixes/fix_tuple_params.py (and associated tests). I > still want to work on the code that generates the new names (so that > they'll be more appropriate), but the functionality is there. > > I definitely want to sit down and abstract out the "insert these stmts > at the beginning of a suite" logic. fix_tuple_params includes support > for "def foo(): x = 5; y = 7"-style functions, which leads to some > less-than-attractive code. > > Collin Winter > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gareth.mccaughan at pobox.com Sat Mar 3 17:02:52 2007 From: gareth.mccaughan at pobox.com (Gareth McCaughan) Date: Sat, 3 Mar 2007 16:02:52 +0000 Subject: [Python-3000] Thoughts on new I/O library and bytecode In-Reply-To: <45E37DD5.2050809@canterbury.ac.nz> References: <9e804ac0702251429i36623d1bg7ea69d0ad7945429@mail.gmail.com> <20070225194742.AE4B.JCARLSON@uci.edu> <45E37DD5.2050809@canterbury.ac.nz> Message-ID: <200703031602.53038.gareth.mccaughan@pobox.com> On Tuesday 27 February 2007 00:39, Greg Ewing wrote: > I can't help feeling the people arguing for b"..." as the > repr format haven't really accepted the fact that text and > binary data will be distinct things in py3k, and are thinking > of bytes as being a replacement for the old string type. But > that's not true -- most of the time, *unicode* will be the > replacement for str when it is used to represent characters, > and bytes will mostly be used only for non-text. [etc.] ... but Guido prefers to use b"..." as the repr format, on the grounds that byte-sequences quite often are lightly encoded text, and that when that's true it can be *much* better to report them as such. Here's an ugly, impure, but possibly practical answer: give each bytes object a single-bit flag meaning something like "mostly textual"; make the bytes([1,2,3,4]) constructor set it to false, the b"abcde" constructor set it to true, and arbitrary operations on bytes objects do ... well, something plausible :-). (Textuality/non-textuality is generally preserved; combining texual and non-textual yields non-textual.) Then repr() can look at that flag and decide what to do on the basis of it. This would mean that x==y ==> repr(x)==repr(y) would fail; it can already fail when x,y are of different types (3==3.0; 1==True) and perhaps in some weird situations where they are of the same type (signed IEEE zeros). It would make the behaviour of repr() less predictable, and that's probably bad; it would mean (unlike the examples I gave above) that you can have x==y, with x and y of different types, but have repr(x) and repr(y) not look at all similar. Obviously the flag wouldn't affect comparisons or hashing. I can't say I like this much -- it's exactly the sort of behaviour I've found painful in Perl, with too much magic happening behind the scenes for perhaps-insufficient reason -- but it still might be the best available compromise. (The other obvious compromise approach would be to sniff the contents of the bytes object and see whether it "looks" like a lightly-encoded string. That's a bit too much magic for fuzzy reasons too.) -- Gareth McCaughan From brett at python.org Sat Mar 3 23:09:35 2007 From: brett at python.org (Brett Cannon) Date: Sat, 3 Mar 2007 14:09:35 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: <43aa6ff70703021722j240c4e33x723e81f0c02ae7cc@mail.gmail.com> <43aa6ff70703022025y4072af3fuaba917f0cf18da27@mail.gmail.com> Message-ID: On 3/2/07, Guido van Rossum wrote: > Very cool. This should make the PEP a complete success! > Thanks to Collin for doing this! Makes my life a heck of a lot easier. -Brett > On 3/2/07, Collin Winter wrote: > > On 3/2/07, Guido van Rossum wrote: > > > On 3/2/07, Collin Winter wrote: > > > > On 3/2/07, Guido van Rossum wrote: > > > > > Has anyone tried to create a 2to3 transformer for this? I think it > > > > > should be possible, although there are a few warts (like inserting the > > > > > new code after the docstring, and how to come up with names for the > > > > > anonymous tuple(s), and having to do it for potentially any number of > > > > > arguments. Still, it would require no dataflow analysis, and the > > > > > except transformer already has a similar statement insertion. So I > > > > > think it's doable. > > > > > > > > I can take a crack at this. I had to do similar things for the raise > > > > fixer before switching it over to use with_traceback(). > > > > > > That would be great! You may end up adding some generally useful code > > > for inserting statements into a block... > > > > I just checked in fixes/fix_tuple_params.py (and associated tests). I > > still want to work on the code that generates the new names (so that > > they'll be more appropriate), but the functionality is there. > > > > I definitely want to sit down and abstract out the "insert these stmts > > at the beginning of a suite" logic. fix_tuple_params includes support > > for "def foo(): x = 5; y = 7"-style functions, which leads to some > > less-than-attractive code. > > > > Collin Winter > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From gareth.mccaughan at pobox.com Sat Mar 3 23:58:53 2007 From: gareth.mccaughan at pobox.com (Gareth McCaughan) Date: Sat, 3 Mar 2007 22:58:53 +0000 Subject: [Python-3000] Thoughts on new I/O library and bytecode In-Reply-To: <200703031602.53038.gareth.mccaughan@pobox.com> References: <9e804ac0702251429i36623d1bg7ea69d0ad7945429@mail.gmail.com> <45E37DD5.2050809@canterbury.ac.nz> <200703031602.53038.gareth.mccaughan@pobox.com> Message-ID: <200703032258.53830.gareth.mccaughan@pobox.com> On Saturday 03 March 2007 16:02, I wrote: > Here's an ugly, impure, but possibly practical answer: > give each bytes object a single-bit flag meaning something > like "mostly textual"; ... > Obviously the flag wouldn't affect comparisons or hashing. Josiah Carlson mailed me to point out that, duh, obviously the flag wouldn't affect hashing since bytes objects are mutable and therefore unhashable. :-) -- g From bob at redivi.com Sun Mar 4 00:52:39 2007 From: bob at redivi.com (Bob Ippolito) Date: Sat, 3 Mar 2007 15:52:39 -0800 Subject: [Python-3000] Thoughts on new I/O library and bytecode In-Reply-To: <200703031602.53038.gareth.mccaughan@pobox.com> References: <9e804ac0702251429i36623d1bg7ea69d0ad7945429@mail.gmail.com> <20070225194742.AE4B.JCARLSON@uci.edu> <45E37DD5.2050809@canterbury.ac.nz> <200703031602.53038.gareth.mccaughan@pobox.com> Message-ID: <6a36e7290703031552i7a7da308o996d248f44e147c6@mail.gmail.com> On 3/3/07, Gareth McCaughan wrote: > On Tuesday 27 February 2007 00:39, Greg Ewing wrote: > > > I can't help feeling the people arguing for b"..." as the > > repr format haven't really accepted the fact that text and > > binary data will be distinct things in py3k, and are thinking > > of bytes as being a replacement for the old string type. But > > that's not true -- most of the time, *unicode* will be the > > replacement for str when it is used to represent characters, > > and bytes will mostly be used only for non-text. > [etc.] > > ... but Guido prefers to use b"..." as the repr format, > on the grounds that byte-sequences quite often are > lightly encoded text, and that when that's true it > can be *much* better to report them as such. I agree with Guido here. As a person that's written a lot of protocol implementations and parser/generators for a few strange binary formats... the literal syntax that lets me use ASCII is what I would prefer. I would have to say that most protocols these days are lightly encoded text anyway, so it's most beneficial to optimize for the ASCII case. > Here's an ugly, impure, but possibly practical answer: > give each bytes object a single-bit flag meaning something > like "mostly textual"; make the bytes([1,2,3,4]) constructor > set it to false, the b"abcde" constructor set it to true, > and arbitrary operations on bytes objects do ... well, > something plausible :-). (Textuality/non-textuality is > generally preserved; combining texual and non-textual > yields non-textual.) Then repr() can look at that flag > and decide what to do on the basis of it. That sounds like a generally bad idea... Even if a protocol is "mostly binary" a dump of single byte decimal integers is likely to be *less* useful than b"\x01\x02\x03\x04". Almost all protocols deal in integers larger than one byte, so a sequence of decimal bytes is really the worst thing to see in those cases. Erlang is in general really good about dealing with bytes (their binary type) but the printed representation is suboptimal because it behaves kinda like that. 1> Chunk = <<11:16/big, "foo bar baz">>. <<0,11,102,111,111,32,98,97,114,32,98,97,122>> 2> <> = Chunk. <<0,11,102,111,111,32,98,97,114,32,98,97,122>> 3> <> = Rest. <<"foo bar baz">> 4> {Length, String, Extra}. {11,<<"foo bar baz">>,<<"">>} When Erlang is printing the "repr" of a list or binary term to the shell it first checks to see if every item is printable ASCII integer. If so, then it prints as an ASCII string. Otherwise, it prints as a list of decimal integers. It doesn't work out well in these kinds of situations. If it was printed out as ASCII with hex escapes then it would make a lot more sense at a glance. -bob From daniel at stutzbachenterprises.com Sun Mar 4 01:59:17 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 3 Mar 2007 18:59:17 -0600 Subject: [Python-3000] Thoughts on new I/O library and bytecode In-Reply-To: <6a36e7290703031552i7a7da308o996d248f44e147c6@mail.gmail.com> References: <9e804ac0702251429i36623d1bg7ea69d0ad7945429@mail.gmail.com> <20070225194742.AE4B.JCARLSON@uci.edu> <45E37DD5.2050809@canterbury.ac.nz> <200703031602.53038.gareth.mccaughan@pobox.com> <6a36e7290703031552i7a7da308o996d248f44e147c6@mail.gmail.com> Message-ID: On 3/3/07, Bob Ippolito wrote: > When Erlang is printing the "repr" of a list or binary term to the > shell it first checks to see if every item is printable ASCII integer. > If so, then it prints as an ASCII string. Otherwise, it prints as a > list of decimal integers. It doesn't work out well in these kinds of > situations. If it was printed out as ASCII with hex escapes then it > would make a lot more sense at a glance. Perhaps it would be best to make one format the default, but provide a convenience method on the bytes type for the other format? repr(b) -> bytes("spam spam spam")' b.hex() -> "7370616d 20737061 6d207370 616d" -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From bob at redivi.com Sun Mar 4 02:06:39 2007 From: bob at redivi.com (Bob Ippolito) Date: Sat, 3 Mar 2007 17:06:39 -0800 Subject: [Python-3000] Thoughts on new I/O library and bytecode In-Reply-To: References: <9e804ac0702251429i36623d1bg7ea69d0ad7945429@mail.gmail.com> <20070225194742.AE4B.JCARLSON@uci.edu> <45E37DD5.2050809@canterbury.ac.nz> <200703031602.53038.gareth.mccaughan@pobox.com> <6a36e7290703031552i7a7da308o996d248f44e147c6@mail.gmail.com> Message-ID: <6a36e7290703031706i1878562oa7552c9257ea102a@mail.gmail.com> On 3/3/07, Daniel Stutzbach wrote: > On 3/3/07, Bob Ippolito wrote: > > When Erlang is printing the "repr" of a list or binary term to the > > shell it first checks to see if every item is printable ASCII integer. > > If so, then it prints as an ASCII string. Otherwise, it prints as a > > list of decimal integers. It doesn't work out well in these kinds of > > situations. If it was printed out as ASCII with hex escapes then it > > would make a lot more sense at a glance. > > Perhaps it would be best to make one format the default, but provide a > convenience method on the bytes type for the other format? > > repr(b) -> bytes("spam spam spam")' > b.hex() -> "7370616d 20737061 6d207370 616d" That's exactly what Guido said and what I was agreeing with. With use cases. -bob From jimjjewett at gmail.com Sun Mar 4 02:57:39 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sat, 3 Mar 2007 20:57:39 -0500 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: I have mixed feelings; I won't go so far as to say I oppose removing tuple-arguments, but some of the problems do have other solutions. On 3/2/07, Brett Cannon wrote: > But there is great difficulty when it comes to tuple parameters. The > existence of a tuple parameter is denoted by it name being made of a > ``.`` and a number in the ``co_varnames`` attribute of the function's > code object. I think this is just an implementation detail. For example, co_varnames could itself hold tuples as well as names. (That would itself break backwards compatibility, but only for introspection.) > Consider PEP 3102 (keyword-only arguments) and PEP 3107 (function > annotations) [#pep-3102]_ [#pep-3107]_. Both PEPs have been accepted and > introduce new functionality within a function's signature. And yet > for both PEPs the new feature cannot be applied to tuple parameters. I hadn't realized that they couldn't be annotated. That could be fixed. On the other hand, the existence of annotations does take away part of the value -- now you could just annotate a_b_c as having internal structure: (a, (b, c)) I'm not sure what keyword-only could even mean for strictly positional arguments. > The existence of tuple parameters also places sequence objects > separately from mapping objects in a function signature. There is no > way to pass in a mapping object (e.g., a dict) as a parameter and have > it unpack in the same fashion as a sequence does into a tuple > parameter. Either **kwargs or f(**mapping) is pretty close, depending on your perspective. The main difference is that sequence users have to supply all the names (keys) in advance, so you can't default only part of a tuple. (This oddness does strengthen the "but it really represents a unitary something" aspect.) > Iterating over every line in Python's code repository in the ``Lib/`` > directory using the regular expression ``^\s*def\s*\w+\s*\(`` to > detect function and method definitions there were 22,252 matches in > the trunk. > Tacking on ``.*,\s*\(`` to find ``def`` statements that contained a > tuple parameter, only 41 matches were found. This means that for > ``def`` statements, only 0.18% of them seem to use a tuple parameter. That misses first-argument tuples, such as cgitb.text and pydoc.fixup. Searching '^\\s*def\\s*\\w+\\s*\\(\\s*\\(' in C:\Python25\Lib\*.py ... I got 19 first-argument hits, but ten were in test, and another 4 in site-packages. I think it would also miss multiline defs whose tuple argument was not on the first line. That said, the frequency is almost certainly still less than 0.5%. -jJ From ncoghlan at gmail.com Sun Mar 4 03:24:22 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 04 Mar 2007 12:24:22 +1000 Subject: [Python-3000] Thoughts on new I/O library and bytecode In-Reply-To: References: <9e804ac0702251429i36623d1bg7ea69d0ad7945429@mail.gmail.com> <20070225194742.AE4B.JCARLSON@uci.edu> <45E37DD5.2050809@canterbury.ac.nz> <200703031602.53038.gareth.mccaughan@pobox.com> <6a36e7290703031552i7a7da308o996d248f44e147c6@mail.gmail.com> Message-ID: <45EA2DD6.7020600@gmail.com> Daniel Stutzbach wrote: > On 3/3/07, Bob Ippolito wrote: >> When Erlang is printing the "repr" of a list or binary term to the >> shell it first checks to see if every item is printable ASCII integer. >> If so, then it prints as an ASCII string. Otherwise, it prints as a >> list of decimal integers. It doesn't work out well in these kinds of >> situations. If it was printed out as ASCII with hex escapes then it >> would make a lot more sense at a glance. > > Perhaps it would be best to make one format the default, but provide a > convenience method on the bytes type for the other format? > > repr(b) -> bytes("spam spam spam")' > b.hex() -> "7370616d 20737061 6d207370 616d" The hex() method isn't implemented yet, but a really simple listcomp/gencomp already gives the 'list of integers' format: >>> data = b"what's in a name?" >>> repr(data) "b'what\\'s in a name?'" >>> [x for x in data] [119, 104, 97, 116, 39, 115, 32, 105, 110, 32, 97, 32, 110, 97, 109, 101, 63] Given the simplicity of retrieving the underlying integers, I think the string representation makes a good default repr implementation. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From brett at python.org Sun Mar 4 05:43:43 2007 From: brett at python.org (Brett Cannon) Date: Sat, 3 Mar 2007 20:43:43 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/3/07, Jim Jewett wrote: > I have mixed feelings; I won't go so far as to say I oppose removing > tuple-arguments, but some of the problems do have other solutions. > Sure some of them have other solutions, but that does not mean that they should be fixed just to save this feature. > On 3/2/07, Brett Cannon wrote: > > > But there is great difficulty when it comes to tuple parameters. The > > existence of a tuple parameter is denoted by it name being made of a > > ``.`` and a number in the ``co_varnames`` attribute of the function's > > code object. > > I think this is just an implementation detail. For example, > co_varnames could itself hold tuples as well as names. (That would > itself break backwards compatibility, but only for introspection.) > Possibly, but no one has stepped forward to rectify the situation (I am obviously not going to since I have other issues with them). > > Consider PEP 3102 (keyword-only arguments) and PEP 3107 (function > > annotations) [#pep-3102]_ [#pep-3107]_. Both PEPs have been accepted and > > introduce new functionality within a function's signature. And yet > > for both PEPs the new feature cannot be applied to tuple parameters. > > I hadn't realized that they couldn't be annotated. That could be fixed. > > On the other hand, the existence of annotations does take away part of > the value -- now you could just annotate a_b_c as having internal > structure: (a, (b, c)) > Exactly. > I'm not sure what keyword-only could even mean for strictly positional > arguments. > They don't make sense for them. Jiwon and I discussed this when he was designed keyword-only arguments. > > The existence of tuple parameters also places sequence objects > > separately from mapping objects in a function signature. There is no > > way to pass in a mapping object (e.g., a dict) as a parameter and have > > it unpack in the same fashion as a sequence does into a tuple > > parameter. > > Either **kwargs or f(**mapping) is pretty close, depending on your perspective. > In terms of unpacking, there is no equivalent. You can't pass in a dictionary as an argument and then signify that the dict must have certain keys whose values are bound to a specific parameter. > The main difference is that sequence users have to supply all the > names (keys) in advance, so you can't default only part of a tuple. > (This oddness does strengthen the "but it really represents a unitary > something" aspect.) > > > Iterating over every line in Python's code repository in the ``Lib/`` > > directory using the regular expression ``^\s*def\s*\w+\s*\(`` to > > detect function and method definitions there were 22,252 matches in > > the trunk. > > > Tacking on ``.*,\s*\(`` to find ``def`` statements that contained a > > tuple parameter, only 41 matches were found. This means that for > > ``def`` statements, only 0.18% of them seem to use a tuple parameter. > > That misses first-argument tuples, such as cgitb.text and pydoc.fixup. > Searching '^\\s*def\\s*\\w+\\s*\\(\\s*\\(' in C:\Python25\Lib\*.py ... > > I got 19 first-argument hits, but ten were in test, and another 4 in > site-packages. > > I think it would also miss multiline defs whose tuple argument was not > on the first line. > It's rought, I know. > That said, the frequency is almost certainly still less than 0.5%. Yep, still paltry. -Brett From guido at python.org Sun Mar 4 20:14:11 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 4 Mar 2007 11:14:11 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/3/07, Brett Cannon wrote: > On 3/3/07, Jim Jewett wrote: > > I have mixed feelings; I won't go so far as to say I oppose removing > > tuple-arguments, but some of the problems do have other solutions. > > Sure some of them have other solutions, but that does not mean that > they should be fixed just to save this feature. I see lukewarm support for keeping these at most, and probably lukewarm support for removing them at well. That means I get to decide and nobody will care much (for once :-). So my decision is to get rid of them. > > > Consider PEP 3102 (keyword-only arguments) and PEP 3107 (function > > > annotations) [#pep-3102]_ [#pep-3107]_. Both PEPs have been accepted and > > > introduce new functionality within a function's signature. And yet > > > for both PEPs the new feature cannot be applied to tuple parameters. > > > > I hadn't realized that they couldn't be annotated. That could be fixed. Actually they can be annotated. But that's no reason to keep them either. :-) Though the PEP might be fixed. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Sun Mar 4 20:23:40 2007 From: brett at python.org (Brett Cannon) Date: Sun, 4 Mar 2007 11:23:40 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/4/07, Guido van Rossum wrote: > On 3/3/07, Brett Cannon wrote: > > On 3/3/07, Jim Jewett wrote: > > > I have mixed feelings; I won't go so far as to say I oppose removing > > > tuple-arguments, but some of the problems do have other solutions. > > > > Sure some of them have other solutions, but that does not mean that > > they should be fixed just to save this feature. > > I see lukewarm support for keeping these at most, and probably > lukewarm support for removing them at well. That means I get to decide > and nobody will care much (for once :-). So my decision is to get rid > of them. > Woohoo! Can I go ahead and mark the PEP as accepted then? > > > > Consider PEP 3102 (keyword-only arguments) and PEP 3107 (function > > > > annotations) [#pep-3102]_ [#pep-3107]_. Both PEPs have been accepted and > > > > introduce new functionality within a function's signature. And yet > > > > for both PEPs the new feature cannot be applied to tuple parameters. > > > > > > I hadn't realized that they couldn't be annotated. That could be fixed. > > Actually they can be annotated. But that's no reason to keep them either. :-) > I actually meant they can't be annotated like ``def fxn((a, b):int): pass``. I think what Guido is thinking of is ``def fxn((a:int, b:int)): pass`` (although that causes an assertion error: Python/compile.c:2430: failed assertion `scope || PyString_AS_STRING(name)[0] == '_''). > Though the PEP might be fixed. I will do that right now. -Brett From guido at python.org Sun Mar 4 20:35:19 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 4 Mar 2007 11:35:19 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/4/07, Brett Cannon wrote: > On 3/4/07, Guido van Rossum wrote: > > I see lukewarm support for keeping these at most, and probably > > lukewarm support for removing them at well. That means I get to decide > > and nobody will care much (for once :-). So my decision is to get rid > > of them. > > Woohoo! Can I go ahead and mark the PEP as accepted then? Please do! > > Actually they can be annotated. But that's no reason to keep them either. :-) > > I actually meant they can't be annotated like ``def fxn((a, b):int): > pass``. I think what Guido is thinking of is ``def fxn((a:int, > b:int)): pass`` (although that causes an assertion error: > Python/compile.c:2430: failed assertion `scope || > PyString_AS_STRING(name)[0] == '_''). Hm, I've never seen that assert. How to provoke it? Anyway, it will be ripped out when the tuple parameters are ripped out. You might even get the prize for ripping out the most code! :-) > > Though the PEP might be fixed. > > I will do that right now. Great! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Sun Mar 4 21:11:27 2007 From: brett at python.org (Brett Cannon) Date: Sun, 4 Mar 2007 12:11:27 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/4/07, Guido van Rossum wrote: > On 3/4/07, Brett Cannon wrote: > > On 3/4/07, Guido van Rossum wrote: > > > I see lukewarm support for keeping these at most, and probably > > > lukewarm support for removing them at well. That means I get to decide > > > and nobody will care much (for once :-). So my decision is to get rid > > > of them. > > > > Woohoo! Can I go ahead and mark the PEP as accepted then? > > Please do! > Done! I also did some other cleanup in PEP 0: marked PEP 3102 (keyword-only arguments), PEP 3104 (nonlocal), and 3107 (annotations) as finished. Made PEP 3106 (dict views) as accepted but left as not implemented as there is still an Open Issues section. I reclassified PEP 3108 (stdlib cleanup) as Standards Track instead of Informational as it will need a pronouncement some day and Informational PEPs basically don't. We really need to make PEP 0 be auto-generated so we stop having to edit the darn thing and having it outdated. > > > Actually they can be annotated. But that's no reason to keep them either. :-) > > > > I actually meant they can't be annotated like ``def fxn((a, b):int): > > pass``. I think what Guido is thinking of is ``def fxn((a:int, > > b:int)): pass`` (although that causes an assertion error: > > Python/compile.c:2430: failed assertion `scope || > > PyString_AS_STRING(name)[0] == '_''). > > Hm, I've never seen that assert. How to provoke it? I literally pasted in that example function and that triggered it. > Anyway, it will be > ripped out when the tuple parameters are ripped out. You might even > get the prize for ripping out the most code! :-) > =) Neal better watch out. -Brett From python at zesty.ca Sun Mar 4 20:37:48 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Sun, 4 Mar 2007 13:37:48 -0600 (CST) Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On Fri, 2 Mar 2007, Brett Cannon wrote: > the removal of the automatic unpacking of sequence > arguments when a tuple parameter is present in a function's signature > (e.g., the ``(b, c)`` in ``def fxn(a, (b, c), d): pass``). As I think most people know by now, it makes me quite sad to see these going away. This is one of those really nice flexibilities in Python that makes Python Python (kind of like for/else). Brett's arguments for making them go (in my opinion) say little about the feature itself; most are merely observations that other parts of the language implementation fail to fully support this feature. For example, the "Introspection Issues" argument is mostly irrelevant: we are discussing a new incompatible version of the Python language, so it doesn't make sense to argue based on shortcomings of previous versions. The weakness is in the introspection capabilities. (It's difficult to introspect inside local scopes -- of course that does not mean we should remove local scopes from the language). Similarly, the complaints about poor error messages are irrelevant. If the error messages are inadequate, simply fix them. "Exception to the Rule" is also about other features that don't support tuple parameters, not about tuple parameters themselves. If these features are incomplete, they can be completed. "No Loss of Abilities If Removed" is not an argument to remove anything; it's a neutral claim that removing it won't hurt. All for/else loops could be easily rewritten with a break flag instead and no functionality would be lost -- but functionality is not the question; the expressiveness of the language suffers. In summary, all of the arguments for removing this feature are of the form "It won't hurt very much if we remove the feature"; the arguments for keeping the feature are of the form "This feature is useful and good for the language." Notice the asymmetry: there are no arguments that removing it will actually yield a better language, only arguments that the harm caused by removing it will be small. -- ?!ng From guido at python.org Sun Mar 4 22:00:14 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 4 Mar 2007 13:00:14 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/4/07, Ka-Ping Yee wrote: > On Fri, 2 Mar 2007, Brett Cannon wrote: > > the removal of the automatic unpacking of sequence > > arguments when a tuple parameter is present in a function's signature > > (e.g., the ``(b, c)`` in ``def fxn(a, (b, c), d): pass``). > > As I think most people know by now, it makes me quite sad to see these > going away. This is one of those really nice flexibilities in Python > that makes Python Python (kind of like for/else). > > Brett's arguments for making them go (in my opinion) say little about > the feature itself; most are merely observations that other parts of > the language implementation fail to fully support this feature. > > For example, the "Introspection Issues" argument is mostly irrelevant: > we are discussing a new incompatible version of the Python language, > so it doesn't make sense to argue based on shortcomings of previous > versions. The weakness is in the introspection capabilities. (It's > difficult to introspect inside local scopes -- of course that does not > mean we should remove local scopes from the language). > > Similarly, the complaints about poor error messages are irrelevant. > If the error messages are inadequate, simply fix them. > > "Exception to the Rule" is also about other features that don't > support tuple parameters, not about tuple parameters themselves. > If these features are incomplete, they can be completed. > > "No Loss of Abilities If Removed" is not an argument to remove > anything; it's a neutral claim that removing it won't hurt. All > for/else loops could be easily rewritten with a break flag instead > and no functionality would be lost -- but functionality is not the > question; the expressiveness of the language suffers. > > In summary, all of the arguments for removing this feature are of the > form "It won't hurt very much if we remove the feature"; the arguments > for keeping the feature are of the form "This feature is useful and > good for the language." Notice the asymmetry: there are no arguments > that removing it will actually yield a better language, only arguments > that the harm caused by removing it will be small. I don't see the feature useful for the language. It's a wart that startles most experienced users they first come across it. In order to make room for new features we need to make the language smaller. This cut is one of the least controversial cuts imaginable (your opposition notwithstanding). You'll get over your sadness. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Sun Mar 4 22:22:14 2007 From: python at rcn.com (Raymond Hettinger) Date: Sun, 4 Mar 2007 13:22:14 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) References: Message-ID: <001a01c75ea3$2ee099a0$6800a8c0@RaymondLaptop1> [Ka-Ping Yee] > In summary, all of the arguments for removing this feature are of the > form "It won't hurt very much if we remove the feature"; the arguments > for keeping the feature are of the form "This feature is useful and > good for the language." Notice the asymmetry: there are no arguments > that removing it will actually yield a better language, only arguments > that the harm caused by removing it will be small. Well said. I agree whole-heartedly; however, Brett did present another argument beyond "it won't hurt much". I think his root motivation was that the feature was difficult to implement for some reason or another. FWIW, I would like the feature to be kept. I've found it useful in that it documents the function signature more completely when dealing with arguments that are already pre-packed into tuples (such as records returned from SQL queries or CSV row readers): def contact_info_update(timestamp, sequence_number, (name, address, phone, email), backup=True): . . . contact_info_update(now(), seqnum, cursor.fetchone(), backup=False) I think it is important to make explicit that the function depends on a specific layout for the contact record tuple and have that dependency show-up in tooltips when I write the function call. Another example comes from a spherical navigation module with several tuple representations for movement (lat_offset, long_offset) versus (dist, direction). Raymond From daniel.stutzbach at gmail.com Sun Mar 4 22:34:44 2007 From: daniel.stutzbach at gmail.com (Daniel Stutzbach) Date: Sun, 4 Mar 2007 15:34:44 -0600 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: On 3/1/07, Adam Olsen wrote: > Why do non-blocking operations need to use the same methods when > they're clearly not the same semantics? Although long, > .nonblockflush() would be explicit and allow .flush() to still block. .nonblockflush() would be fine with me, but I don't think .flush() should block on a non-blocking object. To accomplish that, it would either have to be smart enough to switch the object into blocking mode, or internally use select(). How about .flush() write as much as it can, and throw an exception if not all of the bytes can be written to the device? (again, this would only come up when a user has set the underlying file descriptor to non-blocking mode) > I'm especially wary of infinite buffers. They allow a malicious peer > to consume all your memory, DoSing the process or even the whole box > if Linux's OOM killer doesn't kick in fast enough. For a write-buffer, you start eating up memory only if an application is buffer-ignorant and tries to dump a massive amount of data to the socket all at once. A naive HTTP server implementation might do this by calling something like s.write(open(filename)). This isn't a DoS by a peer though, it's a local implementation problem. For a read-buffer, you start eating up all of memory only if you call .read() with no arguments and the peer dumps a few gig on you. If you call read(n) to get as much data as you need, then the buffer class will only grab reasonably sized chunks from the network. Network applications don't normally call .read() without arguments, since they need to communicate both ways. If the object is an ordinary file, then DoS isn't so much of an issue and reading the whole files seems very reasonable. I suppose for an Text object wrapped around a socket, .readline() could be dangerous if a malicious peer sends a few gig all on one line. That's a problem for the encoding layer to sort out, not the buffering layer though. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From daniel.stutzbach at gmail.com Sun Mar 4 22:42:21 2007 From: daniel.stutzbach at gmail.com (Daniel Stutzbach) Date: Sun, 4 Mar 2007 15:42:21 -0600 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> Message-ID: On 3/1/07, Mike Verdone wrote: > When write is called check if the buffer is bigger than buffer_size. > If it is attempt to pre-flush the buffer. If we can't pre-flush, raise > BlockingIO error. I'd much rather see dynamically-sized buffers. If it works as you describe, then any application that wants to use reliable [1] non-blocking I/O is going to have keep its own buffer of data waiting to be written to the Buffered IO object. That's a pain and needlessly redundant. One could just set the buffer "large enough" for the applications current needs, but this can lead to strange failures later when the application needs to hold buffer_size+1 bytes in the buffer and suddenly dies. [1] = "reliable" in the sense of "no data loss unless the connection drops" -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From bob at redivi.com Sun Mar 4 22:50:15 2007 From: bob at redivi.com (Bob Ippolito) Date: Sun, 4 Mar 2007 13:50:15 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: <001a01c75ea3$2ee099a0$6800a8c0@RaymondLaptop1> References: <001a01c75ea3$2ee099a0$6800a8c0@RaymondLaptop1> Message-ID: <6a36e7290703041350l16e638eas6773b72c9d416004@mail.gmail.com> On 3/4/07, Raymond Hettinger wrote: > [Ka-Ping Yee] > > In summary, all of the arguments for removing this feature are of the > > form "It won't hurt very much if we remove the feature"; the arguments > > for keeping the feature are of the form "This feature is useful and > > good for the language." Notice the asymmetry: there are no arguments > > that removing it will actually yield a better language, only arguments > > that the harm caused by removing it will be small. > > Well said. I agree whole-heartedly; however, Brett did present another > argument beyond "it won't hurt much". I think his root motivation was > that the feature was difficult to implement for some reason or another. > > FWIW, I would like the feature to be kept. I've found it useful in that it > documents the function signature more completely when dealing > with arguments that are already pre-packed into tuples (such as records > returned from SQL queries or CSV row readers): > > def contact_info_update(timestamp, sequence_number, (name, address, phone, > email), backup=True): > . . . > > contact_info_update(now(), seqnum, cursor.fetchone(), backup=False) > > I think it is important to make explicit that the function depends on a specific > layout for > the contact record tuple and have that dependency show-up in tooltips when I > write > the function call. > > Another example comes from a spherical navigation module with several tuple > representations for movement (lat_offset, long_offset) versus (dist, > direction). This is a feature I use pretty often. I do a lot of data processing (e.g. CSV or SQL) and it comes up all of the time. It would be annoying to lose it. FWIW Erlang does tuple and record unpacking, but that programming model pretty much requires it. -bob From rhamph at gmail.com Sun Mar 4 23:42:17 2007 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 4 Mar 2007 15:42:17 -0700 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: On 3/4/07, Daniel Stutzbach wrote: > On 3/1/07, Adam Olsen wrote: > > Why do non-blocking operations need to use the same methods when > > they're clearly not the same semantics? Although long, > > .nonblockflush() would be explicit and allow .flush() to still block. > > .nonblockflush() would be fine with me, but I don't think .flush() > should block on a non-blocking object. To accomplish that, it would > either have to be smart enough to switch the object into blocking > mode, or internally use select(). Either would work, if we decide to support it. > How about .flush() write as much as it can, and throw an exception if > not all of the bytes can be written to the device? (again, this would > only come up when a user has set the underlying file descriptor to > non-blocking mode) I see little point in having an interface that randomly fails depending on the stars, phase of the moon, etc. If the application is using the interface wrong then we should fail every time. Errors should never pass silently. > > I'm especially wary of infinite buffers. They allow a malicious peer > > to consume all your memory, DoSing the process or even the whole box > > if Linux's OOM killer doesn't kick in fast enough. > > For a write-buffer, you start eating up memory only if an application > is buffer-ignorant and tries to dump a massive amount of data to the > socket all at once. A naive HTTP server implementation might do this > by calling something like s.write(open(filename)). This isn't a DoS > by a peer though, it's a local implementation problem. Any application expecting a blocking file and getting a non-blocking one is buffer-ignorant. How is this odd way of failing useful? > For a read-buffer, you start eating up all of memory only if you call > .read() with no arguments and the peer dumps a few gig on you. If you > call read(n) to get as much data as you need, then the buffer class > will only grab reasonably sized chunks from the network. Network > applications don't normally call .read() without arguments, since they > need to communicate both ways. If the object is an ordinary file, > then DoS isn't so much of an issue and reading the whole files seems > very reasonable. > > I suppose for an Text object wrapped around a socket, .readline() > could be dangerous if a malicious peer sends a few gig all on one > line. That's a problem for the encoding layer to sort out, not the > buffering layer though. A networked application should never read an unlimited amount of a socket, it should always used fixed-size blocks or fixed-size lines. The rare application that requires processing all of the contents at once should first write it to disk (which has a much larger capacity), then read back in only limited amount at a time. I can see three different behaviours when reading from a file or socket: * Blocking, read all. Returns exactly the amount specified, raising an exception is there is a short read. If n is None (the default) it reads the full contents of the file. * Blocking, chunked. Returns n bytes unless the end is hit, in which case it returns a short read. If n is None it uses a default size. * Non-blocking, chunked. Returns however many bytes it feels like, up to a maximum of n. If n is None it uses a default size. Empty reads indicate an error. Both blocking modes are very similar, differing only in their default (read all vs read whatever) and their handling of the end of the file. I'm not convinced combining them (as python has traditionally done) is optimal, but it's not a big deal. The non-blocking, chunked mode is very different however. It can return a short read at any point. Applications expecting blocking mode may get empty strings (or exceptions indicating such, perhaps the only saving grace.) Using .nonblockingread() is long, but I expect it to be wrapped by the event loop anyway. -- Adam Olsen, aka Rhamphoryncus From greg.ewing at canterbury.ac.nz Mon Mar 5 01:02:06 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 05 Mar 2007 13:02:06 +1300 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: <001a01c75ea3$2ee099a0$6800a8c0@RaymondLaptop1> References: <001a01c75ea3$2ee099a0$6800a8c0@RaymondLaptop1> Message-ID: <45EB5DFE.60702@canterbury.ac.nz> Raymond Hettinger wrote: > def contact_info_update(timestamp, sequence_number, (name, address, phone, > email), backup=True): Things like that are probably better done with something like the named-tuple idea that was discussed recently. Writing code that depends on long sequences having things in particular positions is an anti-pattern, IMO. It's convenient at first, but it rapidly becomes unmaintainable. I wrote a lot of code like that in one of my recent projects, and it's one of the first things I'll change if I get around to re-doing it. > Another example comes from a spherical navigation module with several tuple > representations for movement (lat_offset, long_offset) versus (dist, > direction). That could be addressed by giving the arguments descriptive names, such as lat_long and dist_dir. -- Greg From greg.ewing at canterbury.ac.nz Mon Mar 5 01:19:49 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 05 Mar 2007 13:19:49 +1300 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> Message-ID: <45EB6225.7060802@canterbury.ac.nz> I'm having trouble seeing what the use case is for the buffered non-blocking writes being discussed here. Doing asynchronous I/O usually *doesn't* involve putting the file descriptor into non-blocking mode. Instead you use select() or equivalent, and only try to read or write when the file is reported as being ready. For this to work properly, the select() needs to operate at the *bottom* of the I/O stack. Any buffering layers sit above that, with requests for data propagating up the stack as the file becomes ready. In other words, the whole thing has to have the control flow inverted and work in "pull" mode rather than "push" mode. It's hard to see how this could fit into the model as a minor variation on how writes are done. -- Greg From jimjjewett at gmail.com Mon Mar 5 02:50:37 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 4 Mar 2007 20:50:37 -0500 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: On 3/4/07, Adam Olsen wrote: > On 3/4/07, Daniel Stutzbach wrote: > > .nonblockflush() would be fine with me, but I don't think .flush() > > should block on a non-blocking object. ... I don't think flush should even be called on an object that is (intentionally) non-blocking. The fact that it was called suggests that the application doesn't realize/care about the non-blocking mode. (Or at least doesn't like it at this particular moment.) > > How about .flush() write as much as it can, and throw an exception if > > not all of the bytes can be written to the device? (again, this would > > only come up when a user has set the underlying file descriptor to > > non-blocking mode) > I see little point in having an interface that randomly fails > depending on the stars, phase of the moon, etc. If the application is > using the interface wrong then we should fail every time. (Based on the fflush information at http://opengroup.org/onlinepubs/007908799/xsh/fflush.html) Random failures are unavoidable; the application can't know how full the disk already was. (ENOSPC) The reason not to raise a would-block exception (EAGAIN) is that the application *can* recover by just waiting a while. The boilerplate to do that wait-and-recover should be handled once by python, instead of repeated in every paranoid application that worries it *might* be handed a device that *happens* to be non-blocking. -jJ From rhamph at gmail.com Mon Mar 5 03:54:06 2007 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 4 Mar 2007 19:54:06 -0700 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: <45EB6225.7060802@canterbury.ac.nz> References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/4/07, Greg Ewing wrote: > I'm having trouble seeing what the use case is for > the buffered non-blocking writes being discussed here. > > Doing asynchronous I/O usually *doesn't* involve > putting the file descriptor into non-blocking mode. > Instead you use select() or equivalent, and only > try to read or write when the file is reported as > being ready. I can't say which is more common, but non-blocking has a safer feel. Normal code would be select-driven in both, but if you screw up with non-blocking you get an error, whereas blocking you get a mysterious hang. accept() is the exception. It's possible for a connection to disappear between the time select() returns and the time you call accept(), so you need to be non-blocking to avoid hanging. > > For this to work properly, the select() needs to > operate at the *bottom* of the I/O stack. Any > buffering layers sit above that, with requests for > data propagating up the stack as the file becomes > ready. > > In other words, the whole thing has to have the > control flow inverted and work in "pull" mode > rather than "push" mode. It's hard to see how this > could fit into the model as a minor variation on > how writes are done. Meaning it needs to be a distinct interface and explicitly designed as such. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Mon Mar 5 04:12:07 2007 From: rhamph at gmail.com (Adam Olsen) Date: Sun, 4 Mar 2007 20:12:07 -0700 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: On 3/4/07, Jim Jewett wrote: > On 3/4/07, Adam Olsen wrote: > > On 3/4/07, Daniel Stutzbach wrote: > > > > .nonblockflush() would be fine with me, but I don't think .flush() > > > should block on a non-blocking object. ... > > I don't think flush should even be called on an object that is > (intentionally) non-blocking. The fact that it was called suggests > that the application doesn't realize/care about the non-blocking mode. > (Or at least doesn't like it at this particular moment.) > > > > How about .flush() write as much as it can, and throw an exception if > > > not all of the bytes can be written to the device? (again, this would > > > only come up when a user has set the underlying file descriptor to > > > non-blocking mode) > > > I see little point in having an interface that randomly fails > > depending on the stars, phase of the moon, etc. If the application is > > using the interface wrong then we should fail every time. > > (Based on the fflush information at > http://opengroup.org/onlinepubs/007908799/xsh/fflush.html) > > Random failures are unavoidable; the application can't know how full > the disk already was. (ENOSPC) I wouldn't call a full disk a random failure, at least not in the same sense. What I meant was during development, with no disk load by other processes, the program would work fine. When deployed a user may have other apps running and once a week there's just enough disk load that the flush can't complete immediately. Then the process exits, it's still non-blocking so it won't stop, so the file never gets flushed properly. > > The reason not to raise a would-block exception (EAGAIN) is that the > application *can* recover by just waiting a while. The boilerplate to > do that wait-and-recover should be handled once by python, instead of > repeated in every paranoid application that worries it *might* be > handed a device that *happens* to be non-blocking. Making .flush() internally use select() so that it still behaves in a blocking fashion to the application is perfectly acceptable to me. The alternative, in my mind at least, is that a TypeError of ValueError be raised because this interface is not designed to be used in non-blocking mode and it's bug in the applicatino to attempt it. The app should be using the right interface instead, .nonblockflush() or the like. One interface, one behaviour, no guessing. -- Adam Olsen, aka Rhamphoryncus From python at zesty.ca Mon Mar 5 09:42:27 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 02:42:27 -0600 (CST) Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: <6a36e7290703041350l16e638eas6773b72c9d416004@mail.gmail.com> References: <001a01c75ea3$2ee099a0$6800a8c0@RaymondLaptop1> <6a36e7290703041350l16e638eas6773b72c9d416004@mail.gmail.com> Message-ID: On 3/4/07, Raymond Hettinger wrote: > FWIW, I would like the feature to be kept. I've found it useful in that it > documents the function signature more completely when dealing > with arguments that are already pre-packed into tuples I just noticed that this has a more noticeable effect on lambda, since you don't have room for another statement to do the unpacking. To sort a dictionary by value, you can currently write sorted(dict.items(), key=lambda (key, value): value) Without tuple unpacking, this idiom becomes sorted(dict.items(), key=lambda item: item[1]) obscuring the fact that the item is a (key, value) pair. -- ?!ng From tanzer at swing.co.at Mon Mar 5 09:54:48 2007 From: tanzer at swing.co.at (Christian Tanzer) Date: Mon, 05 Mar 2007 09:54:48 +0100 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: Your message of "Mon, 05 Mar 2007 02:42:27 CST." Message-ID: Ka-Ping Yee wrote > On 3/4/07, Raymond Hettinger wrote: > > FWIW, I would like the feature to be kept. I've found it useful in that it > > documents the function signature more completely when dealing > > with arguments that are already pre-packed into tuples > > I just noticed that this has a more noticeable effect on lambda, since > you don't have room for another statement to do the unpacking. > > To sort a dictionary by value, you can currently write > > sorted(dict.items(), key=lambda (key, value): value) > > Without tuple unpacking, this idiom becomes > > sorted(dict.items(), key=lambda item: item[1]) > > obscuring the fact that the item is a (key, value) pair. Oooops. I'd pretty much ignored the discussion because I thought that I didn't use tuple parameter unpacking anyway. Unfortunately, I use it quite a bit with lambda. I shudder at the thought how those are going to look without tuple unpacking. FWIW, I always liked the `parameter passing is assignment` semantics of Python. I sure hope nobody is going to start a crus^H^H^H^HPEP to remove tuple unpacking in general from the language! -- Christian Tanzer http://www.c-tanzer.at/ From ncoghlan at gmail.com Mon Mar 5 11:17:36 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 05 Mar 2007 20:17:36 +1000 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: <45EBEE40.4070601@gmail.com> Ka-Ping Yee wrote: > In summary, all of the arguments for removing this feature are of the > form "It won't hurt very much if we remove the feature"; the arguments > for keeping the feature are of the form "This feature is useful and > good for the language." Notice the asymmetry: there are no arguments > that removing it will actually yield a better language, only arguments > that the harm caused by removing it will be small. The way I interpret the argument is more along the lines of "this feature has been in the language for years, but look at all these other aspects of the language and standard library that still don't support it correctly". Now we have more features we want to add which would *also* require additional work in order to make tuple parameters work correctly. I see the PEP as asking the question as to whether or not we really want to keep supporting a feature which makes it harder to do other things we want to do, when nobody has cared enough to even make that feature fully supported in the Python 2.x series. As I recall, tuple parameter unpacking was also a cause of a number of tricky bugs during the transition to the AST-based compiler. A little used feature that imposes a significant maintenance burden strikes me as well worth getting rid of - maintaining the status quo appears to be far from free once development effort is taken into account. Further, by stripping the current version of this ability out, it opens the door to the introduction of a far less painful pre-AST form of syntactic sugar (similar to the way try-except-finally unification was handled). Such a re-introduction of automatic parameter unpacking might also be able to be designed such that the tuple itself is still assigned a name (which would alleviate many of the problems that make the current version of tuple parameter unpacking such a pain to deal with on the implementation side). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From python-dev at zesty.ca Mon Mar 5 11:24:11 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 04:24:11 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in Message-ID: At Guido's prompting, I drafted the following PEP. How does it sound? -- ?!ng Title: Renaming iterator.next() to iterator.__next__() ====================================================== Abstract ======== The iterator protocol in Python 2.x consists of two methods: ``__iter__()`` called on an iterable object to yield an iterator, and ``next()`` called on an iterator object to yield the next item in the sequence. Using a ``for`` loop to iterate over an iterable object implicitly calls both of these methods. This PEP proposes that the ``next`` method be renamed to ``__next__``, consistent with all the other protocols in Python in which a method is implicitly called as part of a language-level protocol, and that a built-in function named ``next`` be introduced to invoke ``__next__`` method, consistent with the manner in which other protocols are explicitly invoked. Names With Double Underscores ============================= In Python, double underscores before and after a name are used to distinguish names that belong to the language itself. Attributes and methods that are implicitly used or created by the interpreter employ this naming convention; some examples are: * ``__file__`` - an attribute automatically created by the interpreter * ``__dict__`` - an attribute with special meaning to the interpreter * ``__init__`` - a method implicitly called by the interpreter Note that this convention applies to methods such as ``__init__`` that are explicitly defined by the programmer, as well as attributes such as ``__file__`` that can only be accessed by naming them explicitly, so it includes names that are used *or* created by the interpreter. (Not all things that are called "protocols" are made of methods with double-underscore names. For example, the ``__contains__`` method has double underscores because the language construct ``x in y`` implicitly calls ``__contains__``. But even though the ``read`` method is part of the file protocol, it does not have double underscores because there is no language construct that implicitly invokes ``x.read()``.) The use of double underscores creates a separate namespace for names that are part of the Python language definition, so that programmers are free to create variables, attributes, and methods that start with letters, without fear of silently colliding with names that have a language-defined purpose. (Colliding with reserved keywords is still a concern, but at least this will immediately yield a syntax error.) The naming of the ``next`` method on iterators is an exception to this convention. Code that nowhere contains an explicit call to a ``next`` method can nonetheless be silently affected by the presence of such a method. Therefore, this PEP proposes that iterators should have a ``__next__`` method instead of a ``next`` method (with no change in semantics). Double-Underscore Methods and Built-In Functions ================================================ The Python language defines several protocols that are implemented or customized by defining methods with double-underscore names. In each case, the protocol is provided by an internal method implemented as a C function in the interpreter. For objects defined in Python, this C function supports customization by implicitly invoking a Python method with a double-underscore name (it often does a little bit of additional work beyond just calling the Python method.) Sometimes the protocol is invoked by a syntactic construct: * ``x[y]`` --> internal ``tp_getitem`` --> ``x.__getitem__(y)`` * ``x + y`` --> internal ``nb_add`` --> ``x.__add__(y)`` * ``-x`` --> internal ``nb_negative`` --> ``x.__neg__()`` Sometimes there is no syntactic construct, but it is still useful to be able to explicitly invoke the protocol. For such cases Python offers a built-in function of the same name but without the double underscores. * ``len(x)`` --> internal ``sq_length`` --> ``x.__len__()`` * ``hash(x)`` --> internal ``tp_hash`` --> ``x.__hash__()`` * ``iter(x)`` --> internal ``tp_iter`` --> ``x.__iter__()`` Following this pattern, the natural way to handle ``next`` is to add a ``next`` built-in function that behaves in exactly the same fashion. * ``next(x)`` --> internal ``tp_iternext`` --> ``x.__next__()`` Previous Proposals ================== This proposal is not a new idea. The idea proposed here was supported by the BDFL on python-dev [1]_ and is even mentioned in the original iterator PEP, PEP 234:: (In retrospect, it might have been better to go for __next__() and have a new built-in, next(it), which calls it.__next__(). But alas, it's too late; this has been deployed in Python 2.2 since December 2001.) Transition Plan =============== Two additional transformations will be added to the 2to3 translation tool [2]_: * Method definitions named ``next`` will be renamed to ``__next__``. * Explicit calls to the ``next`` method will be replaced with calls to the built-in ``next`` function. For example, ``x.next()`` will become ``next(x)``. If the module being processed already contains a top-level function definition named ``next``, the second transformation will not be done; instead, calls to ``x.next()`` will be replaced with ``x.__next__()`` and a warning will be emitted. References ========== .. [1] Single- vs. Multi-pass iterability (Guido van Rossum) http://mail.python.org/pipermail/python-dev/2002-July/026814.html .. [2] 2to3 refactoring tool http://svn.python.org/view/sandbox/trunk/2to3/ Copyright ========= This document has been placed in the public domain. From ark-mlist at att.net Mon Mar 5 15:06:56 2007 From: ark-mlist at att.net (Andrew Koenig) Date: Mon, 5 Mar 2007 09:06:56 -0500 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: Message-ID: <000a01c75f2f$88e6d7c0$6402a8c0@arkdesktop> > FWIW, I always liked the `parameter passing is assignment` semantics > of Python. I sure hope nobody is going to start a crus^H^H^H^HPEP to > remove tuple unpacking in general from the language! Isn't the point of this discussion that it is already gone? From tanzer at swing.co.at Mon Mar 5 15:11:23 2007 From: tanzer at swing.co.at (Christian Tanzer) Date: Mon, 05 Mar 2007 15:11:23 +0100 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: Your message of "Mon, 05 Mar 2007 09:06:56 EST." <000a01c75f2f$88e6d7c0$6402a8c0@arkdesktop> Message-ID: ark-mlist at att.net wrote: > > FWIW, I always liked the `parameter passing is assignment` semantics > > of Python. I sure hope nobody is going to start a crus^H^H^H^HPEP to > > remove tuple unpacking in general from the language! > > Isn't the point of this discussion that it is already gone? AFAIU, you'll still be able to say: for k, v in dict.values() : whatever_needs_to_be_done(k, v) or x, y = some_tuple And I hope that's going to stay... -- Christian Tanzer http://www.c-tanzer.at/ From jimjjewett at gmail.com Mon Mar 5 16:06:23 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 5 Mar 2007 10:06:23 -0500 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: <000a01c75f2f$88e6d7c0$6402a8c0@arkdesktop> References: <000a01c75f2f$88e6d7c0$6402a8c0@arkdesktop> Message-ID: On 3/5/07, Andrew Koenig wrote: > > FWIW, I always liked the `parameter passing is assignment` semantics > > of Python. I sure hope nobody is going to start a crus^H^H^H^HPEP to > > remove tuple unpacking in general from the language! > Isn't the point of this discussion that it is already gone? Nobody has even suggested removing tuple-unpacking in general. The question is whether to continue doing it implicitly in function parameters. Today, you can write: >>> def f(name, (street_num, street, city, state, zip)): In python 3, you will probably have to write: >>> def f(name, addr): ... street_num, street, city, state, zip = addr or possibly the redundant >>> def f(name, addr:tuple(street_num, street, city, state, zip)): ... street_num, street, city, state, zip = addr if you want the documentation value. -jJ From collinw at gmail.com Mon Mar 5 17:28:44 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 5 Mar 2007 10:28:44 -0600 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <43aa6ff70703050828g3a720efeu6dd172af7246a43d@mail.gmail.com> On 3/5/07, Ka-Ping Yee wrote: [snip] > Transition Plan > =============== > > Two additional transformations will be added to the 2to3 translation > tool [2]_: > > * Method definitions named ``next`` will be renamed to ``__next__``. > > * Explicit calls to the ``next`` method will be replaced with calls > to the built-in ``next`` function. For example, ``x.next()`` will > become ``next(x)``. > > If the module being processed already contains a top-level function > definition named ``next``, the second transformation will not be done; > instead, calls to ``x.next()`` will be replaced with ``x.__next__()`` > and a warning will be emitted. The basic transformations for this are (relatively) easy to implement. Basing the second transformation on the absence of a top-level function definition (don't forget assignments and imports!) for "next", though, is going to require some surgery to 2to3's internals. I'll see what I can do. Collin Winter From python at rcn.com Mon Mar 5 18:31:01 2007 From: python at rcn.com (python at rcn.com) Date: Mon, 5 Mar 2007 12:31:01 -0500 (EST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in Message-ID: <20070305123101.BAH59075@ms09.lnh.mail.rcn.net> Can I suggest that next() and __next__() be dropped entirely and that iterators just be made callable. The current approach seems to make people think there is something wrong with using an iterator directly (inside of in a for-loop or function that consumes an iterator. >>> ### What we do now >>> import itertools >>> count = itertools.count(1).next >>> count() 1 >>> count() 2 >>> ### What would be nice >>> import itertools >>> cnt = itertools.count(1) >>> cnt() 1 >>> cnt() 2 >>> def f(x): ... while x > 1: ... yield x ... if x % 2 == 0: ... x //= 2 ... else: ... x = x * 3 + 1 ... >>> g = f(12) >>> g() 12 >>> g() 6 >>> g() 3 >>> g() 10 Raymond Hettinger From martin at v.loewis.de Mon Mar 5 19:36:23 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 05 Mar 2007 19:36:23 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <45EC6327.8010906@v.loewis.de> Ka-Ping Yee schrieb: > The iterator protocol in Python 2.x consists of two methods: > ``__iter__()`` called on an iterable object to yield an iterator, and > ``next()`` called on an iterator object to yield the next item in the > sequence. Using a ``for`` loop to iterate over an iterable object > implicitly calls both of these methods. This PEP proposes that the > ``next`` method be renamed to ``__next__``, consistent with all the > other protocols in Python in which a method is implicitly called as > part of a language-level protocol +1 > and that a built-in function named > ``next`` be introduced to invoke ``__next__`` method, consistent with > the manner in which other protocols are explicitly invoked. -1. I dislike the introduction of more builtins unless they have a true generality (i.e. are likely to be needed in many programs). For this one, I think the normal usage of __next__ will be with a for loop, so I don't think one would often need an explicit next() invocation. It is also not true that most protocols are explicitly invoked through builtin functions. Instead, most protocols are can be explicitly invoked through methods in the operator module. So following tradition, it should be operator.next. Of those protocols that are not operators, it is still not true that they normally invocation through builtin functions is available. Things like data attributes (__dict__, __slots__) don't have accessor functions; __class__ is accessed through type() (not class()), __init__ and __del__ don't have any wrappers although they are methods, same for __copy__, __deepcopy__, __new__, __length_hint__; and reduce() doesn't invoke __reduce__. I do know that len(), str(), unicode() etc all call their protocols. However, it is *not* the case that these builtin exists because the protocol was first. It is vice versa: the builtin was there, and a way was need to extend it for user-defined types. As an alternative, I propose that object grows a .next() method, which calls __next__ by default. Please record this objection in the PEP even if you don't change it otherwise. Regards, Martin From barry at python.org Mon Mar 5 19:44:12 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 5 Mar 2007 13:44:12 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC6327.8010906@v.loewis.de> References: <45EC6327.8010906@v.loewis.de> Message-ID: <81F49932-D799-4126-B129-E6218125048C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 5, 2007, at 1:36 PM, Martin v. L?wis wrote: > -1. I dislike the introduction of more builtins unless they have a > true > generality (i.e. are likely to be needed in many programs). For this > one, I think the normal usage of __next__ will be with a for loop, so > I don't think one would often need an explicit next() invocation. I have had occasional need to call .next() explicitly. IIRC the code in question was handed an iterator but only needed the first object out of the iterator. It would be really ugly to have to use __next__ () or force a once-through for loop. OTOH, making iterators callable as equivalent to .next() as I think Raymond suggested sounds about perfect. You get __next__(), plus explicit iteration with no additional built in necessary. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRexk/XEjvBPtnXfVAQIq0gP+KULlyQNTAEsrxqQIQYO8e5eP/zbIazsa idKofJaQVEp9QbpIUBWj4RdzfesEkhjhy3dtNo3APGMCej4YsPhuHQMeoucVSKA/ 1FUXwyVUliqa4yrN9pFadXYTd9dZcep++Hbgb76REp2wUejdNkO0k/lnIZ0JERcv 33Wt64r21x4= =ZWgi -----END PGP SIGNATURE----- From guido at python.org Mon Mar 5 19:48:50 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 5 Mar 2007 10:48:50 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC6327.8010906@v.loewis.de> References: <45EC6327.8010906@v.loewis.de> Message-ID: On 3/5/07, "Martin v. L?wis" wrote: > As an alternative, I propose that object grows a .next() method, > which calls __next__ by default. This would seem slated to confuse users; they will attempt to implement next() instead of __next__(). Or people would attempt to override these differently. If the builtin next() bites the dust, I would rather learn to live with calling __next__() directly, just as for certain occasions one resorts to call __import__(). On 3/5/07, python at rcn.com wrote: > Can I suggest that next() and __next__() be dropped entirely > and that iterators just be made callable. This sounds attractive, except that I fear that the optimizations we've implemented for calling tp_next (especially for built-in iterators like list or dict iterators). Since tp_call must be usable in other contexts, and has much more optional features, it will be hard to carry these optimizations over. It would also invalidate classes that serve as their own iterator (like files) and already implement __call__() for sa different purpose. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Mon Mar 5 19:54:21 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 05 Mar 2007 10:54:21 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: <000a01c75f2f$88e6d7c0$6402a8c0@arkdesktop> Message-ID: <20070305105049.7486.JCARLSON@uci.edu> "Jim Jewett" wrote: > >>> def f(name, addr:tuple(street_num, street, city, state, zip)): > ... street_num, street, city, state, zip = addr Except that it won't work; annotations are evaluated at function definition time, so street_num, street, city, state, zip would need to exist as actual names. It could certainly be faked with X.street_num, but it's all up to the desires of the person doing the annotations. - Josiah From guido at python.org Mon Mar 5 20:04:08 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 5 Mar 2007 11:04:08 -0800 Subject: [Python-3000] PEP 3113 (Removal of Tuple Parameter Unpacking) In-Reply-To: References: Message-ID: On 3/5/07, Christian Tanzer wrote: > FWIW, I always liked the `parameter passing is assignment` semantics > of Python. The parameter list is already so chock full of special cases (keyword params, *varargs, **kwds, default values, and in Py3k required keyword args and argument annotations) that the fiction that the parameter list is just an assignment target is hard to take seriously. Instead, it has become yet another special case that clutters up implementation, documentation, and users' understanding. This sums up my reasons for removing the features. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Mon Mar 5 20:10:58 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 05 Mar 2007 11:10:58 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <20070305110807.7489.JCARLSON@uci.edu> Ka-Ping Yee wrote: > > At Guido's prompting, I drafted the following PEP. How does it sound? > > Title: Renaming iterator.next() to iterator.__next__() > ====================================================== It sounds and reads just fine, though I'm -1 on the change if only because 1) it adds a builtin, 2) it renames a protocol method, making it potentially difficult and/or buggy to write forward or backward compatible code, 3) it smells like gratuitous breakage. - Josiah From python-dev at zesty.ca Mon Mar 5 20:07:52 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 13:07:52 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC6327.8010906@v.loewis.de> References: <45EC6327.8010906@v.loewis.de> Message-ID: One possibility for making a next() built-in more useful: Just like getattr, two-argument next(iter, sentinel) returns sentinel if StopException is caught. If you've used .next() explicitly, would this be handy? (Or, what convenience would you want in a next() builtin?) -- ?!ng From barry at python.org Mon Mar 5 20:12:01 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 5 Mar 2007 14:12:01 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> Message-ID: <37892E8D-DD94-46E6-B6A6-CA8504A7129C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 5, 2007, at 2:07 PM, Ka-Ping Yee wrote: > One possibility for making a next() built-in more useful: > > Just like getattr, two-argument next(iter, sentinel) > returns sentinel if StopException is caught. > > If you've used .next() explicitly, would this be handy? Yes. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRexrgXEjvBPtnXfVAQLjsAP/UxgXDQ37pLPvIVcE8RouqZVwWAUZtYef d0vubjBFXfaT1aBpnrk0YLcg55Xqd/+LkOXl5UZdO8FYyi4ucGRAZ8Kz1P1T9g+k TbTumxCa9Qzj8ockoJv7vJ3+K1jOS0A05hjeOq93jROTWtdyxqcBUDvTY2lQn34J ZK5L0KFwz+E= =qjFx -----END PGP SIGNATURE----- From python at rcn.com Mon Mar 5 20:26:07 2007 From: python at rcn.com (python at rcn.com) Date: Mon, 5 Mar 2007 14:26:07 -0500 (EST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in Message-ID: <20070305142607.BAH99221@ms09.lnh.mail.rcn.net> [Raymond]: > Can I suggest that next() and __next__() be dropped entirely > and that iterators just be made callable. [GvR] > This sounds attractive, except that I fear that > the optimizations we've implemented for calling > tp_next (especially for built-in iterators like > list or dict iterators). Since tp_call must be > usable in other contexts, and has much more > optional features, it will be hard to carry > these optimizations over. I should mull this over for a bit. It may be possible to keep the tp_iter slot for builtins and have the tp_call slot forward the invocation. That keeps the speed for the C iterators, simplifies the API, and will still be faster than next() because it bypasses the method name lookup. OTOH, I haven't thought this through and might encounter a roadblock when trying to implement it. > It would also invalidate classes that serve as their > own iterator (like files) and already implement > __call__() for sa different purpose. My thought here is that iterators should always be a separate object -- there is no good reason for dir(iter(myfile)) to expose methods that have nothing to do with iteration. In the case of files, it would not be hard to have a singleton file-iterator object. my-two-cents, Raymond P.S. I have to credit David Mertz with being one of the original proponents of making iterators callable instead of pushing the functionality into the next() method. From python at rcn.com Mon Mar 5 20:29:55 2007 From: python at rcn.com (python at rcn.com) Date: Mon, 5 Mar 2007 14:29:55 -0500 (EST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in Message-ID: <20070305142955.BAI00560@ms09.lnh.mail.rcn.net> [Ka-Ping] > Title: Renaming iterator.next() to iterator.__next__() > ====================================================== [Josiah] > It sounds and reads just fine, though I'm -1 on the change > if only because 1) it adds a builtin, 2) it renames a > protocol method, making it potentially difficult and/or > buggy to write forward or backward compatible code, > 3) it smells like gratuitous breakage. And, more importantly, it is butt-ugly. Raymond From pje at telecommunity.com Mon Mar 5 21:09:09 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 05 Mar 2007 15:09:09 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC6327.8010906@v.loewis.de> Message-ID: <5.1.1.6.0.20070305145912.02ba9a10@sparrow.telecommunity.com> At 10:48 AM 3/5/2007 -0800, Guido van Rossum wrote: >I fear that the optimizations >we've implemented for calling tp_next (especially for built-in >iterators like list or dict iterators). Since tp_call must be usable >in other contexts, and has much more optional features, it will be >hard to carry these optimizations over. Ah... I sense an opportunity disguised as a problem. :) How about we add a tp_call0 slot for fast zero-argument calling? This could be the first step towards general IronPython-style NxN call dispatch optimization. There are probably some interesting problems to be solved there, but the payoff could be broader. I suspect we would actually want to also implement a tp_call1 (and maybe a tp_call2), because we would want e.g. method objects' tp_call0 to forward to the tp_call1 of their im_func. For the greatest optimization payoff, of course, we would need to support C function/method types that support this convention. Once there's a manual implementation for N up to say, 2, then we could script some code generation to do the permutations up to some reasonable number. This idea is of course independent of what happens with iterators, but perhaps Raymond will be more incentivized to take on this particular optimization challenge if it will weigh in favor of dropping next(). :) From guido at python.org Mon Mar 5 23:04:39 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 5 Mar 2007 14:04:39 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305110807.7489.JCARLSON@uci.edu> References: <20070305110807.7489.JCARLSON@uci.edu> Message-ID: On 3/5/07, Josiah Carlson wrote: > because [...] 2) it renames a protocol method, making it > potentially difficult and/or buggy to write forward or backward > compatible code, [...] Did you see my slides and/or video of my talk about Py3k yet? This change can be carried out automatically. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 5 23:07:47 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 5 Mar 2007 14:07:47 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305142607.BAH99221@ms09.lnh.mail.rcn.net> References: <20070305142607.BAH99221@ms09.lnh.mail.rcn.net> Message-ID: On 3/5/07, python at rcn.com wrote: > My thought here is that iterators should always be a separate object -- there is no good reason for dir(iter(myfile)) to expose methods that have nothing to do with iteration. In the case of files, it would not be hard to have a singleton file-iterator object. The question isn't how hard it would be, but whether it would be the right thing to do. For iterators where (by nature of the object being iterated over) there's only one iterator possible, I think returning self is the right thing to do, as it makes it abundantly clear that multiple parallel iterations are unsupported. (And let's not get into the discussion of how to support multiple iterators over files -- we went over that when we first decided to make files iterable.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Mar 5 23:16:08 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 11:16:08 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <81F49932-D799-4126-B129-E6218125048C@python.org> References: <45EC6327.8010906@v.loewis.de> <81F49932-D799-4126-B129-E6218125048C@python.org> Message-ID: <45EC96A8.4080501@canterbury.ac.nz> Barry Warsaw wrote: > OTOH, making iterators callable as equivalent to .next() as I think > Raymond suggested sounds about perfect. You get __next__(), plus > explicit iteration with no additional built in necessary. But that would raise the question of why __next__ wasn't called __call__ instead. Also I'm not sure I like this idea. It smells like an abuse of the notion of calling to me. One doesn't normally expect that calling a callable object has a side effect on the callable itself. -- Greg From tjreedy at udel.edu Mon Mar 5 23:20:54 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 5 Mar 2007 17:20:54 -0500 Subject: [Python-3000] rename it.next() to it.__next__(), add a next() built-in References: Message-ID: "Ka-Ping Yee" wrote in message news:Pine.LNX.4.58.0703041845250.11751 at server1.LFW.org... On the consistency argument: sorted(somelist) calls copy(somelist).sort, not copy(somelist).__sort__. But I don't expect to see this change proposed. | Transition Plan | =============== | | Two additional transformations will be added to the 2to3 translation | tool [2]_: | | * Method definitions named ``next`` will be renamed to ``__next__``. Consistently renaming 'next' to '__next__', including at call sites, would be one thing... As explained below, you *must* translate at least non-call name lookups to not break programs. | * Explicit calls to the ``next`` method will be replaced with calls | to the built-in ``next`` function. For example, ``x.next()`` will | become ``next(x)``. But this is something else -- a de-optimization. One of the virtues, and as I recall, design purposes, of .next calls is to be fast. After the first call, execution jumps directly into the pre-existing stack frame. This change would interpose a regular old slow call process, thereby undoing this speedup (relative to the older .__getitem__ protocol). The next 'method' is uniquely intended to be called repeatedly, even billions of times, on the same instance, with the same args (none). So it should be as fast as possible. It is also useless for a large class of explicit uses. Explicit calls are usually done within a while loop. It is then fairly standard to factor out the attribute lookup with with 'xnext = x.next' or 'xnext = genx(args).next' before the loop and use the bound method within the loop. (I presume the for loop machinery does the same.) * So the translation tool must also change the name to '__next__' in such attribute lookups. Terry Jan Reedy From barry at python.org Mon Mar 5 23:26:01 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 5 Mar 2007 17:26:01 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC96A8.4080501@canterbury.ac.nz> References: <45EC6327.8010906@v.loewis.de> <81F49932-D799-4126-B129-E6218125048C@python.org> <45EC96A8.4080501@canterbury.ac.nz> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 5, 2007, at 5:16 PM, Greg Ewing wrote: > Also I'm not sure I like this idea. It smells like > an abuse of the notion of calling to me. One doesn't > normally expect that calling a callable object has > a side effect on the callable itself. Perhaps, but I think there's a valid use case for calling "next" explicitly, and I would really hate for that to be spelled "it.__next__()". - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBReyY+XEjvBPtnXfVAQLdmgP9EAJzYtvuXHFJ1DoWj42QOT79xCqpxNMs qcLDF9sRCprrzMWHrjlUcC0NZmBJGXotB4TNMO2CtkwbidiIBweo0xwdgwaUDCED gV+tzrtQtIx516Lv/7gfR08yDl3TmkckVOmaQrQ2RT6QiRuT59+IkhIG9yKlbezn oo+dODRTYRQ= =6eYL -----END PGP SIGNATURE----- From python-dev at zesty.ca Mon Mar 5 23:31:44 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 16:31:44 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305123101.BAH59075@ms09.lnh.mail.rcn.net> References: <20070305123101.BAH59075@ms09.lnh.mail.rcn.net> Message-ID: On Mon, 5 Mar 2007 python at rcn.com wrote: > Can I suggest that next() and __next__() be dropped entirely > and that iterators just be made callable. We went through a long discussion about this when iterators were being figured out the first time around -- in fact, using __call__ was the original proposal, and then it was changed to __next__. I imagine we must have had good reasons for it. PEP 234 (it amuses me that the PEP number for iterators is three digits in arithmetic sequence) says: Arguments against __call__() (the original proposal): taken out of context, x() is not very readable, while x.next() is clear; there's a danger that every special-purpose object wants to use __call__() for its most common operation, causing more confusion than clarity. I think __next__ is better than __call__ because EIBTI. It's a lot more obvious that you're doing explicit iteration (and have all the expectations that go along with it) when you see the call to the next built-in (or method). Explicit iteration (calling next) is quite a bit less common than implicit iteration (with for loops), so it's probably a good idea for it to have a distinct look to it. Function calls happen all over the place; it would be easy not to notice that iteration is occurring with an x() call buried among lots of other stuff. -- ?!ng From python-dev at zesty.ca Mon Mar 5 23:36:00 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 16:36:00 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305142955.BAI00560@ms09.lnh.mail.rcn.net> References: <20070305142955.BAI00560@ms09.lnh.mail.rcn.net> Message-ID: On Mon, 5 Mar 2007 python at rcn.com wrote: > And, more importantly, it is butt-ugly. What part exactly? Does this bother you: def __next__(self): ... or this: while whatever: x = next(i) ... They look pretty much like the rest of Python to me. -- ?!ng From python at rcn.com Mon Mar 5 23:46:39 2007 From: python at rcn.com (Raymond Hettinger) Date: Mon, 5 Mar 2007 17:46:39 -0500 (EST) Subject: [Python-3000] Fwd: Re: PEP: rename it.next() to it.__next__(), add a next() built-in Message-ID: <20070305174639.BAI65900@ms09.lnh.mail.rcn.net> [Raymond] >> My thought here is that iterators should always be a separate object -- there is no good reason for dir(iter(myfile)) to expose methods that have nothing to do with iteration. In the case of files, it would not be hard to have a singleton file-iterator object. [GvR] >The question isn't how hard it would be, but whether it would be the >right thing to do. For iterators where (by nature of the object being >iterated over) there's only one iterator possible, I think returning >self is the right thing to do, as it makes it abundantly clear that >multiple parallel iterations are unsupported. I don't follow how returning self makes anything clear -- the only way to know that self was returned is to check object ids or to guess why dir() on the iterator returns all of the file methods: >>> f = open('tmp.txt') >>> it1 = iter(f) >>> it2 = iter(f) >>> map(id, [f, it1, it2]) # only this check shows that 'iter(f) is f' [3083536096L, 3083536096L, 3083536096L] >>> dir(it1) ['__class__', '__delattr__', '__doc__', '__enter__', '__exit__', '__getattribute__', '__hash__', '__init__', '__iter__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'close', 'closed', 'encoding', 'fileno', 'flush', 'isatty', 'mode', 'name', 'newlines', 'next', 'read', 'readinto', 'readline', 'readlines', 'seek', 'softspace', 'tell', 'truncate', 'write', 'writelines', 'xreadlines'] ISTM, we might as well make the singletonness explicit and hide the file methods from the iterator object (nothing good can come from conflating file ops with iteration). Raymond From greg.ewing at canterbury.ac.nz Mon Mar 5 23:45:18 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 11:45:18 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> Message-ID: <45EC9D7E.6000702@canterbury.ac.nz> Ka-Ping Yee wrote: > Just like getattr, two-argument next(iter, sentinel) > returns sentinel if StopException is caught. +1. I've written a number of pieces of code where this would have made things neater. Just about any place where I've used .next() explicitly, in fact -- it always seems awkward having to deal with StopIteration. -- Greg From g.brandl at gmx.net Mon Mar 5 23:54:25 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 05 Mar 2007 23:54:25 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC9D7E.6000702@canterbury.ac.nz> References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Ka-Ping Yee wrote: > >> Just like getattr, two-argument next(iter, sentinel) >> returns sentinel if StopException is caught. > > +1. I've written a number of pieces of code where this > would have made things neater. Just about any place > where I've used .next() explicitly, in fact -- it > always seems awkward having to deal with StopIteration. Indeed, you almost always have to have a try-except StopIteration- wrapper around "manual" calls to .next(). In many of these cases, a default value would make things nicer: +1. Georg From tjreedy at udel.edu Mon Mar 5 23:54:58 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 5 Mar 2007 17:54:58 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in References: <20070305123101.BAH59075@ms09.lnh.mail.rcn.net> Message-ID: wrote in message news:20070305123101.BAH59075 at ms09.lnh.mail.rcn.net... | Can I suggest that next() and __next__() be dropped entirely and that iterators just be made callable. Can you do that without making the call a regular, slow call each time? Or at least without adding a lookup of '__call__' each time? For generators, I could imagine renaming .next to .__call__, but in general, how would you mark something as an iterator (versus non-iterator iterable or anything else) without .next (or the proposed .__next__), since that is the very thing that now marks it (along with .__iter__) as an iterator. In other words, what would a user-defined iterator class look like? If you say .__call__ along with .__iter__, then you are stipulating that non-iterator iterables cannot be callable themselves. I have no idea it this would break code. | The current approach seems to make people think there is something wrong with using an iterator directly (inside of in a for-loop or function that consumes an iterator. No me. So 'some people' rather than 'people'. How many? Learning about method binding is fairly important. Would changing .next to .__call__ really solve whatever problem you think there is? | >>> ### What we do now | >>> import itertools | >>> count = itertools.count(1).next | >>> count() | 1 | >>> count() | 2 | | | >>> ### What would be nice But, I believe slower, due to the repeated .__call__ method name lookup | >>> import itertools | >>> cnt = itertools.count(1) | >>> cnt() | 1 | >>> cnt() | 2 so I expect the sophisticated user, if expecting to call cnt perhaps millions of times, would write cnt = itertools.count(1).__call__ # or whatever Bound .next methods are perhaps unique in potentially massive reuse. Terry Jan Reedy From jcarlson at uci.edu Tue Mar 6 00:01:33 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 05 Mar 2007 15:01:33 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <20070305110807.7489.JCARLSON@uci.edu> Message-ID: <20070305144950.7491.JCARLSON@uci.edu> "Guido van Rossum" wrote: > On 3/5/07, Josiah Carlson wrote: > > because [...] 2) it renames a protocol method, making it > > potentially difficult and/or buggy to write forward or backward > > compatible code, [...] > > Did you see my slides and/or video of my talk about Py3k yet? This > change can be carried out automatically. I read the slides. However, I have written methods named 'next' which have *nothing* to do with the iterator protocol. I probably should have written them as something like 'get_next' so as not to confuse later readers, but the lack of an __iter__, specific documentation about what .next() means, and the context in which it was written, read much better with .next() than .get_next(), or any one of the handful of other names (and alternate naming semantics) I considered at the time. Barry Warsaw wrote: > Perhaps, but I think there's a valid use case for calling "next" > explicitly, and I would really hate for that to be spelled > "it.__next__()". I agree, which is one of the reasons why I personally prefer the status-quo: it.next() . It works. Has always worked. And the 2to3 translator won't get confused by non-iterator-based .next() methods (which I'm not claiming are common, merely that they exist). - Josiah From python-dev at zesty.ca Tue Mar 6 00:03:48 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 17:03:48 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305144950.7491.JCARLSON@uci.edu> References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: On Mon, 5 Mar 2007, Josiah Carlson wrote: > "Guido van Rossum" wrote: > > On 3/5/07, Josiah Carlson wrote: > However, I have written methods named 'next' which have *nothing* to do > with the iterator protocol. This is exactly why the iterator protocol method should be named __next__: so it can't collide with method names used for other purposes. The translator will be imperfect, and probably will need some guidance to do its work correctly. But the problem you point out here is an issue primarily with the language, not the translator, and Python 3.0 is a good opportunity for incompatible changes like this that will future-proof the language. -- ?!ng From barry at python.org Tue Mar 6 00:12:56 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 5 Mar 2007 18:12:56 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC9D7E.6000702@canterbury.ac.nz> References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: <110B2DA0-335F-4FA2-8A46-E80A3D5D80D2@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 5, 2007, at 5:45 PM, Greg Ewing wrote: > Ka-Ping Yee wrote: > >> Just like getattr, two-argument next(iter, sentinel) >> returns sentinel if StopException is caught. > > +1. I've written a number of pieces of code where this > would have made things neater. Just about any place > where I've used .next() explicitly, in fact -- it > always seems awkward having to deal with StopIteration. I agree. Seems like enough of a use case to me to justify built-in fancy schmancy next() and moving .next() to .__next__(). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBReyj+XEjvBPtnXfVAQK0VAQAufa87C9RM8eBBy5DrSPW8n1ERMUHHreA HtdT6rXt9SlRVckOrz5sHf/QdHa76jpSrLpk6mDBx3i1R93NPSrI+gkagOZhd21a eRLWlW84UotY6kzLFqLSyH1DnlHgad9Oimy9eMZjuf4xK50hU9BKgZNxcl1ogzYa V6udox5tEqU= =pm9U -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Tue Mar 6 00:17:47 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:17:47 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <20070305123101.BAH59075@ms09.lnh.mail.rcn.net> Message-ID: <45ECA51B.4060002@canterbury.ac.nz> Ka-Ping Yee wrote: > PEP 234 (it amuses me that the PEP number for iterators is > three digits in arithmetic sequence) I guess the equivalent PEP for Py3k should be numbered PEP 3456 then. :-) -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 6 00:24:58 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:24:58 +1300 Subject: [Python-3000] rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <45ECA6CA.6010309@canterbury.ac.nz> Terry Reedy wrote: > One of the virtues, and as I recall, design purposes, of .next calls is to > be fast. After the first call, execution jumps directly into the > pre-existing stack frame. You're thinking of generators, but not all iterators are implemented by generators. The built-in ones are implemented in C. Invoking a builtin .next() method via a protocol function shouldn't be any slower than it is now. In both cases you've got one name lookup and one Python call, after which you go through the type slot directly to a C function. > It is then fairly > standard to factor out the attribute lookup with with 'xnext = x.next' You could still do xnext = x.__next__ if you wanted. -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 6 00:31:34 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:31:34 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: <45ECA856.2090606@canterbury.ac.nz> Georg Brandl wrote: > Indeed, you almost always have to have a try-except StopIteration- > wrapper around "manual" calls to .next(). An alternative way of addressing this would be to have a new control structure. We already have the 'for' statement for when you want all the values; why not another one for when you just want one value? Maybe something like get item from iter: # code for when we got something else: # code for when we didn't get anything -- Greg From python at rcn.com Tue Mar 6 00:35:14 2007 From: python at rcn.com (Raymond Hettinger) Date: Mon, 5 Mar 2007 18:35:14 -0500 (EST) Subject: [Python-3000] Fwd: Re: rename it.next() to it.__next__(), add a next() built-in Message-ID: <20070305183514.BAI78440@ms09.lnh.mail.rcn.net> [Greg Ewing] >Invoking a builtin .next() method via a protocol >function shouldn't be any slower than it is now. >In both cases you've got one name lookup and one >Python call, after which you go through the type >slot directly to a C function. Do you expect that next(someiterator) will be just as fast and calling a boundmethod (such as: counter=itertools.counter(1); counter())? Raymond > >> It is then fairly >> standard to factor out the attribute lookup with with 'xnext = x.next' > >You could still do xnext = x.__next__ if you wanted. > >-- >Greg >_______________________________________________ >Python-3000 mailing list >Python-3000 at python.org >http://mail.python.org/mailman/listinfo/python-3000 >Unsubscribe: http://mail.python.org/mailman/options/python-3000/python%40rcn.com From greg.ewing at canterbury.ac.nz Tue Mar 6 00:35:01 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:35:01 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305144950.7491.JCARLSON@uci.edu> References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: <45ECA925.9030602@canterbury.ac.nz> Josiah Carlson wrote: > I have written methods named 'next' which have *nothing* to do > with the iterator protocol. That would be another reason for renaming .next() to .__next__() -- to avoid intruding on the user's namespace. -- Greg From python at rcn.com Tue Mar 6 00:46:57 2007 From: python at rcn.com (Raymond Hettinger) Date: Mon, 5 Mar 2007 18:46:57 -0500 (EST) Subject: [Python-3000] Fwd: Re: PEP: rename it.next() to it.__next__(), add a next() built-in Message-ID: <20070305184657.BAI81150@ms09.lnh.mail.rcn.net> [Josiah Carlson] >> I have written methods named 'next' which have *nothing* to do >> with the iterator protocol. [Greg] >That would be another reason for renaming .next() to >.__next__() -- to avoid intruding on the user's >namespace. Another read is that iterators should be separate objects so that the namespace is coherent (with __iter__, next, and whatnot all being iterator related). Who is going to create a custon iterator that defines BOTH __next__ and next(), that would be less than ideal and not something we want to encourage. Raymond From greg.ewing at canterbury.ac.nz Tue Mar 6 00:44:46 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:44:46 +1300 Subject: [Python-3000] Fwd: Re: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305183514.BAI78440@ms09.lnh.mail.rcn.net> References: <20070305183514.BAI78440@ms09.lnh.mail.rcn.net> Message-ID: <45ECAB6E.6050203@canterbury.ac.nz> Raymond Hettinger wrote: > Do you expect that next(someiterator) will be just > as fast and calling a boundmethod (such as: > counter=itertools.counter(1); counter())? If you snarf a local reference to the next() function, I expect it will be comparable in the case where the iterator is implemented in C, and probably also when it's implemented by a Python generator. -- Greg From aahz at pythoncraft.com Tue Mar 6 00:47:21 2007 From: aahz at pythoncraft.com (Aahz) Date: Mon, 5 Mar 2007 15:47:21 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <20070305234721.GA10038@panix.com> On Mon, Mar 05, 2007, Ka-Ping Yee wrote: > > Abstract > ======== > > The iterator protocol in Python 2.x consists of two methods: > ``__iter__()`` called on an iterable object to yield an iterator, and > ``next()`` called on an iterator object to yield the next item in the > sequence. Using a ``for`` loop to iterate over an iterable object > implicitly calls both of these methods. This PEP proposes that the > ``next`` method be renamed to ``__next__``, consistent with all the > other protocols in Python in which a method is implicitly called as > part of a language-level protocol, and that a built-in function named > ``next`` be introduced to invoke ``__next__`` method, consistent with > the manner in which other protocols are explicitly invoked. +1 -- I was always against next() in the first place. I'm +0 on operator.next() relative to builtin next(). -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "I disrespectfully agree." --SJM From jcarlson at uci.edu Tue Mar 6 00:56:11 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 05 Mar 2007 15:56:11 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45ECA856.2090606@canterbury.ac.nz> References: <45ECA856.2090606@canterbury.ac.nz> Message-ID: <20070305154815.749A.JCARLSON@uci.edu> Greg Ewing wrote: > > Georg Brandl wrote: > > > Indeed, you almost always have to have a try-except StopIteration- > > wrapper around "manual" calls to .next(). The use-cases would likely be analagous to the .find() vs. .index() methods on strings. Then again, while I preferred .find() on strings, I don't mind catching StopIteration in .next() - probably because I do far more string manipulations than I do iterator manipulations, so I'm not bitten as much by the annoyance of having to try/except StopIteration. What about just adding a global next() (or operator.next()) and not renaming the current .next() methods to __next__()? It wouldn't be consistant with other builtins that do automatic dispatch to __magic__ methods, but then the 2to3 translator wouldn't even need to get involved (and potentially muck up non-method .next instance attributes, xnext = x.next method or value caching, etc.). > An alternative way of addressing this would be to > have a new control structure. We already have the > 'for' statement for when you want all the values; > why not another one for when you just want one > value? Maybe something like > > get item from iter: > # code for when we got something > else: > # code for when we didn't get anything It's already spelled... for item in iter: #code for when we got something break else: #code for when we didn't get anything - Josiah From greg.ewing at canterbury.ac.nz Tue Mar 6 00:52:20 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:52:20 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305234721.GA10038@panix.com> References: <20070305234721.GA10038@panix.com> Message-ID: <45ECAD34.5010709@canterbury.ac.nz> Aahz wrote: > I'm +0 on operator.next() relative to builtin next(). I'm -0.7. Classifying next() as an operator doesn't seem right -- what "operator" does it correspond to? Unlike all the other functions in the operator module, there is no piece of syntax that corresponds directly to what next() would do. There's the for-loop, but that does considerably more. -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 6 00:59:05 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 06 Mar 2007 12:59:05 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305154815.749A.JCARLSON@uci.edu> References: <45ECA856.2090606@canterbury.ac.nz> <20070305154815.749A.JCARLSON@uci.edu> Message-ID: <45ECAEC9.5020405@canterbury.ac.nz> Josiah Carlson wrote: > It's already spelled... > > for item in iter: > #code for when we got something > break > else: > #code for when we didn't get anything Technically this is true, but I can't help feeling that's a terribly obscure way to use a for-loop. Normally 'for' means you're iterating over the whole sequence, or at least potentially more than one item from it. A different keyword at the beginning would make it much clearer that you only want one item. BTW, I would really have liked to make it with item from iter: ... else: ... but I fear that would be impossibly confusing given what we've already used 'with' for. :-( -- Greg From nas at arctrix.com Tue Mar 6 02:29:31 2007 From: nas at arctrix.com (Neil Schemenauer) Date: Tue, 6 Mar 2007 01:29:31 +0000 (UTC) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: Ka-Ping Yee wrote: > This is exactly why the iterator protocol method should be named > __next__: so it can't collide with method names used for other > purposes. And yet people are suggesting that __call__ be used instead of __next__. If people believe iterators should be separate objects then there is no problem with name collisions. Even if you don't believe in always using separate iterator objects, using one is not such a big burden in the case of name collisions. The argument that all "protocol" methods should have double underscore names seems to be pretty weak too. It's only an appeal for consistency, I think. We don't suggest that file-like objects should implement __read__() instead of read(), for example. Neil From tjreedy at udel.edu Tue Mar 6 02:37:33 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 5 Mar 2007 20:37:33 -0500 Subject: [Python-3000] rename it.next() to it.__next__(), add a next() built-in References: <45ECA6CA.6010309@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:45ECA6CA.6010309 at canterbury.ac.nz... | Terry Reedy wrote: | | > One of the virtues, and as I recall, design purposes, of .next calls is to | > be fast. After the first call, execution jumps directly into the | > pre-existing stack frame. | | You're thinking of generators, I noticed after sending | > It is then fairly | > standard to factor out the attribute lookup with with 'xnext = x.next' | | You could still do xnext = x.__next__ if you wanted. Of course. And one of my points is that the translater must make this translation, which seems not to be in the current proposal, to avoid breaking code. The other is that this idiom makes the use cases for a builtin rather rare. tjr From python at zesty.ca Tue Mar 6 03:28:04 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 20:28:04 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: On Tue, 6 Mar 2007, Neil Schemenauer wrote: > The argument that all "protocol" methods should have double > underscore names seems to be pretty weak too. It's only an appeal > for consistency, I think. We don't suggest that file-like objects > should implement __read__() instead of read(), for example. There is a convention and it is applied quite consistently: Double-underscores are for methods implicitly invoked by a language construct. In fact, your example was specifically anticipated and addressed in the PEP draft I posted here. file.read() is not invoked by a language construct. When file.read() gets called, it is because the calling code has an explicit call to read() in it, not because the standard semantics of some piece of Python syntax require it to be invoked. Not so for __getitem__, __add__, __iter__, etc. and also __next__. This distinction is important because the "magical" invocation is what makes name collisions more dangerous and confusing to debug. -- ?!ng From python at zesty.ca Tue Mar 6 03:56:29 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Mon, 5 Mar 2007 20:56:29 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <3cdcefb80703051852j28565350h684e2e1504cdcd4e@mail.gmail.com> References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> <3cdcefb80703051852j28565350h684e2e1504cdcd4e@mail.gmail.com> Message-ID: On Mon, 5 Mar 2007, Greg Falcon wrote: > I agree that file.read() is a different beast than iter.next(). > However, file.write() is a counterexample to your argument here. It > gets called by Python syntax, and yet doesn't (and shouldn't) have > double underscores. > > >>> print >> object(), "foo" > Traceback (most recent call last): > ... > AttributeError: 'object' object has no attribute 'write' True... and this is going away in Python 3.0. -- ?!ng From veloso at verylowsodium.com Tue Mar 6 03:52:27 2007 From: veloso at verylowsodium.com (Greg Falcon) Date: Mon, 5 Mar 2007 21:52:27 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: <3cdcefb80703051852j28565350h684e2e1504cdcd4e@mail.gmail.com> On 3/5/07, Ka-Ping Yee wrote: > On Tue, 6 Mar 2007, Neil Schemenauer wrote: > > We don't suggest that file-like objects > > should implement __read__() instead of read(), for example. > > There is a convention and it is applied quite consistently: > > Double-underscores are for methods implicitly invoked > by a language construct. > > In fact, your example was specifically anticipated and addressed in > the PEP draft I posted here. file.read() is not invoked by a language > construct. When file.read() gets called, it is because the calling > code has an explicit call to read() in it, not because the standard > semantics of some piece of Python syntax require it to be invoked. I agree that file.read() is a different beast than iter.next(). However, file.write() is a counterexample to your argument here. It gets called by Python syntax, and yet doesn't (and shouldn't) have double underscores. >>> print >> object(), "foo" Traceback (most recent call last): ... AttributeError: 'object' object has no attribute 'write' Greg F From pmaupin at gmail.com Tue Mar 6 04:28:04 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 5 Mar 2007 21:28:04 -0600 Subject: [Python-3000] Exception tracebacks and PEP 3101 Message-ID: I'm a newbie when it comes to deep Python internals, so maybe I'm missing something obvious. I'm trying to figure out the right answer for exceptions/tracebacks in PEP 3101 (advanced string formatting). Because the PEP 3101 string formatting is so much more powerful than the existing %s, it seems that users will be much more tempted to write large strings with complicated expressions inside them. When an exception is thrown and the user needs to debug such a string, it would be very useful to be able to show the user line number/character information for the failing location inside the format string itself. (The utility of this cannot be overstated -- I believe it will make or break the out-of-the-box experience for this new feature for many users.) But because Python is so dynamic, if an exception is thrown during an attribute or index operation during a call to the format() function, the exception _might_ be because of something several execution frames deep into the attribute operation, and it might be useful to show the traceback stack to the user. However, it seems that the Python traceback scheme is intimately tied to the concept of Python execution frames, which are assumed to have associated text information from a file. I'm not sure that I can legitimately add to a preexisting traceback from a C function in a non-fragile fashion. Even if I could, I don't really have a "file" to show the user (unless I dynamically create one). So it seems the only answer might have to have options like "let the user see the existing traceback" or "get rid of the traceback and raise an exception which shows exactly where in the format string the error occured", or "print the preexisting traceback to stderr, then toast it and raise a new exception showing the string location in the exception text." I guess there are two parts to my question: 1) What is the best thing to do right now for PEP3101? 2) If the "right now" answer isn't very good, is this really a more general problem that needs work? As more "little languages" do things like manipulate the AST, it might become very useful to have the ability to place almost arbitrary objects on the traceback stack. Thanks, Pat From talin at acm.org Tue Mar 6 07:21:40 2007 From: talin at acm.org (Talin) Date: Mon, 05 Mar 2007 22:21:40 -0800 Subject: [Python-3000] Exception tracebacks and PEP 3101 In-Reply-To: References: Message-ID: <45ED0874.80200@acm.org> Patrick Maupin wrote: > 1) What is the best thing to do right now for PEP3101? > > 2) If the "right now" answer isn't very good, is this really a more > general problem that needs work? As more "little languages" do things > like manipulate the AST, it might become very useful to have the > ability to place almost arbitrary objects on the traceback stack. Off the top of my head (without thinking about it too deeply) the way I would approach it is to catch the exception inside the string formatting function, wrap it in another exception that has additional information about where in the formatting string the error occurred, and then re-raise that second exception, using the traceback provided from the first exception. So for example, at the application level, you might get an exception with a message such as: "Error in format string, line 2, character 7: Argument Exception" -- Talin From pmaupin at gmail.com Tue Mar 6 02:44:53 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 5 Mar 2007 19:44:53 -0600 Subject: [Python-3000] Exception tracebacks and PEP 3101 Message-ID: I'm a newbie when it comes to deep Python internals, so maybe I'm missing something obvious. I'm trying to figure out the right answer for exceptions/tracebacks in PEP 3101 (advanced string formatting). Because the PEP 3101 string formatting is so much more powerful than the existing %s, it seems that users will be much more tempted to write large strings with complicated expressions inside them. When an exception is thrown and the user needs to debug such a string, it would be very useful to be able to show the user line number/character information for the failing location inside the format string itself. (The utility of this cannot be overstated -- I believe it will make or break the out-of-the-box experience for this new feature for many users.) But because Python is so dynamic, if an exception is thrown during an attribute or index operation during a call to the format() function, the exception _might_ be because of something several execution frames deep into the attribute operation, and it might be useful to show the traceback stack to the user. However, it seems that the Python traceback scheme is intimately tied to the concept of Python execution frames, which are assumed to have associated text information from a file. I'm not sure that I can legitimately add to a preexisting traceback from a C function in a non-fragile fashion. Even if I could, I don't really have a "file" to show the user (unless I dynamically create one). So it seems the only answer might have to have options like "let the user see the existing traceback" or "get rid of the traceback and raise an exception which shows exactly where in the format string the error occured", or "print the preexisting traceback to stderr, then toast it and raise a new exception showing the string location in the exception text." I guess there are two parts to my question: 1) What is the best thing to do right now for PEP3101? 2) If the "right now" answer isn't very good, is this really a more general problem that needs work? As more "little languages" do things like manipulate the AST, it might become very useful to have the ability to place almost arbitrary objects on the traceback stack. Thanks, Pat From martin at v.loewis.de Tue Mar 6 12:18:44 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 06 Mar 2007 12:18:44 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45ECAD34.5010709@canterbury.ac.nz> References: <20070305234721.GA10038@panix.com> <45ECAD34.5010709@canterbury.ac.nz> Message-ID: <45ED4E14.30907@v.loewis.de> Greg Ewing schrieb: > I'm -0.7. Classifying next() as an operator doesn't > seem right -- what "operator" does it correspond to? > > Unlike all the other functions in the operator > module, there is no piece of syntax that corresponds > directly to what next() would do. Not all of them. operator.attrgetter/itemgetter don't directly correspond to a piece of syntax, either - they return callables that correspond to the syntax. operator.countOf relates to no syntax whatsoever. operator.index is quite close to the proposed next() method: __index__ doesn't directly correspond to syntax, either, but is used as part of a "larger" protocol (o[i], in this case, for an o that needs numbers as indices). operator.isNumberType,isMappingType,isSequenceType don't correspond to syntax, either. operator.truth again is similar to the proposed next() builtin: the protocol it exposes is used in several constructs (if, while), yet it doesn't correspond to an operator (interestingly enough, it is not an alias for the bool builtin type) Regards, Martin From thomas at python.org Tue Mar 6 12:19:26 2007 From: thomas at python.org (Thomas Wouters) Date: Tue, 6 Mar 2007 12:19:26 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: <9e804ac0703060319q42fef66q6cf47ecf80ccbaee@mail.gmail.com> On 3/6/07, Ka-Ping Yee wrote: > > On Tue, 6 Mar 2007, Neil Schemenauer wrote: > > The argument that all "protocol" methods should have double > > underscore names seems to be pretty weak too. It's only an appeal > > for consistency, I think. We don't suggest that file-like objects > > should implement __read__() instead of read(), for example. > > There is a convention and it is applied quite consistently: > > Double-underscores are for methods implicitly invoked > by a language construct. I believe you misinterpret the convention. Rather, as I see it, it is: Double-underscores are for methods implicitly invoked by a language construct, and should not be called directly on foreign objects. (So perhaps it's not much of a convention, if we can't agree on it :) The __*__ methods can be seen as a private API. 'next' is not part of it. I see no problem what so ever with the interpreter calling a non-underscored function. I do not understand why this is a problem. I'm -1 on the change. I would be -0 (*maybe* +0) if I thought we had a chance in hell of doing this migration gracefully, but we don't. The translator can't really handle all the cornercases we currently have. The best we can do is *add* a __next__ method to classes that have a next method but no __next__ method, and I simply do not think that's worth it. I am not convinced there is an *actual problem* we are solving, other than that some people are confused about the current situation. Other people would be confused if it was different, so I do not subscribe to the validity of that argument. The single reason I like the builtin next() is the two-argument version, which is quite similar to the two-argument iter(). I would prefer adding it *without* renaming .next to .__next__. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070306/461ddf34/attachment.html From martin at v.loewis.de Tue Mar 6 12:28:38 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 06 Mar 2007 12:28:38 +0100 Subject: [Python-3000] rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <45ED5066.4050601@v.loewis.de> Terry Reedy schrieb: > On the consistency argument: sorted(somelist) calls copy(somelist).sort, > not copy(somelist).__sort__. Actually, it calls list(somelist).sort, so it requires iteration, not copying, and the result will be known to be a list. So the important difference is that .sort is not a method that the argument to sorted needs to implement (indeed, it has no option to define its own sorting algorithm). This is different from __next__, which user-defined types will need to implement. > One of the virtues, and as I recall, design purposes, of .next calls is to > be fast. I think this holds for the C level only. Python-level invocations will create a method-wrapper (or bound method) every time, so that won't be fast. Regards, Martin From martin at v.loewis.de Tue Mar 6 12:40:03 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 06 Mar 2007 12:40:03 +0100 Subject: [Python-3000] Fwd: Re: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070305183514.BAI78440@ms09.lnh.mail.rcn.net> References: <20070305183514.BAI78440@ms09.lnh.mail.rcn.net> Message-ID: <45ED5313.7020708@v.loewis.de> Raymond Hettinger schrieb: >> Invoking a builtin .next() method via a protocol >> function shouldn't be any slower than it is now. >> In both cases you've got one name lookup and one >> Python call, after which you go through the type >> slot directly to a C function. > > Do you expect that next(someiterator) will be just > as fast and calling a boundmethod (such as: > counter=itertools.counter(1); counter())? I would think so, yes. counter will be a method-wrapper, calling it will create an empty argument tuple (or rather pass the singleton one), this will call wrap_next, which will call count_next. A builtin (or otherwise global) next() function will be METH_O, so no argument tuple is created, and tp_iternext is invoked directly. As no memory allocation should happen in either case, and all checks are pretty fast, the difference should be minor, though. Regards, Martin From martin at v.loewis.de Tue Mar 6 12:50:59 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 06 Mar 2007 12:50:59 +0100 Subject: [Python-3000] Exception tracebacks and PEP 3101 In-Reply-To: References: Message-ID: <45ED55A3.8080903@v.loewis.de> Patrick Maupin schrieb: > I'm a newbie when it comes to deep Python internals, so maybe I'm > missing something obvious. I'm trying to figure out the right answer > for exceptions/tracebacks in PEP 3101 (advanced string formatting). You should turn the question then around: "This is what I want to display, how can I achieve that". This seems the tricky part here: What *is* it that you want to display? > However, it seems that the Python traceback scheme is intimately tied > to the concept of Python execution frames, which are assumed to have > associated text information from a file. I'm not sure that I can > legitimately add to a preexisting traceback from a C function in a > non-fragile fashion. Even if I could, I don't really have a "file" to > show the user (unless I dynamically create one). You can certainly add a stack frame into the traceback, see Modules/pyexpat.c (call_with_frame). However, this will get you just a function number, line number, etc: there is no exception object associated with the trace. > So it seems the only answer might have to have options like "let the > user see the existing traceback" or "get rid of the traceback and > raise an exception which shows exactly where in > the format string the error occured", or "print the preexisting > traceback to stderr, then toast it and raise a new exception showing > the string location in the exception text." I still don't understand why you worry about the stack trace. Isn't the exception object what you are after? > I guess there are two parts to my question: > > 1) What is the best thing to do right now for PEP3101? I may be repeating myself, but I think you are looking for some kind of nested exceptions. There is no notion of nested exceptions yet in Python, but you could try to implement some scheme, where the formatting exception has a reference to the "true" exception, and, when printing the formatting exception, the true exception and its traceback get printed. > 2) If the "right now" answer isn't very good, is this really a more > general problem that needs work? As more "little languages" do things > like manipulate the AST, it might become very useful to have the > ability to place almost arbitrary objects on the traceback stack. Same question again. The stack trace seems irrelevant in this problem to me, and so does 'putting objects on it'. Regards, Martin From g.brandl at gmx.net Tue Mar 6 14:41:29 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 06 Mar 2007 14:41:29 +0100 Subject: [Python-3000] The repr() of Ellipsis? Message-ID: Patch http://python.org/sf/1673355 changes Ellipsis' repr() to '...'. While this is consistent, the submitter himself noted that it would probably cause problems with doctest. Additionally, '...' is currently used to denote recursive structures in the repr of lists and dicts, so this would have to be changed too. What do you think? Georg From thomas at python.org Tue Mar 6 14:55:01 2007 From: thomas at python.org (Thomas Wouters) Date: Tue, 6 Mar 2007 14:55:01 +0100 Subject: [Python-3000] The repr() of Ellipsis? In-Reply-To: References: Message-ID: <9e804ac0703060555n50ee5b47necd2dd47a6a7b109@mail.gmail.com> On 3/6/07, Georg Brandl wrote: > > Patch http://python.org/sf/1673355 changes Ellipsis' repr() to '...'. > While > this is consistent, the submitter himself noted that it would probably > cause > problems with doctest. > > Additionally, '...' is currently used to denote recursive structures in > the > repr of lists and dicts, so this would have to be changed too. > > What do you think? I believe it is not worth changing Elipsis' repr. In fact, the problems with ... may indicate that we shouldn't allow '...' everywere after all... But I expect it isn't really a problem (It would be a syntax error when used like in doctests, and in the case of eval()'ing the repr of a dict/list it would otherwise raise a syntax error. The traditional meaning of elipsis is actually correct, there, although it's not how Python programmers are used to dealing with it.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070306/2f333bc7/attachment.htm From pmaupin at gmail.com Tue Mar 6 16:25:49 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Tue, 6 Mar 2007 09:25:49 -0600 Subject: [Python-3000] Exception tracebacks and PEP 3101 In-Reply-To: <45ED55A3.8080903@v.loewis.de> References: <45ED55A3.8080903@v.loewis.de> Message-ID: On 3/6/07, "Martin v. L?wis" wrote: > Patrick Maupin schrieb: > > I'm a newbie when it comes to deep Python internals, so maybe I'm > > missing something obvious. I'm trying to figure out the right answer > > for exceptions/tracebacks in PEP 3101 (advanced string formatting). > > You should turn the question then around: "This is what I want to > display, how can I achieve that". That was a statement. The question comes later :) > This seems the tricky part here: What *is* it that you want to > display? The obvious, greenfield, blue-sky, non-pre-existing-implementation, perfect-world thing to do is to add a record to the stack traceback. This record would be in a different language than Python (it would be in the "format string" language), and would contain location information relevant to that different language that would help the exception system display a snippet of the format string that would be useful to the programmer. Just like the exception and any other stack trace information, this trace record could be printed out to stderr by default, or otherwise manipulated, displayed, analyzed, etc. programmatically, or even ignored by a try/except statement. > > > However, it seems that the Python traceback scheme is intimately tied > > to the concept of Python execution frames, which are assumed to have > > associated text information from a file. I'm not sure that I can > > legitimately add to a preexisting traceback from a C function in a > > non-fragile fashion. Even if I could, I don't really have a "file" to > > show the user (unless I dynamically create one). > > You can certainly add a stack frame into the traceback, see > Modules/pyexpat.c (call_with_frame). However, this will get you just > a function number, line number, etc: there is no exception object > associated with the trace. That's useful information. I might play with this even though it's not exactly what I would like to see (I could dummy up the function name to be something related to the failing lcoation in the string, for example.) > > > So it seems the only answer might have to have options like "let the > > user see the existing traceback" or "get rid of the traceback and > > raise an exception which shows exactly where in > > the format string the error occured", or "print the preexisting > > traceback to stderr, then toast it and raise a new exception showing > > the string location in the exception text." > > I still don't understand why you worry about the stack trace. Isn't > the exception object what you are after? In the current implementation, probably. In the best possible world, certainly not. The exception is whatever it is . That's not my deal. All I really want to do is leave a breadcrumb on how we GOT to the exception to help give a clue to the poor slob who's up at 3:00 AM trying to find that last bug. > > I guess there are two parts to my question: > > > > 1) What is the best thing to do right now for PEP3101? > > I may be repeating myself, but I think you are looking for some kind of > nested exceptions. There is no notion of nested exceptions yet in > Python, but you could try to implement some scheme, where the formatting > exception has a reference to the "true" exception, and, when printing > the formatting exception, the true exception and its traceback get printed. I agree that nested exceptions may be the best I can do in the current implementation, and I can (and probably will) do what you and Talin have independently suggested. > > 2) If the "right now" answer isn't very good, is this really a more > > general problem that needs work? As more "little languages" do things > > like manipulate the AST, it might become very useful to have the > > ability to place almost arbitrary objects on the traceback stack. > > Same question again. The stack trace seems irrelevant in this problem to > me, and so does 'putting objects on it'. Again, the exception is the thing that happened at the bottom. The stack trace is a clue as to how you got there. Nested exceptions are a workaround. The ideal situation would be to have a more flexible stack trace that let you show "how you got there" through execution of code that isn't Python, and even through execution of dynamically generated Python code that has no associated file object (but which may have an associated string object, for example). Regards, Pat From ncoghlan at gmail.com Tue Mar 6 16:46:28 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 07 Mar 2007 01:46:28 +1000 Subject: [Python-3000] List & set comprehensions patch Message-ID: <45ED8CD4.4020905@gmail.com> Georg and I have been working on the implementation of list comprehensions which don't leak their iteration variables, along with the implementation of set comprehensions. The latest patch can be found as SF patch #1660500 [1]. The file new-set-comps.diff is the combined patch which implements both features, and unifies handling of the different kinds of comprehension. In an effort to improve readability, the patch also converts the sets in symtable.c to be actual PySet objects, rather than PyDict objects with None keys and tries to reduce the number of different meanings assigned to the term 'scope'. One of the comments made on Georg's initial attempt at implementing these features was that it would be nice to avoid the function call overhead in the listcomp & setcomp case (as it appears at first glance that the internal scope can be temporary). I tried to do that and essentially failed outright - working through symtable.c and compile.c, I found that dealing with the scoping issues created by the possibility of nested genexps, lambdas and list or set comprehensions would pretty much require reimplementing all of the scoping rules that functions already provide. Here's an example of the scoping issues from the new test_listcomps.py that forms part of the patch: >>> def test_func(): ... items = [(lambda: i) for i in range(5)] ... i = 20 ... return [x() for x in items] >>> test_func() [4, 4, 4, 4, 4] Without creating an actual function object for the body of the list comprehension, it becomes rather difficult to get the lambda expression closure to resolve to the correct value. For list comprehensions at module or class scope, the introduction of the function object can actually lead to a speed increase as the iteration variables and accumulation variable become function locals instead of module globals. Inside a function, however, the additional function call overhead slows things down. Some specific questions related to the current patch: In implementing it, I discovered that list comprehensions don't do SETUP_LOOP/POP_BLOCK around their for loop - I'd like to get confirmation from someone who knows their way around the ceval loop better than I do that omitting those is actually legitimate (I *think* the restriction to a single expression in the body of the comprehension makes it OK, but I'm not sure). There are also a couple of tests we had to disable - one in test_dis, one in test_grammar. Suggestions on how to reinstate those (or agreement that it is OK to get rid of them) would be appreciated. The PySet update code in symtable.c currently uses PyNumber_InplaceOr with a subsequent call to Py_DECREF to counter the implicit call to Py_INCREF. Should this be changed to use PyObject_CallMethod to invoke the Python level update method? There are also two backwards compatibility problems which came up: - code which explicitly deleted the listcomp variable started throwing NameErrors. Several tweaks were needed in the standard library to fix this. - only the outermost iterator expression is evaluated in the scope containing the comprehension (just like generator expressions). This means that the inner expressions can no longer see class variables and values in explicit locals() dictionaries provided to exec & friends. This didn't actually cause any problems in the standard library - I only note it because my initial implementation mistakenly evaluated the outermost iterator in the new scope, which *did* cause severe problems along these lines. Regards, Nick [1] http://www.python.org/sf/1660500 -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jimjjewett at gmail.com Tue Mar 6 18:33:41 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 6 Mar 2007 12:33:41 -0500 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <20070305110807.7489.JCARLSON@uci.edu> <20070305144950.7491.JCARLSON@uci.edu> Message-ID: On 3/5/07, Ka-Ping Yee wrote: > In fact, your example was specifically anticipated and addressed in > the PEP draft I posted here. file.read() is not invoked by a language > construct. When file.read() gets called, it is because the calling > code has an explicit call to read() in it, not because the standard > semantics of some piece of Python syntax require it to be invoked. Except that the "explicit" call could just be "for line in file", which doesn't mention read. And there are warnings not to mix explicit reads with iteration reads, because buffering will cause surprises. I think the inconsistency is in iteration rather than reading, but file.read is affected. -jJ From g.brandl at gmx.net Tue Mar 6 18:38:27 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 06 Mar 2007 18:38:27 +0100 Subject: [Python-3000] List & set comprehensions patch In-Reply-To: <45ED8CD4.4020905@gmail.com> References: <45ED8CD4.4020905@gmail.com> Message-ID: Nick Coghlan schrieb: > Georg and I have been working on the implementation of list > comprehensions which don't leak their iteration variables, along with > the implementation of set comprehensions. The latest patch can be found > as SF patch #1660500 [1]. The file new-set-comps.diff is the combined > patch which implements both features, and unifies handling of the > different kinds of comprehension. In an effort to improve readability, > the patch also converts the sets in symtable.c to be actual PySet > objects, rather than PyDict objects with None keys and tries to reduce > the number of different meanings assigned to the term 'scope'. > > One of the comments made on Georg's initial attempt at implementing > these features was that it would be nice to avoid the function call > overhead in the listcomp & setcomp case (as it appears at first glance > that the internal scope can be temporary). I tried to do that and > essentially failed outright - working through symtable.c and compile.c, > I found that dealing with the scoping issues created by the possibility > of nested genexps, lambdas and list or set comprehensions would pretty > much require reimplementing all of the scoping rules that functions > already provide. I have to thank you for digging through all that code and cases, which I always dreaded ;) > There are also a couple of tests we had to disable - one in test_dis, > one in test_grammar. Suggestions on how to reinstate those (or agreement > that it is OK to get rid of them) would be appreciated. The one in test_grammar is certainly okay, since it uses syntax which was agreed upon to be invalid in Py3k. > The PySet update code in symtable.c currently uses PyNumber_InplaceOr > with a subsequent call to Py_DECREF to counter the implicit call to > Py_INCREF. Should this be changed to use PyObject_CallMethod to invoke > the Python level update method? Why not add a C-level PySet_Update API? > There are also two backwards compatibility problems which came up: > > - code which explicitly deleted the listcomp variable started > throwing NameErrors. Several tweaks were needed in the standard library > to fix this. This may be hard to catch with the 2to3 tool, but perhaps a certain heuristics (del statement after a statement involving a list comprehension) could apply. Furthermore, I don't think that "trick" is mostly used on module-level to avoid cluttering up the namespace, so a codebase which really uses its modules should see early failures. > - only the outermost iterator expression is evaluated in the scope > containing the comprehension (just like generator expressions). This > means that the inner expressions can no longer see class variables and > values in explicit locals() dictionaries provided to exec & friends. > This didn't actually cause any problems in the standard library - I only > note it because my initial implementation mistakenly evaluated the > outermost iterator in the new scope, which *did* cause severe problems > along these lines. It must be noted that this was always the case for generator expressions, so if a LC should only be syntactic sugar for list(genexp) as written in the PEP, this is even correct behavior. cheers, Georg From collinw at gmail.com Tue Mar 6 18:43:26 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 6 Mar 2007 11:43:26 -0600 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <43aa6ff70703050828g3a720efeu6dd172af7246a43d@mail.gmail.com> References: <43aa6ff70703050828g3a720efeu6dd172af7246a43d@mail.gmail.com> Message-ID: <43aa6ff70703060943q7d51ad5dmfacd26d52859ca5c@mail.gmail.com> On 3/5/07, Collin Winter wrote: > On 3/5/07, Ka-Ping Yee wrote: > [snip] > > Transition Plan > > =============== > > > > Two additional transformations will be added to the 2to3 translation > > tool [2]_: > > > > * Method definitions named ``next`` will be renamed to ``__next__``. > > > > * Explicit calls to the ``next`` method will be replaced with calls > > to the built-in ``next`` function. For example, ``x.next()`` will > > become ``next(x)``. > > > > If the module being processed already contains a top-level function > > definition named ``next``, the second transformation will not be done; > > instead, calls to ``x.next()`` will be replaced with ``x.__next__()`` > > and a warning will be emitted. > > The basic transformations for this are (relatively) easy to implement. > Basing the second transformation on the absence of a top-level > function definition (don't forget assignments and imports!) for > "next", though, is going to require some surgery to 2to3's internals. > I'll see what I can do. Preliminary results: After playing around with 2to3 for a while, it doesn't seem possible to base the second transformation on the presence of a top-level "next" binding. I can detect the binding's presence and warn about it, but just not in time to impact the "it.next()" -> "next(it)" transformation. I'm tinkering with an implementation that defers the transformation until 2to3 has finished walking the entire tree, but it's a lot of clutter and hackery for something like this. Collin Winter From guido at python.org Tue Mar 6 19:01:09 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 10:01:09 -0800 Subject: [Python-3000] The repr() of Ellipsis? In-Reply-To: <9e804ac0703060555n50ee5b47necd2dd47a6a7b109@mail.gmail.com> References: <9e804ac0703060555n50ee5b47necd2dd47a6a7b109@mail.gmail.com> Message-ID: I've rejected it. On 3/6/07, Thomas Wouters wrote: > > > On 3/6/07, Georg Brandl wrote: > > Patch http://python.org/sf/1673355 changes Ellipsis' repr() to '...'. > While > > this is consistent, the submitter himself noted that it would probably > cause > > problems with doctest. > > > > Additionally, '...' is currently used to denote recursive structures in > the > > repr of lists and dicts, so this would have to be changed too. > > > > What do you think? > > I believe it is not worth changing Elipsis' repr. In fact, the problems with > ... may indicate that we shouldn't allow '...' everywere after all... But I > expect it isn't really a problem (It would be a syntax error when used like > in doctests, and in the case of eval()'ing the repr of a dict/list it would > otherwise raise a syntax error. The traditional meaning of elipsis is > actually correct, there, although it's not how Python programmers are used > to dealing with it.) > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me > spread! > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Tue Mar 6 19:03:24 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 06 Mar 2007 19:03:24 +0100 Subject: [Python-3000] locals(), closures, and IronPython... In-Reply-To: <20070306155230.GD4528@panix.com> References: <7AD436E4270DD54A94238001769C22276A624C7482@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20070306155230.GD4528@panix.com> Message-ID: Aahz schrieb: > On Tue, Mar 06, 2007, Andrew Dalke wrote: >> On 3/5/07, Guido van Rossum wrote: >>> >>> I don't know too many good use cases for >>> locals() apart from "learning about the implementation" I think this >>> might be okay. >> >> Since I'm watching this list for any discussion on the traceback >> threads, I figured I would point out the most common use I know >> for locals() is in string interpolation when there are many local >> variables, eg: >> >> a = "spam" >> b = "egg" >> ... >> y = "foo" >> z = "bar" >> >> print fmtstr % locals() > > I'll second this one. While we're at it: I've had a thought about string formatting in Py3k. Suppose you do something like name = "He" what = "Ex-Parrot" print "{name} is an {what}".format(name=name, what=what) it seems a bit too verbose. Why not have format() without any arguments default to format(**locals())? This would give those Perl-style interpolation supporters something they can live with, and it would be quite handy for common usage. Georg From martin at v.loewis.de Tue Mar 6 19:07:29 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 06 Mar 2007 19:07:29 +0100 Subject: [Python-3000] Exception tracebacks and PEP 3101 In-Reply-To: References: <45ED55A3.8080903@v.loewis.de> Message-ID: <45EDADE1.4030404@v.loewis.de> Patrick Maupin schrieb: > That was a statement. The question comes later :) > >> This seems the tricky part here: What *is* it that you want to >> display? > > The obvious, greenfield, blue-sky, non-pre-existing-implementation, > perfect-world thing to do is to add a record to the stack traceback. I can't see how adding a record *alone* helps anything whatsoever. Wouldn't you also need to *display* the record somehow? So lets start with that (and all over again): What is it (precisely) that you want to display? Given what program, what error, what should the output be? Or, if there is not to be any output at any time: what is the purpose of the record? Regards, Martin From steven.bethard at gmail.com Tue Mar 6 19:09:09 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 6 Mar 2007 11:09:09 -0700 Subject: [Python-3000] string formatting and locals() (WAS: locals(), closures, and IronPython...) Message-ID: On 3/6/07, Georg Brandl wrote: > While we're at it: I've had a thought about string formatting in Py3k. > Suppose you do something like > > name = "He" > what = "Ex-Parrot" > print "{name} is an {what}".format(name=name, what=what) > > it seems a bit too verbose. Why not have format() without any arguments > default to format(**locals())? Since format() is just a normal string method, isn't that going to require some frame hackery? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Tue Mar 6 19:18:18 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 10:18:18 -0800 Subject: [Python-3000] List & set comprehensions patch In-Reply-To: <45ED8CD4.4020905@gmail.com> References: <45ED8CD4.4020905@gmail.com> Message-ID: Quick responses from just reading the email (I'll try to review the code later today, I'm trying to do Py3k work all day): On 3/6/07, Nick Coghlan wrote: > Georg and I have been working on the implementation of list > comprehensions which don't leak their iteration variables, along with > the implementation of set comprehensions. The latest patch can be found > as SF patch #1660500 [1]. The file new-set-comps.diff is the combined > patch which implements both features, and unifies handling of the > different kinds of comprehension. In an effort to improve readability, > the patch also converts the sets in symtable.c to be actual PySet > objects, rather than PyDict objects with None keys and tries to reduce > the number of different meanings assigned to the term 'scope'. Maybe you could separate that out into a separate patch so it won't old up review or work on the main patch? Or is there a stronger connection? > One of the comments made on Georg's initial attempt at implementing > these features was that it would be nice to avoid the function call > overhead in the listcomp & setcomp case (as it appears at first glance > that the internal scope can be temporary). I tried to do that and > essentially failed outright - working through symtable.c and compile.c, > I found that dealing with the scoping issues created by the possibility > of nested genexps, lambdas and list or set comprehensions would pretty > much require reimplementing all of the scoping rules that functions > already provide. How about determining if it's a *simple* case or not, and doing the variable renaming in the simple case and Georg's original version in non-simple cases? You can define "simple" as whatever makes the determination easy and still treats most common cases as simple. E.g. a lambda would be a non-simple case, and so would using a nonlocal or global variable (note though that nonlocal and global should reach inside the list/set comp!) etc. > Here's an example of the scoping issues from the new test_listcomps.py > that forms part of the patch: > > >>> def test_func(): > ... items = [(lambda: i) for i in range(5)] > ... i = 20 > ... return [x() for x in items] > >>> test_func() > [4, 4, 4, 4, 4] > > Without creating an actual function object for the body of the list > comprehension, it becomes rather difficult to get the lambda expression > closure to resolve to the correct value. > > For list comprehensions at module or class scope, the introduction of > the function object can actually lead to a speed increase as the > iteration variables and accumulation variable become function locals > instead of module globals. Inside a function, however, the additional > function call overhead slows things down. > > Some specific questions related to the current patch: > > In implementing it, I discovered that list comprehensions don't do > SETUP_LOOP/POP_BLOCK around their for loop - I'd like to get > confirmation from someone who knows their way around the ceval loop > better than I do that omitting those is actually legitimate (I *think* > the restriction to a single expression in the body of the comprehension > makes it OK, but I'm not sure). They exist to handle break/continue. Since those don't apply to list/set comps, it's safe. > There are also a couple of tests we had to disable - one in test_dis, > one in test_grammar. Suggestions on how to reinstate those (or agreement > that it is OK to get rid of them) would be appreciated. I'll have to look later. > The PySet update code in symtable.c currently uses PyNumber_InplaceOr > with a subsequent call to Py_DECREF to counter the implicit call to > Py_INCREF. Should this be changed to use PyObject_CallMethod to invoke > the Python level update method? What's wrong with the inplace or? I seem to recall that s |= x and s.update(x) aren't equivalent if x is not a set. > There are also two backwards compatibility problems which came up: > > - code which explicitly deleted the listcomp variable started > throwing NameErrors. Several tweaks were needed in the standard library > to fix this. That's fine. I think it's okay to have this kind of problem. > - only the outermost iterator expression is evaluated in the scope > containing the comprehension (just like generator expressions). This > means that the inner expressions can no longer see class variables and > values in explicit locals() dictionaries provided to exec & friends. > This didn't actually cause any problems in the standard library - I only > note it because my initial implementation mistakenly evaluated the > outermost iterator in the new scope, which *did* cause severe problems > along these lines. This smells fishy. Do you have an example? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 6 19:20:41 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 10:20:41 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <45EC9D7E.6000702@canterbury.ac.nz> References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On 3/5/07, Greg Ewing wrote: > Ka-Ping Yee wrote: > > > Just like getattr, two-argument next(iter, sentinel) > > returns sentinel if StopException is caught. > > +1. I've written a number of pieces of code where this > would have made things neater. Just about any place > where I've used .next() explicitly, in fact -- it > always seems awkward having to deal with StopIteration. Ditto. Also notice the (in some sense ironic) complementary symmetry with iter(function, sentinel). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 6 19:24:50 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 10:24:50 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: Having now read this entire thread I am going to accept Ping's PEP. Adding the sentinel argument to the next() builtin was what did it for me: it neatly solves the problem if having to catch that StopIteration in 99% of the cases. Thanks all for the discussion! --Guido On 3/6/07, Guido van Rossum wrote: > On 3/5/07, Greg Ewing wrote: > > Ka-Ping Yee wrote: > > > > > Just like getattr, two-argument next(iter, sentinel) > > > returns sentinel if StopException is caught. > > > > +1. I've written a number of pieces of code where this > > would have made things neater. Just about any place > > where I've used .next() explicitly, in fact -- it > > always seems awkward having to deal with StopIteration. > > Ditto. > > Also notice the (in some sense ironic) complementary symmetry with > iter(function, sentinel). > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 6 19:32:59 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 10:32:59 -0800 Subject: [Python-3000] locals(), closures, and IronPython... In-Reply-To: References: <7AD436E4270DD54A94238001769C22276A624C7482@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20070306155230.GD4528@panix.com> Message-ID: On 3/6/07, Georg Brandl wrote: > While we're at it: I've had a thought about string formatting in Py3k. > Suppose you do something like > > name = "He" > what = "Ex-Parrot" > print "{name} is an {what}".format(name=name, what=what) > > it seems a bit too verbose. Why not have format() without any arguments > default to format(**locals())? > > This would give those Perl-style interpolation supporters something they > can live with, and it would be quite handy for common usage. I think that's a little too terse. **locals() sounds just right to me; EIBTI. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 6 19:37:14 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 10:37:14 -0800 Subject: [Python-3000] Exception tracebacks and PEP 3101 In-Reply-To: <45EDADE1.4030404@v.loewis.de> References: <45ED55A3.8080903@v.loewis.de> <45EDADE1.4030404@v.loewis.de> Message-ID: Patrick, I would recommend not to worry about that just yet. As a first cut it's quite alright if a traceback in the formatter is just passed through by the format() call without adding any indication of where in the format string it happened. There are tons of ways to debug that. --Guido On 3/6/07, "Martin v. L?wis" wrote: > Patrick Maupin schrieb: > > That was a statement. The question comes later :) > > > >> This seems the tricky part here: What *is* it that you want to > >> display? > > > > The obvious, greenfield, blue-sky, non-pre-existing-implementation, > > perfect-world thing to do is to add a record to the stack traceback. > > I can't see how adding a record *alone* helps anything whatsoever. > Wouldn't you also need to *display* the record somehow? > > So lets start with that (and all over again): What is it (precisely) > that you want to display? Given what program, what error, what should > the output be? > > Or, if there is not to be any output at any time: what is the purpose > of the record? > > Regards, > Martin > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Tue Mar 6 21:05:13 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 06 Mar 2007 12:05:13 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <20070306115033.74A7.JCARLSON@uci.edu> "Guido van Rossum" wrote: > Having now read this entire thread I am going to accept Ping's PEP. > Adding the sentinel argument to the next() builtin was what did it for > me: it neatly solves the problem if having to catch that StopIteration > in 99% of the cases. Have you read the post by Thomas Wouters in regards to .next() vs. .__next__() [1]? The idea is that methods that *shouldn't* be called from user code explicitly have generally received __magic__ methods, while methods that have uses-cases for being called directly get nonmagic methods. He (and I believe most everyone) is in favor of some two-argument builtin or otherwise named next(interator, default=), it's the bulk renaming of .next -> .__next__ that is the real point of contention (with non-iterator .next methods, .next instance variables, etc.) - Josiah [1] http://mail.python.org/pipermail/python-3000/2007-March/006010.html From pmaupin at gmail.com Tue Mar 6 21:02:09 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Tue, 6 Mar 2007 14:02:09 -0600 Subject: [Python-3000] string formatting and locals() (WAS: locals(), closures, and IronPython...) In-Reply-To: References: Message-ID: Actually, the version checked in to http://svn.python.org/projects/sandbox/trunk/pep3101 currently will search both locals() and globals() if no parameters are passed to format. It's still a work-in-progress, but has quite a few passing tests, and builds as an extension module on 2.3, 2.4, and 3.0 (probably also 2.5, haven't tried it). The file pep_differences.txt shows the current differences between that and the original PEP. Regards, Pat On 3/6/07, Steven Bethard wrote: > On 3/6/07, Georg Brandl wrote: > > While we're at it: I've had a thought about string formatting in Py3k. > > Suppose you do something like > > > > name = "He" > > what = "Ex-Parrot" > > print "{name} is an {what}".format(name=name, what=what) > > > > it seems a bit too verbose. Why not have format() without any arguments > > default to format(**locals())? > > Since format() is just a normal string method, isn't that going to > require some frame hackery? > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/pmaupin%40gmail.com > From pmaupin at gmail.com Tue Mar 6 21:20:06 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Tue, 6 Mar 2007 14:20:06 -0600 Subject: [Python-3000] locals(), closures, and IronPython... In-Reply-To: References: <7AD436E4270DD54A94238001769C22276A624C7482@DF-GRTDANE-MSG.exchange.corp.microsoft.com> <20070306155230.GD4528@panix.com> Message-ID: EIBTI, it's true, but in some cases it's awfully handy to be implicit. Ian Bicking and I were tossing this around, and came up with some ideas, all of which are certainly flawed in one way or another :) One idea is that an additional string formatting method which will automatically work with locals() and globals() could be named something like "eval". On first glance, this seems like a bad name because of the builtin eval() function, but perhaps it would be a good thing. People are warned off using eval() on strings they didn't generate, and the same should be true of a string eval() method, for the same reasons. (Or as a few of us have noted: "eval" == "evil") In any case, the initial C implementation of PEP 3101 automagically does this if format() has no parameters, but that is easy enough to rip out. Regards, Pat On 3/6/07, Guido van Rossum wrote: > On 3/6/07, Georg Brandl wrote: > > While we're at it: I've had a thought about string formatting in Py3k. > > Suppose you do something like > > > > name = "He" > > what = "Ex-Parrot" > > print "{name} is an {what}".format(name=name, what=what) > > > > it seems a bit too verbose. Why not have format() without any arguments > > default to format(**locals())? > > > > This would give those Perl-style interpolation supporters something they > > can live with, and it would be quite handy for common usage. > > I think that's a little too terse. **locals() sounds just right to me; EIBTI. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/pmaupin%40gmail.com > From guido at python.org Tue Mar 6 21:21:10 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 12:21:10 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: <20070306115033.74A7.JCARLSON@uci.edu> References: <20070306115033.74A7.JCARLSON@uci.edu> Message-ID: I don't see much point of adding the builtin if we don't rename the method to __next__, since that would just make the wart stick out more. The conversion pain is a one-time cost. We can work with 2to3 and Python 2.6 warnings on the conversions. Hey, if we can "from __future__ import dict_views" then I think we can "from __future__ import next"! Even if the converter can't do a perfect job, it should be possible to write 2.6 code on which the converter *does* do a perfect job, simply by avoiding a few odd corner cases. --Guido On 3/6/07, Josiah Carlson wrote: > > "Guido van Rossum" wrote: > > Having now read this entire thread I am going to accept Ping's PEP. > > Adding the sentinel argument to the next() builtin was what did it for > > me: it neatly solves the problem if having to catch that StopIteration > > in 99% of the cases. > > Have you read the post by Thomas Wouters in regards to .next() vs. > .__next__() [1]? The idea is that methods that *shouldn't* be called > from user code explicitly have generally received __magic__ methods, > while methods that have uses-cases for being called directly get > nonmagic methods. > > He (and I believe most everyone) is in favor of some two-argument > builtin or otherwise named next(interator, default=), it's the bulk > renaming of .next -> .__next__ that is the real point of contention > (with non-iterator .next methods, .next instance variables, etc.) > > - Josiah > > > [1] http://mail.python.org/pipermail/python-3000/2007-March/006010.html > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Tue Mar 6 21:49:20 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 06 Mar 2007 21:49:20 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: Guido van Rossum schrieb: > Having now read this entire thread I am going to accept Ping's PEP. > Adding the sentinel argument to the next() builtin was what did it for > me: it neatly solves the problem if having to catch that StopIteration > in 99% of the cases. Attached is a possible implementation for the next() function, regardless of how the next method is going to be called in the future. Georg -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: next.diff Url: http://mail.python.org/pipermail/python-3000/attachments/20070306/cce9ec49/attachment.diff From brett at python.org Tue Mar 6 21:54:19 2007 From: brett at python.org (Brett Cannon) Date: Tue, 6 Mar 2007 12:54:19 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On 3/6/07, Georg Brandl wrote: > Guido van Rossum schrieb: > > Having now read this entire thread I am going to accept Ping's PEP. > > Adding the sentinel argument to the next() builtin was what did it for > > me: it neatly solves the problem if having to catch that StopIteration > > in 99% of the cases. > > Attached is a possible implementation for the next() function, regardless > of how the next method is going to be called in the future. > > Georg > > > Index: Python/bltinmodule.c > =================================================================== > --- Python/bltinmodule.c (Revision 54016) > +++ Python/bltinmodule.c (Arbeitskopie) > @@ -1029,6 +1029,45 @@ > > > static PyObject * > +builtin_next(PyObject *self, PyObject *args) > +{ > + PyObject *it, *res; > + PyObject *def = NULL; > + > + if (!PyArg_UnpackTuple(args, "next", 1, 2, &it, &def)) > + return NULL; > + if (!PyIter_Check(it)) { > + PyErr_Format(PyExc_TypeError, > + "%.200s object is not an iterator", it->ob_type->tp_name); > + return NULL; > + } > + > + res = (*it->ob_type->tp_iternext)(it); > + if (res == NULL) { > + if (def) { > + if (PyErr_Occurred() && > + !PyErr_ExceptionMatches(PyExc_StopIteration)) > + return NULL; > + Py_INCREF(def); > + return def; > + } else if (PyErr_Occurred()) { > + return NULL; > + } else { > + PyErr_SetNone(PyExc_StopIteration); > + return NULL; > + } > + } > + return res; > +} http://docs.python.org/dev/api/type-structs.html#l2h-1017 doesn't say that a NULL return can be used as a signal that iterator is exhausted without raising StopIteration. That would mean the above could be simplified somewhat in terms of return value checking. -Brett From daniel at stutzbachenterprises.com Tue Mar 6 22:03:11 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 6 Mar 2007 15:03:11 -0600 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On 3/6/07, Brett Cannon wrote: > http://docs.python.org/dev/api/type-structs.html#l2h-1017 doesn't say > that a NULL return can be used as a signal that iterator is exhausted > without raising StopIteration. That would mean the above could be > simplified somewhat in terms of return value checking. FWIW, PEP 234 states that the end of iteration can also be signaled by returning NULL without an exception set. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Tue Mar 6 22:50:09 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 13:50:09 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On 3/6/07, Brett Cannon wrote: > On 3/6/07, Georg Brandl wrote: > > Guido van Rossum schrieb: > > > Having now read this entire thread I am going to accept Ping's PEP. > > > Adding the sentinel argument to the next() builtin was what did it for > > > me: it neatly solves the problem if having to catch that StopIteration > > > in 99% of the cases. > > > > Attached is a possible implementation for the next() function, regardless > > of how the next method is going to be called in the future. > > > > Georg > > > > > > Index: Python/bltinmodule.c > > =================================================================== > > --- Python/bltinmodule.c (Revision 54016) > > +++ Python/bltinmodule.c (Arbeitskopie) > > @@ -1029,6 +1029,45 @@ > > > > > > static PyObject * > > +builtin_next(PyObject *self, PyObject *args) > > +{ > > + PyObject *it, *res; > > + PyObject *def = NULL; > > + > > + if (!PyArg_UnpackTuple(args, "next", 1, 2, &it, &def)) > > + return NULL; > > + if (!PyIter_Check(it)) { > > + PyErr_Format(PyExc_TypeError, > > + "%.200s object is not an iterator", it->ob_type->tp_name); > > + return NULL; > > + } > > + > > + res = (*it->ob_type->tp_iternext)(it); > > + if (res == NULL) { > > + if (def) { > > + if (PyErr_Occurred() && > > + !PyErr_ExceptionMatches(PyExc_StopIteration)) > > + return NULL; > > + Py_INCREF(def); > > + return def; > > + } else if (PyErr_Occurred()) { > > + return NULL; > > + } else { > > + PyErr_SetNone(PyExc_StopIteration); > > + return NULL; > > + } > > + } > > + return res; > > +} > > http://docs.python.org/dev/api/type-structs.html#l2h-1017 doesn't say > that a NULL return can be used as a signal that iterator is exhausted > without raising StopIteration. That would mean the above could be > simplified somewhat in terms of return value checking. But in practice a NULL without an error is often used (and was intended to be valid). The code should clear the exception though if it *is* set and def is returned. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Tue Mar 6 23:15:56 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Mar 2007 11:15:56 +1300 Subject: [Python-3000] Custom traceback entries (Exception tracebacks and PEP 3101) In-Reply-To: References: Message-ID: <45EDE81C.8000208@canterbury.ac.nz> Patrick Maupin wrote: > I'm not sure that I can > legitimately add to a preexisting traceback from a C function in a > non-fragile fashion. Pyrex does this by creating a fake stack frame and filling most of it in with dummy values. It can be done, but it's ugly. Let me know if you're interested and I'll send you the code. I'd like a better way to do this, too. Seems to me the traceback object ought to have an abstract interface for getting the info needed to print the traceback. Then you could easily put custom entries in the traceback showing whatever you wanted. -- Greg From guido at python.org Wed Mar 7 00:15:40 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 15:15:40 -0800 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: Reading this and all the other discussion on the proper semantics for non-blocking I/O I think I may have overreached in trying to support non-blocking I/O at all levels of the new I/O stack. There probably aren't enough use cases for wanting to support readline() returning None if no full line if input is available yet to warrant the additional complexities -- and I haven't even looked very carefully at incremental codecs, which introduce another (small) buffer. I think maybe a useful simplification would be to support special return values to capture EWOULDBLOCK (or equivalent) in the raw I/O interface only. I think it serves a purpose here, since without such support, code doing raw I/O would either require catching IOError all the time and inspecting it for EWOULDBLOCK (or other platform specific values!), or not using the raw I/O interface at all, requiring yet another interface for raw non-blocking I/O. The buffering layer could then raise IOError (or perhaps a special subclass of it) if the raw I/O layer ever returned one of these; e.g. if a buffered read needs to go to the raw layer to satisfy a request and the raw read returns None, then the buffered read needs to raise this error if no data has been taken out of the buffer yet; or it should return a short read if some data was already consumed (since it's hard to "unconsume" data, especially if the requested read length is larger than the buffer size, or if there's an incremental encoder involved). Thus, applications can assume that a short read means either EOF or nonblocking I/O; most apps can safely ignore the latter since it must be explicitly be turned on by the app. For writing, if the buffering layer receives a short write, it should try again; but if it receives an EWOULDBLOCK, it should likewise raise the abovementioned error, since repeated attempts to write in this case would just end up spinning the CPU without making progress. (We should not raise an error if a single short write happens, since AFAIK this is possible for TCP sockets even in blocking mode, witness the addition of the sendall() method.) This means that the buffering layer that sits directly on top of the raw layer must still be prepared to deal with the special return values from non-blocking I/O, but its API to the next layer up doesn't need special return values, since it turns these into IOErrors, and the next layer(s) up won't have to deal with it nor reflect it in their API. Would this satisfy the critics of the current design? --Guido On 3/4/07, Adam Olsen wrote: > On 3/4/07, Greg Ewing wrote: > > I'm having trouble seeing what the use case is for > > the buffered non-blocking writes being discussed here. > > > > Doing asynchronous I/O usually *doesn't* involve > > putting the file descriptor into non-blocking mode. > > Instead you use select() or equivalent, and only > > try to read or write when the file is reported as > > being ready. > > I can't say which is more common, but non-blocking has a safer feel. > Normal code would be select-driven in both, but if you screw up with > non-blocking you get an error, whereas blocking you get a mysterious > hang. > > accept() is the exception. It's possible for a connection to > disappear between the time select() returns and the time you call > accept(), so you need to be non-blocking to avoid hanging. > > > > > For this to work properly, the select() needs to > > operate at the *bottom* of the I/O stack. Any > > buffering layers sit above that, with requests for > > data propagating up the stack as the file becomes > > ready. > > > > In other words, the whole thing has to have the > > control flow inverted and work in "pull" mode > > rather than "push" mode. It's hard to see how this > > could fit into the model as a minor variation on > > how writes are done. > > Meaning it needs to be a distinct interface and explicitly designed as such. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python-dev at zesty.ca Wed Mar 7 00:29:17 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Tue, 6 Mar 2007 17:29:17 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On Tue, 6 Mar 2007, Guido van Rossum wrote: > Having now read this entire thread I am going to accept Ping's PEP. > Adding the sentinel argument to the next() builtin was what did it for > me: it neatly solves the problem if having to catch that StopIteration > in 99% of the cases. Okay, this is checked in as PEP 3114. -- ?!ng From greg.ewing at canterbury.ac.nz Wed Mar 7 00:53:52 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Mar 2007 12:53:52 +1300 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: <45EDFF10.8030402@canterbury.ac.nz> Ka-Ping Yee wrote: > Okay, this is checked in as PEP 3114. Not 3456? :-( -- Greg From g.brandl at gmx.net Wed Mar 7 01:21:57 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Mar 2007 01:21:57 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: Ka-Ping Yee schrieb: > On Tue, 6 Mar 2007, Guido van Rossum wrote: >> Having now read this entire thread I am going to accept Ping's PEP. >> Adding the sentinel argument to the next() builtin was what did it for >> me: it neatly solves the problem if having to catch that StopIteration >> in 99% of the cases. > > Okay, this is checked in as PEP 3114. Patch is at http://python.org/sf/1675363. Georg From python at zesty.ca Wed Mar 7 01:36:40 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Tue, 6 Mar 2007 18:36:40 -0600 (CST) Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On Wed, 7 Mar 2007, Georg Brandl wrote: > Ka-Ping Yee schrieb: > > On Tue, 6 Mar 2007, Guido van Rossum wrote: > >> Having now read this entire thread I am going to accept Ping's PEP. > >> Adding the sentinel argument to the next() builtin was what did it for > >> me: it neatly solves the problem if having to catch that StopIteration > >> in 99% of the cases. > > > > Okay, this is checked in as PEP 3114. > > Patch is at http://python.org/sf/1675363. Thanks for doing this work! -- ?!ng From collinw at gmail.com Wed Mar 7 01:43:35 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 6 Mar 2007 18:43:35 -0600 Subject: [Python-3000] PEP 3113 transition plan Message-ID: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> >From the checked-in version, """ Two additional transformations will be added to the 2to3 translation tool [3]: * Method definitions named next will be renamed to __next__. * Explicit calls to the next method will be replaced with calls to the built-in next function. For example, x.next() will become next(x). If the module being processed already contains a binding for the name next, the second transformation will not be done; instead, calls to x.next() will be replaced with x.__next__() and a warning will be emitted. (Collin Winter has looked into this [4] and found that it's difficult to make the second transformation depend on the presence of a module-level binding; warning about the existence of bindings to next should be possible, though.) """ I'd like to strike the part about making "x.next()" -> "next(x)"/"x.__next__()" determined by the presence of a module-level "next" binding. I'd rather see the transformation always be "x.next()" -> "next(x)" and warn on top-level "next"s. Doing it the way the PEP suggests is possible, but ugly (requires an unusually-complicated fixer) and slow. I can give you warnings on the following items, though: - global assignments to "next". - global definitions of a "next" function. - global imports of anything named "next". - assignments to "__builtin__.next". Will that do? Collin Winter From daniel at stutzbachenterprises.com Wed Mar 7 01:44:37 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 6 Mar 2007 18:44:37 -0600 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: +1 I liked the original design at first, but after fleshing it and out and seeing all of the corner cases raised here, the Buffered I/O interface for non-blocking would have to be so different that it would not make much sense to make it the same type. The raw I/O layer semantics are unchanged, right? (.read() returns 0 bytes on EOF, None on an EAGAIN/EWOULDBLOCK condition, raise IOError on any other problem) On 3/6/07, Guido van Rossum wrote: > Reading this and all the other discussion on the proper semantics for > non-blocking I/O I think I may have overreached in trying to support > non-blocking I/O at all levels of the new I/O stack. There probably > aren't enough use cases for wanting to support readline() returning > None if no full line if input is available yet to warrant the > additional complexities -- and I haven't even looked very carefully at > incremental codecs, which introduce another (small) buffer. > > I think maybe a useful simplification would be to support special > return values to capture EWOULDBLOCK (or equivalent) in the raw I/O > interface only. I think it serves a purpose here, since without such > support, code doing raw I/O would either require catching IOError all > the time and inspecting it for EWOULDBLOCK (or other platform specific > values!), or not using the raw I/O interface at all, requiring yet > another interface for raw non-blocking I/O. > > The buffering layer could then raise IOError (or perhaps a special > subclass of it) if the raw I/O layer ever returned one of these; e.g. > if a buffered read needs to go to the raw layer to satisfy a request > and the raw read returns None, then the buffered read needs to raise > this error if no data has been taken out of the buffer yet; or it > should return a short read if some data was already consumed (since > it's hard to "unconsume" data, especially if the requested read length > is larger than the buffer size, or if there's an incremental encoder > involved). Thus, applications can assume that a short read means > either EOF or nonblocking I/O; most apps can safely ignore the latter > since it must be explicitly be turned on by the app. > > For writing, if the buffering layer receives a short write, it should > try again; but if it receives an EWOULDBLOCK, it should likewise raise > the abovementioned error, since repeated attempts to write in this > case would just end up spinning the CPU without making progress. (We > should not raise an error if a single short write happens, since AFAIK > this is possible for TCP sockets even in blocking mode, witness the > addition of the sendall() method.) > > This means that the buffering layer that sits directly on top of the > raw layer must still be prepared to deal with the special return > values from non-blocking I/O, but its API to the next layer up doesn't > need special return values, since it turns these into IOErrors, and > the next layer(s) up won't have to deal with it nor reflect it in > their API. > > Would this satisfy the critics of the current design? > > --Guido > > On 3/4/07, Adam Olsen wrote: > > On 3/4/07, Greg Ewing wrote: > > > I'm having trouble seeing what the use case is for > > > the buffered non-blocking writes being discussed here. > > > > > > Doing asynchronous I/O usually *doesn't* involve > > > putting the file descriptor into non-blocking mode. > > > Instead you use select() or equivalent, and only > > > try to read or write when the file is reported as > > > being ready. > > > > I can't say which is more common, but non-blocking has a safer feel. > > Normal code would be select-driven in both, but if you screw up with > > non-blocking you get an error, whereas blocking you get a mysterious > > hang. > > > > accept() is the exception. It's possible for a connection to > > disappear between the time select() returns and the time you call > > accept(), so you need to be non-blocking to avoid hanging. > > > > > > > > For this to work properly, the select() needs to > > > operate at the *bottom* of the I/O stack. Any > > > buffering layers sit above that, with requests for > > > data propagating up the stack as the file becomes > > > ready. > > > > > > In other words, the whole thing has to have the > > > control flow inverted and work in "pull" mode > > > rather than "push" mode. It's hard to see how this > > > could fit into the model as a minor variation on > > > how writes are done. > > > > Meaning it needs to be a distinct interface and explicitly designed as such. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/daniel%40stutzbachenterprises.com > -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Wed Mar 7 01:57:54 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 16:57:54 -0800 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/6/07, Daniel Stutzbach wrote: > +1 > > I liked the original design at first, but after fleshing it and out > and seeing all of the corner cases raised here, the Buffered I/O > interface for non-blocking would have to be so different that it would > not make much sense to make it the same type. > > The raw I/O layer semantics are unchanged, right? (.read() returns 0 > bytes on EOF, None on an EAGAIN/EWOULDBLOCK condition, raise IOError > on any other problem) Yup. > On 3/6/07, Guido van Rossum wrote: > > Reading this and all the other discussion on the proper semantics for > > non-blocking I/O I think I may have overreached in trying to support > > non-blocking I/O at all levels of the new I/O stack. There probably > > aren't enough use cases for wanting to support readline() returning > > None if no full line if input is available yet to warrant the > > additional complexities -- and I haven't even looked very carefully at > > incremental codecs, which introduce another (small) buffer. > > > > I think maybe a useful simplification would be to support special > > return values to capture EWOULDBLOCK (or equivalent) in the raw I/O > > interface only. I think it serves a purpose here, since without such > > support, code doing raw I/O would either require catching IOError all > > the time and inspecting it for EWOULDBLOCK (or other platform specific > > values!), or not using the raw I/O interface at all, requiring yet > > another interface for raw non-blocking I/O. > > > > The buffering layer could then raise IOError (or perhaps a special > > subclass of it) if the raw I/O layer ever returned one of these; e.g. > > if a buffered read needs to go to the raw layer to satisfy a request > > and the raw read returns None, then the buffered read needs to raise > > this error if no data has been taken out of the buffer yet; or it > > should return a short read if some data was already consumed (since > > it's hard to "unconsume" data, especially if the requested read length > > is larger than the buffer size, or if there's an incremental encoder > > involved). Thus, applications can assume that a short read means > > either EOF or nonblocking I/O; most apps can safely ignore the latter > > since it must be explicitly be turned on by the app. > > > > For writing, if the buffering layer receives a short write, it should > > try again; but if it receives an EWOULDBLOCK, it should likewise raise > > the abovementioned error, since repeated attempts to write in this > > case would just end up spinning the CPU without making progress. (We > > should not raise an error if a single short write happens, since AFAIK > > this is possible for TCP sockets even in blocking mode, witness the > > addition of the sendall() method.) > > > > This means that the buffering layer that sits directly on top of the > > raw layer must still be prepared to deal with the special return > > values from non-blocking I/O, but its API to the next layer up doesn't > > need special return values, since it turns these into IOErrors, and > > the next layer(s) up won't have to deal with it nor reflect it in > > their API. > > > > Would this satisfy the critics of the current design? > > > > --Guido > > > > On 3/4/07, Adam Olsen wrote: > > > On 3/4/07, Greg Ewing wrote: > > > > I'm having trouble seeing what the use case is for > > > > the buffered non-blocking writes being discussed here. > > > > > > > > Doing asynchronous I/O usually *doesn't* involve > > > > putting the file descriptor into non-blocking mode. > > > > Instead you use select() or equivalent, and only > > > > try to read or write when the file is reported as > > > > being ready. > > > > > > I can't say which is more common, but non-blocking has a safer feel. > > > Normal code would be select-driven in both, but if you screw up with > > > non-blocking you get an error, whereas blocking you get a mysterious > > > hang. > > > > > > accept() is the exception. It's possible for a connection to > > > disappear between the time select() returns and the time you call > > > accept(), so you need to be non-blocking to avoid hanging. > > > > > > > > > > > For this to work properly, the select() needs to > > > > operate at the *bottom* of the I/O stack. Any > > > > buffering layers sit above that, with requests for > > > > data propagating up the stack as the file becomes > > > > ready. > > > > > > > > In other words, the whole thing has to have the > > > > control flow inverted and work in "pull" mode > > > > rather than "push" mode. It's hard to see how this > > > > could fit into the model as a minor variation on > > > > how writes are done. > > > > > > Meaning it needs to be a distinct interface and explicitly designed as such. > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/daniel%40stutzbachenterprises.com > > > > > -- > Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 7 02:08:48 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 17:08:48 -0800 Subject: [Python-3000] PEP Draft: Enhancing the buffer protcol In-Reply-To: <20070228132631.AE7A.JCARLSON@uci.edu> References: <20070228104438.AE6E.JCARLSON@uci.edu> <45E5E5DC.9040103@ee.byu.edu> <20070228132631.AE7A.JCARLSON@uci.edu> Message-ID: On 2/28/07, Josiah Carlson wrote: > > Travis Oliphant wrote: > > Josiah Carlson wrote: > > >Travis Oliphant wrote: > > >>I think you are right. In the discussions for unifying string/unicode I > > >>really like the proposals that are leaning toward having a unicode > > >>object be an immutable string of either ucs-1, ucs-2, or ucs-4 depending > > >>on what is in the string. > > > > > >Except that its not going to happen. The width of the unicode > > >representation is going to be fixed at compile time, generally utf-16 or > > >ucs-4. > > > > Are you sure about this? Guido was still talking about the > > multiple-version representation at PyCon a few days ago. > > I was thinking of Guido's message from August 31, 2006 with the subject > of "Re: [Python-3000] UTF-16", in that message he states that he would > like it to be a configure (presumably during compilation) option. > > If he's talking about different runtime representations, then there's an > entire thread discussing it with the subject of "How will unicode get > used?" in September of 2006, and an earlier thread prior to that. While > I was an early proponent of 'represent minimally', I'm not terribly > worried about it either way at this point, and was merely attempting to > state what had been expressed in the past. I haven't been following that as closely as perhaps I should have. I'd be glad to drop this and go back to a string representation/implementation that's essentially the 2.x unicode type, with a compile-time configuration choice between 16 or 32 bits wide characters only. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Mar 7 00:51:49 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Mar 2007 12:51:49 +1300 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: <45EDFE95.4050908@canterbury.ac.nz> Guido van Rossum wrote: > Would this satisfy the critics of the current design? That all sounds reasonable to me. -- Greg From rhamph at gmail.com Wed Mar 7 03:04:20 2007 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 6 Mar 2007 19:04:20 -0700 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/6/07, Guido van Rossum wrote: > The buffering layer could then raise IOError (or perhaps a special > subclass of it) if the raw I/O layer ever returned one of these; What's the rationale for IOError instead of ValueError? Isn't it an error in the application to apply the buffering layer to a non-blocking socket, and not something related to a connection reset? +1 on the rest -- Adam Olsen, aka Rhamphoryncus From python-dev at zesty.ca Wed Mar 7 03:20:10 2007 From: python-dev at zesty.ca (Ka-Ping Yee) Date: Tue, 6 Mar 2007 20:20:10 -0600 (CST) Subject: [Python-3000] PEP 3113 transition plan In-Reply-To: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> References: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> Message-ID: On Tue, 6 Mar 2007, Collin Winter wrote: > I'd like to strike the part about making "x.next()" -> > "next(x)"/"x.__next__()" determined by the presence of a module-level > "next" binding. I'd rather see the transformation always be "x.next()" > -> "next(x)" and warn on top-level "next"s. Doing it the way the PEP > suggests is possible, but ugly (requires an unusually-complicated > fixer) and slow. I can give you warnings on the following items, > though: > > - global assignments to "next". > - global definitions of a "next" function. > - global imports of anything named "next". > - assignments to "__builtin__.next". That sounds pretty good to me. I've updated the PEP. (In "global assignments" don't forget bindings introduced by "for next in blah".) -- ?!ng From guido at python.org Wed Mar 7 03:30:52 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 18:30:52 -0800 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/6/07, Adam Olsen wrote: > On 3/6/07, Guido van Rossum wrote: > > The buffering layer could then raise IOError (or perhaps a special > > subclass of it) if the raw I/O layer ever returned one of these; > > What's the rationale for IOError instead of ValueError? Isn't it an > error in the application to apply the buffering layer to a > non-blocking socket, and not something related to a connection reset? ValueError would mean that the specific call to read() or write() had invalid argument values. IOError means that there's an I/O-related error, which means that it's somewhat expected during normal operation; *anything* I/O related raises IOError (even if you had closed the file descriptor behind the file's back). ValueError is typically a bug in the immediate code containing the call. IOError is something that could happen even to valid calls (e.g. when a library is passed a stream that happens to have been put in nonblocking mode; if the library has a recovery for other I/O errors like disk full or broken connections, the recovery should be applied in this case, too). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Wed Mar 7 03:54:41 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 06 Mar 2007 18:54:41 -0800 Subject: [Python-3000] PEP Draft: Enhancing the buffer protcol In-Reply-To: References: <20070228132631.AE7A.JCARLSON@uci.edu> Message-ID: <20070306184828.74AA.JCARLSON@uci.edu> "Guido van Rossum" wrote: > I haven't been following that as closely as perhaps I should have. I'd > be glad to drop this and go back to a string > representation/implementation that's essentially the 2.x unicode type, > with a compile-time configuration choice between 16 or 32 bits wide > characters only. Sounds good to me. If the extended buffer interface gets accepted, it may make sense to remove the string encoding "defenc" attribute on unicode objects which is "used for implementing the buffer protocol", as it would no longer be necessary for the buffer interface, or it could be kept around for 'print(unicodeobject)' calls. - Josiah From daniel at stutzbachenterprises.com Wed Mar 7 03:52:09 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 6 Mar 2007 20:52:09 -0600 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/6/07, Adam Olsen wrote: > What's the rationale for IOError instead of ValueError? Isn't it an > error in the application to apply the buffering layer to a > non-blocking socket, and not something related to a connection reset? The buffering layer can't easily detect that the file object is non-blocking a priori. It only finds out when a system call returns -1 and errno is set to EAGAIN. In the meantime, it might have actually read or writen some data. An IOError makes it more clear that something went awry, while a ValueError typically implies that the operation was rejected altogether. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From pmaupin at gmail.com Wed Mar 7 05:57:30 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Tue, 6 Mar 2007 22:57:30 -0600 Subject: [Python-3000] Interaction between unittest and keyword argument machinery Message-ID: It appears that if a C function is called from Python with **kwargs, a new dictionary object is created and passed to the C function even if **kwargs is empty, but if the same C function is called without **kwargs, then the NULL pointer is passed to the C function. Because unittest always creates **kwargs, any code path in a C function which is only executed when the *keywords parameter is NULL will never be correctly tested from the standard unittest methods. doctest doesn't have this issue, but appears to be deprecated. OTOH unless it would be a major performance hit to never pass empty *dictionary parameters (always use a NULL pointer) to C functions, it would remove a whole class of untested potential execution paths to change the interpreter. Regards, Pat From guido at python.org Wed Mar 7 06:32:03 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 6 Mar 2007 21:32:03 -0800 Subject: [Python-3000] Interaction between unittest and keyword argument machinery In-Reply-To: References: Message-ID: On 3/6/07, Patrick Maupin wrote: > It appears that if a C function is called from Python with **kwargs, a > new dictionary object is created and passed to the C function even if > **kwargs is empty, but if the same C function is called without > **kwargs, then the NULL pointer is passed to the C function. > > Because unittest always creates **kwargs, any code path in a C > function which is only executed when the *keywords parameter is NULL > will never be correctly tested from the standard unittest methods. This needs some context; which call from unittest to a C function are you talking about? > doctest doesn't have this issue, but appears to be deprecated. Far from it! Doctest is alive and well. I even used it to test the xreload module I added to Py3k. I admit that I've not been a fan of it in the past, and in many cases I will prefer unittest, but there are definitely very good reasons to support doctest. Consider it a rare deviation from TOOWTDI. > OTOH > unless it would be a major performance hit to never pass empty > *dictionary parameters (always use a NULL pointer) to C functions, it > would remove a whole class of untested potential execution paths to > change the interpreter. You guessed it, it's a major performance hit to create a whole new dict object (even if empty) when the majority of calls don't need it. So, no, this isn't going away any time soon. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Mar 7 08:53:23 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Mar 2007 20:53:23 +1300 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: <45EE6F73.80109@canterbury.ac.nz> Adam Olsen wrote: > What's the rationale for IOError instead of ValueError? Isn't it an > error in the application to apply the buffering layer to a > non-blocking socket, and not something related to a connection reset? Good point -- I often put a try-except for EnvironmentError around things to catch errors resulting from external conditions and report them to the user. So I'd prefer something that didn't descend from EnvironmentError for this. TypeError might even be more appropriate than ValueError, since a non-blocking socket could be considered the wrong type of object to do buffered I/O on. -- Greg From tjreedy at udel.edu Wed Mar 7 09:27:17 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 7 Mar 2007 03:27:17 -0500 Subject: [Python-3000] PEP 3113 transition plan References: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> Message-ID: "Collin Winter" wrote in message news:43aa6ff70703061643j4c722138qee88c7e2b4800b4b at mail.gmail.com... | >From the checked-in version, | | """ | Two additional transformations will be added to the 2to3 translation tool [3]: | | * Method definitions named next will be renamed to __next__. | * Explicit calls to the next method will be replaced with calls to | the built-in next function. For example, x.next() will become next(x). As I mentioned before, the above does not seem to cover the case of non-call attribute lookups, as when making instance bindings, that also need the name changed. IE gnext = g.next needs to be changed to gnext = g.__next__. Have you included such cases? tjr From greg.ewing at canterbury.ac.nz Wed Mar 7 09:56:10 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Mar 2007 21:56:10 +1300 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: <45EE7E2A.8040004@canterbury.ac.nz> Guido van Rossum wrote: > ValueError is > typically a bug in the immediate code containing the call. Not necessarily, because the value may have been stored somewhere for a while before being used. > IOError is > something that could happen even to valid calls (e.g. when a library > is passed a stream that happens to have been put in nonblocking mode; But the calling code has made a mistake by passing such a stream to the library in the first place. > if the library has a recovery for other I/O errors like disk full or > broken connections, the recovery should be applied in this case, too). I don't see how. For something like "Disk full", the library can report it to the application, and the application can notify the user and tell him to make more room somehow. But how can the application recover from trying to use a non-blocking socket inappropriately? It's not the user's fault, and there's nothing the user can to do fix it. I'd much rather the whole EnvironmentError subtree were reserved for external things that the user can do something about. That way the application can easily sort out what should be reported as a user error, and what should be reported as a program bug. I realise this isn't quite true at the moment, because things like passing invalid parameters to a system call will probably produce an IOError or OSError. But there's no sense in muddying things any further if it can be easily avoided. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 7 09:56:17 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 07 Mar 2007 21:56:17 +1300 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: <45EE7E31.8030006@canterbury.ac.nz> Daniel Stutzbach wrote: > An IOError makes it more clear > that something went awry, while a ValueError typically implies that > the operation was rejected altogether. In the code I typically write, the IOError will get reported to the user, who won't know why it happened or what to do about it, and the application will abort that operation and go on to do something else. The ValueError will cause the application to abort altogether and produce a traceback, which is as it should be, since there's a bug. In either case, it doesn't really matter whether something was written or not, as the whole operation is screwed anyway. -- Greg From g.brandl at gmx.net Wed Mar 7 10:26:30 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Mar 2007 10:26:30 +0100 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: Ka-Ping Yee schrieb: > On Wed, 7 Mar 2007, Georg Brandl wrote: >> Ka-Ping Yee schrieb: >> > On Tue, 6 Mar 2007, Guido van Rossum wrote: >> >> Having now read this entire thread I am going to accept Ping's PEP. >> >> Adding the sentinel argument to the next() builtin was what did it for >> >> me: it neatly solves the problem if having to catch that StopIteration >> >> in 99% of the cases. >> > >> > Okay, this is checked in as PEP 3114. >> >> Patch is at http://python.org/sf/1675363. > > Thanks for doing this work! I hope it helps getting a decision about the PEP. One thing that struck me while doing the next -> __next__ transition was the new asymmetry between generator methods; there is now send() and close(), but no next() anymore. Georg From g.brandl at gmx.net Wed Mar 7 13:25:01 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 07 Mar 2007 13:25:01 +0100 Subject: [Python-3000] __methods__ and __members__ Message-ID: While reviewing the patch for __dir__() (which I'll apply then, since it was generally agreed upon at least for Py3k), I came about this: /* Merge in __members__ and __methods__ (if any). XXX Would like this to go away someday; for now, it's XXX needed to get at im_self etc of method objects. */ What is the status of __methods__ and __members__? Is this (and the docs) the only trace that's left of it? Georg From ncoghlan at gmail.com Wed Mar 7 15:11:05 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 08 Mar 2007 00:11:05 +1000 Subject: [Python-3000] List & set comprehensions patch In-Reply-To: References: <45ED8CD4.4020905@gmail.com> Message-ID: <45EEC7F9.6030902@gmail.com> Guido van Rossum wrote: > Quick responses from just reading the email (I'll try to review the > code later today, I'm trying to do Py3k work all day): > > On 3/6/07, Nick Coghlan wrote: >> Georg and I have been working on the implementation of list >> comprehensions which don't leak their iteration variables, along >> with the implementation of set comprehensions. The latest patch can >> be found as SF patch #1660500 [1]. The file new-set-comps.diff is >> the combined patch which implements both features, and unifies >> handling of the different kinds of comprehension. In an effort to >> improve readability, the patch also converts the sets in symtable.c >> to be actual PySet objects, rather than PyDict objects with None >> keys and tries to reduce the number of different meanings assigned >> to the term 'scope'. > > Maybe you could separate that out into a separate patch so it won't > old up review or work on the main patch? Or is there a stronger > connection? I think they can be separated - I didn't do it before posting the patch on SF as I was well and truly over symtable.c by that point :) I'll try to have a look at splitting them this weekend. > How about determining if it's a *simple* case or not, and doing the > variable renaming in the simple case and Georg's original version in > non-simple cases? You can define "simple" as whatever makes the > determination easy and still treats most common cases as simple. E.g. > a lambda would be a non-simple case, and so would using a nonlocal > or global variable (note though that nonlocal and global should reach > inside the list/set comp!) etc. It sounds feasible, but I don't think the lack of it should prevent the change from going in. The implementation in the patch *is* a regression from a speed point of view, but it's also an obvious target for later optimisation - if we find a function scope that cannot outlive it's parent scope (such as a simple comprehension), then we can inline that code in the parent scope by using renamed variables. However, there's then further weirdness with what effect this has on the contents of locals()... It looks like even nailing down what 'simple' means in this situation will be somewhat tricky :P >> In implementing it, I discovered that list comprehensions don't do >> SETUP_LOOP/POP_BLOCK around their for loop - I'd like to get >> confirmation from someone who knows their way around the ceval loop >> better than I do that omitting those is actually legitimate (I >> *think* the restriction to a single expression in the body of the >> comprehension makes it OK, but I'm not sure). > > They exist to handle break/continue. Since those don't apply to > list/set comps, it's safe. Excellent - that's what I thought they were about. Does yield need them? If it doesn't, we can probably also remove them from the bytecode emitted for generator expressions. >> There are also a couple of tests we had to disable - one in >> test_dis, one in test_grammar. Suggestions on how to reinstate >> those (or agreement that it is OK to get rid of them) would be >> appreciated. > > I'll have to look later. According to Georg, it's only the test_dis one that needs a closer look - the removed grammar test was declared invalid some time ago. >> The PySet update code in symtable.c currently uses >> PyNumber_InplaceOr with a subsequent call to Py_DECREF to counter >> the implicit call to Py_INCREF. Should this be changed to use >> PyObject_CallMethod to invoke the Python level update method? > > What's wrong with the inplace or? I seem to recall that s |= x and > s.update(x) aren't equivalent if x is not a set. A while back when Barry wanted to add PySet_Update, Raymond recommended using PyObject_CallMethod instead. In this case, I know I'm dealing with real set objects, so the inplace-or works fine (it's annoyingly easy to forget the decref and leak a reference though... all the more reason to leave that part of the patch out for the moment, I guess) >> - only the outermost iterator expression is evaluated in the scope >> containing the comprehension (just like generator expressions). >> This means that the inner expressions can no longer see class >> variables and values in explicit locals() dictionaries provided to >> exec & friends. This didn't actually cause any problems in the >> standard library - I only note it because my initial implementation >> mistakenly evaluated the outermost iterator in the new scope, which >> *did* cause severe problems along these lines. > > This smells fishy. Do you have an example? The following example uses a generator expression to show the effect of the semantic change: .>>> class Dummy: ... x = 10 ... print [x for i in range(5)] ... print list(x for i in range(5)) ... [10, 10, 10, 10, 10] Traceback (most recent call last): File "", line 1, in ? File "", line 4, in Dummy File "", line 4, in NameError: global name 'x' is not defined With list comprehensions also switching to private iteration variables, the first print statement will throw a NameError instead of printing the list. This was brought to my attention by the fact that my original buggy implementation blew up on really common things like "[i for i in seq if cond(i)]" when the code was executed with execfile() rather than being imported. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From pmaupin at gmail.com Wed Mar 7 16:22:52 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 7 Mar 2007 09:22:52 -0600 Subject: [Python-3000] Interaction between unittest and keyword argument machinery In-Reply-To: References: Message-ID: > > Because unittest always creates **kwargs, any code path in a C > > function which is only executed when the *keywords parameter is NULL > > will never be correctly tested from the standard unittest methods. > > This needs some context; which call from unittest to a C function are > you talking about? Umm, my broken ones? :) In doing some personal doctesting, I noticed an issue in my PEP3101 code with a null pointer, and added a test to the unittest code to catch it in preparation for fixing it. My unittest code never failed, and I realized that the unittest module was always passing down a keyword dictionary, even when no keywords were passed to it. I'm sure that most Python C programmers are more experienced/careful than I am, but even so, it seems that there are a whole class of potential bugs in C functions that unittest cannot check for at present, because it always passes a keywords dictionary. > > > doctest doesn't have this issue, but appears to be deprecated. > > Far from it! Doctest is alive and well. I even used it to test the > xreload module I added to Py3k. I admit that I've not been a fan of it > in the past, and in many cases I will prefer unittest, but there are > definitely very good reasons to support doctest. Consider it a rare > deviation from TOOWTDI. The documentation for the "test" module states that "All new tests should be written using the unittest module; using unittest is not required but makes the tests more flexible and maintenance of the tests easier. Some older tests are written to use doctest and a ``traditional'' testing style; these styles of tests will not be covered." While that language is arguably quite mild, it was enough to scare me off of using doctest, but for a lot of what I want to test on the string formatting, doctest is preferable, so I guess I'll build some doctests to check in! > > OTOH > > unless it would be a major performance hit to never pass empty > > *dictionary parameters (always use a NULL pointer) to C functions, it > > would remove a whole class of untested potential execution paths to > > change the interpreter. > > You guessed it, it's a major performance hit to create a whole new > dict object (even if empty) when the majority of calls don't need it. > So, no, this isn't going away any time soon. That's not actually what I was suggesting. While it still may be too much of a performance hit, I was suggesting always passing NULL if the dictionary is empty. Since the C code has to support _some_ execution path for a NULL dictionary in any case, I thought perhaps you could remove the equivalent empty dictionary case to let the C code practice TOOWTDI for empty or NULL dictionaries. Obviously, some other C function could still call a C function with an empty dictionary, but all calls straight from Python to the C function would take the same execution path for empty as for NULL dictionaries, so the unittest coverage would automatically be better. (The naive implementation of what I am discussing would be to always pass as the keyword parameter the expression: (((keyword != NULL) && (PyDict_Size(keyword) != 0)) ? keyword : NULL) Regards, Pat From collinw at gmail.com Wed Mar 7 16:40:28 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 7 Mar 2007 09:40:28 -0600 Subject: [Python-3000] PEP 3113 transition plan In-Reply-To: References: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> Message-ID: <43aa6ff70703070740x12a74d4elf2c4e4dbf3547888@mail.gmail.com> On 3/7/07, Terry Reedy wrote: > > "Collin Winter" wrote in message > news:43aa6ff70703061643j4c722138qee88c7e2b4800b4b at mail.gmail.com... > | >From the checked-in version, > | > | """ > | Two additional transformations will be added to the 2to3 translation tool > [3]: > | > | * Method definitions named next will be renamed to __next__. > | * Explicit calls to the next method will be replaced with calls to > | the built-in next function. For example, x.next() will become next(x). > > As I mentioned before, the above does not seem to cover the case of > non-call attribute lookups, as when making instance bindings, that also > need the name changed. IE gnext = g.next needs to be changed to gnext = > g.__next__. Have you included such cases? Not yet, but doing so is trivial. Collin Winter From guido at python.org Wed Mar 7 16:49:20 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 07:49:20 -0800 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: <45EE7E31.8030006@canterbury.ac.nz> References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> <45EE7E31.8030006@canterbury.ac.nz> Message-ID: I don't agree with this line of reasoning. It should be a ValueError when passing a nonblocking raw I/O object to BufferedIO, but that's not detectable (at least not without an unpleasant amount of platform-specific code), and it isn't sufficient either, since the file could be blocking initially and made non-blocking later. I/O primitives produce IOError for *anything* I/O-related. *Something* in the app has made the file nonblocking where it shouldn't. It's no different from trying to read from a low-level file descriptor that's open for writing only -- that will raise IOError too, and there's nothing we can do about it, so you better accept it than trying to fight it. --Guido On 3/7/07, Greg Ewing wrote: > Daniel Stutzbach wrote: > > An IOError makes it more clear > > that something went awry, while a ValueError typically implies that > > the operation was rejected altogether. > > In the code I typically write, the IOError will get > reported to the user, who won't know why it happened > or what to do about it, and the application will > abort that operation and go on to do something else. > > The ValueError will cause the application to abort > altogether and produce a traceback, which is as it > should be, since there's a bug. > > In either case, it doesn't really matter whether > something was written or not, as the whole operation > is screwed anyway. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Wed Mar 7 16:53:56 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 7 Mar 2007 09:53:56 -0600 Subject: [Python-3000] Interaction between unittest and keyword argument machinery In-Reply-To: References: Message-ID: <43aa6ff70703070753t22592351v713092ee917459c4@mail.gmail.com> On 3/7/07, Patrick Maupin wrote: > > > Because unittest always creates **kwargs, any code path in a C > > > function which is only executed when the *keywords parameter is NULL > > > will never be correctly tested from the standard unittest methods. > > > > This needs some context; which call from unittest to a C function are > > you talking about? > > Umm, my broken ones? :) In doing some personal doctesting, I noticed > an issue in my PEP3101 code with a null pointer, and added a test to > the unittest code to catch it in preparation for fixing it. My > unittest code never failed, and I realized that the unittest module > was always passing down a keyword dictionary, even when no keywords > were passed to it. I'm sure that most Python C programmers are more > experienced/careful than I am, but even so, it seems that there are a > whole class of potential bugs in C functions that unittest cannot > check for at present, because it always passes a keywords dictionary. The only place in unittest that uses **kwargs is TestCase.failUnlessRaises()/assertRaises(); is this the method you're using when writing your tests? If so, then don't use assertRaises(): just call your function and trap the exception manually. Collin Winter From jimjjewett at gmail.com Wed Mar 7 16:57:56 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 7 Mar 2007 10:57:56 -0500 Subject: [Python-3000] PEP 3113 transition plan In-Reply-To: References: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> Message-ID: On Tue, 6 Mar 2007, Collin Winter wrote: > I can give you warnings on the following items, > - global assignments to "next". > - global definitions of a "next" function. > - global imports of anything named "next". > - assignments to "__builtin__.next". How hard would it be to warn about any assignment to next, even in nested functions? def f(): def next(): def g(): next=47 -jJ From collinw at gmail.com Wed Mar 7 17:05:10 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 7 Mar 2007 10:05:10 -0600 Subject: [Python-3000] PEP 3113 transition plan In-Reply-To: References: <43aa6ff70703061643j4c722138qee88c7e2b4800b4b@mail.gmail.com> Message-ID: <43aa6ff70703070805k3bfd6f41r4c6122737137a948@mail.gmail.com> On 3/7/07, Jim Jewett wrote: > On Tue, 6 Mar 2007, Collin Winter wrote: > > > I can give you warnings on the following items, > > > - global assignments to "next". > > - global definitions of a "next" function. > > - global imports of anything named "next". > > - assignments to "__builtin__.next". > > How hard would it be to warn about any assignment to next, even in > nested functions? > > def f(): > def next(): > def g(): > next=47 Not hard at all. It's actually easier than warning about module-level assignments. Collin Winter From jimjjewett at gmail.com Wed Mar 7 17:14:18 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 7 Mar 2007 11:14:18 -0500 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/6/07, Guido van Rossum wrote: > I think maybe a useful simplification would be to support special > return values to capture EWOULDBLOCK (or equivalent) in the raw I/O > interface only. That makes sense. > The buffering layer could then raise IOError (or perhaps a special > subclass of it) if the raw I/O layer ever returned one of these; Is this a "could", or "should"? I would expect the buffering layer (particularly output) to use its buffer, and to appear blocking (through sleep-and-retry) when that isn't enough. Or are you concerned that if it might really be blocked forever, and should say so at the first opportunity? -jJ From daniel at stutzbachenterprises.com Wed Mar 7 17:32:21 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 7 Mar 2007 10:32:21 -0600 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/7/07, Jim Jewett wrote: > > The buffering layer could then raise IOError (or perhaps a special > > subclass of it) if the raw I/O layer ever returned one of these; > > Is this a "could", or "should"? I would expect the buffering layer > (particularly output) to use its buffer, and to appear blocking > (through sleep-and-retry) when that isn't enough. If programmer has wrapped a non-blocking object with a buffer to give it blocking behavior, I think this is a programming error and Python should treat as such, not work around it so that the problem is harder to detect. An object only becomes non-blocking if the program explicitly makes it non-blocking via fcntl. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From guido at python.org Wed Mar 7 18:43:04 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 09:43:04 -0800 Subject: [Python-3000] __methods__ and __members__ In-Reply-To: References: Message-ID: I don't recall ripping them out, but it's worth trying to do that -- they really shouldn't be needed for modern extensionmodules (2.2+). On 3/7/07, Georg Brandl wrote: > While reviewing the patch for __dir__() (which I'll apply then, since it was > generally agreed upon at least for Py3k), I came about this: > > /* Merge in __members__ and __methods__ (if any). > XXX Would like this to go away someday; for now, it's > XXX needed to get at im_self etc of method objects. */ > > What is the status of __methods__ and __members__? Is this (and the docs) > the only trace that's left of it? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 7 18:55:04 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 09:55:04 -0800 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: On 3/7/07, Jim Jewett wrote: > On 3/6/07, Guido van Rossum wrote: > > I think maybe a useful simplification would be to support special > > return values to capture EWOULDBLOCK (or equivalent) in the raw I/O > > interface only. > > That makes sense. > > > The buffering layer could then raise IOError (or perhaps a special > > subclass of it) if the raw I/O layer ever returned one of these; > > Is this a "could", or "should"? I would expect the buffering layer > (particularly output) to use its buffer, and to appear blocking > (through sleep-and-retry) when that isn't enough. > > Or are you concerned that if it might really be blocked forever, and > should say so at the first opportunity? If the read can be satisifed from the buffer, or with a single raw read call, I see no reason to return an error. But if a raw read call is necessary and the raw read call returns None, the buffered layer should *not* retry since that would just turn it into a busy-wait loop until the read can be satisfied, which is a really bad idea. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Wed Mar 7 21:23:23 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 7 Mar 2007 14:23:23 -0600 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: <43aa6ff70607131312q1d323a07hce351a1d648e1d71@mail.gmail.com> References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> <43aa6ff70607030840l3bf31fbdw45664884578d3f83@mail.gmail.com> <43aa6ff70607131312q1d323a07hce351a1d648e1d71@mail.gmail.com> Message-ID: <43aa6ff70703071223n46d1889fh2cc15b720ce20cff@mail.gmail.com> On 7/13/06, Collin Winter wrote: > On 7/3/06, Collin Winter wrote: > > On 7/3/06, Guido van Rossum wrote: > > > It turns out I was misled by Collin's claim that the PEP wants > > > isCallable and sequenceIncludes removed "because there are better, > > > more obvious ways to spell these things." He must have made up the > > > motivation, as the PEP doesn't give any. > > > > I was inferring that motivation > > So that others don't have to infer in the future, I've posted patch > #1522038, which adds an explanation as to why these items are being > removed. > > Also, in case you (Guido) have changed your mind about removing > operator.truth and operator.abs, I've also posted patch #1522059 which > will remove them from the stdlib. I had forgotten I had posted this patch... I don't suppose you've changed your mind about removing operator.truth and operator.abs in the seven months since this discussion? Collin From brett at python.org Wed Mar 7 22:15:23 2007 From: brett at python.org (Brett Cannon) Date: Wed, 7 Mar 2007 13:15:23 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC6327.8010906@v.loewis.de> <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On 3/7/07, Georg Brandl wrote: > Ka-Ping Yee schrieb: > > On Wed, 7 Mar 2007, Georg Brandl wrote: > >> Ka-Ping Yee schrieb: > >> > On Tue, 6 Mar 2007, Guido van Rossum wrote: > >> >> Having now read this entire thread I am going to accept Ping's PEP. > >> >> Adding the sentinel argument to the next() builtin was what did it for > >> >> me: it neatly solves the problem if having to catch that StopIteration > >> >> in 99% of the cases. > >> > > >> > Okay, this is checked in as PEP 3114. > >> > >> Patch is at http://python.org/sf/1675363. > > > > Thanks for doing this work! > > I hope it helps getting a decision about the PEP. > > One thing that struck me while doing the next -> __next__ transition > was the new asymmetry between generator methods; there is now send() > and close(), but no next() anymore. > Oooh, that's a good point. I guess that would mean generators should keep their 'next' methods for API symmetry with 'send' and 'close'; calling next() just for when you are not sending something in would be icky. But of course having two methods that do the exact same thing seems a little icky as well. I was on the fence with this whole proposal, but this makes me -0 on the rename and +0 on the new built-in. -Brett From greg.ewing at canterbury.ac.nz Wed Mar 7 22:50:38 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 08 Mar 2007 10:50:38 +1300 Subject: [Python-3000] List & set comprehensions patch In-Reply-To: <45ED8CD4.4020905@gmail.com> References: <45ED8CD4.4020905@gmail.com> Message-ID: <45EF33AE.4040004@canterbury.ac.nz> Nick Coghlan wrote: > One of the comments made on Georg's initial attempt at implementing > these features was that it would be nice to avoid the function call > overhead in the listcomp & setcomp case ... I tried to do that and > essentially failed outright Not having seen the code, my thought would be to temporarily change the symtab entry so that it refers to a different locals slot, compile the nested code with that, and then change it back again. Did you consider doing something like that and find that it wouldn't work? -- Greg From guido at python.org Wed Mar 7 23:07:40 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 14:07:40 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: Since next() is equivalent to send(None) we don't really need the next() method do we? On 3/7/07, Brett Cannon wrote: > On 3/7/07, Georg Brandl wrote: > > Ka-Ping Yee schrieb: > > > On Wed, 7 Mar 2007, Georg Brandl wrote: > > >> Ka-Ping Yee schrieb: > > >> > On Tue, 6 Mar 2007, Guido van Rossum wrote: > > >> >> Having now read this entire thread I am going to accept Ping's PEP. > > >> >> Adding the sentinel argument to the next() builtin was what did it for > > >> >> me: it neatly solves the problem if having to catch that StopIteration > > >> >> in 99% of the cases. > > >> > > > >> > Okay, this is checked in as PEP 3114. > > >> > > >> Patch is at http://python.org/sf/1675363. > > > > > > Thanks for doing this work! > > > > I hope it helps getting a decision about the PEP. > > > > One thing that struck me while doing the next -> __next__ transition > > was the new asymmetry between generator methods; there is now send() > > and close(), but no next() anymore. > > > > Oooh, that's a good point. I guess that would mean generators should > keep their 'next' methods for API symmetry with 'send' and 'close'; > calling next() just for when you are not sending something in would be > icky. > > But of course having two methods that do the exact same thing seems a > little icky as well. I was on the fence with this whole proposal, but > this makes me -0 on the rename and +0 on the new built-in. > > -Brett > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 7 23:20:46 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 14:20:46 -0800 Subject: [Python-3000] List & set comprehensions patch In-Reply-To: <45EEC7F9.6030902@gmail.com> References: <45ED8CD4.4020905@gmail.com> <45EEC7F9.6030902@gmail.com> Message-ID: On 3/7/07, Nick Coghlan wrote: > > How about determining if it's a *simple* case or not, and doing the > > variable renaming in the simple case and Georg's original version in > > non-simple cases? You can define "simple" as whatever makes the > > determination easy and still treats most common cases as simple. E.g. > > a lambda would be a non-simple case, and so would using a nonlocal > > or global variable (note though that nonlocal and global should reach > > inside the list/set comp!) etc. > > It sounds feasible, but I don't think the lack of it should prevent the > change from going in. I was worried that if the change went in, the portions of the old code that would still be usable for the "simple" cases would be deleted or at least go stale. But I think it's more important to make the new syntax and semantics go in. > The implementation in the patch *is* a regression > from a speed point of view, but it's also an obvious target for later > optimisation - if we find a function scope that cannot outlive it's > parent scope (such as a simple comprehension), then we can inline that > code in the parent scope by using renamed variables. > > However, there's then further weirdness with what effect this has on the > contents of locals()... Yet another reason to declare locals() an implementation detail. I personally never use it. > It looks like even nailing down what 'simple' means in this situation > will be somewhat tricky :P That's why I left the definition of simple to you. :-) > >> In implementing it, I discovered that list comprehensions don't do > >> SETUP_LOOP/POP_BLOCK around their for loop - I'd like to get > >> confirmation from someone who knows their way around the ceval loop > >> better than I do that omitting those is actually legitimate (I > >> *think* the restriction to a single expression in the body of the > >> comprehension makes it OK, but I'm not sure). > > > > They exist to handle break/continue. Since those don't apply to > > list/set comps, it's safe. > > Excellent - that's what I thought they were about. Does yield need them? > If it doesn't, we can probably also remove them from the bytecode > emitted for generator expressions. Yield shouldn't need them; it doesn't involve flow control within the frame. > >> There are also a couple of tests we had to disable - one in > >> test_dis, one in test_grammar. Suggestions on how to reinstate > >> those (or agreement that it is OK to get rid of them) would be > >> appreciated. > > > > I'll have to look later. > > According to Georg, it's only the test_dis one that needs a closer look > - the removed grammar test was declared invalid some time ago. OK, thanks. > >> The PySet update code in symtable.c currently uses > >> PyNumber_InplaceOr with a subsequent call to Py_DECREF to counter > >> the implicit call to Py_INCREF. Should this be changed to use > >> PyObject_CallMethod to invoke the Python level update method? > > > > What's wrong with the inplace or? I seem to recall that s |= x and > > s.update(x) aren't equivalent if x is not a set. > > A while back when Barry wanted to add PySet_Update, Raymond recommended > using PyObject_CallMethod instead. In this case, I know I'm dealing with > real set objects, so the inplace-or works fine (it's annoyingly easy to > forget the decref and leak a reference though... all the more reason to > leave that part of the patch out for the moment, I guess) This is system level code, we only need to get it right once. > >> - only the outermost iterator expression is evaluated in the scope > >> containing the comprehension (just like generator expressions). > >> This means that the inner expressions can no longer see class > >> variables and values in explicit locals() dictionaries provided to > >> exec & friends. This didn't actually cause any problems in the > >> standard library - I only note it because my initial implementation > >> mistakenly evaluated the outermost iterator in the new scope, which > >> *did* cause severe problems along these lines. > > > > This smells fishy. Do you have an example? > > The following example uses a generator expression to show the effect of > the semantic change: > > .>>> class Dummy: > ... x = 10 > ... print [x for i in range(5)] > ... print list(x for i in range(5)) > ... > [10, 10, 10, 10, 10] > Traceback (most recent call last): > File "", line 1, in ? > File "", line 4, in Dummy > File "", line 4, in > NameError: global name 'x' is not defined > > With list comprehensions also switching to private iteration variables, > the first print statement will throw a NameError instead of printing the > list. > > This was brought to my attention by the fact that my original buggy > implementation blew up on really common things like "[i for i in seq if > cond(i)]" when the code was executed with execfile() rather than being > imported. Ah, I see. I think this is acceptable, since it only happens in cases where we already have somewhat odd scoping rules. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Wed Mar 7 23:24:21 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 08 Mar 2007 11:24:21 +1300 Subject: [Python-3000] Non-blocking I/O? (Draft PEP for New IO system) In-Reply-To: References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <5487f95e0703010807h31ee4813j62bb348a490112ea@mail.gmail.com> <45EB6225.7060802@canterbury.ac.nz> Message-ID: <45EF3B95.1040502@canterbury.ac.nz> Jim Jewett wrote: > Is this a "could", or "should"? I would expect the buffering layer > (particularly output) to use its buffer, and to appear blocking > (through sleep-and-retry) when that isn't enough. No! Busy waiting should *not* be done implicitly. -- Greg From brett at python.org Wed Mar 7 23:33:46 2007 From: brett at python.org (Brett Cannon) Date: Wed, 7 Mar 2007 14:33:46 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: <45EC9D7E.6000702@canterbury.ac.nz> Message-ID: On 3/7/07, Guido van Rossum wrote: > Since next() is equivalent to send(None) we don't really need the > next() method do we? > Guess not, especially if we make send() default to sending None back. -Brett > On 3/7/07, Brett Cannon wrote: > > On 3/7/07, Georg Brandl wrote: > > > Ka-Ping Yee schrieb: > > > > On Wed, 7 Mar 2007, Georg Brandl wrote: > > > >> Ka-Ping Yee schrieb: > > > >> > On Tue, 6 Mar 2007, Guido van Rossum wrote: > > > >> >> Having now read this entire thread I am going to accept Ping's PEP. > > > >> >> Adding the sentinel argument to the next() builtin was what did it for > > > >> >> me: it neatly solves the problem if having to catch that StopIteration > > > >> >> in 99% of the cases. > > > >> > > > > >> > Okay, this is checked in as PEP 3114. > > > >> > > > >> Patch is at http://python.org/sf/1675363. > > > > > > > > Thanks for doing this work! > > > > > > I hope it helps getting a decision about the PEP. > > > > > > One thing that struck me while doing the next -> __next__ transition > > > was the new asymmetry between generator methods; there is now send() > > > and close(), but no next() anymore. > > > > > > > Oooh, that's a good point. I guess that would mean generators should > > keep their 'next' methods for API symmetry with 'send' and 'close'; > > calling next() just for when you are not sending something in would be > > icky. > > > > But of course having two methods that do the exact same thing seems a > > little icky as well. I was on the fence with this whole proposal, but > > this makes me -0 on the rename and +0 on the new built-in. > > > > -Brett > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From jcarlson at uci.edu Wed Mar 7 23:48:15 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 07 Mar 2007 14:48:15 -0800 Subject: [Python-3000] PEP: rename it.next() to it.__next__(), add a next() built-in In-Reply-To: References: Message-ID: <20070307143613.74BC.JCARLSON@uci.edu> "Guido van Rossum" wrote: > Since next() is equivalent to send(None) we don't really need the > next() method do we? As long as its renamed to __next__, then we seemingly wouldn't need next, though users needing to use next(X) or X.send(None) instead of X.next(), may be confusing and inconvenient. Should send get a default value of None? - Josiah From guido at python.org Thu Mar 8 01:38:01 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 16:38:01 -0800 Subject: [Python-3000] Removing functions from the operator module In-Reply-To: References: <43aa6ff70607030605p2ca6af88i3c716f2e4b7d192@mail.gmail.com> <43aa6ff70607030711y32c8072am4197d98eff2ae6b6@mail.gmail.com> <43aa6ff70607030840l3bf31fbdw45664884578d3f83@mail.gmail.com> <43aa6ff70607131312q1d323a07hce351a1d648e1d71@mail.gmail.com> <43aa6ff70703071223n46d1889fh2cc15b720ce20cff@mail.gmail.com> Message-ID: On 3/7/07, Collin Winter wrote: > On 7/13/06, Collin Winter wrote: > > On 7/3/06, Collin Winter wrote: > > > On 7/3/06, Guido van Rossum wrote: > > > > It turns out I was misled by Collin's claim that the PEP wants > > > > isCallable and sequenceIncludes removed "because there are better, > > > > more obvious ways to spell these things." He must have made up the > > > > motivation, as the PEP doesn't give any. > > > > > > I was inferring that motivation > > > > So that others don't have to infer in the future, I've posted patch > > #1522038, which adds an explanation as to why these items are being > > removed. (And this was checked in. I agree for sequenceIncludes which should be spelled as operator.contains. I guess operator.callable can go when callable goes.) > > Also, in case you (Guido) have changed your mind about removing > > operator.truth and operator.abs, I've also posted patch #1522059 which > > will remove them from the stdlib. > > I had forgotten I had posted this patch... > > I don't suppose you've changed your mind about removing operator.truth > and operator.abs in the seven months since this discussion? No, though I think that operator.truth should be renamed to operator.bool. I like the idea that for each built-in op there's a callable in operator. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Thu Mar 8 03:39:35 2007 From: talin at acm.org (Talin) Date: Wed, 07 Mar 2007 18:39:35 -0800 Subject: [Python-3000] Discussions with no PEPs Message-ID: <45EF7767.9050406@acm.org> Going back in this list, I see a number of issues which were discussed at length, but there is no corresponding PEP. I realize that in some cases, this is because no actual consensus was reached. In any case, I would like to know, what is the current status of the following issues: -- Generic Functions -- Interfaces -- Metaclass syntax It's not my intention to start a big thread about these - anyone who wants to can go back and read the old discussions. I mainly would like to know: 1) Was there a final pronouncement, or at least a rough consensus on either of these? If so, what was it? 2) Should these have PEPs? (Even if solely for purposes of rejection?) 3) If they are not ready for PEPs, should the discussions be restarted on Python-ideas? In order to prevent a major thread, I would suggest that any response that is not a direct answer to one of the above questions should be moved to python-ideas. Specifically, I'd like to avoid any discussion of the *merits* of the ideas themselves, or alternative solutions - let's limit the comments to just talking about their *status*. And if you need someone to write those PEPs, I can do it - but I'll need some help on the first two in order to understand all of the sides of the controversy. The third one is pretty easy. -- Talin From guido at python.org Thu Mar 8 04:00:02 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 7 Mar 2007 19:00:02 -0800 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45EF7767.9050406@acm.org> References: <45EF7767.9050406@acm.org> Message-ID: I don't think a consensus has been reached on any of these. My votes: - generic functions: no - interfaces: no, but I'd like to work on ABCs instead - metaclass syntax: I'd like to see your PEP --Guido On 3/7/07, Talin wrote: > Going back in this list, I see a number of issues which were discussed > at length, but there is no corresponding PEP. I realize that in some > cases, this is because no actual consensus was reached. > > In any case, I would like to know, what is the current status of the > following issues: > > -- Generic Functions > -- Interfaces > -- Metaclass syntax > > It's not my intention to start a big thread about these - anyone who > wants to can go back and read the old discussions. I mainly would like > to know: > > 1) Was there a final pronouncement, or at least a rough consensus on > either of these? If so, what was it? > 2) Should these have PEPs? (Even if solely for purposes of rejection?) > 3) If they are not ready for PEPs, should the discussions be > restarted on Python-ideas? > > In order to prevent a major thread, I would suggest that any response > that is not a direct answer to one of the above questions should be > moved to python-ideas. Specifically, I'd like to avoid any discussion of > the *merits* of the ideas themselves, or alternative solutions - let's > limit the comments to just talking about their *status*. > > And if you need someone to write those PEPs, I can do it - but I'll need > some help on the first two in order to understand all of the sides of > the controversy. The third one is pretty easy. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Thu Mar 8 08:49:04 2007 From: talin at acm.org (Talin) Date: Wed, 07 Mar 2007 23:49:04 -0800 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <45EF7767.9050406@acm.org> Message-ID: <45EFBFF0.2050609@acm.org> Guido van Rossum wrote: > I don't think a consensus has been reached on any of these. > > My votes: > > - generic functions: no Here's how I would summarize the current state of generic functions. Function decorators + argument decorators already provide everything needed for a third-party developer to implement generic functions on their own. In other words - you could create an add-on module that provides the proper function decorators that would be capable of interpreting argument types and building whatever kind of dispatch table you wanted, without having any further support in the base language other that what's already there now. I understand that Phillip J. Eby, whose experience with generic functions is well known, is interested in having the ability to replace the standard dispatcher. I believe that this requires no further effort on our part, but I welcome corrections on that point. So, to my mind, there are really only a few issues to be answered: 1) Whether Python 3K should provide a 'standard' implementation of those generic function decorators. 2) if so, should it provide one with the initial release of Py3K (i.e. 3.0), or should we wait for various 3rd party modules to emerge, compete in the marketplace, and then pick a winner to be blessed with 'official' status. 3) if a 'standard' implementation is to be included with the 3.0 release, should it be an optimized, native-code dispatcher, or a 'pure python' implementation. Moreover, will any changes to the Python interpreter be required to get the best possible dispatch performance with generic functions. 4) If 3.0 includes a standard dispatcher, to what extent should the standard libraries be allowed to depend on generic functions? In other words, should things like the new i/o library be written to depend on the existence of a built-in dispatcher? Note that this does not address PJE's 'defop' concerns, which (according to my understanding) isn't so much about dispatch mechanisms as it is about the ability to spread the definition of a generic function over multiple source files, so that the methods that operate on a given class can be grouped of that class's definition. The primary motivation is to customize the behavior of a built-in function or operator on a given type, without having to either define a new __special__ method, or monkey patch the built-in function in some way. (Is this correct?) Before we get a flood of answers to these questions, let's make sure that these are, in fact, the right questions. > - interfaces: no, but I'd like to work on ABCs instead > - metaclass syntax: I'd like to see your PEP > > --Guido > > On 3/7/07, Talin wrote: >> Going back in this list, I see a number of issues which were discussed >> at length, but there is no corresponding PEP. I realize that in some >> cases, this is because no actual consensus was reached. >> >> In any case, I would like to know, what is the current status of the >> following issues: >> >> -- Generic Functions >> -- Interfaces >> -- Metaclass syntax >> >> It's not my intention to start a big thread about these - anyone who >> wants to can go back and read the old discussions. I mainly would like >> to know: >> >> 1) Was there a final pronouncement, or at least a rough consensus on >> either of these? If so, what was it? >> 2) Should these have PEPs? (Even if solely for purposes of rejection?) >> 3) If they are not ready for PEPs, should the discussions be >> restarted on Python-ideas? >> >> In order to prevent a major thread, I would suggest that any response >> that is not a direct answer to one of the above questions should be >> moved to python-ideas. Specifically, I'd like to avoid any discussion of >> the *merits* of the ideas themselves, or alternative solutions - let's >> limit the comments to just talking about their *status*. >> >> And if you need someone to write those PEPs, I can do it - but I'll need >> some help on the first two in order to understand all of the sides of >> the controversy. The third one is pretty easy. > From ncoghlan at gmail.com Thu Mar 8 12:05:23 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 08 Mar 2007 21:05:23 +1000 Subject: [Python-3000] List & set comprehensions patch In-Reply-To: <45EF33AE.4040004@canterbury.ac.nz> References: <45ED8CD4.4020905@gmail.com> <45EF33AE.4040004@canterbury.ac.nz> Message-ID: <45EFEDF3.70404@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > >> One of the comments made on Georg's initial attempt at implementing >> these features was that it would be nice to avoid the function call >> overhead in the listcomp & setcomp case ... I tried to do that and >> essentially failed outright > > Not having seen the code, my thought would be to temporarily > change the symtab entry so that it refers to a different > locals slot, compile the nested code with that, and then > change it back again. > > Did you consider doing something like that and find that > it wouldn't work? I didn't, actually - I was looking mostly at the symtable.c code rather than tinkering with the data structures the compiler stage uses to emit the appropriate LOAD_FAST/STORE_FAST/LOAD_DEREF opcodes. At this stage though, I've become more concerned about making sure we get the semantics the way we want them (i.e. the same as generator expressions) before trying to recover some of the speed lost due to the introduction of the anonymous function. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jimjjewett at gmail.com Thu Mar 8 18:15:51 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 8 Mar 2007 12:15:51 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45EF7767.9050406@acm.org> References: <45EF7767.9050406@acm.org> Message-ID: On 3/7/07, Talin wrote: > I would like to know, what is the current status of the > following issues: > -- Generic Functions > -- Interfaces These two were heavily tied to annotations; one option would have been to introduce them at the same time as an example use. Given that annotations do not require them, another option is to wait (at least) an extra release, while annotations sink in. Since they can be done as 3rd-party libraries, I assumed the decision was to postpone at least until there was a concrete implementation candidate available. (And no, I don't know exactly what "candidate available" would require.) > Should these have PEPs? (Even if solely for purposes of rejection?) If Guido says "definately too late", it would be worth putting that in one of the summary PEPs. I don't think it is worth a separate PEP just to say "maybe later". > -- Metaclass syntax Please do provide a summary or PEP of this. My memory boils down to: (1) Guido agreed that it was reasonable to want to use your own mapping type when creating a class. (2) There were a bunch of other questions involving both additional freedoms and which syntax to use, that never really got decided. So (1) didn't happen yet either. -jJ From barry at python.org Thu Mar 8 18:57:36 2007 From: barry at python.org (Barry Warsaw) Date: Thu, 8 Mar 2007 12:57:36 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <45EF7767.9050406@acm.org> Message-ID: <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 8, 2007, at 12:15 PM, Jim Jewett wrote: >> -- Generic Functions >> -- Interfaces > > These two were heavily tied to annotations; one option would have been > to introduce them at the same time as an example use. > > Given that annotations do not require them, another option is to wait > (at least) an extra release, while annotations sink in. > > Since they can be done as 3rd-party libraries, I assumed the decision > was to postpone at least until there was a concrete implementation > candidate available. (And no, I don't know exactly what "candidate > available" would require.) We already have an established, community accepted implementation of interfaces, so why not just accept that into the core stdlib? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfBOknEjvBPtnXfVAQISgQP9HZyIHMX2pYXheAm+/bv1jhABZ4lroM9v OEgE/bNnsyElc+F40z61yHhNbfY1Ldr8/So9j1CwHcxBaRCdZRoexu1ZJp0c+8zT 0ucpYANQkuEcAZNAS15g8hax/BcAqUWME6nCRRsnwZTczjVm0Jtm6FBvXVtu0mUS 3GTaHYSOUCA= =3sS0 -----END PGP SIGNATURE----- From thomas at python.org Thu Mar 8 19:04:29 2007 From: thomas at python.org (Thomas Wouters) Date: Thu, 8 Mar 2007 19:04:29 +0100 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> References: <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> Message-ID: <9e804ac0703081004x39c1fb7br2c5b3de6f29f3fa6@mail.gmail.com> On 3/8/07, Barry Warsaw wrote: > > > On Mar 8, 2007, at 12:15 PM, Jim Jewett wrote: > [Guido] > >> -- Generic Functions > >> -- Interfaces > > > > These two were heavily tied to annotations; one option would have been > > to introduce them at the same time as an example use. > > > > Given that annotations do not require them, another option is to wait > > (at least) an extra release, while annotations sink in. > > > > Since they can be done as 3rd-party libraries, I assumed the decision > > was to postpone at least until there was a concrete implementation > > candidate available. (And no, I don't know exactly what "candidate > > available" would require.) > > We already have an established, community accepted implementation of > interfaces, so why not just accept that into the core stdlib? Guido already knows this, but I would like to second Barry's excellent idea. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070308/42900150/attachment.htm From pje at telecommunity.com Thu Mar 8 20:32:30 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 08 Mar 2007 14:32:30 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45EFBFF0.2050609@acm.org> References: <45EF7767.9050406@acm.org> Message-ID: <5.1.1.6.0.20070308141926.02b1b4d0@sparrow.telecommunity.com> At 11:49 PM 3/7/2007 -0800, Talin wrote: >Function decorators + argument decorators already provide everything >needed for a third-party developer to implement generic functions on >their own. In other words - you could create an add-on module that >provides the proper function decorators that would be capable of >interpreting argument types and building whatever kind of dispatch table >you wanted, without having any further support in the base language >other that what's already there now. > >I understand that Phillip J. Eby, whose experience with generic >functions is well known, is interested in having the ability to replace >the standard dispatcher. I believe that this requires no further effort >on our part, but I welcome corrections on that point. The above is correct. >So, to my mind, there are really only a few issues to be answered: > >1) Whether Python 3K should provide a 'standard' implementation of those >generic function decorators. > >2) if so, should it provide one with the initial release of Py3K (i.e. >3.0), or should we wait for various 3rd party modules to emerge, compete >in the marketplace, and then pick a winner to be blessed with 'official' >status. I think 3.1 or 3.2 are soon enough, and will give some time for competitors to appear and stabilize. >3) if a 'standard' implementation is to be included with the 3.0 >release, should it be an optimized, native-code dispatcher, or a 'pure >python' implementation. Moreover, will any changes to the Python >interpreter be required to get the best possible dispatch performance >with generic functions. This is a (somewhat) open question. There are some tradeoffs that need to be made between speed and dynamicity. My libraries (and Guido's tuple-caching prototype) assume that you are not (for example) changing class' __bases__ at runtime, in order to be able to cache things. Of course, if some 3.x version grew the ability to register Python-accessible callbacks for class or module changes, then we could implement a few other long-desired optimizations, and this tradeoff for generic functions would go away. >Note that this does not address PJE's 'defop' concerns, which (according >to my understanding) isn't so much about dispatch mechanisms as it is >about the ability to spread the definition of a generic function over >multiple source files, so that the methods that operate on a given class >can be grouped of that class's definition. The primary motivation is to >customize the behavior of a built-in function or operator on a given >type, without having to either define a new __special__ method, or >monkey patch the built-in function in some way. (Is this correct?) Yes and no. The primary motivation (for me, anyway) is to have a more usable syntax for defining overloads or generic function methods, that doesn't require you to create a dummy function name. Being able to possibly get rid of __special__ methods would be a nice bonus. (Of course, if we could get some "blocks" or anonymous functions, that might be another way to solve the problem. However, nobody has yet come up with a workable syntax for that.) From janssen at parc.com Thu Mar 8 20:54:26 2007 From: janssen at parc.com (Bill Janssen) Date: Thu, 8 Mar 2007 11:54:26 PST Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45EF7767.9050406@acm.org> References: <45EF7767.9050406@acm.org> Message-ID: <07Mar8.115427pst."57996"@synergy1.parc.xerox.com> > In any case, I would like to know, what is the current status of the > following issues: > > -- Generic Functions > -- Interfaces > -- Metaclass syntax > > It's not my intention to start a big thread about these I don't see how you can avoid it -- that's the reason there's no PEP. I'd be happy to take this discussion up again. For those who don't remember, I believe that the existing multiple-inheritance object system gives us a reasonable way to define standard sets of operations (interfaces) which are provided to new types through the existing inheritance system, not some additional side-tagging mechanism. I further believe that a standard set of base types (interfaces) should be designed and put into P3 as soon as possible, and that all the standard built-in types should be either one of these base types, or composed from several of them. This approach has apparently been identified as ABC, standing for "abstract base classes", though the amount of abstractness that's necessary is debatable -- I've seen schemes like this where the base classes include working default implementations of the underlying semantics. There's an incomplete wiki page about a possible factoring of types at http://wiki.python.org/moin/AbstractBaseClasses. I further believe that relying on so-called "duck typing" runtime partial information probes is a fragile and error-prone way to build software, and that it should be deprecated in favor of "supports this interface" tests. I've got no opinion (yet :-) on metaclass syntax. How do we make progress on getting this into Py3K? Bill From talin at acm.org Thu Mar 8 23:07:21 2007 From: talin at acm.org (Talin) Date: Thu, 08 Mar 2007 14:07:21 -0800 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <07Mar8.115427pst."57996"@synergy1.parc.xerox.com> References: <45EF7767.9050406@acm.org> <07Mar8.115427pst."57996"@synergy1.parc.xerox.com> Message-ID: <45F08919.5070808@acm.org> Bill Janssen wrote: >> In any case, I would like to know, what is the current status of the >> following issues: >> >> -- Generic Functions >> -- Interfaces >> -- Metaclass syntax >> >> It's not my intention to start a big thread about these > > I don't see how you can avoid it -- that's the reason there's no PEP. What I mean is, that I don't want to start the big thread *just yet*, and when I do, I want it to be in the proper place. Specifically, if there's a lot of unsettled issues and controversies, then probably we should move it to python-ideas; On the other hand, if it's just a matter of wrapping up a few details, then it should be taken up here. > I'd be happy to take this discussion up again. For those who don't > remember, I believe that the existing multiple-inheritance object > system gives us a reasonable way to define standard sets of operations > (interfaces) which are provided to new types through the existing > inheritance system, not some additional side-tagging mechanism. I > further believe that a standard set of base types (interfaces) should > be designed and put into P3 as soon as possible, and that all the > standard built-in types should be either one of these base types, or > composed from several of them. This approach has apparently been > identified as ABC, standing for "abstract base classes", though the > amount of abstractness that's necessary is debatable -- I've seen > schemes like this where the base classes include working default > implementations of the underlying semantics. > > There's an incomplete wiki page about a possible factoring of types at > http://wiki.python.org/moin/AbstractBaseClasses. > > I further believe that relying on so-called "duck typing" runtime > partial information probes is a fragile and error-prone way to build > software, and that it should be deprecated in favor of "supports this > interface" tests. I think that you need to be careful here in order that you aren't misunderstood. I believe what you are specifically against is the use of *introspective* duck typing - in other words, using duck typing to deduce certain characteristics of a class, without any a priori knowledge of that class; This is distinct from duck typing in the general case, where you are calling a method on an object, not introspecting it. The latter is an essential part of Python, and any attempt to deprecate it is not only likely to fail, but will probably elicit enough of a defensive reaction as to drown out your other points. > I've got no opinion (yet :-) on metaclass syntax. > > How do we make progress on getting this into Py3K? > > Bill > From talin at acm.org Thu Mar 8 23:36:42 2007 From: talin at acm.org (Talin) Date: Thu, 08 Mar 2007 14:36:42 -0800 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <45EF7767.9050406@acm.org> Message-ID: <45F08FFA.4090703@acm.org> Guido van Rossum wrote: > I don't think a consensus has been reached on any of these. > > My votes: > > - generic functions: no > - interfaces: no, but I'd like to work on ABCs instead > - metaclass syntax: I'd like to see your PEP I've gone back and read all of the threads on Metaclasses, and here is the current state of affairs: There are really two separate issues. First, there is a purely cosmetic argument about how metaclasses ought to be specified syntactically, which I won't go into in any detail. Most of the proposals centered around what I will call 'Class Definition Keyword Arguments' (CDKA), in other words keyword arguments that are passed in along with the list of base classes. Secondly, there is the desire to allow the metaclass to get involved earlier in the class construction process, allowing the metaclass to customize classes in ways that are more fundamental than what we can do today. The primary method for achieving this was envisioned to be a means to supply a custom, dictionary-like object that would be used to collect the class member definitions as the class was being evaluated. However, there was a great deal of controversy as to how that custom dictionary should itself come into existence. The first idea was that the metaclass should have some method for constructing the dictionary, which it would later receive after it was filled in with all of the class methods. One drawback to this approach was that it required a metaclass to be invoked twice, once to create the dictionary, and once to use that dictionary to create the class object. This required that the metaclass actually be a class, as opposed to the current state of affairs where a metaclass can be any callable. Some people felt that these two operations (creating the custom dictionary and invoking the metaclass) should be specified independently; However, since the design of the metaclass and the design of the dictionary will normally be tightly coupled, it makes no sense to require the user of the metaclass to specify both halves separately. Now, it was pointed out that the only use cases for a custom dictionary that anyone could think of were all about preserving the ordering of declarations. In other words, although a large number of use cases could be generated (C structs, COM objects, ORBs, IDLs, etc.) all of them had to do with interfacing to external systems that operated using an implicit ordering of fields. So the suggestion was made that instead of having the metaclass create a custom dictionary, simply *always* use a special 'order-preserving' dictionary when defining a class. In other words, there would be a subclass (or more likely, wrapper) of dict that kept a record of the insertion order of items, which would be retrieved via an itemorder() method or such. The built-in class construction code would use this special subclass of dict whenever it created a class, and any metaclass that was interested in knowing the order of definitions could find out. I'm also assuming that, for performance reasons, the order-preserving dict is not required to eliminate duplicates if a key is inserted multiple times; It's up to the metaclass to sort all that out, which in most cases will be trivial. With this scheme, there is no longer any need to change the syntax of metaclasses, although there may be other reasons for doing so. Unfortunately, this solution has two drawbacks. First, the order-preserving dict will be slightly slower than a regular dict, which in turn will cause a slight slowdown for the creation of all classes. Perhaps this could be turned off if there's no metaclass declared. Secondly, I believe that it is often the case where the dictionary that is used to collect the member definitions (before the metaclass is invoked) is the same as the one that is used to hold the class members after the metaclass is finished. At this point, you would want some way to disable the recording of insertions, otherwise each time you modified a class member you would be extending this record of insertions. I think there are enough open issues here that it will require discussion; The question is, what is the proper place for the discussion to take place. -- Talin From janssen at parc.com Thu Mar 8 23:43:21 2007 From: janssen at parc.com (Bill Janssen) Date: Thu, 8 Mar 2007 14:43:21 PST Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F08919.5070808@acm.org> References: <45EF7767.9050406@acm.org> <07Mar8.115427pst."57996"@synergy1.parc.xerox.com> <45F08919.5070808@acm.org> Message-ID: <07Mar8.144328pst."57996"@synergy1.parc.xerox.com> > This is distinct from duck typing in the > general case, where you are calling a method on an object, not > introspecting it. The latter is an essential part of Python, and any > attempt to deprecate it is not only likely to fail, but will probably > elicit enough of a defensive reaction as to drown out your other points. All right, I'll ponder your proposed distinction. I think you have it right. Calling a method on an object doesn't seem like duck-typing to me, though. Bill From pje at telecommunity.com Fri Mar 9 00:05:57 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 08 Mar 2007 18:05:57 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <07Mar8.115427pst."57996"@synergy1.parc.xerox.com> References: <45EF7767.9050406@acm.org> <45EF7767.9050406@acm.org> Message-ID: <5.1.1.6.0.20070308180338.04bb2a28@sparrow.telecommunity.com> At 11:54 AM 3/8/2007 -0800, Bill Janssen wrote: >I further believe that relying on so-called "duck typing" runtime >partial information probes is a fragile and error-prone way to build >software, and that it should be deprecated in favor of "supports this >interface" tests. And I believe that "supports this interface" *tests* are equally fragile and error-prone -- in a different sort of way. (I don't have as much reservation about generic functions dispatching based on interfaces, because this is overrideable. Hardcoded interface tests are *not* overrideable, and so are far more fragile.) But we've already had this discussion, I think. :) From talin at acm.org Fri Mar 9 00:18:42 2007 From: talin at acm.org (Talin) Date: Thu, 08 Mar 2007 15:18:42 -0800 Subject: [Python-3000] Cross-compatibility with 2.6 and 3.0 Message-ID: <45F099D2.6010804@acm.org> Guido's Py3K presentation of February 14th (available on Google Video at this URL: http://video.google.com/videoplay?docid=1189446823303316785) contains the following statement: "It is unlikely that you will be able to write both code that is both valid Python 2.6 source code and valid Python 3.0 source code". This seems like a big problem to me: From a systems management approach, its much easier to migrate a system if you can take a gradualist approach rather than being forced to maintain two versions of the code in parallel. It means that you can continue to test your working program as you update each module, rather than trying to do the whole thing in one gulp and then try to figure out what broke. I know that a lot of Python 3K feature will be back-ported to 2.X. What's missing? What stands in the way of me creating a version of my code that runs under both? For example, take the print statement: Clearly, the two uses of 'print' are incompatible. But nothing says that I actually have to use 'print' in my code - I can write a wrapper around the old 'print' that works just like the new 'print', as long as I name it something different than 'print'. Later, when I cut the tie to 2.X, I can do a mass rename of that function back to 'print'. Or, conversely, we could have a __future__ import that simply removes the 'print' keyword from the language. Similarly, the syntax of the 'except' statement is different - but I don't see why 2.6 couldn't support both forms; The presence of the 'as' keyword is a dead giveaway that you are using the new form. Or, again, a __future__ import could work as well. -- Talin From greg.ewing at canterbury.ac.nz Fri Mar 9 00:29:01 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 09 Mar 2007 12:29:01 +1300 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> References: <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> Message-ID: <45F09C3D.2020009@canterbury.ac.nz> Barry Warsaw wrote: > We already have an established, community accepted implementation of > interfaces, Really? Which one is that? -- Greg From rasky at develer.com Fri Mar 9 01:06:24 2007 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 09 Mar 2007 01:06:24 +0100 Subject: [Python-3000] Cross-compatibility with 2.6 and 3.0 In-Reply-To: <45F099D2.6010804@acm.org> References: <45F099D2.6010804@acm.org> Message-ID: On 09/03/2007 0.18, Talin wrote: > Guido's Py3K presentation of February 14th (available on Google Video at > this URL: http://video.google.com/videoplay?docid=1189446823303316785) > contains the following statement: > > "It is unlikely that you will be able to write both code that is both > valid Python 2.6 source code and valid Python 3.0 source code". > > This seems like a big problem to me: From a systems management approach, > its much easier to migrate a system if you can take a gradualist > approach rather than being forced to maintain two versions of the code > in parallel. [...] This was already discussed ad libitum: the point is that you will be able to write and maintain a valid Python 2.6 source code which can be automatically converted to valid Python 3.0 source code with the 2to3 refactoring tool. This is meant to be a good enough compromise. [ Of course, not *every* Python 2.6 valid program can be converted with 2to3, but the goal is making people able to "adjust" their 2.6 source code up to the point that it works with 2to3 ] -- Giovanni Bajo From greg.ewing at canterbury.ac.nz Fri Mar 9 01:41:18 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 09 Mar 2007 13:41:18 +1300 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <07Mar8.144328pst.57996@synergy1.parc.xerox.com> References: <45EF7767.9050406@acm.org> <07Mar8.115427pst.57996@synergy1.parc.xerox.com> <45F08919.5070808@acm.org> <07Mar8.144328pst.57996@synergy1.parc.xerox.com> Message-ID: <45F0AD2E.3@canterbury.ac.nz> Bill Janssen wrote: > Calling a method on an object doesn't seem like duck-typing to > me, though. It's duck typing in the sense that you don't have to formally declare the object as conforming to some interface before you can call the method. -- Greg From collinw at gmail.com Fri Mar 9 04:03:09 2007 From: collinw at gmail.com (Collin Winter) Date: Thu, 8 Mar 2007 21:03:09 -0600 Subject: [Python-3000] Removing sys.exitfunc Message-ID: <43aa6ff70703081903m5f747772qc0d4820fdf4dcb0@mail.gmail.com> I was revisiting my patch on SF to remove sys.exitfunc (per PEP 3100), and I found some usages in the stdlib I had missed the first time around. One of these, I was surprised to learn, is atexit.py; in fact, atexit is implemented in terms of sys.exitfunc. This was especially shocking because PEP 3100 cites that you should "use [the] atexit module instead" of sys.exitfunc as an argument for the latter's deletion. Suggestions, anyone, on how to remove sys.exitfunc and keep atexit working? My own would be to replace atexit.py with a C module that allows manipulation of the exitfuncs array in Python/pythonrun.c (or something similar). Collin Winter From guido at python.org Fri Mar 9 04:09:25 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 8 Mar 2007 19:09:25 -0800 Subject: [Python-3000] Removing sys.exitfunc In-Reply-To: <43aa6ff70703081903m5f747772qc0d4820fdf4dcb0@mail.gmail.com> References: <43aa6ff70703081903m5f747772qc0d4820fdf4dcb0@mail.gmail.com> Message-ID: Rename it to sys._exitfunc? On 3/8/07, Collin Winter wrote: > I was revisiting my patch on SF to remove sys.exitfunc (per PEP 3100), > and I found some usages in the stdlib I had missed the first time > around. One of these, I was surprised to learn, is atexit.py; in fact, > atexit is implemented in terms of sys.exitfunc. This was especially > shocking because PEP 3100 cites that you should "use [the] atexit > module instead" of sys.exitfunc as an argument for the latter's > deletion. > > Suggestions, anyone, on how to remove sys.exitfunc and keep atexit > working? My own would be to replace atexit.py with a C module that > allows manipulation of the exitfuncs array in Python/pythonrun.c (or > something similar). > > Collin Winter > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Fri Mar 9 06:56:11 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 9 Mar 2007 00:56:11 -0500 Subject: [Python-3000] Discussions with no PEPs References: <45EF7767.9050406@acm.org><2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> Message-ID: "Greg Ewing" wrote in message news:45F09C3D.2020009 at canterbury.ac.nz... | Barry Warsaw wrote: | | > We already have an established, community accepted implementation of | > interfaces, | | Really? Which one is that? I was wondering too. Has there been a concrete PEPed proposal that I missed? tjr From jimjjewett at gmail.com Fri Mar 9 15:49:22 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 9 Mar 2007 09:49:22 -0500 Subject: [Python-3000] metaclass syntax [was: Discussions with no PEPs] Message-ID: On 3/8/07, Talin wrote: > First, there is a purely cosmetic argument about how metaclasses ought > to be specified syntactically, which I won't go into in any detail. Most > of the proposals centered around what I will call 'Class Definition > Keyword Arguments' (CDKA), in other words keyword arguments that are > passed in along with the list of base classes. You need to in the PEP though, particularly since class decorators are now available. (These remove the need for some of the existing metaclass usage.) > ... a means to > supply a custom, dictionary-like object that would be used to collect > the class member definitions as the class was being evaluated. ... > Now, it was pointed out that the only use cases for a custom dictionary > that anyone could think of were all about preserving the ordering of > declarations. Not quite true. (1) immutable class dictionaries. These are typical for extension classes, but a real pain for python classes. (2) almost-immutable -- with callbacks when values are added/changed/deleted. PJE just pointed out that doing this on even just the __bases__ attribute could make generic functions safer. (3) actually-private variables -jJ From guido at python.org Fri Mar 9 16:06:56 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 9 Mar 2007 07:06:56 -0800 Subject: [Python-3000] metaclass syntax [was: Discussions with no PEPs] In-Reply-To: References: Message-ID: Those look like use cases for metaclasses, but I don't see how they require setting a custom dict *while the class suite is being executed*. The metaclass can create a new dict from its dict argument and use the new dict to construct the class. On 3/9/07, Jim Jewett wrote: > On 3/8/07, Talin wrote: > > > First, there is a purely cosmetic argument about how metaclasses ought > > to be specified syntactically, which I won't go into in any detail. Most > > of the proposals centered around what I will call 'Class Definition > > Keyword Arguments' (CDKA), in other words keyword arguments that are > > passed in along with the list of base classes. > > You need to in the PEP though, particularly since class decorators are > now available. (These remove the need for some of the existing > metaclass usage.) > > > ... a means to > > supply a custom, dictionary-like object that would be used to collect > > the class member definitions as the class was being evaluated. > > ... > > > Now, it was pointed out that the only use cases for a custom dictionary > > that anyone could think of were all about preserving the ordering of > > declarations. > > Not quite true. > > (1) immutable class dictionaries. These are typical for extension > classes, but a real pain for python classes. > > (2) almost-immutable -- with callbacks when values are > added/changed/deleted. PJE just pointed out that doing this on even > just the __bases__ attribute could make generic functions safer. > > (3) actually-private variables > > -jJ > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Mar 9 17:15:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 09 Mar 2007 11:15:15 -0500 Subject: [Python-3000] metaclass syntax [was: Discussions with no PEPs] In-Reply-To: References: Message-ID: <5.1.1.6.0.20070309110812.02ac41d8@sparrow.telecommunity.com> At 07:06 AM 3/9/2007 -0800, Guido van Rossum wrote: >Those look like use cases for metaclasses, but I don't see how they >require setting a custom dict *while the class suite is being >executed*. The main use case for that is to know what order the items were defined in; currently there's no way to achieve that without say, abusing the debugger hook. There are other interesting use cases that could be achieved with a custom dictionary, too, now that I'm thinking about it, that currently require at least some level of bytecode hacking or string manipulation. For example, there's a Python Cookbook recipe that does propositional logic and works by exec'ing code with a custom dictionary after figuring out what names are being used by a function body. With a custom dictionary for the class body, that could be done without exec or bytecode inspection. I personally can see the possibility of using the feature for implementing database schema definition forward references (where you need to be able to refer to objects that don't yet exist) in a class body. From guido at python.org Fri Mar 9 17:30:00 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 9 Mar 2007 08:30:00 -0800 Subject: [Python-3000] metaclass syntax [was: Discussions with no PEPs] In-Reply-To: <5.1.1.6.0.20070309110812.02ac41d8@sparrow.telecommunity.com> References: <5.1.1.6.0.20070309110812.02ac41d8@sparrow.telecommunity.com> Message-ID: On 3/9/07, Phillip J. Eby wrote: > At 07:06 AM 3/9/2007 -0800, Guido van Rossum wrote: > >Those look like use cases for metaclasses, but I don't see how they > >require setting a custom dict *while the class suite is being > >executed*. > > The main use case for that is to know what order the items were defined in; > currently there's no way to achieve that without say, abusing the debugger > hook. I agree, and I have a proposed solution (which Talin knows and will turn into a PEP). I was just noting that Jim, in a response to Talin's claim that the only use case for a custom dict was to know the declaration order, was replying with use cases for metaclasses that could be solved without a custom dict, hence he wasn't refuting Talin's claim (as he seemed to think he was). > There are other interesting use cases that could be achieved with a custom > dictionary, too, now that I'm thinking about it, that currently require at > least some level of bytecode hacking or string manipulation. For example, > there's a Python Cookbook recipe that does propositional logic and works by > exec'ing code with a custom dictionary after figuring out what names are > being used by a function body. With a custom dictionary for the class > body, that could be done without exec or bytecode inspection. > > I personally can see the possibility of using the feature for implementing > database schema definition forward references (where you need to be able to > refer to objects that don't yet exist) in a class body. Don't worry. This will be possible. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Fri Mar 9 18:35:59 2007 From: janssen at parc.com (Bill Janssen) Date: Fri, 9 Mar 2007 09:35:59 PST Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F0AD2E.3@canterbury.ac.nz> References: <45EF7767.9050406@acm.org> <07Mar8.115427pst.57996@synergy1.parc.xerox.com> <45F08919.5070808@acm.org> <07Mar8.144328pst.57996@synergy1.parc.xerox.com> <45F0AD2E.3@canterbury.ac.nz> Message-ID: <07Mar9.093604pst."57996"@synergy1.parc.xerox.com> > Bill Janssen wrote: > > Calling a method on an object doesn't seem like duck-typing to > > me, though. > > It's duck typing in the sense that you don't have to > formally declare the object as conforming to some > interface before you can call the method. Oh, OK. I'm against (mandatory) type declarations. However, I do believe that the object's type should have to include the expected interface for the method invocation to succeed. I'm not arguing for mandatory enforcement of that "should" in Python itself, but rather for a clear language so that the provider of a value and the user of a value can communicate about what's expected and/or required by the logic of the program. Bill From talin at acm.org Fri Mar 9 20:36:03 2007 From: talin at acm.org (Talin) Date: Fri, 09 Mar 2007 11:36:03 -0800 Subject: [Python-3000] generics [was: Discussions with no PEPs] Message-ID: <45F1B723.8020402@acm.org> Here's my proposal for what additional work should be done with respect to generic functions in Python 3000. 1) In the Python 3000 branch, do nothing at the current time. The 3.0 branch already has sufficient hooks to allow 3rd-party developers to create their own generic function implementations. 2) Backport the argument annotations patch to the 2.X series and get it out to the public as quickly as possible. (According to the conversation I had with Guido, this should be relatively easy since the annotations patch was one of the first patches to diverge between 2.X and 3.0). The reason for doing this is to let the wider community start working with function annotations and developing various dispatcher implementations now, rather than later. Of course, 2.6 isn't going to be coming out much sooner than 3.0 alpha, if at all - but its likely that 2.6 will be used by a much wider audience, whereas 3.0a is likely to only be used by early adopters. The release would also include a note that encourages people to experiment with this feature - letting them know that there's a vacuum here that needs to be filled. What I want to see is a number of different generic function implementations that are competing for mind-share in the Python community. 3) At some point, pick the best / most popular generic function package and formally make it a part of Python 3. -- Talin From talin at acm.org Fri Mar 9 21:44:36 2007 From: talin at acm.org (Talin) Date: Fri, 09 Mar 2007 12:44:36 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 Message-ID: <45F1C734.7080503@acm.org> I had a conversation with Guido last night at the Python user's group meeting, and we hashed out some of the details of how metaclasses should work. I've gone ahead and written up a PEP, which I present for your review. -------------------------------------------- PEP: xxx Title: Metaclasses in Python 3000 Version: $Revision$ Last-Modified: $Date$ Author: Talin Status: Draft Type: Standards Content-Type: text/plain Created: 07-Mar-2007 Python-Version: 3.0 Post-History: Abstract This PEP proposes changing the syntax for declaring metaclasses, and alters the semantics for how classes with metaclasses are constructed. Rationale There are two rationales for this PEP, both of which are somewhat subtle. The primary reason for changing the way metaclasses work, is that there are a number of interesting use cases that require the metaclass to get involved earlier in the class construction process than is currently possible. Currently, the metaclass mechanism is essentially a post-processing step. With the advent of class decorators, much of these post-processing chores can be taken over by the decorator mechanism. In particular, there is an important body of use cases where it would be useful to preserve the order in which a class members are declared. Ordinary Python objects store their members in a dictionary, in which ordering is unimportant, and members are accessed strictly by name. However, Python is often used to interface with external systems in which the members are organized according to an implicit ordering. Examples include declaration of C structs; COM objects; Automatic translation of Python classes into IDL or database schemas, such as used in an ORM; and so on. In such cases, it would be useful for a Python programmer to specify such ordering directly using the declaration order of class members. Currently, such orderings must be specified explicitly, using some other mechanism (see the ctypes module for an example.) Unfortunately, the current method for declaring a metaclass does not allow for this, since the ordering information has already been lost by the time the metaclass comes into play. By allowing the metaclass to get involved in the class construction process earlier, the new system allows the ordering or other early artifacts of construction to be preserved and examined. The other, weaker, rationale is purely cosmetic: The current method for specifying a metaclass is by assignment to the special variable __metaclass__, which is considered by some to be aesthetically less than ideal. Others disagree strongly with that opinion. This PEP will not address this issue, other than to note it, since aesthetic debates cannot be resolved via logically proofs. Specification In the new model, the syntax for specifying a metaclass is via a keyword argument in the list of base classes: class Foo(base1, base2, metaclass=mymeta): ... Additional keywords will also be allowed here, and will be passed to the metaclass, as in the following example: class Foo(base1, base2, metaclass=mymeta, private=True): ... Note that this PEP makes no attempt to define what these other keywords might be - that is up to metaclass implementors to determine. Invoking the Metaclass In the current metaclass system, the metaclass object can be any callable type. This does not change, however in order to fully exploit all of the new features, the metaclass will need to have an extra attribute which is used during class pre-construction. This attribute is a method named __metacreate__, which is invoked before the evaluation of the class body, and which has the following form: classdict = metaclass.__metacreate__(name, bases, keywords) Where: 'name' is the name of the class being created. 'bases' is the list of base classes. 'keywords' is the dictionary of keywords in the base class list. 'classdict' is a custom dictionary object which is created by the metaclass, and which is used to store the class members as they are declared. Note that the Python interpreter will check to insure that the __metacreate__ attribute exists before calling it. This preserves backwards compatibility with existing metaclasses. The 'classdict' object can be a regular dictionary or a custom mapping type. It does not need to implement the full dictionary interface; only the ability to insert items and retrieve them are required. (Note: double check that this is true). When the body of the class is evaluated, the dictionary will be used as the 'locals()' dict for that evaluation. Once the class body has finished evaluating, the metaclass will be called (as a callable) with the class dictionary, which is no different from the current metaclass mechanism. Typically, a metaclass will create a custom dictionary - either a subclass of dict, or a wrapper around it - that will contain additional properties that are set either before or during the evaluation of the class body. Then in the second phase, the metaclass can use these additional properties to further customize the class. An example would be a metaclass that uses information about the ordering of member declarations to create a C struct. The metaclass would provide a custom dictionary that simply keeps a record of the order of insertions. This does not need to be a full 'ordered dict' implementation, but rather just a Python list of (key,value) pairs that is appended to for each insertion. Note that in such a case, the metaclass would be required to deal with the possibility of duplicate keys, but in most cases that is trivial. The metaclass can use the first declaration, the last, combine them in some fashion, or simply throw an exception. It's up to the metaclass to decide how it wants to handle that case. Alternate Proposals Josiah Carlson proposed using the name 'type' instead of 'metaclass', on the theory that what is really being specified is the type of the type. While this is technically correct, it is also confusing from the point of view of a programmer creating a new class. From the application programmer's point of view, the 'type' that they are interested in is the class that they are writing; the type of that type is the metaclass. There were some objections in the discussion to the 'two-phase' creation process, where the metaclass is invoked twice, once to create the class dictionary and once to 'finish' the class. Some people felt that these two phases should be completely separate, in that there ought to be separate syntax for specifying the custom dict as for specifying the metaclass. However, in most cases, the two will be intimately tied together, and the metaclass will most likely have an intimate knowledge of the internal details of the class dict. Requiring the programmer to insure that the correct dict type and the correct metaclass type are used together creates an additional and unneeded burden on the programmer. Another good suggestion was to simply use an ordered dict for all classes, and skip the whole 'custom dict' mechanism. This was based on the observation that most use cases for a custom dict were for the purposes of preserving order information. However, this idea has two drawbacks, first because it means that an ordered dict implementation would have to be added to the set of built-in types in Python, and second because it would impose a slight speed (and complexity) penalty on all class declarations. Backwards Compatibility It would be possible to leave the existing __metaclass__ syntax in place. Alternatively, it would not be too difficult to modify the syntax rules of the Py3K translation tool to convert from the old to the new syntax. References [1] [Python-3000] Metaclasses in Py3K (original proposal) http://mail.python.org/pipermail/python-3000/2006-December/005030.html [2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax) http://mail.python.org/pipermail/python-3000/2006-December/005033.html [3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init) http://mail.python.org/pipermail/python-3000/2006-December/005108.html [4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict) http://mail.python.org/pipermail/python-3000/2006-December/005118.html [5] PEP 359: The 'make' statement - http://www.python.org/dev/peps/pep-0359/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From brett at python.org Fri Mar 9 22:21:49 2007 From: brett at python.org (Brett Cannon) Date: Fri, 9 Mar 2007 13:21:49 -0800 Subject: [Python-3000] PEP Draft: Class Decorators In-Reply-To: <43aa6ff70702281415o2c7ccd75n7fb3db167506abfe@mail.gmail.com> References: <20070228205212.GD5537@performancedrivers.com> <43aa6ff70702281415o2c7ccd75n7fb3db167506abfe@mail.gmail.com> Message-ID: On 2/28/07, Collin Winter wrote: > On 2/28/07, Jack Diederich wrote: > [snip] > > History and Implementation > > ========================== > > > > Class decorators were originally proposed in PEP318 [1]_ and were rejected > > by Guido [2]_ for lack of use cases. Two years later he saw a use case > > he liked and gave the go-ahead for a PEP and patch [3]_. > > While I can look up the use-case that prompted Guido to change his > mind via the footnote, I'd appreciate having a sampling of use-cases > listed in the PEP itself. > It would also help as there is no explicit mention of what the decorator gets passed (I assume the class object, but it is actually not specified anywhere). Some pseudo-code of what exactly is happening wouldn't hurt to more concretely specify it. And you probably want to submit this to the PEP editors to get it checked in as Talin keeps talking about class decorators as if they are already in Py3K. =) -Brett From steven.bethard at gmail.com Fri Mar 9 22:23:39 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 9 Mar 2007 14:23:39 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F1C734.7080503@acm.org> References: <45F1C734.7080503@acm.org> Message-ID: On 3/9/07, Talin wrote: > PEP: xxx > Title: Metaclasses in Python 3000 Thanks for writing this. > This attribute is a method named __metacreate__, which is invoked > before the evaluation of the class body, and which has the > following form: > > classdict = metaclass.__metacreate__(name, bases, keywords) > [snip] > There were some objections in the discussion to the 'two-phase' > creation process, where the metaclass is invoked twice, once to > create the class dictionary and once to 'finish' the class. Some > people felt that these two phases should be completely separate, in > that there ought to be separate syntax for specifying the custom > dict as for specifying the metaclass. However, in most cases, the > two will be intimately tied together, and the metaclass will most > likely have an intimate knowledge of the internal details of the > class dict. Requiring the programmer to insure that the correct dict > type and the correct metaclass type are used together creates an > additional and unneeded burden on the programmer. I think it would really help to give some example code so that people can see why one way is easier than the other. I gather for using an ordered dict, the two alternatives would look something like:: # using __metacreate__ class/static method class meta(type): @staticmethod def __metacreate__(name, bases, keywords): return ordereddict() def __init__(cls, name, bases, bodydict): for key, value in bodydict.items(): # I know this is in order ... class C(metaclass=meta): ... # using separate callables def meta(name, bases, bodydict): for key, value in bodydict.items(): # I know this is in order ... class C(metaclass=meta, bodydict=ordereddict): ... But it would be nice to fill in those ellipses with code that does something useful. I'd rather __metacreate__ be called something like __createdict__, __creationdict__, __metadict__, __getmetadict__, etc. where it's clearer that the purpose is to create a dict object. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Fri Mar 9 22:31:02 2007 From: brett at python.org (Brett Cannon) Date: Fri, 9 Mar 2007 13:31:02 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F1C734.7080503@acm.org> References: <45F1C734.7080503@acm.org> Message-ID: On 3/9/07, Talin wrote: > I had a conversation with Guido last night at the Python user's group > meeting, and we hashed out some of the details of how metaclasses should > work. I've gone ahead and written up a PEP, which I present for your review. > -------------------------------------------- > PEP: xxx > Title: Metaclasses in Python 3000 > Version: $Revision$ > Last-Modified: $Date$ > Author: Talin > Status: Draft > Type: Standards > Content-Type: text/plain > Created: 07-Mar-2007 > Python-Version: 3.0 > Post-History: > > Abstract > > This PEP proposes changing the syntax for declaring metaclasses, > and alters the semantics for how classes with metaclasses are > constructed. > > > Rationale > > There are two rationales for this PEP, both of which are somewhat > subtle. > > The primary reason for changing the way metaclasses work, is that > there are a number of interesting use cases that require the > metaclass to get involved earlier in the class construction process > than is currently possible. Currently, the metaclass mechanism is > essentially a post-processing step. With the advent of class > decorators, much of these post-processing chores can be taken over > by the decorator mechanism. > > In particular, there is an important body of use cases where it > would be useful to preserve the order in which a class members are > declared. Ordinary Python objects store their members in a > dictionary, in which ordering is unimportant, and members are > accessed strictly by name. However, Python is often used to > interface with external systems in which the members are organized > according to an implicit ordering. Examples include declaration of C > structs; COM objects; Automatic translation of Python classes into > IDL or database schemas, such as used in an ORM; and so on. > > In such cases, it would be useful for a Python programmer to specify > such ordering directly using the declaration order of class members. > Currently, such orderings must be specified explicitly, using some > other mechanism (see the ctypes module for an example.) > > Unfortunately, the current method for declaring a metaclass does > not allow for this, since the ordering information has already been > lost by the time the metaclass comes into play. By allowing the > metaclass to get involved in the class construction process earlier, > the new system allows the ordering or other early artifacts of > construction to be preserved and examined. > > The other, weaker, rationale is purely cosmetic: The current method > for specifying a metaclass is by assignment to the special variable > __metaclass__, which is considered by some to be aesthetically less > than ideal. Others disagree strongly with that opinion. This PEP > will not address this issue, other than to note it, since aesthetic > debates cannot be resolved via logically proofs. I think you mean "via logical proofs" or "logically via proofs". > > > Specification > > In the new model, the syntax for specifying a metaclass is via a > keyword argument in the list of base classes: > > class Foo(base1, base2, metaclass=mymeta): > ... > > Additional keywords will also be allowed here, and will be passed to > the metaclass, as in the following example: > > class Foo(base1, base2, metaclass=mymeta, private=True): > ... > > Note that this PEP makes no attempt to define what these other > keywords might be - that is up to metaclass implementors to > determine. > Do the keywords have to follow the metaclass keyword, or is order irrelevant? While order makes sense, it would be a new precedent for keyword arguments to have an important order. > Invoking the Metaclass > > In the current metaclass system, the metaclass object can be any > callable type. This does not change, however in order to fully > exploit all of the new features, the metaclass will need to have an > extra attribute which is used during class pre-construction. > That last sentence felt a little clumsy. I think if you ditch that last comma it reads more easily. > This attribute is a method named __metacreate__, which is invoked > before the evaluation of the class body, and which has the > following form: > > classdict = metaclass.__metacreate__(name, bases, keywords) > > Where: > > 'name' is the name of the class being created. > 'bases' is the list of base classes. > 'keywords' is the dictionary of keywords in the base class list. > 'classdict' is a custom dictionary object which is created by the > metaclass, and which is used to store the class members as > they are declared. > > Note that the Python interpreter will check to insure that the > __metacreate__ attribute exists before calling it. This preserves > backwards compatibility with existing metaclasses. > > The 'classdict' object can be a regular dictionary or a custom > mapping type. It does not need to implement the full dictionary > interface; only the ability to insert items and retrieve them are > required. (Note: double check that this is true). When the body of > the class is evaluated, the dictionary will be used as the > 'locals()' dict for that evaluation. > > Once the class body has finished evaluating, the metaclass will be > called (as a callable) with the class dictionary, which is no > different from the current metaclass mechanism. > > Typically, a metaclass will create a custom dictionary - either a > subclass of dict, or a wrapper around it - that will contain > additional properties that are set either before or during the > evaluation of the class body. Then in the second phase, the > metaclass can use these additional properties to further customize > the class. > > An example would be a metaclass that uses information about the > ordering of member declarations to create a C struct. The metaclass > would provide a custom dictionary that simply keeps a record of the > order of insertions. This does not need to be a full 'ordered dict' > implementation, but rather just a Python list of (key,value) pairs > that is appended to for each insertion. > Does the language spec guarantee that the body of a class will be executed in definition order? Or is that considered implicit by the fact that the class body is executed as code? > Note that in such a case, the metaclass would be required to deal > with the possibility of duplicate keys, but in most cases that is > trivial. The metaclass can use the first declaration, the last, > combine them in some fashion, or simply throw an exception. It's up > to the metaclass to decide how it wants to handle that case. > > > Alternate Proposals > > Josiah Carlson proposed using the name 'type' instead of > 'metaclass', on the theory that what is really being specified is > the type of the type. While this is technically correct, it is also > confusing from the point of view of a programmer creating a new > class. From the application programmer's point of view, the 'type' > that they are interested in is the class that they are writing; the > type of that type is the metaclass. > > There were some objections in the discussion to the 'two-phase' > creation process, where the metaclass is invoked twice, once to > create the class dictionary and once to 'finish' the class. Some > people felt that these two phases should be completely separate, in > that there ought to be separate syntax for specifying the custom > dict as for specifying the metaclass. However, in most cases, the > two will be intimately tied together, and the metaclass will most > likely have an intimate knowledge of the internal details of the > class dict. Requiring the programmer to insure that the correct dict > type and the correct metaclass type are used together creates an > additional and unneeded burden on the programmer. > > Another good suggestion was to simply use an ordered dict for all > classes, and skip the whole 'custom dict' mechanism. This was based > on the observation that most use cases for a custom dict were for > the purposes of preserving order information. However, this idea has > two drawbacks, first because it means that an ordered dict > implementation would have to be added to the set of built-in types > in Python, and second because it would impose a slight speed (and > complexity) penalty on all class declarations. > > > Backwards Compatibility > > It would be possible to leave the existing __metaclass__ syntax in > place. Alternatively, it would not be too difficult to modify the > syntax rules of the Py3K translation tool to convert from the old to > the new syntax. > > > References > > [1] [Python-3000] Metaclasses in Py3K (original proposal) > > http://mail.python.org/pipermail/python-3000/2006-December/005030.html > > [2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax) > > http://mail.python.org/pipermail/python-3000/2006-December/005033.html > > [3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init) > > http://mail.python.org/pipermail/python-3000/2006-December/005108.html > > [4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict) > > http://mail.python.org/pipermail/python-3000/2006-December/005118.html > > [5] PEP 359: The 'make' statement - > http://www.python.org/dev/peps/pep-0359/ > > Copyright > > This document has been placed in the public domain. Seems good, although I hardly ever use metaclasses so that doesn't say a whole lot. =) -Brett From tony at PageDNA.com Fri Mar 9 22:15:24 2007 From: tony at PageDNA.com (Tony Lownds) Date: Fri, 9 Mar 2007 13:15:24 -0800 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: <45F1B723.8020402@acm.org> References: <45F1B723.8020402@acm.org> Message-ID: <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> On Mar 9, 2007, at 11:36 AM, Talin wrote: > 2) Backport the argument annotations patch to the 2.X series and > get it > out to the public as quickly as possible. (According to the > conversation > I had with Guido, this should be relatively easy since the annotations > patch was one of the first patches to diverge between 2.X and 3.0). Do you think that there will be support for this? Annotations seem like a big new feature for Python, more suited to Python 3.X. With tuple parameters removed, there would still be divergence in 2.X. By the way, I would like to work on either removing tuple parameters in 3.0 or fixing the assertion failure currently in CVS. -Tony From pmaupin at gmail.com Fri Mar 9 22:38:52 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Fri, 9 Mar 2007 15:38:52 -0600 Subject: [Python-3000] [Python-Dev] Policy Decisions, Judgment Calls, and Backwards Compatibility (was Re: splitext('.cshrc')) In-Reply-To: <5.1.1.6.0.20070309154617.04a23ec8@sparrow.telecommunity.com> References: <20070306233946.6305.746015806.divmod.xquotient.27@joule.divmod.com> <20070308045229.GQ14306@steerpike.home.puzzling.org> <20070307214117.74CE.JCARLSON@uci.edu> <45EFEB29.7080403@v.loewis.de> <20070309003135.7769.1063793143.divmod.xquotient.177@joule.divmod.com> <5.1.1.6.0.20070309111942.02ac4090@sparrow.telecommunity.com> <45F1A4EF.3030109@v.loewis.de> <5.1.1.6.0.20070309154617.04a23ec8@sparrow.telecommunity.com> Message-ID: On 3/9/07, Phillip J. Eby wrote: > At 07:18 PM 3/9/2007 +0100, Martin v. L?wis wrote: > >Phillip J. Eby schrieb: > The only group the change benefits is people writing *new* code -- so give > them a new function. Actually, it's worse than that. The only group the change benefits is people writing *new* code who *never* want or need it to run on older Python versions. Anybody who wants their code to work across multiple versions now needs to avoid this function like the plague, because it will be different. Not better, not worse, just different. Also, I think there is more potential breakage than some people realize. There are *lots* of directories which are named ".something" and splitext is probably used in parsing these out more often than most people realize. This is not a corner case -- there is probably a ton of code out there relying on the current behavior, which, as you point out, is explicitly documented. Regards, Pat From guido at python.org Fri Mar 9 23:35:29 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 9 Mar 2007 14:35:29 -0800 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: On 3/9/07, Tony Lownds wrote: > > On Mar 9, 2007, at 11:36 AM, Talin wrote: > > 2) Backport the argument annotations patch to the 2.X series and > > get it > > out to the public as quickly as possible. (According to the > > conversation > > I had with Guido, this should be relatively easy since the annotations > > patch was one of the first patches to diverge between 2.X and 3.0). > > Do you think that there will be support for this? Annotations seem > like a > big new feature for Python, more suited to Python 3.X. > > With tuple parameters removed, there would still be divergence in 2.X. But not much; we could drop the annotation feature for tuple parameters (since it won't survive in 3.0) and warn about tuple params if the -Wpy3k flag is given. > By the way, I would like to work on either removing tuple parameters > in 3.0 That would be great! > or fixing the assertion failure currently in CVS. Perhaps you should switch to SVN? :-) More seriously, what assertion failure? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Fri Mar 9 23:59:30 2007 From: collinw at gmail.com (Collin Winter) Date: Fri, 9 Mar 2007 16:59:30 -0600 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: <43aa6ff70703091459j25431e9rad15a7da0de0f244@mail.gmail.com> On 3/9/07, Guido van Rossum wrote: > On 3/9/07, Tony Lownds wrote: > > > > On Mar 9, 2007, at 11:36 AM, Talin wrote: > > > 2) Backport the argument annotations patch to the 2.X series and > > > get it > > > out to the public as quickly as possible. (According to the > > > conversation > > > I had with Guido, this should be relatively easy since the annotations > > > patch was one of the first patches to diverge between 2.X and 3.0). > > > > Do you think that there will be support for this? Annotations seem > > like a > > big new feature for Python, more suited to Python 3.X. > > > > With tuple parameters removed, there would still be divergence in 2.X. > > But not much; we could drop the annotation feature for tuple > parameters (since it won't survive in 3.0) and warn about tuple params > if the -Wpy3k flag is given. This sounds right to me. Collin Winter From thomas at python.org Sat Mar 10 01:00:01 2007 From: thomas at python.org (Thomas Wouters) Date: Sat, 10 Mar 2007 01:00:01 +0100 Subject: [Python-3000] Cross-compatibility with 2.6 and 3.0 In-Reply-To: <45F099D2.6010804@acm.org> References: <45F099D2.6010804@acm.org> Message-ID: <9e804ac0703091600s1509e567v83ac27f7817a8399@mail.gmail.com> On 3/9/07, Talin wrote: > "It is unlikely that you will be able to write both code that is both > valid Python 2.6 source code and valid Python 3.0 source code". Please see http://mail.python.org/pipermail/python-dev/2007-February/071372.html (and subsequent messages, if you want.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070310/07170314/attachment.htm From jackdied at jackdied.com Sat Mar 10 01:01:01 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Fri, 9 Mar 2007 19:01:01 -0500 Subject: [Python-3000] PEP Draft: Class Decorators In-Reply-To: References: <20070228205212.GD5537@performancedrivers.com> <43aa6ff70702281415o2c7ccd75n7fb3db167506abfe@mail.gmail.com> Message-ID: <20070310000101.GM18179@performancedrivers.com> On Fri, Mar 09, 2007 at 01:21:49PM -0800, Brett Cannon wrote: > On 2/28/07, Collin Winter wrote: > >On 2/28/07, Jack Diederich wrote: > >[snip] > >> History and Implementation > >> ========================== > >> > >> Class decorators were originally proposed in PEP318 [1]_ and were > >rejected > >> by Guido [2]_ for lack of use cases. Two years later he saw a use case > >> he liked and gave the go-ahead for a PEP and patch [3]_. > > > >While I can look up the use-case that prompted Guido to change his > >mind via the footnote, I'd appreciate having a sampling of use-cases > >listed in the PEP itself. > > > > It would also help as there is no explicit mention of what the > decorator gets passed (I assume the class object, but it is actually > not specified anywhere). Some pseudo-code of what exactly is > happening wouldn't hurt to more concretely specify it. > > And you probably want to submit this to the PEP editors to get it > checked in as Talin keeps talking about class decorators as if they > are already in Py3K. =) Yes, Talin's metaclass post lit a fire under by behind. New version with all the feedback I got getting posted in five .. -Jack From tony at PageDNA.com Sat Mar 10 01:10:06 2007 From: tony at PageDNA.com (Tony Lownds) Date: Fri, 9 Mar 2007 16:10:06 -0800 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: <87DA7784-E9C9-4647-B5A0-6ED36A169696@PageDNA.com> On Mar 9, 2007, at 2:35 PM, Guido van Rossum wrote: > On 3/9/07, Tony Lownds wrote: >> With tuple parameters removed, there would still be divergence in >> 2.X. > > But not much; we could drop the annotation feature for tuple > parameters (since it won't survive in 3.0) and warn about tuple params > if the -Wpy3k flag is given. That would work >> By the way, I would like to work on either removing tuple parameters >> in 3.0 > > That would be great! Ok > >> or fixing the assertion failure currently in CVS. > > Perhaps you should switch to SVN? :-) Heh, clearly I am behind the times. I should switch to Mercurial to get ahead of the curve. > > More seriously, what assertion failure? This one, noted recently by Brett: >>> def f((a: int, b: int)): pass ... python: Python/compile.c:2430: compiler_nameop: Assertion `scope || (((PyStringObject *)(name))->ob_sval)[0] == '_'' failed. Abort (core dumped) It's moot with tuple parameters removed. -Tony ps. sorry for the duplicate. From jackdied at jackdied.com Sat Mar 10 01:10:54 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Fri, 9 Mar 2007 19:10:54 -0500 Subject: [Python-3000] PEP Draft: Class Decorators In-Reply-To: <20070228205212.GD5537@performancedrivers.com> References: <20070228205212.GD5537@performancedrivers.com> Message-ID: <20070310001054.GN18179@performancedrivers.com> I got lots of feedback on the PEP at PyCon and the two main responses were "heck yeah!" and "what does a class decorator look like?" I've added some simple examples for the "what is?" and some other examples that should help explain the "heck yeah" responses too. I'm cc'ing peps at python.org for a number assignment (or can I just check it in myself?) PEP: 3XXX Title: Class Decorators Version: 1 Last-Modified: 28-Feb-2007 Authors: Jack Diederich Implementation: SF#1671208 Status: Draft Type: Standards Track Created: 26-Feb-2007 Abstract ======== Extending the decorator syntax to allow the decoration of classes. Rationale ========= Class decorators have an identical signature to function decorators. The identity decorator is defined as def identity(cls): return cls @identity class Example: pass To be useful class decorators should return a class-like thing but as with function decorators this is not enforced by the language. All the strong existing use cases are decorators that just register or annotate classes. import myfactory @myfactory.register class MyClass: pass Decorators are stackable so more than one registration can happen. import myfactory import mycron @mycron.schedule('nightly') @myfactory.register class MyClass: pass The same effect is currently possible by making a functioon call after class definition time but as with function decorators decorating the class moves an important piece of information (i.e. 'this class participates in a factory') to the top. Decorators vs Metaclasses ========================= Decorators are executed once for each time they appear. Metaclasses are executed for every class that defines a metaclass and every class that inherits from a class that defines a metaclass. Decorators are also easilly stackable because of their takes-a-class returns-a-class signature. Metaclasses are combinable too but with their richer interface it requires more trouble (see Alex Martelli's PyCon 2005 talk[6]). Even with some tricks the combined metaclasses may need to coordinate who owns __new__ or __init__. Note that class decorators, like metaclasses, aren't garunteed to return the same class they were passed. History and Implementation ========================== Class decorators were originally proposed alongside function decorators in PEP318 [1]_ and were rejected by Guido [2]_ for lack of use cases. Two years later he saw a use case he liked and gave the go-ahead for a PEP and patch [3]_. The current patch is loosely based on a pre-2.4 patch [4]_ and updated to use the newer 2.5 (and 3k) AST. Grammar/Grammar is changed from funcdef: [decorators] 'def' NAME parameters ['->' test] ':' suite to decorated_thing: decorators (classdef | funcdef) funcdef: 'def' NAME parameters ['->' test] ':' suite "decorated_thing"s are premitted everywhere that funcdef and classdef are premitted. An alternate change to the grammar would be to make a 'decoratable_thing' which would include funcdefs and classdefs even if they had no decorators. Motivation ========== Below is an actual production metaclass followed by its class decorator equivalent. It is used to produce factory classes which each keep track of which account_id web forms are associated with. class Register(type): """A tiny metaclass to help classes register themselves automagically""" def __init__(cls, name, bases, dict): if 'DO_NOT_REGISTER' in dict: # this is a non-concrete class pass elif object not in bases: # this is yet another metaclass pass elif 'register' in dict: # this is a top level factory class setattr(cls, 'register', staticmethod(dict['register']))p else: # typical case, register with the factory cls.register(name, cls) return class FormFactory(object): id_to_form = {} # { account_id : { form_id : form_class } } def register(cls): FormFactory.id_to_form.setdefault(cls.account_id, {}) assert cls.form_id not in FormFactory.id_to_form[cls.account_id], cls.form_id FormFactory.id_to_form[cls.account_id][cls.form_id] = cls class FormCaputreA(FormFactory, Form): account_id = 17 form_id = 3 # .. cgi param to sql mapping defined, etc The decorated version eliminates the metaclass and loses some of the if-else checks because it won't be applied by the programmer to intermediate classes, metaclasses, or factories. class FormFactory(Form): id_to_form = {} # { account_id : { form_id : form_class } } @staticmethod def register(cls, account_id, form_id): FormFactory.id_to_form.setdefault(cls.account_id, {}) assert form_id not in FormFactory.id_to_form[cls.account_id], form_id FormFactory.id_to_form[cls.account_id][form_id] = cls # return the class unchanged return cls @FormFactory.register(account_id=17, form_id=3) class FormCaptureA(object): # .. cgi param to sql mapping defined, etc References ========== If you enjoyed this PEP you might also enjoy: .. [1] PEP 318, "Decorators for Functions and Methods" http://www.python.org/dev/peps/pep-0318/ .. [2] Class decorators rejection http://mail.python.org/pipermail/python-dev/2004-March/043458.html .. [3] Class decorator go-ahead http://mail.python.org/pipermail/python-dev/2006-March/062942.html .. [4] 2.4 class decorator patch http://python.org/sf/1007991 .. [5] 3.x class decorator patch http://python.org/sf/1671208 .. [6] Alex Martelli's PyCon 2005 "Python's Black Magic" http://www.python.org/pycon/2005/papers/36/ From collinw at gmail.com Sat Mar 10 01:19:49 2007 From: collinw at gmail.com (Collin Winter) Date: Fri, 9 Mar 2007 18:19:49 -0600 Subject: [Python-3000] PEP Draft: Class Decorators In-Reply-To: <20070310001054.GN18179@performancedrivers.com> References: <20070228205212.GD5537@performancedrivers.com> <20070310001054.GN18179@performancedrivers.com> Message-ID: <43aa6ff70703091619j78c1403ck341223b2c4e4b19f@mail.gmail.com> You need to run this through a spellchecker; I noticed at least a dozen typos. Other comments inline. Collin Winter On 3/9/07, Jack Diederich wrote: > PEP: 3XXX > Title: Class Decorators > Version: 1 > Last-Modified: 28-Feb-2007 > Authors: Jack Diederich > Implementation: SF#1671208 > Status: Draft > Type: Standards Track > Created: 26-Feb-2007 > > Abstract > ======== > > Extending the decorator syntax to allow the decoration of classes. > > Rationale > ========= > > Class decorators have an identical signature to function decorators. > The identity decorator is defined as > > def identity(cls): > return cls > > @identity > class Example: > pass > > To be useful class decorators should return a class-like thing but > as with function decorators this is not enforced by the language. > > All the strong existing use cases are decorators that just register > or annotate classes. > > import myfactory > > @myfactory.register > class MyClass: > pass > > Decorators are stackable so more than one registration can happen. > > import myfactory > import mycron > > @mycron.schedule('nightly') > @myfactory.register > class MyClass: > pass > > The same effect is currently possible by making a functioon call > after class definition time but as with function decorators decorating > the class moves an important piece of information (i.e. 'this class > participates in a factory') to the top. "but as with function decorators decorating" -> "but, as with function decorators, decorating" (adding commas makes it easier to parse). > Decorators vs Metaclasses > ========================= > > Decorators are executed once for each time they appear. Metaclasses > are executed for every class that defines a metaclass and every class > that inherits from a class that defines a metaclass. > > Decorators are also easilly stackable because of their takes-a-class > returns-a-class signature. Metaclasses are combinable too but with > their richer interface it requires more trouble (see Alex Martelli's > PyCon 2005 talk[6]). Even with some tricks the combined metaclasses > may need to coordinate who owns __new__ or __init__. > > Note that class decorators, like metaclasses, aren't garunteed to > return the same class they were passed. > > History and Implementation > ========================== > > Class decorators were originally proposed alongside function decorators > in PEP318 [1]_ and were rejected by Guido [2]_ for lack of use cases. > Two years later he saw a use case he liked and gave the go-ahead for a > PEP and patch [3]_. I'd still prefer it if this use case -- which is so central to the PEP -- were provided in the PEP itself. > The current patch is loosely based on a pre-2.4 patch [4]_ and updated to > use the newer 2.5 (and 3k) AST. > > Grammar/Grammar is changed from > > funcdef: [decorators] 'def' NAME parameters ['->' test] ':' suite > > to > > decorated_thing: decorators (classdef | funcdef) > funcdef: 'def' NAME parameters ['->' test] ':' suite > > "decorated_thing"s are premitted everywhere that funcdef and classdef > are premitted. > > An alternate change to the grammar would be to make a 'decoratable_thing' > which would include funcdefs and classdefs even if they had no decorators. Is it "decorated_thing" or "decoratable_thing"? > Motivation > ========== > > Below is an actual production metaclass followed by its class > decorator equivalent. It is used to produce factory classes > which each keep track of which account_id web forms are associated > with. > > class Register(type): > """A tiny metaclass to help classes register themselves automagically""" > > def __init__(cls, name, bases, dict): > if 'DO_NOT_REGISTER' in dict: # this is a non-concrete class > pass > elif object not in bases: # this is yet another metaclass > pass > elif 'register' in dict: # this is a top level factory class > setattr(cls, 'register', staticmethod(dict['register']))p > else: # typical case, register with the factory > cls.register(name, cls) > return > > class FormFactory(object): > id_to_form = {} # { account_id : { form_id : form_class } } > def register(cls): > FormFactory.id_to_form.setdefault(cls.account_id, {}) > assert cls.form_id not in FormFactory.id_to_form[cls.account_id], cls.form_id > FormFactory.id_to_form[cls.account_id][cls.form_id] = cls > > class FormCaputreA(FormFactory, Form): > account_id = 17 > form_id = 3 > # .. cgi param to sql mapping defined, etc > > The decorated version eliminates the metaclass and loses some of > the if-else checks because it won't be applied by the programmer to > intermediate classes, metaclasses, or factories. > > class FormFactory(Form): > id_to_form = {} # { account_id : { form_id : form_class } } > @staticmethod > def register(cls, account_id, form_id): > FormFactory.id_to_form.setdefault(cls.account_id, {}) > assert form_id not in FormFactory.id_to_form[cls.account_id], form_id > FormFactory.id_to_form[cls.account_id][form_id] = cls > # return the class unchanged > return cls > > @FormFactory.register(account_id=17, form_id=3) > class FormCaptureA(object): > # .. cgi param to sql mapping defined, etc > > References > ========== > If you enjoyed this PEP you might also enjoy: > > .. [1] PEP 318, "Decorators for Functions and Methods" > http://www.python.org/dev/peps/pep-0318/ > > .. [2] Class decorators rejection > http://mail.python.org/pipermail/python-dev/2004-March/043458.html > > .. [3] Class decorator go-ahead > http://mail.python.org/pipermail/python-dev/2006-March/062942.html > > .. [4] 2.4 class decorator patch > http://python.org/sf/1007991 > > .. [5] 3.x class decorator patch > http://python.org/sf/1671208 > > .. [6] Alex Martelli's PyCon 2005 "Python's Black Magic" > http://www.python.org/pycon/2005/papers/36/ From jackdied at jackdied.com Sat Mar 10 01:26:52 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Fri, 9 Mar 2007 19:26:52 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F1C734.7080503@acm.org> References: <45F1C734.7080503@acm.org> Message-ID: <20070310002652.GO18179@performancedrivers.com> On Fri, Mar 09, 2007 at 12:44:36PM -0800, Talin wrote: > I had a conversation with Guido last night at the Python user's group > meeting, and we hashed out some of the details of how metaclasses should > work. I've gone ahead and written up a PEP, which I present for your review. > -------------------------------------------- > PEP: xxx > Title: Metaclasses in Python 3000 > Version: $Revision$ > Last-Modified: $Date$ > Author: Talin > Rationale > > There are two rationales for this PEP, both of which are somewhat > subtle. > > In particular, there is an important body of use cases where it > would be useful to preserve the order in which a class members are > declared. Ordinary Python objects store their members in a > dictionary, in which ordering is unimportant, and members are > accessed strictly by name. However, Python is often used to > interface with external systems in which the members are organized > according to an implicit ordering. Examples include declaration of C > structs; COM objects; Automatic translation of Python classes into > IDL or database schemas, such as used in an ORM; and so on. > > Alternate Proposals > > Another good suggestion was to simply use an ordered dict for all > classes, and skip the whole 'custom dict' mechanism. This was based > on the observation that most use cases for a custom dict were for > the purposes of preserving order information. However, this idea has > two drawbacks, first because it means that an ordered dict > implementation would have to be added to the set of built-in types > in Python, and second because it would impose a slight speed (and > complexity) penalty on all class declarations. (I just posted a much expanded class decotator PEP so read that first if you haven't already.) I am a very big fan of ordered dicts in classes. One possibility is that suites in classes always store their order in a special dict that keeps a side list of key order. A final invisible class decorator around every class would then toss out the order and leave only a regular dict. An advantage is that the order of the keys would be available to both metaclasses and class decorators. It would also require no changes in signature for either. As far as I can tell ordered dicts are the only demonstrated use case (and common! looking at any SQL/ORM library). People that really wanted to maintain order forever could make a copy of the original order and then implement magic methods to keep the list up to date. In my personal use cases I would use the ordering at class creation time and then never need it again. -Jack From greg.ewing at canterbury.ac.nz Sat Mar 10 01:33:13 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 10 Mar 2007 13:33:13 +1300 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <07Mar9.093604pst.57996@synergy1.parc.xerox.com> References: <45EF7767.9050406@acm.org> <07Mar8.115427pst.57996@synergy1.parc.xerox.com> <45F08919.5070808@acm.org> <07Mar8.144328pst.57996@synergy1.parc.xerox.com> <45F0AD2E.3@canterbury.ac.nz> <07Mar9.093604pst.57996@synergy1.parc.xerox.com> Message-ID: <45F1FCC9.3070701@canterbury.ac.nz> Bill Janssen wrote: > Oh, OK. I'm against (mandatory) type declarations. However, I do > believe that the object's type should have to include the expected > interface for the method invocation to succeed. I don't understand what you mean by that. If there is no formal embodiment of interfaces, then I don't see what it means for the type to "include the expected interface" beyond simply having the method. -- Greg From jackdied at jackdied.com Sat Mar 10 01:39:12 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Fri, 9 Mar 2007 19:39:12 -0500 Subject: [Python-3000] PEP Draft: Class Decorators In-Reply-To: <43aa6ff70703091619j78c1403ck341223b2c4e4b19f@mail.gmail.com> References: <20070228205212.GD5537@performancedrivers.com> <20070310001054.GN18179@performancedrivers.com> <43aa6ff70703091619j78c1403ck341223b2c4e4b19f@mail.gmail.com> Message-ID: <20070310003912.GP18179@performancedrivers.com> On Fri, Mar 09, 2007 at 06:19:49PM -0600, Collin Winter wrote: > >History and Implementation > >========================== > > > >Class decorators were originally proposed alongside function decorators > >in PEP318 [1]_ and were rejected by Guido [2]_ for lack of use cases. > >Two years later he saw a use case he liked and gave the go-ahead for a > >PEP and patch [3]_. > > I'd still prefer it if this use case -- which is so central to the PEP > -- were provided in the PEP itself. I'll make that more clear. The use cas was factory registration right next to the class definition (as opposed to afterwards or through a metaclass). This is the example I use most frequently in the PEP. > >Grammar/Grammar is changed from > > > > funcdef: [decorators] 'def' NAME parameters ['->' test] ':' suite > > > >to > > > > decorated_thing: decorators (classdef | funcdef) > > funcdef: 'def' NAME parameters ['->' test] ':' suite > > > >"decorated_thing"s are premitted everywhere that funcdef and classdef > >are premitted. > > > >An alternate change to the grammar would be to make a 'decoratable_thing' > >which would include funcdefs and classdefs even if they had no decorators. > > Is it "decorated_thing" or "decoratable_thing"? The current production is 'decorated_thing' because it requires at least one decorator so it is definitely decorated. If there are zero or more decorators it would become 'decoratable_thing' as it would include the case where something could be decorated but isn't. Thanks, and I'll go typo hunting. -Jack From greg.ewing at canterbury.ac.nz Sat Mar 10 01:44:38 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 10 Mar 2007 13:44:38 +1300 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: <45F1B723.8020402@acm.org> References: <45F1B723.8020402@acm.org> Message-ID: <45F1FF76.9020805@canterbury.ac.nz> Talin wrote: > 3) At some point, pick the best / most popular generic function package > and formally make it a part of Python 3. Shouldn't there be 2.5) At some point, decide whether we want generic functions as a formal feature at all. -- Greg From brett at python.org Sat Mar 10 03:23:51 2007 From: brett at python.org (Brett Cannon) Date: Fri, 9 Mar 2007 18:23:51 -0800 Subject: [Python-3000] PEP Draft: Class Decorators In-Reply-To: <20070310001054.GN18179@performancedrivers.com> References: <20070228205212.GD5537@performancedrivers.com> <20070310001054.GN18179@performancedrivers.com> Message-ID: [SNIP] > class FormFactory(Form): > id_to_form = {} # { account_id : { form_id : form_class } } > @staticmethod > def register(cls, account_id, form_id): > FormFactory.id_to_form.setdefault(cls.account_id, {}) > assert form_id not in FormFactory.id_to_form[cls.account_id], form_id > FormFactory.id_to_form[cls.account_id][form_id] = cls > # return the class unchanged > return cls > > @FormFactory.register(account_id=17, form_id=3) > class FormCaptureA(object): > # .. cgi param to sql mapping defined, etc I don't think this example works. You have a method call that is lacking its first argument (did you meant to have it be a classmethod?). Plus that call itself will just return FormFactory for the decorator which itself does not define __call__ and thus will raise an exception when the decorator is called with FormCapture. I think you meant something like the following:: class FormFactory(Form): id_to_form = {} # { account_id : { form_id : form_class } } @classmethod def register(cls, account_id, form_id): def inner(to_decorate): FormFactory.id_to_form.setdefault(cls.account_id, {}) assert form_id not in FormFactory.id_to_form[cls.account_id], form_id FormFactory.id_to_form[cls.account_id][form_id] = cls # return the class unchanged return to_decorate return inner Otherwise define a __call__ method on FormFactory or use functools.partial to create the closure for partial function application for the arguments you want. And yes, Jack, you can check it in yourself once the initial big problems with the PEP have been addressed. -Brett From greg.ewing at canterbury.ac.nz Sat Mar 10 08:31:32 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 10 Mar 2007 20:31:32 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F1C734.7080503@acm.org> References: <45F1C734.7080503@acm.org> Message-ID: <45F25ED4.6030302@canterbury.ac.nz> Talin wrote: > class Foo(base1, base2, metaclass=mymeta): > ... -1 on this syntax, I think it's ugly. Alternative proposals: class Foo(base1, base2) as MyMeta: ... or class Foo(base1, base2) = MyMeta: ... > class Foo(base1, base2, metaclass=mymeta, private=True): > ... class Foo(base1, base2) as MyMeta(private=True): ... or class Foo(base1, base2) = MyMeta(private=True): ... > This attribute is a method named __metacreate__ Very bad choice of name -- it's not creating a meta-anything, and gives no clue what it is creating. > It would be possible to leave the existing __metaclass__ syntax in > place. Alternatively, it would not be too difficult to modify the > syntax rules of the Py3K translation tool to convert from the old to > the new syntax. The existing syntax actually has some advantages, e.g. you can do things like class Foo: class __metaclass__: # Real class methods defined here which I think is rather elegant, so I'm not sure I'd like the new syntax to completely replace the old one. -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 10 08:45:30 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 10 Mar 2007 20:45:30 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310002652.GO18179@performancedrivers.com> References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> Message-ID: <45F2621A.8000707@canterbury.ac.nz> Jack Diederich wrote: > I am a very big fan of ordered dicts in classes. One possibility is that > suites in classes always store their order in a special dict that keeps a > side list of key order. A final invisible class decorator around > every class would then toss out the order and leave only a regular dict. Is it really necessary to toss out the order at all? I'm skeptical that class dictionaries are either created or modified anywhere near often enough for there to be any noticeable performance penalty here. In any case, I've thought of a way to reduce the performance penalty to near-zero. Consider that the compiler knows all the names that will be assigned in the class dict and what order they are in. So all it needs to do is compile class Foo: a = 5 def b(self): ... def c(self, x): ... as though it were class Foo: __names__ = ('a', 'b', 'c') a = 5 def b(self): ... def c(self, x): ... The tuple can be a constant of the code object, so the only overhead is one extra item added to the dict at class creation time, and none at all for subsequent access to the class dict. And you don't even need a special kind of dict. -- Greg From g.brandl at gmx.net Sat Mar 10 08:55:28 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 Mar 2007 08:55:28 +0100 Subject: [Python-3000] __methods__ and __members__ In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > I don't recall ripping them out, but it's worth trying to do that -- > they really shouldn't be needed for modern extensionmodules (2.2+). > > On 3/7/07, Georg Brandl wrote: >> While reviewing the patch for __dir__() (which I'll apply then, since it was >> generally agreed upon at least for Py3k), I came about this: >> >> /* Merge in __members__ and __methods__ (if any). >> XXX Would like this to go away someday; for now, it's >> XXX needed to get at im_self etc of method objects. */ >> >> What is the status of __methods__ and __members__? Is this (and the docs) >> the only trace that's left of it? Okay, I looked into it. The old attribute semantics of offering a custom tp_getattr which dispatches to methods and members is mainly used in very old and platform-specific extension modules like RISCOS/*, Modules/{al,fl,sunaudiodev}module. If we decide to drop some of them, it would be nice to do it before working on them ;) Then, we have cases like pyexpat.c, where attributes are dynamically read in getattr. This could also be done with tp_getset, but it may be quite a bit slower. But this is only one type, and it could get a custom __dir__ function. There are also quite a few modules whose types have a getattr which only calls Py_FindMethod. This can easily be replaced with a tp_methods slot entry. Another oddity is that Py_FindMethod checks if __doc__ is requested, and returns it. Is that still necessary? Georg From jcarlson at uci.edu Sat Mar 10 09:55:00 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 10 Mar 2007 00:55:00 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F2621A.8000707@canterbury.ac.nz> References: <20070310002652.GO18179@performancedrivers.com> <45F2621A.8000707@canterbury.ac.nz> Message-ID: <20070310004809.74F4.JCARLSON@uci.edu> Greg Ewing wrote: > > Jack Diederich wrote: > > > I am a very big fan of ordered dicts in classes. One possibility is that > > suites in classes always store their order in a special dict that keeps a > > side list of key order. A final invisible class decorator around > > every class would then toss out the order and leave only a regular dict. > > Is it really necessary to toss out the order at all? > I'm skeptical that class dictionaries are either created > or modified anywhere near often enough for there to > be any noticeable performance penalty here. I agree. Very few classes have significantly more than a few hundred assignments or accesses during class body execution. I don't think speed is going to matter enough for us to care. [snip] > The tuple can be a constant of the code object, so the > only overhead is one extra item added to the dict at > class creation time, and none at all for subsequent > access to the class dict. And you don't even need a > special kind of dict. Strawman: class foo: if ...: a = 1 b = 2 else: b = 1 a = 2 c = 3 Which is first, a or b? What if ... is runtime-dependent? Also, depending on the use, one may want to know the order in a 'move to end' fashion (if a is assigned to multiple times, it ends up in the ordering as if only the last assignment was done). - Josiah From tony at PageDNA.com Sat Mar 10 01:09:06 2007 From: tony at PageDNA.com (Tony Lownds) Date: Fri, 9 Mar 2007 16:09:06 -0800 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: <10BC3DA5-CAEC-41D9-B6CD-120FA3D42577@PageDNA.com> On Mar 9, 2007, at 2:35 PM, Guido van Rossum wrote: > On 3/9/07, Tony Lownds wrote: >> With tuple parameters removed, there would still be divergence in >> 2.X. > > But not much; we could drop the annotation feature for tuple > parameters (since it won't survive in 3.0) and warn about tuple params > if the -Wpy3k flag is given. That would work >> By the way, I would like to work on either removing tuple parameters >> in 3.0 > > That would be great! Ok > >> or fixing the assertion failure currently in CVS. > > Perhaps you should switch to SVN? :-) Heh, clearly I am behind the times. I should switch to Mercurial to get ahead of the curve. > > More seriously, what assertion failure? This one, noted recently by Brett: >>> def f((a: int, b: int)): pass ... python: Python/compile.c:2430: compiler_nameop: Assertion `scope || (((PyStringObject *)(name))->ob_sval)[0] == '_'' failed. Abort (core dumped) It's moot with tuple parameters removed. -Tony From talin at acm.org Sat Mar 10 10:13:44 2007 From: talin at acm.org (Talin) Date: Sat, 10 Mar 2007 01:13:44 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310004809.74F4.JCARLSON@uci.edu> References: <20070310002652.GO18179@performancedrivers.com> <45F2621A.8000707@canterbury.ac.nz> <20070310004809.74F4.JCARLSON@uci.edu> Message-ID: <45F276C8.8020908@acm.org> Josiah Carlson wrote: > Also, depending on the use, one may want to know the order in a 'move to > end' fashion (if a is assigned to multiple times, it ends up in the > ordering as if only the last assignment was done). This is why I feel it is insufficient to keep just a record of what key names were inserted, it should be a record of both keys and values. So for example, if we have: class A: b = 1 b = 2 ...the resulting list should be [(b, 1), (b, 2)] This gives the metaclass a lot of flexibility for handling duplicates. Suppose, for example, a metaclass wants to handle duplicates by keeping only the last definition. This is easy to do: newdict = {} for key, value in orderlist: newdict[ key ] = value # Overwrite any previous definition What if you only want the *first* definition? Then check to see if the key is already in newdict and skip the assignment. What if you want to flag duplicates as an error? Check to see if the key is already in newdict, and throw an exception if so. You can even combine or concatenate the duplicates in some way if you keep enough information around to do so. It's up to the metaclass to decide. -- Talin From talin at acm.org Sat Mar 10 10:21:46 2007 From: talin at acm.org (Talin) Date: Sat, 10 Mar 2007 01:21:46 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F25ED4.6030302@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <45F25ED4.6030302@canterbury.ac.nz> Message-ID: <45F278AA.3090709@acm.org> Greg Ewing wrote: > Talin wrote: > >> class Foo(base1, base2, metaclass=mymeta): >> ... > > -1 on this syntax, I think it's ugly. All I can say is, beauty is in the eye, etc...in any case, I'm not the one to decide what's pretty and what's not. > Alternative proposals: > > class Foo(base1, base2) as MyMeta: > ... > > or > > class Foo(base1, base2) = MyMeta: > ... Neither of these work for me, in the sense that when I read them, they don't elicit concepts that match the metaclass concept. The relation between a class and a metaclass isn't well-expressed with 'as', 'is', or 'equals'. >> class Foo(base1, base2, metaclass=mymeta, private=True): >> ... > > class Foo(base1, base2) as MyMeta(private=True): > ... > > or > > class Foo(base1, base2) = MyMeta(private=True): > ... > >> This attribute is a method named __metacreate__ > > Very bad choice of name -- it's not creating a meta-anything, > and gives no clue what it is creating. I didn't want to call it __metadict__, as it was in the earlier proposal, because in the current version it's doing more than just creating the class dict, it's also processing any additional keywords that are passed in to the class base list. >> It would be possible to leave the existing __metaclass__ syntax in >> place. Alternatively, it would not be too difficult to modify the >> syntax rules of the Py3K translation tool to convert from the old to >> the new syntax. > > The existing syntax actually has some advantages, e.g. > you can do things like > > class Foo: > class __metaclass__: > # Real class methods defined here > > which I think is rather elegant, so I'm not sure I'd > like the new syntax to completely replace the old one. I have no opinion on this - I've never had a need to declare a metaclass inline like that. > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org > From g.brandl at gmx.net Sat Mar 10 10:26:09 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 Mar 2007 10:26:09 +0100 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: <10BC3DA5-CAEC-41D9-B6CD-120FA3D42577@PageDNA.com> References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <10BC3DA5-CAEC-41D9-B6CD-120FA3D42577@PageDNA.com> Message-ID: Tony Lownds schrieb: >>> or fixing the assertion failure currently in CVS. >> >> Perhaps you should switch to SVN? :-) > > Heh, clearly I am behind the times. I should switch to Mercurial > to get ahead of the curve. You should rather switch to Bazaar and Mercurial at the same time, as I expect a hard battle between the two should Python ever choose to change version control systems again... ;) Georg From g.brandl at gmx.net Sat Mar 10 10:30:31 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 Mar 2007 10:30:31 +0100 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310002652.GO18179@performancedrivers.com> References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> Message-ID: Jack Diederich schrieb: > I am a very big fan of ordered dicts in classes. One possibility is that > suites in classes always store their order in a special dict that keeps a > side list of key order. A final invisible class decorator around > every class would then toss out the order and leave only a regular dict. So perhaps, thanks to metaclasses, all those people wanting ordered dicts in Python's core finally get their wish. ;) Seriously, I'm not very convinced of the "metaclass-as-keyword-args" syntax. It appears too arbitrary. Then again, setting something (__metaclass__) *in the class* which affects the creation *of the class* is not very idiomatic either. "class Foo as A" is altogether confusing, since all usages of "as" so far at least assigned *something* to "A". cheers, Georg From jackdied at jackdied.com Sat Mar 10 19:39:09 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Sat, 10 Mar 2007 13:39:09 -0500 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <10BC3DA5-CAEC-41D9-B6CD-120FA3D42577@PageDNA.com> Message-ID: <20070310183909.GQ18179@performancedrivers.com> On Sat, Mar 10, 2007 at 10:26:09AM +0100, Georg Brandl wrote: > Tony Lownds schrieb: > > >>> or fixing the assertion failure currently in CVS. > >> > >> Perhaps you should switch to SVN? :-) > > > > Heh, clearly I am behind the times. I should switch to Mercurial > > to get ahead of the curve. > > You should rather switch to Bazaar and Mercurial at the same time, > as I expect a hard battle between the two should Python ever choose > to change version control systems again... ;) Bazaar has a leg up this front. Canonical was handing out free bzr T-shirts at PyCon. -Jack From jcarlson at uci.edu Sat Mar 10 20:18:31 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 10 Mar 2007 11:18:31 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F25ED4.6030302@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <45F25ED4.6030302@canterbury.ac.nz> Message-ID: <20070310005505.74F7.JCARLSON@uci.edu> Greg Ewing wrote: > > Talin wrote: > > > class Foo(base1, base2, metaclass=mymeta): > > ... > > -1 on this syntax, I think it's ugly. > > Alternative proposals: > > class Foo(base1, base2) as MyMeta: > ... While I don't particularly like the foo(base, metaclass=...) syntax, (__metaclass__ also doesn't seem broken to me), the whole 'x as y' approach to syntax is starting to wear on me. > class Foo(base1, base2) = MyMeta: This particular variant make me want to claw out my eyes. Please don't mention it again - I like my eyes! > > This attribute is a method named __metacreate__ > > Very bad choice of name -- it's not creating a meta-anything, > and gives no clue what it is creating. Agreed. > > It would be possible to leave the existing __metaclass__ syntax in > > place. Alternatively, it would not be too difficult to modify the > > syntax rules of the Py3K translation tool to convert from the old to > > the new syntax. > > The existing syntax actually has some advantages, e.g. > you can do things like > > class Foo: > class __metaclass__: > # Real class methods defined here > > which I think is rather elegant, so I'm not sure I'd > like the new syntax to completely replace the old one. I agree; while I've never used the inline ability of metaclasses (I rarely use metaclasses in the first place), losing this ability would seem to be premature optimization. Generally, about the only need I forsee for arbitrary keyword arguments in the class 'foo(...)' signature is to pass arguments to the 'metadict' factory (method, function, etc.). Otherwise it seems more like a syntax looking for a purpose than a purpose looking for a syntax. There are two semantics that I would be happy with. Either force metaclasses to have a .metadict() or equivalent method (or a callable attribute in the case of arbitrary callables), or even allow __metaclass__ to be a tuple: class foo(...): __metaclass__ = mytype, newdict Newdict needs to be callable, and if you want to pass certain semantic-altering arguments, make newdict a factory can call it directly... class foo(...): __metaclass__ = mytype, newdict(no_dupes=True) One wouldn't even need any of the default arguments only stuff to make the two 'newdict' examples above use the same callable. - Josiah From talin at acm.org Sat Mar 10 20:27:13 2007 From: talin at acm.org (Talin) Date: Sat, 10 Mar 2007 11:27:13 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310005505.74F7.JCARLSON@uci.edu> References: <45F1C734.7080503@acm.org> <45F25ED4.6030302@canterbury.ac.nz> <20070310005505.74F7.JCARLSON@uci.edu> Message-ID: <45F30691.6040103@acm.org> Josiah Carlson wrote: > There are two semantics that I would be happy with. Either force > metaclasses to have a .metadict() or equivalent method (or a callable > attribute in the case of arbitrary callables), or even allow > __metaclass__ to be a tuple: > > class foo(...): > __metaclass__ = mytype, newdict > > Newdict needs to be callable, and if you want to pass certain > semantic-altering arguments, make newdict a factory can call it directly... > > class foo(...): > __metaclass__ = mytype, newdict(no_dupes=True) > > One wouldn't even need any of the default arguments only stuff to make > the two 'newdict' examples above use the same callable. I strongly feel that this makes *using* metaclasses way too complex. A person wanting to create a C struct or a database record should simply have to say "metaclass=cstruct" - they shouldn't have to declare a bunch of individual pieces, all of which have to match with each other. There's no utility in being able to "mix n' match" metaclasses and dicts. In any case, we're just going over old ground here. -- Talin From mark.russell at zen.co.uk Sat Mar 10 15:22:52 2007 From: mark.russell at zen.co.uk (Mark Russell) Date: Sat, 10 Mar 2007 14:22:52 +0000 Subject: [Python-3000] Reversing through text files with the new IO library Message-ID: As an experiment with the new IO library (http://docs.google.com/Doc? id=dfksfvqd_1cn5g5m), I added support for walking through the lines of a file backwards using the reversed() iterator: for line in reversed(open(path)): ... It works by scanning backwards through the file a block at a time (using seek()) but will handle arbitrary length lines. The patch is at http://www.python.org/sf/1677872. The code is currently directly in Lib/io.py, but in fact the only internal access it needs is adding the __reversed__ hook to TextIOWrapper. It's useful for scanning backwards through large log files, but it's also IMHO a nice example of the benefits of the new library. The version of this that I used under python 2.5 had to use os.seek() etc on file descriptors, whereas now it just wraps a new buffering class (io.TextLineReverser) around the raw IO object. Among other things it makes unit tests simpler - instead of messing around with temporary files the tests can do things like: b = io.BytesIO(b'one\ntwo\nthree\n') assert list(io.TextLineReverser(b)) == [ 'three\n', 'two\n', 'one \n' ] Mark Russell From pje at telecommunity.com Sat Mar 10 21:22:30 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Mar 2007 15:22:30 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F2621A.8000707@canterbury.ac.nz> References: <20070310002652.GO18179@performancedrivers.com> <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> Message-ID: <5.1.1.6.0.20070310152034.02a62d70@sparrow.telecommunity.com> At 08:45 PM 3/10/2007 +1300, Greg Ewing wrote: >Jack Diederich wrote: > > > I am a very big fan of ordered dicts in classes. One possibility is that > > suites in classes always store their order in a special dict that keeps a > > side list of key order. A final invisible class decorator around > > every class would then toss out the order and leave only a regular dict. > >Is it really necessary to toss out the order at all? >I'm skeptical that class dictionaries are either created >or modified anywhere near often enough for there to >be any noticeable performance penalty here. > >In any case, I've thought of a way to reduce the performance >penalty to near-zero. Consider that the compiler knows >all the names that will be assigned in the class dict >and what order they are in. So all it needs to do is >compile... This wouldn't help any of the other use cases for custom metaclass dictionaries. I.e., the ability to use a custom dictionary type is a feature, not a bug. (Note that using a custom dictionary means you can override its __getitem__ as well as its __setitem__, thereby influencing the execution of the class body, defining special names, etc.) From jcarlson at uci.edu Sat Mar 10 21:40:24 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 10 Mar 2007 12:40:24 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F30691.6040103@acm.org> References: <20070310005505.74F7.JCARLSON@uci.edu> <45F30691.6040103@acm.org> Message-ID: <20070310123707.750D.JCARLSON@uci.edu> Talin wrote: > I strongly feel that this makes *using* metaclasses way too complex. A > person wanting to create a C struct or a database record should simply > have to say "metaclass=cstruct" - they shouldn't have to declare a bunch > of individual pieces, all of which have to match with each other. > There's no utility in being able to "mix n' match" metaclasses and dicts. > > In any case, we're just going over old ground here. Great! If there is no need to mix and match metaclasses and dicts arbitrarily, then there is no need to pass alternate parameters to the dictionary construction method, and thusly, no need to offer arbitrary keyword arguments to class foo(...) signatures. As such, the compiler, etc., needs to be taught to pull out the __metaclass__ definition before executing the class body, but that's better than mucking with the syntax of class foo(...). - Josiah From pje at telecommunity.com Sat Mar 10 22:08:35 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 10 Mar 2007 16:08:35 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310123707.750D.JCARLSON@uci.edu> References: <45F30691.6040103@acm.org> <20070310005505.74F7.JCARLSON@uci.edu> <45F30691.6040103@acm.org> Message-ID: <5.1.1.6.0.20070310155616.056aaf70@sparrow.telecommunity.com> At 12:40 PM 3/10/2007 -0800, Josiah Carlson wrote: >Talin wrote: > > I strongly feel that this makes *using* metaclasses way too complex. A > > person wanting to create a C struct or a database record should simply > > have to say "metaclass=cstruct" - they shouldn't have to declare a bunch > > of individual pieces, all of which have to match with each other. > > There's no utility in being able to "mix n' match" metaclasses and dicts. > > > > In any case, we're just going over old ground here. > >Great! If there is no need to mix and match metaclasses and dicts >arbitrarily, then there is no need to pass alternate parameters to the >dictionary construction method, Er, wha? Those two things are utterly unrelated. From jcarlson at uci.edu Sat Mar 10 23:00:31 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 10 Mar 2007 14:00:31 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <5.1.1.6.0.20070310155616.056aaf70@sparrow.telecommunity.com> References: <20070310123707.750D.JCARLSON@uci.edu> <5.1.1.6.0.20070310155616.056aaf70@sparrow.telecommunity.com> Message-ID: <20070310133901.7510.JCARLSON@uci.edu> "Phillip J. Eby" wrote: > At 12:40 PM 3/10/2007 -0800, Josiah Carlson wrote: > > >Talin wrote: > > > I strongly feel that this makes *using* metaclasses way too complex. A > > > person wanting to create a C struct or a database record should simply > > > have to say "metaclass=cstruct" - they shouldn't have to declare a bunch > > > of individual pieces, all of which have to match with each other. > > > There's no utility in being able to "mix n' match" metaclasses and dicts. > > > > > > In any case, we're just going over old ground here. > > > >Great! If there is no need to mix and match metaclasses and dicts > >arbitrarily, then there is no need to pass alternate parameters to the > >dictionary construction method, > > Er, wha? Those two things are utterly unrelated. I did have a post that was going to show that they were related, but then I remembered that because of __metaclass__ assignment semantics, it needs to necessarily have access to the class body dictionary at the point of assignment, so the compiler, etc., cannot be taught to extract everything. Syntax is necessary to offer an ordered dictionary semantics. I guess I'll toss my hat in for +0 on... class foo(..., metaclass=..., arbitrary_keyword=i_suppose): ... - Josiah From tony at pagedna.com Sat Mar 10 23:03:45 2007 From: tony at pagedna.com (Tony Lownds) Date: Sat, 10 Mar 2007 14:03:45 -0800 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: On Mar 9, 2007, at 2:35 PM, Guido van Rossum wrote: > On 3/9/07, Tony Lownds wrote: >> By the way, I would like to work on either removing tuple parameters >> in 3.0 > > That would be great! > I've posted a patch removing tuple parameters as #1678060. There was one case in itertools tests/docs where IMO readability was lost: ->>> for k, g in groupby(enumerate(data), lambda (i,x):i-x): +>>> for k, g in groupby(enumerate(data), lambda t:t[0]-t[1]): ... print(map(operator.itemgetter(1), g)) Is it OK to replace this with: >>> for k, g in groupby(data, lambda i, c=count(): c.next()-i): ... print(list(g)) ... -Tony From g.brandl at gmx.net Sat Mar 10 23:06:56 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 Mar 2007 23:06:56 +0100 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: Tony Lownds schrieb: > On Mar 9, 2007, at 2:35 PM, Guido van Rossum wrote: > >> On 3/9/07, Tony Lownds wrote: >>> By the way, I would like to work on either removing tuple parameters >>> in 3.0 >> >> That would be great! >> > > I've posted a patch removing tuple parameters as #1678060. > > There was one case in itertools tests/docs where IMO readability was > lost: > > ->>> for k, g in groupby(enumerate(data), lambda (i,x):i-x): > +>>> for k, g in groupby(enumerate(data), lambda t:t[0]-t[1]): > ... print(map(operator.itemgetter(1), g)) > > Is it OK to replace this with: > > >>> for k, g in groupby(data, lambda i, c=count(): c.next()-i): > ... print(list(g)) > ... If you do that, please parenthesize the lambda. Georg From g.brandl at gmx.net Sat Mar 10 23:22:35 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 10 Mar 2007 23:22:35 +0100 Subject: [Python-3000] Moving files around Message-ID: There are a few files which seemingly don't belong in their subdirectories: * Python/bltinmodule.c -> Modules * Python/sysmodule.c -> Modules * parts of Python/import.c -> Modules/impmodule.c * Python/traceback.c -> Objects * Modules/getpath.c -> Python * Modules/getbuildinfo.c -> Python * Modules/python.c -> Python * Modules/main.c -> Python However, I guess moving them would cause much more confusion for those merging changes between branches than for those having to find the files in the wrong directory in the first place ;) Georg From greg.ewing at canterbury.ac.nz Sat Mar 10 23:43:46 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Mar 2007 11:43:46 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F278AA.3090709@acm.org> References: <45F1C734.7080503@acm.org> <45F25ED4.6030302@canterbury.ac.nz> <45F278AA.3090709@acm.org> Message-ID: <45F334A2.7030805@canterbury.ac.nz> Talin wrote: > The relation > between a class and a metaclass isn't well-expressed with 'as', 'is', or > 'equals'. Okay, then make it class Foo(base1, base2) isa MyMeta: ... -- Greg From greg.ewing at canterbury.ac.nz Sat Mar 10 23:51:04 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Mar 2007 11:51:04 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> Message-ID: <45F33658.501@canterbury.ac.nz> Some more metaclass syntax ideas: class Foo[Meta](bases): ... Meta class Foo(bases): ... although I don't like the way the latter moves the 'class' keyword away from the beginning. Another thought: It's not strictly necessary to allow for arguments to the metaclass, since the same effect can be achieved with a factory function, like we do for decorators. -- Greg From brett at python.org Sat Mar 10 23:59:57 2007 From: brett at python.org (Brett Cannon) Date: Sat, 10 Mar 2007 14:59:57 -0800 Subject: [Python-3000] Moving files around In-Reply-To: References: Message-ID: On 3/10/07, Georg Brandl wrote: > There are a few files which seemingly don't belong in their subdirectories: > > * Python/bltinmodule.c -> Modules > * Python/sysmodule.c -> Modules > * parts of Python/import.c -> Modules/impmodule.c > > * Python/traceback.c -> Objects > The above are where they are as they are compiled into the interpreter and are not optional. > * Modules/getpath.c -> Python > * Modules/getbuildinfo.c -> Python > * Modules/python.c -> Python > * Modules/main.c -> Python > Yeah, those are weird. And does anyone else find test.test_support an odd name; it isn't testing anything! I really want to rename that module test.support or make it the __init__ module for 'test'. -Brett From guido at python.org Sun Mar 11 01:21:09 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 10 Mar 2007 16:21:09 -0800 Subject: [Python-3000] __methods__ and __members__ In-Reply-To: References: Message-ID: On 3/9/07, Georg Brandl wrote: > Guido van Rossum schrieb: > > I don't recall ripping them out, but it's worth trying to do that -- > > they really shouldn't be needed for modern extensionmodules (2.2+). > > > > On 3/7/07, Georg Brandl wrote: > >> While reviewing the patch for __dir__() (which I'll apply then, since it was > >> generally agreed upon at least for Py3k), I came about this: > >> > >> /* Merge in __members__ and __methods__ (if any). > >> XXX Would like this to go away someday; for now, it's > >> XXX needed to get at im_self etc of method objects. */ > >> > >> What is the status of __methods__ and __members__? Is this (and the docs) > >> the only trace that's left of it? > > Okay, I looked into it. > The old attribute semantics of offering a custom tp_getattr > which dispatches to methods and members is mainly used in very old > and platform-specific extension modules like RISCOS/*, > Modules/{al,fl,sunaudiodev}module. If we decide to drop some of them, > it would be nice to do it before working on them ;) I'm sure we can drop al and fl. I expect we can drop sunaudiodev too, and I seem to recall there's no-one to maintain the RISCOS code; but for sunaudiodev and RISCOS it would be better to ask on python-dev. > Then, we have cases like pyexpat.c, where attributes are dynamically > read in getattr. This could also be done with tp_getset, but it may > be quite a bit slower. But this is only one type, and it could get > a custom __dir__ function. I recommend doing the simplest thing that could possibly work. If it's too slow someone will speed it up. > There are also quite a few modules whose types have a getattr which > only calls Py_FindMethod. This can easily be replaced with a tp_methods > slot entry. Yeah, they didn't get changed earlier because there was no incentive. :-) > Another oddity is that Py_FindMethod checks if __doc__ is requested, and > returns it. Is that still necessary? I'm pretty sure the new code in typeobject.py handles that too -- otherwise how would __doc__ for modern types work? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Sun Mar 11 01:24:21 2007 From: talin at acm.org (Talin) Date: Sat, 10 Mar 2007 16:24:21 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F33658.501@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> <45F33658.501@canterbury.ac.nz> Message-ID: <45F34C35.9000308@acm.org> Greg Ewing wrote: > Another thought: It's not strictly necessary to > allow for arguments to the metaclass, since the same > effect can be achieved with a factory function, like > we do for decorators. True. The main reason for doing it the way that I did, is to allow for the future possibility of the *default* metaclass interpreting some of those keywords. I'll give you an example: Suppose the author of Pyrex realizes that there is a potential optimization available if classes can be declared 'sealed', in other words the class author is telling the compiler that this class will never be used as a base class. This allows the dispatch table to be simplified considerably - in many cases, eliminating the need for a run-time dict lookup of the method. So in this case, the default metaclass might accept the keyword 'sealed' to indicate this fact. Now, it's true that I don't know exactly what uses people will make of this keyword-argument facility, but I don't consider that alone to be a show-stopper. I agree that, in the general case, one shouldn't add features to a language for no reason. However, its often the case that when we add a new feature, such as generators, we don't really know all of the potential uses to which it will be put. And that's a good thing - it would be an awfully impoverished programming language if it could only be used for the purposes its creators intended! I realize that you are trying to hold back the chaos of everyone wanting to put their half-baked idea into Python's syntax and I appreciate that. However, there are two ways to accomplish this, which I would call a 'soft' prohibition whereby you set usage guidelines and style rules, and the 'hard' prohibition where you make it physically impossible to violate the rules. The problem with the 'soft' method is obvious - since the rules are only enforced by social contract, people can break them. The problem with the 'hard' method is that you may be closing off cases where it would be reasonable to make an exception to the rule - in other words, you are taking the decision out of the hands of the potential rule-breakers, which in some instances may have better judgment in a particular case than you do. For class-definition keyword arguments, I am envisioning that there would be some opportunities for useful innovations using this syntax, although I don't know what those might be. (If I knew, then they wouldn't be innovations!) Moreover, by granting people this 'hook', we would be allowing people to accomplish things that they would otherwise have to lobby for syntactical changes to accomplish. With respect to holding back the chaos, I'm all in favor of 'soft' prohibitions here - i.e. guidelines as to when this feature ought to be used. For one thing, anything that can be done via class decorator instead of a metaclass ought to use the former rather than the latter IMHO. The new metaclass mechanism should only be used when you need to 'go deep' - in other words, when you need to make a customization to the class that fundamentally alters its representation. -- Talin From brett at python.org Sun Mar 11 02:52:53 2007 From: brett at python.org (Brett Cannon) Date: Sat, 10 Mar 2007 17:52:53 -0800 Subject: [Python-3000] __methods__ and __members__ In-Reply-To: References: Message-ID: On 3/10/07, Guido van Rossum wrote: > On 3/9/07, Georg Brandl wrote: [SNIP] > > Okay, I looked into it. > > The old attribute semantics of offering a custom tp_getattr > > which dispatches to methods and members is mainly used in very old > > and platform-specific extension modules like RISCOS/*, > > Modules/{al,fl,sunaudiodev}module. If we decide to drop some of them, > > it would be nice to do it before working on them ;) > > I'm sure we can drop al and fl. I expect we can drop sunaudiodev too, Both are on the list to go in PEP 3108 (assuming you OK the PEP =). -Brett From talin at acm.org Sun Mar 11 03:49:40 2007 From: talin at acm.org (Talin) Date: Sat, 10 Mar 2007 18:49:40 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F34C35.9000308@acm.org> References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> <45F33658.501@canterbury.ac.nz> <45F34C35.9000308@acm.org> Message-ID: <45F36E44.7040207@acm.org> Talin wrote: > I'll give you an example: Suppose the author of Pyrex realizes that Of course, only an idiot would be unaware that "the author of Pyrex" is in fact the person who he is replying to. Which means I must have meant to do that. Uh-huh. Right. :) -- Talin From greg.ewing at canterbury.ac.nz Sun Mar 11 06:15:34 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Mar 2007 18:15:34 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310123707.750D.JCARLSON@uci.edu> References: <20070310005505.74F7.JCARLSON@uci.edu> <45F30691.6040103@acm.org> <20070310123707.750D.JCARLSON@uci.edu> Message-ID: <45F39076.8000706@canterbury.ac.nz> Josiah Carlson wrote: > As such, the compiler, etc., needs to be taught to pull out the > __metaclass__ definition before executing the class body, but that's > better than mucking with the syntax of class foo(...). I'd be happy with that. Also, for the name of the method for creating the dict, I suggest __newdict__. -- Greg From greg.ewing at canterbury.ac.nz Sun Mar 11 06:20:53 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Mar 2007 18:20:53 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310133901.7510.JCARLSON@uci.edu> References: <20070310123707.750D.JCARLSON@uci.edu> <5.1.1.6.0.20070310155616.056aaf70@sparrow.telecommunity.com> <20070310133901.7510.JCARLSON@uci.edu> Message-ID: <45F391B5.3040505@canterbury.ac.nz> Josiah Carlson wrote: > I did have a post that was going to show that they were related, but > then I remembered that because of __metaclass__ assignment semantics, it > needs to necessarily have access to the class body dictionary at the > point of assignment, so the compiler, etc., cannot be taught to extract > everything. Syntax is necessary to offer an ordered dictionary > semantics. I still think that extra arguments are unnecessary in any case, because anything you could do by passing args to the dict creation function could be done by using a function instead of a class for the "metaclass". Does anyone have a counter-example? -- Greg From collinw at gmail.com Sun Mar 11 06:34:53 2007 From: collinw at gmail.com (Collin Winter) Date: Sat, 10 Mar 2007 23:34:53 -0600 Subject: [Python-3000] Moving files around In-Reply-To: References: Message-ID: <43aa6ff70703102134k7bfbbb24p88dff30171786b97@mail.gmail.com> On 3/10/07, Brett Cannon wrote: [snip] > And does anyone else find test.test_support an odd name; it isn't > testing anything! I really want to rename that module test.support or > make it the __init__ module for 'test'. +1 for test.support, if it's to be renamed. Collin Winter From jcarlson at uci.edu Sun Mar 11 06:55:02 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 10 Mar 2007 21:55:02 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F391B5.3040505@canterbury.ac.nz> References: <20070310133901.7510.JCARLSON@uci.edu> <45F391B5.3040505@canterbury.ac.nz> Message-ID: <20070310214811.7516.JCARLSON@uci.edu> Greg Ewing wrote: > Josiah Carlson wrote: > > I did have a post that was going to show that they were related, but > > then I remembered that because of __metaclass__ assignment semantics, it > > needs to necessarily have access to the class body dictionary at the > > point of assignment, so the compiler, etc., cannot be taught to extract > > everything. Syntax is necessary to offer an ordered dictionary > > semantics. > > I still think that extra arguments are unnecessary in > any case, because anything you could do by passing args > to the dict creation function could be done by using a > function instead of a class for the "metaclass". Generally, yes. Also, I believe that the PEP offered as a requirement that the newdict creation stuff be a callable method on the metaclass, or at least a callable attribute on the metaclass function. What I was talking about with regards to the metaclass needing access to the class body is for cases like the following which are currently allowed... class foo(...): a = ... __metaclass__ = callable(a, ...) The above is syntactically and semantically kosher, and only by pulling the metaclass definition outside of the class body can we 1) prevent it from happening, and more importantly 2) be able to discover the metaclass and the ordered dictionary *prior* to the execution of the class body. Yes, you lose the ability to inline metaclasses, but that can't really be helped. - Josiah From greg.ewing at canterbury.ac.nz Sun Mar 11 07:01:15 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Mar 2007 19:01:15 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070310214811.7516.JCARLSON@uci.edu> References: <20070310133901.7510.JCARLSON@uci.edu> <45F391B5.3040505@canterbury.ac.nz> <20070310214811.7516.JCARLSON@uci.edu> Message-ID: <45F39B2B.7080000@canterbury.ac.nz> Josiah Carlson wrote: > class foo(...): > a = ... > __metaclass__ = callable(a, ...) Personally I wouldn't mind if it were made a requirement that __metaclass__, if it is present, must be the first name bound in the class body (and can't be inside an if statement, etc.) Does anyone know of any actual code out there that would be broken by this? -- Greg From greg.ewing at canterbury.ac.nz Sun Mar 11 07:13:07 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 11 Mar 2007 19:13:07 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F34C35.9000308@acm.org> References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> <45F33658.501@canterbury.ac.nz> <45F34C35.9000308@acm.org> Message-ID: <45F39DF3.4090402@canterbury.ac.nz> Talin wrote: > The main reason for doing it the way that I did, is to allow for > the future possibility of the *default* metaclass interpreting some of > those keywords. Hmmm. I can see how you would need some way of passing keywords in order to do that. But in the absence of any real use cases, it still smells like a big YAGNI to me. Note that you can already pass information to the metaclass itself using special attributes in the class namespace. This would only be need if you absolutely had to pass information when creating the *dict*. Things like your 'sealed' option don't seem to fall into that category. (I hope Python never gets anything like 'sealed', btw, but I realise it was just an example.) -- Greg From g.brandl at gmx.net Sun Mar 11 09:06:47 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 11 Mar 2007 09:06:47 +0100 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F33658.501@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <20070310002652.GO18179@performancedrivers.com> <45F33658.501@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Some more metaclass syntax ideas: > > class Foo[Meta](bases): > ... > > Meta class Foo(bases): > ... > > although I don't like the way the latter moves the > 'class' keyword away from the beginning. You could even unify metaclass and class decorator, if you alter the latter's definition; give it not a "class" argument, but "type", "name", "bases" and "dict" and have it return the same, but possibly altered or different objects. :D Georg From benji at benjiyork.com Sun Mar 11 12:50:22 2007 From: benji at benjiyork.com (Benji York) Date: Sun, 11 Mar 2007 07:50:22 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F09C3D.2020009@canterbury.ac.nz> References: <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> Message-ID: <45F3ECFE.1090100@benjiyork.com> Greg Ewing wrote: > Barry Warsaw wrote: > >> We already have an established, community accepted implementation of >> interfaces, > > Really? Which one is that? I believe Barry was referring to zope.interface. -- Benji York http://benjiyork.com From python at zesty.ca Mon Mar 12 01:37:51 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Sun, 11 Mar 2007 18:37:51 -0600 (CST) Subject: [Python-3000] __builtin__ and __builtins__ Message-ID: For a long time __builtin__ and __builtins__ have mystified me. Usually I end up guessing -- in any given namespace, one of the two will exist, and it will either be a module or a dictionary -- but I never committed to memory which name or type to expect. A little investigation reveals: In module __main__: __builtins__ is a reference to module __builtin__. __builtin__ only exists if you import it. In any other module: __builtins__ is a reference to module __builtin__'s __dict__. __builtin__ only exists if you import it. In order of increasing boldness, here are some possible fixes: 1. Make __builtins__ always refer to a dict. I think there's quite a strong argument for this -- in general, you should be able to take the same code and run it as a script or a module with similar behaviour. 2. Rename __builtins__ so it isn't so easily mistaken for __builtin__. I haven't thought of any awesome names to propose. 3. Unify the roles of the names __builtins__ and __builtin__ -- having both is too confusing. a. Instead of putting __builtins__ into every module namespace, import __builtin__ into every module namespace. b. Do the last resort for name lookups in __builtin__.__dict__ instead of __builtins__. c. To make this no slower than it is today, make each module cache a pointer to its __builtin__.__dict__ and update that pointer when __builtin__ is reassigned. Going all the way to #3 would be the cleanest, I think. Maybe there are other solutions for unifying the two, as well -- any ideas? -- ?!ng From greg.ewing at canterbury.ac.nz Mon Mar 12 05:37:20 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 12 Mar 2007 17:37:20 +1300 Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: References: Message-ID: <45F4D900.4030602@canterbury.ac.nz> Ka-Ping Yee wrote: > A little investigation reveals: > > In module __main__: > __builtins__ is a reference to module __builtin__. > __builtin__ only exists if you import it. > > In any other module: > __builtins__ is a reference to module __builtin__'s __dict__. > __builtin__ only exists if you import it. As I understand it, __builtins__ has always been regarded as an implementation detail, and if you want to get hold of the builtins, importing __builtin__ is the correct way to do it. The difference between __main__ and other modules is a bit weird. Phil Hassey posted some code recently that seemed to assume __builtins__ was always a module, which makes me think it was changed at some point to speed things up, but __main__ was somehow inadvertently excluded from the change. Changing __main__ to match would seem to be a good idea. I don't think it would be such a good idea to unify __builtin__ and __builtins__, because then importing __builtin__ would clobber the existing builtin namespace being used by the code -- which may not be the same thing. There might be merit in renaming __builtins__ to something less confusable, at the expense of breaking existing code which refers to it. -- Greg From python at zesty.ca Mon Mar 12 08:17:33 2007 From: python at zesty.ca (Ka-Ping Yee) Date: Mon, 12 Mar 2007 01:17:33 -0600 (CST) Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: <45F4D900.4030602@canterbury.ac.nz> References: <45F4D900.4030602@canterbury.ac.nz> Message-ID: On Mon, 12 Mar 2007, Greg Ewing wrote: > Changing __main__ to match would seem to be a > good idea. [...] > There might be merit in renaming __builtins__ > to something less confusable, at the expense of > breaking existing code which refers to it. Cool. > I don't think it would be such a good > idea to unify __builtin__ and __builtins__, > because then importing __builtin__ would clobber > the existing builtin namespace being used by > the code -- which may not be the same thing. We have "import as", though. If you want to import the default builtins without using them as the builtins, you can say import __builtin__ as default_builtin If you want to reset the builtins to the default builtins: import __builtin__ If you want to replace them with your own builtins: import my_builtin as __builtin__ That doesn't seem so bad. Hmm... when do you find yourself importing __builtin__? -- ?!ng From thomas at python.org Mon Mar 12 18:02:22 2007 From: thomas at python.org (Thomas Wouters) Date: Mon, 12 Mar 2007 18:02:22 +0100 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <-8873556614695186988@unknownmsgid> References: <45EF7767.9050406@acm.org> <-8873556614695186988@unknownmsgid> Message-ID: <9e804ac0703121002gbe0a565hf4ba3254d0211265@mail.gmail.com> On 3/8/07, Bill Janssen wrote: > There's an incomplete wiki page about a possible factoring of types at > http://wiki.python.org/moin/AbstractBaseClasses. The one thing that I have been unable to figure out (from that page, the python-dev/python-3000 messages on the subject and talking with Guido) is why ABCs are better than explicit interfaces like Twisted and Zope use. I think mixing in abstract baseclasses in the normal inheritance tree is a mistake, and if you have a separate tree, you really just have interfaces. I can think of a half dozen nagging warts that will pop up with using ABCs instead of interfaces, most of which I already griped to Guido about (but failed to convince him.) - You can't take a third-party object, not participating in the ABC/interface craze, and say 'it inherits from this ABC' (which you can do for interfaces.) - If the abstract classes hold actual attributes (as is the suggestion, as far as I can tell) rather than be completely empty, they can end up in the middle of the inheritance tree and mess up MRO. - If the abstract classes are really completely empty, you need another mechanism for figuring out what a particular abstract class entails. - You can't define *any* 'magic' on the abstract classes, things that are supposed to work on ABCs only, because any inherited class will of course automatically inherit the magic. If you use a magic metaclass for ABCs that works around that, people 'implementing' classes with their own metaclass will have to remember to subclass their metaclass from the ABC-magic-metaclass. - Because of the no-magic thing, you can't have the ABC do useful things, like asking it to verify if an implementation class has the right attributes or whether it can find an adapter for one ABC to another. - You can't un-inherit an ABC, ever. You can't inherit from a particular class to get some free implementation, but provide a slightly different (conflicting) interface. (Alright, probably not realistically a problem, but still.) - Mixing interface-definition and implementation like that makes it hard for new programmers to pick up the difference. It will be harder to explain the difference between 'dict' and 'Mapping' when they are both in the list of bases. How can you tell one from the other? As far as I can see, there is tremendously little difference between ABCs and interfaces, at least the way Guido explained his ideas for ABCs to me. The entirely Wiki page can be turned into a list of interfaces with a few textual edits. The only difference is that ABCs nestle in the normal MRO, which, in my eyes, is not a benefit at all. So... what's the actual benefit of ABCs over interfaces? -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070312/65c98503/attachment.html From guido at python.org Mon Mar 12 18:56:50 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 10:56:50 -0700 Subject: [Python-3000] Reversing through text files with the new IO library In-Reply-To: References: Message-ID: Thanks! This is a very interesting idea, I'd like to keep this around somehow. I also see that you noticed a problem with text I/O in the current design; there's no easy way to implement readline() efficiently. I want readline() to be as efficient as possible -- "for line in " should *scream*, like it does in 2.x. --Guido On 3/10/07, Mark Russell wrote: > As an experiment with the new IO library (http://docs.google.com/Doc? > id=dfksfvqd_1cn5g5m), I added support for walking through the lines > of a file backwards using the reversed() iterator: > > for line in reversed(open(path)): > ... > > It works by scanning backwards through the file a block at a time > (using seek()) but will handle arbitrary length lines. > > The patch is at http://www.python.org/sf/1677872. The code is > currently directly in Lib/io.py, but in fact the only internal access > it needs is adding the __reversed__ hook to TextIOWrapper. > > It's useful for scanning backwards through large log files, but it's > also IMHO a nice example of the benefits of the new library. The > version of this that I used under python 2.5 had to use os.seek() etc > on file descriptors, whereas now it just wraps a new buffering class > (io.TextLineReverser) around the raw IO object. Among other things > it makes unit tests simpler - instead of messing around with > temporary files the tests can do things like: > > b = io.BytesIO(b'one\ntwo\nthree\n') > assert list(io.TextLineReverser(b)) == [ 'three\n', 'two\n', 'one > \n' ] > > Mark Russell > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 12 19:09:42 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 11:09:42 -0700 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: Actually, I like the version with t[0]-t[1] better. This is supposed to be an exposition about groupby. Working out how it works, the version with count() seems to take more effort. But thanks for doing this! Is it done, apart from this nit? Then I'll check it in. --Guido On 3/10/07, Tony Lownds wrote: > > On Mar 9, 2007, at 2:35 PM, Guido van Rossum wrote: > > > On 3/9/07, Tony Lownds wrote: > >> By the way, I would like to work on either removing tuple parameters > >> in 3.0 > > > > That would be great! > > > > I've posted a patch removing tuple parameters as #1678060. > > There was one case in itertools tests/docs where IMO readability was > lost: > > ->>> for k, g in groupby(enumerate(data), lambda (i,x):i-x): > +>>> for k, g in groupby(enumerate(data), lambda t:t[0]-t[1]): > ... print(map(operator.itemgetter(1), g)) > > Is it OK to replace this with: > > >>> for k, g in groupby(data, lambda i, c=count(): c.next()-i): > ... print(list(g)) > ... > > > -Tony > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Mar 12 19:11:01 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 11:11:01 -0700 Subject: [Python-3000] Moving files around In-Reply-To: <43aa6ff70703102134k7bfbbb24p88dff30171786b97@mail.gmail.com> References: <43aa6ff70703102134k7bfbbb24p88dff30171786b97@mail.gmail.com> Message-ID: +1 for test.support too. -1 for putting code in test/__init__.py. On 3/10/07, Collin Winter wrote: > On 3/10/07, Brett Cannon wrote: > [snip] > > And does anyone else find test.test_support an odd name; it isn't > > testing anything! I really want to rename that module test.support or > > make it the __init__ module for 'test'. > > +1 for test.support, if it's to be renamed. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Mar 12 19:14:24 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 12 Mar 2007 14:14:24 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F3ECFE.1090100@benjiyork.com> References: <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 11, 2007, at 7:50 AM, Benji York wrote: > Greg Ewing wrote: >> Barry Warsaw wrote: >>> We already have an established, community accepted implementation >>> of interfaces, >> Really? Which one is that? > > I believe Barry was referring to zope.interface. Correct. AFAIK, there is only one other competing, mature package (but please correct me if I'm wrong!): PyProtocols. My impression is that the latter is essentially a superset of the former and that the latter is compatible with the former. The question then becomes whether we really want to invent a third way of doing interfaces in Python, or whether our time is better spent selecting and promoting one of the existing, tried-and-true packages. Perhaps it would be better for us to figure out how and if the two packages can be integrated. If a third way is still advocated, then I think it must have a fairly high barrier to acceptance in order to push aside such long accepted libraries. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfWYhnEjvBPtnXfVAQLIYgP/WbVv1ezQriZ+Vcbens8vl25q8RwCudik aTcJCDEPXCykhYDbj9KQIaNQd/+HBRgoBmUwEbQgCUI18zUBCNjanvODCcI9Kb/s Ad9w1gsTT1gqTPA2yLggfpYCS7mGttSLPvZJvXw9Vc60wjfpVtgCJphZG6WXza/i IrLpOYcOklc= =FNNm -----END PGP SIGNATURE----- From barry at python.org Mon Mar 12 19:18:37 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 12 Mar 2007 14:18:37 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <9e804ac0703121002gbe0a565hf4ba3254d0211265@mail.gmail.com> References: <45EF7767.9050406@acm.org> <-8873556614695186988@unknownmsgid> <9e804ac0703121002gbe0a565hf4ba3254d0211265@mail.gmail.com> Message-ID: <195B1676-5DA5-498B-91F8-EB70996DF5C0@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 12, 2007, at 1:02 PM, Thomas Wouters wrote: > - Mixing interface-definition and implementation like that makes > it hard for new programmers to pick up the difference. It will be > harder to explain the difference between 'dict' and 'Mapping' when > they are both in the list of bases. How can you tell one from the > other? I'm in agreement with everything Thomas said here, but I think this is one of the most important points. If you're looking at a class definition, it really should be plainly obvious what contributions to implementation and what contributes to interface declarations, without having to traipse through a bunch of documentation or source code. I think separating implementation and interface declarations aids in readability and discoverability. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfWZfXEjvBPtnXfVAQLP/QP9HrhR/Vbyc94jnXNR/cwSAnxCE0X6p1jj n+DTL4fb4zNUr0qTgK9gf82/Pa8xSPsDsLousxkFLIejrll+gqZ5UtWYM7NdKHgq PVDMly++rJ0yeAYVWe5aZvYMBKiJETcBbZKyGyEqPGeIMs0gYcf0F3zBQ96CeYrI q1O+rmbQ1tI= =k1kl -----END PGP SIGNATURE----- From guido at python.org Mon Mar 12 19:53:57 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 11:53:57 -0700 Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: <45F4D900.4030602@canterbury.ac.nz> References: <45F4D900.4030602@canterbury.ac.nz> Message-ID: This is all my fault. Long ago, there was only __builtin__, and all was well; you had to import it before using it. Then I added __builtins__ as an internal hook for restricted execution mode. The idea was that if __builtins__ was set to something nonstandard you were automatically in restricted mode (which just turned off a few instrospection features); all your build-in lookups were done here. But why is __builtins__ a module in __main__ and a dict elsewhere? Because in *interactive* mode, printing vars() would include __builtins__, which would be rather large. Code that incorrectly assumes it's always one or the other is incorrect and has always been incorrect; the wart was present when this feature first appeared. Since this has never been documented AFAIK, it's probably just been (incorrectly) reverse-engineered and copied around. I am all for unifying __builtin__ and __builtins__, exactly as Ping's point #3. Normally, importing __builtin__ again would not change anything because it'll replace the object with the same object. If you use some hack to import something else as __builtin__, yes, it would be good to trap that but I woudn't mind if it simply didn't work right, being a useless idea. --Guido On 3/11/07, Greg Ewing wrote: > Ka-Ping Yee wrote: > > > A little investigation reveals: > > > > In module __main__: > > __builtins__ is a reference to module __builtin__. > > __builtin__ only exists if you import it. > > > > In any other module: > > __builtins__ is a reference to module __builtin__'s __dict__. > > __builtin__ only exists if you import it. > > As I understand it, __builtins__ has always been > regarded as an implementation detail, and if you > want to get hold of the builtins, importing > __builtin__ is the correct way to do it. > > The difference between __main__ and other modules > is a bit weird. Phil Hassey posted some code > recently that seemed to assume __builtins__ was > always a module, which makes me think it was > changed at some point to speed things up, but > __main__ was somehow inadvertently excluded from > the change. > > Changing __main__ to match would seem to be a > good idea. I don't think it would be such a good > idea to unify __builtin__ and __builtins__, > because then importing __builtin__ would clobber > the existing builtin namespace being used by > the code -- which may not be the same thing. > > There might be merit in renaming __builtins__ > to something less confusable, at the expense of > breaking existing code which refers to it. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tony at pagedna.com Mon Mar 12 20:04:48 2007 From: tony at pagedna.com (Tony Lownds) Date: Mon, 12 Mar 2007 12:04:48 -0700 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: On Mar 12, 2007, at 11:09 AM, Guido van Rossum wrote: > Actually, I like the version with t[0]-t[1] better. This is supposed > to be an exposition about groupby. Working out how it works, the > version with count() seems to take more effort. > > But thanks for doing this! Is it done, apart from this nit? Then I'll > check it in. > Yep, it's done. I posted another version this morning. I can look at adding lambda support to 2to3/fixes/fix_tuple_params.py, if Collin isn't. Thanks -Tony From collinw at gmail.com Mon Mar 12 20:06:51 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 Mar 2007 14:06:51 -0500 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> Message-ID: <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> On 3/12/07, Tony Lownds wrote: > > On Mar 12, 2007, at 11:09 AM, Guido van Rossum wrote: > > > Actually, I like the version with t[0]-t[1] better. This is supposed > > to be an exposition about groupby. Working out how it works, the > > version with count() seems to take more effort. > > > > But thanks for doing this! Is it done, apart from this nit? Then I'll > > check it in. > > > > Yep, it's done. I posted another version this morning. > I can look at adding lambda support to 2to3/fixes/fix_tuple_params.py, > if Collin isn't. I can take care of it. It'll be a good excuse to get back to 2to3 after a few days spent in the depths of the patch tracker : ) Collin Winter From tony at pagedna.com Mon Mar 12 20:15:40 2007 From: tony at pagedna.com (Tony Lownds) Date: Mon, 12 Mar 2007 12:15:40 -0700 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> Message-ID: On Mar 12, 2007, at 12:06 PM, Collin Winter wrote: > I can take care of it. It'll be a good excuse to get back to 2to3 > after a few days spent in the depths of the patch tracker : ) > Cool. Out of curiosity, what versions of Python should 2to3 need to run? I know that 2to3/refactor.py needs 2.3 < Python < 3.0. Guess which versions I have handy :) Also, since I believe sys.exc_info() is staying, fix_sysexcinfo should change... -Tony From collinw at gmail.com Mon Mar 12 20:20:51 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 Mar 2007 14:20:51 -0500 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> Message-ID: <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> On 3/12/07, Tony Lownds wrote: > > On Mar 12, 2007, at 12:06 PM, Collin Winter wrote: > > > I can take care of it. It'll be a good excuse to get back to 2to3 > > after a few days spent in the depths of the patch tracker : ) > > > > Cool. Out of curiosity, what versions of Python should 2to3 need to > run? I know that 2to3/refactor.py needs 2.3 < Python < 3.0. Guess which > versions I have handy :) I use 2.5. 2.3 and 2.4 both fail the test suite, bailing out with SyntaxErrors in various places. > Also, since I believe sys.exc_info() is staying, fix_sysexcinfo > should change... Right. It's already on my todo list. Collin Winter From pje at telecommunity.com Mon Mar 12 20:33:14 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 14:33:14 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> Message-ID: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> At 02:14 PM 3/12/2007 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mar 11, 2007, at 7:50 AM, Benji York wrote: > > > Greg Ewing wrote: > >> Barry Warsaw wrote: > >>> We already have an established, community accepted implementation > >>> of interfaces, > >> Really? Which one is that? > > > > I believe Barry was referring to zope.interface. > >Correct. AFAIK, there is only one other competing, mature package >(but please correct me if I'm wrong!): PyProtocols. My impression is >that the latter is essentially a superset of the former and that the >latter is compatible with the former. > >The question then becomes whether we really want to invent a third >way of doing interfaces in Python, or whether our time is better >spent selecting and promoting one of the existing, tried-and-true >packages. Perhaps it would be better for us to figure out how and if >the two packages can be integrated. If a third way is still >advocated, then I think it must have a fairly high barrier to >acceptance in order to push aside such long accepted libraries. Personally I think both packages are overkill for the language and/or stdlib, if only because they're solutions for the wrong problem. (Note that PEPs 245 and 246 were rejected for related reasons.) For maybe 80-90% of the purposes that I originally created PyProtocols for, I have found that "simplegeneric" ( http://cheeseshop.python.org/simplegeneric/ ) is more than adequate -- and it's only 80 lines of code. Guido's more featureful GF prototype isn't that much bigger, I don't believe - heck, it might be smaller if memory serves. Of course, generic functions require you to say 'foo(bar)' instead of 'bar.foo()' (and IIUC, that's the big sticking point for Guido wrt to GF's in Py3K). But then again, interfaces and adapters require you to say 'IFoo(bar).foo()' also, so that's not really an improvement. From guido at python.org Mon Mar 12 21:04:24 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 13:04:24 -0700 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> Message-ID: On 3/12/07, Collin Winter wrote: > On 3/12/07, Tony Lownds wrote: > > > > On Mar 12, 2007, at 12:06 PM, Collin Winter wrote: > > > > > I can take care of it. It'll be a good excuse to get back to 2to3 > > > after a few days spent in the depths of the patch tracker : ) Great! I'll leave this to you then. > > Cool. Out of curiosity, what versions of Python should 2to3 need to > > run? I know that 2to3/refactor.py needs 2.3 < Python < 3.0. Guess which > > versions I have handy :) > > I use 2.5. 2.3 and 2.4 both fail the test suite, bailing out with > SyntaxErrors in various places. I think that if it becomes a requirement it shouldn't be too hard to make the refactoring tool work with older versions though, up to 2.2. but we should determine need first. > > Also, since I believe sys.exc_info() is staying, fix_sysexcinfo > > should change... > > Right. It's already on my todo list. I must've missed that. What happened? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mark.russell at zen.co.uk Mon Mar 12 21:10:17 2007 From: mark.russell at zen.co.uk (Mark Russell) Date: Mon, 12 Mar 2007 20:10:17 +0000 Subject: [Python-3000] Reversing through text files with the new IO library In-Reply-To: References: Message-ID: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> On 12 Mar 2007, at 17:56, Guido van Rossum wrote: > Thanks! This is a very interesting idea, I'd like to keep this > around somehow. Thanks for the positive feedback - much appreciated. > I also see that you noticed a problem with text I/O in the current > design; there's no easy way to implement readline() efficiently. I > want readline() to be as efficient as possible -- "for line in " > should *scream*, like it does in 2.x. Yes, I suspect that BufferedReader needs some kind of readuntil() method, so that (at least for sane encodings like utf-8) each line is read via a single readuntil() followed by a decode() call for the entire line. Maybe something like this (although the only way to be sure is to experiment): line, endindex = buffer.readuntil(line_endings) Read until we see one of the byte strings in line_endings, which is a sequence of one or more byte strings. If there are multiple line endings with a common prefix, use the longest. Return the line complete with the ending, with endindex being the index within line of the line ending (or None if EOF was encountered). Is anyone working on io.py btw? If not I'd be willing to put some time into it. I guess the todo list is something like this: - Finish off the python prototypes in io.py (using and maybe tweaking the API spec) - Get unit tests working with __builtin__.open = io.open - Profile and optimize (e.g. by selective conversion to C) Mark From guido at python.org Mon Mar 12 21:18:09 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 13:18:09 -0700 Subject: [Python-3000] Reversing through text files with the new IO library In-Reply-To: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> References: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> Message-ID: On 3/12/07, Mark Russell wrote: > On 12 Mar 2007, at 17:56, Guido van Rossum wrote: > > Thanks! This is a very interesting idea, I'd like to keep this > > around somehow. > > Thanks for the positive feedback - much appreciated. > > > I also see that you noticed a problem with text I/O in the current > > design; there's no easy way to implement readline() efficiently. I > > want readline() to be as efficient as possible -- "for line in " > > should *scream*, like it does in 2.x. > > Yes, I suspect that BufferedReader needs some kind of readuntil() > method, so that (at least for sane encodings like utf-8) each line is > read via a single readuntil() followed by a decode() call for the > entire line. > > Maybe something like this (although the only way to be sure is to > experiment): > > line, endindex = buffer.readuntil(line_endings) > > Read until we see one of the byte strings in line_endings, which > is a sequence of one or > more byte strings. If there are multiple line endings with a > common prefix, use the longest. > Return the line complete with the ending, with endindex being > the index within line of the > line ending (or None if EOF was encountered). > > Is anyone working on io.py btw? If not I'd be willing to put some > time into it. I guess the todo list is something like this: I am, when I have time (which seems rarely) and Mike Verdone and Daniel Stutzbach are (though I may have unintentionally discouraged them by not providing feedback soon enough). > - Finish off the python prototypes in io.py (using and maybe > tweaking the API spec) Yes. I am positive that attempting to implement the entire PEP (and trying to do it relatively efficiently) will require us to go back to the API design several times. Note that some of the binary prototypes don't work right yet; the unittests don't cover everything that's been implemented yet. I would love for you to start working on this. Let me know off-line if you need more guidance (but CC Daniel and Mike so they know what's going on). > - Get unit tests working with __builtin__.open = io.open I'm not even sure about this one; we may have to do that simultaneously with the str/unicode conversion. If we attempt do to it before then, I expect that we'll get lots of failures because the new I/O text layer always returns unicode and the new binary layer returns bytes objects. We may have to do it more piecemeal. Perhaps a good start would be to convert selected modules that use binary I/O to switch to the new io module explicitly by importing it and recoding them to deal with bytes. > - Profile and optimize (e.g. by selective conversion to C) I'd be okay with doing that after the 3.0 alpha 1 release (planned for June). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From mark.russell at zen.co.uk Mon Mar 12 21:51:32 2007 From: mark.russell at zen.co.uk (Mark Russell) Date: Mon, 12 Mar 2007 20:51:32 +0000 Subject: [Python-3000] Reversing through text files with the new IO library In-Reply-To: References: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> Message-ID: <89C96DFC-FDD1-47E2-8E5C-1E7BFED27FB9@zen.co.uk> On 12 Mar 2007, at 20:18, Guido van Rossum wrote: > I would love for you to start working on this. Let me know off-line if > you need more guidance (but CC Daniel and Mike so they know what's > going on). Great! I'll start off by working up a patch that implements any easy missing stuff from http://docs.google.com/Doc?id=dfksfvqd_1cn5g5m (aiming for simplicity and reasonable but not extreme efficiency) along with more unit tests. I'll also have a go at a BufferedReader method to support a reasonably efficient TextIOWrapper.readline(). Hopefully I'll have something in the next week or so. Daniel and Mike - please let me know if I'm overlapping with anything you're working on - I don't want to tread on anyone's toes. Mark From collinw at gmail.com Mon Mar 12 22:24:06 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 Mar 2007 16:24:06 -0500 Subject: [Python-3000] generics [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> Message-ID: <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> On 3/12/07, Guido van Rossum wrote: > On 3/12/07, Collin Winter wrote: > > On 3/12/07, Tony Lownds wrote: > > > Cool. Out of curiosity, what versions of Python should 2to3 need to > > > run? I know that 2to3/refactor.py needs 2.3 < Python < 3.0. Guess which > > > versions I have handy :) > > > > I use 2.5. 2.3 and 2.4 both fail the test suite, bailing out with > > SyntaxErrors in various places. > > I think that if it becomes a requirement it shouldn't be too hard to > make the refactoring tool work with older versions though, up to 2.2. > but we should determine need first. As of r54311, 2to3 now works with Python 2.4. I tried making it work with Python 2.3, but it's going to require changes wrt. how 2to3 interacts with the logging module, and I don't know enough about logging to know what needs to change. > > > Also, since I believe sys.exc_info() is staying, fix_sysexcinfo > > > should change... > > > > Right. It's already on my todo list. > > I must've missed that. What happened? I thought the decision had been made (or at least, the consensus reached) that sys.exc_info() would stick around in Python 3. Collin Winter From guido at python.org Mon Mar 12 22:27:59 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 14:27:59 -0700 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <9e804ac0703121002gbe0a565hf4ba3254d0211265@mail.gmail.com> References: <45EF7767.9050406@acm.org> <-8873556614695186988@unknownmsgid> <9e804ac0703121002gbe0a565hf4ba3254d0211265@mail.gmail.com> Message-ID: On 3/12/07, Thomas Wouters wrote: > > On 3/8/07, Bill Janssen wrote: > > There's an incomplete wiki page about a possible factoring of types at > > http://wiki.python.org/moin/AbstractBaseClasses. > > The one thing that I have been unable to figure out (from that page, the > python-dev/python-3000 messages on the subject and talking with Guido) is > why ABCs are better than explicit interfaces like Twisted and Zope use. I > think mixing in abstract baseclasses in the normal inheritance tree is a > mistake, and if you have a separate tree, you really just have interfaces. I > can think of a half dozen nagging warts that will pop up with using ABCs > instead of interfaces, most of which I already griped to Guido about (but > failed to convince him.) And, to the contrary, I believe that if one mechanism can handle two requirements, that's better than having two separate mechanisms. I have a mandate to try and keep the language small. > - You can't take a third-party object, not participating in the > ABC/interface craze, and say 'it inherits from this ABC' (which you can do > for interfaces.) In some cases, at least, this could be done by patching __bases__; not that I'm particularly advocating that. But I believe the last time we had this discussion this wasn't considered a particularly important requirement, as long as the standard library plays the ABC game consistently. I believe it is a requirement for zope/twisted interfaces mostly because the stdlib *doesn't* play their game. > - If the abstract classes hold actual attributes (as is the suggestion, as > far as I can tell) rather than be completely empty, they can end up in the > middle of the inheritance tree and mess up MRO. I don't believe this. Can you show me an example? I would think that if done properly, the ABC defining a particular behavior would always be included as a base class for any class implementing that behavior, and thus it would automatically come in the MRO *after* all classes implementing the behavior, as it should. That seems entirely correct to me. Of course, there may be problems if one of the subclasses doesn't call its super method, but that problem can occur regardless of the use of ABCs. > - If the abstract classes are really completely empty, you need another > mechanism for figuring out what a particular abstract class entails. That's not what Bill or I are proposing, however. Take a look at the Py3k Lib/io.py module; it has several abstract base classes, esp. RawIOBase is close to my ideal use of ABC. > - You can't define *any* 'magic' on the abstract classes, things that are > supposed to work on ABCs only, because any inherited class will of course > automatically inherit the magic. If you use a magic metaclass for ABCs that > works around that, people 'implementing' classes with their own metaclass > will have to remember to subclass their metaclass from the > ABC-magic-metaclass. That's exactly what metaclasses are for, so I'm not sure what your problem is; obviously mixing metaclasses requires a convention for metaclasses to cooperate, but I imagine that unknowingly combining custom metaclasses with zope/twisted interfaces can *also* easily screw up. > - Because of the no-magic thing, you can't have the ABC do useful things, > like asking it to verify if an implementation class has the right attributes > or whether it can find an adapter for one ABC to another. That's easily done as separate function though. Most meta-behavior is already done as separate functions; e.g. you write type(x), isinstance(x, C), issubclass(C, B) rather than using method notations for these. I don't see this as a disadvantage at all, unless you have suddenly got the OO-religion-disease (typically found in Java programmers migrating to Python and questioning len() :-). > - You can't un-inherit an ABC, ever. You can't inherit from a particular > class to get some free implementation, but provide a slightly different > (conflicting) interface. (Alright, probably not realistically a problem, but > still.) This should be done using proxies or wrappers or containment anyway, not inheritance. > - Mixing interface-definition and implementation like that makes it hard > for new programmers to pick up the difference. It will be harder to explain > the difference between 'dict' and 'Mapping' when they are both in the list > of bases. How can you tell one from the other? A naming convention? Documentation? The presence of unimplemented methods? > As far as I can see, there is tremendously little difference between ABCs > and interfaces, at least the way Guido explained his ideas for ABCs to me. Then why are you fighting them? :-) > The entirely Wiki page can be turned into a list of interfaces with a few > textual edits. The only difference is that ABCs nestle in the normal MRO, > which, in my eyes, is not a benefit at all. So... what's the actual benefit > of ABCs over interfaces? Well, to others, the nestling in the MRO is a feature, not a bug. Please stay posted as I (slowly) work on a set of proposals for ABCs to go into the stdlib. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Mon Mar 12 22:42:15 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 12 Mar 2007 22:42:15 +0100 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> References: <45F1B723.8020402@acm.org> <14B3D84D-5C9C-4083-89E4-AF3E0332DE5D@PageDNA.com> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> Message-ID: Collin Winter schrieb: >> > > Also, since I believe sys.exc_info() is staying, fix_sysexcinfo >> > > should change... >> > >> > Right. It's already on my todo list. >> >> I must've missed that. What happened? > > I thought the decision had been made (or at least, the consensus > reached) that sys.exc_info() would stick around in Python 3. Has the final decision now been made whether the traceback should be stuck on the exception or not? Georg From guido at python.org Mon Mar 12 23:03:02 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 15:03:02 -0700 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> Message-ID: On 3/12/07, Georg Brandl wrote: > Has the final decision now been made whether the traceback should be > stuck on the exception or not? If it has, I wasn't there. :-( I'm still hoping we can get agreement that storing the traceback on the exception object is the way to go. Only a handful of modules using pre-baked exception objects would have to be fixed, and likely those were doing it out of a misunderstanding of the performance characteristics. Then sys.exc_info() would be a roundabout way to ask for the tuple (e.__class__, e, e.__traceback__) and would constrain implementations to keep the most recently caught exception around per stack frame to maintain sys.exc_info()'s precise semantics. I'd much rather add something to 2.6 that stores __traceback__ on the exception instance when it is caught using 'as' syntax (but not using ',' syntax). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Mon Mar 12 23:12:07 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 11:12:07 +1300 Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: References: <45F4D900.4030602@canterbury.ac.nz> Message-ID: <45F5D037.6050204@canterbury.ac.nz> Ka-Ping Yee wrote: > We have "import as", though. If you want to import the default > builtins without using them as the builtins, you can say > > import __builtin__ as default_builtin Seems to me it would be better to rename the module to a non-double-underscore name. There's really nothing magical about the module itself, only the name used by global variable lookups. So just call the module 'builtins', and then you can do import builtins # get the module with no magic happening import mystuff as __builtins__ # change my builtin namespace import builtins as __builtins__ # restore default builtin namespace -- Greg From barry at python.org Mon Mar 12 23:27:09 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 12 Mar 2007 18:27:09 -0400 Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: <45F5D037.6050204@canterbury.ac.nz> References: <45F4D900.4030602@canterbury.ac.nz> <45F5D037.6050204@canterbury.ac.nz> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 12, 2007, at 6:12 PM, Greg Ewing wrote: > Seems to me it would be better to rename the module > to a non-double-underscore name. There's really nothing > magical about the module itself, only the name used > by global variable lookups. So just call the module > 'builtins', and then you can do > > import builtins # get the module with no magic happening > > import mystuff as __builtins__ # change my builtin namespace > > import builtins as __builtins__ # restore default builtin > namespace +1 - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfXTvnEjvBPtnXfVAQLRGAQAoZFNRze7dmJE6CvIKY1fNeUegfhBcJ4G NMkCjiWdjUXbfx7kqF66f/acQJvA9Zpw6EMlOkvsNdsgIDRB5VQsvc4EIpmLj2Mu DnPHPQt5hfK9J0HghUbTh0hVe+OatCArAKe/EkdcHDrgnohQGvAJx8ljFifvECx9 SgACSxZwvRk= =0axI -----END PGP SIGNATURE----- From guido at python.org Mon Mar 12 23:35:15 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 15:35:15 -0700 Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: <45F5D037.6050204@canterbury.ac.nz> References: <45F4D900.4030602@canterbury.ac.nz> <45F5D037.6050204@canterbury.ac.nz> Message-ID: On 3/12/07, Greg Ewing wrote: > Ka-Ping Yee wrote: > > > We have "import as", though. If you want to import the default > > builtins without using them as the builtins, you can say > > > > import __builtin__ as default_builtin > > Seems to me it would be better to rename the module > to a non-double-underscore name. There's really nothing > magical about the module itself, only the name used > by global variable lookups. Not true; modifying the module affects every other module (except those that have imported some alternative). This is why I renamed it to __builtin__ sometime in the dark ages (way before introducing __builtins__). I strongly disagree that it's "just" a regular module. > So just call the module 'builtins', and then you can do > > import builtins # get the module with no magic happening > > import mystuff as __builtins__ # change my builtin namespace > > import builtins as __builtins__ # restore default builtin namespace I'm still +1 on Ping's original proposal #3. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Mar 13 00:00:43 2007 From: barry at python.org (Barry Warsaw) Date: Mon, 12 Mar 2007 19:00:43 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> References: <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 12, 2007, at 3:33 PM, Phillip J. Eby wrote: > For maybe 80-90% of the purposes that I originally created > PyProtocols for, I have found that "simplegeneric" ( http:// > cheeseshop.python.org/simplegeneric/ ) is more than adequate -- and > it's only 80 lines of code. Guido's more featureful GF prototype > isn't that much bigger, I don't believe - heck, it might be smaller > if memory serves. Interesting. Maybe one of the problems here is that we're trying to solve different problems. For me one of the most important aspects of interfaces is the documentation it gives to people wanting to implement alternatives or plugins. Yes, there are other ways to do it, but interfaces seems to strike a good balance documentation-in- the-code and discoverability. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfXbnHEjvBPtnXfVAQLWwwP+OhnhoMmwVBINavffs8iHhKKeqV2pmlwO yzS2PTvSHdVHpij4MEh+XQlKQ1cD04p2KHHEfo3erixYu/TT0ILdEJWHK5oiYjXF v5yargeAVMf2aYLVESeSmlRJ2yzk36s1UL6i5CinabZVvxV26whgzrndY6u/vrV/ IYCCYNwHhOI= =6TZ5 -----END PGP SIGNATURE----- From greg.ewing at canterbury.ac.nz Tue Mar 13 00:06:47 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 12:06:47 +1300 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> Message-ID: <45F5DD07.3070408@canterbury.ac.nz> Guido van Rossum wrote: > I'm still hoping we can get agreement that storing the traceback on > the exception object is the way to go. Only a handful of modules using > pre-baked exception objects would have to be fixed, and likely those > were doing it out of a misunderstanding of the performance > characteristics. One more thought on this -- do we still think it's a good idea to make it possible to raise and catch an exception in C code without instantiating anything? If so, how does this interact with the idea of attaching the traceback to the exception? -- Greg From pje at telecommunity.com Tue Mar 13 00:40:29 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 18:40:29 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> At 07:00 PM 3/12/2007 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mar 12, 2007, at 3:33 PM, Phillip J. Eby wrote: > > > For maybe 80-90% of the purposes that I originally created > > PyProtocols for, I have found that "simplegeneric" ( http:// > > cheeseshop.python.org/simplegeneric/ ) is more than adequate -- and > > it's only 80 lines of code. Guido's more featureful GF prototype > > isn't that much bigger, I don't believe - heck, it might be smaller > > if memory serves. > >Interesting. Maybe one of the problems here is that we're trying to >solve different problems. For me one of the most important aspects >of interfaces is the documentation it gives to people wanting to >implement alternatives or plugins. Yes, there are other ways to do >it, but interfaces seems to strike a good balance documentation-in- >the-code and discoverability. Remember, len(), iter(), and suchlike are all generic functions -- that just happen to be implemented using reserved method names instead of a method registry. Do we need an ILengthMeasurable interface to indicate that you can call len() on something? Or does it suffice to say, "objects passed to this API need to be len()-compatible"? With regard to alternatives and plugins, this is precisely what generic functions enable, with the added benefit that a third party can provide (e.g.) the len() implementation for a class, without having to be the author of it. Yes, adaptation gives you much the same thing, but then, iter() is a generic adaptation function. (That is, adaptation can be trivially implemnented using generic functions, but not the other way around). The principle difference between Python's fixed, built-in generics (len(), iter(), etc.) and simplegeneric et al is that the former requires you to be the author of the type in order to implement the behavior. (Some of Python's stdlib generics like "pickle" do allow you to register implementations, however.) Finally, I posted here several months ago a short "Interface" class that used generic functions for its base - it would let you define generic functions in its body and then do Zope/PyProtocols-style 'IFoo(bar).foo()' operations instead of 'foo(bar)'. It was, I believe, only another 30-40 lines of code. Unless you want all of the fancy introspection features of zope.interface (which I personally think tempt less-experienced people to write code that will bite extenders in the butt) or the exotic meta-features of PyProtocols (which I have found can be replaced by generic functions with "method combination" support), a few hundred lines of code should be more than sufficient to implement both generic functions *and* interfaces for the stdlib's needs. (Btw, when I say that introspection produces butt-biting code, I mean that when *I* first started using zope.interface, I created all sorts of horrors until I realized that interface introspection is an attractive but unnecessary evil, directly analagous to LBYL vs EAFP and type-testing vs. duck-typing issues. See e.g.: http://www.mockobjects.com/2006/10/tell-dont-ask-and-mock-objects.html and http://peak.telecommunity.com/protocol_ref/replintrowadapt.html for some further thoughts on this. Basically, the problem is that if you use interface detection in order to decide what to do with something, you're basically taking away the ability for the developer who comes *after* you to control what happens with the objects they give you. Adaptation and generic functions don't have this problem.) From brett at python.org Tue Mar 13 00:46:10 2007 From: brett at python.org (Brett Cannon) Date: Mon, 12 Mar 2007 16:46:10 -0700 Subject: [Python-3000] Moving files around In-Reply-To: References: <43aa6ff70703102134k7bfbbb24p88dff30171786b97@mail.gmail.com> Message-ID: I will go ahead and add it this to PEP 3108. -Brett On 3/12/07, Guido van Rossum wrote: > +1 for test.support too. -1 for putting code in test/__init__.py. > > On 3/10/07, Collin Winter wrote: > > On 3/10/07, Brett Cannon wrote: > > [snip] > > > And does anyone else find test.test_support an odd name; it isn't > > > testing anything! I really want to rename that module test.support or > > > make it the __init__ module for 'test'. > > > > +1 for test.support, if it's to be renamed. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From brett at python.org Tue Mar 13 00:59:50 2007 From: brett at python.org (Brett Cannon) Date: Mon, 12 Mar 2007 16:59:50 -0700 Subject: [Python-3000] __builtin__ and __builtins__ In-Reply-To: References: <45F4D900.4030602@canterbury.ac.nz> <45F5D037.6050204@canterbury.ac.nz> Message-ID: On 3/12/07, Guido van Rossum wrote: > On 3/12/07, Greg Ewing wrote: > > Ka-Ping Yee wrote: > > > > > We have "import as", though. If you want to import the default > > > builtins without using them as the builtins, you can say > > > > > > import __builtin__ as default_builtin > > > > Seems to me it would be better to rename the module > > to a non-double-underscore name. There's really nothing > > magical about the module itself, only the name used > > by global variable lookups. > > Not true; modifying the module affects every other module (except > those that have imported some alternative). This is why I renamed it > to __builtin__ sometime in the dark ages (way before introducing > __builtins__). I strongly disagree that it's "just" a regular module. > > > So just call the module 'builtins', and then you can do > > > > import builtins # get the module with no magic happening > > > > import mystuff as __builtins__ # change my builtin namespace > > > > import builtins as __builtins__ # restore default builtin namespace > > I'm still +1 on Ping's original proposal #3. > +1 from me as well. If you want to change the built-in values then just import the individual objects into the global namespace, using ``import *`` if you really want to shadow everything. -Brett From guido at python.org Tue Mar 13 01:30:48 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 17:30:48 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F1C734.7080503@acm.org> References: <45F1C734.7080503@acm.org> Message-ID: On 3/9/07, Talin wrote: > I had a conversation with Guido last night at the Python user's group > meeting, and we hashed out some of the details of how metaclasses should > work. I've gone ahead and written up a PEP, which I present for your review. Executive summary: I'm defending the PEP and the metaclass syntax it proposes, and in fact I want the "clas arguments" to have the same syntax as a call, including *args and **kwds. I'm only suggesting to find a new name instead of __metacreate__. While I'm responding to the first message in the thread, I've actually read the entire thread, and I'm responding in tit-for-tat mode to much of it. (Yes, you'll find that there are actually a few messages in the thread so far to which I *didn't* respond, and even some by Greg Ewing. :-) > There were some objections in the discussion to the 'two-phase' > creation process, where the metaclass is invoked twice, once to > create the class dictionary and once to 'finish' the class. Some > people felt that these two phases should be completely separate, in > that there ought to be separate syntax for specifying the custom > dict as for specifying the metaclass. However, in most cases, the > two will be intimately tied together, and the metaclass will most > likely have an intimate knowledge of the internal details of the > class dict. Requiring the programmer to insure that the correct dict > type and the correct metaclass type are used together creates an > additional and unneeded burden on the programmer. Moreover, I expect that in most cases the metaclass will be implicitly provided by one of the base classes. It would be insane to require the *user* of the metaclass to remember to also specify the custom dict creator if the metaclass needs it. > It would be possible to leave the existing __metaclass__ syntax in > place. Alternatively, it would not be too difficult to modify the > syntax rules of the Py3K translation tool to convert from the old to > the new syntax. I'm in favor of ripping it out. Also the module level __metaclass__, which was mostly intended as a way to make all baseless classes in a given module new-style by default. On 3/9/07, Steven Bethard wrote: > > I'd rather __metacreate__ be called something like __createdict__, > __creationdict__, __metadict__, __getmetadict__, etc. where it's > clearer that the purpose is to create a dict object. > Agreed; there's no need that the attribute name contains the word 'meta' again, that's already implied in it being an attribute of the metaclass. On 3/9/07, Brett Cannon wrote: > Do the keywords have to follow the metaclass keyword, or is order > irrelevant? While order makes sense, it would be a new precedent for > keyword arguments to have an important order. I'd like the syntax between () to be identical to that for a function call, i.e. including *args and **kwds. That way the cognitive load for the user is the smallest. Sorting out the semantics of the keywords is purely a runtime activity anywaty. > Does the language spec guarantee that the body of a class will be > executed in definition order? Or is that considered implicit by the > fact that the class body is executed as code? Yes, and yes. On 3/9/07, Jack Diederich wrote: > I am a very big fan of ordered dicts in classes. Would you mind showing us some of your use cases / examples? > One possibility is that > suites in classes always store their order in a special dict that keeps a > side list of key order. A final invisible class decorator around > every class would then toss out the order and leave only a regular dict. This was Talin's initial suggestion (in private conversation with me). I dissuaded him against it because: (a) it would force us to provide a specific ordered dict implementation and anything it didn't handle would remain impossible; and (b) I'm sure there are other use cases such as trapping duplicate method definitions (either to issue errors or to assign special semantics, as may be useful for certain generic function packages). In general I find hardcoding a specific mechanism hard to defend at such a meta-level; I'm much more in favor of providing policy and letting libraries pick their own mechanism. This also lets us provide an ordered-dict-like implementation that only provides just enough information needed for a typical implementation (e.g. something that just keeps a log of all __setitem__ calls) without limiting future uses. > An advantage is that the order of the keys would be available to both > metaclasses and class decorators. It would also require no changes in > signature for either. As far as I can tell ordered dicts are the only > demonstrated use case (and common! looking at any SQL/ORM library). I think that whether the order of the keys is made available to the class decorator should be up to the metaclass. After all it is in charge of what it returns -- it may nor return something resembling a class at all! On 3/10/07, Greg Ewing wrote: > > class Foo(base1, base2, metaclass=mymeta): > > ... > > -1 on this syntax, I think it's ugly. Like beauty, ugliness is in the eye of the beholder. > Alternative proposals: > > class Foo(base1, base2) as MyMeta: > ... -1000; the 'as' keyword in Python suggests assignment or renaming; here you seem to be using the VB meaning of 'as'. > or [snip] -1 on all of these; '=' is just as arbitrary here as 'as' or '<<' or '@'. Anyway the syntax debate is mostly bikeshed color discussion. > The existing syntax actually has some advantages, e.g. > you can do things like > > class Foo: > class __metaclass__: > # Real class methods defined here > > which I think is rather elegant, so I'm not sure I'd > like the new syntax to completely replace the old one. Again, tastes differ. :-) Personally, I find code that does this completely unreadable and I would never accept it in a code review. On 3/10/07, Greg Ewing wrote: > In any case, I've thought of a way to reduce the performance > penalty to near-zero. Consider that the compiler knows > all the names that will be assigned in the class dict > and what order they are in. So all it needs to do is > compile > > class Foo: > a = 5 > def b(self): > ... > def c(self, x): > ... > > as though it were > > class Foo: > __names__ = ('a', 'b', 'c') > a = 5 > def b(self): > ... > def c(self, x): > ... > > The tuple can be a constant of the code object, so the > only overhead is one extra item added to the dict at > class creation time, and none at all for subsequent > access to the class dict. And you don't even need a > special kind of dict. -1; I like Python's compiler to remain stupid. It would also easily be thwarted by use of locals(), exec(), etc. (And yes, there's code using exec in a class def in the stdlib!) Also, Josiah's example of conditional code branches, etc. On 3/10/07, Talin wrote: > I didn't want to call it __metadict__, as it was in the earlier > proposal, because in the current version it's doing more than just > creating the class dict, it's also processing any additional keywords > that are passed in to the class base list. How about __prepare__? On 3/10/07, Josiah Carlson wrote: > Generally, about the only need I forsee for arbitrary keyword arguments > in the class 'foo(...)' signature is to pass arguments to the 'metadict' > factory (method, function, etc.). Otherwise it seems more like a syntax > looking for a purpose than a purpose looking for a syntax. Or to the metaclass later on. I see keyword args as a way to pass parameters to a metaclass that would otherwise have to be assigned to class variables, in a namespace that's much more crowded. Imagine being able to write this: class C(implements=(I1, I2)): ... > Newdict needs to be callable, and if you want to pass certain > semantic-altering arguments, make newdict a factory can call it directly... > > class foo(...): > __metaclass__ = mytype, newdict(no_dupes=True) > > One wouldn't even need any of the default arguments only stuff to make > the two 'newdict' examples above use the same callable. As I said before, this seems to break abstraction to me; if a metaclass requires an ordered dict, the user shouldn't have to remember to pass the appropriate constructor each time that metaclass is used. And most of the time the metaclass is implied by the base classes, not explicitly provided using __metaclass__ -- the latter is mostly an escape hook to resolve metaclass conflicts when combining bases with different metaclasses. On 3/10/07, Phillip J. Eby wrote: > At 12:40 PM 3/10/2007 -0800, Josiah Carlson wrote: > >Great! If there is no need to mix and match metaclasses and dicts > >arbitrarily, then there is no need to pass alternate parameters to the > >dictionary construction method, > > Er, wha? Those two things are utterly unrelated. Also, I'm in utter disagreement with anyone who believes that assignment to __metaclass__ in the class body is somehow better or more elegant than a keyword argument in the class "parameter list" (formerly known as bases :-). On 3/10/07, Greg Ewing wrote: > Okay, then make it > > class Foo(base1, base2) isa MyMeta: > ... And why is that better than a metaclass keyword? 'isa' isn't even an English word. Bah. On 3/10/07, Greg Ewing wrote: > Josiah Carlson wrote: > > As such, the compiler, etc., needs to be taught to pull out the > > __metaclass__ definition before executing the class body, but that's > > better than mucking with the syntax of class foo(...). > > I'd be happy with that. And I think "execution-out-of-order-because-we-don't-want-new-syntax" is a poor excuse for evolutionary language design. On 3/10/07, Greg Ewing wrote: > I still think that extra arguments are unnecessary in > any case, because anything you could do by passing args > to the dict creation function could be done by using a > function instead of a class for the "metaclass". > > Does anyone have a counter-example? Not that you couldn't write it your way, but I find this rather cool looking: class C(implements=(I1, I2)): ... On 3/10/07, Greg Ewing wrote: > Note that you can already pass information to the > metaclass itself using special attributes in the class > namespace. This would only be need if you absolutely > had to pass information when creating the *dict*. > Things like your 'sealed' option don't seem to fall > into that category. I don't know about sealed, but using class attributes topassing parameter (other than the namespace itself) to the metaclass seems a pretty lousy mechanism, and keyword arguments in the classdef are a much cleaner solution for this need. For example, this solves the problem that those attributes remain in the class dict but aren't necessarily candidates for inheritance (example: __slots__, whose inheritance story is, um, complicated; in general C.__slots__ does not necessarily tell you all the slots that the class has). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Mar 13 01:36:20 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 17:36:20 -0700 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: <45F5DD07.3070408@canterbury.ac.nz> References: <45F1B723.8020402@acm.org> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> <45F5DD07.3070408@canterbury.ac.nz> Message-ID: On 3/12/07, Greg Ewing wrote: > Guido van Rossum wrote: > > I'm still hoping we can get agreement that storing the traceback on > > the exception object is the way to go. Only a handful of modules using > > pre-baked exception objects would have to be fixed, and likely those > > were doing it out of a misunderstanding of the performance > > characteristics. > > One more thought on this -- do we still think it's a > good idea to make it possible to raise and catch an > exception in C code without instantiating anything? I don't see it's contradictory to the proposal. > If so, how does this interact with the idea of > attaching the traceback to the exception? The internal mechanism for bubbling an exception up the stack until an except clause catches it could continue to use the (class, instance, traceback) triple, and if raise is passed a class instead of an instance, the instance would be NULL; when forced to instantiate the exception, the traceback collected up to that point is attached to it. If an instance was passed to raise, the __traceback__ pointer in the instance is updated each time we leave a stack frame. I think this means the __traceback__ attribute needs to correspond to a C-level slot in the BaseException instance lay-out. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Tue Mar 13 01:58:07 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 Mar 2007 19:58:07 -0500 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> <45F5DD07.3070408@canterbury.ac.nz> Message-ID: <43aa6ff70703121758h57dca216ia18d62fbb659a469@mail.gmail.com> On 3/12/07, Guido van Rossum wrote: > On 3/12/07, Greg Ewing wrote: > > Guido van Rossum wrote: > > > I'm still hoping we can get agreement that storing the traceback on > > > the exception object is the way to go. Only a handful of modules using > > > pre-baked exception objects would have to be fixed, and likely those > > > were doing it out of a misunderstanding of the performance > > > characteristics. > > > > One more thought on this -- do we still think it's a > > good idea to make it possible to raise and catch an > > exception in C code without instantiating anything? > > I don't see it's contradictory to the proposal. I think it should be kept. During my first pass at implementing PEPs 3110, I tried replacing all the places in C that raise non-instantiated exceptions with instantiating alternatives. It turned out to be a tedious pain in the ass. Of course, if someone else is doing the work... ; ) > > If so, how does this interact with the idea of > > attaching the traceback to the exception? > > The internal mechanism for bubbling an exception up the stack until an > except clause catches it could continue to use the (class, instance, > traceback) triple, and if raise is passed a class instead of an > instance, the instance would be NULL; when forced to instantiate the > exception, the traceback collected up to that point is attached to it. > If an instance was passed to raise, the __traceback__ pointer in the > instance is updated each time we leave a stack frame. I think this > means the __traceback__ attribute needs to correspond to a C-level > slot in the BaseException instance lay-out. This a good summary of the strategy I've taken while implementing PEP 344. Collin Winter From janssen at parc.com Tue Mar 13 02:06:33 2007 From: janssen at parc.com (Bill Janssen) Date: Mon, 12 Mar 2007 18:06:33 PDT Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> Message-ID: <07Mar12.170642pst."57996"@synergy1.parc.xerox.com> Phillip, I think there is a certain schizophrenia in Python between the older original procedural/functional orientation (itself somewhat split), and the newer-fangled (for Python) object style, between len(obj) and obj.len(). That's what makes it hard to evaluate the pros and cons between adding generic functions (as Lisp did, to support its own functional style), and adding interfaces (in whatever fashion) to support the object-oriented view of life. Bill From collinw at gmail.com Tue Mar 13 02:15:16 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 12 Mar 2007 20:15:16 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: <43aa6ff70703121815m1d639693mfa4d9dd244e06983@mail.gmail.com> On 3/12/07, Guido van Rossum wrote: [snip stuff I agree with] > On 3/9/07, Brett Cannon wrote: > > Do the keywords have to follow the metaclass keyword, or is order > > irrelevant? While order makes sense, it would be a new precedent for > > keyword arguments to have an important order. > > I'd like the syntax between () to be identical to that for a function > call, i.e. including *args and **kwds. That way the cognitive load for > the user is the smallest. Sorting out the semantics of the keywords is > purely a runtime activity anywaty. Woohoo! I've wanted to be able to say "class X(*args):" before, so this is great news. On a 2to3-related topic, will a __metaclass__ class attribute do the same thing as a metaclass keyword argument, or will a fixer be needed for this? Collin Winter From pje at telecommunity.com Tue Mar 13 02:36:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 20:36:15 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <45F1C734.7080503@acm.org> Message-ID: <5.1.1.6.0.20070312203341.044f1e80@sparrow.telecommunity.com> At 05:30 PM 3/12/2007 -0700, Guido van Rossum wrote: >I don't know about sealed, but using class attributes topassing >parameter (other than the namespace itself) to the metaclass seems a >pretty lousy mechanism, and keyword arguments in the classdef are a >much cleaner solution for this need. For example, this solves the >problem that those attributes remain in the class dict but aren't >necessarily candidates for inheritance Ah, I missed that point about the new kwargs mechanism! Before I was +0 on *args/**kw, now I'm +1. I've actually needed that feature more than once, but have been doing it with specialized in-body class decorators, like this: class FooService(context.Service): context.replaces(BarService) ... Which could presumably now become: class FooService(context.Service, replaces=BarService): ... Right? From pje at telecommunity.com Tue Mar 13 02:46:27 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 20:46:27 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <07Mar12.170642pst."57996"@synergy1.parc.xerox.com> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070312203632.044f3008@sparrow.telecommunity.com> At 06:06 PM 3/12/2007 -0700, Bill Janssen wrote: >Phillip, > >I think there is a certain schizophrenia in Python between the older >original procedural/functional orientation (itself somewhat split), >and the newer-fangled (for Python) object style, between len(obj) and >obj.len(). Where you see schizophrenia, I see a set of mostly very wise choices, actually. >That's what makes it hard to evaluate the pros and cons >between adding generic functions (as Lisp did, to support its own >functional style), and adding interfaces (in whatever fashion) to >support the object-oriented view of life. And one of the benefits of Python is that it doesn't kowtow, Java-like to an exclusively "object-oriented view of life". Life involves more than objects, thank goodness. :) Btw, I don't think the evaluation is difficult at all, and in any case it's a false dichotomy. It's easy to implement interfaces and adaptation over generic functions, but much harder to do the reverse. So if we have to start somewhere, we might as well start with the more powerful feature that's less prone to error and abuse. In other words, if we must have interfaces, then let's steal more from Haskell (typeclasses and instances defined in terms of generic functions) and less from Java (ABCs and name-based abstract method collections). :) From guido at python.org Tue Mar 13 02:48:06 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 17:48:06 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <43aa6ff70703121815m1d639693mfa4d9dd244e06983@mail.gmail.com> References: <45F1C734.7080503@acm.org> <43aa6ff70703121815m1d639693mfa4d9dd244e06983@mail.gmail.com> Message-ID: On 3/12/07, Collin Winter wrote: > On 3/12/07, Guido van Rossum wrote: > [snip stuff I agree with] > > On 3/9/07, Brett Cannon wrote: > > > Do the keywords have to follow the metaclass keyword, or is order > > > irrelevant? While order makes sense, it would be a new precedent for > > > keyword arguments to have an important order. > > > > I'd like the syntax between () to be identical to that for a function > > call, i.e. including *args and **kwds. That way the cognitive load for > > the user is the smallest. Sorting out the semantics of the keywords is > > purely a runtime activity anywaty. > > Woohoo! I've wanted to be able to say "class X(*args):" before, so > this is great news. > > On a 2to3-related topic, will a __metaclass__ class attribute do the > same thing as a metaclass keyword argument, or will a fixer be needed > for this? I'd like a fixer. The fixer could assume the __metaclass__ ass't is first in the class, possibly after a docstring; I'm guessing this would capture 99% of actual use cases and I'm sure people who put it somewhere else can move it forward so they'll have code that converts cleanly. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benji at benjiyork.com Tue Mar 13 03:03:35 2007 From: benji at benjiyork.com (Benji York) Date: Mon, 12 Mar 2007 22:03:35 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> Message-ID: <45F60677.5030304@benjiyork.com> Phillip J. Eby wrote: > I posted here several months ago a short "Interface" class that > used generic functions for its base How could Python use generic functions to accomplish what the proposed ABCs aim to? By first implementing interfaces? Perhaps by implementing is_file() with a generic function individuals could add implementations for that return True? -- Benji York http://benjiyork.com From pmaupin at gmail.com Tue Mar 13 03:47:21 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 12 Mar 2007 21:47:21 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! Message-ID: Eric Smith and I have a reasonable first-cut of a C implementation for Talin's PEP3101 (it runs as an extension module and has been tested on Python 2.3, 2.4, and 3.0) along with some test cases. It's sort of experimental, in that it mostly implements the PEP, but also implements a few possible enhancements and changes, to give us (well, me, mostly) an idea of what works and what doesn't, and ideas about changes that might be useful. This list of potential changes to the PEP is in order of (what I believe to be) most contentious first. (It will be interesting to contrast that with the actual votes :). I apologize for the long list, but it's quite a comprehensive PEP, and the implementation work Eric and I have done has probably encompassed the most critical examination of the PEP to date, so here goes: Feature: Alternate syntaxes for escape to markup. There is naturally a strong desire to resist having more than one way to do things. However, in my experience using Python to generate lots of text (which experience is why I volunteered to work on the PEP implementation in the first place!), I have found that there are different application domains which naturally lend themselves to different approaches for escape to markup. The issues are: - In some application domains, the number of {} characters in the actual character data is dizzying. This leads to two problems: first that there are an inordinate number of { and } characters to escape by doubling, and second (and even more troubling) is that it can be very difficult to locate the markup inside the text file, without an editor (and user!) that understands fancy regexps. - In other application domains, the sheer number of {} characters is not so bad (and markup can be readily noticed), but the use of {{ for { is very confusing, because { has a particular technical meaning (rather than just offsetting some natural language text), and it is hard for people to mentally parse the underlying document when the braces are doubled. To deal with these issues, I propose having a total of 3 markup syntaxes, with one of them being the default, and with a readable, defined method for the string to declare it is using one of the other markup syntaxes. (If this is accepted, I feel that EIBTI says the string itself should declare if it is using other than the standard markup transition sequence. This also makes it easier for any automated tools to understand the insides of the strings.) The default markup syntax is the one proposed in the initial PEP, where literal { and } characters are denoted by doubling them. It works well for many problem domains, and is the best for short strings (where it would be burdensome to have the string declare the markup syntax it is using). The second method is the well-understood ${} syntax. The $ is easy to find in a sea of { characters, and the only special handling required is that every $ must be doubled. The third method is something I came up with a couple of years ago when I was generating a lot of C code from templates. It would make any non-Python programmer blanch, because it relies on significant whitespace, but it made for very readable technical templates. WIth this method "{foo}" escapes to markup, but when there is whitespace after the leading "{", e.g. "{ foo}", the brace is not an escape to markup. If the whitespace is a space, it is removed from the output, but if it is '\r', '\n', or '\t', then it is left in the output. The result is that, for example, most braces in most C texts do not need to have spaces inserted after them, because they immediately precede a newline. The syntaxes are similar enough that they can all be efficiently parsed by the same loop, so there are no real implementation issues. The currently contemplated method for declaring a markup syntax is by using decorator-style markup, e.g. {@syntax1} inside the string, although I am open to suggestions about better-looking ways to do this. Feature: Automatic search of locals() and globals() for name lookups if no parameters are given. This is contentious because it violates EIBTI. However, it is extremely convenient. To me, the reasons for allowing or disallowing this feature on 'somestring'.format() appear to be exactly the same as the reasons for allowing or disallowing this feature on eval('somestring'). Barring a distinction between these cases that I have not noticed, I think that if we don't want to allow this for 'somestring'.format(), then we should seriously consider removing the capability in Python 3000 for eval('somestring'). Feature: Ability to pass in a dictionary or tuple of dictionaries of namespaces to search. This feature allows, in some cases, for much more dynamic code than *kwargs. (You could manually smush multiple dictionaries together to build kwargs, but that can be ugly, tedious, and slow.) Implementation-wise, this feature and locals() / globals() go hand in hand. Feature: Placement of a dummy record on the traceback stack for underlying errors. There are two classes of exceptions which can occur during formatting: exceptions generated by the formatter code itself, and exceptions generated by user code (such as a field object's getattr function, or the field_hook function). In general, exceptions generated by the formatter code itself are of the "ValueError" variety -- there is an error in the actual "value" of the format string. (This is not strictly true; for example, the string.format() function might be passed a non-string as its first parameter, which would result in a TypeError.) The text associated with these internally generated ValueError exceptions will indicate the location of the exception inside the format string, as well as the nature of the exception. For exceptions generated by user code, a trace record and dummy frame will be added to the traceback stack to help in determining the location in the string where the exception occurred. The inserted traceback will indicate that the error occurred at: File "?format_string?", line X, in column_Y where X and Y represent the line and character position of where the exception occurred inside the format string. THIS IS A HACK! There is currently no general mechanism for non-Python source code to be added to a traceback (which might be the subject of another PEP), but there is some precedent and some examples, for example, in PyExpat and in Pyrex, of adding non-Python code information to the traceback stack. Even though this is a hack, the necessity of debugging format strings makes this a very useful hack -- the hack means that the location information of the error within the format string is available and can be manipulated, just like the location of the error within any Python module for which source is not available. Removed feature: Ability to dump error information into the output string. The original PEP added this capability for the same reason I am proposing the traceback hack. OTOH, with the traceback hack in place, it would be extremely easy to write a pure-Python wrapper for format that would trap an exception, parse the traceback, and place the error message at the appropriate place in the string. (For a bit more effort, the exception wrapper could re-call the formatter with the remainder of the string if you really wanted to see ALL the errors, but this is actually much more functionality than you get with Python itself for debugging, so while I see the usefulness, I don't know if it justifies the effort.) Feature: Addition of functions and "constants" to string module. The PEP proposes doing everything as string methods, with a "cformat" method allowing some access to the underlying machinery. I propose only having a 'format' method of the string (unicode) type, and a corresponding 'format' and extended 'flag_format' function in the string module, along with definitions for the flags for access to non-default underlying format machinery. Feature: Ability for "field hook" user code function to only be called on some fields. The PEP takes an all-or-nothing approach to the field hook -- it is either called on every field or no fields. Furthermore, it is only available for calling if the extended function ('somestring'.cformat() in the spec, string.flag_format() in this proposal) is called. The proposed change keeps this functionality, but also adds a field type specifier 'h' which causes the field hook to be called as needed on a per-field basis. This latter method can even be used from the default 'somestring'.format() method. Changed feature: By default, not using all arguments is not an exception The original PEP says that, by default, the formatting operation should check that all arguments are used. In the original % operator, the exception reporting that an extra argument is present is symmetrical to the exception reporting that not enough arguments are present. Both these errors are easy to commit, because it is hard to count the number of arguments and the number of % specifiers in your string and make sure they match. In theory, the new string formatting should make it easier to get the arguments right, because all arguments in the format string are numbered or even named, and with the new string formatting, the corresponding error is that a _specific_ argument is missing, not just "you didn't supply enough arguments." Also, it is arguably not Pythonic to require a check that all arguments to a function are actually used by the execution of the function (but see interfaces!), and format() is, after all, just another function. So it seems that the default should be to not check that all the arguments are used. In fact, there are similar reasons for not using all the arguments here as with any other function. For example, for customization, the format method of a string might be called with a superset of all the information which might be useful to view. Because the ability to check that all arguments is used might be useful (and because I don't want to be accused of leaving this feature out because of laziness :), this feature remains available if string.flag_format() is called. Feature: Ability to insert non-printing comments in format strings This feature is implemented in a very intuitive way, e.g. " text {# your comment here} more text" (example shown with the default transition to markup syntax). One of the nice benefits of this feature is the ability to break up long source lines (if you have lots of long variable names and attribute lookups). Feature: Exception raised if attribute with leading underscore accessed. The syntax supported by the PEP is deliberately limited in an attempt to increase security. This is an additional security measure, which is on by default, but can be optionally disabled if string.flag_format() is used instead of 'somestring'.format(). Feature: Support for "center" alignment. The field specifier uses "<" and ">" for left and right alignment. This adds "^" for center alignment. Feature: support of earlier versions of Python (This was contemplated but not mandated in the earlier PEP, but implementation has proven to be quite easy.) This should work with both string and unicode objects in 2.6, and should be available as a separate compilable extension module for versions back to 2.3 (functions only -- no string method support). Feature: no global state The original PEP specified a global error-handling mode, which controlled a few capabilities. The current implementation has no global state -- any non-default options are accessed by using the string.flag_format() function. Thanks in advance for any feedback you can provide on these changes. Regards, Pat From steven.bethard at gmail.com Tue Mar 13 03:48:19 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 12 Mar 2007 20:48:19 -0600 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> References: <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> Message-ID: On 3/12/07, Phillip J. Eby wrote: > At 02:14 PM 3/12/2007 -0400, Barry Warsaw wrote: > >The question then becomes whether we really want to invent a third > >way of doing interfaces in Python, or whether our time is better > >spent selecting and promoting one of the existing, tried-and-true > >packages. Perhaps it would be better for us to figure out how and if > >the two packages can be integrated. If a third way is still > >advocated, then I think it must have a fairly high barrier to > >acceptance in order to push aside such long accepted libraries. > > Personally I think both packages are overkill for the language and/or > stdlib, if only because they're solutions for the wrong problem. (Note > that PEPs 245 and 246 were rejected for related reasons.) > > For maybe 80-90% of the purposes that I originally created PyProtocols for, > I have found that "simplegeneric" ( > http://cheeseshop.python.org/simplegeneric/ ) is more than adequate -- and > it's only 80 lines of code. I believe the correct URL is: http://cheeseshop.python.org/pypi/simplegeneric/ Nice, clear examples with this BTW. > Of course, generic functions require you to say 'foo(bar)' instead of > 'bar.foo()' (and IIUC, that's the big sticking point for Guido wrt to GF's > in Py3K). Yeah, I'd be happy to see things like ``len()`` and ``iter()`` become generic functions like these (they're already most of the way there) but I'm not sure I'm ready to start writing ``dict.update(d, ...)`` instead of ``d.update(...)``. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From steven.bethard at gmail.com Tue Mar 13 03:57:19 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 12 Mar 2007 20:57:19 -0600 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: On 3/12/07, Guido van Rossum wrote: > On 3/9/07, Brett Cannon wrote: > > Do the keywords have to follow the metaclass keyword, or is order > > irrelevant? While order makes sense, it would be a new precedent for > > keyword arguments to have an important order. > > I'd like the syntax between () to be identical to that for a function > call, i.e. including *args and **kwds. That way the cognitive load for > the user is the smallest. Sorting out the semantics of the keywords is > purely a runtime activity anywaty. Just to clarify, while the syntax between the () in the class definition will be the same as that of a function call, the signature of the method called will be:: __prepare__(name, args, kwargs) not __prepare__(name, *args, **kwargs) right? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From tjreedy at udel.edu Tue Mar 13 03:58:24 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 12 Mar 2007 22:58:24 -0400 Subject: [Python-3000] PEP for Metaclasses in Python 3000 References: <45F1C734.7080503@acm.org> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20703121730r244f0314j38e8ffe9ebb5efcf at mail.gmail.com... FWIW, I agree that class definition meta-information should go in the header rather that the body. To me, metaclass=mymeta at the top is much prettier than __metaclass__ = mymeta in the body. It will also make it obvious to the reader immediately that something different is going to happen. I always understood the module level assignment to be a temporary kludge until the new class transition was completed. tjr From guido at python.org Tue Mar 13 04:03:06 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 12 Mar 2007 19:03:06 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: Unsure why you present this as a question; I'm not sure anyone has thought much about it yet. I wonder if the call shouldn't be made like this: __prepare__(name, bases, **kwargs) so that if you only expect certain specific keyword args you can define it like this: def __prepare__(name, base, metaclass=X): ... On 3/12/07, Steven Bethard wrote: > On 3/12/07, Guido van Rossum wrote: > > On 3/9/07, Brett Cannon wrote: > > > Do the keywords have to follow the metaclass keyword, or is order > > > irrelevant? While order makes sense, it would be a new precedent for > > > keyword arguments to have an important order. > > > > I'd like the syntax between () to be identical to that for a function > > call, i.e. including *args and **kwds. That way the cognitive load for > > the user is the smallest. Sorting out the semantics of the keywords is > > purely a runtime activity anywaty. > > Just to clarify, while the syntax between the () in the class > definition will be the same as that of a function call, the signature > of the method called will be:: > __prepare__(name, args, kwargs) > not > __prepare__(name, *args, **kwargs) > right? > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Mar 13 04:06:05 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 22:06:05 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F60677.5030304@benjiyork.com> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> At 10:03 PM 3/12/2007 -0400, Benji York wrote: >Phillip J. Eby wrote: > > I posted here several months ago a short "Interface" class that > > used generic functions for its base > >How could Python use generic functions to accomplish what the proposed >ABCs aim to? Which specific aim do you have in mind? > By first implementing interfaces? Perhaps by implementing >is_file() with a generic function individuals could add implementations >for that return True? In most cases, having such introspection functions is an attractive nuisance. It's attractive in Python because it grows out of reusing built-in types as syntactic shorthand, which is all very well and good until you start trying to apply it to non-builtin types too. I assert that an is_file(x) function that does something different than isinstance(x,file) is a bad idea, because all of the use cases that lead to somebody wanting to know this, are better satisfied by either: 1. simply trying to use the object as if it were a file (i.e., rely on the caller to pass you the right type), or 2. defining a generic function for whatever it was you were going to *do* with the file-like object, and leaving it to others to implement *that* method (which can always call your 'file'-type implementation if it needs to). The presence of an is_file() on the other hand, convinces people that it's okay to write a bunch of if-then tests... which leads to broken and non-extensible code like what is found in pydoc and epydoc, that won't work properly with any Python objects whose existence the authors didn't anticipate. I routinely have to reverse engineer Pydoc's oddities so that my objects don't produce bizarre or misleading information in help() (and I recently saw a blog post from Robert Brewer that indicates it's not just me who has that problem). Now, in *theory* pydoc should be just dandy because it's full of calls to 'isroutine()' and 'ismethod()' and similar "abstract interface" stuff. In *practice*, those calls occur in big if-then trees that it uses to decide, based on what a thing *is*, how to document it. What it *should* be doing instead, is invoking generics like get_call_signature(), iter_members(), defined_by_module(), and so on. Now, you could argue that the problem is that Python lacks ABCs or well-defined interfaces for these things. But it wouldn't actually help this problem at *all*. See, just because you've defined an interface for how the calling signature is stored in a function's attributes, doesn't mean that that's how every object can (or even *should*) store its calling signature. For example, calling a class usually ends up calling its __init__, but it could also go to the metaclass' __call__. Pydoc just ignores the issue altogether, because it assumes that 'isclass' and 'isroutine' (for example) are mutually exclusive. Meanwhile, just because you've defined interfaces about class or module dictionaries, doesn't mean that everything you might wish to document, wants to use a dictionary to define its members. Maybe its members are *ordered*, for example! Thus, the *right* way to do interfaces is not for component *providers* to offer them, but rather for component *consumers* to define what interfaces they *require*. In other words, formalizing the interfaces provided by "classes", "modules", "routines", etc., won't fix pydoc's problems in the slightest. What's needed is not for the language to define interfaces for the things to be used by pydoc, but rather for *pydoc* to define "documentable object" interface(s), instead. (And provide implementations of these interfaces/generics for the builtin types.) Then, you could add documentation support to *any* object, instead of having to put all sorts of phony attributes on your objects to trick Pydoc into thinking they're sort of like some object type that pydoc knows about. (And hoping that in the process you won't cause some *other* library to decide to do something weird with your objects as a result of having those phony attributes!) In short, this is why I see the ABC and interface stuff as a non-solution to the wrong problem. It's starting at the wrong end of the telescope, so to speak, dealing in the minutiae of the solution domain (the language), instead of in the big picture of the problem domain (what people want to *do* with the language). In short, the very idea of 'is_file()' is wrong, wrong, wrong. At least, if your goal is to make libraries more robust and reusable. It leads inevitably to the brokenness seen in Pydoc -- and the comparable brokenness that existed in Zope prior to its replacing most introspection by adaptation. (To be honest, I'm *assuming* that those broken bits went away as soon as adaptation became the recommended option -- I don't know if it really did or not, as I haven't done any real Zope work in a few years.) From steven.bethard at gmail.com Tue Mar 13 04:32:30 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 12 Mar 2007 21:32:30 -0600 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: On 3/12/07, Steven Bethard wrote: > the signature of the method called will be:: > __prepare__(name, args, kwargs) > not > __prepare__(name, *args, **kwargs) > right? On 3/12/07, Guido van Rossum wrote: > I'm not sure anyone has thought much about it yet. I wonder > if the call shouldn't be made like this: > > __prepare__(name, bases, **kwargs) > > so that if you only expect certain specific keyword args you can > define it like this: > > def __prepare__(name, base, metaclass=X): ... Yeah, seems like **kwargs would be more useful in practice. Initially, I was concerned that this would break the symmetry with the __metaclass__ signature:: def __metaclass__(name, bases, bodydict): def __prepare__(name, bases, kwargs): But then I realized that there's really no need for any symmetry here -- they're getting passed two different things (kwargs != bodydict) and they're serving two pretty different purposes. I think there might be a parallel argument for passing in the bases as *args instead of as a tuple. If you want to write a metaclass that expects exactly zero or one base classes, you could then write:: def __prepare__(name, **kwargs): def __prepare__(name, base, **kwargs): STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Tue Mar 13 04:36:03 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 22:36:03 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070312223226.028ed9c0@sparrow.telecommunity.com> At 08:48 PM 3/12/2007 -0600, Steven Bethard wrote: >On 3/12/07, Phillip J. Eby wrote: > > For maybe 80-90% of the purposes that I originally created PyProtocols for, > > I have found that "simplegeneric" ( > > http://cheeseshop.python.org/simplegeneric/ ) is more than adequate -- and > > it's only 80 lines of code. > >I believe the correct URL is: > http://cheeseshop.python.org/pypi/simplegeneric/ Oops. > > Of course, generic functions require you to say 'foo(bar)' instead of > > 'bar.foo()' (and IIUC, that's the big sticking point for Guido wrt to GF's > > in Py3K). > >Yeah, I'd be happy to see things like ``len()`` and ``iter()`` become >generic functions like these (they're already most of the way there) >but I'm not sure I'm ready to start writing ``dict.update(d, ...)`` >instead of ``d.update(...)``. If you *know* you're using a dict, then of course d.update() is preferable. But wouldn't it be *nice* if you *could* call dict.update(d, ...) on anything that had a __setitem__? :) From pje at telecommunity.com Tue Mar 13 04:42:06 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Mon, 12 Mar 2007 22:42:06 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> At 09:32 PM 3/12/2007 -0600, Steven Bethard wrote: >On 3/12/07, Steven Bethard wrote: > > the signature of the method called will be:: > > __prepare__(name, args, kwargs) > > not > > __prepare__(name, *args, **kwargs) > > right? > >On 3/12/07, Guido van Rossum wrote: > > I'm not sure anyone has thought much about it yet. I wonder > > if the call shouldn't be made like this: > > > > __prepare__(name, bases, **kwargs) > > > > so that if you only expect certain specific keyword args you can > > define it like this: > > > > def __prepare__(name, base, metaclass=X): ... > >Yeah, seems like **kwargs would be more useful in practice. Really? Why? I can more easily see circumstances where you'd want to restrict the arguments. In the cases I have where I'd switch from a class decorator to a class keyword, I have a small set of decorators and would have an equally small set of keywords. >I think there might be a parallel argument for passing in the bases as >*args instead of as a tuple. If you want to write a metaclass that >expects exactly zero or one base classes, you could then write:: > > def __prepare__(name, **kwargs): > def __prepare__(name, base, **kwargs): Ugh. In principle, this isn't an issue if you can have keyword-only arguments, but I just don't like it. Maybe that's because in my mind, this *is* a parallel with the type() signature, and I don't see a reason for it to be different. From steven.bethard at gmail.com Tue Mar 13 04:40:57 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 12 Mar 2007 21:40:57 -0600 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312223226.028ed9c0@sparrow.telecommunity.com> References: <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312223226.028ed9c0@sparrow.telecommunity.com> Message-ID: On 3/12/07, Phillip J. Eby wrote: > At 08:48 PM 3/12/2007 -0600, Steven Bethard wrote: > >On 3/12/07, Phillip J. Eby wrote: > > > Of course, generic functions require you to say 'foo(bar)' instead of > > > 'bar.foo()' (and IIUC, that's the big sticking point for Guido wrt to GF's > > > in Py3K). > > > >Yeah, I'd be happy to see things like ``len()`` and ``iter()`` become > >generic functions like these (they're already most of the way there) > >but I'm not sure I'm ready to start writing ``dict.update(d, ...)`` > >instead of ``d.update(...)``. > > If you *know* you're using a dict, then of course d.update() is > preferable. But wouldn't it be *nice* if you *could* call dict.update(d, > ...) on anything that had a __setitem__? :) Definitely. It would certainly make implementing DictMixin simpler (if it didn't eliminate the need for it entirely). STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From steven.bethard at gmail.com Tue Mar 13 04:45:26 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 12 Mar 2007 21:45:26 -0600 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: On 3/12/07, Phillip J. Eby wrote: > At 09:32 PM 3/12/2007 -0600, Steven Bethard wrote: > >On 3/12/07, Steven Bethard wrote: > > > the signature of the method called will be:: > > > __prepare__(name, args, kwargs) > > > not > > > __prepare__(name, *args, **kwargs) > > > right? > > > >On 3/12/07, Guido van Rossum wrote: > > > I'm not sure anyone has thought much about it yet. I wonder > > > if the call shouldn't be made like this: > > > > > > __prepare__(name, bases, **kwargs) > > > > > > so that if you only expect certain specific keyword args you can > > > define it like this: > > > > > > def __prepare__(name, base, metaclass=X): ... > > > >Yeah, seems like **kwargs would be more useful in practice. > > Really? Why? I can more easily see circumstances where you'd want to > restrict the arguments. In the cases I have where I'd switch from a class > decorator to a class keyword, I have a small set of decorators and would > have an equally small set of keywords. Maybe I'm misunderstanding but isn't this a reason that you'd *want* the ``**kwargs`` signature? With the plain ``kwargs`` signature from the PEP you'd have to do something like:: def __prepare__(name, args, kwargs): required = kwargs.pop('required', None) if required is not None: ... STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From jseutter at gmail.com Tue Mar 13 06:05:03 2007 From: jseutter at gmail.com (Jerry Seutter) Date: Mon, 12 Mar 2007 23:05:03 -0600 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: Message-ID: <2c8d48d70703122205s4b10429k3945546605a5a020@mail.gmail.com> On 3/12/07, Patrick Maupin wrote: > > Eric Smith and I have a reasonable first-cut of a C implementation for > Talin's PEP3101 (it runs as an extension module and has been tested on > Python 2.3, 2.4, and 3.0) along with some test cases. It's sort of > experimental, in that it mostly implements the PEP, but also > implements a few possible enhancements and changes, to give us (well, > me, mostly) an idea of what works and what doesn't, and ideas about > changes that might be useful. > > This list of potential changes to the PEP is in order of (what I > believe to be) most contentious first. (It will be interesting to > contrast that with the actual votes :). I apologize for the long > list, but it's quite a comprehensive PEP, and the implementation work > Eric and I have done has probably encompassed the most critical > examination of the PEP to date, so here goes: > > > Feature: Alternate syntaxes for escape to markup. > > There is naturally a strong desire to resist having more than one way > to do things. However, in my experience using Python to generate lots > of text (which experience is why I volunteered to work on the PEP > implementation in the first place!), I have found that there are > different application domains which naturally lend themselves to > different approaches for escape to markup. The issues are: > > - In some application domains, the number of {} characters in the > actual character data is dizzying. This leads to two problems: first > that there are an inordinate number of { and } characters to escape by > doubling, and second (and even more troubling) is that it can be very > difficult to locate the markup inside the text file, without an editor > (and user!) that understands fancy regexps. > > - In other application domains, the sheer number of {} characters is > not so bad (and markup can be readily noticed), but the use of {{ for > { is very confusing, because { has a particular technical meaning > (rather than just offsetting some natural language text), and it is > hard for people to mentally parse the underlying document when the > braces are doubled. > > To deal with these issues, I propose having a total of 3 markup > syntaxes, with one of them being the default, and with a readable, > defined method for the string to declare it is using one of the other > markup syntaxes. (If this is accepted, I feel that EIBTI says the > string itself should declare if it is using other than the standard > markup transition sequence. This also makes it easier for any > automated tools to understand the insides of the strings.) > > The default markup syntax is the one proposed in the initial PEP, > where literal { and } characters are denoted by doubling them. It > works well for many problem domains, and is the best for short strings > (where it would be burdensome to have the string declare the markup > syntax it is using). > > The second method is the well-understood ${} syntax. The $ is easy to > find in a sea of { characters, and the only special handling required > is that every $ must be doubled. I like ${}. I have admired Ruby's #{} when I've had to use it, as it is easy to read. It kind of reminds me of #!/bin/sh in shell scripts, reminding me that it is something magic. Disclaimer: I know nothing of the consequences to enabling this. Jerry Seutter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070312/422b8135/attachment.htm From ncoghlan at gmail.com Tue Mar 13 10:16:42 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Mar 2007 19:16:42 +1000 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: <45F66BFA.1040609@gmail.com> Guido van Rossum wrote: > On 3/9/07, Steven Bethard wrote: >> >> I'd rather __metacreate__ be called something like __createdict__, >> __creationdict__, __metadict__, __getmetadict__, etc. where it's >> clearer that the purpose is to create a dict object. >> > > Agreed; there's no need that the attribute name contains the word > 'meta' again, that's already implied in it being an attribute of the > metaclass. Along similar lines, I'd be marginally happier with: class Bob(meta=Planet): pass over what is currently proposed in the PEP. Repeating the 'class' part within a single line feels redundant. > On 3/9/07, Brett Cannon wrote: >> Do the keywords have to follow the metaclass keyword, or is order >> irrelevant? While order makes sense, it would be a new precedent for >> keyword arguments to have an important order. > > I'd like the syntax between () to be identical to that for a function > call, i.e. including *args and **kwds. That way the cognitive load for > the user is the smallest. Sorting out the semantics of the keywords is > purely a runtime activity anywaty. I like this idea. I think the signature of the method that creates the namespace dictionary requires some further thought though: - is it intended to typically be a static method or a class method of the metaclass? (as it will be called before __new__, an instance method of the metaclass wouldn't make sense) - is the metaclass passed in to the namespace creation function as a normal keyword argument, or is it automatically removed by the interpreter? (note that implementing it as a class method would still allow access to the actual metaclass) - if the metaclass is derived rather than passed in explicitly, is the 'meta' argument still passed in to the namespace creation function? - what's the actual signature of the new function? The answers to these will be somewhat intertwined - if the 'meta' keyword argument is always going to be passed in to the function, then it makes more sense to use a static method in the typical case. If the meta keyword is stripped (or only provided when passed in explicitly), then a class method is needed in order to reliably get at the metaclass itself. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Tue Mar 13 10:17:56 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 22:17:56 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: <45F66C44.60207@canterbury.ac.nz> Steven Bethard wrote: > Initially, I was concerned that this would break the symmetry with the > __metaclass__ signature:: > > def __metaclass__(name, bases, bodydict): > def __prepare__(name, bases, kwargs): Can someone clarify something here: Are the keywords going to be passed to the prepare function only, or to both the the prepare function and the metaclass? If they're only passed to the prepare function, it means that any metaclass that's interested in them will have to implement a prepare function that creates some object to hold onto them, even if it has no other need for a custom namespace. If they're passed to both, then the signatures become metaclass.__prepare__(name, bases, **kwargs) metaclass(name, bases, body, **kwargs) BTW, I don't think I like the name "__prepare__" much more than "__metacreate__". It seems just as wooly. What's being prepared? How? What for? -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 13 10:31:35 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 22:31:35 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: <45F66F77.5050402@canterbury.ac.nz> Guido van Rossum wrote: > > On 3/10/07, Greg Ewing wrote: > > > class Foo: > > class __metaclass__: > > Personally, I find code that does this completely unreadable The reason I'd be a little disappointed to lose this is that it provides a concise way of defining genuine class methods (as opposed to the odd beasts created by classmethod()). Sometimes you need them, e.g. for __xxx__ methods, or if you want inheritance to work properly. Having said that, I can't remember whether I've ever actually used this, and I probably wouldn't miss it all that much. > I find this rather cool looking: > > class C(implements=(I1, I2)): ... I'm a bit worried that class headers are going to become rather cluttered if we start putting all sorts of stuff in there. E.g. are you intending to put __slots__ there? It makes sense, but it could lead to some rather long and unwieldy class headers. -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 13 10:39:03 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 22:39:03 +1300 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <43aa6ff70703121206o3590eb28xfa78628dc52686cf@mail.gmail.com> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> <45F5DD07.3070408@canterbury.ac.nz> Message-ID: <45F67137.1070707@canterbury.ac.nz> Guido van Rossum wrote: > The internal mechanism for bubbling an exception up the stack until an > except clause catches it could continue to use the (class, instance, > traceback) triple, Hmmm, so most of that complexity will still be there at the C level. I was hoping there might be some simplification there as well. However... > and if raise is passed a class instead of an > instance, the instance would be NULL; when forced to instantiate the > exception, the traceback collected up to that point is attached to it. Would this apply to Python code as well? I.e. if you use a raise statement with a class, it doesn't get instantiated immediately? And if you catch it with an except clause which doesn't capture the exception, it's never instantiated? That would be a bonus. > If an instance was passed to raise, the __traceback__ pointer in the > instance is updated each time we leave a stack frame. So it seems we're back to declaring pre-instantiated exceptions to be bad style, which you said you didn't like earlier -- have you changed your mind? What about thread safety? Do we just document that using pre-instantiated exceptions is not thread-safe? -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 13 11:04:07 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 23:04:07 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F66BFA.1040609@gmail.com> References: <45F1C734.7080503@acm.org> <45F66BFA.1040609@gmail.com> Message-ID: <45F67717.4070107@canterbury.ac.nz> Nick Coghlan wrote: > - is it intended to typically be a static method or a class method of > the metaclass? (as it will be called before __new__, an instance method > of the metaclass wouldn't make sense) Looks like it will have to be either a staticmethod or classmethod, or else you'll have to give your metaclass a metametaclass and make it an instance method of that. Maybe this is another case for an implicitly-static method, a la __new__? > - is the metaclass passed in to the namespace creation function as a > normal keyword argument, or is it automatically removed by the > interpreter? I would hope it's removed. > If the > meta keyword is stripped (or only provided when passed in explicitly), > then a class method is needed in order to reliably get at the metaclass > itself. Or the keyword is stripped and the metaclass is passed as the first argument, i.e. __prepare__(metaclass, name, bases, **kwargs_without_metaclass) I don't like the idea of requiring the use of classmethod(), since as far as I can tell it's badly broken when it comes to making inherited method calls -- unless there's some technique I don't know about it? -- Greg From greg.ewing at canterbury.ac.nz Tue Mar 13 11:10:06 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 13 Mar 2007 23:10:06 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F66BFA.1040609@gmail.com> References: <45F1C734.7080503@acm.org> <45F66BFA.1040609@gmail.com> Message-ID: <45F6787E.3010401@canterbury.ac.nz> Nick Coghlan wrote: > Along similar lines, I'd be marginally happier with: > > class Bob(meta=Planet): pass Is there some way we could remove the word "meta" from the syntax altogether? I don't mind using it in conversation, but it seems a tad too geeky to have as an actual part of the language. How about class Bob(Base1, Base2, class Planet): ... i.e. we're saying what we want the class of the class to be. A benefit would be that the metaclass doesn't end up as being one of the keyword args in the first place, so there's no issue of whether to strip it out or not. -- Greg From benji at benjiyork.com Tue Mar 13 12:16:05 2007 From: benji at benjiyork.com (Benji York) Date: Tue, 13 Mar 2007 07:16:05 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> Message-ID: <45F687F5.5010407@benjiyork.com> Phillip J. Eby wrote: > In short, the very idea of 'is_file()' is wrong, wrong, wrong. At least, > if your goal is to make libraries more robust and reusable. First a note: I personally hope a generic function system (with multiple dispatch) makes it into 3.0 and I don't particularly like the ABC proposal. Having said that, the origin of this discussion was (paraphrased) "instead of doing ABCs, let's just do interfaces". Which seems reasonably to me. It would seem that the same arguments against is_file (or any other check of that ilk) would apply to the ABCs proposed (http://wiki.python.org/moin/AbstractBaseClasses) as well, but I don't recall much argument about the idea that ability to check for a few basic types should be included in the language. If that idea isn't widely accepted, I would have expected more vigorous argument (from many corners) now that it seems ABCs are destined to appear in 3.0. If 3.0 /is/ going to give the "typecheck" ability to basic types, then the argument is: should it be ABCs, interfaces, generic functions, or something else. -- Benji York http://benjiyork.com From ncoghlan at gmail.com Tue Mar 13 13:33:10 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Mar 2007 22:33:10 +1000 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F66C44.60207@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <45F66C44.60207@canterbury.ac.nz> Message-ID: <45F69A06.7010404@gmail.com> Greg Ewing wrote: > If they're passed to both, then the signatures become > > metaclass.__prepare__(name, bases, **kwargs) > metaclass(name, bases, body, **kwargs) > > BTW, I don't think I like the name "__prepare__" much > more than "__metacreate__". It seems just as wooly. > What's being prepared? How? What for? __namespace__ and __instancedict__ are the most literally descriptive names that have occurred to me (after all, the role of the method is to return the locals() dictionary for the creation of a class). That said, __new__ & __init__ could also be said to be somewhat woolly if you didn't already know what they did - preparing a namespace for evaluation of a class body shouldn't be hard to remember once someone has learned what the term is referring to. If Guido decides he likes __prepare__, I can certainly live with it :) (I actually quite like the connotation that what the method returns has been prepared for something, but isn't really finished yet) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Mar 13 13:47:51 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 13 Mar 2007 22:47:51 +1000 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F6787E.3010401@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <45F66BFA.1040609@gmail.com> <45F6787E.3010401@canterbury.ac.nz> Message-ID: <45F69D77.6050102@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: >> Along similar lines, I'd be marginally happier with: >> >> class Bob(meta=Planet): pass > > Is there some way we could remove the word "meta" > from the syntax altogether? I don't mind using > it in conversation, but it seems a tad too geeky > to have as an actual part of the language. If anyone wants to understand metaclass code, they're probably going to need to know what the 'm' in 'mcl' stands for... > How about > > class Bob(Base1, Base2, class Planet): > ... > > i.e. we're saying what we want the class of the > class to be. While this is true, calling the class of a class a metaclass provides a comforting illusion that the infinite 'class of' recursion terminates neatly. (The fact that 'type(type) is type' can be ignored most of the time...) > A benefit would be that the metaclass doesn't > end up as being one of the keyword args in the > first place, so there's no issue of whether to > strip it out or not. There will already be some special cases here I think - for example, if I use the signature: def __prepare__(mcl, name, bases, **kwargs): #... def __new__(mcl, name, bases, attrs, **kwargs): #... def __init__(cls, name, bases, attrs, **kwargs): #... can a user then do: class Bob(meta=Planet, bases=(A, B, C)): pass and still get a usable class that inherits from A, B & C? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Tue Mar 13 14:30:18 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 13 Mar 2007 09:30:18 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 12, 2007, at 7:40 PM, Phillip J. Eby wrote: > Remember, len(), iter(), and suchlike are all generic functions -- > that just happen to be implemented using reserved method names > instead of a method registry. Do we need an ILengthMeasurable > interface to indicate that you can call len() on something? Or > does it suffice to say, "objects passed to this API need to be len > ()-compatible"? No, and yes. :) Okay, for stuff like this I agree that something like simplegenerics is a nice solution. But I'm having a hard time generalizing from a simplegenerics approach to the use cases where I (still) want to use interfaces. For example, what do you do if you want to talk about 8 attributes and a dozen methods as a cohesive unit. Let's say I want to allow people to implement an IUser which has an email address, a realname, a company, a reference to an IProfile, and let's me do thing like email them, remove them, subscribe them to a list, etc. In a generics approach, where do I document all this? > Finally, I posted here several months ago a short "Interface" class > that used generic functions for its base - it would let you define > generic functions in its body and then do Zope/PyProtocols-style > 'IFoo(bar).foo()' operations instead of 'foo(bar)'. It was, I > believe, only another 30-40 lines of code. > > Unless you want all of the fancy introspection features of > zope.interface (which I personally think tempt less-experienced > people to write code that will bite extenders in the butt) or the > exotic meta-features of PyProtocols (which I have found can be > replaced by generic functions with "method combination" support), a > few hundred lines of code should be more than sufficient to > implement both generic functions *and* interfaces for the stdlib's > needs. Generics seem like a good approach for "built-in" Python functionality, like len()-able, or file-like, or map-ish, where there's only one or two methods (or attributes) that contribute to the thing-ness you're trying to model. But for more complex and complicated interfaces, ISTM you need more elaborate mechanisms to document, declare, and verify compliance. The things I want something like zope.interfaces for is a way to put all the documentation about the contract between my stuff and your stuff in one place. I want a way for you to declare "here's an object that conforms to your contract" and I want a way to find all your objects that conform to my contract. Generally, I'm not concerned with whether you're lying or not, although in a debug or development mode, IWBNI you could turn on assertions or whatnot to verify these things (or perhaps have a lint-like mode that trolled through the code to do such verification). So a simplified Interface class might be just the trick for the majority of these use cases. Again, I'm mostly concerned with documentation, rather than strict enforcement or introspection. > (Btw, when I say that introspection produces butt-biting code, I > mean that when *I* first started using zope.interface, I created > all sorts of horrors until I realized that interface introspection > is an attractive but unnecessary evil, directly analagous to LBYL > vs EAFP and type-testing vs. duck-typing issues. See e.g.: http:// > www.mockobjects.com/2006/10/tell-dont-ask-and-mock-objects.html and > http://peak.telecommunity.com/protocol_ref/replintrowadapt.html for > some further thoughts on this. Basically, the problem is that if > you use interface detection in order to decide what to do with > something, you're basically taking away the ability for the > developer who comes *after* you to control what happens with the > objects they give you. Adaptation and generic functions don't have > this problem.) I haven't read those articles yet, but I still think I agree with this. :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfana3EjvBPtnXfVAQJ7QgP8DSSdGQOi4H9F9DIAC+pOMBDr0641HgXp X1v8uZKXN+A6iOdht4SaghX7W5AUL2g8nP5CvoxF7zHMgsQB9pLUor8zMTwWplKe cxzi+hows9qalWkwZfdyXt+rYZS6reknPJPdgSDqJN4lmba/f7zuFyYiEVSKNgbw NH041Y5QGSc= =ejqN -----END PGP SIGNATURE----- From barry at python.org Tue Mar 13 14:41:22 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 13 Mar 2007 09:41:22 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> Message-ID: <3090D998-B052-4E02-AAEC-14777274132C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 12, 2007, at 11:06 PM, Phillip J. Eby wrote: > In short, the very idea of 'is_file()' is wrong, wrong, wrong. At > least, > if your goal is to make libraries more robust and reusable. It leads > inevitably to the brokenness seen in Pydoc -- and the comparable > brokenness > that existed in Zope prior to its replacing most introspection by > adaptation. (To be honest, I'm *assuming* that those broken bits > went away > as soon as adaptation became the recommended option -- I don't know > if it > really did or not, as I haven't done any real Zope work in a few > years.) Two other things I wanted to mention. One, I do think adaptation is an important (necessary?) aspect to any interface solution for exactly the reasons you state. Second, I think the other thing that bugs me about a pure-generics solution is that all the generic functions seem to want to live in a global namespace. That's fine for len() or iter() or keys(), but not so good for all_nonbouncing_regular_delivery_members(). In that sense, for more domain-specific functionality, it just seems that interfaces (w/ adaptation for that extra level of abstraction) is the object- oriented approach to generics. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfaqAnEjvBPtnXfVAQKH7gQAsLqXQ1/v+yU8XXXdWyN8Ear65JrXrurz RS/YwvjJEeASaq5p34jqTf6tzKL7txM7mxZmqTNUvGb5OW/hc1sENQHVn4tgeTD4 +U3KMR4xg31+84QWIzUB3VU88WNOkUqmwWATguiuNWrQZHSG7DEgxBxzLYOS9iTA VehDU4WUNHE= =Rfqn -----END PGP SIGNATURE----- From ncoghlan at gmail.com Tue Mar 13 15:12:26 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Mar 2007 00:12:26 +1000 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <3090D998-B052-4E02-AAEC-14777274132C@python.org> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> Message-ID: <45F6B14A.2080504@gmail.com> Barry Warsaw wrote: > Two other things I wanted to mention. One, I do think adaptation is > an important (necessary?) aspect to any interface solution for > exactly the reasons you state. Second, I think the other thing that > bugs me about a pure-generics solution is that all the generic > functions seem to want to live in a global namespace. That's fine > for len() or iter() or keys(), but not so good for > all_nonbouncing_regular_delivery_members(). In that sense, for more > domain-specific functionality, it just seems that interfaces (w/ > adaptation for that extra level of abstraction) is the object- > oriented approach to generics. Generics can live quite happily inside modules and as methods on objects. The important aspect is the ability to tell the generic function "here's how to frobnicate a doodad", even though the basic frobnicator knows nothing about doodads, and doodads know nothing about frobnicating. That machinery can be designed to work for any callable. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Tue Mar 13 15:16:22 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 13 Mar 2007 10:16:22 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F6B14A.2080504@gmail.com> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> Message-ID: <28CB6854-F165-48EB-BBD1-0AC83AABE0A8@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 13, 2007, at 10:12 AM, Nick Coghlan wrote: > Generics can live quite happily inside modules and as methods on > objects. > > The important aspect is the ability to tell the generic function > "here's how to frobnicate a doodad", even though the basic > frobnicator knows nothing about doodads, and doodads know nothing > about frobnicating. That machinery can be designed to work for any > callable. Are generics then just a way of doing adaptation on a per-function basis rather than a per-object basis? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfayNnEjvBPtnXfVAQKK1QP+MdlrrA/JSXJb4z81f1LHjrymHrtrjip6 +1dmxmj2MQanglqMyoDP8NVplBkX7UH0dWounk2qLl8sxxWJbeugl1U5mCIfH1or zvzkEepZ587bCT0lmc11JpWlwcZG55ehaczo2l5gtLtYBDcAM4g7OJvQbbOWbNfQ PWrlILqUTwQ= =36V0 -----END PGP SIGNATURE----- From pje at telecommunity.com Tue Mar 13 16:06:46 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 10:06:46 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <28CB6854-F165-48EB-BBD1-0AC83AABE0A8@python.org> References: <45F6B14A.2080504@gmail.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> Message-ID: <5.1.1.6.0.20070313095913.048be588@sparrow.telecommunity.com> At 10:16 AM 3/13/2007 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mar 13, 2007, at 10:12 AM, Nick Coghlan wrote: > > > Generics can live quite happily inside modules and as methods on > > objects. > > > > The important aspect is the ability to tell the generic function > > "here's how to frobnicate a doodad", even though the basic > > frobnicator knows nothing about doodads, and doodads know nothing > > about frobnicating. That machinery can be designed to work for any > > callable. > >Are generics then just a way of doing adaptation on a per-function >basis rather than a per-object basis? I prefer to think of adaptation as being a way to provide a namespace for generic functions. :) See http://mail.python.org/pipermail/python-3000/2006-November/004717.html for an implementation of this idea. Basically, you define an interface by creating generic functions in the body of an 'Interface' subclass... and the resulting Interface can then be used to adapt any object to it. The 26 lines of code I gave can be used are not specific to any particular generic function implementation, so they could be used with simplegeneric, RuleDispatch, PEAK-Rules, Guido's prototype, or anything else you might come up with. From pje at telecommunity.com Tue Mar 13 16:41:47 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 10:41:47 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070313104017.048b5db8@sparrow.telecommunity.com> At 09:45 PM 3/12/2007 -0600, Steven Bethard wrote: >On 3/12/07, Phillip J. Eby wrote: > > At 09:32 PM 3/12/2007 -0600, Steven Bethard wrote: > > >On 3/12/07, Steven Bethard wrote: > > > > the signature of the method called will be:: > > > > __prepare__(name, args, kwargs) > > > > not > > > > __prepare__(name, *args, **kwargs) > > > > right? > > > > > >On 3/12/07, Guido van Rossum wrote: > > > > I'm not sure anyone has thought much about it yet. I wonder > > > > if the call shouldn't be made like this: > > > > > > > > __prepare__(name, bases, **kwargs) > > > > > > > > so that if you only expect certain specific keyword args you can > > > > define it like this: > > > > > > > > def __prepare__(name, base, metaclass=X): ... > > > > > >Yeah, seems like **kwargs would be more useful in practice. > > > > Really? Why? I can more easily see circumstances where you'd want to > > restrict the arguments. In the cases I have where I'd switch from a class > > decorator to a class keyword, I have a small set of decorators and would > > have an equally small set of keywords. > >Maybe I'm misunderstanding but isn't this a reason that you'd *want* >the ``**kwargs`` signature? Sorry, I thought you were saying that **kwargs in the *receiver signature* would be more useful... not **kwargs in the *calling* signature, which is of course what Guido proposed. From daniel at stutzbachenterprises.com Tue Mar 13 16:49:42 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 13 Mar 2007 10:49:42 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: Message-ID: On 3/12/07, Patrick Maupin wrote: > Feature: Ability to pass in a dictionary or tuple of dictionaries of > namespaces to search. > > This feature allows, in some cases, for much more dynamic code than > *kwargs. (You could manually smush multiple dictionaries together to > build kwargs, but that can be ugly, tedious, and slow.) > Implementation-wise, this feature and locals() / globals() go hand in > hand. +1 I can think of many cases where it'd be nice to pass locals() along with some_row_from_SQL. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From barry at python.org Tue Mar 13 16:51:47 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 13 Mar 2007 11:51:47 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070313095913.048be588@sparrow.telecommunity.com> References: <45F6B14A.2080504@gmail.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> <5.1.1.6.0.20070313095913.048be588@sparrow.telecommunity.com> Message-ID: <16E25C7D-05A4-45E0-9A57-98C820EA8D70@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 13, 2007, at 11:06 AM, Phillip J. Eby wrote: > I prefer to think of adaptation as being a way to provide a > namespace for generic functions. :) > > See http://mail.python.org/pipermail/python-3000/2006-November/ > 004717.html for an implementation of this idea. Basically, you > define an interface by creating generic functions in the body of an > 'Interface' subclass... and the resulting Interface can then be > used to adapt any object to it. The 26 lines of code I gave can be > used are not specific to any particular generic function > implementation, so they could be used with simplegeneric, > RuleDispatch, PEAK-Rules, Guido's prototype, or anything else you > might come up with. Interesting, and thanks for the links. I'm not 100% I got it all on the first reading, but I think I see where you're going. I'm glad to see my question is just plowing the same ground for the third tiem (at least :). One thing is still missing though, and that's the documentation aspects of something like zope.interfaces. You could say that generics.interfaces could just use docstrings, which is fine for methods, but how would you document attributes in a discoverable way? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfbIk3EjvBPtnXfVAQJroQQAuhxK74A2UKF2suwPPuO98ZG7LmEdPjX6 mS/R7grkFzCrWVlawM8DLYorI6dHh5I/5fKORSdrqqFDezNBysd3gGeyFNbe59Am AOt8MViPkVNtkA9GJvO3gGvy1tPYKdp1pd/0RDUeT01dYTEfARNsw0qsAOaiJ4tq w3ddavRf+0k= =6YUD -----END PGP SIGNATURE----- From agthorr at barsoom.org Tue Mar 13 17:02:22 2007 From: agthorr at barsoom.org (Daniel Stutzbach) Date: Tue, 13 Mar 2007 11:02:22 -0500 Subject: [Python-3000] Reversing through text files with the new IO library In-Reply-To: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> References: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> Message-ID: On 3/12/07, Mark Russell wrote: > Yes, I suspect that BufferedReader needs some kind of readuntil() > method, so that (at least for sane encodings like utf-8) each line is > read via a single readuntil() followed by a decode() call for the > entire line. +1 .readuntil() would also be incredibly useful for writing network protocol code when the protocol isn't line-based. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From daniel at stutzbachenterprises.com Tue Mar 13 17:03:41 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 13 Mar 2007 11:03:41 -0500 Subject: [Python-3000] Reversing through text files with the new IO library In-Reply-To: <89C96DFC-FDD1-47E2-8E5C-1E7BFED27FB9@zen.co.uk> References: <7775D972-3D7A-4803-BB19-810BF6637CC5@zen.co.uk> <89C96DFC-FDD1-47E2-8E5C-1E7BFED27FB9@zen.co.uk> Message-ID: On 3/12/07, Mark Russell wrote: > Hopefully I'll have something in the next week or so. Daniel and > Mike - please let me know if I'm overlapping with anything you're > working on - I don't want to tread on anyone's toes. I'm not doing any work on this at the moment, but feel free to fire me an email asking "Could you do X?". -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From pje at telecommunity.com Tue Mar 13 17:05:10 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 11:05:10 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F687F5.5010407@benjiyork.com> References: <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> At 07:16 AM 3/13/2007 -0400, Benji York wrote: >Phillip J. Eby wrote: > > In short, the very idea of 'is_file()' is wrong, wrong, wrong. At least, > > if your goal is to make libraries more robust and reusable. > >First a note: I personally hope a generic function system (with multiple >dispatch) makes it into 3.0 and I don't particularly like the ABC proposal. > >Having said that, the origin of this discussion was (paraphrased) >"instead of doing ABCs, let's just do interfaces". Which seems >reasonably to me. > >It would seem that the same arguments against is_file (or any other >check of that ilk) would apply to the ABCs proposed >(http://wiki.python.org/moin/AbstractBaseClasses) as well, but I don't >recall much argument about the idea that ability to check for a few >basic types should be included in the language. I'm not *as* opposed to that, because subclassing is more of a commitment and 'isinstance()' tests still "smell" enough that people can (probably) be convinced they're bad. However, more people are under the impression that interface testing is safe and pure and smell-free, so there's more of a barrier to overcome for them to learn why it's wrong, without them first getting bitten -- either by their own code or somebody else's. >If that idea isn't widely accepted, I would have expected more vigorous >argument (from many corners) now that it seems ABCs are destined to >appear in 3.0. Mainly, my argument about the ABC effort is that it's doomed to non-usefulness, in part because of the too-high granularity, but also because it's the *receivers* who need to declare the interfaces they require, and these are inevitably a mismatch with the interfaces that objects (especially generic containers) want to declare. Libraries and applications generally don't care about something being "a sequence" - they care about say, a priority-ordered sequence of things that can be notified about a socket event. Or a postcode-ordered sequence of things that can have shipping labels printed for them. In a nutshell, this is why most "type system" efforts in Python arrive with much fanfare and then gradually fade into the background -- they're attacking the wrong end of the problem. Java, ironically, avoids this problem by forcing you to have new types for every single damn thing. The idiomatic way to have a priority-ordered sequence of socket event listeners is to make a new base class or interface just for that purpose -- so you end up with receiver-oriented interfaces as a side effect! (In addition, Java has just enough overloading support that a lot of the common cases for what would be type tests in Python, are done using overloads in Java.) Whereas in Python, the idiomatic way to do things like that is to just use a list object, unless you absolutely need something like a database or a btree or something fancy, or you want to create a list that automatically maintains the ordering. Even then, you're probably *not* going to subclass 'list', since you'd have to override *all* the manipulation methods. Anyway, I don't think the ABC effort is a bad thing in itself. In fact, there may be *some* benefit to documenting what Python's type concepts are. But I think it'll mostly be a waste of time and effort with respect to practical libraries and applciations -- including the stdlib itself. Basically, all of the past efforts at producing such things (including zope.interface and my own PyProtocols) were an attempt to transplant a Java idea of interfaces into a language where they really don't fit. If we had Java-style overloading and didn't have truly generic container protocols (iter, getitem, slices, mappings), they *might* make sense. But the reality is that Python's built-in types and protocols are far superior to what's in Java, and it simply makes a mess of things to try to shoehorn them into a Java-oriented way of defining interfaces. >If 3.0 /is/ going to give the "typecheck" ability to basic types, We have this now - it's called isinstance(). :) >then >the argument is: should it be ABCs, interfaces, generic functions, or >something else. Of these, ABCs are the least problematic because they don't introduce a new kind of type checking that's lauded as safe when in fact it's pure evil. From pje at telecommunity.com Tue Mar 13 17:06:55 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 11:06:55 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F69A06.7010404@gmail.com> References: <45F66C44.60207@canterbury.ac.nz> <45F1C734.7080503@acm.org> <45F66C44.60207@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20070313110611.02a1f698@sparrow.telecommunity.com> At 10:33 PM 3/13/2007 +1000, Nick Coghlan wrote: >Greg Ewing wrote: > > If they're passed to both, then the signatures become > > > > metaclass.__prepare__(name, bases, **kwargs) > > metaclass(name, bases, body, **kwargs) > > > > BTW, I don't think I like the name "__prepare__" much > > more than "__metacreate__". It seems just as wooly. > > What's being prepared? How? What for? > >__namespace__ and __instancedict__ are the most literally descriptive >names that have occurred to me (after all, the role of the method is to >return the locals() dictionary for the creation of a class). > >That said, __new__ & __init__ could also be said to be somewhat woolly >if you didn't already know what they did - preparing a namespace for >evaluation of a class body shouldn't be hard to remember once someone >has learned what the term is referring to. > >If Guido decides he likes __prepare__, I can certainly live with it :) >(I actually quite like the connotation that what the method returns has >been prepared for something, but isn't really finished yet) How about __class_locals__()? This would at least say exactly what the return value is used for. From pje at telecommunity.com Tue Mar 13 17:10:15 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 11:10:15 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070313110740.02b23b68@sparrow.telecommunity.com> At 09:30 AM 3/13/2007 -0400, Barry Warsaw wrote: >For example, what do you do if you want to talk about 8 attributes >and a dozen methods as a cohesive unit. Let's say I want to allow >people to implement an IUser which has an email address, a realname, >a company, a reference to an IProfile, and let's me do thing like >email them, remove them, subscribe them to a list, etc. In a >generics approach, where do I document all this? http://mail.python.org/pipermail/python-3000/2006-November/004717.html From pje at telecommunity.com Tue Mar 13 17:12:32 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 11:12:32 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <16E25C7D-05A4-45E0-9A57-98C820EA8D70@python.org> References: <5.1.1.6.0.20070313095913.048be588@sparrow.telecommunity.com> <45F6B14A.2080504@gmail.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> <5.1.1.6.0.20070313095913.048be588@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070313111041.02a1ba18@sparrow.telecommunity.com> At 11:51 AM 3/13/2007 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mar 13, 2007, at 11:06 AM, Phillip J. Eby wrote: > > > I prefer to think of adaptation as being a way to provide a > > namespace for generic functions. :) > > > > See http://mail.python.org/pipermail/python-3000/2006-November/ > > 004717.html for an implementation of this idea. Basically, you > > define an interface by creating generic functions in the body of an > > 'Interface' subclass... and the resulting Interface can then be > > used to adapt any object to it. The 26 lines of code I gave can be > > used are not specific to any particular generic function > > implementation, so they could be used with simplegeneric, > > RuleDispatch, PEAK-Rules, Guido's prototype, or anything else you > > might come up with. > >Interesting, and thanks for the links. I'm not 100% I got it all on >the first reading, but I think I see where you're going. I'm glad to >see my question is just plowing the same ground for the third tiem >(at least :). > >One thing is still missing though, and that's the documentation >aspects of something like zope.interfaces. You could say that >generics.interfaces could just use docstrings, which is fine for >methods, but how would you document attributes in a discoverable way? property() is one way, of course, but the implementation sketched in that email supports *any* sort of descriptor you want. The AdaptingDescriptor should probably support __set__ and __delete__ as well, but you see the general idea. From python3now at gmail.com Tue Mar 13 17:47:11 2007 From: python3now at gmail.com (James Thiele) Date: Tue, 13 Mar 2007 09:47:11 -0700 Subject: [Python-3000] Fwd: Cross-compatibility with 2.6 and 3.0 In-Reply-To: <9e804ac0703091600s1509e567v83ac27f7817a8399@mail.gmail.com> References: <45F099D2.6010804@acm.org> <9e804ac0703091600s1509e567v83ac27f7817a8399@mail.gmail.com> Message-ID: <8f01efd00703130947j7c049477vfda99e9c3e8ef8da@mail.gmail.com> ---------- Forwarded message ---------- From: Thomas Wouters Date: Mar 9, 2007 5:00 PM Subject: Re: [Python-3000] Cross-compatibility with 2.6 and 3.0 To: Talin Cc: Python-3000 at python.org On 3/9/07, Talin wrote: > "It is unlikely that you will be able to write both code that is both > valid Python 2.6 source code and valid Python 3.0 source code". Please see http://mail.python.org/pipermail/python-dev/2007-February/071372.html (and subsequent messages, if you want.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! _______________________________________________ Python-3000 mailing list Python-3000 at python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/python3now%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070313/d2812fa0/attachment.htm From guido at python.org Tue Mar 13 17:56:55 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 13 Mar 2007 09:56:55 -0700 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: <45F67137.1070707@canterbury.ac.nz> References: <45F1B723.8020402@acm.org> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> <45F5DD07.3070408@canterbury.ac.nz> <45F67137.1070707@canterbury.ac.nz> Message-ID: On 3/13/07, Greg Ewing wrote: > Guido van Rossum wrote: > > > The internal mechanism for bubbling an exception up the stack until an > > except clause catches it could continue to use the (class, instance, > > traceback) triple, > > Hmmm, so most of that complexity will still be there at > the C level. I was hoping there might be some simplification > there as well. We can still gain some simplification there, as we shouldn't have to keep shadow copies of these triples in each frame and in each thread state. > However... > > > and if raise is passed a class instead of an > > instance, the instance would be NULL; when forced to instantiate the > > exception, the traceback collected up to that point is attached to it. > > Would this apply to Python code as well? I.e. if you use > a raise statement with a class, it doesn't get instantiated > immediately? And if you catch it with an except clause which > doesn't capture the exception, it's never instantiated? > That would be a bonus. I *think* that's how it works now, and I don't intend to break this. > > If an instance was passed to raise, the __traceback__ pointer in the > > instance is updated each time we leave a stack frame. > > So it seems we're back to declaring pre-instantiated > exceptions to be bad style, which you said you didn't > like earlier -- have you changed your mind? Yes. After learning that there isn't much of a performance gain and there aren't many users, and after looking into other solutions, I really think that having the tb attached to the exc is the best approach. > What about thread safety? Do we just document that using > pre-instantiated exceptions is not thread-safe? It's more than non-thread-safe. It's unsafe, period. And yes, we document this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Tue Mar 13 18:29:08 2007 From: barry at python.org (Barry Warsaw) Date: Tue, 13 Mar 2007 13:29:08 -0400 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> Message-ID: <0E5BE476-9229-46B0-A215-668A539B9F08@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 13, 2007, at 12:05 PM, Phillip J. Eby wrote: > However, more people are under the impression that interface > testing is > safe and pure and smell-free, so there's more of a barrier to > overcome for > them to learn why it's wrong, without them first getting bitten -- > either > by their own code or somebody else's. Phillip, could you sum the argument up as this: being against interface testing as just a fancy isinstance() test, but that adaptation and/or generics is a better way to go because the former is a LBYL pattern while the latter is a Just Do It pattern. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfbfanEjvBPtnXfVAQImlgP/bZguuw5GhZsV/nXOaaivrISAaaQN7PCX MYclQdI1R38hJCZSrHpDzoyTAbY+uMwuoCLSjW33f6oVNStspUb3weIJnx3fcYb4 NXTAZhs7VQ5oDcD0Fpy1zTkQ9tn9sCIomWNcWQCQG0DRJ+biwqNqG39FqeS9lsEV pp2sTunxtIU= =MgHy -----END PGP SIGNATURE----- From guido at python.org Tue Mar 13 18:38:29 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 13 Mar 2007 10:38:29 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: Another bulk reply... On 3/12/07, Steven Bethard wrote: > Maybe I'm misunderstanding but isn't this a reason that you'd *want* > the ``**kwargs`` signature? With the plain ``kwargs`` signature from > the PEP you'd have to do something like:: > > def __prepare__(name, args, kwargs): > required = kwargs.pop('required', None) > if required is not None: > ... There seems some confusion here. Let me try to clarify this by specifying exactly what I meant and I betcha you'll all agree that that's what you meant too. :-) The __prepare__ function will be called with two positional arguments, corresponding to the name and the tuple of base classes, respectively; and with as many keyword arguments as were present in the class header. Use of *args in the class header adds to the bases; use of **kwargs adds to the keyword args. PS I'm not 100% sure of the name __prepare__ (though I will stick for it until we find a better one), but I *am* pretty sure that this is the most useful signature. On 3/13/07, Nick Coghlan wrote: > Along similar lines, I'd be marginally happier with: > > class Bob(meta=Planet): pass > > over what is currently proposed in the PEP. Repeating the 'class' part > within a single line feels redundant. Disagree. 'meta' is an extremely generic term, and needs qualification. Also, we've been calling these things metaclasses since before they were properly supported. Let's stick with metaclass. [Guido] > > I'd like the syntax between () to be identical to that for a function > > call, i.e. including *args and **kwds. That way the cognitive load for > > the user is the smallest. Sorting out the semantics of the keywords is > > purely a runtime activity anywaty. [Nick] > I like this idea. > > I think the signature of the method that creates the namespace > dictionary requires some further thought though: > > - is it intended to typically be a static method or a class method of > the metaclass? (as it will be called before __new__, an instance method > of the metaclass wouldn't make sense) We have to be very careful, because the 'metaclass' doesn't even need to be a class. I propose to use exactly these syntax: prepare = getattr(metaclass, '__prepare__', None) if prepare is not None: prepare(name, bases, **kwargs) > - is the metaclass passed in to the namespace creation function as a > normal keyword argument, or is it automatically removed by the > interpreter? (note that implementing it as a class method would still > allow access to the actual metaclass) For uniformity and generality I propose to keep it. So the absolute minimal signature for __prepare__ is actually: def __prepare__(name, bases, *, metaclass=None): ... > - if the metaclass is derived rather than passed in explicitly, is the > 'meta' argument still passed in to the namespace creation function? I think it should not be; this way the prepare function can distinguish between an explicit and an implied metaclass. Hence the default None above. > - what's the actual signature of the new function? See above. > The answers to these will be somewhat intertwined - if the 'meta' > keyword argument is always going to be passed in to the function, then > it makes more sense to use a static method in the typical case. If the > meta keyword is stripped (or only provided when passed in explicitly), > then a class method is needed in order to reliably get at the metaclass > itself. I think we shouldn't care about whether it's a class method or a static method. It should be *callable* like above, and there are many ways to accept this call signature. Again, remember that the 'metaclass' could be a refugar function and __prepare__ another function stored on the former as a function attribute. On 3/13/07, Greg Ewing wrote: > Can someone clarify something here: Are the keywords going > to be passed to the prepare function only, or to both the > the prepare function and the metaclass? I think they should go to both -- otherwise you might end up having to create a dummy __prepare__ just to pass the keyword args to the metaclass itself. > If they're only passed to the prepare function, it means > that any metaclass that's interested in them will have to > implement a prepare function that creates some object to > hold onto them, even if it has no other need for a custom > namespace. > > If they're passed to both, then the signatures become > > metaclass.__prepare__(name, bases, **kwargs) > metaclass(name, bases, body, **kwargs) Right. > BTW, I don't think I like the name "__prepare__" much > more than "__metacreate__". It seems just as wooly. > What's being prepared? How? What for? We're preparing a dict for use by the class body. But I'm open for suggestions of alternate names (though this is quickly reaching the bikeshed color level of interest). We could use __begin__, or something else including the word "begin", as the call signals the beginning of the class construction. On 3/13/07, Greg Ewing wrote: [Greg] > > > class Foo: > > > class __metaclass__: [Guido] > > Personally, I find code that does this completely unreadable [Greg] > The reason I'd be a little disappointed to lose this is that > it provides a concise way of defining genuine class methods > (as opposed to the odd beasts created by classmethod()). > Sometimes you need them, e.g. for __xxx__ methods, or if you > want inheritance to work properly. All I obect to is the nesting of the metaclass *inside* the class. It makes the metaclass essentially non-reusable (except by inheriting from the class in which it appears) and it ends up having a generic name which may make it hard to debug code in which that name appears. Also there are general issues with nested classes (e.g. they're hard to pickle right). > Having said that, I can't remember whether I've ever actually > used this, and I probably wouldn't miss it all that much. Spoken like a man. :-) > > I find this rather cool looking: > > > > class C(implements=(I1, I2)): ... Me too. :-) > I'm a bit worried that class headers are going to become > rather cluttered if we start putting all sorts of stuff > in there. > > E.g. are you intending to put __slots__ there? It makes > sense, but it could lead to some rather long and > unwieldy class headers. It makes sense to put slots there, doesn't it? I'm not too worried about the long class headers; a formatting convention can do wonders here. On 3/13/07, Greg Ewing wrote: > Is there some way we could remove the word "meta" > from the syntax altogether? I don't mind using > it in conversation, but it seems a tad too geeky > to have as an actual part of the language. Being geeky is what it's all about. :-) > How about > > class Bob(Base1, Base2, class Planet): > ... > > i.e. we're saying what we want the class of the > class to be. Too confusing for the reader, and breaks the IMO rather nice property that this is now syntactically a call, with exactly all the bells and whistles that entails, and no others. > A benefit would be that the metaclass doesn't > end up as being one of the keyword args in the > first place, so there's no issue of whether to > strip it out or not. I don't see that as a problem (see above). On 3/13/07, Phillip J. Eby wrote: > How about __class_locals__()? This would at least say exactly what the > return value is used for. But it doesn't convey the idea that this *creates* such a dict; it could be misconstrued as being an accessor method for the class locals (even though that makes no sense). I'm still in favor of __prepare__ or __begin__. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Tue Mar 13 18:42:33 2007 From: thomas at python.org (Thomas Wouters) Date: Tue, 13 Mar 2007 18:42:33 +0100 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: References: <45F1B723.8020402@acm.org> <43aa6ff70703121220i52b420a7p2d3fa7e81a5f1cc8@mail.gmail.com> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> <45F5DD07.3070408@canterbury.ac.nz> <45F67137.1070707@canterbury.ac.nz> Message-ID: <9e804ac0703131042g4bd697e7j26f70ec3fad0ffe8@mail.gmail.com> On 3/13/07, Guido van Rossum wrote: > > > > > Would this apply to Python code as well? I.e. if you use > > a raise statement with a class, it doesn't get instantiated > > immediately? And if you catch it with an except clause which > > doesn't capture the exception, it's never instantiated? > > That would be a bonus. > > I *think* that's how it works now, and I don't intend to break this. Eh, I don't see how it can, when you do 'raise ExceptionClass(args)' (like you want everyone to use :). It certainly doesn't, now; the arguments to 'raise' are just expressions, there is no specialcasing of calling anything. I don't believe it does so with the two-argument raise either -- at least, when experimenting, the body of an exception class's __init__ method is executed before the body of a 'finally' suite. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070313/e150e424/attachment.htm From thomas at python.org Tue Mar 13 18:47:51 2007 From: thomas at python.org (Thomas Wouters) Date: Tue, 13 Mar 2007 18:47:51 +0100 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: <9e804ac0703131047neca1bdbk982e216c344183c3@mail.gmail.com> On 3/13/07, Guido van Rossum wrote: > > [Quoting Greg E. quoting Guido] > > > I find this rather cool looking: > > > > > > class C(implements=(I1, I2)): ... > > Me too. :-) Boy, you really like it, don't you. Me, too, though. Will that be the syntax for ABCs then, rather than that silly normal-inheritance business? :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070313/85a7a9a7/attachment.html From guido at python.org Tue Mar 13 18:51:27 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 13 Mar 2007 10:51:27 -0700 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: <9e804ac0703131042g4bd697e7j26f70ec3fad0ffe8@mail.gmail.com> References: <45F1B723.8020402@acm.org> <43aa6ff70703121424q58ed593dle98330beee7e0efd@mail.gmail.com> <45F5DD07.3070408@canterbury.ac.nz> <45F67137.1070707@canterbury.ac.nz> <9e804ac0703131042g4bd697e7j26f70ec3fad0ffe8@mail.gmail.com> Message-ID: He was talking about raise ExceptionClass which I don't want to forbid; it's the raise ExceptionClass, args that will be killed in 3.0. You're correct though that in 2.x, "raise E" instantiates E immediately. We could change this though, as it works from C... --Guido On 3/13/07, Thomas Wouters wrote: > > > On 3/13/07, Guido van Rossum wrote: > > > > > > Would this apply to Python code as well? I.e. if you use > > > a raise statement with a class, it doesn't get instantiated > > > immediately? And if you catch it with an except clause which > > > doesn't capture the exception, it's never instantiated? > > > That would be a bonus. > > > > I *think* that's how it works now, and I don't intend to break this. > > Eh, I don't see how it can, when you do 'raise ExceptionClass(args)' (like > you want everyone to use :). It certainly doesn't, now; the arguments to > 'raise' are just expressions, there is no specialcasing of calling anything. > I don't believe it does so with the two-argument raise either -- at least, > when experimenting, the body of an exception class's __init__ method is > executed before the body of a 'finally' suite. > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me > spread! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bjourne at gmail.com Tue Mar 13 18:57:54 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Tue, 13 Mar 2007 18:57:54 +0100 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: <740c3aec0703131057s6a6452fn579e35c260fb78d1@mail.gmail.com> On 3/13/07, Guido van Rossum wrote: > > > I find this rather cool looking: > > > > > > class C(implements=(I1, I2)): ... > > Me too. :-) But... What does it do? PEP says: In the new model, the syntax for specifying a metaclass is via a keyword argument in the list of base classes: class Foo(base1, base2, metaclass=mymeta): ... Additional keywords will also be allowed here, and will be passed to the metaclass, as in the following example: class Foo(base1, base2, metaclass=mymeta, private=True): ... Who is the receiver of the implements keyword argument in your example? Should it not be class class C(metaclass = SomethingHere, implements = (I1, I2))? -- mvh Bj?rn From pje at telecommunity.com Tue Mar 13 19:33:20 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 13:33:20 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <0E5BE476-9229-46B0-A215-668A539B9F08@python.org> References: <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070313131610.02798dc0@sparrow.telecommunity.com> At 01:29 PM 3/13/2007 -0400, Barry Warsaw wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Mar 13, 2007, at 12:05 PM, Phillip J. Eby wrote: > > > However, more people are under the impression that interface > > testing is > > safe and pure and smell-free, so there's more of a barrier to > > overcome for > > them to learn why it's wrong, without them first getting bitten -- > > either > > by their own code or somebody else's. > >Phillip, could you sum the argument up as this: Only if I were willing to making it sound like an argument over style instead of one of substance. :) >being against >interface testing as just a fancy isinstance() test, but that >adaptation and/or generics is a better way to go because the former >is a LBYL pattern while the latter is a Just Do It pattern. It's *similar* to this, certainly. However, that's like saying that murder is bad because it's an instance of the Sin pattern. If the Sin pattern also includes things like overindulgence in chocolate, then it seriously understates the part where somebody dies! In other words, there are plenty of instances where LBYL might be "sinful", but it doesn't do any harm besides making your code "fat". Whereas, type and interface testing of the variety practiced by pydoc and older Zopes, well, that *hurts other people too*. So I want to keep it the hell away from the stdlib, because we don't need more pydoc-style hassles. Sure, the parts of pydoc that work, work very nicely -- for built-in types. But it's a terrible headache for people who want to use it on anything *else* -- which is precisely the thing that interfaces are theoretically *supposed* to avoid -- but don't in practice. Superficially, pydoc is clean code because it uses "interfaces" (the inspect module's various isfoo() functions), but in reality it's just a perfect example of how that approach goes horribly wrong. (It shouldn't be necessary for me to make all my callable objects have 'func_code' attributes in order to make pydoc present their signatures, for example!) So, if I wanted to try to sum up my argument as briefly as possible, I would say: Using interfaces for type testing provides a false sense of confidence that one's code is "clean", extensible, and reusable, while in fact creating endless headaches for anyone who actually tries to reuse it. In contrast, using isinstance() may sometimes be a necessary evil, but few people are under the false impression that the resulting code is actually clean or reusable. So it does comparatively little harm, because people usually practice it in moderation. Had pydoc only had isinstance() to play with, it might have been less tempting to spread isfoo() junk throughout its entire code base. And if it had generic functions to play with (or adaptation to an IDocumentable interface), it would've been able to avoid the whole issue in the first place. From g.brandl at gmx.net Tue Mar 13 19:56:41 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 13 Mar 2007 19:56:41 +0100 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <740c3aec0703131057s6a6452fn579e35c260fb78d1@mail.gmail.com> References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> <740c3aec0703131057s6a6452fn579e35c260fb78d1@mail.gmail.com> Message-ID: BJ?rn Lindqvist schrieb: > On 3/13/07, Guido van Rossum wrote: >> > > I find this rather cool looking: >> > > >> > > class C(implements=(I1, I2)): ... >> >> Me too. :-) > > But... What does it do? PEP says: > > In the new model, the syntax for specifying a metaclass is via a > keyword argument in the list of base classes: > > class Foo(base1, base2, metaclass=mymeta): > ... > > Additional keywords will also be allowed here, and will be passed to > the metaclass, as in the following example: > > class Foo(base1, base2, metaclass=mymeta, private=True): > ... > > Who is the receiver of the implements keyword argument in your > example? Should it not be class class C(metaclass = SomethingHere, > implements = (I1, I2))? If Foo inherits from another class whose metaclass already accepts "implements", it should work. I can't see how "class C(implements=(I1, I2))" would work though, without a hook to set a default metaclass. Georg From jcarlson at uci.edu Tue Mar 13 22:26:48 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 13 Mar 2007 14:26:48 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: Message-ID: <20070313140656.FB71.JCARLSON@uci.edu> "Guido van Rossum" wrote: > On 3/13/07, Greg Ewing wrote: > > I'm a bit worried that class headers are going to become > > rather cluttered if we start putting all sorts of stuff > > in there. > > > > E.g. are you intending to put __slots__ there? It makes > > sense, but it could lead to some rather long and > > unwieldy class headers. > > It makes sense to put slots there, doesn't it? I'm not too worried > about the long class headers; a formatting convention can do wonders > here. I agree that formatting can do wonders, but it smells a bit like the discussion that was had way back when we were discussing function decorators. If I remember correctly, one reason why decorators didn't end up in the function signature was *because* it makes the function signature unruly in the case of many decorators. While class headers are significantly more rare than function or method definitions, I can't help but think that some sort of 'pre-decorator' syntax wouldn't look a bit better. Say something like @@(*args, **kwargs). For a shorter example, it doesn't look much (if any) better... class F(implements=(I1, I2)): ... vs. @@(implements=(I1, I2)) class F: ... But in slightly longer examples, I think that the looks are significantly improved... class Foo(base1, base2, *otherbases, metaclass=mymeta, private=True, **kwargs): ... vs. @@(metaclass=mymeta, private=True, **kwargs) class Foo(base1, base2, *otherbases): ... The 'pre-decorator' (which only makes sense for classes) could serve as the location where all **kwargs are passed to the __prepare__ method on the provided metaclass, with the class header allowing *args, which are provided to both __prepare__ and whatever method is called on the metaclass. - Josiah From pje at telecommunity.com Tue Mar 13 22:37:39 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 16:37:39 -0500 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070313140656.FB71.JCARLSON@uci.edu> References: Message-ID: <5.1.1.6.0.20070313163307.04695a20@sparrow.telecommunity.com> At 02:26 PM 3/13/2007 -0700, Josiah Carlson wrote: >But in slightly longer examples, I think that the looks are >significantly improved... > > class Foo(base1, base2, *otherbases, metaclass=mymeta, > private=True, **kwargs): > ... > >vs. > > @@(metaclass=mymeta, private=True, **kwargs) > class Foo(base1, base2, *otherbases): > ... I was confused for a minute, because I thought you were contradicting yourself, because it was so obvious that the *first* one looks better. It took me a while to realize you're saying you like the bottom one better, whereas to me it's like "ugh". If it was a choice between using that syntax and not having the feature, I'd frankly choose not having the feature, as I'm making do right now with magic functions in the class body to do this sort of thing. They look a heckuva lot better than the @@ thing. Fortunately, I think Guido has all but Pronounced on the first syntax, and IMO with good reason. The symmetry with normal function syntax is just too good to pass up. From greg.ewing at canterbury.ac.nz Tue Mar 13 23:10:58 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 11:10:58 +1300 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F6B14A.2080504@gmail.com> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> Message-ID: <45F72172.4040509@canterbury.ac.nz> Nick Coghlan wrote: > Generics can live quite happily inside modules and as methods on objects. I can see a problem with this when applied to the pydoc problem discussed earlier. Suppose pydoc were to define some generic functions for documenting an object. To avoid polluting the global namespace, the objects representing them are placed into the pydoc module. Now I have a type that I want to make documentable, so I provide implementations of those functions. But to register those implementations, I have to import pydoc. So any time anyone uses my type, for whatever reason, they end up dragging in pydoc even if they have no intention of using it. If instead of pydoc it's some third party module, the situation is even worse, since the module may not even be installed on the user's system. So this scheme would seem to create undesirable dependencies. -- Greg From ncoghlan at gmail.com Tue Mar 13 23:19:48 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Mar 2007 08:19:48 +1000 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070313140656.FB71.JCARLSON@uci.edu> References: <20070313140656.FB71.JCARLSON@uci.edu> Message-ID: <45F72384.5040708@gmail.com> Josiah Carlson wrote: > But in slightly longer examples, I think that the looks are > significantly improved... > > class Foo(base1, base2, *otherbases, metaclass=mymeta, > private=True, **kwargs): > ... > > vs. > > @@(metaclass=mymeta, private=True, **kwargs) > class Foo(base1, base2, *otherbases): > ... > vs. class Foo(base1, base2, *otherbases, metaclass=mymeta, private=True, **kwargs): When it comes to long class headers, I think vertical whitespace will prove to be useful for separating out the keywords and the list of bases. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Tue Mar 13 23:29:03 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 13 Mar 2007 15:29:03 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F72384.5040708@gmail.com> References: <20070313140656.FB71.JCARLSON@uci.edu> <45F72384.5040708@gmail.com> Message-ID: I think we've definitely entered the bikeshed color discussion now. I'm holding out for Talin's revised version of the PEP. On 3/13/07, Nick Coghlan wrote: > Josiah Carlson wrote: > > But in slightly longer examples, I think that the looks are > > significantly improved... > > > > class Foo(base1, base2, *otherbases, metaclass=mymeta, > > private=True, **kwargs): > > ... > > > > vs. > > > > @@(metaclass=mymeta, private=True, **kwargs) > > class Foo(base1, base2, *otherbases): > > ... > > > > vs. > > class Foo(base1, base2, *otherbases, > metaclass=mymeta, private=True, **kwargs): > > When it comes to long class headers, I think vertical whitespace will > prove to be useful for separating out the keywords and the list of bases. > > Regards, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > http://www.boredomandlaziness.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Tue Mar 13 23:37:13 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Mar 2007 08:37:13 +1000 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F72172.4040509@canterbury.ac.nz> References: <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> <45F72172.4040509@canterbury.ac.nz> Message-ID: <45F72799.1080005@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > >> Generics can live quite happily inside modules and as methods on objects. > > If instead of pydoc it's some third party module, > the situation is even worse, since the module may > not even be installed on the user's system. > > So this scheme would seem to create undesirable > dependencies. This is why the idea of lazy importing comes up pretty fast whenever generics are discussed. However, this criticism regarding importing otherwise unneeded modules also applies to interfaces and ABC's, only more so. For example, if I wanted to declare that a class implements (or inherits from) pydoc.IDocumentable, then I would have to import pydoc first, and as I need to do it at class definition time (assuming we're using the normal inheritance mechanism to declare our interfaces), lazy importing can't help. 3rd-party registration of 'A implements I' or 'here's how to convert A to something that implements I' is useful not only for adapting two libraries to play well together, but also in allowing the libraries themselves to delay these definitions until they're known to be at least potentially neeeded (e.g. don't register a class as being documentable until the application actually imports and starts using pydoc). Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From thomas at python.org Tue Mar 13 23:57:53 2007 From: thomas at python.org (Thomas Wouters) Date: Tue, 13 Mar 2007 23:57:53 +0100 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <740c3aec0703131057s6a6452fn579e35c260fb78d1@mail.gmail.com> References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> <740c3aec0703131057s6a6452fn579e35c260fb78d1@mail.gmail.com> Message-ID: <9e804ac0703131557h7ec5c578r194ee28216e8a5d9@mail.gmail.com> On 3/13/07, BJ?rn Lindqvist wrote: > > On 3/13/07, Guido van Rossum wrote: > > > > I find this rather cool looking: > > > > > > > > class C(implements=(I1, I2)): ... > > > > Me too. :-) Who is the receiver of the implements keyword argument in your > example? Should it not be class class C(metaclass = SomethingHere, > implements = (I1, I2))? The receiver in Guido's example is of course the default metaclass, 'type'. (You can believe either that, or that it's just an example :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070313/0c149257/attachment.html From pje at telecommunity.com Wed Mar 14 00:12:54 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 13 Mar 2007 18:12:54 -0500 Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <45F72799.1080005@gmail.com> References: <45F72172.4040509@canterbury.ac.nz> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <3090D998-B052-4E02-AAEC-14777274132C@python.org> <45F6B14A.2080504@gmail.com> <45F72172.4040509@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20070313180707.03dbb3c8@sparrow.telecommunity.com> At 08:37 AM 3/14/2007 +1000, Nick Coghlan wrote: >Greg Ewing wrote: > > Nick Coghlan wrote: > > > >> Generics can live quite happily inside modules and as methods on objects. > > > > If instead of pydoc it's some third party module, > > the situation is even worse, since the module may > > not even be installed on the user's system. > > > > So this scheme would seem to create undesirable > > dependencies. In cases where you don't already have such a dependency, yeah. >This is why the idea of lazy importing comes up pretty fast whenever >generics are discussed. I call 'em "weak" imports, myself: http://peak.telecommunity.com/DevCenter/Importing#weak-imports As it happens, you need the same thing for adaptation, which is why I put this into PEAK way back in the heydey of Python 2.2. >However, this criticism regarding importing otherwise unneeded modules >also applies to interfaces and ABC's, only more so. For example, if I >wanted to declare that a class implements (or inherits from) >pydoc.IDocumentable, then I would have to import pydoc first, and as I >need to do it at class definition time (assuming we're using the normal >inheritance mechanism to declare our interfaces), lazy importing can't help. > >3rd-party registration of 'A implements I' or 'here's how to convert A >to something that implements I' is useful not only for adapting two >libraries to play well together, but also in allowing the libraries >themselves to delay these definitions until they're known to be at least >potentially neeeded (e.g. don't register a class as being documentable >until the application actually imports and starts using pydoc). An excellent point, indeed. From jcarlson at uci.edu Wed Mar 14 00:21:34 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 13 Mar 2007 16:21:34 -0700 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: Message-ID: <20070313160221.FB77.JCARLSON@uci.edu> "Patrick Maupin" wrote: [snip] > Feature: Alternate syntaxes for escape to markup. [snip] > The second method is the well-understood ${} syntax. The $ is easy to > find in a sea of { characters, and the only special handling required > is that every $ must be doubled. I think this is reasonable. > The third method is something I came up with a couple of years ago > when I was generating a lot of C code from templates. It would make > any non-Python programmer blanch, because it relies on significant > whitespace, but it made for very readable technical templates. WIth > this method "{foo}" escapes to markup, but when there is whitespace > after the leading "{", e.g. "{ foo}", the brace is not an escape to > markup. If the whitespace is a space, it is removed from the output, > but if it is '\r', '\n', or '\t', then it is left in the output. The > result is that, for example, most braces in most C texts do not need > to have spaces inserted after them, because they immediately precede a > newline. -1 Sounds far too implicit to me. I would much prefer a completely different syntax. Having just had a patch that implements Windows shell variable expansion as $var, ${var} and %var% (the last being new), I can't help but think that %var% would be a better alternate explicit syntax than the 'space follows {' implicit syntax you are offering. [snip] > Feature: Automatic search of locals() and globals() for name lookups > if no parameters are given. -1 for the obvious 'Explicit is better than implicit'. > Feature: Ability to pass in a dictionary or tuple of dictionaries of > namespaces to search. +1 I can see where it would be useful for code using something like "...".format(obj.__dict__, defaults) [snip] > Feature: Addition of functions and "constants" to string module. > > The PEP proposes doing everything as string methods, with a "cformat" > method allowing some access to the underlying machinery. I propose > only having a 'format' method of the string (unicode) type, and a > corresponding 'format' and extended 'flag_format' function in the > string module, along with definitions for the flags for access to > non-default underlying format machinery. Sounds good to me. > Feature: Ability for "field hook" user code function to only be called > on some fields. > > The PEP takes an all-or-nothing approach to the field hook -- it is > either called on every field or no fields. Furthermore, it is only > available for calling if the extended function ('somestring'.cformat() > in the spec, string.flag_format() in this proposal) is called. The > proposed change keeps this functionality, but also adds a field type > specifier 'h' which causes the field hook to be called as needed on a > per-field basis. This latter method can even be used from the default > 'somestring'.format() method. > > > Changed feature: By default, not using all arguments is not an exception +1 I can see how it would be useful to have '...'.format(obj.__dict__) to format portions of some object. Requiring the explicit passing of names, and/or the construction of a new dictionary would be foolish. Practicality beats purity. > Feature: Ability to insert non-printing comments in format strings > > This feature is implemented in a very intuitive way, e.g. " text {# > your comment here} more text" (example shown with the default > transition to markup syntax). One of the nice benefits of this > feature is the ability to break up long source lines (if you have lots > of long variable names and attribute lookups). -1 The user can use the parser/compiler to get this behavior for free. (" text " #your comment here "more text").format(...) > Feature: Exception raised if attribute with leading underscore accessed. > > The syntax supported by the PEP is deliberately limited in an attempt > to increase security. This is an additional security measure, which > is on by default, but can be optionally disabled if > string.flag_format() is used instead of 'somestring'.format(). +1 > Feature: Support for "center" alignment. > > The field specifier uses "<" and ">" for left and right alignment. > This adds "^" for center alignment. +.5 There is use for this, but I'm not sure that '^' is as clear as '<' or '>' in the left/right alignment case. Its meaning certainly doesn't jump out to me as 'center' (then again, < and > don't jump out as left and right alignment either...). > Feature: support of earlier versions of Python +1 > Feature: no global state > > The original PEP specified a global error-handling mode, which > controlled a few capabilities. The current implementation has no > global state -- any non-default options are accessed by using the > string.flag_format() function. Great. Global state is a great way of having things break when threads get involved. - Josiah From jcarlson at uci.edu Wed Mar 14 00:24:57 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 13 Mar 2007 16:24:57 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <9e804ac0703131557h7ec5c578r194ee28216e8a5d9@mail.gmail.com> References: <740c3aec0703131057s6a6452fn579e35c260fb78d1@mail.gmail.com> <9e804ac0703131557h7ec5c578r194ee28216e8a5d9@mail.gmail.com> Message-ID: <20070313162151.FB7A.JCARLSON@uci.edu> "Thomas Wouters" wrote: > On 3/13/07, BJorn Lindqvist wrote: > > > > On 3/13/07, Guido van Rossum wrote: > > > > > I find this rather cool looking: > > > > > > > > > > class C(implements=(I1, I2)): ... > > > > > > Me too. :-) > > > Who is the receiver of the implements keyword argument in your > > example? Should it not be class class C(metaclass = SomethingHere, > > implements = (I1, I2))? > > > The receiver in Guido's example is of course the default metaclass, 'type'. > (You can believe either that, or that it's just an example :-) If I read the proposal correctly, it's actually metaclass.__prepare__, as in metaclass.__prepare__(name, bases, kwargs) . - Josiah From greg.ewing at canterbury.ac.nz Wed Mar 14 00:51:16 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 12:51:16 +1300 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: <45F738F4.2020206@canterbury.ac.nz> Guido van Rossum wrote: > For uniformity and generality I propose to keep it. So the absolute > minimal signature for __prepare__ is actually: > > def __prepare__(name, bases, *, metaclass=None): ... > > this way the prepare function can > distinguish between an explicit and an implied metaclass. Can you envisage any use case for distinguishing those? Seems to me you should be able to specify the metaclass either way and not have to worry about it behaving differently. > I think they should go to both If they go to both, and the metaclass keyword is left in both times, then every existing metaclass is going to need its constructor signature changed to have a metaclass=None in it. Is that really what you want? > All I obect to is the nesting of the metaclass *inside* the class. It > makes the metaclass essentially non-reusable ... it ends up having a > generic name ... hard to pickle right. For the use I had in mind (defining class methods) most of those things wouldn't be a problem. The metaclass is only intended for one purpose -- holding the class methods of the class being defined -- so you're not likely to want to reuse it, except when subclassing, in which case the subclass will get it as its implied metaclass. You're also not likely to need to pickle it, since its only instance is a top-level class, which will get pickled by name. The naming is an issue, though, and also the fact that if any of your bases have a custom metaclass, you would need to do something a bit awkward to make your nested metaclass inherit from it. What I really want is something like Smalltalk where you just say "I want these methods to be class methods" and a metaclass is automatically created behind the scenes to hold them, subclassed from the metaclass of the base class and given a suitable name. But that's something for another discussion. > It makes sense to put slots there, doesn't it? What bothers me about __slots__ in particular is that it's defining attributes, so conceptually it really belongs in the body. Or at least the *contents* of it does -- it's just the name '__slots__' that we want to keep out. I'm not sure what to do about that. -- Greg From guido at python.org Wed Mar 14 01:10:12 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 13 Mar 2007 17:10:12 -0700 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: <20070313152406.FB74.JCARLSON@uci.edu> References: <20070313152406.FB74.JCARLSON@uci.edu> Message-ID: [adding back the list] On 3/13/07, Josiah Carlson wrote: > What if the thing to the right of 'as' were assigned different values > depending on what it was? If it were one name, it would be the > exception (without traceback) "except ... as foo". If it was a 3-tuple > of names, it would get all three: "except ... as ecls, e, tb:". (with 2, > maybe e and tb) Anything else could raise a SyntaxError during the code > generation phase. > > In cases where we get a tb, we can clear the traceback at the end of the > except clause without needing to clear the exception, etc. Whether or > not the traceback exists as an attribute of the exception would then > become an implementation detail. For cases where we pass an exception > instance around, I would introduce a function called get_tb(e) (in > either sys or traceback) that takes the exception object and returns the > traceback. Whether or not it should print a warning unless the > keyword-only argument of 'yes_I_know_I_need_to_clear_the_traceback' is > provided and is True, or if it returns a weakref, I'll leave to someone > else's discretion. That sounds like too much complexity (both the syntax proposal and the get_tb() API). With the new GC, clearing the traceback is really not all that important any more except in the vicinity of buggy code that expects that leaving a scope GC's its locals; apart from that it's more of a performance issue than anything else, so I don't see why we can't just have __traceback__ on the exception object. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Wed Mar 14 01:50:27 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Tue, 13 Mar 2007 19:50:27 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <20070313160221.FB77.JCARLSON@uci.edu> References: <20070313160221.FB77.JCARLSON@uci.edu> Message-ID: On 3/13/07, Josiah Carlson wrote: > > Sounds far too implicit to me. I would much prefer a completely > different syntax. Having just had a patch that implements Windows shell > variable expansion as $var, ${var} and %var% (the last being new), I > can't help but think that %var% would be a better alternate explicit > syntax than the 'space follows {' implicit syntax you are offering. I think the syntax could be considered 'subtle', but I'm not really sure about how it is 'implicit' -- the rules of engagement are quite explicit. In any case, the current PEP has a % format specifier operator, so you would need a way to have % inside your %%. There may be some other issues with having the "transition to markup" and "transition back to text" characters be the same, but I can't think of them at the moment. > [snip] > > Feature: Automatic search of locals() and globals() for name lookups > > if no parameters are given. > > -1 > > for the obvious 'Explicit is better than implicit'. Out of curiosity, do you feel that eval()'s use of this is somehow different? I haven't been able to figure out a real difference yet. > > (Description of comments deleted) > > -1 > > The user can use the parser/compiler to get this behavior for free. > > (" text " #your comment here > "more text").format(...) I think breaking up strings this way can be really ugly. (I might be alone here, but OTOH if I am, I'm not sure why triple-quoted strings were invented :) I also think there may be use cases which involve, e.g. strings read in from a separate text file. It might be handy to have comments in that other file, and it would be nice not to require a separate user-code parser to strip them out. Thanks for the feedback. For some reason, my post hasn't garnered that much attention yet. Do I need to post it on python-dev or c.l.p., or are people just really busy with other things, or have I breached some etiquette I don't yet understand? Thanks, Pat From jcarlson at uci.edu Wed Mar 14 02:37:58 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 13 Mar 2007 18:37:58 -0700 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: <20070313160221.FB77.JCARLSON@uci.edu> Message-ID: <20070313181049.FB82.JCARLSON@uci.edu> "Patrick Maupin" wrote: > > On 3/13/07, Josiah Carlson wrote: > > > > Sounds far too implicit to me. I would much prefer a completely > > different syntax. Having just had a patch that implements Windows shell > > variable expansion as $var, ${var} and %var% (the last being new), I > > can't help but think that %var% would be a better alternate explicit > > syntax than the 'space follows {' implicit syntax you are offering. > > I think the syntax could be considered 'subtle', but I'm not really > sure about how it is 'implicit' -- the rules of engagement are quite > explicit. In any case, the current PEP has a % format specifier > operator, so you would need a way to have % inside your %%. There may > be some other issues with having the "transition to markup" and > "transition back to text" characters be the same, but I can't think of > them at the moment. It's all a matter of opinion regarding the meaning of 'implicit' and 'subtle'. Going with a couple dictionary definitions of implicit and subtle on dictionary.com: "Implied or understood though not directly expressed" "difficult to perceive or understand" I would say that it is not uncommon for one person's implied meaning to be difficult to perceive or understand to another. That is to say, in this case, I believe implicit is subtle, or really, your use of a construct with implied meaning that is not obvious to the casual observer, makes the meaning of the construct difficult to understand. Now, in the case of curly braces in a language like C/C++ or even Java, there was likely a secondary implied requirement, which constructs some kind of counter to keep track of the number of implied and required closing curly braces, for things like... void foo() { if (bar) { ... } } I'm still -1. Define the words how you want, escaping should be explicit via doubling, or through the use of an alternate (but equivalent) syntax. Heck, if you feel really ambitious, consider offering explicit start/end symbols with your .flag_format() method. > > [snip] > > > Feature: Automatic search of locals() and globals() for name lookups > > > if no parameters are given. > > > > -1 > > > > for the obvious 'Explicit is better than implicit'. > > Out of curiosity, do you feel that eval()'s use of this is somehow > different? I haven't been able to figure out a real difference yet. eval() should require explicit locals() and globals() if the user wants to use them. Not providing a locals() at least should be an error. > > > (Description of comments deleted) > > > > -1 > > > > The user can use the parser/compiler to get this behavior for free. > > > > (" text " #your comment here > > "more text").format(...) > > I think breaking up strings this way can be really ugly. (I might be > alone here, but OTOH if I am, I'm not sure why triple-quoted strings > were invented :) I also think there may be use cases which involve, > e.g. strings read in from a separate text file. It might be handy to > have comments in that other file, and it would be nice not to require > a separate user-code parser to strip them out. from other import formatx from other import __doc__ as formatx formatx = ConfigObj.unrepr(open('formatx', 'r').read()) I'm not saying that breaking up strings as I did is pretty. What I'm saying is that you can do it now. I'll also say that inlining comments is ugly and confusing. I'd rather not have the feature than have an ugly and confusing feature. Also, when breaking up the strings, you can exploit your editor's syntax highlighting to comment as necessary. Yes, it requires using Python's import machinery, exec, or even a mangled variant of ConfigObj.unrepr() (what I would personally suggest), but those are relatively small prices to pay to keep the syntax from being confusing. > Thanks for the feedback. For some reason, my post hasn't garnered > that much attention yet. Do I need to post it on python-dev or > c.l.p., or are people just really busy with other things, or have I > breached some etiquette I don't yet understand? I would guess that people are busy and/or somewhat uninterested. Personally, I'm happy with '...'%(...), and commented on this feature because (like many other features) I would much prefer Python to be pretty. - Josiah From jcarlson at uci.edu Wed Mar 14 02:56:47 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 13 Mar 2007 18:56:47 -0700 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: References: <20070313152406.FB74.JCARLSON@uci.edu> Message-ID: <20070313183808.FB85.JCARLSON@uci.edu> "Guido van Rossum" wrote: > [adding back the list] (oops) > On 3/13/07, Josiah Carlson wrote: > > What if the thing to the right of 'as' were assigned different values > > depending on what it was? If it were one name, it would be the > > exception (without traceback) "except ... as foo". If it was a 3-tuple > > of names, it would get all three: "except ... as ecls, e, tb:". (with 2, > > maybe e and tb) Anything else could raise a SyntaxError during the code > > generation phase. > > > > In cases where we get a tb, we can clear the traceback at the end of the > > except clause without needing to clear the exception, etc. Whether or > > not the traceback exists as an attribute of the exception would then > > become an implementation detail. For cases where we pass an exception > > instance around, I would introduce a function called get_tb(e) (in > > either sys or traceback) that takes the exception object and returns the > > traceback. Whether or not it should print a warning unless the > > keyword-only argument of 'yes_I_know_I_need_to_clear_the_traceback' is > > provided and is True, or if it returns a weakref, I'll leave to someone > > else's discretion. > > That sounds like too much complexity (both the syntax proposal and the > get_tb() API). With the new GC, clearing the traceback is really not > all that important any more except in the vicinity of buggy code that > expects that leaving a scope GC's its locals; apart from that it's > more of a performance issue than anything else, so I don't see why we > can't just have __traceback__ on the exception object. The point of get_tb() would be to help prevent people from getting at the traceback unless they really needed it. If I remember the discussion about 'x = sys.exc_info()' and/or the automatic clearing of the name in the 'except ... as name:' syntax, the reason for clearing the exception and traceback from the locals was to prevent cycle issues, etc. By requiring people to be explicit about when they wanted the traceback (rather than giving it to them all the time via e.__traceback__), we can prevent many of those issues. Also, with a little work, we can force users to at least acknowledge proper cleanup semantics. I'll presume this idea is rejected, no need to reply. - Josiah In order to get maximum benefit from the 'give me the traceback', really it would take two dictionaries, one that is a mapping from exception instances to threads, and a second that is a mapping from threads to (exception instance, traceback) tuples. All current and proposed semantics are easily derived from these two structures (my reading of the sys.exc_* functions suggest that something like this already exists, though is perhaps attached to the thread, and not in a dictionary). From nnorwitz at gmail.com Wed Mar 14 04:19:23 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 13 Mar 2007 19:19:23 -0800 Subject: [Python-3000] __special__ attrs looked up on the type, not instance Message-ID: ---------- Forwarded message from python-3000-checkins ---------- Neal Norwitz schrieb: > I assume this is not the desired behaviour? > >>>> class F: > ... def __dir__(self): > ... return [5] > ... >>>> dir(F()) > [5] >>>> f = F() >>>> dir(f) > [5] >>>> def __dir__(): return [10] > ... >>>> f.__dir__ = __dir__ >>>> dir(f) > [5] > > I think the problem is in _dir_object() > > + PyObject * dirfunc = PyObject_GetAttrString((PyObject*)obj->ob_type, > + "__dir__"); > > Shouldn't the first arg just be obj, not obj->ob_type? [Georg] This is modeled after the principle that for new-style objects, __special__ methods are looked up on the type, not the instance. ----- 1) I didn't remember this, do we have it documented somewhere? 2) Assuming #1 is correct, is this rule consistently applied? 3) How does (should) this affect 2.6 and migration to 3.0, if at all? n From python at rcn.com Wed Mar 14 03:53:48 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 13 Mar 2007 22:53:48 -0400 (EDT) Subject: [Python-3000] Octal Message-ID: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> Now that the 20th century is safely behind us, do we still want literals with leading zeroes to be interpreted as octal? Raymond From jcarlson at uci.edu Wed Mar 14 05:59:39 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 13 Mar 2007 21:59:39 -0700 Subject: [Python-3000] Octal In-Reply-To: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> Message-ID: <20070313215636.FB8B.JCARLSON@uci.edu> Raymond Hettinger wrote: > Now that the 20th century is safely behind us, do we still want > literals with leading zeroes to be interpreted as octal? Do we deprecate it followed by a later removal (so as to "resist the temptation to guess")? If so, sounds good to me (I've never had a use for octal literals). Making them decimal instead, I think, would be a mistake. - Josiah From greg.ewing at canterbury.ac.nz Wed Mar 14 06:07:13 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 18:07:13 +1300 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: <20070313160221.FB77.JCARLSON@uci.edu> Message-ID: <45F78301.8050903@canterbury.ac.nz> Patrick Maupin wrote: > Out of curiosity, do you feel that eval()'s use of this is somehow > different? I haven't been able to figure out a real difference yet. I think the difference is that when you see eval() you know something dangerous is going on, but most uses of format() would be harmless -- *except* when given no arguments. It would be an easy thing to overlook when reading code. If this functionality is to be provided, I think it should at least have a different method name to make it stand out more clearly. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 14 06:11:38 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 18:11:38 +1300 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <20070313181049.FB82.JCARLSON@uci.edu> References: <20070313160221.FB77.JCARLSON@uci.edu> <20070313181049.FB82.JCARLSON@uci.edu> Message-ID: <45F7840A.6020308@canterbury.ac.nz> Josiah Carlson wrote: > eval() should require explicit locals() and globals() if the user wants > to use them. Not providing a locals() at least should be an error. Or should default to an empty dict. You might want to use eval() on something known to be a constant expresssion that doesn't refer to any names. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 14 06:16:08 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 18:16:08 +1300 Subject: [Python-3000] exception info [was: Discussions with no PEPs] In-Reply-To: <20070313183808.FB85.JCARLSON@uci.edu> References: <20070313152406.FB74.JCARLSON@uci.edu> <20070313183808.FB85.JCARLSON@uci.edu> Message-ID: <45F78518.5040003@canterbury.ac.nz> Josiah Carlson wrote: > By requiring people to be explicit about when they wanted the > traceback (rather than giving it to them all the time via > e.__traceback__), we can prevent many of those issues. If any way is provided of getting a traceback given an exception object, whether it's get_tb(e) or e.__traceback__, it's necessary to keep the traceback around as long as the exception object lives. So I don't see what you would gain from this. -- Greg From greg.ewing at canterbury.ac.nz Wed Mar 14 06:38:27 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 18:38:27 +1300 Subject: [Python-3000] Octal In-Reply-To: <20070313215636.FB8B.JCARLSON@uci.edu> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> Message-ID: <45F78A53.3000704@canterbury.ac.nz> Josiah Carlson wrote: > Do we deprecate it followed by a later removal (so as to "resist the > temptation to guess")? If so, sounds good to me (I've never had a use > for octal literals). I think that *some* syntax should be provided for octal literals. They're useful when you're translating constants from a C header file that are expressed in octal. I'd suggest 0o123 except that the lower case 'o' might be a bit hard to spot. :-( Maybe something more general could be used to indicate a number base, such as 1101(2) # binary 1234(8) # octal 1c3a(16) # hexadecimal 12g7(35) # why stop at 16? Since calling a built-in integer never makes sense, this would be unambiguous. > Making them decimal instead, I think, would be a > mistake. Perhaps an all-digits literal with a leading zero should be disallowed altogether. That ought to prevent any accidents. -- Greg From pmaupin at gmail.com Wed Mar 14 06:57:10 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 00:57:10 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <45F78301.8050903@canterbury.ac.nz> References: <20070313160221.FB77.JCARLSON@uci.edu> <45F78301.8050903@canterbury.ac.nz> Message-ID: On 3/14/07, Greg Ewing wrote: > > I think the difference is that when you see eval() > you know something dangerous is going on, but most > uses of format() would be harmless -- *except* when > given no arguments. It would be an easy thing to > overlook when reading code. > > If this functionality is to be provided, I think > it should at least have a different method name > to make it stand out more clearly. That might be a reasonable difference. I can think of two possible counterarguments: 1) if there is always "something dangerous" going on with eval(), making it harder to write eval(), and making the user think about which set of variables he really wants to pass in might not be such a bad thing, either; and 2) the actual percentage of use cases of ''.format() where "something dangerous" is going on, e.g. where the programmer has passed a string that he has accepted at face value from J. Random User, is, realistically, going to be vanishingly small. (In most cases, .format will probably immediately follow a literal string.) Ian Bicking actually suggested 'somestring'.eval() for this functionality. I like that a lot, in the sense that it gives the same sense of danger as eval('somestring'), but at the same time, I dislike it intensely because eval('somestring') and 'somestring'.eval() appear so similar but would give such drastically different results. If the default usage of locals() / globals() is really that objectionable, we can certainly take it out. In that case, I would suggest that one possible thing to do is: 1) Make calling format() with no variables raise an exception; and 2) Monitor the usage and see how often people do 'somestring'.format(namespace=locals()) or 'somestring'.format(namespace=(locals(), globals)) If the number of times this extra boilerplate appears in code seems excessive, then we could either add another method to do the automatic locals() and globals(), or (since we would know there are no uses of format() with no variables in the code) reenable the functionality. Regards, Pat From pmaupin at gmail.com Wed Mar 14 07:08:19 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 01:08:19 -0500 Subject: [Python-3000] Octal In-Reply-To: <45F78A53.3000704@canterbury.ac.nz> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <45F78A53.3000704@canterbury.ac.nz> Message-ID: On 3/14/07, Greg Ewing wrote: > 1101(2) # binary > 1234(8) # octal > 1c3a(16) # hexadecimal > 12g7(35) # why stop at 16? > > Since calling a built-in integer never makes sense, > this would be unambiguous. That's a great idea all the way around. Another possible syntax would be 1101 at 2, 1234 at 8, etc. I don't think that could mean anything currently. > Perhaps an all-digits literal with a leading zero > should be disallowed altogether. That ought to > prevent any accidents. Yeah, you can't change it; you have to deprecate it and then start raising an exception. Pat From talin at acm.org Wed Mar 14 07:08:27 2007 From: talin at acm.org (Talin) Date: Tue, 13 Mar 2007 23:08:27 -0700 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: <20070313160221.FB77.JCARLSON@uci.edu> <45F78301.8050903@canterbury.ac.nz> Message-ID: <45F7915B.9040505@acm.org> Patrick Maupin wrote: > Ian Bicking actually suggested 'somestring'.eval() for this > functionality. I like that a lot, in the sense that it gives the same > sense of danger as eval('somestring'), but at the same time, I dislike > it intensely because eval('somestring') and 'somestring'.eval() appear > so similar but would give such drastically different results. Then I would call that 'somestring'.expand(), or perhaps 'substitute' or 'interpolate' - implying that you are doing an expansion of the named fields within the current scope. > If the default usage of locals() / globals() is really that > objectionable, we can certainly take it out. In that case, I would > suggest that one possible thing to do is: > > 1) Make calling format() with no variables raise an exception; and > 2) Monitor the usage and see how often people do > > 'somestring'.format(namespace=locals()) or > 'somestring'.format(namespace=(locals(), globals)) > > If the number of times this extra boilerplate appears in code seems > excessive, then we could either add another method to do the automatic > locals() and globals(), or (since we would know there are no uses of > format() with no variables in the code) reenable the functionality. From cvrebert at gmail.com Wed Mar 14 07:17:15 2007 From: cvrebert at gmail.com (Chris Rebert) Date: Tue, 13 Mar 2007 23:17:15 -0700 Subject: [Python-3000] Octal In-Reply-To: <45F78A53.3000704@canterbury.ac.nz> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <45F78A53.3000704@canterbury.ac.nz> Message-ID: <45F7936B.8040209@gmail.com> I agree that octal should still have a syntax. However, as you say, 0o doesn't jump out as much typographically. And I'm wary of adding arbitrary base literals to the language. It sounds like unnecessary complexity, though a standard library function to convert arbitrary base representations of ints to ints might be useful. At any rate, I'd like to propose the octal syntax: 0c123 I like this because "0c" looks like the start of the word "octal", and, as with your suggestion, the 0[character-here] prefix makes for a nice symmetry with "0x" for hex. - Chris Rebert Greg Ewing wrote: > Josiah Carlson wrote: > >> Do we deprecate it followed by a later removal (so as to "resist the >> temptation to guess")? If so, sounds good to me (I've never had a use >> for octal literals). > > I think that *some* syntax should be provided for octal > literals. They're useful when you're translating constants > from a C header file that are expressed in octal. I'd > suggest > > 0o123 > > except that the lower case 'o' might be a bit hard > to spot. :-( > > Maybe something more general could be used to > indicate a number base, such as > > 1101(2) # binary > 1234(8) # octal > 1c3a(16) # hexadecimal > 12g7(35) # why stop at 16? > > Since calling a built-in integer never makes sense, > this would be unambiguous. > > > Making them decimal instead, I think, would be a > > mistake. > > Perhaps an all-digits literal with a leading zero > should be disallowed altogether. That ought to > prevent any accidents. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/cvrebert%40gmail.com > From pmaupin at gmail.com Wed Mar 14 07:20:52 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 01:20:52 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <45F7915B.9040505@acm.org> References: <20070313160221.FB77.JCARLSON@uci.edu> <45F78301.8050903@canterbury.ac.nz> <45F7915B.9040505@acm.org> Message-ID: On 3/14/07, Talin wrote: > Then I would call that 'somestring'.expand(), or perhaps 'substitute' or > 'interpolate' - implying that you are doing an expansion of the named > fields within the current scope. 'expand' isn't bad. On the one hand, I really would like the ability to do automatic locals() and globals() lookup, but on the other hand, I'm not personally so much hung up on the amount of typing as I am on the esthetics, so to me, import string string.expand('somestring') would be equally as appealing as having a string expand method, and would probably be more difficult for someone to do accidentally. Pat From python at rcn.com Wed Mar 14 07:22:30 2007 From: python at rcn.com (Raymond Hettinger) Date: Tue, 13 Mar 2007 23:22:30 -0700 Subject: [Python-3000] Octal References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> Message-ID: <026001c76601$2ae83550$f101a8c0@RaymondLaptop1> > Raymond Hettinger wrote: >> Now that the 20th century is safely behind us, do we still want >> literals with leading zeroes to be interpreted as octal? [Josiah] > Do we deprecate it followed by a later removal Nope. Just drop them from Python 3000. No one (except Greg) will miss them. The 2-to-3 tool can convert 0123 constants to decimal or hex. Any need to process external octal value can already be done explicitly through: int(octstring, 8). Raymond P.S. My note was occasioned by a colleague encountering errors with integer data that had leading zeroes. He was surprised to find-out that the lead zero notation for octal literals had been around for a long time. From rasky at develer.com Wed Mar 14 08:04:32 2007 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 14 Mar 2007 08:04:32 +0100 Subject: [Python-3000] Octal In-Reply-To: <026001c76601$2ae83550$f101a8c0@RaymondLaptop1> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <026001c76601$2ae83550$f101a8c0@RaymondLaptop1> Message-ID: On 14/03/2007 7.22, Raymond Hettinger wrote: > [Josiah] >> Do we deprecate it followed by a later removal > > Nope. Just drop them from Python 3000. No one (except Greg) will miss them. > The 2-to-3 tool can convert 0123 constants to decimal or hex. I don't think anyone use octal unless there is a very compelling reason. Thus, I'd make it convert the octal literal to int("...", 8). If the programmer used octal, it probably means something in his mind (eg: UNIX attributes) so I'd leave it that way in the source code. The corresponding decimal/hex could be totally meaningless in that context (or he would have not used octal in the first place...). -- Giovanni Bajo From talin at acm.org Wed Mar 14 08:08:31 2007 From: talin at acm.org (Talin) Date: Wed, 14 Mar 2007 00:08:31 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> Message-ID: <45F79F6F.6090701@acm.org> Guido van Rossum wrote: >> I'm a bit worried that class headers are going to become >> rather cluttered if we start putting all sorts of stuff >> in there. >> >> E.g. are you intending to put __slots__ there? It makes >> sense, but it could lead to some rather long and >> unwieldy class headers. > > It makes sense to put slots there, doesn't it? I'm not too worried > about the long class headers; a formatting convention can do wonders > here. Actually, with the new system of metaclasses, we might not need __slots__ anymore: class Foo(metaclass=Slottabl From rasky at develer.com Wed Mar 14 08:06:22 2007 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 14 Mar 2007 08:06:22 +0100 Subject: [Python-3000] Octal In-Reply-To: <45F78A53.3000704@canterbury.ac.nz> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <45F78A53.3000704@canterbury.ac.nz> Message-ID: On 14/03/2007 6.38, Greg Ewing wrote: >> Do we deprecate it followed by a later removal (so as to "resist the >> temptation to guess")? If so, sounds good to me (I've never had a use >> for octal literals). > > I think that *some* syntax should be provided for octal > literals. If we get to the point that builtin calls are compile-time optimized (and we really really should), int("1234", 8) could even be folded at byte-code level. -- Giovanni Bajo From talin at acm.org Wed Mar 14 08:09:46 2007 From: talin at acm.org (Talin) Date: Wed, 14 Mar 2007 00:09:46 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F79F6F.6090701@acm.org> References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> <45F79F6F.6090701@acm.org> Message-ID: <45F79FBA.6090000@acm.org> Ack, I meant to hit cancel and not Send. Please ignore that. Talin wrote: > Guido van Rossum wrote: >>> I'm a bit worried that class headers are going to become >>> rather cluttered if we start putting all sorts of stuff >>> in there. >>> >>> E.g. are you intending to put __slots__ there? It makes >>> sense, but it could lead to some rather long and >>> unwieldy class headers. >> It makes sense to put slots there, doesn't it? I'm not too worried >> about the long class headers; a formatting convention can do wonders >> here. > > Actually, with the new system of metaclasses, we might not need > __slots__ anymore: > > class Foo(metaclass=Slottabl > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/talin%40acm.org > From talin at acm.org Wed Mar 14 08:29:09 2007 From: talin at acm.org (Talin) Date: Wed, 14 Mar 2007 00:29:09 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 Message-ID: <45F7A445.5090507@acm.org> PEP: xxx Title: Metaclasses in Python 3000 Version: $Revision$ Last-Modified: $Date$ Author: Talin Status: Draft Type: Standards Content-Type: text/plain Created: 07-Mar-2007 Python-Version: 3.0 Post-History: 11-March-2007 Abstract This PEP proposes changing the syntax for declaring metaclasses, and alters the semantics for how classes with metaclasses are constructed. Rationale There are two rationales for this PEP, both of which are somewhat subtle. The primary reason for changing the way metaclasses work, is that there are a number of interesting use cases that require the metaclass to get involved earlier in the class construction process than is currently possible. Currently, the metaclass mechanism is essentially a post-processing step. With the advent of class decorators, much of these post-processing chores can be taken over by the decorator mechanism. In particular, there is an important body of use cases where it would be useful to preserve the order in which a class members are declared. Ordinary Python objects store their members in a dictionary, in which ordering is unimportant, and members are accessed strictly by name. However, Python is often used to interface with external systems in which the members are organized according to an implicit ordering. Examples include declaration of C structs; COM objects; Automatic translation of Python classes into IDL or database schemas, such as used in an ORM; and so on. In such cases, it would be useful for a Python programmer to specify such ordering directly using the declaration order of class members. Currently, such orderings must be specified explicitly, using some other mechanism (see the ctypes module for an example.) Unfortunately, the current method for declaring a metaclass does not allow for this, since the ordering information has already been lost by the time the metaclass comes into play. By allowing the metaclass to get involved in the class construction process earlier, the new system allows the ordering or other early artifacts of construction to be preserved and examined. The other, weaker, rationale is purely cosmetic: The current method for specifying a metaclass is by assignment to the special variable __metaclass__, which is considered by some to be aesthetically less than ideal. Others disagree strongly with that opinion. This PEP will not address this issue, other than to note it, since aesthetic debates cannot be resolved via logical proofs. Specification In the new model, the syntax for specifying a metaclass is via a keyword argument in the list of base classes: class Foo(base1, base2, metaclass=mymeta): ... Additional keywords will also be allowed here, and will be passed to the metaclass, as in the following example: class Foo(base1, base2, metaclass=mymeta, private=True): ... Note that this PEP makes no attempt to define what these other keywords might be - that is up to metaclass implementors to determine. More generally, the parameter list passed to a class definition will now support all of the features of a function call, meaning that you can now use *args and **kwargs-style arguments in the class base list: class Foo(*bases, **kwds): ... Invoking the Metaclass In the current metaclass system, the metaclass object can be any callable type. This does not change, however in order to fully exploit all of the new features the metaclass will need to have an extra attribute which is used during class pre-construction. This attribute is named __prepare__, which is invoked as a function before the evaluation of the class body. The __prepare__ function takes two positional arguments, and an arbitrary number of keyword arguments. The two positional arguments are: 'name' - the name of the class being created. 'bases' - the list of base classes. The interpreter always tests for the existence of __prepare__ before calling it; If it is not present, then a regular dictionary is used, as illustrated in the following Python snippet. def prepare_class(name, *bases, metaclass=type, **kwargs): prepare = getattr(metaclass, '__prepare__', None) if prepare is not None: return prepare(name, bases, **kwargs) else: return dict() The example above illustrates how the arguments to 'class' are interpreted. The class name is the first argument, followed by an arbitrary length list of base classes. After the base classes, there may be one or more keyword arguments, one of which can be 'metaclass'. Note that the 'metaclass' argument is not included in kwargs, since it is filtered out by the normal parameter assignment algorithm. (Note also that 'metaclass' is a keyword- only argument as per PEP 3102 [6].) __prepare__ returns a dictionary-like object which is used to store the class member definitions during evaluation of the class body. In other words, the class body is evaluated as a function block (just like it is now), except that the local variables dictionary is replaced by the dictionary returned from __prepare__. This dictionary object can be a regular dictionary or a custom mapping type. It does not need to implement the full dictionary interface; only the ability to insert items and retrieve them are required. (Note: double check that this is true). Note that __prepare__ is generally a class method, not an instance method because it is called before the metaclass instance (i.e. the class itself) is created. Once the class body has finished evaluating, the metaclass will be called (as a callable) with the class dictionary, which is no different from the current metaclass mechanism. Typically, a metaclass will create a custom dictionary - either a subclass of dict, or a wrapper around it - that will contain additional properties that are set either before or during the evaluation of the class body. Then in the second phase, the metaclass can use these additional properties to further customize the class. An example would be a metaclass that uses information about the ordering of member declarations to create a C struct. The metaclass would provide a custom dictionary that simply keeps a record of the order of insertions. This does not need to be a full 'ordered dict' implementation, but rather just a Python list of (key,value) pairs that is appended to for each insertion. Note that in such a case, the metaclass would be required to deal with the possibility of duplicate keys, but in most cases that is trivial. The metaclass can use the first declaration, the last, combine them in some fashion, or simply throw an exception. It's up to the metaclass to decide how it wants to handle that case. Example: Here's a simple example of a metaclass which creates a list of the names of all class members, in the order that they were declared: # The metaclass class OrderedClass(type): # The custom dictionary class member_table(dict): def __init__(self): self.member_names = [] def __setitem__(self, key, value): # if the key is not already defined, add to the # list of keys. if key not in self: self.member_names.append( key ) # Call superclass dict.setitem(self, key value) # The prepare function @classmethod def __prepare__(cls, name, bases): # No keywords in this case return self.member_table() # The metaclass invocation def __init__(self, name, bases, classdict): # Note that we replace the classdict with a regular # dict before passing it to the superclass, so that we # don't continue to record member names after the class # has been created. result = type(name, bases, dict(classdict)) result.member_names = classdict.member_names return result class MyClass(metaclass=OrderedClass): # method1 goes in array element 0 def method1(self): pass # method2 goes in array element 1 def method2(self): pass Alternate Proposals Josiah Carlson proposed using the name 'type' instead of 'metaclass', on the theory that what is really being specified is the type of the type. While this is technically correct, it is also confusing from the point of view of a programmer creating a new class. From the application programmer's point of view, the 'type' that they are interested in is the class that they are writing; the type of that type is the metaclass. There were some objections in the discussion to the 'two-phase' creation process, where the metaclass is invoked twice, once to create the class dictionary and once to 'finish' the class. Some people felt that these two phases should be completely separate, in that there ought to be separate syntax for specifying the custom dict as for specifying the metaclass. However, in most cases, the two will be intimately tied together, and the metaclass will most likely have an intimate knowledge of the internal details of the class dict. Requiring the programmer to insure that the correct dict type and the correct metaclass type are used together creates an additional and unneeded burden on the programmer. Another good suggestion was to simply use an ordered dict for all classes, and skip the whole 'custom dict' mechanism. This was based on the observation that most use cases for a custom dict were for the purposes of preserving order information. However, this idea has two drawbacks, first because it means that an ordered dict implementation would have to be added to the set of built-in types in Python, and second because it would impose a slight speed (and complexity) penalty on all class declarations. Backwards Compatibility It would be possible to leave the existing __metaclass__ syntax in place. Alternatively, it would not be too difficult to modify the syntax rules of the Py3K translation tool to convert from the old to the new syntax. References [1] [Python-3000] Metaclasses in Py3K (original proposal) http://mail.python.org/pipermail/python-3000/2006-December/005030.html [2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax) http://mail.python.org/pipermail/python-3000/2006-December/005033.html [3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init) http://mail.python.org/pipermail/python-3000/2006-December/005108.html [4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict) http://mail.python.org/pipermail/python-3000/2006-December/005118.html [5] PEP 359: The 'make' statement - http://www.python.org/dev/peps/pep-0359/ [6] PEP 3102: Keyword-only arguments - http://www.python.org/dev/peps/pep-3102/ Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From greg.ewing at canterbury.ac.nz Wed Mar 14 08:49:43 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 14 Mar 2007 20:49:43 +1300 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F7A445.5090507@acm.org> References: <45F7A445.5090507@acm.org> Message-ID: <45F7A917.50405@canterbury.ac.nz> Talin wrote: > It does not need to implement the full dictionary interface; > only the ability to insert items and retrieve them are > required. (Note: double check that this is true). Deletion ability might be required as well, in case del is used in the class body. -- Greg From jan.grant at bristol.ac.uk Wed Mar 14 10:25:39 2007 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 14 Mar 2007 09:25:39 +0000 (GMT) Subject: [Python-3000] Discussions with no PEPs In-Reply-To: <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> References: <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <45F3ECFE.1090100@benjiyork.com> <45EF7767.9050406@acm.org> <2621FBA7-CC11-4463-85E6-A6330949AADB@python.org> <45F09C3D.2020009@canterbury.ac.nz> <45F3ECFE.1090100@benjiyork.com> <5.1.1.6.0.20070312142248.02d59658@sparrow.telecommunity.com> <5.1.1.6.0.20070312181728.045dfd88@sparrow.telecommunity.com> <5.1.1.6.0.20070312213458.02d2fc70@sparrow.telecommunity.com> <5.1.1.6.0.20070313104418.02a31658@sparrow.telecommunity.com> Message-ID: <20070314091750.O8223@tribble.ilrt.bris.ac.uk> On Tue, 13 Mar 2007, Phillip J. Eby wrote: > Java, ironically, avoids this problem by forcing you to have new types for > every single damn thing. The idiomatic way to have a priority-ordered > sequence of socket event listeners is to make a new base class or interface > just for that purpose -- so you end up with receiver-oriented interfaces as > a side effect! (In addition, Java has just enough overloading support that > a lot of the common cases for what would be type tests in Python, are done > using overloads in Java.) I believe this parenthetical statement is the most important one*. I've done a lot of recent work in Java, and my liking of interfaces is that they encourage developers to think in terms of contracts (for some arguably weak value thereof). A set of interfaces offer a nice way to document the APIs of coarse-grained components. Java has, as Phillip puts it, "receiver-oriented interfaces"; adapter classes are a fairly trivial thing to generate (my IDE does it for me) and crop up somewhat often as different large-scale components are plumbed together. So they do require some machinery, and java has no magical adaption (which I must confess, I like). But then Java is by design a language lacking many kinds of magic. The key point is that java methods are overloaded. That makes the interface notion as expressed in Java useful. Python methods are not overloaded. So interfaces (ABCs) as the sole feature of Python would perhaps retain the documentation features that I find compelling; but they would, alone, be much klunkier in use. Cheers, jan * it convinces me anyway. -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ "My army boots contain everything not in them." - Russell's pair o' Docs. From thomas at python.org Wed Mar 14 10:44:59 2007 From: thomas at python.org (Thomas Wouters) Date: Wed, 14 Mar 2007 10:44:59 +0100 Subject: [Python-3000] __special__ attrs looked up on the type, not instance In-Reply-To: References: Message-ID: <9e804ac0703140244l58210070x78975bfa4a93b32b@mail.gmail.com> On 3/14/07, Neal Norwitz wrote: > > ---------- Forwarded message from python-3000-checkins ---------- > > Neal Norwitz schrieb: > > I assume this is not the desired behaviour? > > > >>>> class F: > > ... def __dir__(self): > > ... return [5] > > ... > >>>> dir(F()) > > [5] > >>>> f = F() > >>>> dir(f) > > [5] > >>>> def __dir__(): return [10] > > ... > >>>> f.__dir__ = __dir__ > >>>> dir(f) > > [5] > > > > I think the problem is in _dir_object() > > > > + PyObject * dirfunc = > PyObject_GetAttrString((PyObject*)obj->ob_type, > > + "__dir__"); > > > > Shouldn't the first arg just be obj, not obj->ob_type? > > [Georg] > This is modeled after the principle that for new-style objects, > __special__ > methods are looked up on the type, not the instance. > > ----- > > 1) I didn't remember this, do we have it documented somewhere? > 2) Assuming #1 is correct, is this rule consistently applied? > 3) How does (should) this affect 2.6 and migration to 3.0, if at all? I don't remember seeing it documented, but it's the biggest (and hardest to detect) incompatibility between classic and new-style classes. It does, however, make sense to me. (The type is what defines behaviour, not the instance.) It seems to be quite consistently applied throughout the interpreter, but not throughout other modules that use their own __hooks__ -- say, pickle. That's because they (naturally) do 'obj.__hook__()', not 'obj.__class__.__hook__(obj)'. We could make this entirely consistent by making all __*__ methods be data descriptors on the class, which takes precedence over instance attributes. (So make them like property() rather than methods like now; you wouldn't be able to shadow them in an instance's __dict__.) A decorator would make sense too if we controlled all places the hooks get defined, but we don't, so it doesn't. As for 2.6->3.0 migration, it's all part of classic vs. new-style, but yes, we should warn about this. And yes, we can, although only by somewhat evil magic: make a list of affected __methods__ (unless we're going to solve it generically like I sketched above, which I doubt), make a special data descriptor on classic instances for each one, have that data descriptor check to see if the instance has a shadowing attribute in __dict__ and if so, warn and use it. n > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/thomas%40python.org > -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070314/e5cf379c/attachment.htm From thomas at python.org Wed Mar 14 10:59:30 2007 From: thomas at python.org (Thomas Wouters) Date: Wed, 14 Mar 2007 10:59:30 +0100 Subject: [Python-3000] Octal In-Reply-To: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> Message-ID: <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> On 3/14/07, Raymond Hettinger wrote: > > Now that the 20th century is safely behind us, do we still want literals > with leading zeroes to be interpreted as octal? What, this again? Without reading the rest of the thread I just know it'll devolve into discussion about arbitrary-base-integer-literals :-) (Ok, I snuck a peak just to make sure I won't get egg on my face.) I'm -1 on removing octal and hexadecimal literals, as I do use them both, and because of the fact that many, many other languages also use them. Some things are just stumbling blocks. FWIW, we get a great many complete-newbie questions on #python, and I do not believe I've *ever* seen anyone surprised about octal literals. They're surprised about other representation issues, like "?" coming out as "\xe2\x82\xac", every now and then about how to handle 'binary literals' and occasionally about os.chmod(755) not doing the right thing, but never about 010 coming out wrong. Oh, that's not entirely true: once, someone complained that 0101110 didn't come out right, but the problem wasn't the leading 0: he expected it to be interpreted as a binary number and come out as 46, and was equally surprised by 1101110. (Boy, did we have a hell of a time convincing that guy.) (And no, I don't think the number of questions #python gets about binary literals is enough to warrant binary literals. They're invariably new programmers tackling the problem as an exercise, or trying to get at the 'pure bits' of an int to perform bitwise operations in an inefficient manner.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070314/229296e0/attachment.html From p.f.moore at gmail.com Wed Mar 14 11:09:52 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 14 Mar 2007 10:09:52 +0000 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <20070313181049.FB82.JCARLSON@uci.edu> References: <20070313160221.FB77.JCARLSON@uci.edu> <20070313181049.FB82.JCARLSON@uci.edu> Message-ID: <79990c6b0703140309ieaa72dcua63f34637d01fe2@mail.gmail.com> On 14/03/07, Josiah Carlson wrote: > > "Patrick Maupin" wrote: > > Thanks for the feedback. For some reason, my post hasn't garnered > > that much attention yet. Do I need to post it on python-dev or > > c.l.p., or are people just really busy with other things, or have I > > breached some etiquette I don't yet understand? > > I would guess that people are busy and/or somewhat uninterested. > Personally, I'm happy with '...'%(...), and commented on this feature > because (like many other features) I would much prefer Python to be > pretty. I'm interested, but haven't had much time to review the PEP - as far as I can see, there's a lot to it, and I'm not sure I'd have good intuitions without trying the new functionality (something I haven't had the opportunity or need to do so far). FWIW, I tend to agree with most of Josiah's responses. In particular, I'm afraid I hate the whitespace-sensitive alternative syntax. %...% syntax would be a reasonable alternative for Windows users, but I take your point that % is already magical. I'd stick with {foo} and ${foo} (would you expect to allow the normal shortcut of $foo where foo is in the form of a simple identifier?) I get the impression you are close to having a publishable implementation - I'd say, get it released and get some feedback from real use. (If it's portable to 2.5, or even older versions, that is). You can mark it as alpha and explicitly note that some design issues still remain to be resolved. Paul. From ncoghlan at gmail.com Wed Mar 14 12:09:48 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Mar 2007 21:09:48 +1000 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: Message-ID: <45F7D7FC.5080908@gmail.com> (I've read the whole thread as it currently stands - responding to the initial post to cover the various topics) Patrick Maupin wrote: > Feature: Alternate syntaxes for escape to markup. > The syntaxes are similar enough that they can all be efficiently > parsed by the same loop, so there are no real implementation issues. > The currently contemplated method for declaring a markup syntax is by > using decorator-style markup, e.g. {@syntax1} inside the string, > although I am open to suggestions about better-looking ways to do > this. -1 Including a very limited set of alternate syntaxes in the base formatting operation seems like a bad idea from a readability point of view - the fact that the proposed alternatives happen to be easy to implement is no realy justification at all. For serious template usage, string.Template is a much stronger foundation for modification (as it allows complete alteration of the regular expression used to detect fields for substitution). A string.FormattedTemplate class that combined the pattern matching flexibility of string.Template with the formatting options of PEP 3101 would provide a significantly more powerful approach to supporting alternate syntaxes (as well as providing a good workout for the reusability of the formatting internals). > Feature: Automatic search of locals() and globals() for name lookups > if no parameters are given. > > This is contentious because it violates EIBTI. However, it is > extremely convenient. To me, the reasons for allowing or disallowing > this feature on 'somestring'.format() appear to be exactly the same as > the reasons for allowing or disallowing this feature on > eval('somestring'). Barring a distinction between these cases that I > have not noticed, I think that if we don't want to allow this for > 'somestring'.format(), then we should seriously consider removing the > capability in Python 3000 for eval('somestring'). -1 As others have noted, any use of eval() is already enough to raise alarm bells for a reviewer. It would be a pain if every use of string formatting had to be subjected to the same level of scrutiny. I suggest trawling through the py3k archive for a bit before deciding whether or not you feel it is worth getting distracted by this argument in order to save typing a dozen characters (or writing a two line utility function that you put in a support module somewhere). A reasonable place to start might be: http://mail.python.org/pipermail/python-3000/2006-December/004971.html > Feature: Ability to pass in a dictionary or tuple of dictionaries of > namespaces to search. > > This feature allows, in some cases, for much more dynamic code than > *kwargs. (You could manually smush multiple dictionaries together to > build kwargs, but that can be ugly, tedious, and slow.) > Implementation-wise, this feature and locals() / globals() go hand in > hand. +1 > Feature: Placement of a dummy record on the traceback stack for > underlying errors. > Removed feature: Ability to dump error information into the output string. +0 > Feature: Addition of functions and "constants" to string module. > > The PEP proposes doing everything as string methods, with a "cformat" > method allowing some access to the underlying machinery. I propose > only having a 'format' method of the string (unicode) type, and a > corresponding 'format' and extended 'flag_format' function in the > string module, along with definitions for the flags for access to > non-default underlying format machinery. +1 I significantly prefer this to the approach currently in the PEP - it keeps the string type's namespace comparatively clean, while providing access to the underlying building blocks when someone needs something which is 'similar but different'. I detest the name 'flag_format', though - the function doesn't format a flag! > Feature: Ability for "field hook" user code function to only be called > on some fields. > > The PEP takes an all-or-nothing approach to the field hook -- it is > either called on every field or no fields. Furthermore, it is only > available for calling if the extended function ('somestring'.cformat() > in the spec, string.flag_format() in this proposal) is called. The > proposed change keeps this functionality, but also adds a field type > specifier 'h' which causes the field hook to be called as needed on a > per-field basis. This latter method can even be used from the default > 'somestring'.format() method. -1. I don't like this - the caller has to provide a template that uses a hook specifier in the appropriate place, as well as providing the correct hook function. That kind of cross-dependency is ugly. The approach in the PEP is simple - every field is passed to the hook function, and the hook function decides whether or not it wants to override the default handling. Keep the string method simple, leave the flexibility and configurability for the underlying functions in the string module. > Changed feature: By default, not using all arguments is not an exception > > Also, it is arguably not Pythonic to require a check that all > arguments to a function are actually used by the execution of the > function (but see interfaces!), and format() is, after all, just > another function. So it seems that the default should be to not check > that all the arguments are used. In fact, there are similar reasons > for not using all the arguments here as with any other function. For > example, for customization, the format method of a string might be > called with a superset of all the information which might be useful to > view. +1 This seems like a good idea to me. > Feature: Ability to insert non-printing comments in format strings > > This feature is implemented in a very intuitive way, e.g. " text {# > your comment here} more text" (example shown with the default > transition to markup syntax). One of the nice benefits of this > feature is the ability to break up long source lines (if you have lots > of long variable names and attribute lookups). +0 Also a reasonable idea. > Feature: Exception raised if attribute with leading underscore accessed. > > The syntax supported by the PEP is deliberately limited in an attempt > to increase security. This is an additional security measure, which > is on by default, but can be optionally disabled if > string.flag_format() is used instead of 'somestring'.format(). -0 This is only an issue if implicit access to locals()/globals() is permitted, and is unlikely to help much in that case (underscores are rarely used with local variables, and those are the most likely to contain juicy information which may be leaked) > Feature: Support for "center" alignment. > > The field specifier uses "<" and ">" for left and right alignment. > This adds "^" for center alignment. +0 > Feature: support of earlier versions of Python > Feature: no global state Both significant improvements, in my opinion. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 14 12:14:00 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Mar 2007 21:14:00 +1000 Subject: [Python-3000] __special__ attrs looked up on the type, not instance In-Reply-To: References: Message-ID: <45F7D8F8.9090100@gmail.com> Neal Norwitz wrote: > [Georg] > This is modeled after the principle that for new-style objects, __special__ > methods are looked up on the type, not the instance. > > ----- > > 1) I didn't remember this, do we have it documented somewhere? > 2) Assuming #1 is correct, is this rule consistently applied? > 3) How does (should) this affect 2.6 and migration to 3.0, if at all? The principle isn't consistently applied - the with statement generates standard GET_ATTR opcodes, so it checks the instance first when looking for __enter__ and __exit__. It was questioned at the time, and Guido was OK with it - I believe his position was that defining special methods on instances may or may not affect behaviour, and whether or not it does so is implementation dependent. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Mar 14 12:17:12 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 14 Mar 2007 21:17:12 +1000 Subject: [Python-3000] Octal In-Reply-To: <45F7936B.8040209@gmail.com> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <45F78A53.3000704@canterbury.ac.nz> <45F7936B.8040209@gmail.com> Message-ID: <45F7D9B8.8090908@gmail.com> Chris Rebert wrote: > though a standard library function to convert arbitrary base > representations of ints to ints might be useful. >>> int('beef', 36) 531879 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From eric+python-dev at trueblade.com Wed Mar 14 12:36:43 2007 From: eric+python-dev at trueblade.com (Eric V. Smith) Date: Wed, 14 Mar 2007 07:36:43 -0400 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <45F7D7FC.5080908@gmail.com> References: <45F7D7FC.5080908@gmail.com> Message-ID: <45F7DE4B.7040208@trueblade.com> Nick Coghlan wrote: >> Feature: Exception raised if attribute with leading underscore accessed. >> >> The syntax supported by the PEP is deliberately limited in an attempt >> to increase security. This is an additional security measure, which >> is on by default, but can be optionally disabled if >> string.flag_format() is used instead of 'somestring'.format(). > > -0 > > This is only an issue if implicit access to locals()/globals() is > permitted, and is unlikely to help much in that case (underscores are > rarely used with local variables, and those are the most likely to > contain juicy information which may be leaked) That's not true. What this feature is trying to prevent is access to attributes of the passed in objects. For example: >>> from pep3101 import format >>> class Foo: pass ... >>> format("{0.__module__}", Foo()) Traceback (most recent call last): File "", line 1, in ? ValueError: Leading underscores not allowed in attribute/index strings at format_string[3] >>> format("{0.__module__}", Foo(), _allow_leading_underscores=1) '__main__' >>> format('{0.__module__.lower}', Foo(), _allow_leading_underscores=1) '' The thinking is that the format strings might come from a translation, or otherwise not be under the direct control of the original programmer. (I won't go so far as to say it's likely they'll be user-supplied, but I guess it's possible.) So be preventing access to attributes with leading underscores, we're trying to prevent access to arguably private attributes. I'm not sure it's much of a security measure, but it's something. Eric. From barry at python.org Wed Mar 14 14:14:35 2007 From: barry at python.org (Barry Warsaw) Date: Wed, 14 Mar 2007 09:14:35 -0400 Subject: [Python-3000] Octal In-Reply-To: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> Message-ID: <7BE92BBC-968D-4D86-83ED-6A5CB48559AA@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 13, 2007, at 10:53 PM, Raymond Hettinger wrote: > Now that the 20th century is safely behind us, do we still want > literals with leading zeroes to be interpreted as octal? Can we wait until Unix file permissions are expressed in something other than octal? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRff1PHEjvBPtnXfVAQI5jQP7BC2Uo99Rw/A5DhwYNa/itZH/NgYRdj5H h72a1yGcZXvzd9gqLqd83niA4GUTz/CdrqGmV0aiKs94tSjNGCppJSP/nJfyYnQV ZAZyupm+mE3433HU38YaroKQfgenKCP/7fbxnWTwYAo9UzyWvoc1brkr7/yGzQAt m9S16I/cMtU= =jvm3 -----END PGP SIGNATURE----- From barry at python.org Wed Mar 14 14:17:07 2007 From: barry at python.org (Barry Warsaw) Date: Wed, 14 Mar 2007 09:17:07 -0400 Subject: [Python-3000] Octal In-Reply-To: <026001c76601$2ae83550$f101a8c0@RaymondLaptop1> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <026001c76601$2ae83550$f101a8c0@RaymondLaptop1> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 14, 2007, at 2:22 AM, Raymond Hettinger wrote: > Nope. Just drop them from Python 3000. No one (except Greg) will > miss them. > The 2-to-3 tool can convert 0123 constants to decimal or hex. Any > need to > process external octal value can already be done explicitly through: > int(octstring, 8). os.mkdir('/tmp/foo', int('02755', 8)) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRff103EjvBPtnXfVAQJyCAP/WfF/zjH5RgHS1rBlSIvVElI8u/iqFGVs xRrszWaIUkL3665BhzaFjgvqJE3/Cxx3Hcs308uaBicefzy2daR8o+bX+81xKNIU 23BY3BgQAMOR6nDYP4ZtdeWUm5pDvOJuWGKuasn/GaLUASusP35EzrT6uOYF4wQc NuhIkEOveV0= =KQc3 -----END PGP SIGNATURE----- From eric+python-dev at trueblade.com Wed Mar 14 14:55:28 2007 From: eric+python-dev at trueblade.com (Eric V. Smith) Date: Wed, 14 Mar 2007 09:55:28 -0400 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <45F7DE4B.7040208@trueblade.com> References: <45F7D7FC.5080908@gmail.com> <45F7DE4B.7040208@trueblade.com> Message-ID: <45F7FED0.4010005@trueblade.com> Eric V. Smith wrote: > Nick Coghlan wrote: >>> Feature: Exception raised if attribute with leading underscore accessed. >>> >>> The syntax supported by the PEP is deliberately limited in an attempt >>> to increase security. This is an additional security measure, which >>> is on by default, but can be optionally disabled if >>> string.flag_format() is used instead of 'somestring'.format(). >> -0 >> >> This is only an issue if implicit access to locals()/globals() is >> permitted, and is unlikely to help much in that case (underscores are >> rarely used with local variables, and those are the most likely to >> contain juicy information which may be leaked) > > That's not true. What this feature is trying to prevent is access to > attributes of the passed in objects. For example: I should have said "It's not an issue only if locals()/globals() are allowed". It is true that automatic locals()/globals() magnifies this issue, but it exists even without it. Eric. From ncoghlan at gmail.com Wed Mar 14 15:43:58 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 15 Mar 2007 00:43:58 +1000 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <45F7DE4B.7040208@trueblade.com> References: <45F7D7FC.5080908@gmail.com> <45F7DE4B.7040208@trueblade.com> Message-ID: <45F80A2E.8020000@gmail.com> Eric V. Smith wrote: > Nick Coghlan wrote: >>> Feature: Exception raised if attribute with leading underscore accessed. >>> >>> The syntax supported by the PEP is deliberately limited in an attempt >>> to increase security. This is an additional security measure, which >>> is on by default, but can be optionally disabled if >>> string.flag_format() is used instead of 'somestring'.format(). >> -0 >> >> This is only an issue if implicit access to locals()/globals() is >> permitted, and is unlikely to help much in that case (underscores are >> rarely used with local variables, and those are the most likely to >> contain juicy information which may be leaked) > > That's not true. What this feature is trying to prevent is access to > attributes of the passed in objects. Ah, I misread it. Change that vote to a +1 then. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Wed Mar 14 15:50:54 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 06:50:54 -0800 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F738F4.2020206@canterbury.ac.nz> References: <45F1C734.7080503@acm.org> <5.1.1.6.0.20070312223834.040fdae8@sparrow.telecommunity.com> <45F738F4.2020206@canterbury.ac.nz> Message-ID: On 3/13/07, Greg Ewing wrote: > Guido van Rossum wrote: > > > For uniformity and generality I propose to keep it. So the absolute > > minimal signature for __prepare__ is actually: > > > > def __prepare__(name, bases, *, metaclass=None): ... > > > > this way the prepare function can > > distinguish between an explicit and an implied metaclass. > > Can you envisage any use case for distinguishing those? > Seems to me you should be able to specify the metaclass > either way and not have to worry about it behaving > differently. > > > I think they should go to both > > If they go to both, and the metaclass keyword is left > in both times, then every existing metaclass is going > to need its constructor signature changed to have > a metaclass=None in it. Is that really what you want? OK, you've convinced me. The metaclass= keyword, if present, should not be passed on to __prepare__ (by whatever name) or to the metaclass constructor itself. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 14 15:59:03 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 06:59:03 -0800 Subject: [Python-3000] Octal In-Reply-To: <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> Message-ID: I would miss them too (again, as unix perms, which have a large swath of my brain tissue devoted to them through nearly 30 years of use) but I see the problem that Raymond mentioned (one of his colleagues fell into the trap -- he didn't make this up) I really don't like to have to write int('660', 8). How about 0t660? This is most symmetric with 0xffe for hex, as it is the 3rd letter of the word in both cases. I think we also approved 0b101010 for binary numbers, didn't we? Other bases don't have enough of a following to bother. --Guido On 3/14/07, Thomas Wouters wrote: > > On 3/14/07, Raymond Hettinger wrote: > > Now that the 20th century is safely behind us, do we still want literals > with leading zeroes to be interpreted as octal? > > What, this again? Without reading the rest of the thread I just know it'll > devolve into discussion about arbitrary-base-integer-literals :-) (Ok, I > snuck a peak just to make sure I won't get egg on my face.) > > I'm -1 on removing octal and hexadecimal literals, as I do use them both, > and because of the fact that many, many other languages also use them. Some > things are just stumbling blocks. FWIW, we get a great many complete-newbie > questions on #python, and I do not believe I've *ever* seen anyone surprised > about octal literals. They're surprised about other representation issues, > like "?" coming out as "\xe2\x82\xac", every now and then about how to > handle 'binary literals' and occasionally about os.chmod(755) not doing the > right thing, but never about 010 coming out wrong. Oh, that's not entirely > true: once, someone complained that 0101110 didn't come out right, but the > problem wasn't the leading 0: he expected it to be interpreted as a binary > number and come out as 46, and was equally surprised by 1101110. (Boy, did > we have a hell of a time convincing that guy.) > > (And no, I don't think the number of questions #python gets about binary > literals is enough to warrant binary literals. They're invariably new > programmers tackling the problem as an exercise, or trying to get at the > 'pure bits' of an int to perform bitwise operations in an inefficient > manner.) > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me > spread! > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Wed Mar 14 16:21:32 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 10:21:32 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <45F7D7FC.5080908@gmail.com> References: <45F7D7FC.5080908@gmail.com> Message-ID: On 3/14/07, Nick Coghlan wrote: > Including a very limited set of alternate syntaxes in the base > formatting operation seems like a bad idea from a readability point of > view. The proposal includes the selection of (non-default) format options via markup in the string itself. It's easy to see when something different is going on. > the fact that the proposed alternatives happen to be easy to > implement is no realy justification at all. I absolutely agree with this statement. But sometimes even good ideas are left unimplemented because the implementation is difficult or error-prone, so I wasn't presenting this data point as a justification, just pointing out the lack of an impediment. > For serious template usage, string.Template is a much stronger > foundation for modification (as it allows complete alteration of the > regular expression used to detect fields for substitution). Agreed, but the question is really are there intermediate use-cases where it would be nice to support some different data domains without requiring the programmer to learn/understand all the features of yet another package, especially when there are obvious, known, simple use-cases for different syntaxes. > A string.FormattedTemplate class that combined the pattern matching > flexibility of string.Template with the formatting options of PEP 3101 > would provide a significantly more powerful approach to supporting > alternate syntaxes (as well as providing a good workout for the > reusability of the formatting internals). Agreed. I view templates as more "heavyweight", though. In my viewpoint, templates are optimized for reusing multiple times, and 'somestring'.format() is optimized for single one-off calls. Another issue is just the issue of learning something for one-time use. Or to put it another way, if I am doing floating point math, I won't bother to learn and start using numeric python until it really hurts. The same thing is probably true of templates. There is certainly a delicate balance to be struck in deciding the right amount of functionality for built-in stuff vs. libraries, and multiple syntaxes might just trip that balance. So another question here is, would this extra functionality be as frowned upon if it were only available in the string module functions? > I suggest trawling through the py3k archive for a bit before deciding > whether or not you feel it is worth getting distracted by this argument > in order to save typing a dozen characters (or writing a two line > utility function that you put in a support module somewhere). For me, it's not about either of thoses things. (See my earlier post on the possibility of doing string.expand()). The stackframe/utility function you mention might not work on all versions of Python, but it is easy for a builtin to access the stackframe. Do you have a real concern with the "utility function" being built in, like the proposed expand()? > I detest the name 'flag_format', though - the function doesn't format a > flag! I don't like it that much myself, either. I considered "extended_format" and a few others, but wasn't happy with any of them. Name suggestions are certainly welcome. > I don't like this - the caller has to provide a template that uses a > hook specifier in the appropriate place, as well as providing the > correct hook function. That kind of cross-dependency is ugly. But the other way, you have a hook function which has to know exactly which objects it is expected to format. You've really just moved the cross-dependency. > The approach in the PEP is simple - every field is passed to the hook > function, and the hook function decides whether or not it wants to > override the default handling. Which requires a lot of intelligence in the hook function. If you are displaying two integers, and you want to display one of them in the default format, and the other one in base 37, you will need to specify, in the field specifier of the format string, exactly how you want a particular integer displayed, in a fashion which is understandable to the hook function. So you really haven't even removed the dependency between the format string and the hook function. All you've done is made the hook function more complicated by forcing it to return None on any object that isn't an integer, or any integer field specifier that doesn't say 'base37", or something. > Keep the string method simple, leave the flexibility and configurability > for the underlying functions in the string module. OK, but it would be extra (and unnecessary IMO) work to allow a function in the string module to support this particular functionality but disallow it in the string method. > This is only an issue if implicit access to locals()/globals() is > permitted, and is unlikely to help much in that case (underscores are > rarely used with local variables, and those are the most likely to > contain juicy information which may be leaked) Eric already clarified this, but I wanted to reiterate that this is about attribute lookup as well as variable name lookup (and it's most consistent and easier to explain in the final docs if we just say that "identifiers cannot have leading underscores"). Thanks for the feedback. Regards, Pat From mattias at virtutech.se Wed Mar 14 16:58:08 2007 From: mattias at virtutech.se (=?iso-8859-1?q?Mattias_Engdeg=E5rd?=) Date: Wed, 14 Mar 2007 15:58:08 +0000 (UTC) Subject: [Python-3000] Octal References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <20070313215636.FB8B.JCARLSON@uci.edu> <45F78A53.3000704@canterbury.ac.nz> Message-ID: "Patrick Maupin" writes: >That's a great idea all the way around. Another possible syntax would >be 1101 at 2, 1234 at 8, etc. I don't think that could mean anything >currently. It's a terrible idea all the way around. Be reasonable: in modern code, only decimal, hex and binary constants are of general use at all. Anything else is very much special-purpose code. I have heard users of languages that use a general number base notation (16#fce2# in Ada and the similar syntax in Erlang) complain about how this buys them nothing compared to the much nicer 0x... syntax of Python and C for the bases that people actually use. I have frequently wanted binary constants when doing actual programming in Python - the 0b1101 syntax would certainly be handy. Perl allows it, but I hope that is not an argument against it - it would be even more useful in Python. From pje at telecommunity.com Wed Mar 14 17:15:39 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Mar 2007 11:15:39 -0500 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F7A917.50405@canterbury.ac.nz> References: <45F7A445.5090507@acm.org> <45F7A445.5090507@acm.org> Message-ID: <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> At 08:49 PM 3/14/2007 +1300, Greg Ewing wrote: >Talin wrote: > > It does not need to implement the full dictionary interface; > > only the ability to insert items and retrieve them are > > required. (Note: double check that this is true). > >Deletion ability might be required as well, in case >del is used in the class body. Right, and some weird metaclasses might only need the ability to set, or only the ability to get. (Actually, 90%+ of today's classes could get by with just setting, and no getting or deleting.) So the subset required is determined entirely by what the class body (and metaclass __new__) require. It should probably be noted that type.__new__ is still going to want a *real* dictionary, which it will then proxy. So the metaclass' __new__ is going to need to read out the contents of the pseudo-dict somehow, or generate alternative contents. From phd at phd.pp.ru Wed Mar 14 17:26:56 2007 From: phd at phd.pp.ru (Oleg Broytmann) Date: Wed, 14 Mar 2007 19:26:56 +0300 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> Message-ID: <20070314162656.GA11074@phd.pp.ru> On Wed, Mar 14, 2007 at 06:59:03AM -0800, Guido van Rossum wrote: > I really don't like to have to write int('660', 8). How about 0t660? > This is most symmetric with 0xffe for hex, as it is the 3rd letter of > the word in both cases. > > I think we also approved 0b101010 for binary numbers, didn't we? Other > bases don't have enough of a following to bother. 0b101010 0c660 0xffe I.e. the first letter from "bin", the second from "oct", the third from "hex". Also "0c" resembles "oc" from "oct". Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From guido at python.org Wed Mar 14 17:44:45 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 09:44:45 -0700 Subject: [Python-3000] New I/O PEP to Subversion Message-ID: I think we're far enough along with the new I/O PEP and trial implementation that I'm uncomfortable with the PEP living in Google docs only (http://docs.google.com/Doc?id=dfksfvqd_1cn5g5m). Does someone have half an hour of time available to convert it to standard PEP mark-up? See PEP 9 or 12. If you mail it to me I'll check it in. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Wed Mar 14 17:55:23 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 14 Mar 2007 10:55:23 -0600 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F7A445.5090507@acm.org> References: <45F7A445.5090507@acm.org> Message-ID: On 3/14/07, Talin wrote: > PEP: xxx > Title: Metaclasses in Python 3000 Looks good. > # The metaclass > class OrderedClass(type): > > # The custom dictionary > class member_table(dict): > def __init__(self): > self.member_names = [] > > def __setitem__(self, key, value): > # if the key is not already defined, add to the > # list of keys. > if key not in self: > self.member_names.append( key ) > > # Call superclass > dict.setitem(self, key value) Should probably be ``dict.__setitem__(self, key, value)``. > # The prepare function > @classmethod > def __prepare__(cls, name, bases): # No keywords in this case > return self.member_table() Should probably be: @classmethod def __prepare__(meta, name, bases): return meta.member_table() Perhaps __prepare__ should be special-cased like __new__ so that it's always a @classmethod? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From jackdied at jackdied.com Wed Mar 14 17:57:40 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Wed, 14 Mar 2007 12:57:40 -0400 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: References: <45F1C734.7080503@acm.org> Message-ID: <20070314165740.GB5293@performancedrivers.com> On Mon, Mar 12, 2007 at 05:30:48PM -0700, Guido van Rossum wrote: > Executive summary: I'm defending the PEP and the metaclass syntax it > proposes, and in fact I want the "clas arguments" to have the same > syntax as a call, including *args and **kwds. I'm only suggesting to > find a new name instead of __metacreate__. > > While I'm responding to the first message in the thread, I've actually > read the entire thread, and I'm responding in tit-for-tat mode to much > of it. (Yes, you'll find that there are actually a few messages in the > thread so far to which I *didn't* respond, and even some by Greg > Ewing. :-) I've snipped all the bits I understand and/or agree with. First, is this an accurate summary of the current proposal? class A(metaclass=meta): pass mydict = meta.__prepare__(name, bases, **kwds) cls = meta.__new__(name, bases, mydict, **kwds) meta.__init__(cls, name, bases, mydict, **kwds) A = cls > On 3/9/07, Jack Diederich wrote: > > I am a very big fan of ordered dicts in classes. > > Would you mind showing us some of your use cases / examples? This has only come up for me when using classes as namespaces. In a rule based application it was important to know the order of precedence. There are ways to work around it but they are a bit hackey. next_id = iter(xrange(1000)).next class Series700: housing = Plastics(order=next_id()) pins = Integer(0, 50, order=next_id()) (in reality the next_id() was called in the Plastics and Integer __init__s) I did a similar thing when making PLY class based instead of module based so I could have many grammars in a module. The other approach is what PLY does - When you look at the top of a PLY module you will see a list of strings that are the names of rules. The order of the list is the order in which the rules are applied. Also, the only non-cosmetic reasons listed in the PEP are all for ordered dicts! > On 3/10/07, Josiah Carlson wrote: > > Generally, about the only need I forsee for arbitrary keyword arguments > > in the class 'foo(...)' signature is to pass arguments to the 'metadict' > > factory (method, function, etc.). Otherwise it seems more like a syntax > > looking for a purpose than a purpose looking for a syntax. > > Or to the metaclass later on. I see keyword args as a way to pass > parameters to a metaclass that would otherwise have to be assigned to > class variables, in a namespace that's much more crowded. Imagine > being able to write this: > > class C(implements=(I1, I2)): > ... I don't understand the keyword args other than 'metaclass.' If class D inherits from class C does it also get passed the 'Implements' keyword? If it does the same effect could be acheived by making a metaclass maker function and just using the metaclass keyword. class C(metaclass=Implements(I1, I2)): ... If D doesn't get the 'Implements' keyword then the one-off behavior would be easier to put in a class decorator than a metaclass. Keywords also strike me as hard to stack. If a class has two keywords defining unrelated behaviors the metaclass would have to be a library-like thing that understood both of them. Mixing metaclasses from two seperate metaclass libraries would be hard - no harder than it currently is but keywords will invite more use. > On 3/10/07, Greg Ewing wrote: > > I still think that extra arguments are unnecessary in > > any case, because anything you could do by passing args > > to the dict creation function could be done by using a > > function instead of a class for the "metaclass". > > > > Does anyone have a counter-example? > > Not that you couldn't write it your way, but I find this rather cool looking: > > class C(implements=(I1, I2)): ... I like the more explicit class C(metaclass=Implements(I1, I2)): ... > On 3/10/07, Greg Ewing wrote: > > Note that you can already pass information to the > > metaclass itself using special attributes in the class > > namespace. This would only be need if you absolutely > > had to pass information when creating the *dict*. > > Things like your 'sealed' option don't seem to fall > > into that category. > > I don't know about sealed, but using class attributes topassing > parameter (other than the namespace itself) to the metaclass seems a > pretty lousy mechanism, and keyword arguments in the classdef are a > much cleaner solution for this need. For example, this solves the > problem that those attributes remain in the class dict but aren't > necessarily candidates for inheritance (example: __slots__, whose > inheritance story is, um, complicated; in general C.__slots__ does not > necessarily tell you all the slots that the class has). Agreed, I currently use metaclasses that pull behavior flags from the class body and it is less than pretty. In my experience when you have those behavior flags in the class they change for every subclass too. If they didn't they would be defined in the original metaclass. I think of metaclasses as 'apply always' and class decorators as 'apply once.' Everyone seems to have some loose ideas of what they would use the keywords for but I'm hung up on why this class C(metaclass=meta_library, sealed=True, implements=(I1, I2)): ... is preferable to this @seal @implements(I1, I2) class C(): ... or even this class C(metaclass=meta_library(sealed=True, implements=(I1, I2))): ... -Jack From jackdied at jackdied.com Wed Mar 14 18:13:43 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Wed, 14 Mar 2007 13:13:43 -0400 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F7A445.5090507@acm.org> References: <45F7A445.5090507@acm.org> Message-ID: <20070314171343.GC5293@performancedrivers.com> On Wed, Mar 14, 2007 at 12:29:09AM -0700, Talin wrote: > # The metaclass > class OrderedClass(type): > > # The custom dictionary > class member_table(dict): I would move the member_table class out of the OrderedClass namespace so no one gets any funny ideas that it has to be nested. > Alternate Proposals > > Another good suggestion was to simply use an ordered dict for all > classes, and skip the whole 'custom dict' mechanism. This was based > on the observation that most use cases for a custom dict were for > the purposes of preserving order information. However, this idea has > two drawbacks, first because it means that an ordered dict > implementation would have to be added to the set of built-in types > in Python, and second because it would impose a slight speed (and > complexity) penalty on all class declarations. FYI, I did a hacky implementation to test speed by making a copy of dictobject.c and storing key names in a PyList on the object. The slowdown for inserting 100k unique keys (/usr/share/dict/words) was 10%. There was no appreciable slowdown for lookup (it uses the normal hash lookup). I didn't implement deletion so I don't know how much it would suffer. -Jack From jcarlson at uci.edu Wed Mar 14 18:51:34 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 14 Mar 2007 10:51:34 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070314165740.GB5293@performancedrivers.com> References: <20070314165740.GB5293@performancedrivers.com> Message-ID: <20070314103429.FBA0.JCARLSON@uci.edu> Jack Diederich wrote: [snip] > I don't understand the keyword args other than 'metaclass.' > If class D inherits from class C does it also get passed the 'Implements' > keyword? If it does the same effect could be acheived by making a metaclass > maker function and just using the metaclass keyword. The keyword arguments to the class ( implements=(I1,I2) ) get passed as as the **kwargs in meta.__prepare__ (..., **kwargs), as well as the **kwargs in meta.__new__(..., **kwargs) (though the metaclass= keyword is removed when calling meta.__new__ ). > class C(metaclass=Implements(I1, I2)): > ... > > If D doesn't get the 'Implements' keyword then the one-off behavior > would be easier to put in a class decorator than a metaclass. > Keywords also strike me as hard to stack. If a class has two keywords > defining unrelated behaviors the metaclass would have to be a library-like > thing that understood both of them. Mixing metaclasses from two seperate > metaclass libraries would be hard - no harder than it currently is but > keywords will invite more use. As you say, stacking metaclasses is already hard and this would make them no more difficult. Combining metaclasses is not a stated purpose of this particular PEP (at least according to my reading), so I don't see as how that is in any way a strike against it. [snip] > is preferable to this > > @seal > @implements(I1, I2) > class C(): > ... It's pre-creation vs. post-creation. As defined, decorators are applied after the class creation, whereas meta.__prepare__ and meta.__new__ occur before. Note that one of the driving use-cases for meta.__prepare__, which is called prior to meta.__new__, is to offer ordered dictionary availability for things like... class mystruct(structbase): field1 = int4 field2 = uint8 Or even... class Contact(row): _if_not_exists_create = True firstname = text middlename = text lastname = text The point of offering metaclass=meta is to make it unambiguous what the metaclass is going to be. As it stands, a user is free to do any of the following things. class foo(object): a = 8 __metaclass__ = True ... __metaclass__ = fcn(a, __metaclass__) ... del __metaclass__ __metaclass__ = ... ... However, because we need the metaclass to be able to call metaclass.__prepare__, and to discover the metaclass we would have to emulate the above, we get to a chicken and egg issue. So we pull them out of the body and define it in the class header. > or even this > > class C(metaclass=meta_library(sealed=True, implements=(I1, I2))): > ... If one is allowing metaclass=, I see no point in disallowing other keywords. - Josiah From guido at python.org Wed Mar 14 19:15:54 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 11:15:54 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F7A445.5090507@acm.org> References: <45F7A445.5090507@acm.org> Message-ID: On 3/14/07, Talin wrote: > PEP: xxx > Title: Metaclasses in Python 3000 Checked in as PEP 3115. I fixed the two typos that were noted so far (missing comma and inconsistency in name of first arg to __prepare__; I renamed the latter metacls) and cleaned up one case of extra whitespace (.append( key ) -> .append(key)). Now on to the content: > def prepare_class(name, *bases, metaclass=type, **kwargs): > prepare = getattr(metaclass, '__prepare__', None) > if prepare is not None: > return prepare(name, bases, **kwargs) > else: > return dict() This is missing the extraction of the metaclass default from the bases. It's probably okay to default metaclass to None and add this code to the body: if metaclass is None: metaclass = compute_default_metaclass(bases) > type. It does not need to implement the full dictionary interface; > only the ability to insert items and retrieve them are > required. (Note: double check that this is true). As was mentioned, getitem, setitem and delitem are used. No other APIs are, unless the dict is accessed via locals(), or unless dir() is used. IMO it's up to the prepare function to provide the features that it wants to support; if it doesn't want to support deletions, that is between the metaclass and the users; the language spec doesn't have to say anything about this. > # The custom dictionary > class member_table(dict): > def __init__(self): Despite this just being an example, I'd like for the member_table class to be defined outside the OrderedClass class. it also probably ought to have a conforming name, i.e. MemberTable. > Another good suggestion was to simply use an ordered dict for all > classes, and skip the whole 'custom dict' mechanism. This was based > on the observation that most use cases for a custom dict were for > the purposes of preserving order information. However, this idea has > two drawbacks, first because it means that an ordered dict > implementation would have to be added to the set of built-in types > in Python, and second because it would impose a slight speed (and > complexity) penalty on all class declarations. I think this rebuttal isn't strong enough; I'm sure there *will* be use cases where a custom prepare method can solve a problem that a standard ordered dict couldn't. On 3/14/07, Steven Bethard wrote: > Perhaps __prepare__ should be special-cased like __new__ so that it's > always a @classmethod? No, I don't think __prepare__ should get special treatment. (And __new__ is a static method anyway, not a class method.) All in all, I think the PEP is on the fast track to acceptance. Thanks, Talin! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 14 19:37:37 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 11:37:37 -0700 Subject: [Python-3000] __special__ attrs looked up on the type, not instance In-Reply-To: <45F7D8F8.9090100@gmail.com> References: <45F7D8F8.9090100@gmail.com> Message-ID: On 3/14/07, Nick Coghlan wrote: > > 1) I didn't remember this, do we have it documented somewhere? > > 2) Assuming #1 is correct, is this rule consistently applied? > > 3) How does (should) this affect 2.6 and migration to 3.0, if at all? > > The principle isn't consistently applied - the with statement generates > standard GET_ATTR opcodes, so it checks the instance first when looking > for __enter__ and __exit__. > > It was questioned at the time, and Guido was OK with it - I believe his > position was that defining special methods on instances may or may not > affect behaviour, and whether or not it does so is implementation dependent. Right. At the same time, we could be more rigorous in 3.0 if we wanted to. We could even do this by hacking the default getattr implementation to skip the instance dict if the name starts and ends with two underscores. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Wed Mar 14 19:37:43 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 13:37:43 -0500 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: <20070313160221.FB77.JCARLSON@uci.edu> <20070313181049.FB82.JCARLSON@uci.edu> <79990c6b0703140309ieaa72dcua63f34637d01fe2@mail.gmail.com> Message-ID: On 3/14/07, Paul Moore wrote: > On 14/03/07, Josiah Carlson wrote: > > FWIW, I tend to agree with most of Josiah's responses. In particular, > I'm afraid I hate the whitespace-sensitive alternative syntax. %...% > syntax would be a reasonable alternative for Windows users, but I take > your point that % is already magical. I'd stick with {foo} and ${foo} > (would you expect to allow the normal shortcut of $foo where foo is in > the form of a simple identifier?) I originally wasn't going to support $foo, but when it was mentioned to me earlier (being a congenial sort of guy) I decided to put it in (it's in one version of the documentation checked into subversion), but after thinking about it some more I decided against it. I'm not really that big on saving keystrokes, and to me, $foo instead of ${foo} is a significant impairment to readability because there is more than one way to do it within the same format string. (It also drives complexity into the documentation and into the parser -- the rules about when you need {} and when you don't can get complicated.) > I get the impression you are close to having a publishable > implementation - I'd say, get it released and get some feedback from > real use. (If it's portable to 2.5, or even older versions, that is). > You can mark it as alpha and explicitly note that some design issues > still remain to be resolved. Yeah, it works pretty well, so after incorporating changes from this round of feedback, I think that's probably a pretty good plan. (Anybody who doesn't mind the occasional arrow in their back can already download and play with it. It's under sandbox/trunk/pep3101.) Thanks, Pat From pmaupin at gmail.com Wed Mar 14 19:48:26 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 13:48:26 -0500 Subject: [Python-3000] Octal In-Reply-To: <20070314162656.GA11074@phd.pp.ru> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: On 3/14/07, Oleg Broytmann wrote: > > 0b101010 > 0c660 > 0xffe > > I.e. the first letter from "bin", the second from "oct", the third from > "hex". Also "0c" resembles "oc" from "oct". -1 on "c" It's too visually close to "0" in some fonts. +1 on "t" "t" does not appear in 'binary' or 'hexadecimal' "x" does not appear in 'binary' or 'octal' "b" does not appear in 'octal' or 'hexadecimal' And finally "c" means "character" in %s or PEP3101, and "t" is not yet defined as a type specifier. So just to couch it all in terms of a proposal: - In 2.6 and 3.0, we add 0t1234 as a valid octal number - In 2.6, we issue a deprecation warning for a leading literal 0 which is followed immediately by another digit. - In 3.0, that becomes an exception - If people really are still using octal that much, we should also consider adding it in to PEP3101. Regards, Pat From guido at python.org Wed Mar 14 19:50:25 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 11:50:25 -0700 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <20070314165740.GB5293@performancedrivers.com> References: <45F1C734.7080503@acm.org> <20070314165740.GB5293@performancedrivers.com> Message-ID: On 3/14/07, Jack Diederich wrote: > I've snipped all the bits I understand and/or agree with. I'm only quoting what I think needs clarification and/or hasn't already been addressed. Note you're responding to an old thread; Talin posted a new version of the PEP and I checked it in: http://www.python.org/dev/peps/pep-3115/ > I don't understand the keyword args other than 'metaclass.' > If class D inherits from class C does it also get passed the 'Implements' > keyword? No, it's up to the metaclass to define what happens. It has enough info to tell which interfaces C implements, so if it wants to it can easily make D implement them too without being explicitly told so. > I like the more explicit > > class C(metaclass=Implements(I1, I2)): > ... You can do it either way. The PEP however won't force people to do it this way if they like the other way. > Agreed, I currently use metaclasses that pull behavior flags from the > class body and it is less than pretty. In my experience when you have > those behavior flags in the class they change for every subclass too. > If they didn't they would be defined in the original metaclass. I think > of metaclasses as 'apply always' and class decorators as 'apply once.' But that's really between the metaclass and the decorator; the language doesn't constrain you. (I feel I'm repeating myself; we're so deep into meta-land that we can only provide mechanism, not policy on how to use it.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 14 19:58:32 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 11:58:32 -0700 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: Great! Mind writing up writing up a PEP that summarizes the discussion (a bit)? In particular it should explain (a) why we need octal literals; (b) why leading-zero is bad; (c) why we don't need general bases; (d) why 0t is the best choice. Oh, and please add 0b too; there's no formal proposal for that yet. Thanks! --Guido On 3/14/07, Patrick Maupin wrote: > On 3/14/07, Oleg Broytmann wrote: > > > > 0b101010 > > 0c660 > > 0xffe > > > > I.e. the first letter from "bin", the second from "oct", the third from > > "hex". Also "0c" resembles "oc" from "oct". > > -1 on "c" It's too visually close to "0" in some fonts. > > +1 on "t" > > "t" does not appear in 'binary' or 'hexadecimal' > "x" does not appear in 'binary' or 'octal' > "b" does not appear in 'octal' or 'hexadecimal' > > And finally "c" means "character" in %s or PEP3101, and "t" is not yet > defined as a type specifier. > > So just to couch it all in terms of a proposal: > > - In 2.6 and 3.0, we add 0t1234 as a valid octal number > - In 2.6, we issue a deprecation warning for a leading literal 0 which > is followed immediately by another digit. > - In 3.0, that becomes an exception > - If people really are still using octal that much, we should also > consider adding it in to PEP3101. > > Regards, > Pat > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Wed Mar 14 20:07:34 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 14:07:34 -0500 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: Sure. I'll do that tonight or tomorrow. It would be great to get my feet wet on the process on a relatively simple PEP. One other question, first though -- not that I want to open a huge can of worms or anything, but if we are trying to make things nice and consistent, how about: x = int("0x500") I know I can do int("500", 16) (and I think we want to keep that for sure), but for the cases of binary, octal, and hexadecimal which we have decided are special and useful, should the standard integer constructor also take these strings? Thanks, Pat On 3/14/07, Guido van Rossum wrote: > Great! Mind writing up writing up a PEP that summarizes the discussion > (a bit)? In particular it should explain (a) why we need octal > literals; (b) why leading-zero is bad; (c) why we don't need general > bases; (d) why 0t is the best choice. Oh, and please add 0b too; > there's no formal proposal for that yet. Thanks! > > --Guido > > On 3/14/07, Patrick Maupin wrote: > > On 3/14/07, Oleg Broytmann wrote: > > > > > > 0b101010 > > > 0c660 > > > 0xffe > > > > > > I.e. the first letter from "bin", the second from "oct", the third from > > > "hex". Also "0c" resembles "oc" from "oct". > > > > -1 on "c" It's too visually close to "0" in some fonts. > > > > +1 on "t" > > > > "t" does not appear in 'binary' or 'hexadecimal' > > "x" does not appear in 'binary' or 'octal' > > "b" does not appear in 'octal' or 'hexadecimal' > > > > And finally "c" means "character" in %s or PEP3101, and "t" is not yet > > defined as a type specifier. > > > > So just to couch it all in terms of a proposal: > > > > - In 2.6 and 3.0, we add 0t1234 as a valid octal number > > - In 2.6, we issue a deprecation warning for a leading literal 0 which > > is followed immediately by another digit. > > - In 3.0, that becomes an exception > > - If people really are still using octal that much, we should also > > consider adding it in to PEP3101. > > > > Regards, > > Pat > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From python at rcn.com Wed Mar 14 20:18:03 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 14 Mar 2007 15:18:03 -0400 (EDT) Subject: [Python-3000] Fwd: Re: Octal Message-ID: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> [Patrick Maupin]> >So just to couch it all in terms of a proposal: > >- In 2.6 and 3.0, we add 0t1234 as a valid octal number >- In 2.6, we issue a deprecation warning for a leading literal 0 which >is followed immediately by another digit. >- In 3.0, that becomes an exception My suggestion: - In 2.6 and 3.0, add 0t1234 as a valid octal number - In 2.6, use the special 3.0 warning gizmo that is off by default and only turned-on with a command-line switch. A normal deprecation warning is unwarranted and irritating; besides, we do intend leave the existing notation intact in the 2.x series. - In 3.0, we don't want an exception. With floats and decimals, leading zeroes are allowed (i.e. 0.123). In 3.0, I would like the leading zero distinction to disappear from our collective memory. Somelike like eval('00987') should give 987, not an exception. The use cases for zfill() correspond to the cases where leading zeros are meaningfully interpreted as decimals (this includes phone and social security numbers, account numbers, and whatnot). - Enhance the 2-to-3 conversion tool to move 0123 literals to 0t123. Raymond From guido at python.org Wed Mar 14 20:22:46 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 12:22:46 -0700 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: On 3/14/07, Patrick Maupin wrote: > Sure. I'll do that tonight or tomorrow. > > It would be great to get my feet wet on the process on a relatively > simple PEP. One other question, first though -- not that I want to > open a huge can of worms or anything, but if we are trying to make > things nice and consistent, how about: > > x = int("0x500") > > I know I can do int("500", 16) (and I think we want to keep that for > sure), but for the cases of binary, octal, and hexadecimal which we > have decided are special and useful, should the standard integer > constructor also take these strings? You can do this with x = int("0x500", 0) > Thanks, > Pat > > > On 3/14/07, Guido van Rossum wrote: > > Great! Mind writing up writing up a PEP that summarizes the discussion > > (a bit)? In particular it should explain (a) why we need octal > > literals; (b) why leading-zero is bad; (c) why we don't need general > > bases; (d) why 0t is the best choice. Oh, and please add 0b too; > > there's no formal proposal for that yet. Thanks! > > > > --Guido > > > > On 3/14/07, Patrick Maupin wrote: > > > On 3/14/07, Oleg Broytmann wrote: > > > > > > > > 0b101010 > > > > 0c660 > > > > 0xffe > > > > > > > > I.e. the first letter from "bin", the second from "oct", the third from > > > > "hex". Also "0c" resembles "oc" from "oct". > > > > > > -1 on "c" It's too visually close to "0" in some fonts. > > > > > > +1 on "t" > > > > > > "t" does not appear in 'binary' or 'hexadecimal' > > > "x" does not appear in 'binary' or 'octal' > > > "b" does not appear in 'octal' or 'hexadecimal' > > > > > > And finally "c" means "character" in %s or PEP3101, and "t" is not yet > > > defined as a type specifier. > > > > > > So just to couch it all in terms of a proposal: > > > > > > - In 2.6 and 3.0, we add 0t1234 as a valid octal number > > > - In 2.6, we issue a deprecation warning for a leading literal 0 which > > > is followed immediately by another digit. > > > - In 3.0, that becomes an exception > > > - If people really are still using octal that much, we should also > > > consider adding it in to PEP3101. > > > > > > Regards, > > > Pat > > > _______________________________________________ > > > Python-3000 mailing list > > > Python-3000 at python.org > > > http://mail.python.org/mailman/listinfo/python-3000 > > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Wed Mar 14 21:07:58 2007 From: talin at acm.org (Talin) Date: Wed, 14 Mar 2007 13:07:58 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: References: <45F7A445.5090507@acm.org> Message-ID: <45F8561E.6070509@acm.org> Guido van Rossum wrote: > On 3/14/07, Talin wrote: >> PEP: xxx >> Title: Metaclasses in Python 3000 > > Checked in as PEP 3115. I fixed the two typos that were noted so far > (missing comma and inconsistency in name of first arg to __prepare__; > I renamed the latter metacls) and cleaned up one case of extra > whitespace (.append( key ) -> .append(key)). Now on to the content: > >> def prepare_class(name, *bases, metaclass=type, **kwargs): >> prepare = getattr(metaclass, '__prepare__', None) >> if prepare is not None: >> return prepare(name, bases, **kwargs) >> else: >> return dict() > > This is missing the extraction of the metaclass default from the > bases. It's probably okay to default metaclass to None and add this > code to the body: > > if metaclass is None: > metaclass = compute_default_metaclass(bases) > >> type. It does not need to implement the full dictionary interface; >> only the ability to insert items and retrieve them are >> required. (Note: double check that this is true). > > As was mentioned, getitem, setitem and delitem are used. No other APIs > are, unless the dict is accessed via locals(), or unless dir() is > used. IMO it's up to the prepare function to provide the features that > it wants to support; if it doesn't want to support deletions, that is > between the metaclass and the users; the language spec doesn't have to > say anything about this. > >> # The custom dictionary >> class member_table(dict): >> def __init__(self): > > Despite this just being an example, I'd like for the > member_table class to be defined outside the OrderedClass class. it > also probably ought to have a conforming name, i.e. MemberTable. > > >> Another good suggestion was to simply use an ordered dict for all >> classes, and skip the whole 'custom dict' mechanism. This was based >> on the observation that most use cases for a custom dict were for >> the purposes of preserving order information. However, this idea has >> two drawbacks, first because it means that an ordered dict >> implementation would have to be added to the set of built-in types >> in Python, and second because it would impose a slight speed (and >> complexity) penalty on all class declarations. > > I think this rebuttal isn't strong enough; I'm sure there *will* be > use cases where a custom prepare method can solve a problem that a > standard ordered dict couldn't. This all looks good to me - I think I still have permission to check in changes to the PEPs dir, want me to go ahead and make the changes directly? -- Talin From pmaupin at gmail.com Wed Mar 14 21:12:43 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 15:12:43 -0500 Subject: [Python-3000] Fwd: Re: Octal In-Reply-To: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> References: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> Message-ID: On 3/14/07, Raymond Hettinger wrote: > > - In 3.0, we don't want an exception. With floats and decimals, leading zeroes are allowed (i.e. 0.123). In 3.0, I would like the leading zero distinction to disappear from our collective memory. Somelike like eval('00987') should give 987, not an exception. The use cases for zfill() correspond to the cases where leading zeros are meaningfully interpreted as decimals (this includes phone and social security numbers, account numbers, and whatnot). This is the only part I'm not sure I agree with. I agree it would be nice if there were no collective memory, but it is etched into the collective memory, and anybody who uses C or some other language as well as Python is living that collective memory on a daily basis and could easily do the wrong thing (either in Python or in the other language). We are removing the ability to specify an octal number via a leading 0, because it caused confusion. I think adding the ability to specify a decimal number with a leading 0 has the potential to cause confusion in a completely different group of users. And it doesn't seem to be something people are clamoring for in a major way. In fact, I would go so far as to argue that the behavior of int() should be changed as well: int('012') -> exception (used to return 12) int('012', 0) -> exception (used to return 10) int('012', 10) -> should return 12, like always So for your use-cases: 1) you shouldn't hard code an SSN into the program; 2) you shouldn't use eval() on user data; and 3) you should call int() with a second parameter of 10 to get what you want. Regards, Pat From pje at telecommunity.com Wed Mar 14 21:23:22 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 14 Mar 2007 15:23:22 -0500 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: References: <45F7A445.5090507@acm.org> <45F7A445.5090507@acm.org> Message-ID: <5.1.1.6.0.20070314151901.048a2d10@sparrow.telecommunity.com> At 11:15 AM 3/14/2007 -0700, Guido van Rossum wrote: >I think this rebuttal isn't strong enough; I'm sure there *will* be >use cases where a custom prepare method can solve a problem that a >standard ordered dict couldn't. For example, a custom prepare could provide access to special method names to do things in the body, ala Ruby on Rails' ActiveRecord DSL. e.g.: class SomeRecord(Record, table_name="foo") x = many(OtherRecord) Where 'many' is actually an object returned by the special dictionary's __getitem__. In addition, it might be the case that "OtherRecord" is a class that doesn't exist yet, and the special dictionary returns a placeholder object that will serve until a record type with that name is defined. This use case doesn't even *need* ordering, but it does need __prepare__. From martin at v.loewis.de Wed Mar 14 21:27:15 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 14 Mar 2007 21:27:15 +0100 Subject: [Python-3000] Octal In-Reply-To: <7BE92BBC-968D-4D86-83ED-6A5CB48559AA@python.org> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <7BE92BBC-968D-4D86-83ED-6A5CB48559AA@python.org> Message-ID: <45F85AA3.1050004@v.loewis.de> > Can we wait until Unix file permissions are expressed in something > other than octal? The time machine at work: machine:~/work/25 loewis$ ls -l x.py -rw-r--r-- 1 loewis admin 22 Aug 25 2006 x.py machine:~/work/25 loewis$ python Python 2.3.5 (#1, Aug 19 2006, 21:31:42) [GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> from stat import * >>> os.chmod("x.py", S_IRUSR|S_IRGRP|S_IWUSR|S_IWGRP) >>> machine:~/work/25 loewis$ ls -l x.py -rw-rw---- 1 loewis admin 22 Aug 25 2006 x.py Not as brief as 0660, but more readable. Regards, Martin From thomas at python.org Wed Mar 14 21:48:58 2007 From: thomas at python.org (Thomas Wouters) Date: Wed, 14 Mar 2007 21:48:58 +0100 Subject: [Python-3000] Fwd: Re: Octal In-Reply-To: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> References: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> Message-ID: <9e804ac0703141348yc97eb26uc29490542589c916@mail.gmail.com> On 3/14/07, Raymond Hettinger wrote: > - In 3.0, we don't want an exception. Eh, no, you might not want one, but I most assuredly do want an exception. Having formerly-octal literals suddenly give wrong results would be much more of a stumbling block than having them in the first place, especially considering we forgot to change all the other languages out there. An exception can make the difference between '0t60' and '60' clear in a single message, not to mention refuse the temptation to guess. Maybe, *maybe*, after Python is in general use for a couple of years we could remove that exception (although I would keep it a warning, myself.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070314/6c5400bc/attachment.htm From steven.bethard at gmail.com Wed Mar 14 21:56:32 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 14 Mar 2007 14:56:32 -0600 Subject: [Python-3000] Fwd: Re: Octal In-Reply-To: <9e804ac0703141348yc97eb26uc29490542589c916@mail.gmail.com> References: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> <9e804ac0703141348yc97eb26uc29490542589c916@mail.gmail.com> Message-ID: On 3/14/07, Thomas Wouters wrote: > > > On 3/14/07, Raymond Hettinger wrote: > > - In 3.0, we don't want an exception. > > Eh, no, you might not want one, but I most assuredly do want an exception. > Having formerly-octal literals suddenly give wrong results would be much > more of a stumbling block than having them in the first place, especially > considering we forgot to change all the other languages out there. An > exception can make the difference between '0t60' and '60' clear in a single > message, not to mention refuse the temptation to guess. Sorry, but could you explain why having the -py3k flag raise the exception for your 2.X code wouldn't be sufficient? Is it because you expect your fingers will continue to type 0660 instead of 0t660 when you're writing Python 3000 code? STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From thomas at python.org Wed Mar 14 21:57:06 2007 From: thomas at python.org (Thomas Wouters) Date: Wed, 14 Mar 2007 21:57:06 +0100 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: <9e804ac0703141357h14801e08o503f0159086e3dba@mail.gmail.com> When you're done with the PEP, here's the code: http://python.org/sf/1681002 On 3/14/07, Patrick Maupin wrote: > > Sure. I'll do that tonight or tomorrow. > > It would be great to get my feet wet on the process on a relatively > simple PEP. One other question, first though -- not that I want to > open a huge can of worms or anything, but if we are trying to make > things nice and consistent, how about: > > x = int("0x500") > > I know I can do int("500", 16) (and I think we want to keep that for > sure), but for the cases of binary, octal, and hexadecimal which we > have decided are special and useful, should the standard integer > constructor also take these strings? > > Thanks, > Pat > > > On 3/14/07, Guido van Rossum wrote: > > Great! Mind writing up writing up a PEP that summarizes the discussion > > (a bit)? In particular it should explain (a) why we need octal > > literals; (b) why leading-zero is bad; (c) why we don't need general > > bases; (d) why 0t is the best choice. Oh, and please add 0b too; > > there's no formal proposal for that yet. Thanks! > > > > --Guido > > > > On 3/14/07, Patrick Maupin wrote: > > > On 3/14/07, Oleg Broytmann wrote: > > > > > > > > 0b101010 > > > > 0c660 > > > > 0xffe > > > > > > > > I.e. the first letter from "bin", the second from "oct", the > third from > > > > "hex". Also "0c" resembles "oc" from "oct". > > > > > > -1 on "c" It's too visually close to "0" in some fonts. > > > > > > +1 on "t" > > > > > > "t" does not appear in 'binary' or 'hexadecimal' > > > "x" does not appear in 'binary' or 'octal' > > > "b" does not appear in 'octal' or 'hexadecimal' > > > > > > And finally "c" means "character" in %s or PEP3101, and "t" is not yet > > > defined as a type specifier. > > > > > > So just to couch it all in terms of a proposal: > > > > > > - In 2.6 and 3.0, we add 0t1234 as a valid octal number > > > - In 2.6, we issue a deprecation warning for a leading literal 0 which > > > is followed immediately by another digit. > > > - In 3.0, that becomes an exception > > > - If people really are still using octal that much, we should also > > > consider adding it in to PEP3101. > > > > > > Regards, > > > Pat > > > _______________________________________________ > > > Python-3000 mailing list > > > Python-3000 at python.org > > > http://mail.python.org/mailman/listinfo/python-3000 > > > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/thomas%40python.org > -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070314/1572b95d/attachment.html From thomas at python.org Wed Mar 14 21:59:36 2007 From: thomas at python.org (Thomas Wouters) Date: Wed, 14 Mar 2007 21:59:36 +0100 Subject: [Python-3000] Fwd: Re: Octal In-Reply-To: References: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> <9e804ac0703141348yc97eb26uc29490542589c916@mail.gmail.com> Message-ID: <9e804ac0703141359n38ce4689t7517e2ad19ae3cf1@mail.gmail.com> On 3/14/07, Steven Bethard wrote: > > On 3/14/07, Thomas Wouters wrote: > > > > > > On 3/14/07, Raymond Hettinger wrote: > > > - In 3.0, we don't want an exception. > > > > Eh, no, you might not want one, but I most assuredly do want an > exception. > > Having formerly-octal literals suddenly give wrong results would be much > > more of a stumbling block than having them in the first place, > especially > > considering we forgot to change all the other languages out there. An > > exception can make the difference between '0t60' and '60' clear in a > single > > message, not to mention refuse the temptation to guess. > > Sorry, but could you explain why having the -py3k flag raise the > exception for your 2.X code wouldn't be sufficient? Is it because you > expect your fingers will continue to type 0660 instead of 0t660 when > you're writing Python 3000 code? Not just me. The world. This isn't a "re-educate people used to a wart in Python 2.x" kind of thing. This is a "re-educate new programmers coming from other languages" kind of thing. The stuff we warn about with -Wpy3k in Python 2.6 is stuff that is a change in how Python 3.0 does things compared to 2.x. This isn't just a change compared to 2.6, this is a change compared to quite a lot of popular programming languages out there. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070314/471d7af0/attachment.htm From python at rcn.com Wed Mar 14 22:04:58 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 14 Mar 2007 17:04:58 -0400 (EDT) Subject: [Python-3000] Fwd: Re: Fwd: Re: Octal Message-ID: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> >> - In 3.0, we don't want an exception. With floats and decimals, >> leading zeroes are allowed (i.e. 0.123). In 3.0, I would like the >> leading zero distinction to disappear from our collective memory. >> Somelike like eval('00987') should give 987, not an exception. >> The use cases for zfill() correspond to the cases where leading zeroes >> are meaningfully interpreted as decimals (this includes phone and >> social security numbers, account numbers, and whatnot). [Pat] >This is the only part I'm not sure I agree with. Leading zeroes are not a syntax error. Excel supports them for both input and output formats because accountants (like me) ocassionally need them. Schoolkids have an expectation that leading zeroes do not change the value. Your pocket calculator accepts leading zeroes. I do not support adding yet another SyntaxError to the language -- that will not improve its ease of use. With decimal and float literals supporting lead zeroes, it would be a wart to introduce a syntax error for ints. >In fact, I would go so far as to argue that the behavior of int() >should be changed as well: > > int('012') -> exception (used to return 12) Gratuitous breakage. Major inconvenience when parsing zfilled inputs (which are ubiquitous). hour, minutes, seconds = map(int, '08:30:00'.split(':')) Raymond From pmaupin at gmail.com Wed Mar 14 22:19:39 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 16:19:39 -0500 Subject: [Python-3000] Fwd: Re: Octal In-Reply-To: <9e804ac0703141359n38ce4689t7517e2ad19ae3cf1@mail.gmail.com> References: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> <9e804ac0703141348yc97eb26uc29490542589c916@mail.gmail.com> <9e804ac0703141359n38ce4689t7517e2ad19ae3cf1@mail.gmail.com> Message-ID: On 3/14/07, Thomas Wouters wrote: > On 3/14/07, Steven Bethard wrote: > > Sorry, but could you explain why having the -py3k flag raise the > > exception for your 2.X code wouldn't be sufficient? Is it because you > > expect your fingers will continue to type 0660 instead of 0t660 when > > you're writing Python 3000 code? > > Not just me. The world. This isn't a "re-educate people used to a wart in > Python 2.x" kind of thing. This is a "re-educate new programmers coming from > other languages" kind of thing. The stuff we warn about with -Wpy3k in > Python 2.6 is stuff that is a change in how Python 3.0 does things compared > to 2.x. This isn't just a change compared to 2.6, this is a change compared > to quite a lot of popular programming languages out there. I absolutely agree with Thomas about this problem and this issue, but I would like to additionally address the idea of "fixing" things with a -py3k flag. Many (most) Python programmers will not migrate all of their stuff to 3.0 overnight. They will be working on 2.x and 3.x stuff for years, so the idea of "fix the code once and then forget about it" does not address the fact that a human working in two different environments will invariably screw up in one or the other occasionally. I understand that the floodgates are open on 3.0 and all sorts of stuff is changing and being added and deleted, but (except for things that are irretrievably broken), changes should not be made in such a fashion that code will silently do different things on 2.6 and 3.0. (Guido has explicitly addressed this in policy about the C API.) Now, lots of things might be considered "irretrievably broken" by those with deep experience and knowledge in the matter, but (with the possible exception of Raymond) I have not seen anybody argue that "the refusal of Python to parse a literal string of multiple digits with a leading 0 as a decimal number is irretrievably broken." The point that Raymond has made that I (and several others) can agree with is "Python's parsing of a literal string of multiple digits with a leading 0 as an octal number can be confusing to someone who expected a decimal number." But this latter problem can be fixed without silently doing something different in 2.6 and 3.0, namely by refusing to guess whether the number should be octal or decimal. Regards, pat From bjourne at gmail.com Wed Mar 14 22:20:11 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 14 Mar 2007 22:20:11 +0100 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws Message-ID: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> I have a question about PEP 3001: """The last and most disruptive change is the overhaul of a module's public interface. If a module's interface is to be changed, a justification should be made beforehand, or a PEP should be written. The change must be fully documented as "New in Version 3.0", and the python3warn.py script must know about it.""" It seems like its wording makes it very hard to fix broken or bad stuff in old modules. Many modules are very old and contain lots of legacy stuff that is kept to preserve backwards compatibility. Other modules exposes a non-optimal (or even bad) API because it must not be changed. So I wonder what is the policy for fixing mistakes in the API design? Is a PEP really needed? For example, here are some bugs in the threading module: activeCount() # Redundant, use len(threading.enumerate()) currentThread() # PEP8 -> current_thread() class local # PEP8 -> class Local Is there anything that can be done about it? For another example, take the Telnet class in the telnetlib module. It has a method set_option_negotiation_callback() which takes a function that will be called for each telnet option read. The default behaviour for the Telnet class is to refuse all negotiation requests, but using a negotiation callback you can override that. However, using a callback does not work so well because the function acting on the telnet options read still needs to reference the Telnet class to get hold of negotiation data using read_sb_data(). The problem is non-lethal but a small annoyance to advanced Telnet users. See SourceForge patches #1520081, #664020 and #1678077. The right design would have been to have a method (handle_option) in the class that handles all options and, by default, refuses them. Users of the Telnet class could then subclass Telnet and override the handle_option method to implement their application specific option handling. Can or should API bugs like these be fixed for Python 3.0? -- mvh Bj?rn From pmaupin at gmail.com Wed Mar 14 22:24:31 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 16:24:31 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Octal In-Reply-To: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> References: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> Message-ID: > Leading zeroes are not a syntax error. Excel supports them for both input and output formats because accountants (like me) ocassionally need them. Schoolkids have an expectation that leading zeroes do not change the value. Your pocket calculator accepts leading zeroes. I do not support adding yet another SyntaxError to the language -- that will not improve its ease of use. With decimal and float literals supporting lead zeroes, it would be a wart to introduce a syntax error for ints. I agree that, in a perfect world, these numbers would be considered decimal. But when it comes to backwards compatibility, it is interesting you mention Excel -- there seems to be another thread going on about its CSV parsing :) > >In fact, I would go so far as to argue that the behavior of int() > >should be changed as well: > > > > int('012') -> exception (used to return 12) > > Gratuitous breakage. > Major inconvenience when parsing zfilled inputs (which are ubiquitous). > > hour, minutes, seconds = map(int, '08:30:00'.split(':')) I can agree that should stay, since it has been there all along giving the "right" answer, but I particularly don't want int('012', 0) to return 12 and not throw an exception, since it has been giving the "wrong" answer all along. Pat From python at rcn.com Wed Mar 14 23:41:47 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 14 Mar 2007 18:41:47 -0400 (EDT) Subject: [Python-3000] Py3.0 impacts on Py2.6 Message-ID: <20070314184147.BBL48652@ms09.lnh.mail.rcn.net> >> I would like to additionally address the idea of "fixing" things >> with a -py3k flag. Many (most) Python programmers will not migrate >> all of their stuff to 3.0 overnight . . . After hearing Guido's talk at PyCon, ISTM that Py3.0 will be accepted widely and gracefully on its own merits. However, I sense in this forum a strong inclination to hedge bets by monkeying with Py2.6 in ways that go beyond the -py3k flag or backporting compatible new features. It is important to not delude ourselves about the role of Py2.6. We don't get to decide that -- the users do. We all know, it is a big deal for a shop to migrate from 2.5 to 2.6 (reloading/rebuilding third-party and homegrown modules/extension, suppressing/addressing warnings, code updates, etc). Accordingly, users will NOT upgrade unless Py2.6 is BETTER than Py2.5. In our shop, if Py2.6 is slower or is just a pile of deprecations and warnings, then we will likely limit the upgrade to just a single person who can use it as tool to transition our code and third-party tools to Py3.0 when the time comes. The rest of the shop will go straight from Py2.5 to Py3.0 unless Py2.6 shows some compelling advantage over what we have now. If Py2.6 is to be accepted, it needs improvements, not second-class status as a stepping stone. It should be the best of the 2.x series, not a dumping ground for transitional stuff; otherwise, why bother with it at all? Raymond From guido at python.org Thu Mar 15 00:08:03 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 16:08:03 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Octal In-Reply-To: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> References: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> Message-ID: int('0012') should still return 12, no doubt about it, since the default base is 10 and this is currently supported. Some business users need to easily parse numbers output by Cobol. So should int('0012', 10), which is really the same case. For int('0012', 0), (the auto-base option) I'm not sure, as this may well be used to parse e.g. command-line options by folks who don't know or care about Python syntax. Of course, it ought to be sufficient to handle this in optparse; I'm sure a simple regexp can distinguish between all the different cases. (Of course the auto-base option should support 0x, 0t and 0b prefixes; it's just the combination of auto-base with decimal leading zeros that might be confusing.) I agree with Thomas that allowing the literal 0012 in a program to return the value 10 would trip up old-timers too easily. I don't expect Cobol code to be producing a lot of Python souce code, and it's currently not supported (well, it is, but with a different meaning) so I think it's OK to say that leading zeros are out (except for 0 itself, and 0x/0b/0t). IOW, the only thing I'm unsure about is whether int('0012', 0) should return 10 or complain like eval('0012') would. Perhaps it's best to say that int('0012', 0) is a safer way to parse Python int literals, and that would make it an error. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Mar 15 00:13:45 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 12:13:45 +1300 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> References: <45F7A445.5090507@acm.org> <45F7A445.5090507@acm.org> <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> Message-ID: <45F881A9.8050205@canterbury.ac.nz> Phillip J. Eby wrote: > So the subset required is determined entirely by what the class body > (and metaclass __new__) require. Perhaps the PEP should say something to the effect that get, set and delete need to be implemented in order to support all the things the class body might want to do, but a metaclass can omit them if it wants to restrict what can be done in the class body. > So the metaclass' __new__ > is going to need to read out the contents of the pseudo-dict somehow, or > generate alternative contents. Maybe the PEP should specify a protocol that the pseudo-dict must provide for doing this, such as an iteritems() method (or whatever the Py3k equivalent is). -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 15 00:14:47 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 12:14:47 +1300 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> Message-ID: <45F881E7.3060308@canterbury.ac.nz> Guido van Rossum wrote: > How about 0t660? > This is most symmetric with 0xffe for hex, as it is the 3rd letter of > the word in both cases. I like it. > I think we also approved 0b101010 for binary numbers, didn't we? Although for consistency with 0x an 0t it would have to be 0n, wouldn't it? :-) -- Greg From talin at acm.org Thu Mar 15 00:12:31 2007 From: talin at acm.org (Talin) Date: Wed, 14 Mar 2007 16:12:31 -0700 Subject: [Python-3000] Py3.0 impacts on Py2.6 In-Reply-To: <20070314184147.BBL48652@ms09.lnh.mail.rcn.net> References: <20070314184147.BBL48652@ms09.lnh.mail.rcn.net> Message-ID: <45F8815F.7010901@acm.org> Raymond Hettinger wrote: >>> I would like to additionally address the idea of "fixing" things >>> with a -py3k flag. Many (most) Python programmers will not migrate >>> all of their stuff to 3.0 overnight . . . > > After hearing Guido's talk at PyCon, ISTM that Py3.0 will be accepted > widely and gracefully on its own merits. However, I sense in this forum > a strong inclination to hedge bets by monkeying with Py2.6 in ways > that go beyond the -py3k flag or backporting compatible new features. > > It is important to not delude ourselves about the role of Py2.6. > We don't get to decide that -- the users do. We all know, it is a > big deal for a shop to migrate from 2.5 to 2.6 (reloading/rebuilding > third-party and homegrown modules/extension, suppressing/addressing > warnings, code updates, etc). Accordingly, users will NOT upgrade > unless Py2.6 is BETTER than Py2.5. > > In our shop, if Py2.6 is slower or is just a pile of deprecations > and warnings, then we will likely limit the upgrade to just a single > person who can use it as tool to transition our code and third-party > tools to Py3.0 when the time comes. The rest of the shop will go > straight from Py2.5 to Py3.0 unless Py2.6 shows some compelling > advantage over what we have now. If Py2.6 is to be accepted, it > needs improvements, not second-class status as a stepping stone. > It should be the best of the 2.x series, not a dumping ground for > transitional stuff; otherwise, why bother with it at all? I think you make good points; However I think that the backporting of Python 3000 features may be sufficient to give Python 2.X users a compelling reason to upgrade to 2.6. Here's the way I see it: The 'official' reason for the existence of Python 3000 is to allow us to go back in time and revisit some early design decisions, which cannot be changed in 2.X because they would cause backwards incompatibility. However, many of the 3000-series PEPs are not of this nature at all (including all of mine.) Instead, they are PEPs that propose dramatic new features, which have little or no backwards-compatibility impact. I think that part of the reason for this is that all of us who live in the software world have certain expectations about what happens when a major release number gets bumped. For any given piece of software - especially in the commercial world - when a major version number bumps from "2" to "3" we expect a whole boat load of new and interesting functionality. We as customers would be disappointed if Photoshop 7 was simply Photoshop 6 + bug fixes. (Although perhaps it would be a better world if we didn't - but that's another topic.) The fact that we've used the name "3000" instead of "3" further adds to the "marketing hype" that feeds that expectation, that Python 3000 is going to be a "brand new, radically better" Python. I think that there's also a perception in the development community that the Python 3 series is "more open" to radical feature suggestions. The result of all this is that a lot of the creative energy that would have gone into Python 2.6 has instead been drawn off to Python 3. Most of the PEPs in the 3000 series could have just as easily been 2.X PEPs. Of the PEPs that propose new features (as opposed to removing old ones) there's only a few cases, I think, where a PEP is 'enabled' by the freedom to break backwards compatibility. What this means to me is that many of these PEPs could in fact be backported to 2.6, if it was felt useful enough to do so. Which in turn says to me that the compelling marketing tag line for 2.6 is "most of the features of Python 3000, but still compatible." I should also say that I have a slightly different view of the relationship between the 2.X series and the 3.0 series. Part of the attraction of 3.0 is the opportunity to make a "clean break", unhampered by the mistakes of the past. I think that it's good for language designers sometimes to be able to stretch out and not be constrained too much by the past. But I don't think that, in the end, there will actually be a "break". What I mean is, I think that eventually there will be a convergence between the 2.X and 3.0 series. Each subsequent 2.X release will be more compatible with 3.0 than the previous ones. The reason for this has to do with the way that innovation in the 2.X series is going to happen from now on: I very much doubt that people are going to be innovating new features in 2.X that aren't also going to be ported to 3.0; Which means that every change to 2.X is going to bring it closer to 3.0 in some way, and there won't be any changes that move 2.X farther away from 3.0. So as long as 2.X keeps changing and is not entirely frozen, it will gradually become more like 3.0. The result is, I think, that eventually there will be a smooth transition path between the two major versions. -- Talin From greg.ewing at canterbury.ac.nz Thu Mar 15 00:17:55 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 12:17:55 +1300 Subject: [Python-3000] Octal In-Reply-To: <20070314162656.GA11074@phd.pp.ru> References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: <45F882A3.1080309@canterbury.ac.nz> Oleg Broytmann wrote: > 0b101010 > 0c660 > 0xffe > > I.e. the first letter from "bin", the second from "oct", the third from > "hex". Also "0c" resembles "oc" from "oct". 0c bothers me a bit, because it looks like it might have something to do with characters. What about 0k (pronounced "oktal") ? -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 15 00:31:40 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 12:31:40 +1300 Subject: [Python-3000] __special__ attrs looked up on the type, not instance In-Reply-To: References: <45F7D8F8.9090100@gmail.com> Message-ID: <45F885DC.60008@canterbury.ac.nz> Guido van Rossum wrote: > We could even do this by hacking the default getattr > implementation to skip the instance dict if the name starts and ends > with two underscores. But unless I'm mistaken, this behaviour is only appropriate for *methods*, and you can't tell just from whether the name has double underscores whether it's a method or not. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 15 00:39:10 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 12:39:10 +1300 Subject: [Python-3000] Octal In-Reply-To: References: <20070313225348.BBI60895@ms09.lnh.mail.rcn.net> <9e804ac0703140259q78a0a6c4qb4e790cd0b92ce46@mail.gmail.com> <20070314162656.GA11074@phd.pp.ru> Message-ID: <45F8879E.50306@canterbury.ac.nz> Patrick Maupin wrote: > how about: > > x = int("0x500") I don't think that should be done by default. The string passed to int often comes from user input, and typical users aren't familiar with bases other than 10, so it's more likely to be a typo if the string contains non-digits. Maybe there should be an option for it, though, such as x = int("0x500", anybase = True) or perhaps a different function altogether. -- Greg From greg.ewing at canterbury.ac.nz Thu Mar 15 00:44:33 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 12:44:33 +1300 Subject: [Python-3000] Fwd: Re: Octal In-Reply-To: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> References: <20070314151803.BBK89916@ms09.lnh.mail.rcn.net> Message-ID: <45F888E1.7060508@canterbury.ac.nz> Raymond Hettinger wrote: > The use cases for > zfill() correspond to the cases where leading zeros are meaningfully > interpreted as decimals (this includes phone and social security numbers, > account numbers, and whatnot). They're NOT numbers! They're STRINGS! Leading zeroes would only be disallowed in *python source code*. Things like int("00123") would still work and be decimal. -- Greg From thomas at python.org Thu Mar 15 01:02:18 2007 From: thomas at python.org (Thomas Wouters) Date: Thu, 15 Mar 2007 01:02:18 +0100 Subject: [Python-3000] __special__ attrs looked up on the type, not instance In-Reply-To: <45F885DC.60008@canterbury.ac.nz> References: <45F7D8F8.9090100@gmail.com> <45F885DC.60008@canterbury.ac.nz> Message-ID: <9e804ac0703141702h5c7d13e4x6c717f164bc6d05b@mail.gmail.com> On 3/15/07, Greg Ewing wrote: > > Guido van Rossum wrote: > > We could even do this by hacking the default getattr > > implementation to skip the instance dict if the name starts and ends > > with two underscores. > > But unless I'm mistaken, this behaviour is only > appropriate for *methods*, and you can't tell > just from whether the name has double underscores > whether it's a method or not. We can make __*__ methods be a different kind of method (by making the metaclass wrap them up in a different kind of descriptor, one with __set__ set. Python treats descriptors with a __set__ slightly different than those without __set__: it bypasses __dict__ entirely, for new-style classes. This trick doesn't work for classic classes.) For instance: import types class SpecialMethodDescr(object): def __init__(self, func): self.func = func def __get__(self, inst, cls=None): # For backward compatibility, insert inst.__dict__ checks and warns here (and we should get our name passed in.) return self.func.__get__(inst, cls) def __set__(self, inst, value): # Possibly allow assignment here, by assigning to inst.__dict__, but warn about it not being used. raise AttributeError class SMDType(type): def __new__(cls, name, bases, attrs): for attr in attrs: if (attr.startswith("__") and attr.endswith("__") and isinstance(attrs[attr], types.FunctionType)): attrs[attr] = SpecialMethodDescr(attrs[attr]) return super(SMDType, cls).__new__(cls, name, bases, attrs) class smd_object(object): __metaclass__ = SMDType And to see it in action: >>> class Old(object): ... def __repr__(self): ... return "Old.__repr__" ... >>> class New(smd_object): ... def __repr__(self): ... return "New.__repr__" ... >>> o = Old() >>> n = New() >>> def o_repr(): ... return "o_repr" ... >>> o.__dict__['__repr__'] = o_repr >>> def n_repr(): ... return "n_repr" ... >>> n.__dict__['__repr__'] = n_repr >>> o Old.__repr__ >>> n New.__repr__ >>> o.__repr__() 'o_repr' >>> n.__repr__() 'New.__repr__' -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070315/f2cb887e/attachment.htm From guido at python.org Thu Mar 15 01:14:35 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 17:14:35 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F8561E.6070509@acm.org> References: <45F7A445.5090507@acm.org> <45F8561E.6070509@acm.org> Message-ID: While you're at it, could you also add a pointer to this patch as a sample implementation? http://python.org/sf/1681101 It's far from complete at this point; so far, I've got the syntax working and it seems to compile old code just fine, but the keyword args and */** notation are simply ignored. If someone wants to work on this, drop me a note to see if I haven't made progress already. --Guido On 3/14/07, Talin wrote: > Guido van Rossum wrote: > > On 3/14/07, Talin wrote: > >> PEP: xxx > >> Title: Metaclasses in Python 3000 > > > > Checked in as PEP 3115. I fixed the two typos that were noted so far > > (missing comma and inconsistency in name of first arg to __prepare__; > > I renamed the latter metacls) and cleaned up one case of extra > > whitespace (.append( key ) -> .append(key)). Now on to the content: > > > >> def prepare_class(name, *bases, metaclass=type, **kwargs): > >> prepare = getattr(metaclass, '__prepare__', None) > >> if prepare is not None: > >> return prepare(name, bases, **kwargs) > >> else: > >> return dict() > > > > This is missing the extraction of the metaclass default from the > > bases. It's probably okay to default metaclass to None and add this > > code to the body: > > > > if metaclass is None: > > metaclass = compute_default_metaclass(bases) > > > >> type. It does not need to implement the full dictionary interface; > >> only the ability to insert items and retrieve them are > >> required. (Note: double check that this is true). > > > > As was mentioned, getitem, setitem and delitem are used. No other APIs > > are, unless the dict is accessed via locals(), or unless dir() is > > used. IMO it's up to the prepare function to provide the features that > > it wants to support; if it doesn't want to support deletions, that is > > between the metaclass and the users; the language spec doesn't have to > > say anything about this. > > > >> # The custom dictionary > >> class member_table(dict): > >> def __init__(self): > > > > Despite this just being an example, I'd like for the > > member_table class to be defined outside the OrderedClass class. it > > also probably ought to have a conforming name, i.e. MemberTable. > > > > > >> Another good suggestion was to simply use an ordered dict for all > >> classes, and skip the whole 'custom dict' mechanism. This was based > >> on the observation that most use cases for a custom dict were for > >> the purposes of preserving order information. However, this idea has > >> two drawbacks, first because it means that an ordered dict > >> implementation would have to be added to the set of built-in types > >> in Python, and second because it would impose a slight speed (and > >> complexity) penalty on all class declarations. > > > > I think this rebuttal isn't strong enough; I'm sure there *will* be > > use cases where a custom prepare method can solve a problem that a > > standard ordered dict couldn't. > > This all looks good to me - I think I still have permission to check in > changes to the PEPs dir, want me to go ahead and make the changes directly? > > -- Talin > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Thu Mar 15 01:26:27 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 14 Mar 2007 20:26:27 -0400 (EDT) Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal Message-ID: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> [GvR] >I agree with Thomas that allowing the literal 0012 in a program to >return the value 10 would trip up old-timers too easily. Did you mean "return the value 12 when 10 was intended"? My worry focuses on new-timers and SyntaxErrors. The worst part of the new user experience is encountering a SyntaxError. Because of the way we generate them, invalid syntax errors are not very informative and it is not always self-evident how to fix them. I don't think we're helping the language by adding a new special rule (lead zeros are okay on floats and decimals but not ints) that results in a new source of SyntaxErrors. IMO, Py3.0 would be cleaner with 0012 being 12 and not raising a SyntaxError. Raymond From guido at python.org Thu Mar 15 01:33:59 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 16:33:59 -0800 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> Message-ID: On 3/14/07, Raymond Hettinger wrote: > [GvR] > >I agree with Thomas that allowing the literal 0012 in a program to > >return the value 10 would trip up old-timers too easily. > > Did you mean "return the value 12 when 10 was intended"? Yes, sorry. > My worry focuses on new-timers and SyntaxErrors. The worst part of the new user experience is encountering a SyntaxError. Because of the way we generate them, invalid syntax errors are not very informative and it is not always self-evident how to fix them. I don't think we're helping the language by adding a new special rule (lead zeros are okay on floats and decimals but not ints) that results in a new source of SyntaxErrors. Then let's focus on improving the error message. > IMO, Py3.0 would be cleaner with 0012 being 12 and not raising a SyntaxError. While I would agree in an ideal world, I find Thomas's argument compelling. Not all newbie Python users are first-time computer users; if they have C/C++ or other Unixoid languages under their belt (even Java has leading-zero-as-octal!) they might type 0600 and never notice it returns the wrong value until it's too late. Swapping the confusion of one group for the confusion of another group is not quite worth it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jbauer at rubic.com Thu Mar 15 01:34:38 2007 From: jbauer at rubic.com (Jeff Bauer) Date: Wed, 14 Mar 2007 19:34:38 -0500 Subject: [Python-3000] Octal Message-ID: <45F8949E.1000304@rubic.com> I really didn't want to add to the traffic on this thread, but would like to keep leading zeros from raising a SyntaxError. If this passes, it will look as archaic in retrospect as the original decision to use '/' for integer division. And I'll have to wait until 2017 for this to get rescinded in Python 4000. ;-) The coders who use octal (including myself) can adapt. There's not a lot of ambiguity when int('042') evaluates to 42. -- Jeff Bauer Rubicon, Inc. From guido at python.org Thu Mar 15 02:26:14 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 14 Mar 2007 18:26:14 -0700 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax Message-ID: The PEP proposes that the class statement accepts keyword arguments, *args, and **kwds syntax as well as positional bases. This is a bit messy to compile and execute, but we already have this, of course, in the code for calling regular functions. So I think it would be acceptable to this into a call to a new (hidden) built-in function, named __build_class__. Then that this class definition: class C(A, B, metaclass=M, other=42, *more_bases, *more_kwds): ... would translate into this: C = __build_class__(, 'C', A, B, metaclass=M, other=42, *more_bases, *more_kwds) where is a function object for the class body. (It's a slightly different function than currently; the current function *returns* the locals, while the new one *takes* the locals as an argument; instead of a LOAD_LOCALS opcode we need a STORE_LOCALS opcode.) Then __build_class__ could be roughly like this (but implemented in C): def __build_class__(func, name, *bases, metaclass=None, **kwds): if metaclass is None: metaclass = extract_metaclass(bases) # may raise an exception prepare = getattr(metaclass, "__prepare__", None) if prepare: locals = prepare(name, bases, **kwds) else: locals = {} func(locals) return metaclass(name, bases, locals, **kwds) What do folks think? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Thu Mar 15 02:33:55 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 20:33:55 -0500 Subject: [Python-3000] Octal In-Reply-To: <45F8949E.1000304@rubic.com> References: <45F8949E.1000304@rubic.com> Message-ID: On 3/14/07, Jeff Bauer wrote: > I really didn't want to add to the traffic on this > thread, but would like to keep leading zeros > from raising a SyntaxError. If this passes, it > will look as archaic in retrospect as the original > decision to use '/' for integer division. And I'll > have to wait until 2017 for this to get rescinded in > Python 4000. ;-) > > The coders who use octal (including myself) can > adapt. There's not a lot of ambiguity when > int('042') evaluates to 42. 1) int('042') will keep evaluating to 42 (it does that right now) 2) x = 042 will raise a SyntaxError 3) If this is a huge mistake, we can easily fix it before 2017. It is much easier to stop raising SyntaxErrors if we are sure that it is not going to confuse anybody, than it would be to start raising them if we decided to make x = 042 bind 'x' to the decimal value '42'. In other words, I view this as a two step process. First, stop accepting 042 as an octal number. Next, if we revisit this and there is consensus, we could start accepting 042 as a decimal number. I know this is the most important issue for some people, but as the Romans used to say, let's "make haste slowly." Regards, Pat From jbauer at rubic.com Thu Mar 15 03:36:05 2007 From: jbauer at rubic.com (Jeff Bauer) Date: Wed, 14 Mar 2007 21:36:05 -0500 Subject: [Python-3000] Octal In-Reply-To: References: <45F8949E.1000304@rubic.com> Message-ID: <45F8B115.7090008@rubic.com> Patrick Maupin wrote: > 1) int('042') will keep evaluating to 42 (it does that right now) Argh. Brain flatulence. > 2) x = 042 will raise a SyntaxError #2 is what I meant. > 3) If this is a huge mistake, we can easily fix it before 2017. Agreed that it's easier to fix than something like integer division, but it would be nice to address in 2to3 (when there's already some understanding of backward incompatibility) than in 3.0 -> 3.1. -- Jeff Bauer Rubicon, Inc. From pmaupin at gmail.com Thu Mar 15 03:41:58 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 21:41:58 -0500 Subject: [Python-3000] Octal In-Reply-To: <45F8B115.7090008@rubic.com> References: <45F8949E.1000304@rubic.com> <45F8B115.7090008@rubic.com> Message-ID: On 3/14/07, Jeff Bauer wrote: > > 3) If this is a huge mistake, we can easily fix it before 2017. > > Agreed that it's easier to fix than something like > integer division, but it would be nice to address > in 2to3 (when there's already some understanding of > backward incompatibility) than in 3.0 -> 3.1. Maybe I'm missing something. 2to3 needs to insert 't' after leading zeros for octal. My point was that (I believe) that being more permissive (removing the syntax error) later is an easy fix. There is no need for a conversion tool when things that you didn't used to be able to do start working. The only issue would be if somebody wrote code that DEPENDED on x = 042 raising a SyntaxError. I suppose somebody could do this, but I can't work up the energy to care if their code is broken later... Regards, Pat From pmaupin at gmail.com Thu Mar 15 04:18:43 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 22:18:43 -0500 Subject: [Python-3000] One more question on binary/octal Message-ID: As Greg Ewing pointed out (tongue in cheek, I think, but I'm never sure), it would be consistent with 'x' for hexadecimal and 't' for octal to use 'n' for binary rather than 'b', e.g. theanswer = 0n101010 Part of me recoils at this, part of me agrees that this is not any stranger than using 'x' for hex or 't' for octal; and the real reason I am writing this email is that part of me thinks that it would be no fun at all to be staring at a line of Python at 3:00AM wondering why, after executing: x = 0b0 x doesn't have the value of 176 (0xb0). Like 'b', 'n' does not appear in the literal words 'hexadecimal' or 'octal', so that is good. 'n' has been chosen to be a "general number" character in PEP3101, but it is not a valid type specifier for the current % operator, so PEP3101 could easily change. 'b' has the advantage of being used in some other languages. OTOH, to a large extent we're eschewing that for internal consistency with Python 3.0, and Greg's 'n' IS more consistent with 'x' and 't' than 'b' is, and if we use 'n' for binary, '0b0' would be quickly caught as an exception, so the disadvantage is momentary. The primary advantage of 'n' over 'b', of course, is that 'n' is not a valid hexadecimal digit. So, after I get over my initial shock at thinking about biNary numbers, I have to give Greg's "proposal" a +1. Regards, Pat From fumanchu at amor.org Thu Mar 15 04:12:44 2007 From: fumanchu at amor.org (Robert Brewer) Date: Wed, 14 Mar 2007 20:12:44 -0700 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax References: Message-ID: <435DF58A933BA74397B42CDEB8145A86224D62@ex9.hostedexchange.local> Guido van Rossum wrote: > The PEP proposes that the class statement accepts > keyword arguments, *args, and **kwds syntax as well > as positional bases. This is a bit messy to compile > and execute, but we already have this, of course, in > the code for calling regular functions. > > So I think it would be acceptable to this into a > call to a new (hidden) built-in function, named > __build_class__. Then that this class definition: > > class C(A, B, metaclass=M, other=42, *more_bases, > *more_kwds): > ... > > would translate into this: > > C = __build_class__(, 'C', A, B, metaclass=M, > other=42, *more_bases, *more_kwds) This sounds familiar ;) http://mail.python.org/pipermail/python-dev/2004-March/043562.html Why not just call that function "class" and get it over with? ;) Robert Brewer System Architect Amor Ministries fumanchu at amor.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070314/5b4a8ee6/attachment.htm From steven.bethard at gmail.com Thu Mar 15 04:43:31 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 14 Mar 2007 21:43:31 -0600 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax In-Reply-To: <435DF58A933BA74397B42CDEB8145A86224D62@ex9.hostedexchange.local> References: <435DF58A933BA74397B42CDEB8145A86224D62@ex9.hostedexchange.local> Message-ID: On 3/14/07, Robert Brewer wrote: > Guido van Rossum wrote: > > The PEP proposes that the class statement accepts > > keyword arguments, *args, and **kwds syntax as well > > as positional bases. This is a bit messy to compile > > and execute, but we already have this, of course, in > > the code for calling regular functions. > > > > So I think it would be acceptable to this into a > > call to a new (hidden) built-in function, named > > __build_class__. Then that this class definition: > > > > class C(A, B, metaclass=M, other=42, *more_bases, > > *more_kwds): > > ... > > > > would translate into this: > > > > C = __build_class__(, 'C', A, B, metaclass=M, > > other=42, *more_bases, *more_kwds) > > This sounds familiar ;) > http://mail.python.org/pipermail/python-dev/2004-March/043562.html > > Why not just call that function "class" and get it over with? ;) That certainly has an appeal. Then you'd have:: class (<*args>, <**kwargs>): translated into:: = class(, , <*args>, <**kwargs>) And in Python 4000, we can allow any function instead of just "class", and we can basically get the "make" statement PEP. ;-) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From jason.orendorff at gmail.com Thu Mar 15 04:46:23 2007 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Wed, 14 Mar 2007 23:46:23 -0400 Subject: [Python-3000] New I/O PEP to Subversion In-Reply-To: References: Message-ID: On 3/14/07, Guido van Rossum wrote: > I think we're far enough along with the new I/O PEP and trial > implementation that I'm uncomfortable with the PEP living in Google > docs only (http://docs.google.com/Doc?id=dfksfvqd_1cn5g5m). Does > someone have half an hour of time available to convert it to standard > PEP mark-up? See PEP 9 or 12. > > If you mail it to me I'll check it in. I'll do it. -j From greg.ewing at canterbury.ac.nz Thu Mar 15 05:01:17 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 17:01:17 +1300 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> Message-ID: <45F8C50D.3010009@canterbury.ac.nz> Raymond Hettinger wrote: > The worst part of the new user experience is encountering a > SyntaxError. Because of the way we generate them, invalid > syntax errors are not very informative This seems like an argument for producing more informative error messages. Something like Syntax error: Numeric literal with leading zeroes is ambiguous. Use 0t123 for an octal literal, or omit the leading zeroes for a decimal literal. should make it pretty clear what the problem is and how to fix it. -- Greg From pmaupin at gmail.com Thu Mar 15 05:25:10 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 23:25:10 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45F8C50D.3010009@canterbury.ac.nz> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <45F8C50D.3010009@canterbury.ac.nz> Message-ID: On 3/14/07, Greg Ewing wrote: > Raymond Hettinger wrote: > > > The worst part of the new user experience is encountering a > > SyntaxError. Because of the way we generate them, invalid > > syntax errors are not very informative > > This seems like an argument for producing more informative > error messages. Something like > > Syntax error: Numeric literal with leading zeroes is > ambiguous. Use 0t123 for an octal literal, or omit the > leading zeroes for a decimal literal. > > should make it pretty clear what the problem is and > how to fix it. What, you don't like: >>> 09 File "", line 1 09 ^ SyntaxError: invalid token I can't imagine why :) Raymond makes a very good point that syntax errors are the bane of newbies. When I started using Python, I was very impressed with the accuracy and utility of the exception system. It may be that most experienced programmers have similar feelings, so it would seem that nobody has turned a critical eye on the reporting of errors to nervous first-time programmers who are already worried they are going to break the machine. From pmaupin at gmail.com Thu Mar 15 05:30:24 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Wed, 14 Mar 2007 23:30:24 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <45F8C50D.3010009@canterbury.ac.nz> Message-ID: Let me restate that. It's obvious that a lot of error message have had a lot of thought put into them, so when I said: > so it would seem that > nobody has turned a critical eye on the reporting of errors to nervous > first-time programmers who are already worried they are going to break > the machine. I really should have said that: a) there is no apparent process for approving messages from a newbie's viewpoint; and b) the exception reporting system, which has been little changed in a very long time, should also be given some real thought in 3.0. For example, perhaps it can even have an option to link to a web how-to which is specific to a given error... Regards, Pat From tony at PageDNA.com Thu Mar 15 05:33:42 2007 From: tony at PageDNA.com (Tony Lownds) Date: Wed, 14 Mar 2007 21:33:42 -0700 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax In-Reply-To: References: Message-ID: <101D80A5-7D5C-48C6-9C75-60F2E3E0FC03@PageDNA.com> On Mar 14, 2007, at 6:26 PM, Guido van Rossum wrote: > The PEP proposes that the class statement accepts keyword arguments, > *args, and **kwds syntax as well as positional bases. This is a bit > messy to compile and execute, but we already have this, of course, in > the code for calling regular functions. > > So I think it would be acceptable to this into a call to a new > (hidden) built-in function, named __build_class__. Then that this > class definition: > > class C(A, B, metaclass=M, other=42, *more_bases, *more_kwds): > ... > > would translate into this: > > C = __build_class__(, 'C', A, B, metaclass=M, other=42, > *more_bases, *more_kwds) > > where is a function object for the class body. (It's a slightly > different function than currently; the current function *returns* the > locals, while the new one *takes* the locals as an argument; instead > of a LOAD_LOCALS opcode we need a STORE_LOCALS opcode.) > > Then __build_class__ could be roughly like this (but implemented in > C): > > def __build_class__(func, name, *bases, metaclass=None, **kwds): > if metaclass is None: > metaclass = extract_metaclass(bases) # may raise an exception > prepare = getattr(metaclass, "__prepare__", None) > if prepare: > locals = prepare(name, bases, **kwds) > else: > locals = {} > func(locals) > return metaclass(name, bases, locals, **kwds) > > What do folks think? > This seems elegant to me. Does **kwds need to be passed to prepare and also to metaclass constructor? Removing that would make it easier to provide nice error messages. Custom __prepare__ implementations can always store the **kwds that are needed in or on the dictionary-like object. Instead of a function, maybe __build_class__ can just take a code object. Then STORE_LOCALS won't be needed. def __build_class__(code, name, *bases, metaclass=None, **kwds): if metaclass is None: metaclass = extract_metaclass(bases) # may raise an exception prepare = getattr(metaclass, "__prepare__", None) if prepare: locals = prepare(name, bases, **kwds) else: if kwds: raise TypeError, "default metaclass takes no extra arguments" locals = {} exec(code, locals) return metaclass(name, bases, locals) How do you load a hidden built-in? An new opcode, like LOAD_BUILTIN? Or is it hidden by convention only? LOAD_BUILTIN 0 (__build_class__) LOAD_CONST 1 () LOAD_CONST 0 ('classname') LOAD_NAME 1 (base) CALL_FUNCTION 3 STORE_NAME 2 (name) Thanks -Tony From jbauer at rubic.com Thu Mar 15 05:36:12 2007 From: jbauer at rubic.com (Jeff Bauer) Date: Wed, 14 Mar 2007 23:36:12 -0500 Subject: [Python-3000] Octal In-Reply-To: References: <45F8949E.1000304@rubic.com> <45F8B115.7090008@rubic.com> Message-ID: <45F8CD3C.5080700@rubic.com> Patrick Maupin wrote: > Maybe I'm missing something. 2to3 needs to insert 't' after leading > zeros for octal. By "2to3" I meant the forthcoming 2-to-3 conversion tool. -- Jeff Bauer Rubicon, Inc. From greg.ewing at canterbury.ac.nz Thu Mar 15 06:34:11 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 15 Mar 2007 18:34:11 +1300 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax In-Reply-To: References: Message-ID: <45F8DAD3.7090705@canterbury.ac.nz> Guido van Rossum wrote: > So I think it would be acceptable to this into a call to a new > (hidden) built-in function, named __build_class__. > > > What do folks think? Looks good to me. It'll also provide some interesting new possibilities for overriding it in creative ways. :-) -- Greg From thomas at python.org Thu Mar 15 09:39:19 2007 From: thomas at python.org (Thomas Wouters) Date: Thu, 15 Mar 2007 09:39:19 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> Message-ID: <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> On 3/15/07, Raymond Hettinger wrote: > My worry focuses on new-timers and SyntaxErrors. The worst part of the new > user experience is encountering a SyntaxError. Please. I already said it would be a SyntaxError *with explanation*. It's pretty easy to make a syntax error that goes "Did you mean 0t6065 or 6065?". This would not be a syntax error handled by the parser, the tokenizer really doesn't care about leading 0's and ast.c's parsenumber (which would be the spot to raise it) has all the necessary information. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070315/c081c677/attachment.html From g.brandl at gmx.net Thu Mar 15 09:47:06 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 15 Mar 2007 09:47:06 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Thomas Wouters schrieb: > > > On 3/15/07, *Raymond Hettinger* > > wrote: > > My worry focuses on new-timers and SyntaxErrors. The worst part of > the new user experience is encountering a SyntaxError. > > > Please. I already said it would be a SyntaxError *with explanation*. > It's pretty easy to make a syntax error that goes "Did you mean 0t6065 > or 6065?". This would not be a syntax error handled by the parser, the > tokenizer really doesn't care about leading 0's and ast.c's parsenumber > (which would be the spot to raise it) has all the necessary information. Until now, I skipped the octal thread. Now I read this and wondered, "what the hell is 0t6065 supposed to mean" and only from the context I gathered that it would be an octal literal... Seriously, nobody, even coming from another language, will be able to look at it and say, "yes, that's an octal literal." Georg From talin at acm.org Thu Mar 15 10:11:40 2007 From: talin at acm.org (Talin) Date: Thu, 15 Mar 2007 02:11:40 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: References: <45F7A445.5090507@acm.org> <45F8561E.6070509@acm.org> Message-ID: <45F90DCC.4090503@acm.org> Guido van Rossum wrote: > While you're at it, could you also add a pointer to this patch as a > sample implementation? > > http://python.org/sf/1681101 > > It's far from complete at this point; so far, I've got the syntax > working and it seems to compile old code just fine, but the keyword > args and */** notation are simply ignored. If someone wants to work on > this, drop me a note to see if I haven't made progress already. > > --Guido That is way cool. One other thing that I think might be good to have (at some point after we get a working implementation) is some sort of HOWTO document (apart from the PEP) that lays out a set of guidelines or advice on creating metaclasses. Perhaps a Python cookbook recipe or something. The reason I say this is because one of the things that I noticed is that writing a metaclass that handles all corner cases properly can be quite tricky. As an example, if you're doing anything with ordered fields, you need to make sure that the base class's ordered fields are properly incorporated into the calculation. Exploiting the new metaclass mechanism fully requires a deeper knowledge of the way function bodies are evaluated and how the type() constructor works. The fact that there was some confusion as to what methods of the dict interface needed to be supported is evidence of this. The PEP gives you a mechanism for declaring and using metaclasses, and invites you to play with the class machinery at a deeper level, but it doesn't tell much about the internal mechanism and structure of classes themselves, or what is/isn't valid in terms of tweaking them. Thus, for example, if I want a class whose instances don't have a dictionary, but instead store their members in a flat array (a la __slots__), I would need to know what parts of the system will choke, expecting a dict to be there when there isn't one, and how to get around that. -- Talin From thomas at python.org Thu Mar 15 10:19:14 2007 From: thomas at python.org (Thomas Wouters) Date: Thu, 15 Mar 2007 10:19:14 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: <9e804ac0703150219l4fcaffd4l6e182aa42051f49c@mail.gmail.com> On 3/15/07, Georg Brandl wrote: > Until now, I skipped the octal thread. Now I read this and wondered, "what > the > hell is 0t6065 supposed to mean" and only from the context I gathered that > it > would be an octal literal... > > Seriously, nobody, even coming from another language, will be able to look > at it > and say, "yes, that's an octal literal." Yes. Raymond's (and Guido's and others') point is that hardly anybody knows what an octal literal is in the first place, and the less-explicit syntax of '06065' gives confusing results to those who don't. You can't accidentily use this new syntax. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070315/1987e338/attachment.htm From nicko at nicko.org Thu Mar 15 10:29:26 2007 From: nicko at nicko.org (Nicko van Someren) Date: Thu, 15 Mar 2007 09:29:26 +0000 Subject: [Python-3000] Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> Message-ID: On 14 Mar 2007, at 23:08, Guido van Rossum wrote: > IOW, the only thing I'm unsure about is whether int('0012', 0) should > return 10 or complain like eval('0012') would. Perhaps it's best to > say that int('0012', 0) is a safer way to parse Python int literals, > and that would make it an error. The current documentation for the int() says "If radix is zero ... the interpretation is the same as for integer literals." I very much think that this should remain true. Nicko From martin at v.loewis.de Thu Mar 15 11:29:29 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 15 Mar 2007 11:29:29 +0100 Subject: [Python-3000] I18N identifiers Message-ID: <45F92009.9060500@v.loewis.de> What is the status of I18N identifiers? Has any progress been made? If not, I'd like to write a PEP, and mentor a SoC student to work on that (if one shows up). Regards, Martin From barry at python.org Thu Mar 15 11:54:06 2007 From: barry at python.org (Barry Warsaw) Date: Thu, 15 Mar 2007 06:54:06 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> Message-ID: <4E95D860-C77B-418D-8C1A-B27EE617A0F1@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mar 14, 2007, at 8:33 PM, Guido van Rossum wrote: > While I would agree in an ideal world, I find Thomas's argument > compelling. Not all newbie Python users are first-time computer users; > if they have C/C++ or other Unixoid languages under their belt (even > Java has leading-zero-as-octal!) they might type 0600 and never notice > it returns the wrong value until it's too late. Swapping the confusion > of one group for the confusion of another group is not quite worth it. I agree completely. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRfklznEjvBPtnXfVAQLF2AQAjkuZRnma4EVziqqWimzNznJCNGytI20J KMSz+VCxv3JON8sFZbTS2kO+G8TjG/sF7ecD/tv91dyht92qgnC4Ywy1kUGNTfY7 sG8qCksB7rSpfoVNX6iWNOnFJQ0ymlxt2dNecMCNxGp2lYkXX/N3JySrabh+JTCD M4hjDX4DN8Y= =UgZe -----END PGP SIGNATURE----- From nicko at nicko.org Thu Mar 15 14:30:15 2007 From: nicko at nicko.org (Nicko van Someren) Date: Thu, 15 Mar 2007 13:30:15 +0000 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax In-Reply-To: <101D80A5-7D5C-48C6-9C75-60F2E3E0FC03@PageDNA.com> References: <101D80A5-7D5C-48C6-9C75-60F2E3E0FC03@PageDNA.com> Message-ID: On 15 Mar 2007, at 04:33, Tony Lownds wrote: > Does **kwds need to be passed to prepare and also to metaclass > constructor? > Removing that would make it easier to provide nice error messages. > Custom __prepare__ > implementations can always store the **kwds that are needed in or > on the > dictionary-like object. If you don't pass it to both the constructor and the preparer then all metaclasses which take keyword arguments must have a __prepare__ function whether they have custom local storage or not. This seems to be non-obvious and it also unnecessarily conflates the issues of custom locals storage and passing arguments to metaclasses. Nicko From jimjjewett at gmail.com Thu Mar 15 14:31:53 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 15 Mar 2007 09:31:53 -0400 Subject: [Python-3000] PEP for Metaclasses in Python 3000 In-Reply-To: <45F1C734.7080503@acm.org> References: <45F1C734.7080503@acm.org> Message-ID: On 3/9/07, Talin wrote: > Note that the Python interpreter will check to insure that the > __metacreate__ attribute exists before calling it. This preserves > backwards compatibility with existing metaclasses. You might want to be even more explicit and say If the metaclass does not have a __prepare__ attribute, then the interpreter will supply an empty mapping. (I was tempted to say "ordinary dict", but that might not be true in, say, a secured mode.) -jJ From jimjjewett at gmail.com Thu Mar 15 14:45:59 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 15 Mar 2007 09:45:59 -0400 Subject: [Python-3000] metaclass syntax [was: Discussions with no PEPs] In-Reply-To: References: Message-ID: (reasons to use a custom dict, other than ordering) > > (1) immutable class dictionaries. These are typical for > > extension classes, but a real pain for python classes. > > (2) almost-immutable -- with callbacks when values are > > added/changed/deleted. > > (3) actually-private variables On 3/9/07, Guido van Rossum wrote: > Those look like use cases for metaclasses, but I don't see how > they require setting a custom dict *while the class suite is being > executed*. I was thinking too much of attempts to subclass module. The newly created objects retain a reference to the original dictionary as their globals. I should at least have made that assumption clear. -jJ From collinw at gmail.com Thu Mar 15 16:50:35 2007 From: collinw at gmail.com (Collin Winter) Date: Thu, 15 Mar 2007 10:50:35 -0500 Subject: [Python-3000] Removing sys.exitfunc In-Reply-To: References: <43aa6ff70703081903m5f747772qc0d4820fdf4dcb0@mail.gmail.com> Message-ID: <43aa6ff70703150850n3ea96fe4h877d79638a8ad147@mail.gmail.com> On 3/8/07, Guido van Rossum wrote: > Rename it to sys._exitfunc? Georg and I have independently reimplemented atexit in C: see patch #1680961. Collin Winter From pje at telecommunity.com Thu Mar 15 17:19:35 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Mar 2007 11:19:35 -0500 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F881A9.8050205@canterbury.ac.nz> References: <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> <45F7A445.5090507@acm.org> <45F7A445.5090507@acm.org> <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20070315111634.04e59688@sparrow.telecommunity.com> At 12:13 PM 3/15/2007 +1300, Greg Ewing wrote: >Phillip J. Eby wrote: > > > So the subset required is determined entirely by what the class body > > (and metaclass __new__) require. > >Perhaps the PEP should say something to the effect that >get, set and delete need to be implemented in order to >support all the things the class body might want to do, >but a metaclass can omit them if it wants to restrict >what can be done in the class body. > > > So the metaclass' __new__ > > is going to need to read out the contents of the pseudo-dict somehow, or > > generate alternative contents. > >Maybe the PEP should specify a protocol that the >pseudo-dict must provide for doing this, such as an >iteritems() method (or whatever the Py3k equivalent >is). That's not necessary; the metaclass instance __call__ (or class __new__) should be responsible for passing a real dictionary to type.__new__, if it in fact wants to create a type object all. Remember, metaclasses need not be types, nor to create types. For that matter, it isn't necessarily the case that the metaclass will directly map the dictionary "contents" into a dictionary at all. There may be renaming of keys, creation of new keys, code generation, and who-knows-what-else. So, having a protocol for transforming the suite locals into a type dictionary would be unnecessarily restrictive. From pje at telecommunity.com Thu Mar 15 17:23:48 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Thu, 15 Mar 2007 11:23:48 -0500 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax In-Reply-To: Message-ID: <5.1.1.6.0.20070315112033.04c6a6a0@sparrow.telecommunity.com> At 06:26 PM 3/14/2007 -0700, Guido van Rossum wrote: >(It's a slightly >different function than currently; the current function *returns* the >locals, while the new one *takes* the locals as an argument; instead >of a LOAD_LOCALS opcode we need a STORE_LOCALS opcode.) Wouldn't it suffice to exec a code object with the specified locals? Then the class suite wouldn't even need to be a function, since it would neither accept parameters nor return results. And no new opcode would be needed. So in this: >def __build_class__(func, name, *bases, metaclass=None, **kwds): > if metaclass is None: > metaclass = extract_metaclass(bases) # may raise an exception > prepare = getattr(metaclass, "__prepare__", None) > if prepare: > locals = prepare(name, bases, **kwds) > else: > locals = {} > func(locals) > return metaclass(name, bases, locals, **kwds) we would change 'func' to 'code' and instead of func(locals) we would 'exec code in globals, locals'. From mattias at virtutech.se Thu Mar 15 17:28:27 2007 From: mattias at virtutech.se (=?iso-8859-1?q?Mattias_Engdeg=E5rd?=) Date: Thu, 15 Mar 2007 16:28:27 +0000 (UTC) Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Georg Brandl writes: >Seriously, nobody, even coming from another language, will be able to look at it >and say, "yes, that's an octal literal." Scientific googling reveals that most other languages that use 0x for hex use 0b for binary and either 0o or 0 for octal, so if "cultural compatibility" is important, this may influence the choice. Whatever prefix is used, it should be clearly offset visually from the number, so it's clear what prefix it is and where the digits start. 0 as a prefix does not really have this property - a good reason to change; 0o and 0b both work. 0c may be slightly better than 0t, graphically speaking. From guido at python.org Thu Mar 15 18:02:36 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 15 Mar 2007 10:02:36 -0700 Subject: [Python-3000] Removing sys.exitfunc In-Reply-To: <43aa6ff70703150850n3ea96fe4h877d79638a8ad147@mail.gmail.com> References: <43aa6ff70703081903m5f747772qc0d4820fdf4dcb0@mail.gmail.com> <43aa6ff70703150850n3ea96fe4h877d79638a8ad147@mail.gmail.com> Message-ID: OK, if you two agree on what to do, just go ahead! On 3/15/07, Collin Winter wrote: > On 3/8/07, Guido van Rossum wrote: > > Rename it to sys._exitfunc? > > Georg and I have independently reimplemented atexit in C: see patch #1680961. > > Collin Winter > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Mar 15 18:12:53 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 15 Mar 2007 10:12:53 -0700 Subject: [Python-3000] Compiling the PEP 3115 metaclass syntax In-Reply-To: <5.1.1.6.0.20070315112033.04c6a6a0@sparrow.telecommunity.com> References: <5.1.1.6.0.20070315112033.04c6a6a0@sparrow.telecommunity.com> Message-ID: On 3/15/07, Phillip J. Eby wrote: > At 06:26 PM 3/14/2007 -0700, Guido van Rossum wrote: > >(It's a slightly > >different function than currently; the current function *returns* the > >locals, while the new one *takes* the locals as an argument; instead > >of a LOAD_LOCALS opcode we need a STORE_LOCALS opcode.) > > Wouldn't it suffice to exec a code object with the specified locals? Then > the class suite wouldn't even need to be a function, since it would neither > accept parameters nor return results. And no new opcode would be needed. This was brought up twice now, and I initially drafted it that way, but then I started to think about closures and after looking at the byte code for a class whose methods reference closures I realized that making it a function is necessary to pass the closures in. Unless there's a different way to deal with closures at the C level. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Mar 15 18:21:41 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 15 Mar 2007 10:21:41 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/15/07, Mattias Engdeg?rd wrote: > Scientific googling reveals that most other languages that use 0x for hex use > 0b for binary and either 0o or 0 for octal, so if "cultural compatibility" > is important, this may influence the choice. > > Whatever prefix is used, it should be clearly offset visually from the number, > so it's clear what prefix it is and where the digits start. 0 as a prefix > does not really have this property - a good reason to change; 0o and 0b both > work. 0c may be slightly better than 0t, graphically speaking. 0c was rejected because in some fonts 'c' looks too much like a zero; though it wasn't me and I don't recall every having this issue. I'm somewhat uncomfortable with 0o because the zero in some fonts is about the same size as a lowercase oh; but I'd be willing to support it in the name of conventionality. Frankly, the choice of letter is a minor issue to me that I'd gladly leave to others to discuss and decide. My main interest is that we should move away from just leading zero as octal, disallow leading zero altogether because of the confusion it would cause for folks coming from other neutral about the choice between 0o, 0c or 0t. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Thu Mar 15 18:24:03 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 15 Mar 2007 10:24:03 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/15/07, Guido van Rossum wrote: > My main interest is that we should move away from just leading zero as > octal, disallow leading zero altogether because of the confusion it > would cause for folks coming from other neutral about the choice > between 0o, 0c or 0t. Something got lost there in a last-minute edit. I meant to say: My main interest is that we should move away from just leading zero as octal; that we should disallow leading zero altogether because of the confusion it would cause for folks coming from other languagesl and that we should avoid the swamp that is generalized numeric bases. I'm neutral about the choice between 0o, 0c or 0t. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Thu Mar 15 19:05:37 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 15 Mar 2007 19:05:37 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Guido van Rossum schrieb: > On 3/15/07, Mattias Engdeg?rd wrote: >> Scientific googling reveals that most other languages that use 0x for hex use >> 0b for binary and either 0o or 0 for octal, so if "cultural compatibility" >> is important, this may influence the choice. >> >> Whatever prefix is used, it should be clearly offset visually from the number, >> so it's clear what prefix it is and where the digits start. 0 as a prefix >> does not really have this property - a good reason to change; 0o and 0b both >> work. 0c may be slightly better than 0t, graphically speaking. > > 0c was rejected because in some fonts 'c' looks too much like a zero; > though it wasn't me and I don't recall every having this issue. I'm > somewhat uncomfortable with 0o because the zero in some fonts is about > the same size as a lowercase oh; but I'd be willing to support it in > the name of conventionality. Frankly, the choice of letter is a minor > issue to me that I'd gladly leave to others to discuss and decide. My concern is that I don't want to have to write something like x = 0t755 # octal literal because then I can also write x = 493 # 0755 in octal and don't need octal literals at all. Georg From pmaupin at gmail.com Thu Mar 15 20:26:14 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Thu, 15 Mar 2007 14:26:14 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/15/07, Georg Brandl wrote: > My concern is that I don't want to have to write something like > > x = 0t755 # octal literal > > because then I can also write > > x = 493 # 0755 in octal But, in general, code comments about how Python works are to be discouraged. If 0t755 becomes part of the language, your first example would be almost as silly as writing: x += 1 # Increment X However, even if you still feel the need to do this, I submit that it takes less time and mental energy to indicate to your reader that it is an octal number, than the time and mental energy for you to convert 0755 to 493, and (more importantly) for your reader to mentally convert 493 BACK to 0755 just to check that the comment is not lying. (A knowledgeable Python reader can verify the veracity of your "# octal literal" comment in milliseconds, but not your other comment.) Regards, Pat From daniel at stutzbachenterprises.com Thu Mar 15 20:29:56 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 15 Mar 2007 14:29:56 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/15/07, Georg Brandl wrote: > My concern is that I don't want to have to write something like > > x = 0t755 # octal literal > > because then I can also write > > x = 493 # 0755 in octal > > and don't need octal literals at all. The later is error-prone and might be hard to debug if you put too much faith in comments. However, I'd be happy with leaving out octal literals in favor of: x = int('755', 8) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From apt.shansen at gmail.com Thu Mar 15 20:48:22 2007 From: apt.shansen at gmail.com (Stephen Hansen) Date: Thu, 15 Mar 2007 12:48:22 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: <7a9c25c20703151248w56c5b297w12477d7290410803@mail.gmail.com> On 3/15/07, Guido van Rossum wrote: > > I'm neutral about the choice between 0o, 0c or 0t. Interested Lurker Opinion: Can it be a "small" character? 0x1 leaps out for me because of the fact that numbers are all "tall" and the x is short-- so having that difference in height makes it very clear this is not a normal base 10 number when I'm scanning text. 0o755 does the same, as does 0c755. Heck, even 0b10100 looks fine even though the 'b' has a stalk, since its bottom-heavy with the stalk off to the side. However, a 't' is a tall and relatively streamlined character... 0t755 looks all jumbled together and doesn't provide as clear a visual indicator that something abnormal is going on here. I periodically have to use octal numbers, and so far am only using them with the int() function because I don't find 0755 as readily expressive of "Hi! I'm Octal!" as I'd like. 0t755 wouldn't be much of an improvement; but anything else short would be, from 0o755 to 0c755 to ... 0a755 .. or whatever. :) --Stephen, who goes back to lurking interestedly. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070315/55fbee23/attachment.html From tim.peters at gmail.com Thu Mar 15 21:26:29 2007 From: tim.peters at gmail.com (Tim Peters) Date: Thu, 15 Mar 2007 16:26:29 -0400 Subject: [Python-3000] Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> [Guido] > Something got lost there in a last-minute edit. I meant to say: > > My main interest is that we should move away from just leading zero as > octal; that we should disallow leading zero altogether because of the > confusion it would cause for folks coming from other languagesl and > that we should avoid the swamp that is generalized numeric bases. I'm > neutral about the choice between 0o, 0c or 0t. While I like the goal, I'd prefer not to be cryptically clever. How about restricting the base in Icon's clean syntax to one of {2, 4, 8, 16}? Here's 10 in those four bases: 2r1010 4r22 8r12 16ra (or) 16rA "r" is mnemonic for "radix" in this context. The Ada syntax is similarly clean, but has both a leading & trailing "bracket" character. IIRC, that's because Ada wants to support embedded whitespace in numeric literals. From tjreedy at udel.edu Thu Mar 15 21:47:15 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 15 Mar 2007 16:47:15 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Octal References: <20070314170458.BBL23900@ms09.lnh.mail.rcn.net> Message-ID: "Nicko van Someren" wrote in message news:F85A7BFC-E4E7-4287-912D-FA38BE617648 at nicko.org... | On 14 Mar 2007, at 23:08, Guido van Rossum wrote: | > IOW, the only thing I'm unsure about is whether int('0012', 0) should | > return 10 or complain like eval('0012') would. Perhaps it's best to | > say that int('0012', 0) is a safer way to parse Python int literals, | > and that would make it an error. | | The current documentation for the int() says "If radix is zero ... | the interpretation is the same as for integer literals." I very much | think that this should remain true. I agree. Adding an exception just slightly clouds the language. Anyway, int('12',8) in one *less* character to type! Terry Jan Reedy From tjreedy at udel.edu Thu Mar 15 21:50:35 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 15 Mar 2007 16:50:35 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net><9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <7a9c25c20703151248w56c5b297w12477d7290410803@mail.gmail.com> Message-ID: "Stephen Hansen" wrote in message news:7a9c25c20703151248w56c5b297w12477d7290410803 at mail.gmail.com... | On 3/15/07, Guido van Rossum wrote: | > | > I'm neutral about the choice between 0o, 0c or 0t. | | | Interested Lurker Opinion: Can it be a "small" character? 0x1 leaps out for | me because of the fact that numbers are all "tall" and the x is short-- so | having that difference in height makes it very clear this is not a normal | base 10 number when I'm scanning text. 0o755 does the same, as does 0c755. | Heck, even 0b10100 looks fine even though the 'b' has a stalk, since its | bottom-heavy with the stalk off to the side. | | However, a 't' is a tall and relatively streamlined character... 0t755 looks | all jumbled together and doesn't provide as clear a visual indicator that | something abnormal is going on here. I periodically have to use octal | numbers, and so far am only using them with the int() function because I | don't find 0755 as readily expressive of "Hi! I'm Octal!" as I'd like. 0t755 | wouldn't be much of an improvement; but anything else short would be, from | 0o755 to 0c755 to ... 0a755 .. or whatever. :) I agree. 0t = base two?, three?, ten?, thirteen?, twenty? ;-) tjr From guido at python.org Thu Mar 15 22:08:29 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 15 Mar 2007 14:08:29 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <45F90DCC.4090503@acm.org> References: <45F7A445.5090507@acm.org> <45F8561E.6070509@acm.org> <45F90DCC.4090503@acm.org> Message-ID: On 3/15/07, Talin wrote: > Guido van Rossum wrote: > > While you're at it, could you also add a pointer to this patch as a > > sample implementation? > > > > http://python.org/sf/1681101 > > That is way cool. It is now way cooler -- it *works*! The build_class function is implemented in Python. To try this out, go to the SF patch. I'm soliciting help at this point!! > One other thing that I think might be good to have (at some point after > we get a working implementation) is some sort of HOWTO document (apart > from the PEP) that lays out a set of guidelines or advice on creating > metaclasses. Perhaps a Python cookbook recipe or something. > > The reason I say this is because one of the things that I noticed is > that writing a metaclass that handles all corner cases properly can be > quite tricky. As an example, if you're doing anything with ordered > fields, you need to make sure that the base class's ordered fields are > properly incorporated into the calculation. > > Exploiting the new metaclass mechanism fully requires a deeper knowledge > of the way function bodies are evaluated and how the type() constructor > works. The fact that there was some confusion as to what methods of the > dict interface needed to be supported is evidence of this. > > The PEP gives you a mechanism for declaring and using metaclasses, and > invites you to play with the class machinery at a deeper level, but it > doesn't tell much about the internal mechanism and structure of classes > themselves, or what is/isn't valid in terms of tweaking them. Thus, for > example, if I want a class whose instances don't have a dictionary, but > instead store their members in a flat array (a la __slots__), I would > need to know what parts of the system will choke, expecting a dict to be > there when there isn't one, and how to get around that. Yes, this would be a great thing to add to the documentation somewhere. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Thu Mar 15 22:15:22 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Thu, 15 Mar 2007 16:15:22 -0500 Subject: [Python-3000] Octal In-Reply-To: <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> Message-ID: On 3/15/07, Tim Peters wrote: > 2r1010 > 4r22 > 8r12 > 16ra (or) 16rA > Part of me would miss 0xABCD terrbly, but I like the concept. The only real rub is the impending huge discussion about the number of allowed radices. The max 36 radix currently supported by int() is numerically suspiciously similar to the sum of the number of ASCII digit characters and the number of same-case ASCII letters, so obviously, a tempting answer to the question "How many radices should we support?" is "All of them." Regards, Pat From guido at python.org Thu Mar 15 22:56:34 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 15 Mar 2007 14:56:34 -0700 Subject: [Python-3000] Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> Message-ID: Please. No discussion of generalized radixes. Also, IMO Tim's proposal changes too much; there's nothing wrong with 0b and 0x. On 3/15/07, Patrick Maupin wrote: > On 3/15/07, Tim Peters wrote: > > > 2r1010 > > 4r22 > > 8r12 > > 16ra (or) 16rA > > > > Part of me would miss 0xABCD terrbly, but I like the concept. The > only real rub is the impending huge discussion about the number of > allowed radices. The max 36 radix currently supported by int() is > numerically suspiciously similar to the sum of the number of ASCII > digit characters and the number of same-case ASCII letters, so > obviously, a tempting answer to the question "How many radices should > we support?" is "All of them." > > Regards, > Pat > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Mar 15 23:48:50 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Mar 2007 11:48:50 +1300 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: <5.1.1.6.0.20070315111634.04e59688@sparrow.telecommunity.com> References: <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> <45F7A445.5090507@acm.org> <45F7A445.5090507@acm.org> <5.1.1.6.0.20070314110851.03f96e80@sparrow.telecommunity.com> <5.1.1.6.0.20070315111634.04e59688@sparrow.telecommunity.com> Message-ID: <45F9CD52.9060706@canterbury.ac.nz> Phillip J. Eby wrote: > > Maybe the PEP should specify a protocol that the > > pseudo-dict must provide for doing this, such as an > > iteritems() method (or whatever the Py3k equivalent > > is). > > That's not necessary; the metaclass instance __call__ (or class __new__) > should be responsible for passing a real dictionary to type.__new__, I was thinking that there could be a default mechanism used by type.__new__ to generate a dict if it wasn't passed a real dict. This would relieve some burden from the majority of metaclasses, I think. If the custom namespace is subclassed from dict or DictMixin or whatever ABC there is for this in Py3k, then it will already support the needed protocol. To be clearer, I perhaps should say that the protocol is only required if the metaclass wants to make use of this default behaviour. If the metaclass is willing to do its own dict extraction, then it doesn't matter. Equivalently, this could be phrased in terms of the protocol that must be supported by the object passed to type.__new__, i.e. must either be exactly a dict or be able to have a dict created from it by calling dict() on it. Then it's up to the metaclass how it satisfies this. And if the "metaclass" is really a function, type.__new__ isn't involved at all, so none of this applies. -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 16 00:31:15 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Mar 2007 12:31:15 +1300 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: <45F9D743.9090107@canterbury.ac.nz> Guido van Rossum wrote: > I'm > somewhat uncomfortable with 0o because the zero in some fonts is about > the same size as a lowercase oh; You already have to be careful what font you use for programming with, so I don't think that's a serious problem. More serious is that if we want to be consistent with 0x and allow upper case, we get 0O, which looks quite confusing in just about any font. -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 16 00:39:34 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Mar 2007 12:39:34 +1300 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: <45F9D936.9060600@canterbury.ac.nz> Georg Brandl wrote: > because then I can also write > > x = 493 # 0755 in octal and provide a chance for the code to get out of step with the comment. -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 16 01:19:47 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Mar 2007 13:19:47 +1300 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <7a9c25c20703151248w56c5b297w12477d7290410803@mail.gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <7a9c25c20703151248w56c5b297w12477d7290410803@mail.gmail.com> Message-ID: <45F9E2A3.6020405@canterbury.ac.nz> Stephen Hansen wrote: > Interested Lurker Opinion: Can it be a "small" character? 0q123 # oqtal Other available short or short-with-stalk letters: 0w123 0r123 0u123 0p123 0a123 0s123 0d123 0g123 0h123 0k123 0z123 0v123 0n123 0m123 -- Greg From greg.ewing at canterbury.ac.nz Fri Mar 16 01:39:12 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 16 Mar 2007 13:39:12 +1300 Subject: [Python-3000] Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> Message-ID: <45F9E730.5020507@canterbury.ac.nz> Patrick Maupin wrote: > The max 36 radix currently supported by int() is > numerically suspiciously similar to the sum of the number of ASCII > digit characters and the number of same-case ASCII letters What would you use for digits in bases higher than 36? Most of the ASCII punctuation is already taken... start plundering unicode for more alphabets? -- Greg From pmaupin at gmail.com Fri Mar 16 03:51:28 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Thu, 15 Mar 2007 21:51:28 -0500 Subject: [Python-3000] Octal In-Reply-To: <45F9E730.5020507@canterbury.ac.nz> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> <45F9E730.5020507@canterbury.ac.nz> Message-ID: On 3/15/07, Greg Ewing wrote: > What would you use for digits in bases higher > than 36? Most of the ASCII punctuation is already > taken... start plundering unicode for more > alphabets? I'm not suggesting that the number of supported bases should grow AT ALL. What I am suggesting is that it is obviously very tempting, since it is easy to do, to support bases up to 36. The PEP I am working on will recommend that we not remove this capability from int(string, base) because that might be gratuitous breakage, but that we not give in to the same temptation for int(string, 0) or for the language syntax. I have not seen any real requests for any bases other than 2, 8, 10, or 16. Regards, Pat From pmaupin at gmail.com Fri Mar 16 03:53:56 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Thu, 15 Mar 2007 21:53:56 -0500 Subject: [Python-3000] Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> Message-ID: On 3/15/07, Guido van Rossum wrote: > Please. No discussion of generalized radixes. I agree. I was just stating the obvious that, if we allow 16r100, somebody will ask why not 21r100. > > Also, IMO Tim's proposal changes too much; there's nothing wrong with 0b and 0x. 0b and 0x it is. Pat From g.brandl at gmx.net Fri Mar 16 08:26:20 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 16 Mar 2007 08:26:20 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45F9D743.9090107@canterbury.ac.nz> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> Message-ID: Greg Ewing schrieb: > Guido van Rossum wrote: >> I'm >> somewhat uncomfortable with 0o because the zero in some fonts is about >> the same size as a lowercase oh; > > You already have to be careful what font you use for > programming with, so I don't think that's a serious > problem. I agree. > More serious is that if we want to be consistent with > 0x and allow upper case, we get 0O, which looks quite > confusing in just about any font. Hm, I don't think there are too many people who write 0XABC today. As discussed earlier, you lose the "short" x. Georg From ncoghlan at gmail.com Fri Mar 16 09:32:22 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 16 Mar 2007 18:32:22 +1000 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> Message-ID: <45FA5616.5050005@gmail.com> Georg Brandl wrote: > Greg Ewing schrieb: >> More serious is that if we want to be consistent with >> 0x and allow upper case, we get 0O, which looks quite >> confusing in just about any font. > > Hm, I don't think there are too many people who write 0XABC today. > As discussed earlier, you lose the "short" x. There aren't many people that write UR"WTF?" either, but odd capitalisation is still legal syntax that can't be ignored completely when making changes. Compare: 0t755 0T755 0o755 0O755 0c755 0C755 0c755 is looking like the best candidate IMO (best looking lowercase, least bad looking uppercase) There's also a nice coincidental pattern that emerges from that option: 0b for Binary 0c for oCtal 0x for heXadecimal Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From apt.shansen at gmail.com Fri Mar 16 10:24:28 2007 From: apt.shansen at gmail.com (Stephen Hansen) Date: Fri, 16 Mar 2007 02:24:28 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45FA5616.5050005@gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> <45FA5616.5050005@gmail.com> Message-ID: <7a9c25c20703160224k2cde9450t17e7d519956cc4e6@mail.gmail.com> > 0c755 is looking like the best candidate IMO (best looking lowercase, > least bad looking uppercase) So agree. :) Please? -- Stephen Hansen Development Advanced Prepress Technology shansen at advpubtech.com (818) 748-9282 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/b8a2847b/attachment.html From rasky at develer.com Fri Mar 16 11:07:59 2007 From: rasky at develer.com (Giovanni Bajo) Date: Fri, 16 Mar 2007 11:07:59 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: <45F92009.9060500@v.loewis.de> References: <45F92009.9060500@v.loewis.de> Message-ID: On 15/03/2007 11.29, Martin v. L?wis wrote: > What is the status of I18N identifiers? Has any progress been made? > If not, I'd like to write a PEP, and mentor a SoC student to work on > that (if one shows up). Is there any agreement over this feature in the first place? FWIW I am a strong -1 on this. I don't want to start downloading packages I can't use because people have started using characters not on my keyboard. There have been strong debates about adding a similar feature to C99 and to GCC, and there are strong opposition about allowing it in common coding conventions (eg: FSF's). Do we want to start going down that route? -- Giovanni Bajo From p.f.moore at gmail.com Fri Mar 16 11:44:26 2007 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 16 Mar 2007 10:44:26 +0000 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45FA5616.5050005@gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> <45FA5616.5050005@gmail.com> Message-ID: <79990c6b0703160344y4fdf9c0cxe19e5ff393dbba6e@mail.gmail.com> On 16/03/07, Nick Coghlan wrote: > There aren't many people that write UR"WTF?" either, but odd > capitalisation is still legal syntax that can't be ignored completely > when making changes. > > Compare: > > 0t755 0T755 > 0o755 0O755 > 0c755 0C755 > > 0c755 is looking like the best candidate IMO (best looking lowercase, > least bad looking uppercase) Rather than agonise about uppercase, which is very infrequently used, why not just remove the option and only allow lowercase? 2to3 could easily convert. If capitals aren't allowed, I prefer 0o755 - justifications for any other letter are frankly pretty thin. Damn, I swore not to get involved in bikeshed discussions. I don't actually use octal constants, so my opinion isn't worth much. Paul. From martin at v.loewis.de Fri Mar 16 14:07:57 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 Mar 2007 14:07:57 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: References: <45F92009.9060500@v.loewis.de> Message-ID: <45FA96AD.6050706@v.loewis.de> Giovanni Bajo schrieb: >> What is the status of I18N identifiers? Has any progress been made? >> If not, I'd like to write a PEP, and mentor a SoC student to work on >> that (if one shows up). > > Is there any agreement over this feature in the first place? This counter-question may be an answer to my original question. If there is no consensus, it is unlikely that anything had been done. I won't discuss the specific concerns now; a PEP needs to be written first. Regards, Martin From thomas at python.org Fri Mar 16 14:27:28 2007 From: thomas at python.org (Thomas Wouters) Date: Fri, 16 Mar 2007 14:27:28 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: <45FA96AD.6050706@v.loewis.de> References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> Message-ID: <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> On 3/16/07, "Martin v. L?wis" wrote: > > Giovanni Bajo schrieb: > >> What is the status of I18N identifiers? Has any progress been made? > >> If not, I'd like to write a PEP, and mentor a SoC student to work on > >> that (if one shows up). > > > > Is there any agreement over this feature in the first place? > > This counter-question may be an answer to my original question. If there > is no consensus, it is unlikely that anything had been done. > > I won't discuss the specific concerns now; a PEP needs to be written > first. My recollection is that Guido said 'no' to non-ASCII identifiers (although that may be wishful thinking on my part.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/dd176aad/attachment.htm From phd at phd.pp.ru Fri Mar 16 14:40:41 2007 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 16 Mar 2007 16:40:41 +0300 Subject: [Python-3000] I18N identifiers In-Reply-To: <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> Message-ID: <20070316134041.GA13200@phd.pp.ru> On Fri, Mar 16, 2007 at 02:27:28PM +0100, Thomas Wouters wrote: > My recollection is that Guido said 'no' to non-ASCII identifiers (although > that may be wishful thinking on my part.) I certainly remember him saying that. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From g.brandl at gmx.net Fri Mar 16 15:14:17 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 16 Mar 2007 15:14:17 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: <20070316134041.GA13200@phd.pp.ru> References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> <20070316134041.GA13200@phd.pp.ru> Message-ID: Oleg Broytmann schrieb: > On Fri, Mar 16, 2007 at 02:27:28PM +0100, Thomas Wouters wrote: >> My recollection is that Guido said 'no' to non-ASCII identifiers (although >> that may be wishful thinking on my part.) > > I certainly remember him saying that. Me too! ;) Perhaps we can even get him to agree to adding that to PEP 3099. Georg From theller at ctypes.org Fri Mar 16 15:36:03 2007 From: theller at ctypes.org (Thomas Heller) Date: Fri, 16 Mar 2007 15:36:03 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: <45FA96AD.6050706@v.loewis.de> References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> Message-ID: Martin v. L?wis schrieb: > Giovanni Bajo schrieb: >>> What is the status of I18N identifiers? Has any progress been made? >>> If not, I'd like to write a PEP, and mentor a SoC student to work on >>> that (if one shows up). >> >> Is there any agreement over this feature in the first place? > > This counter-question may be an answer to my original question. If there > is no consensus, it is unlikely that anything had been done. > > I won't discuss the specific concerns now; a PEP needs to be written first. Since people have voiced their opinions, let me add mine: As a professional programmer I would never use I18N identifiers. OTOH, as native German and father of some children I have a hard time explaining to them why they cannot have a variable named 'B?cher' if they are writing a program that counts books, for example ;-). Thomas From shansen at advpubtech.com Fri Mar 16 17:42:32 2007 From: shansen at advpubtech.com (Stephen Hansen) Date: Fri, 16 Mar 2007 09:42:32 -0700 Subject: [Python-3000] I18N identifiers In-Reply-To: References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> <20070316134041.GA13200@phd.pp.ru> Message-ID: <7a9c25c20703160942ne6aeccdibc59b63f366206@mail.gmail.com> It's already there :) | Python 3000 source code won't use non-ASCII Unicode characters for anything except string literals or comments. | Thread: sets in P3K? http://mail.python.org/pipermail/python-3000/2006-April/001474.html On 3/16/07, Georg Brandl wrote: > > Oleg Broytmann schrieb: > > On Fri, Mar 16, 2007 at 02:27:28PM +0100, Thomas Wouters wrote: > >> My recollection is that Guido said 'no' to non-ASCII identifiers > (although > >> that may be wishful thinking on my part.) > > > > I certainly remember him saying that. > > Me too! ;) > > Perhaps we can even get him to agree to adding that to PEP 3099. > > Georg > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/b016003f/attachment.htm From g.brandl at gmx.net Fri Mar 16 17:49:57 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 16 Mar 2007 17:49:57 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: <7a9c25c20703160942ne6aeccdibc59b63f366206@mail.gmail.com> References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> <20070316134041.GA13200@phd.pp.ru> <7a9c25c20703160942ne6aeccdibc59b63f366206@mail.gmail.com> Message-ID: Stephen Hansen schrieb: > It's already there :) argh -- so what are we discussing here anyway? ;) Georg From jimjjewett at gmail.com Fri Mar 16 18:38:00 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 16 Mar 2007 13:38:00 -0400 Subject: [Python-3000] Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> <45F9E730.5020507@canterbury.ac.nz> Message-ID: On 3/15/07, Patrick Maupin wrote: > I have not seen any real > requests for any bases other than 2, 8, 10, or 16. What is the real need for 8? The only use I've seen on the list is unix file permissions. Even if that weren't obscure enough to stick in a module, I would still rather see chmod g+x than something that requires a man page for translation. FWIW, I have actually used octal literals (in C) for an old wire protocol. It did conform with the surrounding code, but ... that didn't make it readable, either to me or to the usual maintainers. This oddball code wasn't the only thing they had to maintain, and they were young enough (mostly born in the late 1960s) to sometimes forget the "leading 0 <==> octal" rule. -jJ From apt.shansen at gmail.com Fri Mar 16 19:09:47 2007 From: apt.shansen at gmail.com (Stephen Hansen) Date: Fri, 16 Mar 2007 11:09:47 -0700 Subject: [Python-3000] Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> <45F9E730.5020507@canterbury.ac.nz> Message-ID: <7a9c25c20703161109k458cc070g2f544ffe94e766d9@mail.gmail.com> On 3/16/07, Jim Jewett wrote: > > On 3/15/07, Patrick Maupin wrote: > > I have not seen any real > > requests for any bases other than 2, 8, 10, or 16. > > What is the real need for 8? Legacy protocols and such is my use case... and the benefit is not so much "in python", as 0o755 is not anymore readable then 493, but the real benefit of octal literals comes into play when you're comparing your code not to other code but to external, horrifying documentation written for the octal based protocol that has been around forever and will be around forever.. Python is as great tool to wrap around and glue these legacy things that we're forced to support into the modern age; and octal literals help. They aren't essential, but they help. :) --S, who is far too talkative lately. I'm supposed to be a lurker :P -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/7a8e8a1f/attachment.htm From martin at v.loewis.de Fri Mar 16 22:58:49 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 16 Mar 2007 22:58:49 +0100 Subject: [Python-3000] I18N identifiers In-Reply-To: References: <45F92009.9060500@v.loewis.de> <45FA96AD.6050706@v.loewis.de> <9e804ac0703160627l6ca6995by121c54210c3654db@mail.gmail.com> <20070316134041.GA13200@phd.pp.ru> <7a9c25c20703160942ne6aeccdibc59b63f366206@mail.gmail.com> Message-ID: <45FB1319.6090701@v.loewis.de> Georg Brandl schrieb: > Stephen Hansen schrieb: >> It's already there :) > > argh -- so what are we discussing here anyway? > > ;) Georg Nothing - I just wanted to ask what the status is. I still have to write that PEP yet. Regards, Martin From pje at telecommunity.com Sat Mar 17 00:24:43 2007 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 16 Mar 2007 18:24:43 -0500 Subject: [Python-3000] Misc. thoughts & questions re: xreload Message-ID: <5.1.1.6.0.20070316181420.02ab0430@sparrow.telecommunity.com> This is probably one of those cases where I've missed something obvious, but shouldn't it be sufficient for xreload() to use reload() to perform the central step of running the code in the cleared module dictionary? There doesn't seem to be any reason to emulate all that reload() machinery, and it would then work for any and all importers (not to mention chopping out a bunch of code). I'm also not clear on why the module dictionary even needs to be cleared, though it seems reasonable enough. However, *not* clearing it would allow modules to be written so as to accomodate not clearing global data structures during reloading, by checking whether the structure already exists. Btw, the _update() function looks like a great place to use that wonderful generic function prototype of Guido's, since it would allow developers to add extra reloading support for specialty objects, such as say, interfaces. ;-) From avassalotti at acm.org Sat Mar 17 00:27:58 2007 From: avassalotti at acm.org (Alexandre Vassalotti) Date: Fri, 16 Mar 2007 19:27:58 -0400 Subject: [Python-3000] Octal In-Reply-To: <7a9c25c20703161109k458cc070g2f544ffe94e766d9@mail.gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <1f7befae0703151326l2fe75770u7ea45dfb5a9fb440@mail.gmail.com> <45F9E730.5020507@canterbury.ac.nz> <7a9c25c20703161109k458cc070g2f544ffe94e766d9@mail.gmail.com> Message-ID: I just want to mention that the issue, about removing octal literals, has been discussed before: http://www.python.org/dev/summary/2006-02-01_2006-02-15/#id10 -- Alexandre From guido at python.org Sat Mar 17 00:39:44 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 16 Mar 2007 16:39:44 -0700 Subject: [Python-3000] Misc. thoughts & questions re: xreload In-Reply-To: <5.1.1.6.0.20070316181420.02ab0430@sparrow.telecommunity.com> References: <5.1.1.6.0.20070316181420.02ab0430@sparrow.telecommunity.com> Message-ID: You missed that it's a work in progress. I'll come back to it at some point. On 3/16/07, Phillip J. Eby wrote: > This is probably one of those cases where I've missed something obvious, > but shouldn't it be sufficient for xreload() to use reload() to perform the > central step of running the code in the cleared module dictionary? > > There doesn't seem to be any reason to emulate all that reload() machinery, > and it would then work for any and all importers (not to mention chopping > out a bunch of code). I'm also not clear on why the module dictionary even > needs to be cleared, though it seems reasonable enough. However, *not* > clearing it would allow modules to be written so as to accomodate not > clearing global data structures during reloading, by checking whether the > structure already exists. > > Btw, the _update() function looks like a great place to use that wonderful > generic function prototype of Guido's, since it would allow developers to > add extra reloading support for specialty objects, such as say, interfaces. ;-) > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Sat Mar 17 00:52:11 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 17 Mar 2007 12:52:11 +1300 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45FA5616.5050005@gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> <45FA5616.5050005@gmail.com> Message-ID: <45FB2DAB.3040104@canterbury.ac.nz> Nick Coghlan wrote: > 0c755 is looking like the best candidate IMO (best looking lowercase, > least bad looking uppercase) A disadvantage of 0c is that in string formatting, 'c' means 'character', confusing things slightly. -- Greg From jimjjewett at gmail.com Sat Mar 17 00:55:04 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 16 Mar 2007 19:55:04 -0400 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: References: Message-ID: On 3/12/07, Patrick Maupin wrote: > Feature: Alternate syntaxes for escape to markup. I suggest keeping this for templates, or at most the library, because ... > using decorator-style markup, e.g. {@syntax1} inside the string, This is effectively a command switch. It isn't part of the string's data (it shouldn't display) or the template (it isn't replaced). It is just metadata -- and by the time you need that, the complexity is probably enough to justify using a Template. I also don't think you can improve on this, except possibly with a new prefix, like $" ... " # ${} _" ... " # space-replace T" ... " # sugar for string.Template(" ... ") with % mapped to safe_substitute Each of those look OK on their own, but I'm not sure they're important enough to justify the extra complexity in the language as a whole. (And if the T prefix already existed, I'm not sure you would have felt as motivated to do this extension.) > Feature: Automatic search of locals() and globals() for name lookups > if no parameters are given. I do like the search of locals(). It feels more like a guilty pleasure than a good idea, though. Maybe if it were phrased as a call? print "At round {count} the value is {myvar}"() Still feels more like a guilty pleasure than a good idea. > Feature: Ability to pass in a dictionary or tuple of dictionaries of > namespaces to search. I would like this even if nothing else were adopted. But do you mean "sequence of mappings", or exactly a tuple? If you mean exactly a tuple, that needs to be called out clearly in the docstring as well as the docs. If you mean any sequence, then you get the same nasty "Is it a mapping or a sequence of mappings?" problem that requires special-casing for singleton tuples today. > Feature: Placement of a dummy record on the traceback stack for > underlying errors. ... > The text associated with these internally generated ValueError > exceptions will indicate the location of the exception inside the > format string, as well as the nature of the exception. Yes, please. > There is currently no general mechanism for non-Python source code to > be added to a traceback (which might be the subject of another PEP), Yes, please. :D > Feature: Addition of functions and "constants" to string module. Yes. Though once you're importing a module, the extra barrier to templates gets smaller still. > Feature: Ability for "field hook" user code function to only be called > on some fields. So the generic specifiers would change from [[fill]align][sign][width][.precision][type] to [hook][[fill]align][sign][width][.precision][type] where hook is some constant that won't look like a type? (Maybe "h:") > Changed feature: By default, not using all arguments is not an exception I think that not using all *numeric* arguments should be an exception. I'll grant that the case is weaker than with normal %. (If you really want to skip some anyhow, you can still use the "%.0s" trick to at least make it clear that something is being skipped.) > Feature: Ability to insert non-printing comments in format strings I like it. But as with the syntax switch, I wonder if it shouldn't be added to Templates instead. Simply naming things gains most of the value of comments, and beyond that ... maybe it would encourage overly long strings. > Feature: Exception raised if attribute with leading underscore accessed. Sounds good. > Feature: Support for "center" alignment. > The field specifier uses "<" and ">" for left and right alignment. > This adds "^" for center alignment. "^" may be too strongly linked with "start of line" or "not" from regexes. I've seen other things (usually "|") used in other contexts. (Of course, that has its own problems, so "^" may still be a better choice.) -jJ From greg.ewing at canterbury.ac.nz Sat Mar 17 01:02:35 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 17 Mar 2007 13:02:35 +1300 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <79990c6b0703160344y4fdf9c0cxe19e5ff393dbba6e@mail.gmail.com> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> <45FA5616.5050005@gmail.com> <79990c6b0703160344y4fdf9c0cxe19e5ff393dbba6e@mail.gmail.com> Message-ID: <45FB301B.7030400@canterbury.ac.nz> Paul Moore wrote: > Rather than agonise about uppercase, which is very infrequently used, > why not just remove the option and only allow lowercase? That's actually a pretty good idea. Identifiers and keywords are already case-sensitive, so it's hard to see why these shouldn't be. For consistency, do we apply this to string prefixes too? -- Greg From jimjjewett at gmail.com Sat Mar 17 01:09:49 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 16 Mar 2007 20:09:49 -0400 Subject: [Python-3000] Proposed changes to PEP3101 advanced string formatting -- please discuss and vote! In-Reply-To: <20070313160221.FB77.JCARLSON@uci.edu> References: <20070313160221.FB77.JCARLSON@uci.edu> Message-ID: On 3/13/07, Josiah Carlson wrote: > > "Patrick Maupin" wrote: > > Feature: Alternate syntaxes for escape to markup. > > this method "{foo}" escapes to markup, but when there is whitespace > > after the leading "{", e.g. "{ foo}", the brace is not an escape to > > markup. If the whitespace is a space, it is removed from the output, > > but if it is '\r', '\n', or '\t', then it is left in the output. I can see it being very useful. But if I'm using strings that long, and ready to be picky about whitespace -- I'm ready to import a module. To me, this looks like a good example Template extension. > can't help but think that %var% would be a better alternate explicit Some of the same documents that have lots of "{" characters will have lots of "%". Again, Template will let you change the identifier character quite easily. > > Feature: Ability to insert non-printing comments in format strings > The user can use the parser/compiler to get this behavior for free. > (" text " #your comment here > "more text").format(...) If the only alternative were really to push more on eval and to obfuscate code (did you *mean* to stick those together, or did you forget a comma?) ... I would support this. Since we do already have String.template, maybe the answer is to somehow promote it. -jJ From guido at python.org Sat Mar 17 01:13:00 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 16 Mar 2007 17:13:00 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45FB301B.7030400@canterbury.ac.nz> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> <45FA5616.5050005@gmail.com> <79990c6b0703160344y4fdf9c0cxe19e5ff393dbba6e@mail.gmail.com> <45FB301B.7030400@canterbury.ac.nz> Message-ID: +1 On 3/16/07, Greg Ewing wrote: > Paul Moore wrote: > > > Rather than agonise about uppercase, which is very infrequently used, > > why not just remove the option and only allow lowercase? > > That's actually a pretty good idea. Identifiers and keywords > are already case-sensitive, so it's hard to see why these > shouldn't be. > > For consistency, do we apply this to string prefixes too? > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Sat Mar 17 01:13:42 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Fri, 16 Mar 2007 19:13:42 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45FB301B.7030400@canterbury.ac.nz> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> <45FA5616.5050005@gmail.com> <79990c6b0703160344y4fdf9c0cxe19e5ff393dbba6e@mail.gmail.com> <45FB301B.7030400@canterbury.ac.nz> Message-ID: On 3/16/07, Greg Ewing wrote: > Paul Moore wrote: > > > Rather than agonise about uppercase, which is very infrequently used, > > why not just remove the option and only allow lowercase? > > That's actually a pretty good idea. Identifiers and keywords > are already case-sensitive, so it's hard to see why these > shouldn't be. > > For consistency, do we apply this to string prefixes too? Actually, sometimes people use upper-case hexadecimal; sometimes lower, so for output formatting, we need 'x' and 'X'. Obviously, that isn't an issue with octal :) Regards, Pat From python at rcn.com Sat Mar 17 01:22:17 2007 From: python at rcn.com (Raymond Hettinger) Date: Fri, 16 Mar 2007 20:22:17 -0400 (EDT) Subject: [Python-3000] Fwd: Re: Octal Message-ID: <20070316202217.BBS36204@ms09.lnh.mail.rcn.net> [Alexandre Vassalotti] >I just want to mention that the issue, about removing octal literals, >has been discussed before: >http://www.python.org/dev/summary/2006-02-01_2006-02-15/#id10 LOL. Would please calendar this so that twelve months from now we can re-propose removing octal literals ;-) I didn't re-read the whole thread so I didn't find-out how we still have octal literal after Guido suggested they be removed. Raymond From bjourne at gmail.com Sat Mar 17 02:26:12 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Sat, 17 Mar 2007 01:26:12 +0000 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: <45F9D743.9090107@canterbury.ac.nz> References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> <45F9D743.9090107@canterbury.ac.nz> Message-ID: <740c3aec0703161826tfcdc062rf54576e6fa85c6d8@mail.gmail.com> On 3/15/07, Greg Ewing wrote: > More serious is that if we want to be consistent with > 0x and allow upper case, we get 0O, which looks quite > confusing in just about any font. OR you could ban X from hex and B from binary. I have not seen a good reason why 0X0DE and 0B10 needs to be allowed when 0x0DE and 0b10 looks much nicer. -- mvh Bj?rn From monpublic at gmail.com Sat Mar 17 03:32:57 2007 From: monpublic at gmail.com (CM) Date: Fri, 16 Mar 2007 22:32:57 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/15/07, Georg Brandl wrote: > > Thomas Wouters schrieb: > > > > > > On 3/15/07, *Raymond Hettinger* > > > wrote: > > > > My worry focuses on new-timers and SyntaxErrors. The worst part of > > the new user experience is encountering a SyntaxError. > > > > > > Please. I already said it would be a SyntaxError *with explanation*. > > It's pretty easy to make a syntax error that goes "Did you mean 0t6065 > > or 6065?". This would not be a syntax error handled by the parser, the > > tokenizer really doesn't care about leading 0's and ast.c's parsenumber > > (which would be the spot to raise it) has all the necessary information. > > Until now, I skipped the octal thread. Now I read this and wondered, "what > the > hell is 0t6065 supposed to mean" and only from the context I gathered that > it > would be an octal literal... > > Seriously, nobody, even coming from another language, will be able to look > at it > and say, "yes, that's an octal literal." I agree wholeheartedly with this. The argument that 'x' is the third character of 'hexadecimal', so 't' would have some kind of nice symmetry doesn't hold. One of the reasons that 'x' is such a nice label to remember is that its *pronunciation* (at least in English) matches that of the word 'hex'. The letter 't' has no real resemblance to the word 'octal', except that it happens to be a letter in the word. Then again, we all had to learn that '0x' meant 'hex' once, so maybe 't' isn't that big of a stretch. I do think that addressing the pronunciation would be a good idea, though. I wonder if the '0' prefix was meant to be pronounced "aught" in the beginning.... :-) - C Georg > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/monpublic%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/6ca4f2d7/attachment.htm From monpublic at gmail.com Sat Mar 17 03:47:22 2007 From: monpublic at gmail.com (CM) Date: Fri, 16 Mar 2007 22:47:22 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Oops, posted from the wrong email address (sorry Georg): To switch this into a positive idea from a negative one, I am -1 on using 't' -.5 on having octal literals at all, since int('0755',8) does the job nicely when needed +1 on 'o' if we must have them in the language, since the first letter of the radix name is easier to grok than an obscure third letter that doesn't even have the pronunciation excuse that 'x' carries in English. - C On 3/16/07, CM wrote: > > > > On 3/15/07, Georg Brandl wrote: > > > > Thomas Wouters schrieb: > > > > > > > > > On 3/15/07, *Raymond Hettinger* > >> > > > wrote: > > > > > > My worry focuses on new-timers and SyntaxErrors. The worst part of > > > the new user experience is encountering a SyntaxError. > > > > > > > > > Please. I already said it would be a SyntaxError *with explanation*. > > > It's pretty easy to make a syntax error that goes "Did you mean 0t6065 > > > or 6065?". This would not be a syntax error handled by the parser, the > > > tokenizer really doesn't care about leading 0's and ast.c's > > parsenumber > > > (which would be the spot to raise it) has all the necessary > > information. > > > > Until now, I skipped the octal thread. Now I read this and wondered, > > "what the > > hell is 0t6065 supposed to mean" and only from the context I gathered > > that it > > would be an octal literal... > > > > Seriously, nobody, even coming from another language, will be able to > > look at it > > and say, "yes, that's an octal literal." > > > I agree wholeheartedly with this. The argument that 'x' is the third > character of 'hexadecimal', so 't' would have some kind of nice symmetry > doesn't hold. One of the reasons that 'x' is such a nice label to remember > is that its *pronunciation* (at least in English) matches that of the word > 'hex'. The letter 't' has no real resemblance to the word 'octal', except > that it happens to be a letter in the word. > > Then again, we all had to learn that '0x' meant 'hex' once, so maybe 't' > isn't that big of a stretch. I do think that addressing the pronunciation > would be a good idea, though. I wonder if the '0' prefix was meant to be > pronounced "aught" in the beginning.... > > :-) > > - C > > > > > Georg > > > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: > > http://mail.python.org/mailman/options/python-3000/monpublic%40gmail.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/3271c6c3/attachment.html From guido at python.org Sat Mar 17 04:08:21 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 16 Mar 2007 20:08:21 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Um, I like to know who's speaking. "CM", "monpublic", "C" don't mean much to me. Have we met? Do you have a real name? Dos anyone here know you? (Georg perhaps?) While in general this isn't a forum where we count votes much, *anonymous* votes really don't count for much at all. --Guido On 3/16/07, CM wrote: > Oops, posted from the wrong email address (sorry Georg): > > To switch this into a positive idea from a negative one, I am > > -1 on using 't' > -.5 on having octal literals at all, since int('0755',8) does the job nicely > when needed > +1 on 'o' if we must have them in the language, since the first letter of > the radix name is easier to grok than an obscure third letter that doesn't > even have the pronunciation excuse that 'x' carries in English. > > - C -- --Guido van Rossum (home page: http://www.python.org/~guido/) From monpublic at gmail.com Sat Mar 17 04:10:29 2007 From: monpublic at gmail.com (CM) Date: Fri, 16 Mar 2007 23:10:29 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, Guido van Rossum wrote: > > Um, I like to know who's speaking. "CM", "monpublic", "C" don't mean > much to me. Have we met? Do you have a real name? Dos anyone here know > you? (Georg perhaps?) While in general this isn't a forum where we > count votes much, *anonymous* votes really don't count for much at > all. I'm sorry about that. I have been lurking for a long time, for some reason under this mailing address, and was forcibly reminded about that fact when I got a moderator message when posting from a more reasonable address. The name's Chris Monson, and we have definitely met. I'll fix my mail settings. - C --Guido > > On 3/16/07, CM wrote: > > Oops, posted from the wrong email address (sorry Georg): > > > > To switch this into a positive idea from a negative one, I am > > > > -1 on using 't' > > -.5 on having octal literals at all, since int('0755',8) does the job > nicely > > when needed > > +1 on 'o' if we must have them in the language, since the first letter > of > > the radix name is easier to grok than an obscure third letter that > doesn't > > even have the pronunciation excuse that 'x' carries in English. > > > > - C > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/0d2811d7/attachment.htm From guido at python.org Sat Mar 17 04:49:50 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 16 Mar 2007 20:49:50 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, CM wrote: > I'm sorry about that. I have been lurking for a long time, for some reason > under this mailing address, and was forcibly reminded about that fact when I > got a moderator message when posting from a more reasonable address. The > name's Chris Monson, and we have definitely met. I'll fix my mail settings. Cool. Then I'll support your support for 0o, and only using lowercase 0b, 0o, 0x. Maybe that'll settle the discussion about this bikeshed's color. I'm also in favor of only using lowercase j for complex and lowercase r for raw strings. (u will disappear with the Unicode/str unification.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From monpublic at gmail.com Sat Mar 17 05:06:32 2007 From: monpublic at gmail.com (Chris Monson) Date: Sat, 17 Mar 2007 00:06:32 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: I didn't see anyone stepping forward to write a PEP for this. If one is needed, I'll volunteer to do it. - C On 3/16/07, Guido van Rossum wrote: > > On 3/16/07, CM wrote: > > I'm sorry about that. I have been lurking for a long time, for some > reason > > under this mailing address, and was forcibly reminded about that fact > when I > > got a moderator message when posting from a more reasonable > address. The > > name's Chris Monson, and we have definitely met. I'll fix my mail > settings. > > Cool. Then I'll support your support for 0o, and only using lowercase > 0b, 0o, 0x. Maybe that'll settle the discussion about this bikeshed's > color. I'm also in favor of only using lowercase j for complex and > lowercase r for raw strings. (u will disappear with the Unicode/str > unification.) > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070317/9a151f44/attachment.html From pmaupin at gmail.com Sat Mar 17 05:22:47 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Fri, 16 Mar 2007 23:22:47 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, Chris Monson wrote: > I didn't see anyone stepping forward to write a PEP for this. If one is > needed, I'll volunteer to do it. I'm working on it! I will probably submit it tomorrow. Regards, Pat From pmaupin at gmail.com Sat Mar 17 05:40:24 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Fri, 16 Mar 2007 23:40:24 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, Guido van Rossum wrote: > Cool. Then I'll support your support for 0o, and only using lowercase > 0b, 0o, 0x. Maybe that'll settle the discussion about this bikeshed's > color. I'm also in favor of only using lowercase j for complex and > lowercase r for raw strings. (u will disappear with the Unicode/str > unification.) In general, I support this, and weary of discussing and documenting the exact shade of battleship grey we have decided upon. However, I need to ask about one blue racing stripe along the left-hand side :) Hexadecimal numbers traditionally support upper- and lower-case display, chosen by the case of the 'x' character. Python supports this both for input and display, and probably needs to continue to support the option of upper or lowercase hex for display. I think perhaps it should be supported for input, as well, both to have symmetry with the display, and to allow for easy cut and paste (and eval() and int(s, 0)) of hexadecimal numbers from other sources. Obviously, we cannot support this easy eval() or int(s, 0) for octal in the future, but arguably, there are a lot more hexadecimal numbers of either case "in the wild" than there are octal numbers. I understand that the proposal is to allow 0xabc or 0xABC, but not allow 0Xabc or 0XABC, but one of the strengths of Python is that I can easily parse files containing numerical data from almost arbitrary sources, and this may be a case where "practicality beats purity" should rule, although it may also be an easy counterargument that I/O symmetry is not really that important, and that one can easily write "x = x.replace('0X', '0x')" for the example I have given. Your call. Regards, Pat From g.brandl at gmx.net Sat Mar 17 09:57:43 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 17 Mar 2007 09:57:43 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Patrick Maupin schrieb: > On 3/16/07, Guido van Rossum wrote: >> Cool. Then I'll support your support for 0o, and only using lowercase >> 0b, 0o, 0x. Maybe that'll settle the discussion about this bikeshed's >> color. I'm also in favor of only using lowercase j for complex and >> lowercase r for raw strings. (u will disappear with the Unicode/str >> unification.) > > In general, I support this, and weary of discussing and documenting > the exact shade of battleship grey we have decided upon. > > However, I need to ask about one blue racing stripe along the left-hand side :) > > Hexadecimal numbers traditionally support upper- and lower-case > display, chosen by the case of the 'x' character. Python supports > this both for input and display, and probably needs to continue to > support the option of upper or lowercase hex for display. > > I think perhaps it should be supported for input, as well, both to > have symmetry with the display, and to allow for easy cut and paste > (and eval() and int(s, 0)) of hexadecimal numbers from other sources. > > Obviously, we cannot support this easy eval() or int(s, 0) for octal > in the future, but arguably, there are a lot more hexadecimal numbers > of either case "in the wild" than there are octal numbers. > > I understand that the proposal is to allow 0xabc or 0xABC, but not > allow 0Xabc or 0XABC, but one of the strengths of Python is that I can > easily parse files containing numerical data from almost arbitrary > sources, and this may be a case where "practicality beats purity" > should rule, although it may also be an easy counterargument that I/O > symmetry is not really that important, and that one can easily write > "x = x.replace('0X', '0x')" for the example I have given. Your call. Please no gratuitous breakage! I'd continue to allow 0Xabc and 0xABC and 0xabc and 0XABC. Do a Google codesearch, and you'll find them in Python code as well as data files. I'd not allow 0O or 0B prefixes. I'd not allow an "R" string prefix -- I've never seen anyone use it. Georg From shiblon at gmail.com Sat Mar 17 03:42:01 2007 From: shiblon at gmail.com (Chris Monson) Date: Fri, 16 Mar 2007 22:42:01 -0400 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, CM wrote: > > > > On 3/15/07, Georg Brandl wrote: > > > > Thomas Wouters schrieb: > > > > > > > > > On 3/15/07, *Raymond Hettinger* > >> > > > wrote: > > > > > > My worry focuses on new-timers and SyntaxErrors. The worst part of > > > the new user experience is encountering a SyntaxError. > > > > > > > > > Please. I already said it would be a SyntaxError *with explanation*. > > > It's pretty easy to make a syntax error that goes "Did you mean 0t6065 > > > or 6065?". This would not be a syntax error handled by the parser, the > > > tokenizer really doesn't care about leading 0's and ast.c's > > parsenumber > > > (which would be the spot to raise it) has all the necessary > > information. > > > > Until now, I skipped the octal thread. Now I read this and wondered, > > "what the > > hell is 0t6065 supposed to mean" and only from the context I gathered > > that it > > would be an octal literal... > > > > Seriously, nobody, even coming from another language, will be able to > > look at it > > and say, "yes, that's an octal literal." > > > I agree wholeheartedly with this. The argument that 'x' is the third > character of 'hexadecimal', so 't' would have some kind of nice symmetry > doesn't hold. One of the reasons that 'x' is such a nice label to remember > is that its *pronunciation* (at least in English) matches that of the word > 'hex'. The letter 't' has no real resemblance to the word 'octal', except > that it happens to be a letter in the word. > To switch this into a positive idea from a negative one, I am -1 on using 't' -.5 on having octal literals at all, since int('0755',8) does the job nicely when needed +1 on 'o' if we must have them in the language, since the first letter of the radix name is easier to grok than an obscure third letter that doesn't even have the pronunciation excuse that 'x' carries in English. - C Then again, we all had to learn that '0x' meant 'hex' once, so maybe 't' > isn't that big of a stretch. I do think that addressing the pronunciation > would be a good idea, though. I wonder if the '0' prefix was meant to be > pronounced "aught" in the beginning.... > > :-) > > - C > > > > > Georg > > > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: > > http://mail.python.org/mailman/options/python-3000/monpublic%40gmail.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070316/35a499c7/attachment.htm From guido at python.org Sat Mar 17 16:22:24 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 17 Mar 2007 08:22:24 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, Patrick Maupin wrote: > Hexadecimal numbers traditionally support upper- and lower-case > display, chosen by the case of the 'x' character. Python supports > this both for input and display, and probably needs to continue to > support the option of upper or lowercase hex for display. > > I think perhaps it should be supported for input, as well, both to > have symmetry with the display, and to allow for easy cut and paste > (and eval() and int(s, 0)) of hexadecimal numbers from other sources. > > Obviously, we cannot support this easy eval() or int(s, 0) for octal > in the future, but arguably, there are a lot more hexadecimal numbers > of either case "in the wild" than there are octal numbers. > > I understand that the proposal is to allow 0xabc or 0xABC, but not > allow 0Xabc or 0XABC, but one of the strengths of Python is that I can > easily parse files containing numerical data from almost arbitrary > sources, and this may be a case where "practicality beats purity" > should rule, although it may also be an easy counterargument that I/O > symmetry is not really that important, and that one can easily write > "x = x.replace('0X', '0x')" for the example I have given. Your call. I'll take your word that 0X is popular for data files out there. I think I would support that in int(s, 0) but not in eval(s) -- IOW, hex literals in Python source code must start with 0x. After that they can use ABC or abc or AbC for all I care. I used Google's code search to find occurrences of 0X followed by a hex digit (and not preceded by a word character) and found only very few hits, many of which were false positives (e.g. in Python's own test suite there are a bunch of occurrences in string literals verifying that %#X produces the right output). Since this is trivial for the conversion tool, I'm not worried about breaking that code at all. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Sat Mar 17 19:51:09 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sat, 17 Mar 2007 13:51:09 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/17/07, Guido van Rossum wrote: > I'll take your word that 0X is popular for data files out there. I > think I would support that in int(s, 0) but not in eval(s) -- IOW, hex > literals in Python source code must start with 0x. After that they can > use ABC or abc or AbC for all I care. I used Google's code search to > find occurrences of 0X followed by a hex digit (and not preceded by a > word character) and found only very few hits, many of which were false > positives (e.g. in Python's own test suite there are a bunch of > occurrences in string literals verifying that %#X produces the right > output). That's a tool I need to remember to learn to use. I can't say that I have seen lots of examples of 0XABC (although I have seen lots of examples of 0ABCH in assembler), so if Google says it doesn't happen, let's remove it. As I pointed out, if someone has data with 0X in it, it's a trivial one-liner to fix the data. So, for input a hex number is a literal lower-case only "0x" followed by hex digits in either upper or lower case, and for input, both "x" and "X" are used, to indicate the case of the desired hex digits. I think that makes for a much simpler explanation of the rules, which means the breakage is not "gratuitous", and as you point out, the conversion tool can easily cope with this. Thanks, Pat From pmaupin at gmail.com Sat Mar 17 20:07:40 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sat, 17 Mar 2007 14:07:40 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, Guido van Rossum wrote: > Cool. Then I'll support your support for 0o, and only using lowercase > 0b, 0o, 0x. Maybe that'll settle the discussion about this bikeshed's > color. I'm also in favor of only using lowercase j for complex and > lowercase r for raw strings. (u will disappear with the Unicode/str > unification.) What about 'e' for exponentiation? Thanks, Pat From guido at python.org Sat Mar 17 20:20:59 2007 From: guido at python.org (Guido van Rossum) Date: Sat, 17 Mar 2007 12:20:59 -0700 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: I'd say the same rule -- lowercase only. On 3/17/07, Patrick Maupin wrote: > On 3/16/07, Guido van Rossum wrote: > > Cool. Then I'll support your support for 0o, and only using lowercase > > 0b, 0o, 0x. Maybe that'll settle the discussion about this bikeshed's > > color. I'm also in favor of only using lowercase j for complex and > > lowercase r for raw strings. (u will disappear with the Unicode/str > > unification.) > > What about 'e' for exponentiation? > > Thanks, > Pat > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pmaupin at gmail.com Sat Mar 17 20:54:19 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sat, 17 Mar 2007 14:54:19 -0500 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: On 3/16/07, Chris Monson wrote: > I didn't see anyone stepping forward to write a PEP for this. If one is > needed, I'll volunteer to do it. Actually, ISTM that changing all string literal special characters ("r", "e", "x", "j", etc.) is probably really a different PEP. I've been focusing on the rationale for having 0123 not be an octal number yet not be a decimal number either, and on the rationale for adding binary but not arbitrary radices, and on the rationale for choosing "o" for octal instead of "c" or "t" or "q" or "w", yet not changing "x" to "h", etc. While only supporting lowercase is one of the key ideas that makes "o" for octal acceptable, I think it's really a different focus for a different PEP, so if you are volunteering to help, feel free to write that up separately. Thanks, Pat From baptiste13 at altern.org Sun Mar 18 03:24:50 2007 From: baptiste13 at altern.org (Baptiste Carvello) Date: Sun, 18 Mar 2007 03:24:50 +0100 Subject: [Python-3000] Fwd: Re: Fwd: Re: Fwd: Re: Octal In-Reply-To: References: <20070314202627.BBL69377@ms09.lnh.mail.rcn.net> <9e804ac0703150139t43b61131o972933154553f832@mail.gmail.com> Message-ID: Guido van Rossum a ?crit : > I'd say the same rule -- lowercase only. > > On 3/17/07, Patrick Maupin wrote: >> What about 'e' for exponentiation? >> > I guess it's already implied, but just for the record: capital E is very popular in data files, so it has to still work with int(). No use in the litteral, though. Baptiste From guido at python.org Sun Mar 18 16:47:50 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 18 Mar 2007 08:47:50 -0700 Subject: [Python-3000] Metaclasses in Python 3000: Draft 2 In-Reply-To: References: <45F7A445.5090507@acm.org> Message-ID: On 3/14/07, Guido van Rossum wrote: > On 3/14/07, Talin wrote: > > PEP: xxx > > Title: Metaclasses in Python 3000 > > Checked in as PEP 3115. I've accepted this PEP and checked in the implementation. The compiler package is currently broken; someone who knows it better may be able to fix it sooner than I can. The docs aren't updated yet. I expec that Collin Winter will produce a transformer that can fix at least the simpler cases of __metaclass__=xxx, and warn about cases it can't transform. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From armin.ronacher at active-4.com Sun Mar 18 20:45:50 2007 From: armin.ronacher at active-4.com (Armin Ronacher) Date: Sun, 18 Mar 2007 19:45:50 +0000 (UTC) Subject: [Python-3000] =?utf-8?b?VHlwZSBfX3JlcHJfXw==?= Message-ID: Hoi, Some time I proposed changing the type __repr__ to the import name. While I still think this is a good idea ;-) I found a small inconsistency in the __repr__ code of PyTypeObject. Currently the repr checks for the Py_TPFLAGS_HEAPTYPE flag to check if it should use "class" or "type" as name for the repr. Now where types and classes are the same i don't think it's a good idea to have different reprs for objects defined in python and for objects defined in the C API. My proposal: change the repr for all objects to "class". So , etc. Regards, Armin From pmaupin at gmail.com Sun Mar 18 21:37:54 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sun, 18 Mar 2007 15:37:54 -0500 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) Message-ID: This is what I think we agreed on :) (except that I think that lowercase-only 'x', 'e', 'r' is a subject for a different PEP) I don't have a PEP number for this yet, but I emailed it into the PEP editors yesterday. I tried to cover the discussion about letting "0123" be a decimal integer, and the discussion about arbitrary bases, very carefully. Please let me know if this is not what was decided or if there is additional information which should be in the PEP. Thanks, Pat Abstract ======== This PEP proposes changes to the Python core to rationalize the treatment of string literal representations of integers in different radices (bases). These changes are targeted at Python 3.0, but the backward-compatible parts of the changes should be added to Python 2.6, so that all valid 3.0 integer literals will also be valid in 2.6. The proposal is that: a) octal literals must now be specified with a leading "0o" instead of "0"; b) binary literals are now supported via a leading "0b"; and c) provision will be made for binary numbers in string formatting. Motivation ========== This PEP was motivated by two different issues: - The default octal representation of integers is silently confusing to people unfamiliar with C-like languages. It is extremely easy to inadvertently create an integer object with the wrong value, because '013' means 'decimal 11', not 'decimal 13', to the Python language itself, which is not the meaning that most humans would assign to this literal. - There is a strong desire for binary support in the language. Specification ============= Grammar specification --------------------- The grammar will be changed. For Python 2.6, the changed and new token definitions will be:: integer ::= decimalinteger | octinteger | hexinteger | bininteger | oldoctinteger octinteger ::= "0" "o" octdigit+ bininteger ::= "0" "b" bindigit+ oldoctinteger ::= "0" octdigit+ bindigit ::= "0" | "1" For Python 3.0, "oldoctinteger" will not be supported, and an exception will be raised if a literal has a leading "0" and a second character which is a digit. For both versions, this will require changes to PyLong_FromString as well as the grammar. The documentation will have to be changed as well: grammar.txt, as well as the integer literal section of the reference manual. PEP 306 should be checked for other issues, and that PEP should be updated if the procedure described therein is insufficient. int() specification -------------------- int(s, 0) will also match the new grammar definition. This should happen automatically with the changes to PyLong_FromString required for the grammar change. Also the documentation for int() should be changed to explain that int(s) operates identically to int(s, 10), and the word "guess" should be removed from the description of int(s, 0). long() specification -------------------- For Python 2.6, the long() implementation and documentation should be changed to reflect the new grammar. Tokenizer exception handling ---------------------------- If an invalid token contains a leading "0", the exception error message should be more informative than the current "SyntaxError: invalid token". It should explain that decimal numbers may not have a leading zero, and that octal numbers require an "o" after the leading zero. int() exception handling ------------------------ The ValueError raised for any call to int() with a string should at least explicitly contain the base in the error message, e.g.:: ValueError: invalid literal for base 8 int(): 09 Output formatting ----------------- The string (and unicode in 2.6) % operator will have a 'b' format specifier added for binary. PEP 3101 already supports 'b' for binary output. Transition from 2.6 to 3.0 --------------------------- The 2to3 translator will have to insert 'o' into any octal string literal. The Py3K compatible option to Python 2.6 should cause attempts to use oldoctinteger literals to raise an exception. Rationale ========= Most of the discussion on these issues occurred on the Python-3000 mailing list starting 14-Mar-2007, prompted by Raymond Hettinger's observation (cleverly couched as a question) that the average human being would be completely mystified upon finding that prepending a "0" to a string of digits changes the meaning of that digit string entirely. As historian Alexandre Vassalotti pointed out during this discussion, a similar, but shorter, discussion on the subject occurred in January of 2006, prompted by a discovery of the same issue noticed by Raymond. Background ---------- For historical reasons, Python's string representation of integers in different bases (radices), for string formatting and token literals, borrows heavily from C. [1]_ [2]_ Usage has shown that the historical method of specifying an octal number is confusing, and also that it would be nice to have additional support for binary literals. Throughout this document, unless otherwise noted, discussions about the string representation of integers relate to these features: - Literal integer tokens, as used by normal module compilation, by eval(), and by int(token, 0). (int(token) and int(token, 2-36) are not modified by this proposal.) * Under 2.6, long() is treated the same as int() - Formatting of integers into strings, either via the % string operator or the new PEP 3101 advanced string formatting method. It is presumed that: - All of these features should have an identical set of supported radices, for consistency. - Python source code syntax and int(mystring, 0) should continue to share identical behavior. Removal of old octal syntax ---------------------------- This PEP proposes that the ability to specify an octal number by using a leading zero will be removed from the language in Python 3.0 (and the Python 3.0 preview mode of 2.6), and that a SyntaxError will be raised whenever a leading "0" is immediately followed by another digit. During the present discussion, it was almost universally agreed that:: eval('010') == 8 should no longer be true, because that is confusing to new users. It was also proposed that:: eval('0010') == 10 should become true, but that is much more contentious, because it is so inconsistent with usage in other computer languages that mistakes are likely to be made. Almost all currently popular computer languages, including C/C++, Java, Perl, and JavaScript, treat a sequence of digits with a leading zero as an octal number. Proponents of treating these numbers as decimal instead have a very valid point -- as discussed in `Supported radices`_, below, the entire non-computer world uses decimal numbers almost exclusively. There is ample anecdotal evidence that many people are dismayed and confused if they are confronted with non-decimal radices. However, in most situations, most people do not write gratuitous zeros in front of their decimal numbers. The primary exception is when an attempt is being made to line up columns of numbers. But since PEP 8 specifically discourages the use of spaces to try to align Python code, one would suspect the same argument should apply to the use of leading zeros for the same purpose. Finally, although the email discussion often focused on whether anybody actually *uses* octal any more, and whether we should cater to those old-timers in any case, that is almost entirely besides the point. Assume the rare complete newcomer to computing who *does*, either occasionally or as a matter of habit, use leading zeros for decimal numbers. Python could either: a) silently do the wrong thing with his numbers, as it does now; b) immediately disabuse him of the notion that this is viable syntax (and yes, the SyntaxWarning should be more gentle than it currently is, but that is a subject for a different PEP); or c) let him continue to think that computers are happy with multi-digit decimal integers which start with "0". Some people passionately believe that (c) is the correct answer, and they would be absolutely right if we could be sure that new users will never blossom and grow and start writing AJAX applications. So while a new Python user may (currently) be mystified at the delayed discovery that his numbers don't work properly, we can fix it by either explaining to him immediately that Python doesn't like leading zeros (hopefully with a reasonable message!), or we can delegate this teaching experience to the JavaScript interpreter in the Internet Explorer browser, and let him try to debug his issue there. Supported radices ----------------- This PEP proposes that the supported radices for the Python language will be 2, 8, 10, and 16. Once it is agreed that the old syntax for octal (radix 8) representation of integers must be removed from the language, the next obvious question is "Do we actually need a way to specify (and display) numbers in octal?" This question is quickly followed by "What radices does the language need to support?" Because computers are so adept at doing what you tell them to, a tempting answer in the discussion was "all of them." This answer has obviously been given before -- the int() constructor will accept an explicit radix with a value between 2 and 36, inclusive, with the latter number bearing a suspicious arithmetic similarity to the sum of the number of numeric digits and the number of same-case letters in the ASCII alphabet. But the Python community has its share of minimalists (each with his own idea of the proper subset), so, for example, Mattias Engdeg?rd wrote: "only decimal, hex and binary constants are of general use at all" while Thomas Wouters opined that octal and hexadecimal should remain but binary was only for misguided students doing their homework. At the end of the day, nobody was vehemently against support of binary, or octal, or hexadecimal, or even decimal, but none of the other radices seemed to have any significant supporters. Humans use other numeric bases constantly. If I tell you that it is 12:30 PM, I have communicated quantitative information arguably composed of *three* separate bases (12, 60, and 2), only one of which is in the "agreed" list above. But the *communication* of that information used two decimal digits each for the base 12 and base 60 information, and, perversely, two letters for information which could have fit in a single decimal digit. So, in general, humans communicate "normal" (non-computer) numerical information either via names (AM, PM, January, ...) or via use of decimal notation. Obviously, names are seldom used for large sets of items, so decimal is used for everything else. There are studies which attempt to explain why this is so, typically reaching the expected conclusion that the Arabic numeral system is well-suited to human cognition. [3]_ There is even support in the history of the design of computers to indicate that decimal notation is the correct way for computers to communicate with humans. One of the first modern computers, ENIAC [4]_ computed in decimal, even though there were already existing computers which operated in binary. Decimal computer operation was important enough that many computers, including the ubiquitous PC, have instructions designed to operate on "binary coded decimal" (BCD) [5]_ , a representation which devotes 4 bits to each decimal digit. These instructions date from a time when the most strenuous calculations ever performed on many numbers were the calculations actually required to perform textual I/O with them. It is possible to display BCD without having to perform a divide/remainder operation on every displayed digit, and this was a huge computational win when most hardware didn't have fast divide capability. Another factor contributing to the use of BCD is that, with BCD calculations, rounding will happen exactly the same way that a human would do it, so BCD is still sometimes used in fields like finance, despite the computational and storage superiority of binary. So, if it weren't for the fact that computers themselves normally use binary for efficient computation and data storage, string representations of integers would probably always be in decimal. Unfortunately, computer hardware doesn't think like humans, so programmers and hardware engineers must often resort to thinking like the computer, which means that it is important for Python to have the ability to communicate binary data in a form that is understandable to humans. The requirement that the binary data notation must be cognitively easy for humans to process means that it should contain an integral number of binary digits (bits) per symbol, while otherwise conforming quite closely to the standard tried-and-true decimal notation (position indicates power, larger magnitude on the left, not too many symbols in the alphabet, etc.). The obvious "sweet spot" for this binary data notation is thus octal, which packs the largest integral number of bits possible into a single symbol chosen from the Arabic numeral alphabet. In fact, some computer architectures, such as the PDP8 and the 8080/Z80, were defined in terms of octal, in the sense of arranging the bitfields of instructions in groups of three, and using octal representations to describe the instruction set. Even today, octal is important because of bit-packed structures which consist of 3 bits per field, such as Unix file permission masks. But octal has a drawback when used for larger numbers. The number of bits per symbol, while integral, is not itself a power of two. This limitation (given that the word size of most computers these days is a power of two) has resulted in hexadecimal, which is more popular than octal despite the fact that it requires a 60% larger alphabet than decimal, because each symbol contains 4 bits. Some numbers, such as Unix file permission masks, are easily decoded by humans when represented in octal, but difficult to decode in hexadecimal, while other numbers are much easier for humans to handle in hexadecimal. Unfortunately, there are also binary numbers used in computers which are not very well communicated in either hexadecimal or octal. Thankfully, fewer people have to deal with these on a regular basis, but on the other hand, this means that several people on the discussion list questioned the wisdom of adding a straight binary representation to Python. One example of where these numbers is very useful is in reading and writing hardware registers. Sometimes hardware designers will eschew human readability and opt for address space efficiency, by packing multiple bit fields into a single hardware register at unaligned bit locations, and it is tedious and error-prone for a human to reconstruct a 5 bit field which consists of the upper 3 bits of one hex digit, and the lower 2 bits of the next hex digit. Even if the ability of Python to communicate binary information to humans is only useful for a small technical subset of the population, it is exactly that population subset which contains most, if not all, members of the Python core team, so even straight binary, the least useful of these notations, has several enthusiastic supporters and few, if any, staunch opponents, among the Python community. Syntax for supported radices ----------------------------- This proposal is to to use a "0o" prefix with a lowercase "o" for octal, and a "0b" prefix with a lowercase "b" for binary. The syntax for delimiting the different radices received a lot of attention in the discussion on Python-3000. There are several (sometimes conflicting) requirements and "nice-to-haves" for this syntax: - It should be as compatible with other languages and previous versions of Python as is reasonable, both for the input syntax and for the output (e.g. string % operator) syntax. - It should be as obvious to the casual observer as possible. - It should be easy to visually distinguish integers formatted in the different bases. Proposed syntaxes included things like arbitrary radix prefixes, such as 16r100 (256 in hexadecimal), and radix suffixes, similar to the 100h assembler-style suffix. The debate on whether the letter "O" could be used for octal was intense -- an uppercase "O" looks suspiciously similar to a zero in some fonts. Suggestions were made to use a "c" (the second letter of "oCtal"), or even to use a "t" for "ocTal" and an "n" for "biNary" to go along with the "x" for "heXadecimal". For the string % operator, "o" was already being used to denote octal, and "b" was not used for anything, so this works out much better than, for example, using "c" (which means "character" for the % operator). At the end of the day, since uppercase "O" can look like a zero and uppercase "B" can look like an 8, it was decided that these prefixes should be lowercase only. Open Issues =========== It was suggested in the discussion that lowercase should be used for all numeric and string special modifiers, such as 'x' for hexadecimal, 'r' for raw strings, 'e' for exponentiation, and 'j' for complex numbers. This is an issue for a separate PEP. There are still some strong feelings that '0123' should be allowed as a literal decimal in Python 3.0. If this is the right thing to do, this can easily be covered in an additional PEP. This proposal only takes the first step of making '0123' not be a valid octal number, for reasons covered in the rationale. Is there (or should there be) an option for the 2to3 translator which only makes the 2.6 compatible changes? Should this be run on 2.6 library code before the 2.6 release? References ========== .. [1] GNU libc manual printf integer format conversions (http://www.gnu.org/software/libc/manual/html_node/Integer-Conversions.html) .. [2] Python string formatting operations (http://docs.python.org/lib/typesseq-strings.html) .. [3] The Representation of Numbers, Jiajie Zhang and Donald A. Norman (http://acad88.sahs.uth.tmc.edu/research/publications/Number-Representation.pdf) .. [4] ENIAC page at wikipedia (http://en.wikipedia.org/wiki/ENIAC) .. [5] BCD page at wikipedia (http://en.wikipedia.org/wiki/Binary-coded_decimal) Copyright ========= This document has been placed in the public domain. From g.brandl at gmx.net Sun Mar 18 23:56:51 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 18 Mar 2007 23:56:51 +0100 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: Message-ID: Patrick Maupin schrieb: > This is what I think we agreed on :) > > (except that I think that lowercase-only 'x', 'e', 'r' is a subject > for a different PEP) > > I don't have a PEP number for this yet, but I emailed it into the PEP > editors yesterday. > > I tried to cover the discussion about letting "0123" be a decimal > integer, and the discussion about arbitrary bases, very carefully. > > Please let me know if this is not what was decided or if there is > additional information which should be in the PEP. I just want to add that I've adapted and extended Thomas' original patch for the 0o123 syntax. It should be pretty complete. Open issues would probably be: - should int("0755", 0) give 0o755? (IMO yes) - what should "%#o" % 100 result in? "0144" or "0o144"? - should oct(100) return "0144" or "0o144"? Georg From pmaupin at gmail.com Mon Mar 19 00:58:10 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sun, 18 Mar 2007 18:58:10 -0500 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: Message-ID: On 3/18/07, Georg Brandl wrote: > I just want to add that I've adapted and extended Thomas' original patch > for the 0o123 syntax. It should be pretty complete. > > Open issues would probably be: > - should int("0755", 0) give 0o755? (IMO yes) The PEP covers this, with the answer current of "yes" for 2.6 and "exception" for 3.0. (It presumes int(x, 0) should be the same as the compiler tokenizer result.) > - what should "%#o" % 100 result in? "0144" or "0o144"? > - should oct(100) return "0144" or "0o144"? Thanks! I missed that the formatter had the '#' option, and completely forgot about the oct() function. I think TOOWTDI says 0o144 in both cases. OTOH, does anybody actually use the oct() function? Has the issue of hex() and oct() remaining builtin been covered? Should we add a matching bin() or remove oct()? Also, PEP 3101 does not currently contain the '#' "alternate form' specifier. Is this an oversight which needs to be corrected? Thanks, Pat From g.brandl at gmx.net Mon Mar 19 01:36:31 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Mon, 19 Mar 2007 01:36:31 +0100 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: Message-ID: Patrick Maupin schrieb: > On 3/18/07, Georg Brandl wrote: >> I just want to add that I've adapted and extended Thomas' original patch >> for the 0o123 syntax. It should be pretty complete. >> >> Open issues would probably be: >> - should int("0755", 0) give 0o755? (IMO yes) > > The PEP covers this, with the answer current of "yes" for 2.6 and > "exception" for 3.0. (It presumes int(x, 0) should be the same as the > compiler tokenizer result.) It isn't. We already said that int() should continue to accept "0x" and "0X" prefixes for hexadecimal, for instance. >> - what should "%#o" % 100 result in? "0144" or "0o144"? >> - should oct(100) return "0144" or "0o144"? > > Thanks! I missed that the formatter had the '#' option, and > completely forgot about the oct() function. I think TOOWTDI says > 0o144 in both cases. I agree. Georg From thomas at python.org Mon Mar 19 01:48:15 2007 From: thomas at python.org (Thomas Wouters) Date: Mon, 19 Mar 2007 01:48:15 +0100 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: Message-ID: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> On 3/18/07, Patrick Maupin wrote: > the treatment of string literal representations of integers I don't think this is the right term. It's certainly confusing, considering "string literals" are the stuff in quotes. A less confusing name is just 'integer literals'. - There is a strong desire for binary support in the language. I have yet to see this 'strong desire'. I've seen people remark that they think it'd be nicely symmetric, but requests for actual usecases have always gotten low responses, as far as I remember. I've done quite a bit of bitfiddling with Python, with the struct module or with hexadecimals and bitwise operations, and in none of those cases would a binary literal have been helpful; they're way too verbose and impossible to get right/debug. But the Python community has its share of minimalists (each with > his own idea of the proper subset), so, for example, Mattias > Engdeg?rd wrote: "only decimal, hex and binary constants are > of general use at all" while Thomas Wouters opined that octal > and hexadecimal should remain but binary was only for misguided > students doing their homework. This strikes me as a rather snide and childish paragraph -- and not just the part about me, which you got wrong. What I said was "they're invariably new programmers tackling the problem as an exercise, or trying to get at the 'pure bits' of an int to perform bitwise operations in an inefficient manner." So, in general, humans communicate "normal" (non-computer) > numerical information either via names (AM, PM, January, ...) > or via use of decimal notation. Obviously, names are > seldom used for large sets of items, so decimal is used for > everything else. There are studies which attempt to explain > why this is so, typically reaching the expected conclusion > that the Arabic numeral system is well-suited to human > cognition. [3]_ I'm not sure why all this matters to the PEP, really. Do we really have to justify having decimal, hexadecimal and octal literals? It's _way_ oversized if you ask me :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070319/ffaa7b60/attachment.html From thomas at python.org Mon Mar 19 01:50:00 2007 From: thomas at python.org (Thomas Wouters) Date: Mon, 19 Mar 2007 01:50:00 +0100 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: Message-ID: <9e804ac0703181750v3d2d976bnbe52c3488b5fbb1a@mail.gmail.com> On 3/19/07, Georg Brandl wrote: > > Patrick Maupin schrieb: > > On 3/18/07, Georg Brandl wrote: > >> I just want to add that I've adapted and extended Thomas' original > patch > >> for the 0o123 syntax. It should be pretty complete. > >> > >> Open issues would probably be: > >> - should int("0755", 0) give 0o755? (IMO yes) > > > > The PEP covers this, with the answer current of "yes" for 2.6 and > > "exception" for 3.0. (It presumes int(x, 0) should be the same as the > > compiler tokenizer result.) > > It isn't. We already said that int() should continue to accept "0x" and > "0X" > prefixes for hexadecimal, for instance. As far as I understood Guido, int(s, 16) should. int(s, 0) should not. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070319/6cfb3411/attachment.htm From pmaupin at gmail.com Mon Mar 19 02:28:25 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sun, 18 Mar 2007 20:28:25 -0500 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> Message-ID: On 3/18/07, Thomas Wouters wrote: > On 3/18/07, Patrick Maupin wrote: > I don't think this is the right term. It's certainly confusing, considering > "string literals" are the stuff in quotes. A less confusing name is just > 'integer literals'. OK, if it is agreed that 'literals' is not confusing in a PEP title. Like most English words, 'literal' has multiple meanings, and many English speakers seem to believe that "literally" means something entirely different. > > > - There is a strong desire for binary support in the language. > > I have yet to see this 'strong desire'. I've seen people remark that they > think it'd be nicely symmetric, but requests for actual usecases have always > gotten low responses, as far as I remember. I've done quite a bit of > bitfiddling with Python, with the struct module or with hexadecimals and > bitwise operations, and in none of those cases would a binary literal have > been helpful; they're way too verbose and impossible to get right/debug. I have personally written output functions for this multiple times, as apparently, have several others. The discussion about PEP3101 didn't seem to have any detractors, so binary is apparently going to be supported for output. Also, as many in the discussion pointed out, it's precisely this reason (impossible to get right/debug) which is why you might want to cut and paste a literal from, e.g. a register document, but (admittedly) the use case for input is not as strong as the case for output, until you throw the symmetry argument in. > > But the Python community has its share of minimalists (each with > > his own idea of the proper subset), so, for example, Mattias > > Engdeg?rd wrote: "only decimal, hex and binary constants are > > of general use at all" while Thomas Wouters opined that octal > > and hexadecimal should remain but binary was only for misguided > > students doing their homework. > > This strikes me as a rather snide and childish paragraph -- and not just the > part about me, which you got wrong. What I said was "they're invariably new > programmers tackling the problem as an exercise, or trying to get at the > 'pure bits' of an int to perform bitwise operations in an inefficient > manner." The point of the paragraph was to illustrate that there is not 100% convergence on the required set of supported bases, and I think you are currently proving that point for me quite handily. :) I wasn't trying to be snide, but I was trying to condense what you said a bit. (As you note later, the document is already quite long!) To me, a "new programmer tackling the problem as an exercise" infers somebody doing (either mandated or self-inflicted) homework. I can take it out or try again. > > So, in general, humans communicate "normal" (non-computer) > > numerical information either via names (AM, PM, January, ...) > > or via use of decimal notation. Obviously, names are > > seldom used for large sets of items, so decimal is used for > > everything else. There are studies which attempt to explain > > why this is so, typically reaching the expected conclusion > > that the Arabic numeral system is well-suited to human > > cognition. [3]_ > > I'm not sure why all this matters to the PEP, really. Do we really have to > justify having decimal, hexadecimal and octal literals? It's _way_ oversized > if you ask me :) There was some apparently serious discussion about "Hey, why not just support ALL the bases?" I agree that the PEP is quite large, which is why the rationale is at the end. But the discussion had several passionate people (with valid points!) arguing things like the number of supported radices and whether 0123 should be a decimal number, so when Guido said: Great! Mind writing up writing up a PEP that summarizes the discussion (a bit)? In particular it should explain (a) why we need octal literals; (b) why leading-zero is bad; (c) why we don't need general bases; (d) why 0t is the best choice. Oh, and please add 0b too; there's no formal proposal for that yet. Thanks! (some of which, as you can see, has already changed since he wrote this) I wanted to try to capture all sides of the discussion, as much as possible, so that when somebody asks a question about why we did or didn't do something, we can truthfully say that the PEP answers their question. Thanks, Pat From pmaupin at gmail.com Mon Mar 19 02:39:54 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sun, 18 Mar 2007 20:39:54 -0500 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: <9e804ac0703181750v3d2d976bnbe52c3488b5fbb1a@mail.gmail.com> References: <9e804ac0703181750v3d2d976bnbe52c3488b5fbb1a@mail.gmail.com> Message-ID: On 3/18/07, Thomas Wouters wrote: > As far as I understood Guido, int(s, 16) should. int(s, 0) should not. I guess this is still an open item, then. My understanding is that Guido said "no uppercase", I asked about "X", he basically replied, "well, fine for data, if you must, but not code, because Google Code says that's not required", and I said... Well, I thought I said "OK, thanks, we won't do it for either" but looking back, I see I didn't say that very well. My opinion is that int(s, 0) should do the same as eval(). A string replace on a data file is actually much easier than on a code file, and it really is a one-liner, so the potential existence of upper case 'X' in the data file does not seem like reason enough to make these different. However, if there strong sentiment that uppercase 'X' really needs to be supported, that needs to be a discussion for another PEP. This one got so big from the supported bases, and parse of "0123' as exception or int, discussions, that I deliberately left the lower-case only argument as subject matter for a different day. That argument encompasses 'r' and 'j', as well, which really doesn't have anything to do with numeric bases. Regards, Pat From guido at python.org Mon Mar 19 03:36:58 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 18 Mar 2007 19:36:58 -0700 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> Message-ID: On 3/18/07, Thomas Wouters wrote: > > > On 3/18/07, Patrick Maupin wrote: > > the treatment of string literal representations of integers > > I don't think this is the right term. It's certainly confusing, considering > "string literals" are the stuff in quotes. A less confusing name is just > 'integer literals'. Right. This is how the Python reference manual uses the term. "Literals" are all notations for values; e.g. string literals, numeric literals. (We don't talk of list or dict literals since they can contain arbitrary expressions.) > > - There is a strong desire for binary support in the language. > > I have yet to see this 'strong desire'. I've seen people remark that they > think it'd be nicely symmetric, but requests for actual usecases have always > gotten low responses, as far as I remember. I've done quite a bit of > bitfiddling with Python, with the struct module or with hexadecimals and > bitwise operations, and in none of those cases would a binary literal have > been helpful; they're way too verbose and impossible to get right/debug. Well, *I* have seen the strong desire -- not in myself, but I have seen it. It's a done deal. 0b for binary is in. > > But the Python community has its share of minimalists (each with > > his own idea of the proper subset), so, for example, Mattias > > Engdeg?rd wrote: "only decimal, hex and binary constants are > > of general use at all" while Thomas Wouters opined that octal > > and hexadecimal should remain but binary was only for misguided > > students doing their homework. > > This strikes me as a rather snide and childish paragraph [...] Patrick, please remove specific names from the PEP. > > So, in general, humans communicate "normal" (non-computer) > > numerical information either via names (AM, PM, January, ...) > > or via use of decimal notation. Obviously, names are > > seldom used for large sets of items, so decimal is used for > > everything else. There are studies which attempt to explain > > why this is so, typically reaching the expected conclusion > > that the Arabic numeral system is well-suited to human > > cognition. [3]_ > > I'm not sure why all this matters to the PEP, really. Do we really have to > justify having decimal, hexadecimal and octal literals? It's _way_ oversized > if you ask me :) Octal does need to be justified, since some people argued to remove it. I guess binary needs to be justified because Thomas doesn't see the need. :-) > -- > Thomas Wouters -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Mon Mar 19 04:54:05 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 18 Mar 2007 23:54:05 -0400 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> Message-ID: On 3/18/07, Guido van Rossum wrote: > Octal does need to be justified, since some people argued to remove > it. I guess binary needs to be justified because Thomas doesn't see > the need. :-) I see literals for octal and binary as similar to raising a tuple, or automatically unpacking one in a function call. It can be useful, but it isn't needed very often. When it does show up, it can be confusing *because* it is so rare. If I were only worried about one program, I would prefer to write: >>> oct(78) == "0o116" rather than: >>> from math import oct >>> oct(78) == int("123", 8) The preference isn't really overwhelming, though, and the use isn't very frequent. This doesn't seem like a big enough win to justify any extra complexity in the language. (And no, I wouldn't add hex either, but not adding is different from removing.) -jJ From pmaupin at gmail.com Mon Mar 19 05:06:34 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Sun, 18 Mar 2007 23:06:34 -0500 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> Message-ID: On 3/18/07, Jim Jewett wrote: > The preference isn't really overwhelming, though, and the use isn't > very frequent. This doesn't seem like a big enough win to justify any > extra complexity in the language. > > (And no, I wouldn't add hex either, but not adding is different from removing.) Well, octal is in the language as well, so there is some resistance to removing it. But there is somewhat less resistance to changing it. Binary isn't in the language yet, but I hope all the arguments about that are encompassed in the PEP. THanks, Pat From pmaupin at gmail.com Mon Mar 19 06:23:27 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 19 Mar 2007 00:23:27 -0500 Subject: [Python-3000] Updated PEP: Integer literal syntax and radices (was octal/binary discussion) In-Reply-To: References: Message-ID: On 3/19/07, Patrick Maupin wrote: > (new version of the PEP) D'oh. I just realized that the text I used in the title which Thomas pointed out was confusing is similarly scattered throughout the PEP. I will fix this on the next rev., and just refer to "integer literals", as it has been agreed that the "string" and "representation" parts are redundant and confusing. Thanks, Pat From baptiste13 at altern.org Mon Mar 19 11:13:16 2007 From: baptiste13 at altern.org (Baptiste Carvello) Date: Mon, 19 Mar 2007 11:13:16 +0100 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: <9e804ac0703181750v3d2d976bnbe52c3488b5fbb1a@mail.gmail.com> Message-ID: Patrick Maupin a ?crit : > A string > replace on a data file is actually much easier than on a code file, > and it really is a one-liner, so the potential existence of upper case > 'X' in the data file does not seem like reason enough to make these > different. > you don't analyse data files for a living, do you :-) As a physicist, I have to open many kinds of data files with the weirdest formats. Of course, some of them could not reasonably be supported by the python builtins, but it doesn't have to be made unnecessary difficult either. "Be liberal in what you accept". However, I usually don't have to guess the base of my data, so this really applies mostly to float(s), int(s), and int(s, base) with base != 0. Cheers, Baptiste From pmaupin at gmail.com Mon Mar 19 06:14:12 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 19 Mar 2007 00:14:12 -0500 Subject: [Python-3000] Updated PEP: Integer literal syntax and radices (was octal/binary discussion) Message-ID: The update includes issues discussed to date, plus the support of uppercase on input of binary and hex, e.g. '0O123'. It was pointed out to me that, since I suggested upper/lowercase was an issue for another PEP, removal of uppercase octal/binary belonged in that same PEP, if anybody cares enough to write it. (It seems that style guides and user preference will lead most people to write '0o' instead of '0O', so perhaps there is no compelling need.) Abstract ======== This PEP proposes changes to the Python core to rationalize the treatment of string literal representations of integers in different radices (bases). These changes are targeted at Python 3.0, but the backward-compatible parts of the changes should be added to Python 2.6, so that all valid 3.0 integer literals will also be valid in 2.6. The proposal is that: a) octal literals must now be specified with a leading "0o" or "0O" instead of "0"; b) binary literals are now supported via a leading "0b" or "0B"; and c) provision will be made for binary numbers in string formatting. Motivation ========== This PEP was motivated by two different issues: - The default octal representation of integers is silently confusing to people unfamiliar with C-like languages. It is extremely easy to inadvertently create an integer object with the wrong value, because '013' means 'decimal 11', not 'decimal 13', to the Python language itself, which is not the meaning that most humans would assign to this literal. - Some Python users have a strong desire for binary support in the language. Specification ============= Grammar specification --------------------- The grammar will be changed. For Python 2.6, the changed and new token definitions will be:: integer ::= decimalinteger | octinteger | hexinteger | bininteger | oldoctinteger octinteger ::= "0" ("o" | "O") octdigit+ bininteger ::= "0" ("b" | "B") bindigit+ oldoctinteger ::= "0" octdigit+ bindigit ::= "0" | "1" For Python 3.0, "oldoctinteger" will not be supported, and an exception will be raised if a literal has a leading "0" and a second character which is a digit. For both versions, this will require changes to PyLong_FromString as well as the grammar. The documentation will have to be changed as well: grammar.txt, as well as the integer literal section of the reference manual. PEP 306 should be checked for other issues, and that PEP should be updated if the procedure described therein is insufficient. int() specification -------------------- int(s, 0) will also match the new grammar definition. This should happen automatically with the changes to PyLong_FromString required for the grammar change. Also the documentation for int() should be changed to explain that int(s) operates identically to int(s, 10), and the word "guess" should be removed from the description of int(s, 0). long() specification -------------------- For Python 2.6, the long() implementation and documentation should be changed to reflect the new grammar. Tokenizer exception handling ---------------------------- If an invalid token contains a leading "0", the exception error message should be more informative than the current "SyntaxError: invalid token". It should explain that decimal numbers may not have a leading zero, and that octal numbers require an "o" after the leading zero. int() exception handling ------------------------ The ValueError raised for any call to int() with a string should at least explicitly contain the base in the error message, e.g.:: ValueError: invalid literal for base 8 int(): 09 oct() function --------------- oct() should be updated to output '0o' in front of the octal digits (for 3.0, and 2.6 compatibility mode). Output formatting ----------------- The string (and unicode in 2.6) % operator will have 'b' format specifier added for binary, and the alternate syntax of the 'o' option will need to be updated to add '0o' in front, instead of '0'. PEP 3101 already supports 'b' for binary output. Transition from 2.6 to 3.0 --------------------------- The 2to3 translator will have to insert 'o' into any octal string literal. The Py3K compatible option to Python 2.6 should cause attempts to use oldoctinteger literals to raise an exception. Rationale ========= Most of the discussion on these issues occurred on the Python-3000 mailing list starting 14-Mar-2007, prompted by an observation that the average human being would be completely mystified upon finding that prepending a "0" to a string of digits changes the meaning of that digit string entirely. It was pointed out during this discussion that a similar, but shorter, discussion on the subject occurred in January of 2006, prompted by a discovery of the same issue. Background ---------- For historical reasons, Python's string representation of integers in different bases (radices), for string formatting and token literals, borrows heavily from C. [1]_ [2]_ Usage has shown that the historical method of specifying an octal number is confusing, and also that it would be nice to have additional support for binary literals. Throughout this document, unless otherwise noted, discussions about the string representation of integers relate to these features: - Literal integer tokens, as used by normal module compilation, by eval(), and by int(token, 0). (int(token) and int(token, 2-36) are not modified by this proposal.) * Under 2.6, long() is treated the same as int() - Formatting of integers into strings, either via the % string operator or the new PEP 3101 advanced string formatting method. It is presumed that: - All of these features should have an identical set of supported radices, for consistency. - Python source code syntax and int(mystring, 0) should continue to share identical behavior. Removal of old octal syntax ---------------------------- This PEP proposes that the ability to specify an octal number by using a leading zero will be removed from the language in Python 3.0 (and the Python 3.0 preview mode of 2.6), and that a SyntaxError will be raised whenever a leading "0" is immediately followed by another digit. During the present discussion, it was almost universally agreed that:: eval('010') == 8 should no longer be true, because that is confusing to new users. It was also proposed that:: eval('0010') == 10 should become true, but that is much more contentious, because it is so inconsistent with usage in other computer languages that mistakes are likely to be made. Almost all currently popular computer languages, including C/C++, Java, Perl, and JavaScript, treat a sequence of digits with a leading zero as an octal number. Proponents of treating these numbers as decimal instead have a very valid point -- as discussed in `Supported radices`_, below, the entire non-computer world uses decimal numbers almost exclusively. There is ample anecdotal evidence that many people are dismayed and confused if they are confronted with non-decimal radices. However, in most situations, most people do not write gratuitous zeros in front of their decimal numbers. The primary exception is when an attempt is being made to line up columns of numbers. But since PEP 8 specifically discourages the use of spaces to try to align Python code, one would suspect the same argument should apply to the use of leading zeros for the same purpose. Finally, although the email discussion often focused on whether anybody actually *uses* octal any more, and whether we should cater to those old-timers in any case, that is almost entirely besides the point. Assume the rare complete newcomer to computing who *does*, either occasionally or as a matter of habit, use leading zeros for decimal numbers. Python could either: a) silently do the wrong thing with his numbers, as it does now; b) immediately disabuse him of the notion that this is viable syntax (and yes, the SyntaxWarning should be more gentle than it currently is, but that is a subject for a different PEP); or c) let him continue to think that computers are happy with multi-digit decimal integers which start with "0". Some people passionately believe that (c) is the correct answer, and they would be absolutely right if we could be sure that new users will never blossom and grow and start writing AJAX applications. So while a new Python user may (currently) be mystified at the delayed discovery that his numbers don't work properly, we can fix it by explaining to him immediately that Python doesn't like leading zeros (hopefully with a reasonable message!), or we can delegate this teaching experience to the JavaScript interpreter in the Internet Explorer browser, and let him try to debug his issue there. Supported radices ----------------- This PEP proposes that the supported radices for the Python language will be 2, 8, 10, and 16. Once it is agreed that the old syntax for octal (radix 8) representation of integers must be removed from the language, the next obvious question is "Do we actually need a way to specify (and display) numbers in octal?" This question is quickly followed by "What radices does the language need to support?" Because computers are so adept at doing what you tell them to, a tempting answer in the discussion was "all of them." This answer has obviously been given before -- the int() constructor will accept an explicit radix with a value between 2 and 36, inclusive, with the latter number bearing a suspicious arithmetic similarity to the sum of the number of numeric digits and the number of same-case letters in the ASCII alphabet. But the best argument for inclusion will have a use-case to back it up, so the idea of supporting all radices was quickly rejected, and the only radices left with any real support were decimal, hexadecimal, octal, and binary. Just because a particular radix has a vocal supporter on the mailing list does not mean that it really should be in the language, so the rest of this section is a treatise on the utility of these particular radices, vs. other possible choices. Humans use other numeric bases constantly. If I tell you that it is 12:30 PM, I have communicated quantitative information arguably composed of *three* separate bases (12, 60, and 2), only one of which is in the "agreed" list above. But the *communication* of that information used two decimal digits each for the base 12 and base 60 information, and, perversely, two letters for information which could have fit in a single decimal digit. So, in general, humans communicate "normal" (non-computer) numerical information either via names (AM, PM, January, ...) or via use of decimal notation. Obviously, names are seldom used for large sets of items, so decimal is used for everything else. There are studies which attempt to explain why this is so, typically reaching the expected conclusion that the Arabic numeral system is well-suited to human cognition. [3]_ There is even support in the history of the design of computers to indicate that decimal notation is the correct way for computers to communicate with humans. One of the first modern computers, ENIAC [4]_ computed in decimal, even though there were already existing computers which operated in binary. Decimal computer operation was important enough that many computers, including the ubiquitous PC, have instructions designed to operate on "binary coded decimal" (BCD) [5]_ , a representation which devotes 4 bits to each decimal digit. These instructions date from a time when the most strenuous calculations ever performed on many numbers were the calculations actually required to perform textual I/O with them. It is possible to display BCD without having to perform a divide/remainder operation on every displayed digit, and this was a huge computational win when most hardware didn't have fast divide capability. Another factor contributing to the use of BCD is that, with BCD calculations, rounding will happen exactly the same way that a human would do it, so BCD is still sometimes used in fields like finance, despite the computational and storage superiority of binary. So, if it weren't for the fact that computers themselves normally use binary for efficient computation and data storage, string representations of integers would probably always be in decimal. Unfortunately, computer hardware doesn't think like humans, so programmers and hardware engineers must often resort to thinking like the computer, which means that it is important for Python to have the ability to communicate binary data in a form that is understandable to humans. The requirement that the binary data notation must be cognitively easy for humans to process means that it should contain an integral number of binary digits (bits) per symbol, while otherwise conforming quite closely to the standard tried-and-true decimal notation (position indicates power, larger magnitude on the left, not too many symbols in the alphabet, etc.). The obvious "sweet spot" for this binary data notation is thus octal, which packs the largest integral number of bits possible into a single symbol chosen from the Arabic numeral alphabet. In fact, some computer architectures, such as the PDP8 and the 8080/Z80, were defined in terms of octal, in the sense of arranging the bitfields of instructions in groups of three, and using octal representations to describe the instruction set. Even today, octal is important because of bit-packed structures which consist of 3 bits per field, such as Unix file permission masks. But octal has a drawback when used for larger numbers. The number of bits per symbol, while integral, is not itself a power of two. This limitation (given that the word size of most computers these days is a power of two) has resulted in hexadecimal, which is more popular than octal despite the fact that it requires a 60% larger alphabet than decimal, because each symbol contains 4 bits. Some numbers, such as Unix file permission masks, are easily decoded by humans when represented in octal, but difficult to decode in hexadecimal, while other numbers are much easier for humans to handle in hexadecimal. Unfortunately, there are also binary numbers used in computers which are not very well communicated in either hexadecimal or octal. Thankfully, fewer people have to deal with these on a regular basis, but on the other hand, this means that several people on the discussion list questioned the wisdom of adding a straight binary representation to Python. One example of where these numbers is very useful is in reading and writing hardware registers. Sometimes hardware designers will eschew human readability and opt for address space efficiency, by packing multiple bit fields into a single hardware register at unaligned bit locations, and it is tedious and error-prone for a human to reconstruct a 5 bit field which consists of the upper 3 bits of one hex digit, and the lower 2 bits of the next hex digit. Even if the ability of Python to communicate binary information to humans is only useful for a small technical subset of the population, it is exactly that population subset which contains most, if not all, members of the Python core team, so even straight binary, the least useful of these notations, has several enthusiastic supporters and few, if any, staunch opponents, among the Python community. Syntax for supported radices ----------------------------- This proposal is to to use a "0o" prefix with either uppercase or lowercase "o" for octal, and a "0b" prefix with either uppercase or lowercase "b" for binary. There was strong support for not supporting uppercase, but this is a separate subject for a different PEP, as 'j' for complex numbers, 'e' for exponent, and 'r' for raw string (to name a few) already support uppercase. The syntax for delimiting the different radices received a lot of attention in the discussion on Python-3000. There are several (sometimes conflicting) requirements and "nice-to-haves" for this syntax: - It should be as compatible with other languages and previous versions of Python as is reasonable, both for the input syntax and for the output (e.g. string % operator) syntax. - It should be as obvious to the casual observer as possible. - It should be easy to visually distinguish integers formatted in the different bases. Proposed syntaxes included things like arbitrary radix prefixes, such as 16r100 (256 in hexadecimal), and radix suffixes, similar to the 100h assembler-style suffix. The debate on whether the letter "O" could be used for octal was intense -- an uppercase "O" looks suspiciously similar to a zero in some fonts. Suggestions were made to use a "c" (the second letter of "oCtal"), or even to use a "t" for "ocTal" and an "n" for "biNary" to go along with the "x" for "heXadecimal". For the string % operator, "o" was already being used to denote octal, and "b" was not used for anything, so this works out much better than, for example, using "c" (which means "character" for the % operator). At the end of the day, since uppercase "O" can look like a zero and uppercase "B" can look like an 8, it was decided that these prefixes should be lowercase only, but, like 'r' for raw string, that can be a preference or style-guide issue. Open Issues =========== It was suggested in the discussion that lowercase should be used for all numeric and string special modifiers, such as 'x' for hexadecimal, 'r' for raw strings, 'e' for exponentiation, and 'j' for complex numbers. This is an issue for a separate PEP. This PEP takes no position on uppercase or lowercase for input, just noting that, for consistency, if uppercase is not to be removed from input parsing for other letters, it should be added for octal and binary, and documenting the changes under this assumption, as there is not yet a PEP about the case issue. Output formatting may be a different story -- there is already ample precedence for case sensitivity in the output format string, and there would need to be a consensus that there is a valid use-case for the "alternate form" of the string % operator to support uppercase 'B' or 'O' characters for binary or octal output. Currently, PEP3101 does not even support this alternate capability, and the hex() function does not allow the programmer to specify the case of the 'x' character. There are still some strong feelings that '0123' should be allowed as a literal decimal in Python 3.0. If this is the right thing to do, this can easily be covered in an additional PEP. This proposal only takes the first step of making '0123' not be a valid octal number, for reasons covered in the rationale. Is there (or should there be) an option for the 2to3 translator which only makes the 2.6 compatible changes? Should this be run on 2.6 library code before the 2.6 release? Should a bin() function which matches hex() and oct() be added? Is hex() really that useful once we have advanced string formatting? References ========== .. [1] GNU libc manual printf integer format conversions (http://www.gnu.org/software/libc/manual/html_node/Integer-Conversions.html) .. [2] Python string formatting operations (http://docs.python.org/lib/typesseq-strings.html) .. [3] The Representation of Numbers, Jiajie Zhang and Donald A. Norman (http://acad88.sahs.uth.tmc.edu/research/publications/Number-Representation.pdf) .. [4] ENIAC page at wikipedia (http://en.wikipedia.org/wiki/ENIAC) .. [5] BCD page at wikipedia (http://en.wikipedia.org/wiki/Binary-coded_decimal) Copyright ========= This document has been placed in the public domain. From jimjjewett at gmail.com Mon Mar 19 15:29:54 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 19 Mar 2007 10:29:54 -0400 Subject: [Python-3000] Updated PEP: Integer literal syntax and radices (was octal/binary discussion) In-Reply-To: References: Message-ID: On 3/19/07, Patrick Maupin wrote: > It was pointed out to me that, since I suggested > upper/lowercase was an issue for another PEP, > removal of uppercase octal/binary belonged in that same PEP Removal of uppercase from other formats would affect backwards compatibility. For binary and octal, it is simply a matter of not adding additional alternative ways to do it. Consistency with hex and floats is worth something, but I don't think it is important enough to justify adding new support for the confusing O0. (Remember, the choice was 0t when uppercase support was still expected.) Also note that the case distinction can matter in output, because other systems might already assume a certain case for hex or floating point; I haven't heard of any external systems that already recognize O0 but not O0, or 0B but not 0b. If we don't support it on output, then the symmetry is already broken, and there is no reason to support it on input. > ... int(token) and int(token, 2-36) > are not modified by this proposal. int(token, 16) accepts (but does not require) a leading 0x. I assume that the code for 2 and 8 will also begin to allow a leading prefix. -jJ From mccollum at fas.harvard.edu Mon Mar 19 16:39:57 2007 From: mccollum at fas.harvard.edu (Andrew Karem McCollum) Date: Mon, 19 Mar 2007 11:39:57 -0400 (EDT) Subject: [Python-3000] Draft PEP for Requiring Lowercase Literal Modifiers Message-ID: This is my first PEP, and one of my first postings to this list, so I apologize in advance for any glaring errors. I wrote this up because I feel like it is a good companion to the recent octal and binary discussions/PEP. If nothing else, this should at least provide a jumping off point for discussion and someone more experienced could use it as a basis for a more rigorous PEP if they so desired. If it is supported, I am happy to work on an implementation, though I imagine someone else could produce one much more expediently. -Andrew McCollum ------------------------------------------------------------------- PEP: XXX Title: Requiring Lowercase Characters in Literal Modifiers Version: $Revision$ Last-Modified: $Date$ Author: Andrew McCollum Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 19-Mar-2007 Python-Version: 3.0 Post-History: Abstract ======== This PEP proposes to change the syntax for declaring literals with prefixes or modifiers by requiring the prefix or modifier to be in lowercase. Affected modifiers include: * The 'b' and 'r' prefixes for string and bytes literals * The 'b', 'o', and 'x' modifiers for integer literals in other bases * The 'j' suffix for imaginary literals * The 'e' exponent notation for floating point literals Motivation ========== The primary motivation for this change is to avoid confusion and improve the appearance of these literals. The most obvious example is the proposed 'o' modifier for declaring an octal number such as, '0o755', representing the decimal value '493'. When the 'o' character is written in uppercase, the number appears as '0O755', which can be confusing in many fonts because it resembles the string of digits '00755'. In other cases, the lowercase version of the modifier serves to visually separate the two components of the literal, or to make the modifier stand out against the neighboring literal, such as in the following examples:: 0x5A 0b0110 0xab 1.92e21 3.13j 3.14e5j r'\d+\.' b"Hello world" With uppercase modifiers, these literals appear as:: 0X5A 0B0110 0Xab 1.92E21 3.13J 3.14E5J R'\d+\.' B"Hello world" Which are more difficult to visually parse, especially upon initial inspection. There is also an argument for uniformity and so that TOOWTDI. Unlike the case of string literals where ', ", and """ all behave differently making each useful in different situations, in the case of literal modifiers, the difference is purely cosmetic and the behavior of the literal is otherwise unchanged. Grammar Changes =============== The new grammar for string prefixes [1]_ (with the bytes literal) will be: stringprefix ::= "b" | "r" | "br" Integer literals [2]_ will have the following grammar changes: bininteger ::= "0b" bindigit+ octinteger ::= "0o" octdigit+ hexinteger ::= "0x" hexdigit+ Exponents in floating point literals [3]_ will now be defined by: exponent ::= "e" ["+" | "-"] digit+ Imaginary numbers [4]_ will be will now be defined with the syntax: imagnumber ::= (floatnumber | intpart) "j" The grammar of these literals will be otherwise unchanged. For example, the specification of 'hexdigit' will continue to allow both uppercase and lowercase digits. Since this PEP is targeted at Python 3000, the suffix for specifying long integer literals ('l' or 'L') and the prefix for specifying unicode strings ('u' or 'U') are ignored as both forms will disappear as these types are merged with int and str, respectively. Semantic Changes ================ The behavior of the 'int' builtin when passed a radix of 0 will be changed to follow the above grammar. This change is to maintain the specified behavior [5]_ that a radix of 0 mirrors the literal syntax. The behavior of this function will otherwise not be altered. In particular, the behavior of accepting the prefix '0X' when a radix of 16 is specified will be kept for backwards compatibility and easier parsing of data files. Automatic Conversion ==================== It should be trivial for the '2to3' conversion tool to convert literals to the new syntax in all cases. The only possible incompatibility will be from the subtle changes to the 'int' builtin. Open Issues =========== The main issue involves the treatment of hexadecimal values employing the legacy '0X' prefix when passed to the 'int' builtin. Several people showed a desire to maintain parity with the literal syntax and the 'eval' function when 0 was passed in as the radix. The argument against this behavior is that it breaks backwards compatibility and makes parsing integers from arbitrary sources more difficult. This PEP makes the compromise of allowing the use of the prefix '0X' only when the radix is explicitly specified. The rationale for this choice is that when parsing integers from data files, the radix is often know ahead of time, and thus can be supplied as a second argument to maintain the previous behavior, while maintaining the symmetry between the literal syntax and the 0 radix form. BDFL Pronouncements =================== The BDFL supports the disallowing of leading zeros in the syntax for integer literals, and was leaning towards maintaining this behavior when a radix of 0 was passed to the 'int' builtin [6]_. This would break backwards compatibility for automatically parsing octal literals. Later, the BDFL expressed a preference that '0X' be an allowable prefix for hexadecimal numbers when a radix of 0 was passed to the 'int' builtin [7]_. The PEP currently only allows this prefix when the radix is explicitly specified. Reference Implementation ======================== A reference implementation is not yet provided, but since no additional behavior is proposed, simply the removal of previously allowed behavior, changes should be minimal. References ========== .. [1] http://www.python.org/doc/current/ref/strings.html .. [2] http://www.python.org/doc/current/ref/integers.html .. [3] http://www.python.org/doc/current/ref/floating.html .. [4] http://www.python.org/doc/current/ref/imaginary.html .. [5] http://docs.python.org/lib/built-in-funcs.html .. [6] http://mail.python.org/pipermail/python-3000/2007-March/006325.html .. [7] http://mail.python.org/pipermail/python-3000/2007-March/006423.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From guido at python.org Mon Mar 19 17:38:43 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Mar 2007 09:38:43 -0700 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> Message-ID: The point was not to reopen the discussion; we've had the discussion and the outcome is clear. The point was, what should be justified in the PEP. On 3/18/07, Jim Jewett wrote: > On 3/18/07, Guido van Rossum wrote: > > Octal does need to be justified, since some people argued to remove > > it. I guess binary needs to be justified because Thomas doesn't see > > the need. :-) > > I see literals for octal and binary as similar to raising a tuple, or > automatically unpacking one in a function call. It can be useful, but > it isn't needed very often. When it does show up, it can be confusing > *because* it is so rare. > > If I were only worried about one program, I would prefer to write: > > >>> oct(78) == "0o116" > > rather than: > > >>> from math import oct > >>> oct(78) == int("123", 8) > > The preference isn't really overwhelming, though, and the use isn't > very frequent. This doesn't seem like a big enough win to justify any > extra complexity in the language. > > (And no, I wouldn't add hex either, but not adding is different from removing.) > > -jJ > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From raymond at ewtllc.com Mon Mar 19 18:09:38 2007 From: raymond at ewtllc.com (Raymond Hettinger) Date: Mon, 19 Mar 2007 10:09:38 -0700 Subject: [Python-3000] Questions about the Octal literal PEP In-Reply-To: <009e01c76a3b$e801df40$f001a8c0@RaymondLaptop1> References: <009e01c76a3b$e801df40$f001a8c0@RaymondLaptop1> Message-ID: <45FEC3D2.9080004@ewtllc.com> In Py2.6 and Py3.0, what would these emit: map(int, '08:30:00'.split(':')) # handle common decimal string formats with leading zeroes int('0777', 8) # handle externally created octal literal strings myfile.write(oct(44)) # writing to a file format that requires 054 style output eval(oct(44)) == 44 # basic invariant for oct(() If any of those change from Py2.5, it will result in gratuitous code breakage for almost zero benefit. Raymond From pmaupin at gmail.com Mon Mar 19 20:29:18 2007 From: pmaupin at gmail.com (Patrick Maupin) Date: Mon, 19 Mar 2007 14:29:18 -0500 Subject: [Python-3000] Questions about the Octal literal PEP In-Reply-To: <45FEC3D2.9080004@ewtllc.com> References: <009e01c76a3b$e801df40$f001a8c0@RaymondLaptop1> <45FEC3D2.9080004@ewtllc.com> Message-ID: On 3/19/07, Raymond Hettinger wrote: > In Py2.6 and Py3.0, what would these emit: I do not think we are contemplating breaking anything in 2.6, unless the 3.0 compatibility mode is selected. > map(int, '08:30:00'.split(':')) # handle common decimal string > formats with leading zeroes This will work fine. The way to think of int() is that, if it is passed a string, the default for the optional base parameter is 10. The documentation for int() should make this clearer than it is now. > int('0777', 8) # handle externally > created octal literal strings This will work fine. If the base passed to int() is non-zero, it only expects data characters in the string (no characters to determine the base). > myfile.write(oct(44)) # writing to a file format > that requires 054 style output This one won't work (in 3.0). There are many workarounds. One possibility: myfile.write('0%o' % 44) > eval(oct(44)) == 44 # basic invariant for oct(() Agree this needs to be an invariant -- that's what drives breakage on oct(). > If any of those change from Py2.5, it will result in > gratuitous code breakage for almost zero benefit. Should be no breakage on 2.6 in non-3.0 mode, and the one broken feature in 3.0 will be oct(). I did a google search, and there aren't really all that many of those, but it is something I should add to the PEP for the 2to3 tool. Thanks, Pat From tjreedy at udel.edu Mon Mar 19 21:22:19 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 19 Mar 2007 16:22:19 -0400 Subject: [Python-3000] Updated PEP: Integer literal syntax and radices (wasoctal/binary discussion) References: Message-ID: "Patrick Maupin" wrote in message news:d09829f50703182214m7334dn3e2b12cd379cef3f at mail.gmail.com... | The update includes issues discussed to date, plus the support of | uppercase on input of binary and hex, e.g. '0O123'. | | It was pointed out to me that, since I suggested upper/lowercase was | an issue for another PEP, removal of uppercase octal/binary belonged | in that same PEP, if anybody cares enough to write it. No, I disagree. You can't remove something that does not exist ;-) I thought it was agreed to only add 0o and 0b and *not* add 0O and 0B. It is pointless to add something in one PEP that no one(?) wants only to delete it again in another PEP. (It seems that | style guides and user preference will lead most people to write '0o' | instead of '0O', so perhaps there is no compelling need.) There is no need at all for 0O and 0B. Terry Jan Reedy From brett at python.org Mon Mar 19 22:04:35 2007 From: brett at python.org (Brett Cannon) Date: Mon, 19 Mar 2007 14:04:35 -0700 Subject: [Python-3000] Draft PEP for Requiring Lowercase Literal Modifiers In-Reply-To: References: Message-ID: On 3/19/07, Andrew Karem McCollum wrote: [SNIP] > PEP: XXX > Title: Requiring Lowercase Characters in Literal Modifiers > Version: $Revision$ > Last-Modified: $Date$ > Author: Andrew McCollum > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 19-Mar-2007 > Python-Version: 3.0 > Post-History: > > Abstract > ======== > > This PEP proposes to change the syntax for declaring literals with prefixes or > modifiers by requiring the prefix or modifier to be in lowercase. Affected > modifiers include: > I am sick so I am just not in the mood to read a whole PEP, but I do in general support the idea; +0. -Brett From bborcic at gmail.com Mon Mar 19 22:38:50 2007 From: bborcic at gmail.com (Boris Borcic) Date: Mon, 19 Mar 2007 22:38:50 +0100 Subject: [Python-3000] [OT] Re: String literal representation of integers (octal/binary discussion) In-Reply-To: References: Message-ID: Patrick Maupin wrote: > Most of the discussion on these issues occurred on the Python-3000 mailing > list starting 14-Mar-2007, prompted by Raymond Hettinger's observation > (cleverly couched as a question) that the average human being would be > completely mystified upon finding that prepending a "0" to a string of > digits changes the meaning of that digit string entirely. Reminds me a contrario (?) of a .sig I used for a while : 123 ? - the least natural integer (except perhaps for 12) that's symbolizing the whole set just by itself. Successor : 1234 Cheers, BB -- assert _304_ in _340343_, P424D15E_M15M47CH_E2202 From bob at redivi.com Mon Mar 19 23:04:19 2007 From: bob at redivi.com (Bob Ippolito) Date: Mon, 19 Mar 2007 15:04:19 -0700 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> Message-ID: <6a36e7290703191504p5af29065g5ff23c77fd0e9629@mail.gmail.com> On 3/18/07, Jim Jewett wrote: > On 3/18/07, Guido van Rossum wrote: > > Octal does need to be justified, since some people argued to remove > > it. I guess binary needs to be justified because Thomas doesn't see > > the need. :-) > > I see literals for octal and binary as similar to raising a tuple, or > automatically unpacking one in a function call. It can be useful, but > it isn't needed very often. When it does show up, it can be confusing > *because* it is so rare. > Binary is pretty useful for expressing some algorithms or for dealing with bit flags. Not something you do terribly often if you're living in an end to end Python universe but I've certainly wanted to use them now and again, mostly when parsing strange file formats. Instead I just ended up getting better at visualizing things that are conceptually binary as hex digits (4 bits at a time). It would absolutely help if there was an obvious way to get a string of 1s and 0s out of an integer given the number of bits you want to write it as (I've written libraries to do this and to work with bits as lists of 1 and 0). I'm not sure I care about literals anymore though... but I'm at least +0 on them. -bob From guido at python.org Mon Mar 19 23:27:59 2007 From: guido at python.org (Guido van Rossum) Date: Mon, 19 Mar 2007 15:27:59 -0700 Subject: [Python-3000] Compiler package and nonlocal statement? Message-ID: I spent some time today fixing the compiler package so that it uses the new metaclass syntax and generates the correct new-style code for class definitions. But when testing this, I found that there's another newfangled piece of syntax it doesn't understand: nonlocal. Can anyone who is more familiar with the nonlocal implementation fix this? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Tue Mar 20 01:16:33 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 20 Mar 2007 12:16:33 +1200 Subject: [Python-3000] String literal representation of integers (octal/binary discussion) In-Reply-To: <6a36e7290703191504p5af29065g5ff23c77fd0e9629@mail.gmail.com> References: <9e804ac0703181748j790646d4gdb0bdf3554b4c7b8@mail.gmail.com> <6a36e7290703191504p5af29065g5ff23c77fd0e9629@mail.gmail.com> Message-ID: <45FF27E1.2060504@canterbury.ac.nz> Bob Ippolito wrote: > It would absolutely help if there was an obvious way to get a string > of 1s and 0s out of an integer given the number of bits you want to > write it as The obvious thing would be "%bm.n" % i where n = maximum number of bits. -- Greg From mike.klaas at gmail.com Tue Mar 20 02:47:36 2007 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 19 Mar 2007 18:47:36 -0700 Subject: [Python-3000] Draft PEP for Requiring Lowercase Literal Modifiers In-Reply-To: References: Message-ID: <3d2ce8cb0703191847m74dccf0et4df08a082d4f0a46@mail.gmail.com> > Semantic Changes > ================ > > The behavior of the 'int' builtin when passed a radix of 0 will be changed to > follow the above grammar. This change is to maintain the specified behavior > [5]_ that a radix of 0 mirrors the literal syntax. The behavior of this > function will otherwise not be altered. In particular, the behavior of > accepting the prefix '0X' when a radix of 16 is specified will be kept for > backwards compatibility and easier parsing of data files. And float()? Will this break? In [9]: float('%E' % 3e-10) Out[9]: 3e-10 Changes like this don't do much for me. Sure, lowercase modifiers are prettier, but why remove the general functionality and leave in a few special cases, especially when functionality to display numbers in those formats exists? numeric literals: -0 string literals: +0 -Mike From mccollum at fas.harvard.edu Tue Mar 20 04:59:40 2007 From: mccollum at fas.harvard.edu (Andrew McCollum) Date: Mon, 19 Mar 2007 22:59:40 -0500 Subject: [Python-3000] Draft PEP for Requiring Lowercase Literal Modifiers In-Reply-To: <3d2ce8cb0703191847m74dccf0et4df08a082d4f0a46@mail.gmail.com> Message-ID: <200703200359.l2K3xfPM016621@us18.unix.fas.harvard.edu> > And float()? Will this break? > > In [9]: float('%E' % 3e-10) > Out[9]: 3e-10 I don't see any compelling reason to break this behavior. Also, float is different in that it already has special handling for values such as "Inf" and "NaN" (even if they are inconsistent across platforms), and makes no claim to follow the eval behavior, instead defining itself in terms of the underlying C library. > Changes like this don't do much for me. Sure, lowercase modifiers are > prettier, but why remove the general functionality and leave in a few > special cases, especially when functionality to display numbers in > those formats exists? It seems like your main objection is to the change disallowing the "0X" prefix when a 0 radix is passed to int, which is the only functionality suggested for removal by this PEP. I think there are legitimate arguments on both sides, which is why this is listed as the major open issue. I'm curious, how many people even use the autobase functionality. Google Codesearch turns up only a few hits (around 26, by my count), many of which are in the Python test suite itself. Most of the rest seem to be cases where the base of the number is in fact known, but the 0 radix is used anyway, perhaps out of laziness. -Andrew McCollum From mike.klaas at gmail.com Tue Mar 20 05:34:10 2007 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 19 Mar 2007 21:34:10 -0700 Subject: [Python-3000] Draft PEP for Requiring Lowercase Literal Modifiers In-Reply-To: <200703200359.l2K3xfPM016621@us18.unix.fas.harvard.edu> References: <3d2ce8cb0703191847m74dccf0et4df08a082d4f0a46@mail.gmail.com> <200703200359.l2K3xfPM016621@us18.unix.fas.harvard.edu> Message-ID: <3d2ce8cb0703192134p66480d56p75b39bc1d06262bd@mail.gmail.com> On 3/19/07, Andrew McCollum wrote: > > Changes like this don't do much for me. Sure, lowercase modifiers are > > prettier, but why remove the general functionality and leave in a few > > special cases, especially when functionality to display numbers in > > those formats exists? > > It seems like your main objection is to the change disallowing the "0X" > prefix when a 0 radix is passed to int, which is the only functionality > suggested for removal by this PEP. I think there are legitimate arguments > on both sides, which is why this is listed as the major open issue. No, that isn't an objection of mine. I wanted to ensure that float(), int() continued to be case agnostic for things like the exponent. The consistency argument (numeric strings should be parsed as if they were python literals) isn't terribly compelling, as you point out that there are already parseable strings which aren't literal numerals (inf, NaN). -Mike From collinw at gmail.com Tue Mar 20 05:50:47 2007 From: collinw at gmail.com (Collin Winter) Date: Mon, 19 Mar 2007 23:50:47 -0500 Subject: [Python-3000] 2to3 fixers Message-ID: <43aa6ff70703192150q49662e43ob09ac46dcba22173@mail.gmail.com> Hi all, I've added two new fixers to 2to3 this weekend, plus significant new functionality for a third: * fix_next handles the it.next() -> next(it) transition for PEP 3114. * fix_nonzero converts __nonzero__ to __bool__ methods, as mentioned in PEP 3100. * fix_tuple_params now fixes up tuple parameters in lambdas, per PEP 3113. I have a fixer for uppercase literal modifiers (UR"..." -> ur"...") penciled in, but I'm still waiting on a concrete PEP before I start work. Anyone still waiting for a fixer for his/her PEP (except for 3115, which is already on my list) should let me know. On the topic of PEPs, I'd appreciate it if all PEPs that rely on a 2to3 fixer as part of their transition plan could mention exactly which fixer belongs to that PEP. This helps match up fixers and PEPs, and it also keeps me from reading a PEP and having to worry if I've forgotten about a fixer I was supposed to write : ) These PEPs should also include a link to 2to3 (http://svn.python.org/view/sandbox/trunk/2to3/) so people reading the PEP can find the tool easily. Relatedly, it would greatly help me -- and, I imagine, others -- if new and existing PEPs could state exactly what they want their 2to3 fixer to do. Picking on PEP 3115 only because I have it open, saying ""'alternatively, it would not be too difficult to modify the syntax rules of the Py3K translation tool to convert from the old to the new syntax""" and leaving it at that doesn't give me any clue what the fixer should do. If you're writing a PEP and you're unsure as to what 2to3 can do, I'm happy to work with you and help figure out what kind of translation rules are possible/practical for your needs. Thanks, Collin Winter From g.brandl at gmx.net Tue Mar 20 08:59:28 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 20 Mar 2007 08:59:28 +0100 Subject: [Python-3000] 2to3 fixers In-Reply-To: <43aa6ff70703192150q49662e43ob09ac46dcba22173@mail.gmail.com> References: <43aa6ff70703192150q49662e43ob09ac46dcba22173@mail.gmail.com> Message-ID: Collin Winter schrieb: > Hi all, > > I've added two new fixers to 2to3 this weekend, plus significant new > functionality for a third: > > * fix_next handles the it.next() -> next(it) transition for PEP 3114. > * fix_nonzero converts __nonzero__ to __bool__ methods, as mentioned > in PEP 3100. > * fix_tuple_params now fixes up tuple parameters in lambdas, per PEP 3113. > > I have a fixer for uppercase literal modifiers (UR"..." -> ur"...") > penciled in, but I'm still waiting on a concrete PEP before I start > work. Anyone still waiting for a fixer for his/her PEP (except for > 3115, which is already on my list) should let me know. I already have a fixer for octal/hexadecimal literals here, together with my patch. Georg From oliphant.travis at ieee.org Tue Mar 20 11:19:37 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Tue, 20 Mar 2007 04:19:37 -0600 Subject: [Python-3000] Revised PEP for buffer protocol Message-ID: Attached is my revised PEP for the buffer protocol after incorporating suggestions from Greg Ewing. It is as simple as I can make it and still share what I think needs to be sharable. Suggestions are welcome. I will provide and maintain code to implement the PEP when the basic idea of the PEP is accepted. Thanks, -Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: pep_buffer.txt Url: http://mail.python.org/pipermail/python-3000/attachments/20070320/c2800d0e/attachment-0001.txt From jcarlson at uci.edu Tue Mar 20 18:44:58 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 20 Mar 2007 10:44:58 -0700 Subject: [Python-3000] Revised PEP for buffer protocol In-Reply-To: References: Message-ID: <20070320103212.FC4F.JCARLSON@uci.edu> "Travis E. Oliphant" wrote: > abstractly as if it were. I believe, the PIL is where the idea of > multiple buffer segments in the original buffer interface came > from, I believe. Remove the last "I believe" in that sentence and remove the commas. ;) > The buffer interface should allow discontiguous memory areas to > share standard striding information. However, consumers that do > not want to deal with strided memory should also be able to > request a contiguous segment easily. I don't believe this is necessary. While the point of the buffer interface is to offer direct access to memory regions of an object or structure, being able to ask "can I get a contiguous segment" isn't really reasonable. The response is either going to be "yes, that's how I represent it anyways" or "no, that's not how I represent the data". But this bit of meta information is easily acquired by *getting* the buffer and checking the stride. Otherwise, everything looks fine. I'm not convinced that function pointers need to be part of the specification. I can't think of a single positive use-case, and bad data is a security breach waiting to happpen, who has a use for this? - Josiah From brett at python.org Tue Mar 20 22:33:21 2007 From: brett at python.org (Brett Cannon) Date: Tue, 20 Mar 2007 14:33:21 -0700 Subject: [Python-3000] 2to3 fixers In-Reply-To: <43aa6ff70703192150q49662e43ob09ac46dcba22173@mail.gmail.com> References: <43aa6ff70703192150q49662e43ob09ac46dcba22173@mail.gmail.com> Message-ID: On 3/19/07, Collin Winter wrote: > Hi all, > > I've added two new fixers to 2to3 this weekend, plus significant new > functionality for a third: > > * fix_next handles the it.next() -> next(it) transition for PEP 3114. > * fix_nonzero converts __nonzero__ to __bool__ methods, as mentioned > in PEP 3100. > * fix_tuple_params now fixes up tuple parameters in lambdas, per PEP 3113. > As PEP 3113 is my PEP, here is my thanks to Collin for fleshing this fixer out! But thanks in general for helping with the transition toolchain. I will go update my PEPs as requested. -Brett From ncoghlan at gmail.com Tue Mar 20 23:29:39 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Mar 2007 08:29:39 +1000 Subject: [Python-3000] Revised PEP for buffer protocol In-Reply-To: <20070320103212.FC4F.JCARLSON@uci.edu> References: <20070320103212.FC4F.JCARLSON@uci.edu> Message-ID: <46006053.20006@gmail.com> Josiah Carlson wrote: > "Travis E. Oliphant" wrote: >> abstractly as if it were. I believe, the PIL is where the idea of >> multiple buffer segments in the original buffer interface came >> from, I believe. > > Remove the last "I believe" in that sentence and remove the commas. ;) > >> The buffer interface should allow discontiguous memory areas to >> share standard striding information. However, consumers that do >> not want to deal with strided memory should also be able to >> request a contiguous segment easily. > > I don't believe this is necessary. While the point of the buffer > interface is to offer direct access to memory regions of an object or > structure, being able to ask "can I get a contiguous segment" isn't > really reasonable. The response is either going to be "yes, that's how I > represent it anyways" or "no, that's not how I represent the data". But > this bit of meta information is easily acquired by *getting* the buffer > and checking the stride. I think the point is for there to be something in the standard library or Python core that makes it easy for a consumer to *copy* the data to a contiguous memory segment in the event the consumer can't directly handle non-contiguous data (e.g. a C API function that takes the source object, a pointer to the destination memory block, and an optional slice object defining a subsection of the memory block to be retrieved) Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From greg.ewing at canterbury.ac.nz Tue Mar 20 23:56:52 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 21 Mar 2007 10:56:52 +1200 Subject: [Python-3000] Revised PEP for buffer protocol In-Reply-To: <46006053.20006@gmail.com> References: <20070320103212.FC4F.JCARLSON@uci.edu> <46006053.20006@gmail.com> Message-ID: <460066B4.30902@canterbury.ac.nz> Nick Coghlan wrote: > I think the point is for there to be something in the standard library > or Python core that makes it easy for a consumer to *copy* the data to a > contiguous memory segment in the event the consumer can't directly > handle non-contiguous data It would be even more useful if the destination could be non-contiguous as well, but with a different stride. Then you could go from contiguous to non-contiguous, non-contiguous to contiguous, or repack between two different non-contiguous formats. -- Greg From jcarlson at uci.edu Wed Mar 21 00:14:33 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 20 Mar 2007 16:14:33 -0700 Subject: [Python-3000] Revised PEP for buffer protocol In-Reply-To: <46006053.20006@gmail.com> References: <20070320103212.FC4F.JCARLSON@uci.edu> <46006053.20006@gmail.com> Message-ID: <20070320161133.FC71.JCARLSON@uci.edu> Nick Coghlan wrote: > Josiah Carlson wrote: > > "Travis E. Oliphant" wrote: > >> The buffer interface should allow discontiguous memory areas to > >> share standard striding information. However, consumers that do > >> not want to deal with strided memory should also be able to > >> request a contiguous segment easily. > > > > I don't believe this is necessary. While the point of the buffer > > interface is to offer direct access to memory regions of an object or > > structure, being able to ask "can I get a contiguous segment" isn't > > really reasonable. The response is either going to be "yes, that's how I > > represent it anyways" or "no, that's not how I represent the data". But > > this bit of meta information is easily acquired by *getting* the buffer > > and checking the stride. > > I think the point is for there to be something in the standard library > or Python core that makes it easy for a consumer to *copy* the data to a > contiguous memory segment in the event the consumer can't directly > handle non-contiguous data (e.g. a C API function that takes the source > object, a pointer to the destination memory block, and an optional slice > object defining a subsection of the memory block to be retrieved) But that still isn't a use-case for the "I want a contiguous view". The consumer needs to construct a memory region, copy the non-contiguous data, then pass it on somewhere else. The object offering the original view shouldn't need to offer anything special to make it happen. - Josiah From jcarlson at uci.edu Wed Mar 21 00:16:25 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 20 Mar 2007 16:16:25 -0700 Subject: [Python-3000] Revised PEP for buffer protocol In-Reply-To: <460066B4.30902@canterbury.ac.nz> References: <46006053.20006@gmail.com> <460066B4.30902@canterbury.ac.nz> Message-ID: <20070320161456.FC74.JCARLSON@uci.edu> Greg Ewing wrote: > > Nick Coghlan wrote: > > > I think the point is for there to be something in the standard library > > or Python core that makes it easy for a consumer to *copy* the data to a > > contiguous memory segment in the event the consumer can't directly > > handle non-contiguous data > > It would be even more useful if the destination could > be non-contiguous as well, but with a different stride. > Then you could go from contiguous to non-contiguous, > non-contiguous to contiguous, or repack between two > different non-contiguous formats. With writable views, presumably that would be a[w:x] = b[y:z], for any such operation (assuming compatible values, etc.). - Josiah From collinw at gmail.com Wed Mar 21 00:22:27 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 20 Mar 2007 18:22:27 -0500 Subject: [Python-3000] Automated py3k doc builds Message-ID: <43aa6ff70703201622j1076821bo23b17999404349fa@mail.gmail.com> I just discovered that the p3yk/ branch can't do a docs build because almost none of the code in Doc/tools/ is valid Python 3 [1]. How hard would it be to set up something akin to http://docs.python.org/dev/ for p3yk (ie, regular, automated builds), if only to catch problems like this sooner? Collin Winter [1] - a patch to correct Doc/tools/* syntax is available as an attachment on bug #1684811. From brett at python.org Wed Mar 21 01:07:09 2007 From: brett at python.org (Brett Cannon) Date: Tue, 20 Mar 2007 17:07:09 -0700 Subject: [Python-3000] Automated py3k doc builds In-Reply-To: <43aa6ff70703201622j1076821bo23b17999404349fa@mail.gmail.com> References: <43aa6ff70703201622j1076821bo23b17999404349fa@mail.gmail.com> Message-ID: On 3/20/07, Collin Winter wrote: > I just discovered that the p3yk/ branch can't do a docs build because > almost none of the code in Doc/tools/ is valid Python 3 [1]. How hard > would it be to set up something akin to http://docs.python.org/dev/ > for p3yk (ie, regular, automated builds), if only to catch problems > like this sooner? This also begs the question of whether there is anything in our docs toolchain we want to change? I have always found it annoying that I have never been able to get the docs to build for myself on my machine. Raymond's texcheck helps a lot to help prevent errors, but it is unfortunate to not be able to build changed docs to catch errors in the LaTeX. -Brett From collinw at gmail.com Wed Mar 21 02:10:14 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 20 Mar 2007 20:10:14 -0500 Subject: [Python-3000] Total ordering and __cmp__ Message-ID: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Quoting from the commit message for r51533, which removed the default ordering: """ A general problem with getting lots of these tests to pass is the reality that for object types that have a natural total ordering, implementing __cmp__ is much more convenient than implementing __eq__, __ne__, __lt__, and so on. Should we go back to allowing __cmp__ to provide a total ordering? Should we provide some other way to implement rich comparison with a single method override? Alex proposed a __key__() method; I've considered a __richcmp__() method. Or perhaps __cmp__() just shouldn't be killed off... """ What's the status on this? FWIW, I would like to see __cmp__ stay as a fallback measure if __eq__, __ne__, __lt__, etc aren't defined for total ordering purposes. I've run across this while trying to get the docs toolchain working, and I was initially shocked that __cmp__ was no longer called. Collin Winter From guido at python.org Wed Mar 21 02:30:40 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Mar 2007 17:30:40 -0800 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: I haven't taken any action, and it looks like __cmp__ isn't being called. I'd rather not add it back; if you want it back, could you at least write up a brief PEP? A patch would also help; I recall that it was quite a relief being able to cut it out, so I expect that patching it back in would be quite cumbersome. --Guido On 3/20/07, Collin Winter wrote: > Quoting from the commit message for r51533, which removed the default ordering: > > """ > A general problem with getting lots of these tests to pass is > the reality that for object types that have a natural total ordering, > implementing __cmp__ is much more convenient than implementing > __eq__, __ne__, __lt__, and so on. Should we go back to allowing > __cmp__ to provide a total ordering? Should we provide some other > way to implement rich comparison with a single method override? > Alex proposed a __key__() method; I've considered a __richcmp__() > method. Or perhaps __cmp__() just shouldn't be killed off... > """ > > What's the status on this? FWIW, I would like to see __cmp__ stay as a > fallback measure if __eq__, __ne__, __lt__, etc aren't defined for > total ordering purposes. I've run across this while trying to get the > docs toolchain working, and I was initially shocked that __cmp__ was > no longer called. > > Collin Winter > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Wed Mar 21 03:40:56 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 20 Mar 2007 22:40:56 -0400 Subject: [Python-3000] Total ordering and __cmp__ References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: "Collin Winter" wrote in message news:43aa6ff70703201810p9d731b8v3497a48539d5dc84 at mail.gmail.com... | Quoting from the commit message for r51533, which removed the default ordering: | | What's the status on this? FWIW, I would like to see __cmp__ stay as a | fallback measure if __eq__, __ne__, __lt__, etc aren't defined for | total ordering purposes. I've run across this while trying to get the | docs toolchain working, and I was initially shocked that __cmp__ was | no longer called. An alternative would be to give the object class comparison methods defined in terms of a missing __cmp() method. def __eq__(self,other): return __cmp(self,other) == 0 # etc This would put the fallback to and conversion of cmp -1,0,1 output in *one* place instead of sprinkled everywhere. Or these could go into a cmp_order mixin put in some lib module. Terry Jan Reedy From guido at python.org Wed Mar 21 03:46:28 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Mar 2007 19:46:28 -0700 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: But can this work? It might end up calling cmp() on two incomparable objects. For __lt__ etc. that might be the right answer, but for __eq__ it is *not* -- the default __eq__ and __ne__ *must* compare object identity. (What is this __cmp you refer to? A typo for __cmp__ or for cmp?) --Guido On 3/20/07, Terry Reedy wrote: > > "Collin Winter" wrote in message > news:43aa6ff70703201810p9d731b8v3497a48539d5dc84 at mail.gmail.com... > | Quoting from the commit message for r51533, which removed the default > ordering: > | > | What's the status on this? FWIW, I would like to see __cmp__ stay as a > | fallback measure if __eq__, __ne__, __lt__, etc aren't defined for > | total ordering purposes. I've run across this while trying to get the > | docs toolchain working, and I was initially shocked that __cmp__ was > | no longer called. > > An alternative would be to give the object class comparison methods defined > in terms of a missing __cmp() method. > > def __eq__(self,other): return __cmp(self,other) == 0 # etc > > This would put the fallback to and conversion of cmp -1,0,1 output in *one* > place instead of sprinkled everywhere. Or these could go into a cmp_order > mixin put in some lib module. > > Terry Jan Reedy > > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Wed Mar 21 04:36:28 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 20 Mar 2007 21:36:28 -0600 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: On 3/20/07, Collin Winter wrote: > Quoting from the commit message for r51533, which removed the default ordering: > > """ > A general problem with getting lots of these tests to pass is > the reality that for object types that have a natural total ordering, > implementing __cmp__ is much more convenient than implementing > __eq__, __ne__, __lt__, and so on. Should we go back to allowing > __cmp__ to provide a total ordering? Should we provide some other > way to implement rich comparison with a single method override? > Alex proposed a __key__() method I've used a __key__() method quite successfully in my own code. Maybe we should provide a mixin like:: class KeyedComparisonMixin(object): def __eq__(self, other): return self.__key__() == other.__key__() def __ne__(self, other): return self.__key__() != other.__key__() def __lt__(self, other): return self.__key__() < other.__key__() def __le__(self, other): return self.__key__() <= other.__key__() def __gt__(self, other): return self.__key__() > other.__key__() def __ge__(self, other): return self.__key__() >= other.__key__() That way, any classes that needed __cmp__-like behavior could request it explicitly by using the mixin. (If people really want it, we could define a __cmp__-based mixin instead, but I've *always* found the __key__-based mixin to be a better approach in my own code.) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Wed Mar 21 04:44:43 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Mar 2007 20:44:43 -0700 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: How would that produce the desired behavior that == and != are defined on all objects, but <, <=, >= and > are not? We're quickly approaching python-ideas land... :) On 3/20/07, Steven Bethard wrote: > On 3/20/07, Collin Winter wrote: > > Quoting from the commit message for r51533, which removed the default ordering: > > > > """ > > A general problem with getting lots of these tests to pass is > > the reality that for object types that have a natural total ordering, > > implementing __cmp__ is much more convenient than implementing > > __eq__, __ne__, __lt__, and so on. Should we go back to allowing > > __cmp__ to provide a total ordering? Should we provide some other > > way to implement rich comparison with a single method override? > > Alex proposed a __key__() method > > I've used a __key__() method quite successfully in my own code. Maybe > we should provide a mixin like:: > > class KeyedComparisonMixin(object): > def __eq__(self, other): > return self.__key__() == other.__key__() > def __ne__(self, other): > return self.__key__() != other.__key__() > def __lt__(self, other): > return self.__key__() < other.__key__() > def __le__(self, other): > return self.__key__() <= other.__key__() > def __gt__(self, other): > return self.__key__() > other.__key__() > def __ge__(self, other): > return self.__key__() >= other.__key__() > > That way, any classes that needed __cmp__-like behavior could request > it explicitly by using the mixin. (If people really want it, we could > define a __cmp__-based mixin instead, but I've *always* found the > __key__-based mixin to be a better approach in my own code.) > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Wed Mar 21 04:57:49 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 20 Mar 2007 22:57:49 -0500 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> On 3/20/07, Guido van Rossum wrote: > I haven't taken any action, and it looks like __cmp__ isn't being > called. I'd rather not add it back; if you want it back, could you at > least write up a brief PEP? A patch would also help; I recall that it > was quite a relief being able to cut it out, so I expect that patching > it back in would be quite cumbersome. I'm not wild about the idea of reintroducing it. I'd at least like some kind of transition strategy that 2to3 can help with (if not implement entirely). Perhaps something like, "if a class defines a __cmp__ method but not __lt__, __gt__, __ge__, etc, 2to3 will insert those methods, implemented by wrapping calls to cmp() as appropriate". Collin Winter From nnorwitz at gmail.com Wed Mar 21 05:00:31 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 20 Mar 2007 21:00:31 -0700 Subject: [Python-3000] Automated py3k doc builds In-Reply-To: <43aa6ff70703201622j1076821bo23b17999404349fa@mail.gmail.com> References: <43aa6ff70703201622j1076821bo23b17999404349fa@mail.gmail.com> Message-ID: http://docs.python.org/dev/3.x/ will be updated every hour. http://docs.python.org/dev/3.x/results/ contains the results of the last build. n -- On 3/20/07, Collin Winter wrote: > I just discovered that the p3yk/ branch can't do a docs build because > almost none of the code in Doc/tools/ is valid Python 3 [1]. How hard > would it be to set up something akin to http://docs.python.org/dev/ > for p3yk (ie, regular, automated builds), if only to catch problems > like this sooner? > > Collin Winter > > [1] - a patch to correct Doc/tools/* syntax is available as an > attachment on bug #1684811. > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/nnorwitz%40gmail.com > From collinw at gmail.com Wed Mar 21 05:02:41 2007 From: collinw at gmail.com (Collin Winter) Date: Tue, 20 Mar 2007 23:02:41 -0500 Subject: [Python-3000] Automated py3k doc builds In-Reply-To: References: <43aa6ff70703201622j1076821bo23b17999404349fa@mail.gmail.com> Message-ID: <43aa6ff70703202102w4ca31499n1bfe645166c00523@mail.gmail.com> On 3/20/07, Neal Norwitz wrote: > http://docs.python.org/dev/3.x/ > > will be updated every hour. > > http://docs.python.org/dev/3.x/results/ > > contains the results of the last build. Woohoo! Thanks, Neal! From guido at python.org Wed Mar 21 05:16:54 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Mar 2007 21:16:54 -0700 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> Message-ID: On 3/20/07, Collin Winter wrote: > On 3/20/07, Guido van Rossum wrote: > > I haven't taken any action, and it looks like __cmp__ isn't being > > called. I'd rather not add it back; if you want it back, could you at > > least write up a brief PEP? A patch would also help; I recall that it > > was quite a relief being able to cut it out, so I expect that patching > > it back in would be quite cumbersome. > > I'm not wild about the idea of reintroducing it. I'd at least like > some kind of transition strategy that 2to3 can help with (if not > implement entirely). Perhaps something like, "if a class defines a > __cmp__ method but not __lt__, __gt__, __ge__, etc, 2to3 will insert > those methods, implemented by wrapping calls to cmp() as appropriate". Or we could just have 2.6 warn about the presence (or use) of __cmp__ -- users can write equivalent code using __lt__ etc. themselves and probably do a better job. Inserting six new methods sounds like a bit of a heavy operation for 2to3. The goal of 2to3 is not to convert all possible programs; it is to make it reasonable to write code that runs warning-free under 2.6 and and can be converted without manual massaging to correct, equivalent 3.0 code. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steven.bethard at gmail.com Wed Mar 21 06:13:06 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 20 Mar 2007 23:13:06 -0600 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: On 3/20/07, Steven Bethard wrote: > I've used a __key__() method quite successfully in my own code. Maybe > we should provide a mixin like:: > > class KeyedComparisonMixin(object): > def __eq__(self, other): > return self.__key__() == other.__key__() [snip] > def __ge__(self, other): > return self.__key__() >= other.__key__() On 3/20/07, Guido van Rossum wrote: > How would that produce the desired behavior that == and != are defined > on all objects, but <, <=, >= and > are not? The intent was that this would be a mixin like DictMixin -- not default behavior at all. It's roughly parallel to what used to happen when you defined __cmp__, so anyone that felt that defining __lt__, __gt__, etc. was too tedious could "opt in" by adding the mixin to their class bases. I wasn't suggesting that this mixin should ever be added implicitly by Python. As far as Python 2.6 code goes, I think simply warning about any uses of __cmp__ should be fine. (I'd also be nervous about 2to3 generating something like the code above automatically.) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From tjreedy at udel.edu Wed Mar 21 06:52:02 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Mar 2007 01:52:02 -0400 Subject: [Python-3000] Total ordering and __cmp__ References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: "Guido van Rossum" wrote in message news:ca471dc20703201946p722b2ad2q2b4c39e58e12b7d7 at mail.gmail.com... | But can this work? It might end up calling cmp() on two incomparable | objects. For __lt__ etc. that might be the right answer, but for | __eq__ it is *not* -- the default __eq__ and __ne__ *must* compare | object identity. Sorry, leave them out then unless using a mixin. I was responding to Colin's wish that people be able to write one __cmp__ method instead of six and thinking about something like what Steven uses but based on __cmp__ instead of__key__. I am not thinking of any use cases at the moment could imagine this instead being a cookbook recipe. Or something to consider if there is substantantial objection to the demise of cmp as a alternative to keeping it. But maybe there will not be. | (What is this __cmp you refer to? A typo for __cmp__ or for cmp?) A substitute private name for __cmp__ if the latter is to be deprecated (and warned about). Terry Jan Reedy From nnorwitz at gmail.com Wed Mar 21 07:01:22 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 20 Mar 2007 23:01:22 -0700 Subject: [Python-3000] refleaks and other errors Message-ID: regrtest.py -R 4:3: # on 64-bit, but that might only affect the xpickle error test_grammar leaked [14, 14, 14] references test_doctest leaked [84, 84, 84] references test_ctypes leaked [13, 13, 13] references test_descrtut leaked [417, 417, 417] references test_generators leaked [227, 227, 227] references test_metaclass leaked [284, 284, 284] references test_modulefinder leaked [186, 186, 186] references test_scope leaked [78, 78, 78] references test_threading_local leaked [102, 102, 102] references test_unpack leaked [55, 55, 55] references test_weakref leaked [79, 79, 79] references test test_cpickle failed -- errors occurred; run in verbose mode for details test test_io failed -- errors occurred; run in verbose mode for details test test_structmembers failed -- errors occurred; run in verbose mode for details test test_fileio failed -- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/test_fileio.py", line 37, in testSeekTell self.f.seek(0) IOError: [Errno 22] Invalid argument test_socket_ssl skipped -- socket module has no ssl support test test_xpickle failed -- Traceback (most recent call last): File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line 503, in test_ints self.assertEqual(expected, n2) AssertionError: -9223372036854775807 != 1 test test_xreload failed -- Traceback (most recent call last): test_grammar has these two cases leaking 7 refs each: class foo:return 1 class foo:yield 1 From guido at python.org Wed Mar 21 07:42:49 2007 From: guido at python.org (Guido van Rossum) Date: Tue, 20 Mar 2007 23:42:49 -0700 Subject: [Python-3000] refleaks and other errors In-Reply-To: References: Message-ID: I'm pretty sure I introduced leaks with the new metaclass code; it was a bit of a rush job and there's plenty of new C code. I need a hint on how to reproduce this myself for a single test... On 3/20/07, Neal Norwitz wrote: > regrtest.py -R 4:3: # on 64-bit, but that might only affect the xpickle error > > test_grammar leaked [14, 14, 14] references > test_doctest leaked [84, 84, 84] references > test_ctypes leaked [13, 13, 13] references > test_descrtut leaked [417, 417, 417] references > test_generators leaked [227, 227, 227] references > test_metaclass leaked [284, 284, 284] references > test_modulefinder leaked [186, 186, 186] references > test_scope leaked [78, 78, 78] references > test_threading_local leaked [102, 102, 102] references > test_unpack leaked [55, 55, 55] references > test_weakref leaked [79, 79, 79] references > > > test test_cpickle failed -- errors occurred; run in verbose mode for details > test test_io failed -- errors occurred; run in verbose mode for details > test test_structmembers failed -- errors occurred; run in verbose mode > for details > > test test_fileio failed -- Traceback (most recent call last): > File "/home/neal/python/dev/py3k/Lib/test/test_fileio.py", line 37, > in testSeekTell > self.f.seek(0) > IOError: [Errno 22] Invalid argument > > test_socket_ssl skipped -- socket module has no ssl support > > test test_xpickle failed -- Traceback (most recent call last): > File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line > 503, in test_ints > self.assertEqual(expected, n2) > AssertionError: -9223372036854775807 != 1 > > test test_xreload failed -- Traceback (most recent call last): > > > test_grammar has these two cases leaking 7 refs each: > class foo:return 1 > class foo:yield 1 > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Wed Mar 21 07:49:01 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 20 Mar 2007 23:49:01 -0700 Subject: [Python-3000] refleaks and other errors In-Reply-To: References: Message-ID: That was mostly the first line of my message, although it was a bit terse. :-) It's just a cmd line option to regrtest.py -R 4:3: Everything else is the same. The way I run it for test_grammar: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: test_grammar You can also pass "-R ::" instead of "-R 4:3:" however that takes longer and generally doesn't help much. Note that some tests that normally pass will fail (not the refleaks, actually fail) with this option due to the test not cleaning up after itself. n -- On 3/20/07, Guido van Rossum wrote: > I'm pretty sure I introduced leaks with the new metaclass code; it was > a bit of a rush job and there's plenty of new C code. I need a hint on > how to reproduce this myself for a single test... > > On 3/20/07, Neal Norwitz wrote: > > regrtest.py -R 4:3: # on 64-bit, but that might only affect the xpickle error > > > > test_grammar leaked [14, 14, 14] references > > test_doctest leaked [84, 84, 84] references > > test_ctypes leaked [13, 13, 13] references > > test_descrtut leaked [417, 417, 417] references > > test_generators leaked [227, 227, 227] references > > test_metaclass leaked [284, 284, 284] references > > test_modulefinder leaked [186, 186, 186] references > > test_scope leaked [78, 78, 78] references > > test_threading_local leaked [102, 102, 102] references > > test_unpack leaked [55, 55, 55] references > > test_weakref leaked [79, 79, 79] references > > > > > > test test_cpickle failed -- errors occurred; run in verbose mode for details > > test test_io failed -- errors occurred; run in verbose mode for details > > test test_structmembers failed -- errors occurred; run in verbose mode > > for details > > > > test test_fileio failed -- Traceback (most recent call last): > > File "/home/neal/python/dev/py3k/Lib/test/test_fileio.py", line 37, > > in testSeekTell > > self.f.seek(0) > > IOError: [Errno 22] Invalid argument > > > > test_socket_ssl skipped -- socket module has no ssl support > > > > test test_xpickle failed -- Traceback (most recent call last): > > File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line > > 503, in test_ints > > self.assertEqual(expected, n2) > > AssertionError: -9223372036854775807 != 1 > > > > test test_xreload failed -- Traceback (most recent call last): > > > > > > test_grammar has these two cases leaking 7 refs each: > > class foo:return 1 > > class foo:yield 1 > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From g.brandl at gmx.net Wed Mar 21 09:45:03 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Mar 2007 09:45:03 +0100 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: Guido van Rossum schrieb: > I haven't taken any action, and it looks like __cmp__ isn't being > called. I'd rather not add it back; if you want it back, could you at > least write up a brief PEP? A patch would also help; I recall that it > was quite a relief being able to cut it out, so I expect that patching > it back in would be quite cumbersome. BTW, was it intentional that dicts can't be compared via ">" and "<" anymore? Georg From ncoghlan at gmail.com Wed Mar 21 10:47:19 2007 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 21 Mar 2007 19:47:19 +1000 Subject: [Python-3000] Revised PEP for buffer protocol In-Reply-To: <20070320161133.FC71.JCARLSON@uci.edu> References: <20070320103212.FC4F.JCARLSON@uci.edu> <46006053.20006@gmail.com> <20070320161133.FC71.JCARLSON@uci.edu> Message-ID: <4600FF27.7090108@gmail.com> Josiah Carlson wrote: > Nick Coghlan wrote: >> Josiah Carlson wrote: >>> "Travis E. Oliphant" wrote: >>>> The buffer interface should allow discontiguous memory areas to >>>> share standard striding information. However, consumers that do >>>> not want to deal with strided memory should also be able to >>>> request a contiguous segment easily. >>> I don't believe this is necessary. While the point of the buffer >>> interface is to offer direct access to memory regions of an object or >>> structure, being able to ask "can I get a contiguous segment" isn't >>> really reasonable. The response is either going to be "yes, that's how I >>> represent it anyways" or "no, that's not how I represent the data". But >>> this bit of meta information is easily acquired by *getting* the buffer >>> and checking the stride. >> I think the point is for there to be something in the standard library >> or Python core that makes it easy for a consumer to *copy* the data to a >> contiguous memory segment in the event the consumer can't directly >> handle non-contiguous data (e.g. a C API function that takes the source >> object, a pointer to the destination memory block, and an optional slice >> object defining a subsection of the memory block to be retrieved) > > But that still isn't a use-case for the "I want a contiguous view". The > consumer needs to construct a memory region, copy the non-contiguous > data, then pass it on somewhere else. The object offering the original > view shouldn't need to offer anything special to make it happen. It's a use case that any C API updates associated with this PEP need to handle though. It's along the lines of a type supporting 3 different abstract C API functions just by providing a tp_index slot. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From jcarlson at uci.edu Wed Mar 21 16:14:14 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 21 Mar 2007 08:14:14 -0700 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: Message-ID: <20070321080859.FC83.JCARLSON@uci.edu> Georg Brandl wrote: > Guido van Rossum schrieb: > > I haven't taken any action, and it looks like __cmp__ isn't being > > called. I'd rather not add it back; if you want it back, could you at > > least write up a brief PEP? A patch would also help; I recall that it > > was quite a relief being able to cut it out, so I expect that patching > > it back in would be quite cumbersome. > > BTW, was it intentional that dicts can't be compared via ">" and "<" > anymore? Being that dictionaries are *unordered* mappings of keys to values, is there an ordering on dictionaries that makes any sense? If I understand the changes in comparisons correctly, the point is to remove a total ordering on all Python objects, as it is currently broken. For objects that have no reasonable ordering (imaginary vs float, tuples vs unicode, set vs. set, etc.), the only comparisons that return anything useful are x == y and x != y. - Josiah From g.brandl at gmx.net Wed Mar 21 16:29:45 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Mar 2007 16:29:45 +0100 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <20070321080859.FC83.JCARLSON@uci.edu> References: <20070321080859.FC83.JCARLSON@uci.edu> Message-ID: Josiah Carlson schrieb: > Georg Brandl wrote: >> Guido van Rossum schrieb: >> > I haven't taken any action, and it looks like __cmp__ isn't being >> > called. I'd rather not add it back; if you want it back, could you at >> > least write up a brief PEP? A patch would also help; I recall that it >> > was quite a relief being able to cut it out, so I expect that patching >> > it back in would be quite cumbersome. >> >> BTW, was it intentional that dicts can't be compared via ">" and "<" >> anymore? > > Being that dictionaries are *unordered* mappings of keys to values, is > there an ordering on dictionaries that makes any sense? If I understand > the changes in comparisons correctly, the point is to remove a total > ordering on all Python objects, as it is currently broken. For objects > that have no reasonable ordering (imaginary vs float, tuples vs unicode, > set vs. set, etc.), the only comparisons that return anything useful are > x == y and x != y. Yes, but dictionaries had an explicit ordering in dict_compare() which was deleted. Georg From guido at python.org Wed Mar 21 18:19:37 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Mar 2007 10:19:37 -0700 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: On 3/21/07, Georg Brandl wrote: > BTW, was it intentional that dicts can't be compared via ">" and "<" > anymore? Yes. In old Python it made sense since all objects were supposed to be orderable *somehow* (in the truly distant past, comparisons weren't even allowed to raise exceptions!). In Py3k that assumption has been dropped. The ordering for dicts is not particularly useful, hard to explain, and hard to compute. On the other hand, equality testing is straightforward. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Wed Mar 21 19:52:01 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Mar 2007 14:52:01 -0400 Subject: [Python-3000] Total ordering and __cmp__ References: <20070321080859.FC83.JCARLSON@uci.edu> Message-ID: "Georg Brandl" wrote in message news:etrj1a$l9a$1 at sea.gmane.org... | Yes, but dictionaries had an explicit ordering in dict_compare() which was | deleted. Is dict_compare something added in 2.5? It is neither a builtin or dict method in 2.4. In any case, this point is that dict ordering is as arbitrary as ordering, for instance, a dict and a string. Since Guido stopped the experiment of totally ordering all objects when complex nums were added, consistency suggests that all fake orderings be eliminated, leaving only the order of numbers, characters, and sequences of comparable objects. Terry Jan Reedy From bjourne at gmail.com Wed Mar 21 19:54:13 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 21 Mar 2007 19:54:13 +0100 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws In-Reply-To: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> Message-ID: <740c3aec0703211154u5f151dcel817b4f2e253d9d99@mail.gmail.com> No comments at all. :( Did I send the mail to the wrong list? Either or, I still would like to know what the py3k rules are for repairing broken API:s. On 3/14/07, BJ?rn Lindqvist wrote: > I have a question about PEP 3001: > > """The last and most disruptive change is the overhaul of a module's > public interface. If a module's interface is to be changed, a > justification should be made beforehand, or a PEP should be written. > > The change must be fully documented as "New in Version 3.0", and the > python3warn.py script must know about it.""" > > It seems like its wording makes it very hard to fix broken or bad > stuff in old modules. Many modules are very old and contain lots of > legacy stuff that is kept to preserve backwards compatibility. Other > modules exposes a non-optimal (or even bad) API because it must not be > changed. > > So I wonder what is the policy for fixing mistakes in the API design? > Is a PEP really needed? For example, here are some bugs in the > threading module: > > activeCount() # Redundant, use len(threading.enumerate()) > currentThread() # PEP8 -> current_thread() > class local # PEP8 -> class Local > > Is there anything that can be done about it? > > For another example, take the Telnet class in the telnetlib module. It > has a method set_option_negotiation_callback() which takes a function > that will be called for each telnet option read. The default behaviour > for the Telnet class is to refuse all negotiation requests, but using > a negotiation callback you can override that. > > However, using a callback does not work so well because the function > acting on the telnet options read still needs to reference the Telnet > class to get hold of negotiation data using read_sb_data(). The > problem is non-lethal but a small annoyance to advanced Telnet > users. See SourceForge patches #1520081, #664020 and #1678077. > > The right design would have been to have a method (handle_option) in > the class that handles all options and, by default, refuses > them. Users of the Telnet class could then subclass Telnet and > override the handle_option method to implement their application > specific option handling. > > Can or should API bugs like these be fixed for Python 3.0? > > > -- > mvh Bj?rn > -- mvh Bj?rn From collinw at gmail.com Wed Mar 21 20:22:07 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 21 Mar 2007 14:22:07 -0500 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> Message-ID: <43aa6ff70703211222i40e00755o9cf5730b625b5309@mail.gmail.com> On 3/20/07, Guido van Rossum wrote: [snip] > Or we could just have 2.6 warn about the presence (or use) of __cmp__ > -- users can write equivalent code using __lt__ etc. themselves and > probably do a better job. Fair enough. Is anyone collecting a list of these --py3k warnings? There seem to be a lot recent changes that 2.6's py3k-compat mode will be handling, but I haven't seen anything about someone tracking them/working on a PEP on the issue. Or did I just volunteer myself for that? : ) Collin Winter From steven.bethard at gmail.com Wed Mar 21 20:38:32 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 21 Mar 2007 13:38:32 -0600 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws In-Reply-To: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> Message-ID: On 3/14/07, BJ?rn Lindqvist wrote: > So I wonder what is the policy for fixing mistakes in the API design? In general, I think if you can warn about it in 2.6 or you can add to 2to3 so that this gets automatically fixed, then it's okay to fix the API. > Is a PEP really needed? For example, here are some bugs in the > threading module: > > activeCount() # Redundant, use len(threading.enumerate()) Removing this should be alright as long as Python 2.6 warns about it with the -Wpy3k flag. > currentThread() # PEP8 -> current_thread() > class local # PEP8 -> class Local PEP 3108 (http://www.python.org/dev/peps/pep-3108/) and PEP 364 (http://www.python.org/dev/peps/pep-0364/) address some of the issues in renaming things, though I don't think they address renaming anything but modules. Of course, for Python 2.6 it's pretty simple to make one spelling of the class/function/method an alias of the other. But I don't see an obvious way to do this except manually. PEP 364 should probably address this. > For another example, take the Telnet class in the telnetlib module. It > has a method set_option_negotiation_callback() which takes a function > that will be called for each telnet option read. The default behaviour > for the Telnet class is to refuse all negotiation requests, but using > a negotiation callback you can override that. > > However, using a callback does not work so well because the function > acting on the telnet options read still needs to reference the Telnet > class to get hold of negotiation data using read_sb_data(). The > problem is non-lethal but a small annoyance to advanced Telnet > users. See SourceForge patches #1520081, #664020 and #1678077. > > The right design would have been to have a method (handle_option) in > the class that handles all options and, by default, refuses > them. Users of the Telnet class could then subclass Telnet and > override the handle_option method to implement their application > specific option handling. Seems like this could be done now (for Python 2.6), no? Just factor out the code that decides whether or not to call self.option_callback into a handle_option() method and call that. I'm not sure I see how Python 3000 comes into it... STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From tjreedy at udel.edu Wed Mar 21 20:43:07 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Mar 2007 15:43:07 -0400 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> <740c3aec0703211154u5f151dcel817b4f2e253d9d99@mail.gmail.com> Message-ID: "BJ?rn Lindqvist" wrote in message news:740c3aec0703211154u5f151dcel817b4f2e253d9d99 at mail.gmail.com... | No comments at all. :( Did I send the mail to the wrong list? No, perhaps everyone did as I did, and assumed someone else would answer ;-) | Either or, I still would like to know what the py3k rules are for | repairing broken API:s. I don't remember there being much discussion of this yet. I believe there is agreement to change names of modules to comform with PEP 8, but I suspect this a easier to accomodate in the autofixer than names within modules. My personal position, at least for threading and telnet, would be change away, since I have never used either and would not be affected. But I know others are in the opposite position, so I would not give mine much weight. It also seems to me that module fixups might better wait until the kernal and fixup code are more stable. Terry Jan Reedy From rhamph at gmail.com Wed Mar 21 20:47:55 2007 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 21 Mar 2007 13:47:55 -0600 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: On 3/20/07, Steven Bethard wrote: > On 3/20/07, Collin Winter wrote: > > Quoting from the commit message for r51533, which removed the default ordering: > > > > """ > > A general problem with getting lots of these tests to pass is > > the reality that for object types that have a natural total ordering, > > implementing __cmp__ is much more convenient than implementing > > __eq__, __ne__, __lt__, and so on. Should we go back to allowing > > __cmp__ to provide a total ordering? Should we provide some other > > way to implement rich comparison with a single method override? > > Alex proposed a __key__() method > > I've used a __key__() method quite successfully in my own code. Maybe > we should provide a mixin like:: > > class KeyedComparisonMixin(object): > def __eq__(self, other): > return self.__key__() == other.__key__() > def __ne__(self, other): > return self.__key__() != other.__key__() > def __lt__(self, other): > return self.__key__() < other.__key__() > def __le__(self, other): > return self.__key__() <= other.__key__() > def __gt__(self, other): > return self.__key__() > other.__key__() > def __ge__(self, other): > return self.__key__() >= other.__key__() > > That way, any classes that needed __cmp__-like behavior could request > it explicitly by using the mixin. (If people really want it, we could > define a __cmp__-based mixin instead, but I've *always* found the > __key__-based mixin to be a better approach in my own code.) This seems to match what I've usually needed, but I'm not sure it's worth putting in python proper. How about a cookbook entry? It would also be nice to reference in a guide on making code 3.0-ready. -- Adam Olsen, aka Rhamphoryncus From rhamph at gmail.com Wed Mar 21 20:51:40 2007 From: rhamph at gmail.com (Adam Olsen) Date: Wed, 21 Mar 2007 13:51:40 -0600 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <43aa6ff70703211222i40e00755o9cf5730b625b5309@mail.gmail.com> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> <43aa6ff70703211222i40e00755o9cf5730b625b5309@mail.gmail.com> Message-ID: On 3/21/07, Collin Winter wrote: > On 3/20/07, Guido van Rossum wrote: > [snip] > > Or we could just have 2.6 warn about the presence (or use) of __cmp__ > > -- users can write equivalent code using __lt__ etc. themselves and > > probably do a better job. > > Fair enough. > > Is anyone collecting a list of these --py3k warnings? There seem to be > a lot recent changes that 2.6's py3k-compat mode will be handling, but > I haven't seen anything about someone tracking them/working on a PEP > on the issue. Or did I just volunteer myself for that? : ) PEP 361 lists some, although I'm sure it could use some love and care. I would especially appreciate clarity on what the warning flags are, what warnings are active under each flag, etc. Anthony Baxter has patches for two of the warnings: http://sourceforge.net/tracker/index.php?func=detail&aid=1631035&group_id=5470&atid=305470 I have a patch for several more: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1683908&group_id=5470 -- Adam Olsen, aka Rhamphoryncus From steven.bethard at gmail.com Wed Mar 21 21:24:25 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 21 Mar 2007 14:24:25 -0600 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: On 3/21/07, Adam Olsen wrote: > On 3/20/07, Steven Bethard wrote: > > I've used a __key__() method quite successfully in my own code. Maybe > > we should provide a mixin like:: > > > > class KeyedComparisonMixin(object): > > def __eq__(self, other): > > return self.__key__() == other.__key__() [snip] > This seems to match what I've usually needed, but I'm not sure it's > worth putting in python proper. How about a cookbook entry? It would > also be nice to reference in a guide on making code 3.0-ready. Done. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/510403 STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From thomas at python.org Wed Mar 21 21:40:57 2007 From: thomas at python.org (Thomas Wouters) Date: Wed, 21 Mar 2007 21:40:57 +0100 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <43aa6ff70703211222i40e00755o9cf5730b625b5309@mail.gmail.com> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> <43aa6ff70703211222i40e00755o9cf5730b625b5309@mail.gmail.com> Message-ID: <9e804ac0703211340k733ec339j9a7cea58bf5b4be8@mail.gmail.com> On 3/21/07, Collin Winter wrote: > > On 3/20/07, Guido van Rossum wrote: > [snip] > > Or we could just have 2.6 warn about the presence (or use) of __cmp__ > > -- users can write equivalent code using __lt__ etc. themselves and > > probably do a better job. > > Fair enough. > > Is anyone collecting a list of these --py3k warnings? There seem to be > a lot recent changes that 2.6's py3k-compat mode will be handling, but > I haven't seen anything about someone tracking them/working on a PEP > on the issue. Or did I just volunteer myself for that? : ) Neal and I have been updating PEP361, and I will be keeping an eye on it when I refactor the p3yk branch into separate py3k-feature-branches (be it bazaar or mercurial or whatever.) Each conceptual change in py3k will be a separate branch, so keeping track of what needs to be warned will be easier. (But the practical consequences of this will be clearer after I give it some concrete form :) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20070321/e9eabc24/attachment.html From collinw at gmail.com Wed Mar 21 21:43:15 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 21 Mar 2007 15:43:15 -0500 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <9e804ac0703211340k733ec339j9a7cea58bf5b4be8@mail.gmail.com> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <43aa6ff70703202057u1d9bcf46yc4cf11c39c0b3a51@mail.gmail.com> <43aa6ff70703211222i40e00755o9cf5730b625b5309@mail.gmail.com> <9e804ac0703211340k733ec339j9a7cea58bf5b4be8@mail.gmail.com> Message-ID: <43aa6ff70703211343x49180790mbbceacb0525079f@mail.gmail.com> On 3/21/07, Thomas Wouters wrote: > On 3/21/07, Collin Winter wrote: > > On 3/20/07, Guido van Rossum wrote: > > [snip] > > > Or we could just have 2.6 warn about the presence (or use) of __cmp__ > > > -- users can write equivalent code using __lt__ etc. themselves and > > > probably do a better job. > > > > Fair enough. > > > > Is anyone collecting a list of these --py3k warnings? There seem to be > > a lot recent changes that 2.6's py3k-compat mode will be handling, but > > I haven't seen anything about someone tracking them/working on a PEP > > on the issue. Or did I just volunteer myself for that? : ) > > Neal and I have been updating PEP361, and I will be keeping an eye on it > when I refactor the p3yk branch into separate py3k-feature-branches (be it > bazaar or mercurial or whatever.) Each conceptual change in py3k will be a > separate branch, so keeping track of what needs to be warned will be easier. > (But the practical consequences of this will be clearer after I give it some > concrete form :) Good to know. Thanks, Thomas! Collin Winter From g.brandl at gmx.net Wed Mar 21 21:52:20 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 21 Mar 2007 21:52:20 +0100 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <20070321080859.FC83.JCARLSON@uci.edu> Message-ID: Terry Reedy schrieb: > "Georg Brandl" wrote in message > news:etrj1a$l9a$1 at sea.gmane.org... > | Yes, but dictionaries had an explicit ordering in dict_compare() which > was > | deleted. > > Is dict_compare something added in 2.5? It is neither a builtin or dict > method in 2.4. No, it is a C function in dictobject.c, in Python available as dict.__cmp__. > In any case, this point is that dict ordering is as arbitrary as ordering, > for instance, a dict and a string. Since Guido stopped the experiment of > totally ordering all objects when complex nums were added, consistency > suggests that all fake orderings be eliminated, leaving only the order of > numbers, characters, and sequences of comparable objects. It was not really *that* arbitrary. There was a defined algorithm, and it made some sense (at least for 1-item-dicts). Georg From guido at python.org Wed Mar 21 22:28:36 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Mar 2007 14:28:36 -0700 Subject: [Python-3000] refleaks and other errors In-Reply-To: References: Message-ID: Thanks, got it. Found the leak, thanks to gc.get_objects() and your hint about what was leaking in test_grammar. Were you tired when you posted that first msg? It was incredibly terse. :-) Committed revision 54502. --Guido On 3/20/07, Neal Norwitz wrote: > That was mostly the first line of my message, although it was a bit terse. :-) > > It's just a cmd line option to regrtest.py -R 4:3: > Everything else is the same. > > The way I run it for test_grammar: > > ./python -E -tt ./Lib/test/regrtest.py -R 4:3: test_grammar > > You can also pass "-R ::" instead of "-R 4:3:" however that takes > longer and generally doesn't help much. > > Note that some tests that normally pass will fail (not the refleaks, > actually fail) with this option due to the test not cleaning up after > itself. > > n > -- > > On 3/20/07, Guido van Rossum wrote: > > I'm pretty sure I introduced leaks with the new metaclass code; it was > > a bit of a rush job and there's plenty of new C code. I need a hint on > > how to reproduce this myself for a single test... > > > > On 3/20/07, Neal Norwitz wrote: > > > regrtest.py -R 4:3: # on 64-bit, but that might only affect the xpickle error > > > > > > test_grammar leaked [14, 14, 14] references > > > test_doctest leaked [84, 84, 84] references > > > test_ctypes leaked [13, 13, 13] references > > > test_descrtut leaked [417, 417, 417] references > > > test_generators leaked [227, 227, 227] references > > > test_metaclass leaked [284, 284, 284] references > > > test_modulefinder leaked [186, 186, 186] references > > > test_scope leaked [78, 78, 78] references > > > test_threading_local leaked [102, 102, 102] references > > > test_unpack leaked [55, 55, 55] references > > > test_weakref leaked [79, 79, 79] references > > > > > > > > > test test_cpickle failed -- errors occurred; run in verbose mode for details > > > test test_io failed -- errors occurred; run in verbose mode for details > > > test test_structmembers failed -- errors occurred; run in verbose mode > > > for details > > > > > > test test_fileio failed -- Traceback (most recent call last): > > > File "/home/neal/python/dev/py3k/Lib/test/test_fileio.py", line 37, > > > in testSeekTell > > > self.f.seek(0) > > > IOError: [Errno 22] Invalid argument > > > > > > test_socket_ssl skipped -- socket module has no ssl support > > > > > > test test_xpickle failed -- Traceback (most recent call last): > > > File "/home/neal/python/dev/py3k/Lib/test/pickletester.py", line > > > 503, in test_ints > > > self.assertEqual(expected, n2) > > > AssertionError: -9223372036854775807 != 1 > > > > > > test test_xreload failed -- Traceback (most recent call last): > > > > > > > > > test_grammar has these two cases leaking 7 refs each: > > > class foo:return 1 > > > class foo:yield 1 > > > _______________________________________________ > > > Python-3000 mailing list > > > Python-3000 at python.org > > > http://mail.python.org/mailman/listinfo/python-3000 > > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Mar 21 22:36:05 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Mar 2007 14:36:05 -0700 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <20070321080859.FC83.JCARLSON@uci.edu> Message-ID: On 3/21/07, Georg Brandl wrote: > Terry Reedy schrieb: > > "Georg Brandl" wrote in message > > news:etrj1a$l9a$1 at sea.gmane.org... > > | Yes, but dictionaries had an explicit ordering in dict_compare() which > > was > > | deleted. > > > > Is dict_compare something added in 2.5? It is neither a builtin or dict > > method in 2.4. > > No, it is a C function in dictobject.c, in Python available as dict.__cmp__. > > > In any case, this point is that dict ordering is as arbitrary as ordering, > > for instance, a dict and a string. Since Guido stopped the experiment of > > totally ordering all objects when complex nums were added, consistency > > suggests that all fake orderings be eliminated, leaving only the order of > > numbers, characters, and sequences of comparable objects. > > It was not really *that* arbitrary. There was a defined algorithm, and it > made some sense (at least for 1-item-dicts). It was well-defined because I had to do *something* that depended only on the keys and values and not on object identities etc. I never considered it *userful* and I don't know of anyone who used it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Wed Mar 21 22:48:23 2007 From: python at rcn.com (Raymond Hettinger) Date: Wed, 21 Mar 2007 17:48:23 -0400 (EDT) Subject: [Python-3000] Fwd: Re: Total ordering and __cmp__ Message-ID: <20070321174823.BCG89569@ms09.lnh.mail.rcn.net> [Guido van Rossum] >It was well-defined because I had to do *something* that depended only >on the keys and values and not on object identities etc. I never >considered it *userful* and I don't know of anyone who used it. Also, dict ordering would be confusing in Py3k where the views use the inequality operators to express subset/superset relationships. As a side benefit, the dict equality test will speed-up once the dict ordering code is eliminated. Raymond From greg.ewing at canterbury.ac.nz Wed Mar 21 23:16:32 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Mar 2007 10:16:32 +1200 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> Message-ID: <4601AEC0.3030701@canterbury.ac.nz> Adam Olsen wrote: > On 3/20/07, Steven Bethard wrote: > > > I've used a __key__() method quite successfully in my own code. Maybe > > we should provide a mixin like:: > > This seems to match what I've usually needed, but I'm not sure it's > worth putting in python proper. In addition to this, what about the idea of a Python-level __richcmp__ method? It would provide a single point of override for classes that wanted to use it, and it shouldn't upset any default behaviours since it's just exposing what's there at the C level anyway. -- Greg From daniel at stutzbachenterprises.com Wed Mar 21 23:36:50 2007 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 21 Mar 2007 17:36:50 -0500 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: <4601AEC0.3030701@canterbury.ac.nz> References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <4601AEC0.3030701@canterbury.ac.nz> Message-ID: On 3/21/07, Greg Ewing wrote: > In addition to this, what about the idea of a Python-level > __richcmp__ method? It would provide a single point of override > for classes that wanted to use it, and it shouldn't upset any > default behaviours since it's just exposing what's there > at the C level anyway. I apologize if this is a silly question, but what would be the difference between the proposed __richcmp__ and the old __cmp__ that's being removed? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises LLC From bjourne at gmail.com Wed Mar 21 23:45:09 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 21 Mar 2007 23:45:09 +0100 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws In-Reply-To: References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> Message-ID: <740c3aec0703211545i6fc56a9cob397cf6b0ca8bf32@mail.gmail.com> On 3/21/07, Steven Bethard wrote: > On 3/14/07, BJ?rn Lindqvist wrote: > > So I wonder what is the policy for fixing mistakes in the API design? > > In general, I think if you can warn about it in 2.6 or you can add to > 2to3 so that this gets automatically fixed, then it's okay to fix the > API. Is that the general consensus? I think I remember having read Raymond Hettinger complain about to many warnings in 2.6. > > Is a PEP really needed? For example, here are some bugs in the > > threading module: > > > > activeCount() # Redundant, use len(threading.enumerate()) > > Removing this should be alright as long as Python 2.6 warns about it > with the -Wpy3k flag. OK. So every API repair is a two step process? Fix the API in py3k and then add a warning to 2.6 and update the manual. > > currentThread() # PEP8 -> current_thread() > > class local # PEP8 -> class Local > > PEP 3108 (http://www.python.org/dev/peps/pep-3108/) and PEP 364 > (http://www.python.org/dev/peps/pep-0364/) address some of the issues > in renaming things, though I don't think they address renaming > anything but modules. Of course, for Python 2.6 it's pretty simple to > make one spelling of the class/function/method an alias of the other. > But I don't see an obvious way to do this except manually. PEP 364 > should probably address this. I think so too. Aliasing names is easy to do, but uglifies things. Plus, if you do it that way, can you still emit DeprecationWarnings? > > However, using a callback does not work so well because the function > > acting on the telnet options read still needs to reference the Telnet > > class to get hold of negotiation data using read_sb_data(). The > > problem is non-lethal but a small annoyance to advanced Telnet > > users. See SourceForge patches #1520081, #664020 and #1678077. > > > > The right design would have been to have a method (handle_option) in > > the class that handles all options and, by default, refuses > > them. Users of the Telnet class could then subclass Telnet and > > override the handle_option method to implement their application > > specific option handling. > > Seems like this could be done now (for Python 2.6), no? Just factor > out the code that decides whether or not to call self.option_callback > into a handle_option() method and call that. I'm not sure I see how > Python 3000 comes into it... The patches eliminate the set.option_callback variable and the Telnetlib.set_option_negotiation_callback() method. The problem has an extra twist because the old way is mutually exclusive with the new way. Or well, you could support them both but that would make the class extra ugly. -- mvh Bj?rn From collinw at gmail.com Wed Mar 21 23:47:17 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 21 Mar 2007 17:47:17 -0500 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: <20070321222626.83B4E1E400F@bag.python.org> References: <20070321222626.83B4E1E400F@bag.python.org> Message-ID: <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> On 3/21/07, brett.cannon wrote: > Author: brett.cannon > Date: Wed Mar 21 23:26:20 2007 > New Revision: 54510 > > Modified: > python/branches/p3yk/Lib/ConfigParser.py > python/branches/p3yk/Lib/test/test_file.py > Log: > When removing indexing/slicing on exceptions some places were changed > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > reason it needs to be the last option is the dichotomy of 'message' and 'args': > 'message' can be the empty string but args[0] can have a value if more than one > argument was passed. So e.args is going to stick around in 3.0? Collin Winter From guido at python.org Wed Mar 21 23:55:07 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Mar 2007 15:55:07 -0700 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> Message-ID: On 3/21/07, Collin Winter wrote: > On 3/21/07, brett.cannon wrote: > > When removing indexing/slicing on exceptions some places were changed > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > 'message' can be the empty string but args[0] can have a value if more than one > > argument was passed. > > So e.args is going to stick around in 3.0? I think so. e.message OTOH I think we can kill (perhaps deprecate it in 2.6; it was added in 2.5, this may be a record :-). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From bjourne at gmail.com Wed Mar 21 23:56:10 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 21 Mar 2007 23:56:10 +0100 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws In-Reply-To: References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> <740c3aec0703211154u5f151dcel817b4f2e253d9d99@mail.gmail.com> Message-ID: <740c3aec0703211556y2c484a82ja86316cc56f88145@mail.gmail.com> On 3/21/07, Terry Reedy wrote: > "BJ?rn Lindqvist" wrote in message > news:740c3aec0703211154u5f151dcel817b4f2e253d9d99 at mail.gmail.com... > | No comments at all. :( Did I send the mail to the wrong list? > > No, perhaps everyone did as I did, and assumed someone else would answer > ;-) I see.. Thanks for your response then. :) > | Either or, I still would like to know what the py3k rules are for > | repairing broken API:s. > > I don't remember there being much discussion of this yet. I believe there > is agreement to change names of modules to comform with PEP 8, but I > suspect this a easier to accomodate in the autofixer than names within > modules. It is easy to do it manually too, if it is allowed. But the problem becomes what to do for Python 2.6? > My personal position, at least for threading and telnet, would be change > away, since I have never used either and would not be affected. But I know > others are in the opposite position, so I would not give mine much weight. > > It also seems to me that module fixups might better wait until the kernal > and fixup code are more stable. I disagree. Most modules in the Standard Library needs a makeover which potentially is more disruptive than other changes. It is not so hard to adapt to print being a function instead of a statement. It is hard to adapt to 500 different backwards-incompatible API changes in the Standard Library. IMHO, codified guidelines for how to handle that would be most useful. I also think that that could bring many bugs and patches on sourceforge to closure because they require incompatible API changes (such as the telnetlib example). -- mvh Bj?rn From collinw at gmail.com Wed Mar 21 23:58:33 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 21 Mar 2007 17:58:33 -0500 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> Message-ID: <43aa6ff70703211558j1cd15892ja886dbdfa476a04f@mail.gmail.com> On 3/21/07, Guido van Rossum wrote: > On 3/21/07, Collin Winter wrote: > > On 3/21/07, brett.cannon wrote: > > > When removing indexing/slicing on exceptions some places were changed > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > 'message' can be the empty string but args[0] can have a value if more than one > > > argument was passed. > > > > So e.args is going to stick around in 3.0? > > I think so. e.message OTOH I think we can kill (perhaps deprecate it > in 2.6; it was added in 2.5, this may be a record :-). I'll update 2to3's fix_except accordingly. Is PEP 352 the only one that needs to be changed? Collin Winter From brett at python.org Thu Mar 22 00:01:10 2007 From: brett at python.org (Brett Cannon) Date: Wed, 21 Mar 2007 16:01:10 -0700 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> Message-ID: On 3/21/07, Guido van Rossum wrote: > On 3/21/07, Collin Winter wrote: > > On 3/21/07, brett.cannon wrote: > > > When removing indexing/slicing on exceptions some places were changed > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > 'message' can be the empty string but args[0] can have a value if more than one > > > argument was passed. > > > > So e.args is going to stick around in 3.0? > > I think so. I just posted on my blog about this. I am curious to see what the general public think, but I suspect it will stay as well. I was cranky at the sprint because of my 'args' removal. Don't need to make more people cranky. > e.message OTOH I think we can kill (perhaps deprecate it > in 2.6; it was added in 2.5, this may be a record :-). Should we just consider this a PDFL pronouncement, or should we more formally run this past python-dev? -Brett From brett at python.org Thu Mar 22 00:03:19 2007 From: brett at python.org (Brett Cannon) Date: Wed, 21 Mar 2007 16:03:19 -0700 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: <43aa6ff70703211558j1cd15892ja886dbdfa476a04f@mail.gmail.com> References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> <43aa6ff70703211558j1cd15892ja886dbdfa476a04f@mail.gmail.com> Message-ID: On 3/21/07, Collin Winter wrote: > On 3/21/07, Guido van Rossum wrote: > > On 3/21/07, Collin Winter wrote: > > > On 3/21/07, brett.cannon wrote: > > > > When removing indexing/slicing on exceptions some places were changed > > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > > 'message' can be the empty string but args[0] can have a value if more than one > > > > argument was passed. > > > > > > So e.args is going to stick around in 3.0? > > > > I think so. e.message OTOH I think we can kill (perhaps deprecate it > > in 2.6; it was added in 2.5, this may be a record :-). > > I'll update 2to3's fix_except accordingly. So I guess you are just going to notice when exceptions are caught and if exc.message is found change it to exc.args[0]? Could you use this for the removal of indexing/slicing so that exc[0] becomes exc.args[0]? > Is PEP 352 the only one > that needs to be changed? Yep, that would be the only place. -Brett From collinw at gmail.com Thu Mar 22 00:10:40 2007 From: collinw at gmail.com (Collin Winter) Date: Wed, 21 Mar 2007 18:10:40 -0500 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> <43aa6ff70703211558j1cd15892ja886dbdfa476a04f@mail.gmail.com> Message-ID: <43aa6ff70703211610p593caa0bj9f2315bea7b7df4@mail.gmail.com> On 3/21/07, Brett Cannon wrote: > On 3/21/07, Collin Winter wrote: > > On 3/21/07, Guido van Rossum wrote: > > > On 3/21/07, Collin Winter wrote: > > > > On 3/21/07, brett.cannon wrote: > > > > > When removing indexing/slicing on exceptions some places were changed > > > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > > > 'message' can be the empty string but args[0] can have a value if more than one > > > > > argument was passed. > > > > > > > > So e.args is going to stick around in 3.0? > > > > > > I think so. e.message OTOH I think we can kill (perhaps deprecate it > > > in 2.6; it was added in 2.5, this may be a record :-). > > > > I'll update 2to3's fix_except accordingly. > > So I guess you are just going to notice when exceptions are caught and > if exc.message is found change it to exc.args[0]? Could you use this > for the removal of indexing/slicing so that exc[0] becomes > exc.args[0]? No, I'm basically just backing out a transformation that used .message instead of .args. It would be possible to remove slicing/indexing like you suggest, but only when exc is mentioned explicitly; that is, "x = exc; x[0];" would defeat it. Collin Winter From guido at python.org Thu Mar 22 00:13:52 2007 From: guido at python.org (Guido van Rossum) Date: Wed, 21 Mar 2007 16:13:52 -0700 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> Message-ID: PDFL? Petty Dictator For Life? Poodle Dictator For Life? Perhaps Dictator For Life? :-) I suggest that you update the PEP and then repost the pep on python-dev with a summary of the changes (to the PEP). I expect it to go smoothly. --Guido On 3/21/07, Brett Cannon wrote: > On 3/21/07, Guido van Rossum wrote: > > On 3/21/07, Collin Winter wrote: > > > On 3/21/07, brett.cannon wrote: > > > > When removing indexing/slicing on exceptions some places were changed > > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > > 'message' can be the empty string but args[0] can have a value if more than one > > > > argument was passed. > > > > > > So e.args is going to stick around in 3.0? > > > > I think so. > > I just posted on my blog about this. I am curious to see what the > general public think, but I suspect it will stay as well. I was > cranky at the sprint because of my 'args' removal. Don't need to make > more people cranky. > > > e.message OTOH I think we can kill (perhaps deprecate it > > in 2.6; it was added in 2.5, this may be a record :-). > > Should we just consider this a PDFL pronouncement, or should we more > formally run this past python-dev? > > -Brett > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Thu Mar 22 00:27:59 2007 From: brett at python.org (Brett Cannon) Date: Wed, 21 Mar 2007 16:27:59 -0700 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: <43aa6ff70703211610p593caa0bj9f2315bea7b7df4@mail.gmail.com> References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> <43aa6ff70703211558j1cd15892ja886dbdfa476a04f@mail.gmail.com> <43aa6ff70703211610p593caa0bj9f2315bea7b7df4@mail.gmail.com> Message-ID: On 3/21/07, Collin Winter wrote: > On 3/21/07, Brett Cannon wrote: > > On 3/21/07, Collin Winter wrote: > > > On 3/21/07, Guido van Rossum wrote: > > > > On 3/21/07, Collin Winter wrote: > > > > > On 3/21/07, brett.cannon wrote: > > > > > > When removing indexing/slicing on exceptions some places were changed > > > > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > > > > 'message' can be the empty string but args[0] can have a value if more than one > > > > > > argument was passed. > > > > > > > > > > So e.args is going to stick around in 3.0? > > > > > > > > I think so. e.message OTOH I think we can kill (perhaps deprecate it > > > > in 2.6; it was added in 2.5, this may be a record :-). > > > > > > I'll update 2to3's fix_except accordingly. > > > > So I guess you are just going to notice when exceptions are caught and > > if exc.message is found change it to exc.args[0]? Could you use this > > for the removal of indexing/slicing so that exc[0] becomes > > exc.args[0]? > > No, I'm basically just backing out a transformation that used .message > instead of .args. It would be possible to remove slicing/indexing like > you suggest, but only when exc is mentioned explicitly; that is, "x = > exc; x[0];" would defeat it. > In that case I say just go with the Py3K warning thing. -Brett From brett at python.org Thu Mar 22 00:28:46 2007 From: brett at python.org (Brett Cannon) Date: Wed, 21 Mar 2007 16:28:46 -0700 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> Message-ID: On 3/21/07, Guido van Rossum wrote: > PDFL? Petty Dictator For Life? Poodle Dictator For Life? Perhaps > Dictator For Life? :-) > =) Hey, I'm sick so I get to have a typo or four. > I suggest that you update the PEP and then repost the pep on > python-dev with a summary of the changes (to the PEP). I expect it to > go smoothly. > Will do. -Brett > --Guido > > On 3/21/07, Brett Cannon wrote: > > On 3/21/07, Guido van Rossum wrote: > > > On 3/21/07, Collin Winter wrote: > > > > On 3/21/07, brett.cannon wrote: > > > > > When removing indexing/slicing on exceptions some places were changed > > > > > inappropriately from ``e[0]`` to ``e.message`` instead of ``e.args[0]``. The > > > > > reason it needs to be the last option is the dichotomy of 'message' and 'args': > > > > > 'message' can be the empty string but args[0] can have a value if more than one > > > > > argument was passed. > > > > > > > > So e.args is going to stick around in 3.0? > > > > > > I think so. > > > > I just posted on my blog about this. I am curious to see what the > > general public think, but I suspect it will stay as well. I was > > cranky at the sprint because of my 'args' removal. Don't need to make > > more people cranky. > > > > > e.message OTOH I think we can kill (perhaps deprecate it > > > in 2.6; it was added in 2.5, this may be a record :-). > > > > Should we just consider this a PDFL pronouncement, or should we more > > formally run this past python-dev? > > > > -Brett > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From tjreedy at udel.edu Thu Mar 22 02:43:07 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 21 Mar 2007 21:43:07 -0400 Subject: [Python-3000] Total ordering and __cmp__ References: <20070321080859.FC83.JCARLSON@uci.edu> Message-ID: Terry Reedy schrieb: | > In any case, this point is that dict ordering is as arbitrary as ordering, | > for instance, a dict and a string. Since Guido stopped the experiment of | > totally ordering all objects when complex nums were added, consistency | > suggests that all fake orderings be eliminated, leaving only the order of | > numbers, characters, and sequences of comparable objects. I should have added that not having an arbitrary total order allows for useful partial orders. The comparisons on sets appear to be such now (based on inclusion) and RH mentioned doing the same for dicts in 3.0. tjr From greg.ewing at canterbury.ac.nz Thu Mar 22 04:13:36 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Mar 2007 15:13:36 +1200 Subject: [Python-3000] Total ordering and __cmp__ In-Reply-To: References: <43aa6ff70703201810p9d731b8v3497a48539d5dc84@mail.gmail.com> <4601AEC0.3030701@canterbury.ac.nz> Message-ID: <4601F460.3080507@canterbury.ac.nz> Daniel Stutzbach wrote: > I apologize if this is a silly question, but what would be the > difference between the proposed __richcmp__ and the old __cmp__ that's > being removed? __richcmp__ would have the same signature as the C-level tp_richcmp slot, i.e. taking a code indicating which comparison to perform, and returning an arbitary value instead of -1, 0, 1. With this, you would be able to do anything that could be done with __lt__, __eq__, etc., but only have to override a single method. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From greg.ewing at canterbury.ac.nz Thu Mar 22 04:25:25 2007 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 22 Mar 2007 15:25:25 +1200 Subject: [Python-3000] [Python-3000-checkins] r54510 - in python/branches/p3yk/Lib: ConfigParser.py test/test_file.py In-Reply-To: References: <20070321222626.83B4E1E400F@bag.python.org> <43aa6ff70703211547w46dd59e0ne85bf0ad6ed022b9@mail.gmail.com> Message-ID: <4601F725.7030407@canterbury.ac.nz> Brett Cannon wrote: > Should we just consider this a PDFL pronouncement, ^^^^ Interesting acronym... Portable Documentator For Life, maybe? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From steven.bethard at gmail.com Thu Mar 22 05:55:28 2007 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 21 Mar 2007 22:55:28 -0600 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws In-Reply-To: <740c3aec0703211545i6fc56a9cob397cf6b0ca8bf32@mail.gmail.com> References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> <740c3aec0703211545i6fc56a9cob397cf6b0ca8bf32@mail.gmail.com> Message-ID: On 3/21/07, BJ?rn Lindqvist wrote: > On 3/21/07, Steven Bethard wrote: > > On 3/14/07, BJ?rn Lindqvist wrote: > > > So I wonder what is the policy for fixing mistakes in the API design? > > > > In general, I think if you can warn about it in 2.6 or you can add to > > 2to3 so that this gets automatically fixed, then it's okay to fix the > > API. > > Is that the general consensus? I think I remember having read Raymond > Hettinger complain about to many warnings in 2.6. Well, I can't speak for Raymond, but I think much of that is supposed to be addressed by introducing the -Wpy3k flag which will be off by default, and which will set a C-level bit flag so that any warnings that need to go in speed-critical locations can have as little impact as possible. Or at least that was the plan as I understood it. > OK. So every API repair is a two step process? Fix the API in py3k and > then add a warning to 2.6 and update the manual. Yep. You probably don't need a PEP for every one, but maybe we should collect all (or at least as many as possible of) the minor changes in a PEP somewhere just for reference... STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From frank-python at benkstein.net Thu Mar 22 14:36:08 2007 From: frank-python at benkstein.net (Frank Benkstein) Date: Thu, 22 Mar 2007 14:36:08 +0100 Subject: [Python-3000] New builtin exceptions classes? Message-ID: <20070322143608.6a3305e6@localhost> Hi, I'm new to this list and interested in Python 3000 development. The following is not really a proposal that is ready to be implemented. I'm merely interested if this is a problem others want to be solved, too, and if it would be possible to get such a thing into Python 3000. IMHO Exceptions should carry more information about why and where the exception happened. A good example of an exception class carrying that kind of information is IOError with its 'filename' and 'errno' attributes (although these could be documented better). It would be nice if all builtin Exceptions had those kind of attributes. As an example KeyError could have a 'key' and a 'mapping' attribute, IndexError could have a 'index' and an 'sequence' attribute and so on. It is arguable if the second ones ('mapping' and 'sequence') are really needed, but I think it would be really nice to have those as it would make debugging a lot easier. Additionally they shouldn't really hurt because as per PEP 3110 exception instances are deleted when the exception handler is finished. A further addition to this proposal would be to discourage raising exceptions as wrappers around diagnostical messages. Currently it is very hard to programmatically find out what happened here: >>> int([]) Traceback (most recent call last): File "", line 1, in TypeError: int() argument must be a string or a number, not 'list' It would be nice if the recommended way would be something like the following, with only __str__ formatting the message: raise TypeError( '%(target)s() argument must be one of %(allowed)s, not %(given)s', target=int, allowed=(str, int, float, long), given=type(...)) Maybe the the most common messages could be predefined in the Exception class instead of constructed ad-hoc to make processing easier but having them constructed as shown would already help a lot. Best regards Frank Benkstein. -- GPG (Mail): 7093 7A43 CC40 463A 5564 599B 88F6 D625 BE63 866F GPG (XMPP): 2243 DBBA F234 7C5A 6D71 3983 9F28 4D03 7110 6D51 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20070322/dd505aad/attachment.pgp From erno at iki.fi Thu Mar 22 18:47:50 2007 From: erno at iki.fi (Erno Kuusela) Date: 22 Mar 2007 19:47:50 +0200 Subject: [Python-3000] IPv6 address tuple Message-ID: Hello, Would it be out of the question to change the IPv6 address 4-tuple to a 2-tuple (as used with IPv4) in Python 2.0? This is a source of many compatibility problems, and the additional flowinfo and scope elements from the IPv6 tuple are practically never used by normal applications. -- erno From jackdied at jackdied.com Thu Mar 22 19:00:28 2007 From: jackdied at jackdied.com (Jack Diederich) Date: Thu, 22 Mar 2007 14:00:28 -0400 Subject: [Python-3000] Question about PEP 3001 and fixing API flaws In-Reply-To: References: <740c3aec0703141420q1182927fj333e0095bf6d8691@mail.gmail.com> Message-ID: <20070322180028.GI13084@performancedrivers.com> On Wed, Mar 21, 2007 at 01:38:32PM -0600, Steven Bethard wrote: > On 3/14/07, BJ?rn Lindqvist wrote: > > For another example, take the Telnet class in the telnetlib module. It > > has a method set_option_negotiation_callback() which takes a function > > that will be called for each telnet option read. The default behaviour > > for the Telnet class is to refuse all negotiation requests, but using > > a negotiation callback you can override that. > > > > However, using a callback does not work so well because the function > > acting on the telnet options read still needs to reference the Telnet > > class to get hold of negotiation data using read_sb_data(). The > > problem is non-lethal but a small annoyance to advanced Telnet > > users. See SourceForge patches #1520081, #664020 and #1678077. > > > > The right design would have been to have a method (handle_option) in > > the class that handles all options and, by default, refuses > > them. Users of the Telnet class could then subclass Telnet and > > override the handle_option method to implement their application > > specific option handling. > > Seems like this could be done now (for Python 2.6), no? Just factor > out the code that decides whether or not to call self.option_callback > into a handle_option() method and call that. I'm not sure I see how > Python 3000 comes into it... > [I meant to reply too but hadn't gotten back to it] telnetlib isn't usable for anything that isn't extremely simple. It needs a rewrite or at least some major surgery. Lots of people (including me) have written subclasses that extend it but I don't know how compatible they are (see google code search for a half dozen examples). There are a number of outstanding patches in SourceForge but there are no unit tests so it is hard to know if they break stuff. At least a couple of the patches address negotiation. Off the top of my head its two biggest deficiencies are negotiation and being able to manipulate the raw IO. Negotiation is easy to improve for most cases but harder for two-way communication like NAWS (terminal resizing) messages. Manipulating the raw IO is required for telnet extensions like MCCP which expects the raw stream to be gzip'd starting the byte immediately after the SB clause. If someone wrote a test module the SF patches could be applied and tested easily and even major changes to the implementation would be possible. I've been meaning to do that for a couple years now but it never gets near the top of my list. Consider that an invitation! -Jack From brett at python.org Thu Mar 22 20:30:45 2007 From: brett at python.org (Brett Cannon) Date: Thu, 22 Mar 2007 12:30:45 -0700 Subject: [Python-3000] New builtin exceptions classes? In-Reply-To: <20070322143608.6a3305e6@localhost> References: <20070322143608.6a3305e6@localhost> Message-ID: On 3/22/07, Frank Benkstein wrote: > Hi, > > I'm new to this list and interested in Python 3000 development. > > The following is not really a proposal that is ready to be implemented. > I'm merely interested if this is a problem others want to be solved, > too, and if it would be possible to get such a thing into Python 3000. > > IMHO Exceptions should carry more information about why and where the > exception happened. A good example of an exception class carrying that > kind of information is IOError with its 'filename' and 'errno' > attributes (although these could be documented better). It would be > nice if all builtin Exceptions had those kind of attributes. As an > example KeyError could have a 'key' and a 'mapping' attribute, > IndexError could have a 'index' and an 'sequence' attribute and so on. > It is arguable if the second ones ('mapping' and 'sequence') are really > needed, but I think it would be really nice to have those as it would > make debugging a lot easier. Additionally they shouldn't really hurt > because as per PEP 3110 exception instances are deleted when the > exception handler is finished. > > A further addition to this proposal would be to discourage raising > exceptions as wrappers around diagnostical messages. Currently it is > very hard to programmatically find out what happened here: > > >>> int([]) > Traceback (most recent call last): > File "", line 1, in > TypeError: int() argument must be a string or a number, not 'list' > > It would be nice if the recommended way would be something like the > following, with only __str__ formatting the message: > > raise TypeError( > '%(target)s() argument must be one of %(allowed)s, not %(given)s', > target=int, > allowed=(str, int, float, long), > given=type(...)) > > Maybe the the most common messages could be predefined in the Exception > class instead of constructed ad-hoc to make processing easier but > having them constructed as shown would already help a lot. > If you would like to try to come up with a more structured proposal please bring it up on python-ideas. -Brett From brett at python.org Thu Mar 22 20:32:03 2007 From: brett at python.org (Brett Cannon) Date: Thu, 22 Mar 2007 12:32:03 -0700 Subject: [Python-3000] IPv6 address tuple In-Reply-To: References: Message-ID: On 22 Mar 2007 19:47:50 +0200, Erno Kuusela wrote: > Hello, > > Would it be out of the question to change the IPv6 address 4-tuple to > a 2-tuple (as used with IPv4) in Python 2.0? This is a source of many > compatibility problems, and the additional flowinfo and scope elements > from the IPv6 tuple are practically never used by normal applications. > This is the wrong list to ask about Python 2.x issues; that's python-dev. And yes, it would most likely be almost impossible. -Brett From martin at v.loewis.de Thu Mar 22 21:38:45 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 Mar 2007 21:38:45 +0100 Subject: [Python-3000] New builtin exceptions classes? In-Reply-To: <20070322143608.6a3305e6@localhost> References: <20070322143608.6a3305e6@localhost> Message-ID: <4602E955.3090906@v.loewis.de> Frank Benkstein schrieb: > IMHO Exceptions should carry more information about why and where the > exception happened. A good example of an exception class carrying that > kind of information is IOError with its 'filename' and 'errno' > attributes (although these could be documented better). It would be > nice if all builtin Exceptions had those kind of attributes. As an > example KeyError could have a 'key' and a 'mapping' attribute, > IndexError could have a 'index' and an 'sequence' attribute and so on. This is an often-requested feature, e.g. in http://sourceforge.net/tracker/index.php?func=detail&aid=1182143&group_id=5470&atid=355470 The primary reason that it isn't there yet is that nobody bothered to come up with a precise specification what exception should get what attributes. If you would like to contribute such a specification, go ahead. An implementation would be appreciated, as well, although that implementation certainly cannot set these attributes in all places where the exception is raised. This doesn't need to wait for Py3k, either: in many cases, this can be a fully compatible change, so it can be added to 2.6 already. Regards, Martin From martin at v.loewis.de Thu Mar 22 21:40:56 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 22 Mar 2007 21:40:56 +0100 Subject: [Python-3000] IPv6 address tuple In-Reply-To: References: Message-ID: <4602E9D8.6080803@v.loewis.de> > Would it be out of the question to change the IPv6 address 4-tuple to > a 2-tuple (as used with IPv4) in Python 2.0? This is a source of many > compatibility problems, and the additional flowinfo and scope elements > from the IPv6 tuple are practically never used by normal applications. As a starting point of any discussion, you should get into contact with the author of the IPv6 support in Python, Jun-ichiro "itojun" Hagino. Regards, Martin From aahz at pythoncraft.com Fri Mar 23 15:30:29 2007 From: aahz at pythoncraft.com (Aahz) Date: Fri, 23 Mar 2007 07:30:29 -0700 Subject: [Python-3000] Single-line blocks? Message-ID: <20070323143029.GB8509@panix.com> IIRC, there was some discussion about removing single-line blocks like this one in 3.0: if foo: bar() However, I can't find any discussion about making it happen, and it's not in PEP 3099. Guido, does this need a PEP? Do you want it to happen? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Typing is cheap. Thinking is expensive." --Roy Smith From jimjjewett at gmail.com Fri Mar 23 16:27:53 2007 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 23 Mar 2007 11:27:53 -0400 Subject: [Python-3000] telnetlib [was: Question about PEP 3001 and fixing API flaws] Message-ID: On 3/21/07, BJ?rn Lindqvist wrote: > I think so too. Aliasing names is easy to do, but uglifies things. > Plus, if you do it that way, can you still emit DeprecationWarnings? Sure. def oldname(arg): "Deprecated version of newname" warn("Use newname instead", DeprecationWarning) return newname(arg) > The patches eliminate the set.option_callback variable and the > Telnetlib.set_option_negotiation_callback() method. The problem has an > extra twist because the old way is mutually exclusive with the new > way. Or well, you could support them both but that would make the > class extra ugly. Not really. The default handle_option could look for (and call) a callback. If you don't want to mess with refixing the sbdataq, you could just change the call sites. Right now, they already all say if self.option_callback: self.option_callback(...) else: # fallback Make that either # Support the deprecated callback in 2.6 if self.option_callback: self.option_callback(...) else: self.handle_option(...) or # Support the deprecated callback in 2.6 if self.option_callback: self.option_callback(...) elif not self.handle_option(...): # fallback -jJ From tjreedy at udel.edu Fri Mar 23 16:35:55 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 23 Mar 2007 11:35:55 -0400 Subject: [Python-3000] Single-line blocks? References: <20070323143029.GB8509@panix.com> Message-ID: "Aahz" wrote in message news:20070323143029.GB8509 at panix.com... | IIRC, there was some discussion about removing single-line blocks like | this one in 3.0: | | if foo: bar() | | However, I can't find any discussion about making it happen, and it's | not in PEP 3099. Guido, does this need a PEP? Do you want it to happen? To me, such constructions are sometimes more readable than the alternative. So, unless there was a strong, not-yet-presented technical argument for the change, I would strongly oppose mere stylistic imperialism. tjr From guido at python.org Fri Mar 23 18:01:27 2007 From: guido at python.org (Guido van Rossum) Date: Fri, 23 Mar 2007 10:01:27 -0700 Subject: [Python-3000] Single-line blocks? In-Reply-To: References: <20070323143029.GB8509@panix.com> Message-ID: I know this was in my early lists of "clean up the language" plans for Py3k, but I think it's not worth the hassle. On 3/23/07, Terry Reedy wrote: > > "Aahz" wrote in message > news:20070323143029.GB8509 at panix.com... > | IIRC, there was some discussion about removing single-line blocks like > | this one in 3.0: > | > | if foo: bar() > | > | However, I can't find any discussion about making it happen, and it's > | not in PEP 3099. Guido, does this need a PEP? Do you want it to happen? > > To me, such constructions are sometimes more readable than the alternative. > So, unless there was a strong, not-yet-presented technical argument for the > change, I would strongly oppose mere stylistic imperialism. > > tjr > > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Fri Mar 23 18:11:50 2007 From: aahz at pythoncraft.com (Aahz) Date: Fri, 23 Mar 2007 10:11:50 -0700 Subject: [Python-3000] Single-line blocks? In-Reply-To: References: <20070323143029.GB8509@panix.com> Message-ID: <20070323171150.GA22777@panix.com> On Fri, Mar 23, 2007, Guido van Rossum wrote: > > I know this was in my early lists of "clean up the language" plans for > Py3k, but I think it's not worth the hassle. Should it be added to PEP 3099? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Typing is cheap. Thinking is expensive." --Roy Smith From g.brandl at gmx.net Fri Mar 23 19:04:04 2007 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 23 Mar 2007 19:04:04 +0100 Subject: [Python-3000] Single-line blocks? In-Reply-To: <20070323171150.GA22777@panix.com> References: <20070323143029.GB8509@panix.com> <20070323171150.GA22777@panix.com> Message-ID: Aahz schrieb: > On Fri, Mar 23, 2007, Guido van Rossum wrote: >> >> I know this was in my early lists of "clean up the language" plans for >> Py3k, but I think it's not worth the hassle. > > Should it be added to PEP 3099? I've never seen it proposed by anyone recently. So, I think there's no need to do that. Georg From baptiste13 at altern.org Fri Mar 23 23:34:38 2007 From: baptiste13 at altern.org (Baptiste Carvello) Date: Fri, 23 Mar 2007 23:34:38 +0100 Subject: [Python-3000] Single-line blocks? In-Reply-To: <20070323143029.GB8509@panix.com> References: <20070323143029.GB8509@panix.com> Message-ID: Aahz a ?crit : > IIRC, there was some discussion about removing single-line blocks like > this one in 3.0: > > if foo: bar() > > However, I can't find any discussion about making it happen, and it's > not in PEP 3099. Guido, does this need a PEP? Do you want it to happen? please no. Single line blocks may be bad style in programs, but they are very useful in interactive use. Indeed, with readline, re-executing a single line block is as easy as going up in the history. By contrast, multi-line block are a pain. Baptiste From erno at iki.fi Fri Mar 23 07:48:26 2007 From: erno at iki.fi (Erno Kuusela) Date: Fri, 23 Mar 2007 08:48:26 +0200 Subject: [Python-3000] IPv6 address tuple In-Reply-To: References: Message-ID: <20070323064825.GM14803@erno.iki.fi> hello, On Thu, 22 Mar 2007, Brett Cannon wrote: | On 22 Mar 2007 19:47:50 +0200, Erno Kuusela wrote: | >Hello, | > | >Would it be out of the question to change the IPv6 address 4-tuple to | >a 2-tuple (as used with IPv4) in Python 2.0? This is a source of many | >compatibility problems, and the additional flowinfo and scope elements | >from the IPv6 tuple are practically never used by normal applications. | > | | This is the wrong list to ask about Python 2.x issues; that's | python-dev. And yes, it would most likely be almost impossible. That was a typo, sorry - I meant Python 3.0. -- erno From erno at iki.fi Fri Mar 23 10:57:11 2007 From: erno at iki.fi (Erno Kuusela) Date: Fri, 23 Mar 2007 11:57:11 +0200 Subject: [Python-3000] IPv6 address tuple In-Reply-To: <4602E9D8.6080803@v.loewis.de> References: <4602E9D8.6080803@v.loewis.de> Message-ID: <20070323095711.GO14803@erno.iki.fi> hello, On Thu, 22 Mar 2007, "Martin v. L?wis" wrote: | >Would it be out of the question to change the IPv6 address 4-tuple to | >a 2-tuple (as used with IPv4) in Python 2.0? This is a source of many | >compatibility problems, and the additional flowinfo and scope elements | >from the IPv6 tuple are practically never used by normal applications. | | As a starting point of any discussion, you should get into contact with | the author of the IPv6 support in Python, Jun-ichiro "itojun" Hagino. Attached is a reply I got from itojun when asking about this. For those that don't know about IPv6 scoped addresses: there are 1. global addresses, which are the ones you normally use for everything 2. link-local addresses, which work on just one L2 layer, they are used for things like neighbour advertisements which replaces ARP, and DHCP 3. site-local addresses, which were deprecated some time ago and were an abandoned attempt to replace IPv4 RFC1918 addressing (the current way is randomly generated prefixes used with global scope, RFC4193) I first suggested just chopping off the flowid and scopeid parts of the tuple. Itojun's reply seems to indicate we could potentially get away with merging the scopeid to the IP address part in a standard fashion, and discarding the flowid part. In the below mail to itojun I suggested also an alternative way of accepting both the 2-tuple and 4-tuple kinds of objects for IPv6. The socket api would return 2-tuples for scope=global flowid=0 addresses, and 4-tuples for others. Another option that came to my mind is changing the address object from tuple to an object with named fields. This way IPv4 and IPv6 address compatibility could be taken care of with duck typing. This would also require changing the way IPv4 addresses work, but now feels to me like the most clean solution. -- erno -------------- next part -------------- An embedded message was scrubbed... From: itojun at itojun.org (Jun-ichiro itojun Hagino) Subject: Re: (fwd) Re: [Python-3000] IPv6 address tuple Date: Fri, 23 Mar 2007 18:36:28 +0900 (JST) Size: 2678 Url: http://mail.python.org/pipermail/python-3000/attachments/20070323/e7632372/attachment-0001.mht From erno at iki.fi Sat Mar 24 08:19:25 2007 From: erno at iki.fi (Erno Kuusela) Date: Sat, 24 Mar 2007 09:19:25 +0200 Subject: [Python-3000] IPv6 address tuple In-Reply-To: <20070323220556.8C8A21C047@coconut.itojun.org> References: <20070323093628.BC6A61C04E@coconut.itojun.org> <20070323220556.8C8A21C047@coconut.itojun.org> Message-ID: <20070324071925.GA24629@erno.iki.fi> hello, On Sat, 24 Mar 2007, Jun-ichiro itojun Hagino wrote: | > i understand your concern. current 4-tuple is direct mapping from | > struct sockaddr_in6 (which has those two additional members). | | note however, you must rewrite DNS resolving from gethostby* to | get*info anyways, so i'm not too sure if the number of tuple members in | sockaddr matters that much. I agree that when using those functions and treating the address tuple as opaque, things work out OK. The problem is that people do unpack the tuple a lot of the time as the first thing when they get the address. I suppose the elements are sometimes later used for things like logging and debug output. Also people are used to conceptually destructuring the address as "host, port" in their heads so it's logical to unpack it. -- erno From martin at v.loewis.de Sat Mar 24 11:54:08 2007 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 24 Mar 2007 11:54:08 +0100 Subject: [Python-3000] IPv6 address tuple In-Reply-To: <20070323095711.GO14803@erno.iki.fi> References: <4602E9D8.6080803@v.loewis.de> <20070323095711.GO14803@erno.iki.fi> Message-ID: <46050350.600@v.loewis.de> > I first suggested just chopping off the flowid and scopeid parts of the > tuple. Itojun's reply seems to indicate we could potentially get away > with merging the scopeid to the IP address part in a standard fashion, > and discarding the flowid part. Thanks for this research? As a next step, it would be interesting to see how it is done elsewhere (besides C and Python). As you may see, I would be really uncomfortable changing this in a way that locks out interesting applications, or deviates too far from the IPv6 community. So compatibility with the rest of the world is of utmost importance. If you could collect options, established tradition, pros and cons in a document, that would be good. Regards, Martin From martin at v.loewis.de Sat Mar 24 11:57:06 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 24 Mar 2007 11:57:06 +0100 Subject: [Python-3000] IPv6 address tuple In-Reply-To: <20070324071925.GA24629@erno.iki.fi> References: <20070323093628.BC6A61C04E@coconut.itojun.org> <20070323220556.8C8A21C047@coconut.itojun.org> <20070324071925.GA24629@erno.iki.fi> Message-ID: <46050402.4040205@v.loewis.de> > The problem is that people do unpack the tuple a lot of the time as the > first thing when they get the address. I suppose the elements are > sometimes later used for things like logging and debug output. Also > people are used to conceptually destructuring the address as "host, > port" in their heads so it's logical to unpack it. If that is the issue, a object that unpacks as a two-tuple, but has additional named fields (i.e. a StructSequence) may be an option. Regards, Martin From aahz at pythoncraft.com Sun Mar 25 17:26:39 2007 From: aahz at pythoncraft.com (Aahz) Date: Sun, 25 Mar 2007 08:26:39 -0700 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> Message-ID: <20070325152639.GA172@panix.com> I've looked at the most recent version at http://python.org/dev/peps/pep-3116/ and I see nothing in there about the interaction between a BufferedIO object and its underlying RawIO object. That is, what happens if you do this: f = open('foo', buffering=200) f.read(150) f.raw.read(200) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Typing is cheap. Thinking is expensive." --Roy Smith From guido at python.org Sun Mar 25 17:46:57 2007 From: guido at python.org (Guido van Rossum) Date: Sun, 25 Mar 2007 08:46:57 -0700 Subject: [Python-3000] Draft PEP for New IO system In-Reply-To: <20070325152639.GA172@panix.com> References: <5487f95e0702261335k64a8a278sdf628dd7baec4773@mail.gmail.com> <20070325152639.GA172@panix.com> Message-ID: You're not supposed to do that. I guess the PEP is unclear about that, but the effect would (as you understand) depend intricately on the internal state of the BufferedIO object, and while on the one hand I want the BufferedIO object to be more transparent than a C stdio object, on the other hand I don't want to force it into a particular implementation (I want freedom to evolve it). The PEP ought to be explicit about this. The relationship between the two is not unlike that between a C stdio object and its fileno() -- there are certain times that the relationship is well-defined (e.g. after a fsync()) and others that it is not. --Guido On 3/25/07, Aahz wrote: > I've looked at the most recent version at > > http://python.org/dev/peps/pep-3116/ > > and I see nothing in there about the interaction between a BufferedIO > object and its underlying RawIO object. That is, what happens if you do > this: > > f = open('foo', buffering=200) > f.read(150) > f.raw.read(200) > -- > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "Typing is cheap. Thinking is expensive." --Roy Smith > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From frank-python at benkstein.net Tue Mar 27 17:09:47 2007 From: frank-python at benkstein.net (Frank Benkstein) Date: Tue, 27 Mar 2007 17:09:47 +0200 Subject: [Python-3000] Ordered mapping type for Python 3000? Message-ID: <20070327170947.6afbbf7c@localhost> Hi, One thing that has come up quite often IIRC is the request for an ordered mapping type. It would like to propose to add such a type to Python 3000 and make it immutable, to prevent all kind of problems that would occur otherwise. Other than that it should behave just the same as dictionaries for all non-modifying operations. Furthermore it would be nice to have this new type as the default type for **-style keyword parameters. First it would make creating the instances of this new type a lot easier and second it would remove the asymmetry in having a immutable *args and a mutable **kwds. As far as backwards compatibilty is concerned this should only affect functions that rely on a mutable kwds and this could be easily fixed by explicitely creating a dict from the keywords parameter at the begin of the function. Comments? Should I try to wrap this up as a PEP? Best regards Frank Benkstein. -- GPG (Mail): 7093 7A43 CC40 463A 5564 599B 88F6 D625 BE63 866F GPG (XMPP): 2243 DBBA F234 7C5A 6D71 3983 9F28 4D03 7110 6D51 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20070327/2086fba9/attachment.pgp From aahz at pythoncraft.com Tue Mar 27 19:15:53 2007 From: aahz at pythoncraft.com (Aahz) Date: Tue, 27 Mar 2007 10:15:53 -0700 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <20070327170947.6afbbf7c@localhost> References: <20070327170947.6afbbf7c@localhost> Message-ID: <20070327171553.GA13912@panix.com> On Tue, Mar 27, 2007, Frank Benkstein wrote: > > Comments? Should I try to wrap this up as a PEP? You should try to start a discussion on either comp.lang.python or python-ideas. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Need a book? Use your library! From jcarlson at uci.edu Tue Mar 27 20:45:28 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 27 Mar 2007 11:45:28 -0700 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <20070327170947.6afbbf7c@localhost> References: <20070327170947.6afbbf7c@localhost> Message-ID: <20070327094135.FD05.JCARLSON@uci.edu> Frank Benkstein wrote: > One thing that has come up quite often IIRC is the request for an > ordered mapping type. It would like to propose to add such a type to > Python 3000 and make it immutable, to prevent all kind of problems that > would occur otherwise. Other than that it should behave just > the same as dictionaries for all non-modifying operations. In the course of my Python programming, while I have thought it would be convenient to have a mutable sorted mapping (implemented via key, value binary tree), and a mutable indexed sequence (implemented via order-statistic binary tree, for O(logn) insertions and deletions anywhere), about the only time where an ordered mapping would have made sense, whose order depends on definition ordering, has been like ... class foo(...): a = ... b = ... As is the case when creating database schemas via SQLAlchemy, etc. There is a PEP (that is more or less accepted) for offering this *particular* use-case in py3k. > Furthermore it would be nice to have this new type as the default type > for **-style keyword parameters. First it would make creating the > instances of this new type a lot easier and second it would remove the > asymmetry in having a immutable *args and a mutable **kwds. I believe the reason for immutable *args is one of optimization. For quite a while it was faster to create a tuple than a list, and even things like tuple(GENERATOR_EXPRESSION) were faster than list(GENERATOR_EXPRESSION) . I don't believe that the same argument applies to an immutable ordered mapping, and I've heard very few complaints about immutable vs mutable args and kwargs. - Josiah From martin at v.loewis.de Tue Mar 27 22:13:27 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 27 Mar 2007 22:13:27 +0200 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <20070327094135.FD05.JCARLSON@uci.edu> References: <20070327170947.6afbbf7c@localhost> <20070327094135.FD05.JCARLSON@uci.edu> Message-ID: <46097AE7.1040508@v.loewis.de> > I believe the reason for immutable *args is one of optimization. I think there is also a correctness issue at work. For an immutable argument tuple, you can be sure that the tuple won't change "behind you". That, in turn, means that PyArg_ParseTuple can return borrowed references, so that C functions don't have to decref the arguments the have fetched. If the argument tuple was mutable, that would be much more risky. Regards, Martin From bjourne at gmail.com Tue Mar 27 23:44:08 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Tue, 27 Mar 2007 23:44:08 +0200 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <20070327170947.6afbbf7c@localhost> References: <20070327170947.6afbbf7c@localhost> Message-ID: <740c3aec0703271444u5c7bb01ew4fde47125fde9afe@mail.gmail.com> On 3/27/07, Frank Benkstein wrote: > Hi, > > One thing that has come up quite often IIRC is the request for an > ordered mapping type. It would like to propose to add such a type to > Python 3000 and make it immutable, to prevent all kind of problems that > would occur otherwise. Other than that it should behave just > the same as dictionaries for all non-modifying operations. Fantastic idea. If **kwargs where ordered it would finally be possible to create a good (non-hackish!) attribute tuple: col = attrtuple(r = 255, g = 0, b = 128) r, g, b = col Or define XML elements nicely: el = make_el('input', type = 'text', class = 'query') Or interface easier with tabular, order-dependent data. -- mvh Bj?rn From jcarlson at uci.edu Wed Mar 28 00:42:54 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Tue, 27 Mar 2007 15:42:54 -0700 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <740c3aec0703271444u5c7bb01ew4fde47125fde9afe@mail.gmail.com> References: <20070327170947.6afbbf7c@localhost> <740c3aec0703271444u5c7bb01ew4fde47125fde9afe@mail.gmail.com> Message-ID: <20070327154109.FD0A.JCARLSON@uci.edu> "BJ?rn Lindqvist" wrote: > col = attrtuple(r = 255, g = 0, b = 128) > r, g, b = col That is handled reasonably well with Raymond's recipe (or variants thereof). > el = make_el('input', type = 'text', class = 'query') XML elements are not ordering sensitive. > Or interface easier with tabular, order-dependent data. That is handled with the ordered dictionary and metaclass sytax PEP. - Josiah From oliphant.travis at ieee.org Wed Mar 28 07:58:18 2007 From: oliphant.travis at ieee.org (Travis E. Oliphant) Date: Tue, 27 Mar 2007 23:58:18 -0600 Subject: [Python-3000] The latest extended buffer PEP In-Reply-To: <46086189.60108@ieee.org> References: <46086189.60108@ieee.org> Message-ID: <460A03FA.4050902@ieee.org> The latest update is here. Carl and Greg, can I add your names to the PEP author list? I think we are very close. I'd like to start working on the implmentation. The modifications to the struct module is probably where I'll start. I really like the possibilities this will open up for sharing of video, images, audio, databases, between different objects. Algorithms could be written that are object agnostic and work for any object exporting the buffer interface. Are we ready for a pronouncement? -Travis From bjourne at gmail.com Wed Mar 28 10:51:28 2007 From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=) Date: Wed, 28 Mar 2007 10:51:28 +0200 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <20070327154109.FD0A.JCARLSON@uci.edu> References: <20070327170947.6afbbf7c@localhost> <740c3aec0703271444u5c7bb01ew4fde47125fde9afe@mail.gmail.com> <20070327154109.FD0A.JCARLSON@uci.edu> Message-ID: <740c3aec0703280151w5e805cbo478c99b5b4a36d34@mail.gmail.com> On 3/28/07, Josiah Carlson wrote: > > "BJ?rn Lindqvist" wrote: > > col = attrtuple(r = 255, g = 0, b = 128) > > r, g, b = col > > That is handled reasonably well with Raymond's recipe (or variants > thereof). Well, no. NamedTuple('Color', 'r g b')(r = 255, g = 0, b = 128) is not at all the same thing. > > el = make_el('input', type = 'text', class = 'query') > > XML elements are not ordering sensitive. Except when you write them. If you want the output to be exactly you must have an order. > > Or interface easier with tabular, order-dependent data. > > That is handled with the ordered dictionary and metaclass sytax PEP. Yes. Making **kwargs orderable would merely be the syntactic support that would make ordered dictionaries a breeze to work with. -- mvh Bj?rn From jcarlson at uci.edu Wed Mar 28 17:33:08 2007 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 28 Mar 2007 08:33:08 -0700 Subject: [Python-3000] Ordered mapping type for Python 3000? In-Reply-To: <740c3aec0703280151w5e805cbo478c99b5b4a36d34@mail.gmail.com> References: <20070327154109.FD0A.JCARLSON@uci.edu> <740c3aec0703280151w5e805cbo478c99b5b4a36d34@mail.gmail.com> Message-ID: <20070328075712.FD13.JCARLSON@uci.edu> "BJ?rn Lindqvist" wrote: > > On 3/28/07, Josiah Carlson wrote: > > > > "BJ?rn Lindqvist" wrote: > > > el = make_el('input', type = 'text', class = 'query') > > > > XML elements are not ordering sensitive. > > Except when you write them. If you want the output to be exactly > you must have an order. I've not come across a case where web browsers or other XML consumers have cared about those kinds of orderings (except in testing cases). And those cases where order has mattered, sorting by name (so as to consistantly produce as in your example) gets one a consistant ordering in all Pythons. > > > col = attrtuple(r = 255, g = 0, b = 128) > > > r, g, b = col > > > > That is handled reasonably well with Raymond's recipe (or variants > > thereof). > > Well, no. NamedTuple('Color', 'r g b')(r = 255, g = 0, b = 128) is not > at all the same thing. > > > > Or interface easier with tabular, order-dependent data. > > > > That is handled with the ordered dictionary and metaclass sytax PEP. > > Yes. Making **kwargs orderable would merely be the syntactic support > that would make ordered dictionaries a breeze to work with. That is not what I meant. For data defined as columns, etc., where order matters (also in the case of NamedTuple), PEP 3115 handles this case fairly well, generally like: class Color(NamedTuple): r = g = b = None a = Color(r = 255, g = 0, b = 128) or class Contacts(TypedNameTuple): firstname = text middle = text lastname = text ... Yes, you need to take the time to create the prototype. But if taking the time to create one prototype earlier in the source to define ordering isn't reasonable, then it seems to me that one is trying to be too clever. Also, I find Color(r = 255, g = 0, b = 128) to be clearer than ordered_mapping(r = 255, g = 0, b = 128) . I would also point out that neither the OP (nor anyone else) has proposed a possible implementation (or strategy for that matter). - Josiah From amauryfa at gmail.com Thu Mar 29 21:31:16 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 29 Mar 2007 21:31:16 +0200 Subject: [Python-3000] [Python-3000-checkins] r54588 Message-ID: Hello, Sorry if I am wrong, but it seems to me that the change in r54588 has a problem: http://mail.python.org/pipermail/python-3000-checkins/2007-March/000433.html - in the normal case, the return value is INCREF'ed twice - in the error case, Py_INCREF(NULL) is called... One easy way to correct this is to move the last INCREF: (sorry for the approximative patch format) python/branches/p3yk/Objects/typeobject.c: ================================================= static PyObject * object_richcompare(PyObject *self, PyObject *other, int op) { PyObject *res; switch (op) { case Py_EQ: res = (self == other) ? Py_True : Py_False; + Py_INCREF(res); break; case Py_NE: /* By default, != returns the opposite of ==, unless the latter returns NotImplemented. */ res = PyObject_RichCompare(self, other, Py_EQ); if (res != NULL && res != Py_NotImplemented) { int ok = PyObject_IsTrue(res); Py_DECREF(res); if (ok < 0) res = NULL; else { if (ok) res = Py_False; else res = Py_True; Py_INCREF(res); } } break; default: res = Py_NotImplemented; + Py_INCREF(res); break; } - Py_INCREF(res); return res; } Just trying to be faster than Coverity... -- Amaury Forgeot d'Arc From brett at python.org Thu Mar 29 21:52:40 2007 From: brett at python.org (Brett Cannon) Date: Thu, 29 Mar 2007 12:52:40 -0700 Subject: [Python-3000] [Python-3000-checkins] r54588 In-Reply-To: References: Message-ID: On 3/29/07, Amaury Forgeot d'Arc wrote: > Hello, > > Sorry if I am wrong, but it seems to me that the change in r54588 has a problem: > http://mail.python.org/pipermail/python-3000-checkins/2007-March/000433.html > - in the normal case, the return value is INCREF'ed twice > - in the error case, Py_INCREF(NULL) is called... > > One easy way to correct this is to move the last INCREF: > (sorry for the approximative patch format) > > python/branches/p3yk/Objects/typeobject.c: > ================================================= > static PyObject * > object_richcompare(PyObject *self, PyObject *other, int op) > { > PyObject *res; > > switch (op) { > > case Py_EQ: > res = (self == other) ? Py_True : Py_False; > + Py_INCREF(res); > break; > > case Py_NE: > /* By default, != returns the opposite of ==, > unless the latter returns NotImplemented. */ > res = PyObject_RichCompare(self, other, Py_EQ); > if (res != NULL && res != Py_NotImplemented) { > int ok = PyObject_IsTrue(res); > Py_DECREF(res); > if (ok < 0) > res = NULL; > else { > if (ok) > res = Py_False; > else > res = Py_True; > Py_INCREF(res); > } > } > break; > > default: > res = Py_NotImplemented; > + Py_INCREF(res); > break; > } > > - Py_INCREF(res); > return res; > } > It looks right, although you could also remove the INCREF in Py_NE and change the INCREF at the bottom to XINCREF or just return on the error. -Brett From guido at python.org Thu Mar 29 22:51:42 2007 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Mar 2007 13:51:42 -0700 Subject: [Python-3000] [Python-3000-checkins] r54588 In-Reply-To: References: Message-ID: Thanks Amaury! I've submitted your fix: Committed revision 54609. (Brett's fix would require an additional DECREF when NotImplemented is returned, so Amaury's version is better. I bet a compiler can rearrange the code so that the INCREF code is shared. :-) --Guido On 3/29/07, Brett Cannon wrote: > On 3/29/07, Amaury Forgeot d'Arc wrote: > > Hello, > > > > Sorry if I am wrong, but it seems to me that the change in r54588 has a problem: > > http://mail.python.org/pipermail/python-3000-checkins/2007-March/000433.html > > - in the normal case, the return value is INCREF'ed twice > > - in the error case, Py_INCREF(NULL) is called... > > > > One easy way to correct this is to move the last INCREF: > > (sorry for the approximative patch format) > > > > python/branches/p3yk/Objects/typeobject.c: > > ================================================= > > static PyObject * > > object_richcompare(PyObject *self, PyObject *other, int op) > > { > > PyObject *res; > > > > switch (op) { > > > > case Py_EQ: > > res = (self == other) ? Py_True : Py_False; > > + Py_INCREF(res); > > break; > > > > case Py_NE: > > /* By default, != returns the opposite of ==, > > unless the latter returns NotImplemented. */ > > res = PyObject_RichCompare(self, other, Py_EQ); > > if (res != NULL && res != Py_NotImplemented) { > > int ok = PyObject_IsTrue(res); > > Py_DECREF(res); > > if (ok < 0) > > res = NULL; > > else { > > if (ok) > > res = Py_False; > > else > > res = Py_True; > > Py_INCREF(res); > > } > > } > > break; > > > > default: > > res = Py_NotImplemented; > > + Py_INCREF(res); > > break; > > } > > > > - Py_INCREF(res); > > return res; > > } > > > > > It looks right, although you could also remove the INCREF in Py_NE and > change the INCREF at the bottom to XINCREF or just return on the > error. > > -Brett > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/)