From greg.ewing at canterbury.ac.nz Fri Dec 1 00:44:39 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 01 Dec 2006 12:44:39 +1300 Subject: [Python-3000] Low-hanging fruit: change interpreter prompt? In-Reply-To: <456E9240.5010302@develer.com> References: <456E9240.5010302@develer.com> Message-ID: <456F6CE7.6080102@canterbury.ac.nz> Giovanni Bajo wrote: > In fact, there would be another reason to change the prompt in Py3k: to > identify Py3k snippets and tell them from Python 2.x snippets. P3> -- Greg From janssen at parc.com Fri Dec 1 02:08:05 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 30 Nov 2006 17:08:05 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <456F5E4F.6090705@canterbury.ac.nz> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <06Nov29.154543pst.58648@synergy1.parc.xerox.com> <456E2708.6070205@canterbury.ac.nz> <06Nov29.183824pst.58648@synergy1.parc.xerox.com> <456F5E4F.6090705@canterbury.ac.nz> Message-ID: <06Nov30.170806pst."58648"@synergy1.parc.xerox.com> > Bill Janssen wrote: > > So file.readlines() would return an iterable, then? > > I wasn't thinking of having a file.readlines(), just > a stand-alone function. I'd like to see the file > object's interface made quite minimal in Py3k. > > -- > Greg Where would the standalone function live? How would it be associated with whatever open() returns? Bill From janssen at parc.com Fri Dec 1 02:58:38 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 30 Nov 2006 17:58:38 PST Subject: [Python-3000] Generic functions vs. OO In-Reply-To: <06Nov28.100326pst."58648"@synergy1.parc.xerox.com> References: <06Nov28.100326pst."58648"@synergy1.parc.xerox.com> Message-ID: <06Nov30.175847pst."58648"@synergy1.parc.xerox.com> I've updated the page to reflect more of Jim Jewett's comments. And to reflect more of the underlying "special methods". "Container", for instance, now has "get" (for __getattr__), as well as "len" (for __len__). By the way, all these interfaces are already in Python; they just aren't written down anywhere in a group. Instead, they are mostly scattered through the documentation, some in the Library Reference manual, others in the Language Ref manual. (Yes, I have split the file I/O functionality into a number of separate interfaces.) Bill > Guido van Rossum wrote: > > I wonder if a bunch of well thought-out standard ABCs, applied to the > > standard data types, and perhaps more support for setting __bases__, > > wouldn't address most concerns. > > I think it would, but I don't know if we're going to get anywhere > without getting more concrete. Let's tap "the wisdom of crowds". I've > set up a Wiki page at http://wiki.python.org/moin/AbstractBaseClasses, > and spent a little time defining some ABCs and concrete classes. I'll > work on it more over the week, but feel free to pitch in and expand it. > Or create alternative versions (create a new Wiki page, and put a link > to it at the top of the existing page). > > Bill From p.f.moore at gmail.com Fri Dec 1 10:17:49 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 1 Dec 2006 09:17:49 +0000 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <8653224966967895047@unknownmsgid> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> Message-ID: <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> On 11/30/06, Bill Janssen wrote: > > so no offense in advance. > > Sure, no offense taken. I've seen comments like this before on this > list (recently :-). I think both approaches (interface types and duck > typing) are complicated in different ways. Instinctively, I agree with Tomer on this issue. But I'm reasonably relaxed about the matter as long as it's optional. What I'm not sure about, is if that is the intention. For example, class MyMap: def __getitem__(self, key): ... def __setitem__(self, key, value): ... If I pass this into a function (written by a 3rd party, so I have no way of changing it) that *only* uses the syntax m[k] on it (never needs to do len(m), or m.keys() or anything else), will it work? Please note that MyMap is deliberately written to *not* conform to any of your proposed base classes. If it doesn't work, then my code is excluded from using a perfectly good piece of 3rd party code for no reason. Sure, I can "fix" the problem by implementing a few extra methods (just raising an exception if they are called) and possibly declaring that I conform to an interface (which, actually, I don't). But why should I have to? I appreciate that this is an oversimplified argument, and that interfaces/ABCs/GFs/whatever have significant value within certain types of framework - but you're not proposing a hierarchy for framework-specific classes here, but rather for the fundamental types of the language. If the proposed class hierarchy is going to have an impact on code which is not written around a framework (like Twisted) which is built on interfaces, the result won't feel like Python to me, as far as I can see... Paul. From behnel_ml at gkec.informatik.tu-darmstadt.de Fri Dec 1 14:20:42 2006 From: behnel_ml at gkec.informatik.tu-darmstadt.de (Stefan Behnel) Date: Fri, 01 Dec 2006 14:20:42 +0100 Subject: [Python-3000] Low-hanging fruit: change interpreter prompt? In-Reply-To: References: Message-ID: <45702C2A.9060106@gkec.informatik.tu-darmstadt.de> Steven Bethard schrieb: > On 11/29/06, Guido van Rossum wrote: >> But the >>> prompt is Python's trademark! I always get a warm fuzzy >> feeling when I see it, e.g. in a corner of a slide in a NASA >> presentation. > > I was using ``py>`` for a while. That still gave me a warm fuzzy > feeling ;-) and seemed to work with most newsreaders. I think there are enough ways to get around the e-mail problem. One of the least intrusive ways is this: .>>> 1+1 2 Stefan From guido at python.org Fri Dec 1 17:17:55 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Dec 2006 08:17:55 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> Message-ID: On 12/1/06, Paul Moore wrote: > On 11/30/06, Bill Janssen wrote: > > > so no offense in advance. > > > > Sure, no offense taken. I've seen comments like this before on this > > list (recently :-). I think both approaches (interface types and duck > > typing) are complicated in different ways. > > Instinctively, I agree with Tomer on this issue. But I'm reasonably > relaxed about the matter as long as it's optional. What I'm not sure > about, is if that is the intention. For example, > > class MyMap: > def __getitem__(self, key): ... > def __setitem__(self, key, value): ... > > If I pass this into a function (written by a 3rd party, so I have no > way of changing it) that *only* uses the syntax m[k] on it (never > needs to do len(m), or m.keys() or anything else), will it work? > Please note that MyMap is deliberately written to *not* conform to any > of your proposed base classes. It would still work, *unless* the code that receives this object does an explicit check for whether its argument implements some Mapping interface or ABC. > If it doesn't work, then my code is excluded from using a perfectly > good piece of 3rd party code for no reason. Sure, I can "fix" the > problem by implementing a few extra methods (just raising an exception > if they are called) and possibly declaring that I conform to an > interface (which, actually, I don't). But why should I have to? If the 3rd party code makes it part of its API requirements that you must only pass it an object that implements a certain interface, then you had better conform to that requirement. The 3rd party code may have a plan for evolving the implementation in such a way that eventually it *will* require all the functionality of that interface, and they would prefer that you are warned of this possibility in advance. > I appreciate that this is an oversimplified argument, and that > interfaces/ABCs/GFs/whatever have significant value within certain > types of framework - but you're not proposing a hierarchy for > framework-specific classes here, but rather for the fundamental types > of the language. The proposal provides a better way to check whether something implements a particular interface. It doesn't propose adding any automatic type checking to the language. If you were writing a framework today, and you had to design an API that took either a sequence or a mapping -- how would you distinguish the two? Both implement __getitem__, __iter__ and __len__. Would you test for the presence of a keys() method, even if you have no need to ever call it? What makes keys() special? With interfaces or ABCs, you can test whether it implements the Mapping or Sequence API (or some more primitive base for those). > If the proposed class hierarchy is going to have an impact on code > which is not written around a framework (like Twisted) which is built > on interfaces, the result won't feel like Python to me, as far as I > can see... Of course. And nobody is proposing that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From janssen at parc.com Fri Dec 1 17:31:15 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Dec 2006 08:31:15 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> Message-ID: <06Dec1.083117pst."58648"@synergy1.parc.xerox.com> > Instinctively, I agree with Tomer on this issue. But I'm reasonably > relaxed about the matter as long as it's optional. What I'm not sure > about, is if that is the intention. For example, > > class MyMap: > def __getitem__(self, key): ... > def __setitem__(self, key, value): ... > > If I pass this into a function (written by a 3rd party, so I have no > way of changing it) that *only* uses the syntax m[k] on it (never > needs to do len(m), or m.keys() or anything else), will it work? > Please note that MyMap is deliberately written to *not* conform to any > of your proposed base classes. Paul, So long as the third-party function *only* uses m[k] on it (and m[k] is implemented through __getitem__, as it presumably would be), why wouldn't it? I'm not suggesting generic functions which do interface-based method dispatch (though we may want to do that in the future). There could be issues regarding operators, I agree. For instance, the [k] operation may be implemented by the Container interface generic implementation, in which case your type might have to have the Container interface, which currently means it would have to support __len__ and __contains__, as well. But I think that's a separable discussion, about the locality of operator implementation. Bill From pedronis at strakt.com Fri Dec 1 17:45:17 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Fri, 01 Dec 2006 17:45:17 +0100 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> Message-ID: <45705C1D.5090301@strakt.com> Guido van Rossum wrote: > On 12/1/06, Paul Moore wrote: > >> On 11/30/06, Bill Janssen wrote: >> >>>> so no offense in advance. >>>> >>> Sure, no offense taken. I've seen comments like this before on this >>> list (recently :-). I think both approaches (interface types and duck >>> typing) are complicated in different ways. >>> >> Instinctively, I agree with Tomer on this issue. But I'm reasonably >> relaxed about the matter as long as it's optional. What I'm not sure >> about, is if that is the intention. For example, >> >> class MyMap: >> def __getitem__(self, key): ... >> def __setitem__(self, key, value): ... >> >> If I pass this into a function (written by a 3rd party, so I have no >> way of changing it) that *only* uses the syntax m[k] on it (never >> needs to do len(m), or m.keys() or anything else), will it work? >> Please note that MyMap is deliberately written to *not* conform to any >> of your proposed base classes. >> > > It would still work, *unless* the code that receives this object does > an explicit check for whether its argument implements some Mapping > interface or ABC. > > >> If it doesn't work, then my code is excluded from using a perfectly >> good piece of 3rd party code for no reason. Sure, I can "fix" the >> problem by implementing a few extra methods (just raising an exception >> if they are called) and possibly declaring that I conform to an >> interface (which, actually, I don't). But why should I have to? >> > > If the 3rd party code makes it part of its API requirements that you > must only pass it an object that implements a certain interface, then > you had better conform to that requirement. but the hierarchy is too imprecise to avoid the 3rd party making too narrow requirements. From that point of view the ability to refer easily to subsets of interfaces is seems quite crucial to reflect current Python usage. Also the built-in types are concrete types, which tend to maximize convenience. It's unclear that they are a good starting point to define interfaces, because of this maximality. Especially if there's no standard way to do adaptation instead of just checking for the interface, or have derivable methods implemented automatically from a minimal subset. I suppose there's a reason that interfaces and adaptation go together in their current usages in Python. In some sense that from the fact that we would like to distinguish stringish stuff and mappings (in a minimal sense) from sequences, we generalize to wanting interfaces for the core of Python, is quite a jump, especially if then we don't want all the baggage of how interfaces are practiced today by large frameworks, because this is essentially an unproven solution. Comparisons with Java don't really help, the latter is still a statically typed language. > The 3rd party code may > have a plan for evolving the implementation in such a way that > eventually it *will* require all the functionality of that interface, > and they would prefer that you are warned of this possibility in > advance. > > >> I appreciate that this is an oversimplified argument, and that >> interfaces/ABCs/GFs/whatever have significant value within certain >> types of framework - but you're not proposing a hierarchy for >> framework-specific classes here, but rather for the fundamental types >> of the language. >> > > The proposal provides a better way to check whether something > implements a particular interface. It doesn't propose adding any > automatic type checking to the language. > > If you were writing a framework today, and you had to design an API > that took either a sequence or a mapping -- how would you distinguish > the two? Both implement __getitem__, __iter__ and __len__. Would you > test for the presence of a keys() method, even if you have no need to > ever call it? What makes keys() special? With interfaces or ABCs, you > can test whether it implements the Mapping or Sequence API (or some > more primitive base for those). > > >> If the proposed class hierarchy is going to have an impact on code >> which is not written around a framework (like Twisted) which is built >> on interfaces, the result won't feel like Python to me, as far as I >> can see... >> > > Of course. And nobody is proposing that. > > From tomerfiliba at gmail.com Fri Dec 1 18:08:20 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Fri, 1 Dec 2006 19:08:20 +0200 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <-6849499535038592093@unknownmsgid> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> Message-ID: <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> [Bill] > I agree. For instance, the > [k] operation may be implemented by the Container interface generic > implementation, in which case your type might have to have the > Container interface i would still like to understand how object proxying (e.g., weakrefs, RPyC), could work when all/most frameworks/libraries would be based on ABCs. again, the problem is, your proxy is an instance of ProxyType, that refers to another, "concrete" object (e.g., getattr on the proxy performs a getattr on the concrete object). but this only works with duck typing. the concrete object may implement certain interfaces that ProxyType does not/cannot implement (a la 3rd party). so despite the fact that the proxy object *behaves* just like the concrete object, they are of different types and ABCs. checking if the proxy implements an interface may fail, although the proxy is sufficient for whatever purposes you wished to use it. the only way out of this, that i can think of, would require introducing a __does_implement__ special method that will be called by does_implement(), or whatever the name will be. of course you could add a new ABC for that purpose, i.e., CustomImplementer. that way, ProxyType will derive from CustomImplementer, and could perform all the necessary magic inside, to look like the concrete object. if custom proxies couldn't override the default implementation checking mechanism (such as isinstance), i'm strongly against that. -tomer From fdrake at acm.org Fri Dec 1 18:18:59 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 1 Dec 2006 12:18:59 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> Message-ID: <200612011219.00142.fdrake@acm.org> On Friday 01 December 2006 12:08, tomer filiba wrote: > i would still like to understand how object proxying (e.g., weakrefs, > RPyC), could work when all/most frameworks/libraries would be based on > ABCs. That's a realy problem. zope.interface works hard to deal with proxies, but it probably only works with proxies based on zope.proxy. > the only way out of this, that i can think of, would require introducing a > __does_implement__ special method that will be called by > does_implement(), or whatever the name will be. of course you could > add a new ABC for that purpose, i.e., CustomImplementer. This only works if there's only one interface system. As soon as there are multiple systems at work in parallel (possibly because different libraries require them, even if the app doesn't care), this is going to get quite complex. I wish I had some good insight on how to deal with this, but I don't. Jim Fulton might have some ideas since he dealt with all this for zope.interface. He's likely pretty busy right now, though. (And I don't know if he's on this list.) -Fred -- Fred L. Drake, Jr. From guido at python.org Fri Dec 1 19:19:07 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Dec 2006 10:19:07 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <200612011219.00142.fdrake@acm.org> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <200612011219.00142.fdrake@acm.org> Message-ID: On 12/1/06, Fred L. Drake, Jr. wrote: > On Friday 01 December 2006 12:08, tomer filiba wrote: > > i would still like to understand how object proxying (e.g., weakrefs, > > RPyC), could work when all/most frameworks/libraries would be based on > > ABCs. > > That's a realy problem. zope.interface works hard to deal with proxies, but > it probably only works with proxies based on zope.proxy. I think the proper solution is to make isinstance() overridable by the instance. > > the only way out of this, that i can think of, would require introducing a > > __does_implement__ special method that will be called by > > does_implement(), or whatever the name will be. of course you could > > add a new ABC for that purpose, i.e., CustomImplementer. > > This only works if there's only one interface system. As soon as there are > multiple systems at work in parallel (possibly because different libraries > require them, even if the app doesn't care), this is going to get quite > complex. > > I wish I had some good insight on how to deal with this, but I don't. Jim > Fulton might have some ideas since he dealt with all this for zope.interface. > He's likely pretty busy right now, though. (And I don't know if he's on this > list.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Fri Dec 1 19:37:54 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 01 Dec 2006 13:37:54 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <200612011219.00142.fdrake@acm.org> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <200612011219.00142.fdrake@acm.org> Message-ID: <5.1.1.6.0.20061201133551.03a78e50@sparrow.telecommunity.com> At 10:19 AM 12/1/2006 -0800, Guido van Rossum wrote: >On 12/1/06, Fred L. Drake, Jr. wrote: > > On Friday 01 December 2006 12:08, tomer filiba wrote: > > > i would still like to understand how object proxying (e.g., weakrefs, > > > RPyC), could work when all/most frameworks/libraries would be based on > > > ABCs. > > > > That's a realy problem. zope.interface works hard to deal with > proxies, but > > it probably only works with proxies based on zope.proxy. > >I think the proper solution is to make isinstance() overridable by the >instance. It already is, and has been since Python 2.3, in large part to support zope-style proxying. It's also leveraged by my ProxyTypes package in the Cheeseshop, and by a few bits of code in Chandler. From jimjjewett at gmail.com Fri Dec 1 19:45:57 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Fri, 1 Dec 2006 13:45:57 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> Message-ID: On 12/1/06, tomer filiba wrote: > i would still like to understand how object proxying (e.g., weakrefs, RPyC), > could work when all/most frameworks/libraries would be based on ABCs. > again, the problem is, your proxy is an instance of ProxyType, that > refers to another, "concrete" object (e.g., getattr on the proxy performs a > getattr on the concrete object). but this only works with duck typing. > the concrete object may implement certain interfaces that ProxyType > does not/cannot implement (a la 3rd party). so despite the fact that the > proxy object *behaves* just like the concrete object, they are of different > types and ABCs. checking if the proxy implements an interface may > fail, although the proxy is sufficient for whatever purposes you wished > to use it. Why? Why can't your proxy be a derived class that claims to inherit from those same base classes, but just happens to override all the methods (and attributes). If the problem is that an ABC is checked by identity rather than name, and it isn't available to the proxy, then how is it available to the checker? -jJ From pje at telecommunity.com Fri Dec 1 19:57:53 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 01 Dec 2006 13:57:53 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> Message-ID: <5.1.1.6.0.20061201135620.0287d2d0@sparrow.telecommunity.com> At 01:45 PM 12/1/2006 -0500, Jim Jewett wrote: >Why can't your proxy be a derived class that claims to inherit >from those same base classes, but just happens to override all the >methods (and attributes). Because that means you can't use generic proxy objects (e.g. zope.proxy or ProxyTypes). Luckily, however, proxies can lie about their __class__ (since Python 2.3) and thereby fool isinstance(). From janssen at parc.com Fri Dec 1 20:12:11 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Dec 2006 11:12:11 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <45705C1D.5090301@strakt.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <45705C1D.5090301@strakt.com> Message-ID: <06Dec1.111213pst."58648"@synergy1.parc.xerox.com> Samuele Pedroni writes: > Also the built-in types are concrete types, which tend to maximize > convenience. It's unclear > that they are a good starting point to define interfaces, because of > this maximality. I agree with you on this, Samuele. I don't want to deep-end, and put only one method in each interface, but I think the standard types could be decomposed a bit. I've tried to do this with my list (which, by the way, feel free to fiddle with -- that's why it's a Wiki page) at http://wiki.python.org/moin/AbstractBaseClasses. You'll note that the "list" type is constructed of six interfaces: Comparable, Object, Container, MutableContainer, Sequence, and MutableSequence. (Though the inheritance hierarchy makes this easier to think about.) And the "file" object is broken into nine base interfaces. My thinking (about "file") is that in Py3 the actual file type might be implementation-dependent, and in fact there might be several of them, each of which would mix together a different set of file interfaces. I think a similar decomposition would apply to Mapping; I just don't have a good idea of what it would be (yet). Bill From janssen at parc.com Fri Dec 1 20:15:33 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Dec 2006 11:15:33 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201135620.0287d2d0@sparrow.telecommunity.com> References: <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <5.1.1.6.0.20061201135620.0287d2d0@sparrow.telecommunity.com> Message-ID: <06Dec1.111538pst."58648"@synergy1.parc.xerox.com> > Because that means you can't use generic proxy objects (e.g. zope.proxy or > ProxyTypes). > > Luckily, however, proxies can lie about their __class__ (since Python 2.3) > and thereby fool isinstance(). My guess is that support for weakrefs and/or RPC proxies (something with which I have a lot of experience, so I appreciate the point) would want to use some kind of "sneaky" mechanism like that. We should probably make sure that there's an easy-to-use standard "sneaky" mechanism -- perhaps just lying about the __class__ (in a safe way) is enough. Bill From guido at python.org Fri Dec 1 20:25:50 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Dec 2006 11:25:50 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201133551.03a78e50@sparrow.telecommunity.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <200612011219.00142.fdrake@acm.org> <5.1.1.6.0.20061201133551.03a78e50@sparrow.telecommunity.com> Message-ID: On 12/1/06, Phillip J. Eby wrote: > At 10:19 AM 12/1/2006 -0800, Guido van Rossum wrote: > >On 12/1/06, Fred L. Drake, Jr. wrote: > > > On Friday 01 December 2006 12:08, tomer filiba wrote: > > > > i would still like to understand how object proxying (e.g., weakrefs, > > > > RPyC), could work when all/most frameworks/libraries would be based on > > > > ABCs. > > > > > > That's a realy problem. zope.interface works hard to deal with > > proxies, but > > > it probably only works with proxies based on zope.proxy. > > > >I think the proper solution is to make isinstance() overridable by the > >instance. > > It already is, and has been since Python 2.3, in large part to support > zope-style proxying. It's also leveraged by my ProxyTypes package in the > Cheeseshop, and by a few bits of code in Chandler. I know (I should, since I wrote the code :-) but I think the solution could be much cleaner. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From p.f.moore at gmail.com Fri Dec 1 20:34:14 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 1 Dec 2006 19:34:14 +0000 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> Message-ID: <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> On 12/1/06, tomer filiba wrote: > i would still like to understand how object proxying (e.g., weakrefs, RPyC), > could work when all/most frameworks/libraries would be based on ABCs. I don't think that's what is being proposed, if I understand Guido's response to my post. There is no expectation that all/most (or even many!) libraries would be based on ABCs. Duck typing would remain the norm. Although I'm not clear what the value of all this is, if the resulting ABCs *don't* get used in real code. But it's likely that this is a failure in my understanding, rather than the sign of a deep and complex plot to turn Python into Java while I'm not watching :-) So I'm going to continue reading, and wait for enlightenment. Paul. From janssen at parc.com Fri Dec 1 20:52:28 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Dec 2006 11:52:28 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> Message-ID: <06Dec1.115233pst."58648"@synergy1.parc.xerox.com> Paul Moore writes: > There is no expectation that all/most (or even > many!) libraries would be based on ABCs. Duck typing would remain the > norm. Although I'm not clear what the value of all this is, if the > resulting ABCs *don't* get used in real code. I'm not sure what it means to base a library on ABCs. You base types on ABCs. But I'd hope that more library implementors would shift to documenting required interfaces for parameters where appropriate, rather than the current practice of either (1) not specifying anything, and forcing a caller to read the code, or (2) spotty and incomplete documentation of required methods, forcing smart callers to read the code. And lots of library implementors don't know what methods are required if you need a value to support the "[k]" operation. I would hope that eventually, for most widely used library packages, duck typing would *not* be the norm, because I think it's a source of both fragility and FUD in the Python world. Btw, I share your desire not to turn Python into Java (ugh). Bill From pje at telecommunity.com Fri Dec 1 21:55:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 01 Dec 2006 15:55:15 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <06Dec1.115233pst."58648"@synergy1.parc.xerox.com> References: <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> Message-ID: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> At 11:52 AM 12/1/2006 -0800, Bill Janssen wrote: >Paul Moore writes: > > There is no expectation that all/most (or even > > many!) libraries would be based on ABCs. Duck typing would remain the > > norm. Although I'm not clear what the value of all this is, if the > > resulting ABCs *don't* get used in real code. > >I'm not sure what it means to base a library on ABCs. You base types >on ABCs. > >But I'd hope that more library implementors would shift to documenting >required interfaces for parameters where appropriate, rather than the >current practice of either (1) not specifying anything, and forcing a >caller to read the code, or (2) spotty and incomplete documentation of >required methods, forcing smart callers to read the code. And lots of >library implementors don't know what methods are required if you need >a value to support the "[k]" operation. I would hope that eventually, >for most widely used library packages, duck typing would *not* be the >norm, because I think it's a source of both fragility and FUD in the >Python world. I would actually hope that library implementors would shift to using generic functions, since then they can just document what the function(s) are supposed to do, and people can just add methods to those functions for any object types they like, without needing to subclass anything, or have to deal with "interfaces" at all. In other words, I just want to use my object with some operations that a library provides (or requires). An "interface" is excise: something I have to mess with that doesn't directly relate to my goals for using the library. Generic functions are type-safe duck typing because ducklib.quack(ob) doesn't rely on ob having a 'quack()' method that means what you think it means. This solves the problem simply and directly, without introducing interfaces and ABCs and whatnot. Either the operation is defined for the type or it isn't. At most, an interface need only specify what operations must be defined for a type, in order to allow it to be used for a different operation. From janssen at parc.com Fri Dec 1 22:41:23 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Dec 2006 13:41:23 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> References: <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> Message-ID: <06Dec1.134129pst."58648"@synergy1.parc.xerox.com> > I would actually hope that library implementors would shift to using > generic functions, since then they can just document what the function(s) > are supposed to do, and people can just add methods to those functions for > any object types they like, without needing to subclass anything, or have > to deal with "interfaces" at all. I've got sympathy for this point of view, too (the Lispier the better, IMO), particularly for libraries. But note that this still means that people need to talk about "object types", and I think adding interfaces gives us a language for doing that. Remember that the Lisp object system was GF-based because Lisp was already a function-based language, and the GF system (which I regard as very ingenious) fit nicely into the existing framework. I think that shifting Python to a truly GF-based language would be a more radical change, since it has a strongly OO orientation right now (invoking methods on objects). Even GFs don't solve all invocation problems; in Lisp there was a lot of trouble in trying to retrofit the signature for a method for an existing GF into the existing definition. In essence, the inheritance issue with ABCs becomes a signature issue with GFs, only you have it with each function instead of with each type. And adding GFs does not give us a coherent ability to do type introspection, which I think would help. Again, I seem to need to look at values *returned* from calls as much as I worry about the type of parameters passed into my functions. GFs wouldn't help me there, would they? Bill From guido at python.org Fri Dec 1 22:46:40 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 1 Dec 2006 13:46:40 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> Message-ID: On 12/1/06, Phillip J. Eby wrote: > At 11:52 AM 12/1/2006 -0800, Bill Janssen wrote: > >Paul Moore writes: > > > There is no expectation that all/most (or even > > > many!) libraries would be based on ABCs. Duck typing would remain the > > > norm. Although I'm not clear what the value of all this is, if the > > > resulting ABCs *don't* get used in real code. > > > >I'm not sure what it means to base a library on ABCs. You base types > >on ABCs. > > > >But I'd hope that more library implementors would shift to documenting > >required interfaces for parameters where appropriate, rather than the > >current practice of either (1) not specifying anything, and forcing a > >caller to read the code, or (2) spotty and incomplete documentation of > >required methods, forcing smart callers to read the code. And lots of > >library implementors don't know what methods are required if you need > >a value to support the "[k]" operation. I would hope that eventually, > >for most widely used library packages, duck typing would *not* be the > >norm, because I think it's a source of both fragility and FUD in the > >Python world. > > I would actually hope that library implementors would shift to using > generic functions, since then they can just document what the function(s) > are supposed to do, and people can just add methods to those functions for > any object types they like, without needing to subclass anything, or have > to deal with "interfaces" at all. I'm not sure what you mean this. Are you proposing that a library function that might take a mapping would be rewritten as a generic library function with one implementation that takes a mapping? Or are you proposing that the library function, instead of documenting that it takes a mapping, documents which generic functions should be applied to it? Both sound like rather a big stretch from current practice. > In other words, I just want to use my object with some operations that a > library provides (or requires). An "interface" is excise: something I have > to mess with that doesn't directly relate to my goals for using the library. I don't understand the sentence "An "interface" is excise". What does excise mean in this context? Regardless, I find it quite a big change from various ways of saying "this object must have these methods (including perhaps some for which special syntax exists, like __getattr__)" to saying "this object must be supported by these generic functions". A method is just an identifier in the object's attribute namespace. A generic function is an object that may hve to be imported from elsewhere. This may make sense in CLOS (if I understand the summary Bill gave me over lunch Monday correctly) but is quite different from current practice in Python. > Generic functions are type-safe duck typing because ducklib.quack(ob) > doesn't rely on ob having a 'quack()' method that means what you think it > means. This solves the problem simply and directly, without introducing > interfaces and ABCs and whatnot. Either the operation is defined for the > type or it isn't. I object to the suggestion that seems to be implied here that in the future we'll all be writing ducklib.quack(ob) instead of the much simpler ob.quack(). I really like that when I have an object in my hands, I don't need to import anything else in order to manipulate it, as long as the manipulation can be done through methods. I also don't think that the homonym problem that you (and CLOS) are trying to solve here is all that important in practice. For the one homonym that is relevant in practice (__getitem__ ipmlemented by sequences and mappings alike, but with very different invariants) I think I prefer Bill's ABC solution or an interfaces-based solution over a GF-based solution, because the ABC and the interface address the issue that there are invariants *across* operations. (E.g. x in d equals x in d.keys(), len(d) == len(d.keys(), d.items() == zip(d.keys(), d.values(), and and many more.) So knowing that something is an instance of a Mapping ABC or interface vs. a Sequence ditto conveys more information than knowing that it implements __getitem__(). You could probably address this somewhat by having two different oeprations, Mapping.__getitem__ and Sequence.__getitem__, but that doesn't change the fact that the normal spelling is x[y] in both cases, and it doesn't do much about the relationships between different operations either. (Did you see the example I posted about a week ago sketching an ABC or interface hierarchy for sets? That strongly exemplified IMO the need to be able to talk about a group of operations together with some invariants, rather than just a bunch of operations.) > At most, an interface need only specify what operations must be defined for > a type, in order to allow it to be used for a different operation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tomerfiliba at gmail.com Fri Dec 1 23:02:00 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Sat, 2 Dec 2006 00:02:00 +0200 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> Message-ID: <1d85506f0612011402u44e79c06wc11e9c515fd6ba7f@mail.gmail.com> [Phillip] > Luckily, however, proxies can lie about their __class__ (since Python 2.3) > and thereby fool isinstance(). yes, well, you try to override __class__ when the real class is actually in a different process :) types compare by ID, so when a proxy is created for an object in another process, you can only set __class__ to a *proxy* to the remote type... so isinstance fails. i solved the problem with builtin types, but for all other types, it's impossible. [Guido] > I know (I should, since I wrote the code :-) but I think the solution > could be much cleaner. having an __isinstance__ would be nice... that way i could do: def __isinstance__(self, types): return isinstance(self.the_real_object, types) which will ultimately solve all my problems. that, along with generic functions, really adds power to the language. for example, today's help() doesn't work well with proxies, so i could "augment" it... very nice indeed. [Phillip] > In other words, I just want to use my object with some operations that a > library provides (or requires). An "interface" is excise: something I have > to mess with that doesn't directly relate to my goals for using the library. that's exactly my point. i hate restating what i'm doing just to make some tool/compiler/framework happy. i want to express it in code, and let the tool automatically deduce that -- without me manually adding it as metadata. that's duplicating work, without adding real information to the object. it already supports __iter__, so why also derive from BaseIterable? the best example i can think of is... java: in order for a function to throw an exception, it must explicitly declare the exception type it may throw: void Foo() throws Bar { throw new Bar(); } why can't the compiler deduce that automatically? why should a human put time into even thinking about that? > Generic functions are type-safe duck typing because ducklib.quack(ob) > doesn't rely on ob having a 'quack()' method that means what you think it > means. This solves the problem simply and directly, without introducing > interfaces and ABCs and whatnot. Either the operation is defined for the > type or it isn't. i'm +1 on that, but still, as Guido (and others) said, you need a way to differentiate a sequence from a mapping, or a list from a string. dispatching based on *concrete types*, i.e., isinstance(x, dict), is bad, as it means users *must derive* from dict in order to be dispatched as *dict-like* - - - - - - types are something *about* the object, while attributes are something the object *has*. duck-typing is much more agile/flexible, as it's based on attributes -- and as it turns out -- it's much more easy to change something you *have* than something *about* you. (reminds me of The Verve's Bittersweet Symphony :) allow me to suggest a solution in the spirit of duck-typing: class ReadOnlyDict: def __getitem__... def __iter__... ... __adheres__ = ["dict"] # <<< class MyList: def __getitem__... def __iter__... ... __adheres__ = ["list"] # <<< or once we have class decorators -- @adheres("dict") class ReadOnlyDict... this way, ReadOnlyDict simply *states* it adheres to the "dict contract" (or ability), which may be something completely abstract. the *manuals* should defines this contract, not code. a type can just state it adheres to a set of contracts, and the dispatching mechanism will treat it right. def use_a_dict(obj : "dict"): ... use_a_dict( {1:2, 3:4}) # dispatches as a dict use_a_dict(ReadOnlyDict(...)) # dispatches as a dict use_a_dict(MyList(...)) # fails of course we might want to use type-objects / contract-objects instead of plain strings, but i wanted to use strings to emphasize the point -- the *contract* has nothing to do with *types* or subclassing. the contract is something the object *has* instead of something *about* the object. i can just add a new contract tomorrow, without worrying about MROs and a name being multiply defined in several base classes, and whatnot. also, the concrete type can always be a default contract, i.e., if you don't explicitly place an "__adheres__" in your class, the class itself is used instead of the __adheres__ metadata. (perhaps it should be made a magic method instead of an attribute? donno) -tomer From pje at telecommunity.com Sat Dec 2 00:11:30 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 01 Dec 2006 18:11:30 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <06Dec1.134129pst."58648"@synergy1.parc.xerox.com> References: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061201181042.028ca3f0@sparrow.telecommunity.com> At 01:41 PM 12/1/2006 -0800, Bill Janssen wrote: >And adding GFs does not give us a coherent ability to do type >introspection, which I think would help. Again, I seem to need to >look at values *returned* from calls as much as I worry about the type >of parameters passed into my functions. GFs wouldn't help me there, >would they? Why wouldn't they? From pje at telecommunity.com Sat Dec 2 01:29:47 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 01 Dec 2006 19:29:47 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> At 01:46 PM 12/1/2006 -0800, Guido van Rossum wrote: >I'm not sure what you mean this. Are you proposing that a library >function that might take a mapping would be rewritten as a generic >library function with one implementation that takes a mapping? Or are >you proposing that the library function, instead of documenting that >it takes a mapping, documents which generic functions should be >applied to it? > >Both sound like rather a big stretch from current practice. Actually, to me the thing that's a stretch from current practice is the attempt to spell out in detail what a "mapping" is. Note that in the simplest case, where you're only using getitem/setitem operations, you're already using generic functions, just ones that have syntax sugar (i.e. the [] operator). So in that sense, you can say that "mapping" means "supported by operator.getitem and operator.setitem". I'm not saying there is no such thing as "mapping", IOW, I am saying that "mapping" is an informal shorthand for a collection of operations. If you want to be specific, refer to operations. If you wish to be concise (but vague) then refer to "mapping". To put it another way, I'm against halfway measures. It bothers me that people are trying to introduce rigid interfaces, in order to address "quick-and-dirty" use cases. These things are at opposite ends of the spectrum, IMO: if you want a quick-and-dirty type test, test on the bloody concrete type! Testing some abstract interface type to soothe your OO conscience doesn't actually make the code any less rigid or dirty, it just hides the smell. I'd rather that such code continued to obviously smell, rather than pretend it's as fresh as daisies because "interfaces are better". Conversely, if you really want that bit of code to be reusable or extensible, then use a generic function, and the code will be in *fact* extensible, rather than simply pushing the compatibility problem to somebody else to figure out. Example issue: library X demands a "mapping", but really only uses getitem. Its code inspects interfaces to decide how to act on parameters, and behaves differently if it sees "a mapping". I have an object that implements getitem, and I want the special behavior, so I declare that object "a mapping". But now, library Y, that I also pass the same object to, suddenly starts behaving differently, because it sees, "ah, you're a mapping! So I'll use setitem on you..." And now I'm screwed. Things like this used to happen to me all the time in Zope 2, which would introspect methods and attributes a lot so that Zope could "decide" what to do with an object based on what it was. (And it was this experience that made me realize that inspect-and-decide is absolutely the wrong way to write composable code.) Zope 3 at first replaced this attribute-inspection with interface inspection -- with no better result! After all, using an interface as a flag is no different in essence than using a hasattr() check as a flag. As I said, it only *looks* prettier. It wasn't until interface adaptation arrived in Zope 3 that things actually improved, because (like generic functions) adaptation at least allows third-party registration. However, for interface adaptation to work well, the interfaces need to be highly context-specific, or else we just end up back at the problem where "mapping" means different things in different contexts. The PyProtocols approach to solving this was to say, "One use case = one interface", which eventually led me to realize that this generally amounts to having one or more generic functions specific to the thing you're actually trying to do. It takes a lot less time to just *explicitly* add overloads to your code for the things you want to be able to do with an object, than to have to debug the stuff that just suddenly starts happening all over the place, as can happen with interface inspection. Note that quick-and-dirty checks based on *concrete* types are actually *safer* than abstract checks or interface checks, as this generally prevents them from being used as mere behavior flags that can lead to conflicts of the type I've described. In contrast, both duck typing (ie. hasattr) checks and interface checks have proven in practice (Zope 2 and 3 respectively) to produce significant unwanted side-effects. The only way to reduce these side effects is to allow persons other than the code author to decide what should happen in a specific context. Adaptation and generic functions have this ability in common, but mere inspection (regardless of *what* is inspected) does not. > > In other words, I just want to use my object with some operations that a > > library provides (or requires). An "interface" is excise: something I have > > to mess with that doesn't directly relate to my goals for using the > library. > >I don't understand the sentence "An "interface" is excise". What does >excise mean in this context? It's an HCI buzzword meaning "work that doesn't obviously advance your goal, but that you have to do anyway because of the way the system was designed/implemented". >Regardless, I find it quite a big change from various ways of saying >"this object must have these methods (including perhaps some for which >special syntax exists, like __getattr__)" to saying "this object must >be supported by these generic functions". Well, that's why I proposed last week that we allow you to say ISomething(foo).somemethod() to be able to use such things. And, that you be allowed to define your arguments as being of type ISomething, in order to have the correct namespace automatically apply. That proposal doesn't stop you from sticking with duck typing. However, if you *want* to be explicit and extensible and pure, it allows you to do so. Meanwhile, I argue that inspection is inherently quick-and-dirty, just like duck typing. It doesn't actually improve anything, but instead makes you do more work for no new benefit, while keeping the same likelihood of unexpected behavior. What's more inspection is no different from generic functions in terms of being able to produce "spooky action at a distance". However, at least generic functions have tables whose contents can be inspected to find out all the behaviors that might result, whereas interface inspection can happen anywhere! And generic functions are actually extensible by third parties, whereas inspection is not. So: Inspection: * No way to find "spooky" actions * No way to modify broken behaviors without changing the code or trying to "trick" the inspector Generics: * "Spooky" actions are all in a table that can be dumped out and read * Broken or undesirable actions can be overridden >A method is just an >identifier in the object's attribute namespace. A generic function is >an object that may hve to be imported from elsewhere. But this is also true of interfaces, regardless of how they're defined. You can't simultaneously have "safe" duck typing and avoid the use of imports, unless you use some kind of global naming scheme, as in Java. >I object to the suggestion that seems to be implied here that in the >future we'll all be writing ducklib.quack(ob) instead of the much >simpler ob.quack(). No, I'm saying that in any case where current interface proposals would do this: if implements(ob, IDuck): ob.quack() I would argue that you are better off with *either of*: ducklib.quack(ob) OR: ob.quack() And that most code that isn't trying to be explicitly reusable would in fact use the second, "normal" syntax. (In all likelihood, the code using ducklib.quack would only be code *in* ducklib, actually.) GF's really only come into play when you want to make a library extensible or generic, like for pickling, pretty-printing, AST-visiting, etc. etc. Or, if you have some circumstance that requires you to add custom code (like the sendmail() overload example for 'str'). >I really like that when I have an object in my >hands, I don't need to import anything else in order to manipulate it, >as long as the manipulation can be done through methods. I also don't >think that the homonym problem that you (and CLOS) are trying to solve >here is all that important in practice. Homonyms aren't the problem that's being solved, it's context-specific extensibility and library composability. Those are much bigger problems than name collisions. For example, a major source of Zope 2's architectural difficulties was the direct result of using the same type of inspection that's being promoted here. (Here being the Py3K list.) I think it would be wise to learn from that experience. Yes, Zope 2 used hasattr() checks, not interface checks, but the effect is the same: people fudging what they provide in order to trick Zope into doing the right thing(s), instead of being able to just directly define the desired behavior, as they can do with adapters or GF's. Now, I suppose you could look at these things as being no worse than they are in various other languages, and that's probably true. On the other hand, you could look at how much more composable libraries are in languages with generic functions, and observe that as a general rule, such languages do not have massive, all-inclusive branded frameworks like Zope, Twisted, PEAK, etc. And the reason for that, is that in GF-based languages, *libraries are composable on a larger scale*. So, there isn't a need for single integrators to pull together any huge "all-in-one" frameworks. Of course, packaging is also a factor: Twisted, Zope, and PEAK are all-inclusive in part because the historical cost of depending on other Python packages is high. I created setuptools specifically to address that problem. But the other major factor is integration: all-inclusive frameworks work around the difficulties inherent in wrapping other packages. Many was the time in developing PEAK that I wanted to use some existing library (or even stdlib module), but couldn't because there was no way to add the necessary "glue" without awkward monkeypatching. I would thus end up rolling my own libraries more often than I wanted to. GF-based frameworks, on the other hand, don't seem like frameworks at all. They're just libraries that can be combined with other libraries. Interfaces don't give you this ability on their own, and even adaptation only gets you part of the way (and requires writing more code to do the same things). I'd like to see a Python where this type of composability falls out naturally as a side effect of using the language idiomatically. Being able to add overloads to existing functions makes it straightforward to either override behaviors or add adapters as appropriate, when gluing disparate libraries together. From pje at telecommunity.com Sat Dec 2 01:43:23 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 01 Dec 2006 19:43:23 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <1d85506f0612011402u44e79c06wc11e9c515fd6ba7f@mail.gmail.co m> References: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061201193318.02a92cb8@sparrow.telecommunity.com> At 12:02 AM 12/2/2006 +0200, tomer filiba wrote: >i'm +1 on that, but still, as Guido (and others) said, you need a way to >differentiate a sequence from a mapping, or a list from a string. dispatching >based on *concrete types*, i.e., isinstance(x, dict), is bad, as it means >users *must derive* from dict in order to be dispatched as *dict-like* No, it merely means that they must register for the desired *behavior*. If they want the same thing to happen for that type as for a dict, then let them register it explicitly. (e.g. someop[mytype] = someop[dict]) Thinking about things as being "dict-like" (and then desiring to drive if-then tests off of that) is actually the source of the *problem*, not the solution. Guido has argued that it's verbose to have to do this for lots and lots of operations, but I've not observed any situations in practice where this was actually required. What usually happens is that concrete type dispatching perfectly suffices for builtin types and types native to the library that defines the operation. Integrating that library with other types then requires registration of the few special cases that are relevant for *dispatching* (versus simply passing in objects that conform to the required duck typing). Thus, interface checking is a duck in wolf's clothing: it looks like it's an improvement over duck typing or concrete type checks, but actually isn't. In fact, it's worse because it entices people to use it because it *looks* "safer" than concrete type checking! (i.e. it appears more extensible, less error-prone, etc., when it's actually no different in these respects.) From janssen at parc.com Sat Dec 2 02:47:03 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 1 Dec 2006 17:47:03 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> Message-ID: <06Dec1.174704pst."58648"@synergy1.parc.xerox.com> > Actually, to me the thing that's a stretch from current practice is the > attempt to spell out in detail what a "mapping" is. Good point :-). Of course, that's the problem I'd like to see fixed -- not knowing what a mapping is, or whether a particular value is one. > I'm not saying there is no such thing as "mapping", IOW, I am saying that > "mapping" is an informal shorthand for a collection of operations. If you > want to be specific, refer to operations. If you wish to be concise (but > vague) then refer to "mapping". I think we're in agreement here. It's just that using interfaces lets us capture aspects of the semantics of that grouping of operations which may not be expressible in the description of the operations themselves. And using types to define interfaces allows us to re-use code in a way that's just not possible in other interface-based languages, like Java. Bill From greg.ewing at canterbury.ac.nz Sat Dec 2 02:59:39 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 02 Dec 2006 14:59:39 +1300 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> Message-ID: <4570DE0B.8040204@canterbury.ac.nz> Paul Moore wrote: > There is no expectation that all/most (or even > many!) libraries would be based on ABCs. Duck typing would remain the > norm. But even if there are only a few things that don't honour duck typing, it will have an effect on everyone, because there will always be some anxiety that you may trip over one of them. Also, if there is an officially-sanctioned formal mechanism, there's little to stop the "few" from growing into "many" over time. It'll be much harder to argue that such things shouldn't be done as a matter of course. "If it's so bad, why is there all this stuff in the language to support it?" people will ask. And Java programmers will just use it anyway without even thinking... -- Greg From greg.ewing at canterbury.ac.nz Sat Dec 2 03:24:23 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 02 Dec 2006 15:24:23 +1300 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <1d85506f0612011402u44e79c06wc11e9c515fd6ba7f@mail.gmail.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0612011402u44e79c06wc11e9c515fd6ba7f@mail.gmail.com> Message-ID: <4570E3D7.7070703@canterbury.ac.nz> tomer filiba wrote: > when a proxy is created for an object in another > process, you can only set __class__ to a *proxy* to the remote type... > so isinstance fails. But the class being passed to isinstance() can only be a proxy to the remote class as well, so as long as it's the same proxy that the overridden __class__ returns, it should work, shouldn't it? IOW, in process A: x.__class__ is X so isinstance(x, X) == True in process B: x_proxy.__class__ is X_proxy so isinstance(x_proxy, X_proxy) == True -- Greg From pje at telecommunity.com Sat Dec 2 06:46:31 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 02 Dec 2006 00:46:31 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <06Dec1.174704pst."58648"@synergy1.parc.xerox.com> References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061202003508.028ccc38@sparrow.telecommunity.com> At 05:47 PM 12/1/2006 -0800, Bill Janssen wrote: > > Actually, to me the thing that's a stretch from current practice is the > > attempt to spell out in detail what a "mapping" is. > >Good point :-). Of course, that's the problem I'd like to see fixed >-- not knowing what a mapping is, or whether a particular value is >one. Why do you *need* to know? The truth is that you don't need to know if something is "a mapping", you need to know if it does what you want it to do. And what you want a "mapping" to do may not be the same as what somebody else wants a "mapping" to do, which then leads to problems if two libraries are deciding what to do with an object based on it being "a mapping". This was the larger point of my post, actually. :) The thing you're defining as a solution, is actually a source of more problems. At least when the concept of "mapping" is expressed verbally, everybody *knows* that the term is fuzzy. Reifying it into an object doesn't actually eliminate the inherent fuzziness -- which is why I proposed a fuzzy interface implementation, btw. > > I'm not saying there is no such thing as "mapping", IOW, I am saying that > > "mapping" is an informal shorthand for a collection of operations. If you > > want to be specific, refer to operations. If you wish to be concise (but > > vague) then refer to "mapping". > >I think we're in agreement here. It's just that using interfaces lets >us capture aspects of the semantics of that grouping of operations >which may not be expressible in the description of the operations >themselves. So far the only example anyone brought up of such a concept was thread-safety, but even that only makes sense in terms of guarantees about the operations you can perform, in relation to other operations. What we're in disagreement on, is that I'm saying there is no such thing as "the people", only individual persons, while if I understand you correctly, you would make "the people" into an actual thing, rather than a verbal shorthand for referring to a group without listing its individuals. I contend that the group doesn't really exist; it's merely a convenient shorthand fiction that reflects our (fuzzy) notion of "the people", and not something that actually exists in the world. So, we should have fuzzy interfaces, that reflect our fuzzy notions of "mapping", and do not require that an object implement the whole of an interface, just like our fuzzy notion of "bird" does not require that all birds fly, even though most of them do. From talin at acm.org Sat Dec 2 08:29:39 2006 From: talin at acm.org (Talin) Date: Fri, 01 Dec 2006 23:29:39 -0800 Subject: [Python-3000] GF dispatch code in C++ Message-ID: <45712B63.1050604@acm.org> I'm not sure anyone would be interested in this or not. A few months back, I wrote a generic function dispatcher based on the Chambers/Chen paper for my embedded mini-language, Saga. It's written in C++, although it makes fairly minimal use of C++ features. This code is an independent creation from PJE's RuleDispatch, which I've never looked at. The limitations of the current code are: -- It only tests vs. types, not arbitrary predicates, although it could probably be extended to do that (as pointed out in the Chambers/Chen paper.) -- For purposes of reducing memory consumption, it only allows the dispatch to be based on the first 6 arguments or less of the function (obviously the number 6 can be adjusted via a compile-time constant.) -- It's written against the Saga code base, not Python, although it shouldn't be too hard to port one to the other. The dispatching is pretty fast. Each time a new method is added, it rebuilds a discrimination tree (which is stored in a flat array). The tree is constructed by repeatedly partitioning the set of methods by selecting some 'pivot' test against one of the method's arguments. In other words, each method has a set of 'tests' which are run against the input arguments to determine if that method should be called. For any given test, the set of all methods can be divided into three groups: -- methods which require that the test succeed -- methods which require that the test fail -- methods which don't care one way or another To prevent a combinatorial explosion of states, test cases which lead to identical partitionings are folded together. It also uses a heuristic that tries to minimize tree size by choosing a pivot test that results in the least number of 'don't care' cases. Ambiguous cases and failure cases are properly handled. In the case of an ambiguity, it attempts to find the 'most specific' method (the one whose tests are a subset of the tests of all the other methods), and if no such method can be found, it dispatches to a function that raises an 'ambiguity' exception. Anyway, if anyone would like to fool around with this code, drop me a note and I'll mail it to you. It's fairly well commented. You won't be able to compile and run it out of the box (since I'm not going to include the entire project source tree), but it could be used as a starting point for experimentation. --- And along other lines, I'd also be interested if anyone would like to take a crack at porting my Python implementation of PEP 3101 into C. That is the only thing preventing 3101 from being tested out in the real world. The python code can be found here: http://www.viridia.org/hg/python/string_format -- Talin From talin at acm.org Sat Dec 2 09:47:50 2006 From: talin at acm.org (Talin) Date: Sat, 02 Dec 2006 00:47:50 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> Message-ID: <45713DB6.2070908@acm.org> Guido van Rossum wrote: > Nevertheless, I kind of like this, and would like someone to produce a > patch for this approach. It is my opinion that without working code it > is too easy to overlook design flaws. Since you are keen to have this > in 2.6, the solution would have to be backwards compatible with 2.5, > so the patch wouldn't have to change every type initializer in the > system -- it would only have to provide the complete machinery and > convert a few example types to prove it works. I've started working on a patch, and I have a few questions. Most of the members of PyTypeObject are function pointers, and can be initialized via the method table. Here's a list of the members which aren't: PyObject_VAR_HEAD const char *tp_name; /* For printing, in format "." */ Py_ssize_t tp_basicsize, tp_itemsize; /* For allocation */ long tp_flags; const char *tp_doc; /* Documentation string */ Py_ssize_t tp_weaklistoffset; struct PyMethodDef *tp_methods; struct PyMemberDef *tp_members; struct PyGetSetDef *tp_getset; struct _typeobject *tp_base; PyObject *tp_dict; Py_ssize_t tp_dictoffset; PyObject *tp_bases; PyObject *tp_mro; /* method resolution order */ PyObject *tp_cache; PyObject *tp_subclasses; PyObject *tp_weaklist; My question is: Which of these fields need to be initialized by the user? Let's assume that we are going with the idea that types are created dynamically via a function call rather than statically declared. The function that creates a type needs to have a set of parameters that fill in the various non-method fields of a type. So each of the values listed above falls into one of three groups: 1) values which need to be parameters to the creation function, 2) values which can be filled in by the user later, and 3) values which don't need to be supplied by the user because they will be filled in automatically by the VM. I can make some guesses as to which fields fall into which group, but I'm not sure in all cases. I want to avoid having a single C function with a dozen input arguments. One other question is how to deal with initialization of types with multiple base classes. I'm not sure how the API for passing in multiple bases ought to look, and how the type object should be constructed in the case of more than one base type. -- Talin From tomerfiliba at gmail.com Sat Dec 2 10:06:30 2006 From: tomerfiliba at gmail.com (tomer filiba) Date: Sat, 2 Dec 2006 11:06:30 +0200 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <4570E3D7.7070703@canterbury.ac.nz> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0612011402u44e79c06wc11e9c515fd6ba7f@mail.gmail.com> <4570E3D7.7070703@canterbury.ac.nz> Message-ID: <1d85506f0612020106r7bebaac2idf8e3026fd333238@mail.gmail.com> no, because i use the remote isinstance. the code i gave was just a "how to". the real code would be more like: def __isinstance__(self, types): return self.sync_request("isinstance", types) and then isinstance(conn.modules.sys.path, conn.modules.__builtin__.list) # True (of course that's in RPyC notation. in RPyC you get access to the sevrer's module namespace) -tomer On 12/2/06, Greg Ewing wrote: > tomer filiba wrote: > > when a proxy is created for an object in another > > process, you can only set __class__ to a *proxy* to the remote type... > > so isinstance fails. > > But the class being passed to isinstance() can only be > a proxy to the remote class as well, so as long as it's > the same proxy that the overridden __class__ returns, > it should work, shouldn't it? > > IOW, > > in process A: > > x.__class__ is X > > so > > isinstance(x, X) == True > > in process B: > > x_proxy.__class__ is X_proxy > > so > > isinstance(x_proxy, X_proxy) == True > > -- > Greg > From fredrik at pythonware.com Sat Dec 2 13:08:56 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 02 Dec 2006 13:08:56 +0100 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> Message-ID: Guido van Rossum wrote: > Any volunteers? Without code this proposal is dead in the water. given that nobody's even bothered to look at Larry's patch for 2.X, it seems to take a bit more than just code to get something into Py3K. Talin obviously suffers from a serious case of NIH, but does that really have to apply to the entire Py3K project? From qrczak at knm.org.pl Sat Dec 2 16:59:08 2006 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Sat, 02 Dec 2006 16:59:08 +0100 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: (Guido van Rossum's message of "Tue, 28 Nov 2006 07:39:11 -0800") References: <456BFFBF.9000208@acm.org> Message-ID: <87odqmffw3.fsf@qrnik.zagroda> "Guido van Rossum" writes: > - Can't we require a C99 compiler and use C99 struct initialization? > Then the table lines could look like > > tp_new = Noddy_new, > tp_init = Noddy_init, The C99 syntax is: .tp_new = Noddy_new, .tp_init = Noddy_init, GCC also supports the syntax without periods as an extension (it was earlier than C99). I suppose some older GCC versions had supported only the syntax without periods. -- __("< Marcin Kowalczyk \__/ qrczak at knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ From fredrik at pythonware.com Sat Dec 2 17:12:47 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 02 Dec 2006 17:12:47 +0100 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) Message-ID: Marcin 'Qrczak' Kowalczyk wrote: > "Guido van Rossum" writes: > >> - Can't we require a C99 compiler and use C99 struct initialization? >> Then the table lines could look like >> >> tp_new = Noddy_new, >> tp_init = Noddy_init, > > The C99 syntax is: > > .tp_new = Noddy_new, > .tp_init = Noddy_init, I'm beginning to wonder if a preprocessor isn't the right way to get around both the syntax issues related to type objects, and also a bunch of other issues that keep cropping up if you're doing serious extension development (better code generation for ParseTuple is one of those). a preprocessor would also give us more leeway to tweak the Python/C glue layer in future releases. if done right, the preprocessor would only be required during module development; not by people building C extensions. From guido at python.org Sat Dec 2 19:11:33 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 2 Dec 2006 10:11:33 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: I'd love to see this idea explored; it sounds like it might well be the right solution. (It's also an extension of the "separate tool for conversion" idea that I floated earlier.) One requirement: it should be possible to resume development and the preprocessor at any time. I don't like tools that generate code once which you then continue to develop manually. (Numeric Python's history has a good anti-pattern example of this.) On 12/2/06, Fredrik Lundh wrote: > Marcin 'Qrczak' Kowalczyk wrote: > > > "Guido van Rossum" writes: > > > >> - Can't we require a C99 compiler and use C99 struct initialization? > >> Then the table lines could look like > >> > >> tp_new = Noddy_new, > >> tp_init = Noddy_init, > > > > The C99 syntax is: > > > > .tp_new = Noddy_new, > > .tp_init = Noddy_init, > > I'm beginning to wonder if a preprocessor isn't the right way to get > around both the syntax issues related to type objects, and also a bunch > of other issues that keep cropping up if you're doing serious extension > development (better code generation for ParseTuple is one of those). a > preprocessor would also give us more leeway to tweak the Python/C glue > layer in future releases. > > if done right, the preprocessor would only be required during module > development; not by people building C extensions. > > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From reikonmusha at gmail.com Sat Dec 2 19:56:49 2006 From: reikonmusha at gmail.com (Thomas Dixon) Date: Sat, 2 Dec 2006 13:56:49 -0500 Subject: [Python-3000] Switch/Case statement PEP Message-ID: <340FE892-D6AA-4DC8-A8A3-27C23093E59D@gmail.com> I was reading the PEP regarding the Switch/Case statement (http:// www.python.org/dev/peps/pep-3103/), and have a few thoughts about it. 1) Proposed layout Here is how I propose the 'switch' statement to be constructed. Further on I will give the reasoning behind the syntax. As you can see, I am also a fan of Alternative 1. case EXPR: on EXPR: SUITE on EXPR: SUITE continue else: SUITE 2) Statement Keywords You will notice 'switch' was replaced with what is typically found in the body of the switch statement, 'case'. Not only is it shorter, but I feel it also better describes what the statement actually does. You'll also notice that the 'case' keyword in the body was replaced with 'on'. To me, it just sounds natural. Using 'if' as some others have proposed may end up being confusing, and 'case' wasn't used in the body for obvious reasons. 3) Control flow The lone 'continue' in the example is indicative of another one of my proposals. In switch statements the 'break' used in most languages is almost completely redundant, and thus should be implemented by default. I also propose that the switch statement allows use of Python's loop control flow keywords. Which means that by default the flow doesn't 'fall through' in the absence of the 'break', but that one can force it to do so by using 'continue', as you might expect it to behave. Below I have proposed Python examples and their C equivalents to hopefully better explain what I mean. -- Example 1: Python: case EXPR: on EXPR: SUITE C: switch (EXPR) { case EXPR: SUITE break; } -- Example 2: Python: case EXPR: on EXPR: SUITE continue C: switch (EXPR) { case EXPR: SUITE } I hope I explained myself well enough, and that I haven't taken up too much of your time. Have a nice day, Thomas Dixon reikonmusha at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061202/4cc30a56/attachment.html From brett at python.org Sat Dec 2 20:21:39 2006 From: brett at python.org (Brett Cannon) Date: Sat, 2 Dec 2006 11:21:39 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: On 12/2/06, Guido van Rossum wrote: > > I'd love to see this idea explored; it sounds like it might well be > the right solution. (It's also an extension of the "separate tool for > conversion" idea that I floated earlier.) > > One requirement: it should be possible to resume development and the > preprocessor at any time. I don't like tools that generate code once > which you then continue to develop manually. (Numeric Python's history > has a good anti-pattern example of this.) So you are basically saying you want the preprocessor step to be lightweight enough that you always develop with the file before the preprocessor is run instead of with some generated file, right? Regardless, one could take a Pyrex bent on this and having Python-like method declarations but have C as the code body:: class SomeType() { def __str__(self:PyObject) -> PyObject { ... } def fxn(self:PySomeTypeObject, arg1:int, arg2:PyObject, arg3=NULL:char*) -> PyObject { ... } } That should be enough information to fill in the struct fields and auto-generate the ParseTuple() call and variable names properly. It also shouldn't be too expensive to parse and thus would allow developers to work directly with this file and not any generated C code file. Question is whether this is what you and /F had in mind and if you want to go so far as to use Python-like syntax (I only used curly braces to make it blatently obvious it was not actual Python code) or if I am misunderstanding what you two are suggesting. -Brett On 12/2/06, Fredrik Lundh wrote: > > Marcin 'Qrczak' Kowalczyk wrote: > > > > > "Guido van Rossum" writes: > > > > > >> - Can't we require a C99 compiler and use C99 struct initialization? > > >> Then the table lines could look like > > >> > > >> tp_new = Noddy_new, > > >> tp_init = Noddy_init, > > > > > > The C99 syntax is: > > > > > > .tp_new = Noddy_new, > > > .tp_init = Noddy_init, > > > > I'm beginning to wonder if a preprocessor isn't the right way to get > > around both the syntax issues related to type objects, and also a bunch > > of other issues that keep cropping up if you're doing serious extension > > development (better code generation for ParseTuple is one of those). a > > preprocessor would also give us more leeway to tweak the Python/C glue > > layer in future releases. > > > > if done right, the preprocessor would only be required during module > > development; not by people building C extensions. > > > > > > > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061202/4df6ec45/attachment.htm From talin at acm.org Sat Dec 2 20:17:10 2006 From: talin at acm.org (Talin) Date: Sat, 02 Dec 2006 11:17:10 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> Message-ID: <4571D136.2080303@acm.org> Fredrik Lundh wrote: > Guido van Rossum wrote: > >> Any volunteers? Without code this proposal is dead in the water. > > given that nobody's even bothered to look at Larry's patch for 2.X, it > seems to take a bit more than just code to get something into Py3K. > Talin obviously suffers from a serious case of NIH, but does that > really have to apply to the entire Py3K project? Bear in mind that what I am implementing is *not* my original proposal. What I am implementing is the result of the discussion here, which has diverged significantly from both your idea and mine. To summarize the proposals: -- You originally proposed an API that used numeric constants along with a per-method registration function. -- I originally proposed an API that used numeric constants stored in the PyMethodDef table. Based on the feedback that I've read here, there were concerns / criticisms of both approaches. The discussion evolved towards an approach that is table-based and uses string identifiers rather than numeric constants. When Guido said "I kind of like this", I'm presuming that this is in reference to later proposals, not earlier ones. Now, I personally don't have a problem with your way of doing it. As you pointed out, the code-size differences with functions vs. table entries are minimal, and numeric constants are faster to evaluate (although you do lose the ability to transparently promote / demote methods to 'special' status.) If the BDFL had pronounced favorably on it, I would have been happy to support it, and would have dropped the whole issue. However, that didn't happen. So I decided to create a patch based on what I think *can* be approved, although there's no guarantee that it actually will be. So I don't think it's the case that nobody's even bothered to look at Larry's patch; People have looked at the patch and suggested taking a different approach. -- Talin From talin at acm.org Sat Dec 2 20:28:07 2006 From: talin at acm.org (Talin) Date: Sat, 02 Dec 2006 11:28:07 -0800 Subject: [Python-3000] Switch/Case statement PEP In-Reply-To: <340FE892-D6AA-4DC8-A8A3-27C23093E59D@gmail.com> References: <340FE892-D6AA-4DC8-A8A3-27C23093E59D@gmail.com> Message-ID: <4571D3C7.2070602@acm.org> Thomas Dixon wrote: > I hope I explained myself well enough, and that I haven't taken up too > much of your time. Thomas, As you know from reading the PEP, there are some deep unsolved controversies regarding the design of the switch statement, which your suggestion does not address. Syntactical issues such as how the keywords should be named are a minor issue compared to this. Until those issues are resolved, the PEP is likely to remain on hold, and any suggestions as to naming and syntax will likely be regarded as a distraction to the main issue. -- Talin From fredrik at pythonware.com Sat Dec 2 21:18:24 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 02 Dec 2006 21:18:24 +0100 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: <4571D136.2080303@acm.org> References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> Message-ID: Talin wrote: > So I don't think it's the case that nobody's even bothered to look at > Larry's patch so are you basing your patch on his work? > People have looked at the patch and suggested taking a different > approach. really? I haven't seen much of a consensus for the string-literals instead of constants approach, especially not from experienced extension writers. personally, I think it's butt-ugly, a lot more error-prone than any alternative, and I also doubt that it'll save much space in practice. it also ignores history; the Xt developers tried the same thing, and ended up adding #define's for all their string literals to get a least a little help from the compiler... From fredrik at pythonware.com Sat Dec 2 21:34:38 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 02 Dec 2006 21:34:38 +0100 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: Brett Cannon wrote: > So you are basically saying you want the preprocessor step to be > lightweight enough that you always develop with the file before the > preprocessor is run instead of with some generated file, right? exactly. the developer should *never* edit generated data (whether that's a separate file fed to the compiler, an include file included by the original file, or auto-generated sections *inside* the original file). > Regardless, one could take a Pyrex bent on this and having Python-like > method declarations but have C as the code body:: (some days, I wonder if we shouldn't just include Pyrex and tell every- one to use that for *all* their extension work. Greg? how much work would it be to equip Pyrex with a "retargetable" backend?) ::: but back to the preprocessor; I think it might be possible to limit its impact on the C source file to a single #include statement at the end of the file, which includes the generated code. to provide hints for the preprocessor, Python.h would provide a bunch of C macros that are parsed by the module preprocessor, but (mostly) ignored by the C compiler. the spam module from the extension manual could then look something like #include "Python.h" static PyObject *SpamError; PYM_FUNCTION(spam)(char* command) { int sts; sts = system(command); return Py_BuildValue("i", sts); } PYM_INIT(PyObject* module) { SpamError = PyErr_NewException("spam.error", NULL, NULL); Py_INCREF(SpamError); PyModule_AddObject(module, "error", SpamError); } #include "spam.pym" and the preprocessor would then generate the necessary trampolines and tables, and of course the public initspam function. extending this to deal with optional arguments, methods, and perhaps even return values is mostly trivial. (hmm. am I reinventing SWIG here?) more later. From ntoronto at cs.byu.edu Sat Dec 2 21:58:56 2006 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Sat, 02 Dec 2006 13:58:56 -0700 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: <4571E910.70102@cs.byu.edu> Fredrik Lundh wrote: > Brett Cannon wrote: > > >> So you are basically saying you want the preprocessor step to be >> lightweight enough that you always develop with the file before the >> preprocessor is run instead of with some generated file, right? >> > > exactly. the developer should *never* edit generated data (whether > that's a separate file fed to the compiler, an include file included by > the original file, or auto-generated sections *inside* the original file). > > >> Regardless, one could take a Pyrex bent on this and having Python-like >> method declarations but have C as the code body:: >> > > (some days, I wonder if we shouldn't just include Pyrex and tell every- > one to use that for *all* their extension work. Greg? how much work > would it be to equip Pyrex with a "retargetable" backend?) > +10. This is the first thing that popped into my head when you brought up using a preprocessor. I've done extension modules with Pyrex, SWIG, and straight C, and the Pyrex ones are a whole heckuva lot easier to create than anything, and are generally more bug-free. One potential problem with this idea is that you can't drop into C code without calling an external C function, which may not be acceptable in some instances. Another is that if you want to analyze the performance of your code, you at least have to *look* at the C code it generates, which is a bit icky. I think that's pretty much going to happen no matter what though, unless the preprocessor is only a very thin wrapper around C. Neil From barry at python.org Sat Dec 2 22:08:27 2006 From: barry at python.org (Barry Warsaw) Date: Sat, 2 Dec 2006 16:08:27 -0500 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <4571E910.70102@cs.byu.edu> References: <4571E910.70102@cs.byu.edu> Message-ID: <2C6BB017-96B1-4E9D-A370-39D328DEEE7F@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 2, 2006, at 3:58 PM, Neil Toronto wrote: > One potential problem with this idea is that you can't drop into C > code > without calling an external C function, which may not be acceptable in > some instances. Another is that if you want to analyze the performance > of your code, you at least have to *look* at the C code it generates, > which is a bit icky. I think that's pretty much going to happen no > matter what though, unless the preprocessor is only a very thin > wrapper > around C. We need to keep in mind things like debugging and code discovery (IDE, tags, grep), when talking about requiring the use of a preprocessor for extensions. For an application like ours that is very heavily embedded/extended I'm concerned how difficult it will be debug, develop, and maintain these generated extensions. Generated code can be a big time saver while you'r developing the code, but eventually you have to go digging into it, then it's always way more painful. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXHrUHEjvBPtnXfVAQJdGwP/dCFh8+8PeGOdVWeDlhktIEA+ljUKePRW Bu6vbiWNkxiBBRDWyYcNJBWGhwRC/BE7aQuxGZa3s/LyMpjFpU7xY+8DxEwsORUD C9zKYp9hNN3RB+YnB4zrTMb5sNL/Xeblj33pLUpF22YV49mAayKnhI5pK7e/Z+3T VznlRMElACc= =f928 -----END PGP SIGNATURE----- From ntoronto at cs.byu.edu Sat Dec 2 22:22:58 2006 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Sat, 02 Dec 2006 14:22:58 -0700 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <2C6BB017-96B1-4E9D-A370-39D328DEEE7F@python.org> References: <4571E910.70102@cs.byu.edu> <2C6BB017-96B1-4E9D-A370-39D328DEEE7F@python.org> Message-ID: <4571EEB2.403@cs.byu.edu> Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Dec 2, 2006, at 3:58 PM, Neil Toronto wrote: > > >> One potential problem with this idea is that you can't drop into C >> code >> without calling an external C function, which may not be acceptable in >> some instances. Another is that if you want to analyze the performance >> of your code, you at least have to *look* at the C code it generates, >> which is a bit icky. I think that's pretty much going to happen no >> matter what though, unless the preprocessor is only a very thin >> wrapper >> around C. >> > > We need to keep in mind things like debugging and code discovery > (IDE, tags, grep), when talking about requiring the use of a > preprocessor for extensions. For an application like ours that is > very heavily embedded/extended I'm concerned how difficult it will be > debug, develop, and maintain these generated extensions. Generated > code can be a big time saver while you'r developing the code, but > eventually you have to go digging into it, then it's always way more > painful. > Yeah, I've had to dig into my generated code a couple of times to determine why certain string operations weren't working. As far as maintaining goes, though, I regard the C code as temporary - kind of like high-level object code. I wouldn't *want* to maintain it in my most heinous nightmares. This has already been mentioned twice, but whatever the solution is, a developer should never have to do that. A debugger that could step over Pyrex code would be wicked cool. Second to that would be Pyrex-generated code that didn't make your eyes bleed. (No offense intended, Greg. :D) That should be fairly possible. Neil From solipsis at pitrou.net Sat Dec 2 23:28:17 2006 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 02 Dec 2006 23:28:17 +0100 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <4571E910.70102@cs.byu.edu> References: <4571E910.70102@cs.byu.edu> Message-ID: <1165098497.10434.86.camel@fsol> Le samedi 02 d?cembre 2006 ? 13:58 -0700, Neil Toronto a ?crit : > One potential problem with this idea is that you can't drop into C code > without calling an external C function, which may not be acceptable in > some instances. Are you sure ? I think you could declare a C macro as an "external function" in the Pyrex code and Pyrex would probably be happy with that. (I've never tried though ;-)) Antoine. From guido at python.org Sat Dec 2 23:49:03 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 2 Dec 2006 14:49:03 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> Message-ID: My preference is for a table-based version over a "lots-of-calls" approach. I think the idea of using predefined (integer) constants instead of string constants fits fine into the table-based approach. However I'm not sure that a string-based approach is necessarily bad -- we use that now for everything that's not a special method after all. I do get the point that the string-based approach uses more memory and needs more processing time. On 12/2/06, Fredrik Lundh wrote: > Talin wrote: > > > So I don't think it's the case that nobody's even bothered to look at > > Larry's patch > > so are you basing your patch on his work? > > > People have looked at the patch and suggested taking a different > > approach. > > really? I haven't seen much of a consensus for the string-literals > instead of constants approach, especially not from experienced extension > writers. personally, I think it's butt-ugly, a lot more error-prone > than any alternative, and I also doubt that it'll save much space in > practice. it also ignores history; the Xt developers tried the same > thing, and ended up adding #define's for all their string literals to > get a least a little help from the compiler... > > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Sun Dec 3 00:04:27 2006 From: brett at python.org (Brett Cannon) Date: Sat, 2 Dec 2006 15:04:27 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: On 12/2/06, Fredrik Lundh wrote: > > Brett Cannon wrote: > > > So you are basically saying you want the preprocessor step to be > > lightweight enough that you always develop with the file before the > > preprocessor is run instead of with some generated file, right? > > exactly. the developer should *never* edit generated data (whether > that's a separate file fed to the compiler, an include file included by > the original file, or auto-generated sections *inside* the original file). > > > Regardless, one could take a Pyrex bent on this and having Python-like > > method declarations but have C as the code body:: > > (some days, I wonder if we shouldn't just include Pyrex and tell every- > one to use that for *all* their extension work. Greg? how much work > would it be to equip Pyrex with a "retargetable" backend?) > > ::: > > but back to the preprocessor; I think it might be possible to limit its > impact on the C source file to a single #include statement at the end of > the file, which includes the generated code. to provide hints for the > preprocessor, Python.h would provide a bunch of C macros that are parsed > by the module preprocessor, but (mostly) ignored by the C compiler. > the spam module from the extension manual could then look something like > > #include "Python.h" > > static PyObject *SpamError; > > PYM_FUNCTION(spam)(char* command) > { > int sts; > sts = system(command); > return Py_BuildValue("i", sts); > } > > PYM_INIT(PyObject* module) > { > SpamError = PyErr_NewException("spam.error", NULL, NULL); > Py_INCREF(SpamError); > PyModule_AddObject(module, "error", SpamError); > } > > #include "spam.pym" > > and the preprocessor would then generate the necessary trampolines and > tables, and of course the public initspam function. extending this to > deal with optional arguments, methods, and perhaps even return values is > mostly trivial. Assuming spam.pym is the generated C file that should be a good level of separation and minimize the impact of the generated code in debugging (assuming that module is not entirely object-based). (hmm. am I reinventing SWIG here?) =) Maybe. Depends on the level of hinting and syntax change you want from C code compared to the pre-processed code. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061202/d809ab65/attachment.htm From mike.klaas at gmail.com Sun Dec 3 00:09:35 2006 From: mike.klaas at gmail.com (Mike Klaas) Date: Sat, 2 Dec 2006 15:09:35 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> On 12/2/06, Fredrik Lundh wrote: > > Regardless, one could take a Pyrex bent on this and having Python-like > > method declarations but have C as the code body:: > > (some days, I wonder if we shouldn't just include Pyrex and tell every- > one to use that for *all* their extension work. Greg? how much work > would it be to equip Pyrex with a "retargetable" backend?) +1. Pyrex makes extension writing significantly less painful. Perhaps the right path would be to add a mode that de-emphasized the python-compatability aspect of pyrex (automatic python <-> c conversion points), and added support for some of the things necessary in c but currently difficult in pyrex (like calling the initialization macro of a PyCObject). Add string, list, sequence, dict types, and a few restrictions (like on overriding built-ins) and you're close to something that can generate idiomatic c. Reducing the pyrex magic and adding more detailed errors to the code generator should eliminate much of the "code-peering" that is currently necessary (at least, by me) when writing extensions. regards, -MIke From brett at python.org Sun Dec 3 00:10:27 2006 From: brett at python.org (Brett Cannon) Date: Sat, 2 Dec 2006 15:10:27 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <2C6BB017-96B1-4E9D-A370-39D328DEEE7F@python.org> References: <4571E910.70102@cs.byu.edu> <2C6BB017-96B1-4E9D-A370-39D328DEEE7F@python.org> Message-ID: On 12/2/06, Barry Warsaw wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Dec 2, 2006, at 3:58 PM, Neil Toronto wrote: > > > One potential problem with this idea is that you can't drop into C > > code > > without calling an external C function, which may not be acceptable in > > some instances. Another is that if you want to analyze the performance > > of your code, you at least have to *look* at the C code it generates, > > which is a bit icky. I think that's pretty much going to happen no > > matter what though, unless the preprocessor is only a very thin > > wrapper > > around C. > > We need to keep in mind things like debugging and code discovery > (IDE, tags, grep), when talking about requiring the use of a > preprocessor for extensions. For an application like ours that is > very heavily embedded/extended I'm concerned how difficult it will be > debug, develop, and maintain these generated extensions. Generated > code can be a big time saver while you'r developing the code, but > eventually you have to go digging into it, then it's always way more > painful. Right, but how much generated code are we talking about here? There is filling in the type struct, ParseTuple(), and the init function. All of that is boilerplate and not hard to get right in terms of auto-generation. And it also would be easy to have it be very clean to read. I know I am personally in favour of exploring no more than the three steps listed above and not going as far as a totally new syntax ala Pyrex. We don't need a DSL just for Python extensions. I can understand this worry with something like Pyrex where you completely change the syntax, but if we keep the "magic sauce" to the function declaration line then the impact shouldn't be too bad. This is especially true with something that is only going to be a prefix chunk of code for parsing arguments to your C code for methods and the auto-generated init function. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061202/7f89db0c/attachment.html From jcarlson at uci.edu Sun Dec 3 00:17:40 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 02 Dec 2006 15:17:40 -0800 Subject: [Python-3000] Switch/Case statement PEP In-Reply-To: <4571D3C7.2070602@acm.org> References: <340FE892-D6AA-4DC8-A8A3-27C23093E59D@gmail.com> <4571D3C7.2070602@acm.org> Message-ID: <20061202151503.BBB2.JCARLSON@uci.edu> Talin wrote: > Thomas Dixon wrote: > > I hope I explained myself well enough, and that I haven't taken up too > > much of your time. > > As you know from reading the PEP, there are some deep unsolved > controversies regarding the design of the switch statement, which your > suggestion does not address. Syntactical issues such as how the keywords > should be named are a minor issue compared to this. > > Until those issues are resolved, the PEP is likely to remain on hold, > and any suggestions as to naming and syntax will likely be regarded as a > distraction to the main issue. After reading Thomas' post and Talin's response, I'm going to have to agree with Talin; syntax options are moot until we can decide what is acceptable in the case EXPR. - Josiah From greg.ewing at canterbury.ac.nz Sun Dec 3 00:56:40 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Dec 2006 12:56:40 +1300 Subject: [Python-3000] Low-hanging fruit: change interpreter prompt? In-Reply-To: References: Message-ID: <457212B8.3010407@canterbury.ac.nz> Guido van Rossum wrote: > The deficiencies of email shouldn't be dictating the language. The interpreter prompt is hardly part of the language, and I don't think it would be unreasonable to make a small change here to avoid unnecessarily conflicting with a well-established convention (I wouldn't call it a "deficiency"). -- Greg From greg.ewing at canterbury.ac.nz Sun Dec 3 01:03:06 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Dec 2006 13:03:06 +1300 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <06Nov30.170806pst.58648@synergy1.parc.xerox.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <456776D6.8020909@canterbury.ac.nz> <1d85506f0611290941g1aac361ib23247058bbb8223@mail.gmail.com> <06Nov29.154543pst.58648@synergy1.parc.xerox.com> <456E2708.6070205@canterbury.ac.nz> <06Nov29.183824pst.58648@synergy1.parc.xerox.com> <456F5E4F.6090705@canterbury.ac.nz> <06Nov30.170806pst.58648@synergy1.parc.xerox.com> Message-ID: <4572143A.6060700@canterbury.ac.nz> Bill Janssen wrote: > Where would the standalone function live? How would it be associated > with whatever open() returns? Three possibilities come to mind: a) builtin b) imported from a module c) static function of the file type Since there will probably be a number of such functions (readlines, readbytes, readblocks, etc.) I'd probably vote for b) and have a module such as 'files' or 'filetools' to hold generically-implemented file-related functions. -- Greg From greg.ewing at canterbury.ac.nz Sun Dec 3 02:13:42 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Dec 2006 14:13:42 +1300 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> References: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> Message-ID: <457224C6.6040909@canterbury.ac.nz> Mike Klaas wrote: > Reducing the pyrex magic and adding more detailed errors to the code > generator should eliminate much of the "code-peering" that is > currently necessary (at least, by me) when writing extensions. Can you expand on that? What magic would you like to have removed, and what new error messages would you like to see? -- Greg From greg.ewing at canterbury.ac.nz Sun Dec 3 02:13:47 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Dec 2006 14:13:47 +1300 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <1165098497.10434.86.camel@fsol> References: <4571E910.70102@cs.byu.edu> <1165098497.10434.86.camel@fsol> Message-ID: <457224CB.3020207@canterbury.ac.nz> Antoine Pitrou wrote: > I think you could declare a C macro as an "external function" in the > Pyrex code and Pyrex would probably be happy with that. Yes, that works as long as you can declare it as having parameter and return types that make sense to Pyrex. There are a few things that can't be handled that way, such as type-polymorphic macros. I haven't come up with a good solution for those, yet. -- Greg From greg.ewing at canterbury.ac.nz Sun Dec 3 02:14:16 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 03 Dec 2006 14:14:16 +1300 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: Message-ID: <457224E8.3010204@canterbury.ac.nz> Fredrik Lundh wrote: > (some days, I wonder if we shouldn't just include Pyrex and tell every- > one to use that for *all* their extension work. Greg? how much work > would it be to equip Pyrex with a "retargetable" backend?) If you're talking about making it generate code for languages that are very different from C, or for implementations other than CPython, it would require major restructuring. There isn't currently any such thing as a front and back end, except in a very conceptual way. Code generation is done by methods spread all over the parse tree. It would all have to be rearranged to use a visitor pattern or something. -- Greg From talin at acm.org Sun Dec 3 02:59:35 2006 From: talin at acm.org (Talin) Date: Sat, 02 Dec 2006 17:59:35 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> Message-ID: <45722F87.9050903@acm.org> Guido van Rossum wrote: > My preference is for a table-based version over a "lots-of-calls" > approach. I think the idea of using predefined (integer) constants > instead of string constants fits fine into the table-based approach. > However I'm not sure that a string-based approach is necessarily bad > -- we use that now for everything that's not a special method after > all. I do get the point that the string-based approach uses more > memory and needs more processing time. OK. Its easy to implement either way. I coded up a string-based version, but if you guys want integer constants that's fine with me; probably best to let Fredrik & Larry tweak their patch rather than me continuing along this path since their stuff is further along. The two reasons why I thought strings might be good is: 1) I wanted to use the PyMethodDef table to store both special and regular methods; in other words, instead of having a special table for initializing PyTypeObject, I was going to re-use one of the already existing tables, and PyMethodDef already identifies methods via string (although you can type-pun an integer into the same slot.) 2) I wanted regular methods and special methods to be treated uniformly, at least at the API level, and let the runtime decide whether or not a given method should be "special" (in other words, have a dedicated slot in PyTypeObject.) If regular methods are registered by string, and special methods are registered by integer, then you can't really mix them; whereas if they are both registered via the same mechanism, then you can later go back and decide that a given special method doesn't really need to be special and vice versa. So for example, there's no dedicated slot for __unicode__, but if in the future performance measurement should indicate that creating such a slot would be of benefit, you could simply add a new slot for it and all existing modules, when recompiled, would automatically use the new slot. However, all that being said I recognize that string lookups aren't cheap. This is especially true in this case where almost all of the names in the table start with "__", so you have to look at the 3rd character before you detect a mismatch. What I am aiming at is for PyTypeObject to be as opaque as possible in practice, in other words initialization code shouldn't need to know much about the ordering or types of fields. This gives the developers of the Python runtime a great deal of freedom in their ability to refactor the internals of the runtime without breaking existing extension modules. > On 12/2/06, Fredrik Lundh wrote: >> Talin wrote: >> >>> So I don't think it's the case that nobody's even bothered to look at >>> Larry's patch >> so are you basing your patch on his work? >> >> > People have looked at the patch and suggested taking a different >> > approach. >> >> really? I haven't seen much of a consensus for the string-literals >> instead of constants approach, especially not from experienced extension >> writers. personally, I think it's butt-ugly, a lot more error-prone >> than any alternative, and I also doubt that it'll save much space in >> practice. it also ignores history; the Xt developers tried the same >> thing, and ended up adding #define's for all their string literals to >> get a least a little help from the compiler... >> >> From ncoghlan at gmail.com Sun Dec 3 03:13:41 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 03 Dec 2006 12:13:41 +1000 Subject: [Python-3000] Low-hanging fruit: change interpreter prompt? In-Reply-To: <457212B8.3010407@canterbury.ac.nz> References: <457212B8.3010407@canterbury.ac.nz> Message-ID: <457232D5.20709@gmail.com> Greg Ewing wrote: > Guido van Rossum wrote: >> The deficiencies of email shouldn't be dictating the language. > > The interpreter prompt is hardly part of the language, Except that the doctest module and the prevalence of interactive prompt snippets in the documentation mean that it *is* effectively part of at least the standard library, even if not the language proper. > and I don't think it would be unreasonable to make a > small change here to avoid unnecessarily conflicting > with a well-established convention (I wouldn't call > it a "deficiency"). True, but the workaround to align the two conventions if you work with email a lot (i.e., including "sys.ps1 = '.>>> '" or similar in your PYTHONSTARTUP script) is simple enough that it just doesn't seem worth the hassle of changing all the doctests and various other bits of documentation to reflect a new default interactive prompt. All a somewhat moot discussion though, given that Guido has explicitly stated that *he* wants to leave it alone, and he wins as the BDFL* :) Cheers, Nick. *Bikeshed Discussion Finishing Leader, perhaps? ;) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Sun Dec 3 03:29:39 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 2 Dec 2006 18:29:39 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: <45722F87.9050903@acm.org> References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> <45722F87.9050903@acm.org> Message-ID: On 12/2/06, Talin wrote: > Guido van Rossum wrote: > > My preference is for a table-based version over a "lots-of-calls" > > approach. I think the idea of using predefined (integer) constants > > instead of string constants fits fine into the table-based approach. > > However I'm not sure that a string-based approach is necessarily bad > > -- we use that now for everything that's not a special method after > > all. I do get the point that the string-based approach uses more > > memory and needs more processing time. > > OK. Its easy to implement either way. I coded up a string-based version, > but if you guys want integer constants that's fine with me; probably > best to let Fredrik & Larry tweak their patch rather than me continuing > along this path since their stuff is further along. > > The two reasons why I thought strings might be good is: > > 1) I wanted to use the PyMethodDef table to store both special and > regular methods; in other words, instead of having a special table for > initializing PyTypeObject, I was going to re-use one of the already > existing tables, and PyMethodDef already identifies methods via string > (although you can type-pun an integer into the same slot.) Unfortunately putting the special functions in that table doesn't really work for many of the methods that have non-standard signatures. > 2) I wanted regular methods and special methods to be treated uniformly, > at least at the API level, and let the runtime decide whether or not a > given method should be "special" (in other words, have a dedicated slot > in PyTypeObject.) But that's the thing that doesn't work. If a special function takes or returns a C int it is impossible to use it as a regular method. > If regular methods are registered by string, and > special methods are registered by integer, then you can't really mix > them; whereas if they are both registered via the same mechanism, then > you can later go back and decide that a given special method doesn't > really need to be special and vice versa. So for example, there's no > dedicated slot for __unicode__, but if in the future performance > measurement should indicate that creating such a slot would be of > benefit, you could simply add a new slot for it and all existing > modules, when recompiled, would automatically use the new slot. That applies to a very limited set of methods. > However, all that being said I recognize that string lookups aren't > cheap. This is especially true in this case where almost all of the > names in the table start with "__", so you have to look at the 3rd > character before you detect a mismatch. > > What I am aiming at is for PyTypeObject to be as opaque as possible in > practice, in other words initialization code shouldn't need to know much > about the ordering or types of fields. This gives the developers of the > Python runtime a great deal of freedom in their ability to refactor the > internals of the runtime without breaking existing extension modules. The ordering argument is also handled by using C99 initializers. The typing argument seems hard to believe -- if an extension provides a value for a field, it better have the right type. I don't see how we can hide the type or protect against the type changing, except in very limited cases where two types have the same representation. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Sun Dec 3 03:47:53 2006 From: talin at acm.org (Talin) Date: Sat, 02 Dec 2006 18:47:53 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> <45722F87.9050903@acm.org> Message-ID: <45723AD9.2010208@acm.org> Guido van Rossum wrote: > The ordering argument is also handled by using C99 initializers. Not being able to compile extension modules under Visual Studio seems like a show stopper to me. -- Talin From rasky at develer.com Sun Dec 3 15:12:03 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 3 Dec 2006 15:12:03 +0100 (CET) Subject: [Python-3000] Low-hanging fruit: change interpreter prompt? In-Reply-To: <457212B8.3010407@canterbury.ac.nz> References: <457212B8.3010407@canterbury.ac.nz> Message-ID: <1519.62.123.94.250.1165155123.squirrel@www.develer.com> On Sun, December 3, 2006 00:56, Greg Ewing wrote: >> The deficiencies of email shouldn't be dictating the language. > > The interpreter prompt is hardly part of the language, > and I don't think it would be unreasonable to make a > small change here to avoid unnecessarily conflicting > with a well-established convention (I wouldn't call > it a "deficiency"). ... and we're going to confuse beginners a lot when both 2.x and 3k are around. Giovanni Bajo From janssen at parc.com Sun Dec 3 22:53:58 2006 From: janssen at parc.com (Bill Janssen) Date: Sun, 3 Dec 2006 13:53:58 PST Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061202003508.028ccc38@sparrow.telecommunity.com> References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061202003508.028ccc38@sparrow.telecommunity.com> Message-ID: <06Dec3.135401pst."58648"@synergy1.parc.xerox.com> Phillip Eby writes: > Why do you *need* to know? The truth is that you don't need to know if > something is "a mapping", you need to know if it does what you want it to > do. And what you want a "mapping" to do may not be the same as what > somebody else wants a "mapping" to do, which then leads to problems if two > libraries are deciding what to do with an object based on it being "a mapping". That's when they'd use two different interfaces, of course. > The thing you're defining as a solution, is actually a source of more > problems. At least when the concept of "mapping" is expressed verbally, > everybody *knows* that the term is fuzzy. Reifying it into an object > doesn't actually eliminate the inherent fuzziness -- which is why I > proposed a fuzzy interface implementation, btw. Phillip, now I'm just confused. I thought you were advocating changing Python into a really functional language and adding a generic function mechanism, but now I hear you saying that Python modules should, practically speaking, always be badly documented. For instance, when I see a library function (or method) documented as foo (m, f) where "m" is a mapping and "f" is a file... I don't have a clue what I need to pass as "m" and "f". Will the function exception on me if I don't implement "values()" in "m"? Does "f" need to be seekable, or will it have to support "fileno()"? How many of the three different calling modes to "update()" does "m" have to support? Does "__setitem__" need to be supported? What I'd like to see as documentation is more along the lines of foo (types.Mapping m, (types.SeekableStream, types.BinaryOutputStream) f) => types.Integer and *no* human-readable text giving an inaccurate and incomplete description of the parameters. If I could get these kinds of assurances (on return results as well as on parameters), I think some of my need for type introspection would disappear. Someone suggested the name "contracts" for this, which I think (Eiffel aside) fits very well. It's a contract between the writer of the calling code and the writer of the code being called about what gets passed in and what gets returned. There are no pre-condition/post-condition tests in the Eiffel sense (though there may be invariants), no static type checking in the Java sense -- just a well-understood agreement. > If you want to be specific, refer to operations. I agree that in some Utopian wonderland, that would (for the most part) fix the problem. But the reality is that (a) we don't have good ways of talking about collections of operations, and (b) most programmers would rather write code than write documentation, and (c) lots of library implementors don't really know what operations are required because they themselves call poorly documented library functions. So few if any writers are actually going to *get* specific by referring to operations. That's why interface names help; they let us refer to bundles of operations, as well as any semantics for that bundle that aren't captured by the operation signatures, in a few keystrokes. And because of (b), documentation that *looks* like code is more likely to be written. > So, we should have fuzzy interfaces, that reflect our fuzzy notions of > "mapping", and do not require that an object implement the whole of an > interface, I think that's an unworkable idea, and (sorry) I think it really holds Python back. People's "fuzzy notions" of what's in a "mapping" differ; but code has a very unforgiving idea of what it wants. What we need are precise ways of talking about very exact specifications, that still work well with the fuzziness of ordinary human thought. > just like our fuzzy notion of "bird" does not require that all > birds fly, even though most of them do. That's why "big" interfaces are composed through inheritance of smaller interfaces, just as the bird class "Aves" contains more than two dozen orders (and more than 8,000 species). Bill From python3000 at davious.org Mon Dec 4 01:16:00 2006 From: python3000 at davious.org (Dave Anderson) Date: Sun, 03 Dec 2006 19:16:00 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance Message-ID: <457368C0.4010608@davious.org> Summary ======= Implementations are used to make Method Dispatching more flexible by allowing a class to claim that it acts like another class/type even though it is not derived from that class/type. Method Dispatching ================== Class A: ... # method 1 def f(arg): ... # method 2 def f(arg: A): ... method dispatching logic ------------------------ if isinstance(arg, A): [use the method 2] a = A() f(a) # method 2 is used *implements* keyword: A simple Implementation claims it acts like a class/type ======================================================== class B: implements A ... method dispatching logic now becomes ------------------------------------ if arg.does_implement(A): [use method 2] b = B() f(b) # method 2 is used note, isinstance(o, T) -> True will always guarantee o.does_implement(T) -> True An Implementation can be Method-scoped ====================================== class Filelike: implements file.save def save(self): .... def f(file_like_arg: file.save): # will dispatch file and Filelike instances ... An Implementation can be a combination of Classes and Class Methods =================================================================== A_B_C_like = Implementation(A, B, C.f, C.g) class D: implements A_B_C_like ... def f(A_B_C_like_arg: A_B_C_like): # will dispatch D instances ... informal implementation lists are also allowed --------------------------------------------- class D1: implements A, B, C.f, C.g ... def f(A_B_C_like_arg: A, B, C.f, C.g): # will dispatch D and D1 instances ... multiple implementation declarations are also allowed ----------------------------------------------------- class D2: implements A, B implements C.f, C.g def f(A_B_C_like_arg: A, B, C.f, C.g): # will dispatch D, D1, and D2 instances *__adapt__* special method: An Implementation can be tied to a special transformation method ================================================================ class E: implements A def __adapt__(self: A): ... transformation statements ... return [transformed E instance] When a E instance is dispatched as a A instance, __adapt__ will used to generate a transformed instance A default method can be overridden by a type specific method ============================================================ class F implements A.f def f(self): ... def f(self: A): ... When an F instance is dispatched as an A instance, a call to f will use the second method *employs* keyword: Methods can be implemented by borrowing methods directly ========================================================= class G: employs A, B.f, B.g Class G will now use all of A's methods and method f and g of B def f(c_var: A) # will dispatch A and G instances def g(c_var: B.f, B.g) # will dispatch B and G instances Ad Hoc and Runtime Implementation/Employment Declarations ========================================================= class H: ... def adaptation_method(self): ... return [transformed self] H.implements(A, B, C.f) H.adapt(D, adaptation_method) H.employs(E) h = H() h.employs(F) # object h (not class H) implements F with F's methods Implementing non-class-specific methods using special Generic type ================================================================== class I: def f(self): ... def g(self): ... I.does_implement(Generic.i, Generic.g) -> True Implements supports many related conventions ============================================ Class/type-vouching ------------------- Supported with the implements keyword Duck-typing ----------- Supported with method-scope implementation units and, if desired or needed, Generic methods Abstract Interfaces ------------------- Supported by implementing classes with empty method definitions Homonym Method Resolution ------------------------- Resolved by overloading self in method definitions Adaptation ---------- Supported with implements and the __adapt__ special method (Informal) Mix-ins ------------------ Supported using the employs keyword Interesting Examples for Clarification ====================================== In order to be treated as a type/class, a type/class (versus method-unit) implements or employs declaration must be used -------------------------------------------------------------- class A: def f(self): ... def g(self): ... class B: implements A.f, A.g ... B.does_implement(A) => False That is to say, if you want the fine-grain level that duck-typing provides, you must be explicit -------------------------------------------------- a_methods_implementation = Implementation(A.f, A.g) B.does_implement(a_methods_implementation) => True there could be an Object-level property that collects an implementation list based on all the classes methods... B.does_implement(A.duck_implementation) => True there could be an Object-level property that collects an implementation list in terms of Generic methods B.does_implement(A.generic_implementation) => True An Implementation declaration does not affect isinstance -------------------------------------------------------- class I: implements A ... i = I() isinstance(i, A) -> False i.does_implement(A) -> True Implementation declarations are inherited like classes, but through a separate inheritance chain ------------------------------------------------------- class J: implements A ... class K(J): ... K.does_implement(A) -> True Undeclared method assignments aren't added to a class's implementation list automatically ---------------------------------------------- class L: f = B.f L.does_implement(B.f) -> False class M: employs B.f M.does_implement(B.f) -> True From janssen at parc.com Mon Dec 4 01:54:20 2006 From: janssen at parc.com (Bill Janssen) Date: Sun, 3 Dec 2006 16:54:20 PST Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <457368C0.4010608@davious.org> References: <457368C0.4010608@davious.org> Message-ID: <06Dec3.165426pst."58648"@synergy1.parc.xerox.com> Hi, Dave. > Implementations are used to make Method Dispatching more flexible by > allowing a class to claim that it acts like another class/type even > though it is not derived from that class/type. I'm not sure just what this adds over the simple "change_class" function that I've posted twice already. Is there an implementation efficiency argument here? What happens when you call an existing method from one of the (supertypes?) of an implementation? Do you have to re-implement it? Or do you have some way of coercing the value to that type? Bill From jimjjewett at gmail.com Mon Dec 4 02:12:19 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Sun, 3 Dec 2006 20:12:19 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <942744542556869806@unknownmsgid> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061202003508.028ccc38@sparrow.telecommunity.com> <942744542556869806@unknownmsgid> Message-ID: On 12/3/06, Bill Janssen wrote: > Phillip Eby writes: > > what you want a "mapping" to do may not be the same as what > > somebody else wants a "mapping" to do > That's when they'd use two different interfaces, of course. I think Phillip's point is that most people wouldn't, really. You're lucky if someone documents "mapping" rather than "dict". They are almost never more specific. After interfaces, some people probably still won't be more specfic. They might say "MinimalMapping" if it is in the standard library, but they won't (and probably *shouldn't*, as a matter of space) document each particular method. Someone else will come along, peek at the code, and say "They want a mapping, but don't use method X, so I can cheat." And they will, but they'll still *claim* to implement mapping, because that is what the library asks for. The end result is just as much incompatibility as today, but now the library writer can get all huffy about how it was documented, careful writers will be more likely to fill out junk methods, and there will be more boilerplate. I'm not sure these problems can't be worked around, but ... I'm not sure how, particularly without supplying default implementations right on the interface. Simply allowing people to declare an interface probably isn't enough to gain the full benefits, though it might be enough to incur the costs. > What I'd like to see as documentation is more along the lines of > foo (types.Mapping m, (types.SeekableStream, types.BinaryOutputStream) f) => types.Integer That starts to get long enough that people skip it as boilerplate. > and *no* human-readable text giving an inaccurate and incomplete > description of the parameters. But will they really use the minimally correct interface, or will they round up? -jJ From pje at telecommunity.com Mon Dec 4 02:43:44 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sun, 03 Dec 2006 20:43:44 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <942744542556869806@unknownmsgid> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061202003508.028ccc38@sparrow.telecommunity.com> <942744542556869806@unknownmsgid> Message-ID: <5.1.1.6.0.20061203202240.027813a8@sparrow.telecommunity.com> At 08:12 PM 12/3/2006 -0500, Jim Jewett wrote: >On 12/3/06, Bill Janssen wrote: > > Phillip Eby writes: > > > what you want a "mapping" to do may not be the same as what > > > somebody else wants a "mapping" to do > > > That's when they'd use two different interfaces, of course. > >I think Phillip's point is that most people wouldn't, really. You're >lucky if someone documents "mapping" rather than "dict". They are >almost never more specific. Exactly. I'm glad I decided to wait before answering this one, because you nailed it all the way through in your reply. These are all 100% HCI issues, not technical ones. In technical theory, there's absolutely nothing wrong with inspecting interfaces. In practice, there is an awful lot that goes wrong. Alas, there's nothing sadder than a beautiful theory being ruined by an ugly fact, but declaring and inspecting rigid interfaces (as opposed to say, typeclasses), is just such a ruined theory. (Well, actually, interfaces aren't even that beautiful of a theory. Typeclasses don't *declare* anything, they are just rules that determine whether a type qualifies as a member of the typeclass. So they follow the DRY (Don't Repeat Yourself) principle, because you don't have to declare *twice* that you support something, as you do with interfaces. With typeclasses, you can at least only lie about what the *body* of your methods do, not whether you've implemented them. And you get the benefit of default implementations, too.) >I'm not sure these problems can't be worked around, but ... I'm not >sure how, particularly without supplying default implementations right >on the interface. Yep - this is the typeclass solution, and IMO it's ideal for supporting "wide" interfaces like "mapping" and "file". Really, though, if I understand Guido's position correctly, the sticking point re: generic functions and typeclasses is that he doesn't like having either IFace.func(ob) or IFace(ob).func(), and is also opposed to having 'def something(arg:IFace):' automatically adapt 'arg' to IFace so you don't have to use one of the other two syntaxes. Of course, I've already said that ob.func() suffices in cases where you are passing something that is *already* of appropriate type. That is, if your caller invokes 'something(IFace(arg))' or just uses an object known to conform to the desired interface, there's no need for the callee to adapt or use a standalone function. But I think that Guido finds this part somehow unsatisfying, because it lacks the ability to perform a quick-and-dirty test on the interface (instead of the *type*). Which is why I'm trying to highlight the fact that interface *testing* (as opposed to adapting) leads to practical problems that concrete type testing does *not* -- especially if the concrete type testing is done via overloads instead of "if" blocks. From python3000 at davious.org Mon Dec 4 03:43:29 2006 From: python3000 at davious.org (Dave Anderson) Date: Sun, 03 Dec 2006 21:43:29 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <457368C0.4010608@davious.org> References: <457368C0.4010608@davious.org> Message-ID: <45738B51.6000202@davious.org> I'd like to augment one of the sections to my previous post... was: An Implementation can be a combination of Classes and Class Methods I meant to say: An Implementation can be a combination of Classes, Class Methods, and Other Implementations An Implementation can be a combination of Classes, Class Methods, and Other Implementations ================================================= A_B_C_like = Implementation(A, B, C.f, C.g) class D: implements A_B_C_like ... def f(A_B_C_like_arg: A_B_C_like): # will dispatch D instances ... implementations can be composed of other implementations -------------------------------------------------------- A_B = Implemenation(A, B) A_B_C_like = Implementation(A_B, C.f, C.g) informal implementation lists are also allowed --------------------------------------------- class D1: implements A, B, C.f, C.g ... def f(A_B_C_like_arg: A, B, C.f, C.g): # will dispatch D and D1 instances ... multiple implementation declarations are also allowed ----------------------------------------------------- class D2: implements A, B implements C.f, C.g def f(A_B_C_like_arg: A, B, C.f, C.g): # will dispatch D, D1, and D2 instances .... From rrr at ronadam.com Mon Dec 4 05:14:40 2006 From: rrr at ronadam.com (Ron Adam) Date: Sun, 03 Dec 2006 22:14:40 -0600 Subject: [Python-3000] Low-hanging fruit: change interpreter prompt? In-Reply-To: <457232D5.20709@gmail.com> References: <457212B8.3010407@canterbury.ac.nz> <457232D5.20709@gmail.com> Message-ID: Nick Coghlan wrote: > Greg Ewing wrote: >> Guido van Rossum wrote: >>> The deficiencies of email shouldn't be dictating the language. >> The interpreter prompt is hardly part of the language, > > Except that the doctest module and the prevalence of interactive prompt > snippets in the documentation mean that it *is* effectively part of at least > the standard library, even if not the language proper. > >> and I don't think it would be unreasonable to make a >> small change here to avoid unnecessarily conflicting >> with a well-established convention (I wouldn't call >> it a "deficiency"). > > True, but the workaround to align the two conventions if you work with email a > lot (i.e., including "sys.ps1 = '.>>> '" or similar in your PYTHONSTARTUP > script) is simple enough that it just doesn't seem worth the hassle of > changing all the doctests and various other bits of documentation to reflect a > new default interactive prompt. > > All a somewhat moot discussion though, given that Guido has explicitly stated > that *he* wants to leave it alone, and he wins as the BDFL* :) > > Cheers, > Nick. > > *Bikeshed Discussion Finishing Leader, perhaps? ;) Well, maybe finished. I agree with all of the above. But on a some what related note, should it be part of the language in a more formal way. Maybe even go as far as allowing the prompts to be in eval and exec function code strings. It may also be nice if I could cut command line example code (including the prompts) and paste it back into the console (and idle) and just have it work. Or cut from one console window and past it into another and have it work. Would it be doable or too messy to bother with? Cheers, Ron From nnorwitz at gmail.com Mon Dec 4 06:45:21 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Sun, 3 Dec 2006 21:45:21 -0800 Subject: [Python-3000] requiring parens around singleton tuples Message-ID: In looking at this and some other stuff, I thought about this code: x = 1, How about requiring parens around (1,)? It kinda sucks to make it inconsistent by only requiring parens for singleton tuples. But it might prevent more errors and make it much harder to miss the comma. I got this idea while reading about requiring parens in list comps (from 3100): This will make list comprehensions more similar to generator comprehensions. [x for x in 1, 2] will need to be: [x for x in (1, 2)] This would apply to x, = 1 as well (ie, both LHS and RHS). Parens are already required for the empty tuple, this would extend it slightly. I'm not suggestion parens be required for creating/unpacking tuples that have 2 or more elements. n From steven.bethard at gmail.com Mon Dec 4 07:01:05 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 3 Dec 2006 23:01:05 -0700 Subject: [Python-3000] requiring parens around singleton tuples In-Reply-To: References: Message-ID: On 12/3/06, Neal Norwitz wrote: > In looking at this and some other stuff, I thought about this code: > x = 1, > How about requiring parens around (1,)? [snip] > This would apply to x, = 1 as well (ie, both LHS and RHS). FWLIW, I use the latter all the time and would greatly miss it. I can't remember the last time I had a bug from it, though I can certainly believe that they happen. I avoid the ``(x,)`` thing like the plague -- too much punctuation for me -- but if the single comma form goes away, I guess I could still be happy with ``[x]`` for unpacking. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From p.f.moore at gmail.com Mon Dec 4 10:10:44 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 4 Dec 2006 09:10:44 +0000 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061202003508.028ccc38@sparrow.telecommunity.com> <942744542556869806@unknownmsgid> Message-ID: <79990c6b0612040110t23ea7ddw3b2a6c849c6646c8@mail.gmail.com> On 12/4/06, Jim Jewett wrote: > On 12/3/06, Bill Janssen wrote: [Skillped Jim's point that I'm 100% in agreement with, about what people will end up doing in practice...] > > What I'd like to see as documentation is more along the lines of > > > foo (types.Mapping m, (types.SeekableStream, types.BinaryOutputStream) f) => types.Integer > > That starts to get long enough that people skip it as boilerplate. > > > and *no* human-readable text giving an inaccurate and incomplete > > description of the parameters. > > But will they really use the minimally correct interface, or will they round up? A convention I have seen used successfully in C++ templates (where duck typing is used) is to use human-readable text which is *not* "accurate and incomplete", by saying things like: foo(m, f) Here, the following operations on m are required: ret = m[k] for any hashable k m[k] = val for any hashable k m.keys() returnung a list For f, required operations are... This is relatively easy to document, just by looking at the code, as well as being 100% precise. It omits certain "obvious" constraint requirements such as k in m.keys() implies m[k] won't fail, but we're supposed to all be adults here, and not be *trying* to break each other's code :-) Paul. From ben at 666.com Mon Dec 4 11:10:05 2006 From: ben at 666.com (Ben Wing) Date: Mon, 04 Dec 2006 04:10:05 -0600 Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] Message-ID: <4573F3FD.1080800@666.com> on suggestion of someone on python-dev, reposted on python-3000. hope that's ok. ben -------- Original Message -------- Subject: features i'd like [Python 3000] ... #3: fix super() Date: Sun, 03 Dec 2006 21:06:07 -0600 From: Ben Wing To: python-dev at python.org i don't like the current super() at all. having to type super(Foo, self).__init__(...) is messy, hard to remember, and error-prone. it also introduces an unfortunate dependency in that the name of the class (Foo) has to be hard-coded in the call, and if you change the class name you also have to go and change all super() calls -- more chances for errors. as a result, i imagine there's a strong urge to just hardcode the name of the parent -- a bad idea, and the reason why super() was introduced in the first place. instead, `super' should work like in other languages. a couple of ideas: -- super.meth(args) calls the superclass method `meth', in the same way that super(Foo, self).meth(args) currently works. (this is the same syntax as in java. this probably requires making `super' be a keyword, although it might be possible to hack this by examining the current call stack.) -- as an alternative or in addition, super(args) would work like super.meth(args) if we're currently inside of `meth' in a subclass. i suspect that 90% of the time or more, `super' is used to chain to the superclass version of an overridden method, and this optimizes for the common case. it makes for one less possibility of misspelling `__init__' and one less thing that requires renaming if a method is renamed or code copied to another method (granted, this isn't such a big deal as in the case of class renaming). if the form `super.meth(args)' isn't also supported, then a call of this sort would have to be made through the introspection API. (i think it would be a good idea to have this functionality available through the introspection API, in any case. currently i don't see any obvious way in module `inspect' to find out which class a call to `super(...).meth(...)' is invoked on, although maybe a loop with inspect.getmro() and hasattr() would work. it would be nice to have a function like inspect.getsuper(class, 'meth'), i think.) ben From ben at 666.com Mon Dec 4 11:08:58 2006 From: ben at 666.com (Ben Wing) Date: Mon, 04 Dec 2006 04:08:58 -0600 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl Message-ID: <4573F3BA.80603@666.com> i'd already posted this to python-dev, but someone suggested is would belong more on python-3000, so i'm posting it here. hope that's ok. i see in PEP 3101 that there's some work going on to fix up the string formatting capabilities of python. it looks good to me but it still doesn't really address the lack of a simple interpolated string mechanism, as in perl or ruby. i find myself constantly writing stuff like text="Family: %s" % self.name maybe_errout("%s, line %s: %s\n" % (title, lineno, errstr)) def __str__(self): return "CCGFeatval(%s, parents=%s, licensing=%s)" % ( (self.name, self.parents, self.licensing)) and lots of similar examples that are just crying out for perl-style variable interpolation. the proposals in PEP 3101 don't help much; i'd get instead something like maybe_errout("{0}, line {1}: {2}\n".format(title, lineno, errstr)) which isn't any better than the current % notation, or something like maybe_errout("{title}, line {lineno}: {errstr}\n".format( title=title, lineno=lineno, errstr=errstr)) where i have to repeat each interpolated variable three times. yuck yuck yuck. how about something nice like maybe_errout(i"[title], line [lineno]: [errstr]\n") or (with the first example above) text=i"Family: [self.name]" or (third example above) def __str__(self): return i"CCGFeatval([self.name], parents=[self.parents], licensing=[self.licensing])" (that should be one line if it gets broken) the advantage of these is the same as for all such interpolations: the interpolated variable is logically placed exactly where it will be substituted, rather than somewhere else, with the brain needing to do some additional cross-referencing. `i' in front of a string indicates an "interpolated" string just like `r' in the same position means "raw string". if you think this is too invisible, you could maybe use `in' or something easier to see. however, i could see a case being made for combining both `i' and `r' on the same string, and so using a single letter seems to make the most sense. formatting params can follow, e.g. print i"The value of [stock[x]] is [stockval[x]:%6.2f]" some comments: 1. i suggest brackets here so as to parallel but not interfere with PEP 3101, which uses braces; PEP 3101 is somewhat orthogonal to this proposal and the two might want to coexist. i think using something like brackets or braces is better than perl's $ convention (it explicitly indicates the boundaries of the interpolation) and simpler than ruby's #{...} or make's ${...}; since strings are only interpolated if you specifically indicate this, there's less need to use funny characters to avoid accidental interpolation. 2. as shown in the last example, format specifiers can be added to an interpolation, as in PEP 3101. maybe the % is unnecessary. 3. as shown in various examples, things other than just straight variables can be interpolated. the above examples include array/hash references and object attributes. it's not exactly clear what should and shouldn't be allowed. one possibility is just to allow an arbitrary expression. PEP 3101 isn't going to do this because it's working on existing strings (including ones that may come from the user), and so allowing arbitrary expressions could lead to security holes. but here, we're talking about immediate strings, so security holes of this sort should not a concern. 4. the semantics of an interpolated string are exactly that of a series of string concatenations, e.g. return i"CCGFeatval([self.name], parents=[self.parents], licensing=[self.licensing])" (should be one line) is equivalent to return "CCGFeatval(" + self.name + ", parents=" + self.parents + \ ", licensing=" + self.licensing + ")" ben From ncoghlan at gmail.com Mon Dec 4 13:13:58 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 04 Dec 2006 22:13:58 +1000 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <4573F3BA.80603@666.com> References: <4573F3BA.80603@666.com> Message-ID: <45741106.80609@gmail.com> Ben Wing wrote: > i see in PEP 3101 that there's some work going on to fix up the string > formatting capabilities of python. it looks good to me but it still > doesn't really address the lack of a simple interpolated string > mechanism, as in perl or ruby. i find myself constantly writing stuff like > > text="Family: %s" % self.name > > maybe_errout("%s, line %s: %s\n" % (title, lineno, errstr)) > > def __str__(self): > return "CCGFeatval(%s, parents=%s, licensing=%s)" % ( > (self.name, self.parents, self.licensing)) > > and lots of similar examples that are just crying out for perl-style > variable interpolation. the proposals in PEP 3101 don't help much; Have another look at PEP 3101. It already allows your examples to be written as follows: text = "Family: {0.name}".format(self) maybe_errout("{title}, line {lineno}: {errstr}\n".format(**locals())) return ("CCGFeatval({0.name}, " "parents={0.parents}, " "licensing={0.licensing})").format(self) That said, it's conceivable that a separate function could be written that pokes around in the caller's locals instead of accepting a namespace argument: text = interp("Family: {self.name}") maybe_errout(interp("{title}, line {lineno}: {errstr}\n")) return interp(("CCGFeatval({self.name}, " "parents={self.parents}, " "licensing={self.licensing})")) I'm not entirely averse to that idea, even though poking around in the frame stack is always somewhat dubious. I don't recall it coming up directly in previous discussions (certainly PEP 3101 doesn't say anything that explicitly rejects the concept). OTOH, given a string formatting mechanism that accepts names and a mapping (like both %-formatting and PEP 3101), it's pretty easy to write your own implicit interpolation function: >>> def interp(fmt): ... caller = sys._getframe(1) ... return fmt % caller.f_locals ... >>> x = 1 >>> interp("%(x)0.2f") '1.00' Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From barry at python.org Mon Dec 4 14:19:46 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 4 Dec 2006 08:19:46 -0500 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <4573F3BA.80603@666.com> References: <4573F3BA.80603@666.com> Message-ID: <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 4, 2006, at 5:08 AM, Ben Wing wrote: > i see in PEP 3101 that there's some work going on to fix up the string > formatting capabilities of python. it looks good to me but it still > doesn't really address the lack of a simple interpolated string > mechanism, as in perl or ruby. [...] > > maybe_errout("{title}, line {lineno}: {errstr}\n".format( > title=title, lineno=lineno, errstr=errstr)) I don't have time atm, to go into a lot of detail, but I'll say that this latter is exactly what motivated me to add this functionality in Mailman, hooking into the _() function for translation string marking. It's actually a fairly simple thing to implement once you have sys._getframe() (can you guess what motivated me to add that too? :). The implementation in Mailman 2.1 still uses %-strings by default, but Mailman 2.2 will use PEP 292 $-strings by default. Given an implementation of PEP 3101, you could easily make the interpolation use that instead. After several years of use, I'm strongly +1 for this feature in some form or another. Whether it needs to be part of the language or can be provided in a library module, I'm not so sure about. Certainly for Mailman it works fine as a library module because any string that's going to have this interpolation feature is going to be translatable and thus hookable with uber-_(). I tend to think that with the switch to functional print, you'd have a convenient place to hook into with a library somehow. There should be little security concern about feature specifically, but you do need to be aware of an issue once you start hooking into catalogs. It's possible that a nefarious or sloppy translator could expose internal variables if the translated string used an "unsafe" local instead of the "safe" variety. For example, it's often the case that in a web app you'll have one variable which is a user input string from a form, and then another variable which is just the escaped version of the same string. You'll use the latter in your translatable string to avoid XSS and the like, but the unsafe version is still of course available as a local. If the translation used the wrong one, you'd have an XSS vector (but only in the affected language). It might also be handy to allow {name.attr} access in the interpolated string. I know it has been for me occasionally. Anyway, all the parts exist to do this, so I think it's just a matter of deciding whether this should be a language feature or library feature. I tend to think the latter is more appropriate given where Py3k is already headed. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXQgd3EjvBPtnXfVAQKSaQP/fD1bdfLRnxxeXxAYal6QbLH1nn7fJRtK wr55zVw8NyQ2qVeIMF3agd+jsWAPWy1TTj0eR9JBR8N7tbjXY2zauQIpjDlUBXpK 8XucaxYysmjcvz2DhPUUCGHfLzv5Sh9T6oCCno/ja4WcWwCN960wQpgIETIBS2pe 56GHa92876E= =dQB8 -----END PGP SIGNATURE----- From talin at acm.org Mon Dec 4 18:00:32 2006 From: talin at acm.org (Talin) Date: Mon, 04 Dec 2006 09:00:32 -0800 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <45741106.80609@gmail.com> References: <4573F3BA.80603@666.com> <45741106.80609@gmail.com> Message-ID: <45745430.5040605@acm.org> Nick Coghlan wrote: > I'm not entirely averse to that idea, even though poking around in the frame > stack is always somewhat dubious. I don't recall it coming up directly in > previous discussions (certainly PEP 3101 doesn't say anything that explicitly > rejects the concept). PEP 3101 had addressed this in an earlier draft, but it was dropped mainly due to the fact that the PEP was already long and complex enough, and because some folks had some issues with the idea. In the spirit of "pick your battles", I decided to leave that particular battle for someone else to fight. -- Talin From jimjjewett at gmail.com Mon Dec 4 18:16:19 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 4 Dec 2006 12:16:19 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <-3967619051774595947@unknownmsgid> References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> Message-ID: On 12/3/06, Bill Janssen wrote: > > Implementations are used to make Method Dispatching more flexible by > > allowing a class to claim that it acts like another class/type even > > though it is not derived from that class/type. > I'm not sure just what this adds over the simple "change_class" > function that I've posted twice already. Is there an implementation > efficiency argument here? (1) Interfaces can be placed on a separate (presumably light-weight) inheritance hierarchy. (2) There is are efficiency (and security) concerns with changing bases in an arbitrary manner. Changing them only at the back (where they cannot hide existing attributes) may be safer and easier. (I'm not certain of this; but I'm not ready to rule it out either.) -jJ From janssen at parc.com Mon Dec 4 22:28:11 2006 From: janssen at parc.com (Bill Janssen) Date: Mon, 4 Dec 2006 13:28:11 PST Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> Message-ID: <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> Jim Jewett writes: > On 12/3/06, Bill Janssen wrote: > > > Implementations are used to make Method Dispatching more flexible by > > > allowing a class to claim that it acts like another class/type even > > > though it is not derived from that class/type. > > > I'm not sure just what this adds over the simple "change_class" > > function that I've posted twice already. Is there an implementation > > efficiency argument here? > > (1) Interfaces can be placed on a separate (presumably light-weight) > inheritance hierarchy. I know it looks this way, but I think you'd just be changing one inheritance mechanism for another. No one would use the one that's already there; they'd just use this one instead. And I don't really see how it's lighter-weight -- can you explain that? > (2) There is are efficiency (and security) concerns with changing > bases in an arbitrary manner. Changing them only at the back (where > they cannot hide existing attributes) may be safer and easier. (I'm > not certain of this; but I'm not ready to rule it out either.) I'm not sure how this is supposed to work, though, without changing bases. I started to write up a sentence much like you wrote above, then didn't, because I couldn't see how you could do this without changing bases. But Dave has still only provided a sketch. He hasn't explained how it works yet, so we could both be right :-). Bill From python at zesty.ca Tue Dec 5 06:09:25 2006 From: python at zesty.ca (Ka-Ping Yee) Date: Mon, 4 Dec 2006 23:09:25 -0600 (CST) Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <4573F3BA.80603@666.com> References: <4573F3BA.80603@666.com> Message-ID: On Mon, 4 Dec 2006, Ben Wing wrote: > how about something nice like > > maybe_errout(i"[title], line [lineno]: [errstr]\n") See PEP 215. http://www.python.org/dev/peps/pep-0215/ -- ?!ng From mike.klaas at gmail.com Tue Dec 5 06:32:45 2006 From: mike.klaas at gmail.com (Mike Klaas) Date: Mon, 4 Dec 2006 21:32:45 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <457224C6.6040909@canterbury.ac.nz> References: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> <457224C6.6040909@canterbury.ac.nz> Message-ID: <3d2ce8cb0612042132h2aa1572do633e81d7dcfed164@mail.gmail.com> On 12/2/06, Greg Ewing wrote: > Mike Klaas wrote: > > > Reducing the pyrex magic and adding more detailed errors to the code > > generator should eliminate much of the "code-peering" that is > > currently necessary (at least, by me) when writing extensions. > > Can you expand on that? What magic would you like to > have removed, and what new error messages would you > like to see? Sure. The "magic" (and I don't mean that derogatorily) I'm talking about is the Python <-> c type conversion and assumption that all undeclared variables are python objects. I don't think that they should be removed (as having python-compatibility is great), but I find them an impediment when I'm trying to write mostly c code rather than mostly python, and a pyrex mode for extension writing might consider dropping them. It is very easy to forget a cdef, or make a typo in a variable name and have code that still works (due to pyrex's autoconversion) but operates inefficiently (a loop counter being converted back and forth every iteration, for instance). The problem is that variable declaration in pyrex means "this is special", but when you're programming in c, ctypes are not special but the norm. I would be happier to see "error: 'count' undeclared" rather it being a PyObject. Autoconversion is actually less a problem when declarations are required (not to mention ridiculously handy). As for errors, I remember being confused by some the complaints in parameter list declaration, but I can't think of specific instances at the moment. Most of my points of greatest frustration have been when I'm trying to produce a specific line or two of c code, and it becomes a question of trying to trick pyrex into generating it (for instance, using certain types of macros, or trying to pass a PyObject**). You could (safely) argue that I'm using the wrong tool if I'm trying to write too much c in pyrex, but extension writers will need to be able to do the same things. I'd like to postfix this all by mentioning that I think that pyrex is absolutely wonderful and has _huge_ potential. I build large distributed systems with python, and I am always impressed how little of the systems need to be written in c or pyrex to achieve massive increases in performance. Ever consider opening up the project to other committers and becoming the BDFL of pyrex? -Mike From guido at python.org Tue Dec 5 16:59:32 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 5 Dec 2006 09:59:32 -0600 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> Message-ID: Phillip, I have only a little time while I'm sitting in a hotel lobby, but I printed this email out and read it several times over on the plane. It seems as if you purposely ignore a point that I've brought up several times now, without *ever* getting any kind of response from you. I think I first discussed this in the sets example. I mentioned it again in the post to which you're responding below -- but you seem to have cut out those parts and ignored them. My point is that an interface can *document* (at least in English) a "contract" about the invariants between operations. While I'm not into enforcing or verifying such contracts, I'm very interested in documenting them. For example, something that has "mapping" behavior has a very different relationship between x[y] and "y in x" than something that has "sequence" behavior. If you see interfaces as *just* combinations of operations there is no way to put this information. So this is why I keep coming back to interfaces (or ABCs; the difference isn't too important to me, though it seems to be to others and if their argument is convincing enough I'd be happy to accommodate them). Note that I'm not against GFs; I just want interfaces or ABCs too. --Guido On 12/1/06, Phillip J. Eby wrote: > At 01:46 PM 12/1/2006 -0800, Guido van Rossum wrote: > >I'm not sure what you mean this. Are you proposing that a library > >function that might take a mapping would be rewritten as a generic > >library function with one implementation that takes a mapping? Or are > >you proposing that the library function, instead of documenting that > >it takes a mapping, documents which generic functions should be > >applied to it? > > > >Both sound like rather a big stretch from current practice. > > Actually, to me the thing that's a stretch from current practice is the > attempt to spell out in detail what a "mapping" is. > > Note that in the simplest case, where you're only using getitem/setitem > operations, you're already using generic functions, just ones that have > syntax sugar (i.e. the [] operator). So in that sense, you can say that > "mapping" means "supported by operator.getitem and operator.setitem". > > I'm not saying there is no such thing as "mapping", IOW, I am saying that > "mapping" is an informal shorthand for a collection of operations. If you > want to be specific, refer to operations. If you wish to be concise (but > vague) then refer to "mapping". > > To put it another way, I'm against halfway measures. It bothers me that > people are trying to introduce rigid interfaces, in order to address > "quick-and-dirty" use cases. These things are at opposite ends of the > spectrum, IMO: if you want a quick-and-dirty type test, test on the bloody > concrete type! Testing some abstract interface type to soothe your OO > conscience doesn't actually make the code any less rigid or dirty, it just > hides the smell. I'd rather that such code continued to obviously smell, > rather than pretend it's as fresh as daisies because "interfaces are better". > > Conversely, if you really want that bit of code to be reusable or > extensible, then use a generic function, and the code will be in *fact* > extensible, rather than simply pushing the compatibility problem to > somebody else to figure out. > > Example issue: library X demands a "mapping", but really only uses > getitem. Its code inspects interfaces to decide how to act on parameters, > and behaves differently if it sees "a mapping". I have an object that > implements getitem, and I want the special behavior, so I declare that > object "a mapping". > > But now, library Y, that I also pass the same object to, suddenly starts > behaving differently, because it sees, "ah, you're a mapping! So I'll use > setitem on you..." And now I'm screwed. > > Things like this used to happen to me all the time in Zope 2, which would > introspect methods and attributes a lot so that Zope could "decide" what to > do with an object based on what it was. (And it was this experience that > made me realize that inspect-and-decide is absolutely the wrong way to > write composable code.) > > Zope 3 at first replaced this attribute-inspection with interface > inspection -- with no better result! After all, using an interface as a > flag is no different in essence than using a hasattr() check as a flag. As > I said, it only *looks* prettier. It wasn't until interface adaptation > arrived in Zope 3 that things actually improved, because (like generic > functions) adaptation at least allows third-party registration. > > However, for interface adaptation to work well, the interfaces need to be > highly context-specific, or else we just end up back at the problem where > "mapping" means different things in different contexts. The PyProtocols > approach to solving this was to say, "One use case = one interface", which > eventually led me to realize that this generally amounts to having one or > more generic functions specific to the thing you're actually trying to > do. It takes a lot less time to just *explicitly* add overloads to your > code for the things you want to be able to do with an object, than to have > to debug the stuff that just suddenly starts happening all over the place, > as can happen with interface inspection. > > Note that quick-and-dirty checks based on *concrete* types are actually > *safer* than abstract checks or interface checks, as this generally > prevents them from being used as mere behavior flags that can lead to > conflicts of the type I've described. In contrast, both duck typing (ie. > hasattr) checks and interface checks have proven in practice (Zope 2 and 3 > respectively) to produce significant unwanted side-effects. > > The only way to reduce these side effects is to allow persons other than > the code author to decide what should happen in a specific > context. Adaptation and generic functions have this ability in common, but > mere inspection (regardless of *what* is inspected) does not. > > > > > In other words, I just want to use my object with some operations that a > > > library provides (or requires). An "interface" is excise: something I have > > > to mess with that doesn't directly relate to my goals for using the > > library. > > > >I don't understand the sentence "An "interface" is excise". What does > >excise mean in this context? > > It's an HCI buzzword meaning "work that doesn't obviously advance your > goal, but that you have to do anyway because of the way the system was > designed/implemented". > > > >Regardless, I find it quite a big change from various ways of saying > >"this object must have these methods (including perhaps some for which > >special syntax exists, like __getattr__)" to saying "this object must > >be supported by these generic functions". > > Well, that's why I proposed last week that we allow you to say > ISomething(foo).somemethod() to be able to use such things. And, that you > be allowed to define your arguments as being of type ISomething, in order > to have the correct namespace automatically apply. > > That proposal doesn't stop you from sticking with duck typing. However, if > you *want* to be explicit and extensible and pure, it allows you to do so. > > Meanwhile, I argue that inspection is inherently quick-and-dirty, just like > duck typing. It doesn't actually improve anything, but instead makes you > do more work for no new benefit, while keeping the same likelihood of > unexpected behavior. > > What's more inspection is no different from generic functions in terms of > being able to produce "spooky action at a distance". However, at least > generic functions have tables whose contents can be inspected to find out > all the behaviors that might result, whereas interface inspection can > happen anywhere! And generic functions are actually extensible by third > parties, whereas inspection is not. So: > > Inspection: > * No way to find "spooky" actions > * No way to modify broken behaviors without changing the code or trying to > "trick" the inspector > > Generics: > * "Spooky" actions are all in a table that can be dumped out and read > * Broken or undesirable actions can be overridden > > > >A method is just an > >identifier in the object's attribute namespace. A generic function is > >an object that may hve to be imported from elsewhere. > > But this is also true of interfaces, regardless of how they're > defined. You can't simultaneously have "safe" duck typing and avoid the > use of imports, unless you use some kind of global naming scheme, as in Java. > > > >I object to the suggestion that seems to be implied here that in the > >future we'll all be writing ducklib.quack(ob) instead of the much > >simpler ob.quack(). > > No, I'm saying that in any case where current interface proposals would do > this: > > if implements(ob, IDuck): > ob.quack() > > I would argue that you are better off with *either of*: > > ducklib.quack(ob) > > OR: > > ob.quack() > > And that most code that isn't trying to be explicitly reusable would in > fact use the second, "normal" syntax. (In all likelihood, the code using > ducklib.quack would only be code *in* ducklib, actually.) GF's really only > come into play when you want to make a library extensible or generic, like > for pickling, pretty-printing, AST-visiting, etc. etc. Or, if you have > some circumstance that requires you to add custom code (like the sendmail() > overload example for 'str'). > > > >I really like that when I have an object in my > >hands, I don't need to import anything else in order to manipulate it, > >as long as the manipulation can be done through methods. I also don't > >think that the homonym problem that you (and CLOS) are trying to solve > >here is all that important in practice. > > Homonyms aren't the problem that's being solved, it's context-specific > extensibility and library composability. Those are much bigger problems > than name collisions. For example, a major source of Zope 2's > architectural difficulties was the direct result of using the same type > of inspection that's being promoted here. (Here being the Py3K list.) I > think it would be wise to learn from that experience. > > Yes, Zope 2 used hasattr() checks, not interface checks, but the effect is > the same: people fudging what they provide in order to trick Zope into > doing the right thing(s), instead of being able to just directly define the > desired behavior, as they can do with adapters or GF's. > > Now, I suppose you could look at these things as being no worse than they > are in various other languages, and that's probably true. On the other > hand, you could look at how much more composable libraries are in languages > with generic functions, and observe that as a general rule, such languages > do not have massive, all-inclusive branded frameworks like Zope, Twisted, > PEAK, etc. And the reason for that, is that in GF-based languages, > *libraries are composable on a larger scale*. So, there isn't a need for > single integrators to pull together any huge "all-in-one" frameworks. > > Of course, packaging is also a factor: Twisted, Zope, and PEAK are > all-inclusive in part because the historical cost of depending on other > Python packages is high. I created setuptools specifically to address that > problem. > > But the other major factor is integration: all-inclusive frameworks work > around the difficulties inherent in wrapping other packages. Many was the > time in developing PEAK that I wanted to use some existing library (or even > stdlib module), but couldn't because there was no way to add the necessary > "glue" without awkward monkeypatching. I would thus end up rolling my own > libraries more often than I wanted to. > > GF-based frameworks, on the other hand, don't seem like frameworks at > all. They're just libraries that can be combined with other > libraries. Interfaces don't give you this ability on their own, and even > adaptation only gets you part of the way (and requires writing more code to > do the same things). > > I'd like to see a Python where this type of composability falls out > naturally as a side effect of using the language idiomatically. Being able > to add overloads to existing functions makes it straightforward to either > override behaviors or add adapters as appropriate, when gluing disparate > libraries together. > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Tue Dec 5 17:16:37 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 05 Dec 2006 11:16:37 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: >My point is that an interface can *document* (at least in English) a >"contract" about the invariants between operations. While I'm not into >enforcing or verifying such contracts, I'm very interested in >documenting them. For example, something that has "mapping" behavior >has a very different relationship between x[y] and "y in x" than >something that has "sequence" behavior. I assumed this didn't need answering. If you're using the interface solely for documentation, then a namespace-oriented interface suffices to provide it. > If you see interfaces as >*just* combinations of operations there is no way to put this >information. Ah. I've been thinking of this as there being two different 'getitem' operations, like at the C level. From greg.ewing at canterbury.ac.nz Wed Dec 6 00:38:21 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 06 Dec 2006 12:38:21 +1300 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <3d2ce8cb0612042132h2aa1572do633e81d7dcfed164@mail.gmail.com> References: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> <457224C6.6040909@canterbury.ac.nz> <3d2ce8cb0612042132h2aa1572do633e81d7dcfed164@mail.gmail.com> Message-ID: <457602ED.6050703@canterbury.ac.nz> Mike Klaas wrote: > The "magic" (and I don't mean that derogatorily) I'm talking > about is the Python <-> c type conversion and assumption that all > undeclared variables are python objects. It would be fairly easy to have a mode that worked like that. Some new syntax would be needed to force a conversion when you wanted one, though. Unless you were willing to call the appropriate Python/C API functions explicitly whenever you wanted a conversion. > Ever consider opening up the project to other committers and becoming > the BDFL of pyrex? I'm starting to think more seriously about setting up a repository for it somewhere, now that it's no longer my foreground project. My main worry is that if too many people hack on it simultaneously I'll lose track of how it works -- it's hard enough keeping all its intricacies in my head now as it is. :-) So I'd probably want to review patches and apply them myself, at least to begin with. Does anyone have any recommendations for a site to host the repository? I'd really like to avoid SourceForge if possible, as it makes me cringe every time I go there -- cluttered pages, slow response, way too complicated UI. -- Greg From brett at python.org Wed Dec 6 00:46:26 2006 From: brett at python.org (Brett Cannon) Date: Tue, 5 Dec 2006 15:46:26 -0800 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: <457602ED.6050703@canterbury.ac.nz> References: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> <457224C6.6040909@canterbury.ac.nz> <3d2ce8cb0612042132h2aa1572do633e81d7dcfed164@mail.gmail.com> <457602ED.6050703@canterbury.ac.nz> Message-ID: There is Google Code Hosting: http://code.google.com/hosting/ . -Brett On 12/5/06, Greg Ewing wrote: > > Mike Klaas wrote: > > The "magic" (and I don't mean that derogatorily) I'm talking > > about is the Python <-> c type conversion and assumption that all > > undeclared variables are python objects. > > It would be fairly easy to have a mode that worked > like that. Some new syntax would be needed to force > a conversion when you wanted one, though. Unless > you were willing to call the appropriate Python/C > API functions explicitly whenever you wanted a > conversion. > > > Ever consider opening up the project to other committers and becoming > > the BDFL of pyrex? > > I'm starting to think more seriously about setting > up a repository for it somewhere, now that it's > no longer my foreground project. My main worry is > that if too many people hack on it simultaneously > I'll lose track of how it works -- it's hard > enough keeping all its intricacies in my head now > as it is. :-) So I'd probably want to review > patches and apply them myself, at least to begin > with. > > Does anyone have any recommendations for a site > to host the repository? I'd really like to avoid > SourceForge if possible, as it makes me cringe > every time I go there -- cluttered pages, slow > response, way too complicated UI. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061205/a7b2affe/attachment.htm From gsakkis at rutgers.edu Wed Dec 6 01:46:17 2006 From: gsakkis at rutgers.edu (George Sakkis) Date: Tue, 5 Dec 2006 19:46:17 -0500 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) In-Reply-To: References: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> <457224C6.6040909@canterbury.ac.nz> <3d2ce8cb0612042132h2aa1572do633e81d7dcfed164@mail.gmail.com> <457602ED.6050703@canterbury.ac.nz> Message-ID: <91ad5bf80612051646j18073511jc47b99a86c7e65e@mail.gmail.com> Or for less barebones solution, WebFaction offers free Trac and Subversion hosting for open-source python projects: http://www.webfaction.com/freetrac. George On 12/5/06, Brett Cannon wrote: > There is Google Code Hosting: http://code.google.com/hosting/ . > > -Brett > > > On 12/5/06, Greg Ewing < greg.ewing at canterbury.ac.nz> wrote: > > > > Mike Klaas wrote: > > > The "magic" (and I don't mean that derogatorily) I'm talking > > > about is the Python <-> c type conversion and assumption that all > > > undeclared variables are python objects. > > > > It would be fairly easy to have a mode that worked > > like that. Some new syntax would be needed to force > > a conversion when you wanted one, though. Unless > > you were willing to call the appropriate Python/C > > API functions explicitly whenever you wanted a > > conversion. > > > > > Ever consider opening up the project to other committers and becoming > > > the BDFL of pyrex? > > > > I'm starting to think more seriously about setting > > up a repository for it somewhere, now that it's > > no longer my foreground project. My main worry is > > that if too many people hack on it simultaneously > > I'll lose track of how it works -- it's hard > > enough keeping all its intricacies in my head now > > as it is. :-) So I'd probably want to review > > patches and apply them myself, at least to begin > > with. > > > > Does anyone have any recommendations for a site > > to host the repository? I'd really like to avoid > > SourceForge if possible, as it makes me cringe > > every time I go there -- cluttered pages, slow > > response, way too complicated UI. > > > > -- > > Greg > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/brett%40python.org > > > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/george.sakkis%40gmail.com > > > From jan.grant at bristol.ac.uk Wed Dec 6 10:20:43 2006 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 6 Dec 2006 09:20:43 +0000 (GMT) Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> Message-ID: <20061206091118.B30563@tribble.ilrt.bris.ac.uk> On Mon, 4 Dec 2006, Bill Janssen wrote: > Jim Jewett writes: > > On 12/3/06, Bill Janssen wrote: > > > > Implementations are used to make Method Dispatching more flexible by > > > > allowing a class to claim that it acts like another class/type even > > > > though it is not derived from that class/type. > > > > > I'm not sure just what this adds over the simple "change_class" > > > function that I've posted twice already. Is there an implementation > > > efficiency argument here? > > > > (1) Interfaces can be placed on a separate (presumably light-weight) > > inheritance hierarchy. > > I know it looks this way, but I think you'd just be changing one > inheritance mechanism for another. No one would use the one that's > already there; they'd just use this one instead. And I don't really > see how it's lighter-weight -- can you explain that? With interfaces, it's not really about inheritance (although it might look like it). Inheritance is an implementation aspect of classes. Between interfaces, the question is simply one of _polymorphism_. Can members of one type be used as members of another type? Interfaces describe types: they are contracts. They detail the responsiblities of the user of the interface, and the promises that the provider of the interface makes if those responsibilities are met. Classes (and inheritance) are just one way of implementing those types. That is, the class mechanism provides encapsulation; interfaces are the vehicle for abstraction and polymorphism. This might sound like semantic nit-picking, but the description of types and API contracts is the prime thing I'm looking for in an interface mechanism. Although the role that I see interfaces playing can be met by ABCs, there's always a sneaking tendency to slip some implementation detail into the ABC: that is, I see ABCs as an implementation approach to interfaces for a language that doesn't have the latter as a first-class language feature. Given a type system (via interfaces or ABCs), Phillip's GFs are an interesting extension of Python's capabilities. The use of interfaces to express APIs plus the (perhaps only human-readable) semantic constraints on those is a useful feature in its own right. Cheers, jan -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Semantic rules, OK? From jan.grant at bristol.ac.uk Wed Dec 6 10:36:31 2006 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 6 Dec 2006 09:36:31 +0000 (GMT) Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> Message-ID: <20061206092711.G30563@tribble.ilrt.bris.ac.uk> On Mon, 4 Dec 2006, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Dec 4, 2006, at 5:08 AM, Ben Wing wrote: > > > i see in PEP 3101 that there's some work going on to fix up the string > > formatting capabilities of python. it looks good to me but it still > > doesn't really address the lack of a simple interpolated string > > mechanism, as in perl or ruby. I don't think there's such a thing. > After several years of use, I'm strongly +1 for this feature in some > form or another. I think it's a terrible idea in almost every form. > There should be little security concern about feature specifically, > but you do need to be aware of an issue once you start hooking into > catalogs. The reason _why_ I think it's such a bad idea is the above statement just doesn't gel with practical experience. Google will find plenty of reading material if you search for "php sql injection" or "php cross-site scripting". That is to say: In many places where string interpolation is used, what you want is NOT a simple interpolation. Perhaps the feature was OK in the dim distant past of Perl, but back then Perl was aimed at being a tool for sysadmins, and sysadmins often deal with plain text files (in the Unix context). When presented with this misfeature, however, novice programmers see it as a great way to construct SQL queries, or perhaps produce html output with variables interpolated. In both instances the convenience of the expression leads to the dark path. So unless there's a way to get strings to magically know their intended use, and to find a type- and use-safe way of doing interpolation that respects the quoting requirements of the eventual use (and I don't think that level of type inference is going to happen in Py3k), I think this is, on a practical level, inviting disaster. You mention the need to quote variables that are interpolated. You're clearly a careful and thoughtful programmer. Python is a great language for novices too, and they may not have even been exposed to the concept of an injection attack. Don't turn Python into the next PHP. Cheers, jan -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Whose kung-fu is the best? From jan.grant at bristol.ac.uk Wed Dec 6 10:43:15 2006 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 6 Dec 2006 09:43:15 +0000 (GMT) Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <4573F3FD.1080800@666.com> References: <4573F3FD.1080800@666.com> Message-ID: <20061206093755.M30563@tribble.ilrt.bris.ac.uk> On Mon, 4 Dec 2006, Ben Wing wrote: > as a result, i imagine there's a strong urge to just hardcode the name > of the parent ^^^^^^^^^^ > -- super.meth(args) calls the superclass method `meth' ^^^^^^^^^^^^^^ Python supports multiple inheritance, unlike Java; the design mantra is "explicit is better than implicit" and "ambiguity should be an error". Two! The two design mantras are... > -- as an alternative or in addition, super(args) would work like > super.meth(args) if we're currently inside of `meth' in a subclass. i > suspect that 90% of the time or more, `super' is used to chain to the > superclass version of an overridden method, and this optimizes for the > common case. Commonly you know what the method is called at the point where you're typing "super" :-) -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Not as randy or clumsom as a blaster. From python at zesty.ca Wed Dec 6 11:02:11 2006 From: python at zesty.ca (Ka-Ping Yee) Date: Wed, 6 Dec 2006 04:02:11 -0600 (CST) Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <20061206092711.G30563@tribble.ilrt.bris.ac.uk> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> Message-ID: On Wed, 6 Dec 2006, Jan Grant wrote: [regarding string interpolation] > I think it's a terrible idea in almost every form. [...] > The reason _why_ I think it's such a bad idea is the above statement > just doesn't gel with practical experience. Google will find plenty of > reading material if you search for "php sql injection" or "php > cross-site scripting". That is to say: > > In many places where string interpolation is used, what you want is NOT > a simple interpolation. If i understand you correctly, your argument is: Many people use simple string interpolation when they should be using an application-specific quoting mechanism. Therefore string interpolation should be inconvenient. I don't agree that the second statement follows from the first. You are trying to make a "path of least resistance" argument, but the more logical argument would be: Many people use simple string interpolation when they should be using an application-specific quoting mechanism. Therefore application-specific quoting mechanisms should be more convenient to create and use. I see this, therefore, as a design constraint upon the design of a good string interpolation mechanism. In Perl and PHP, the standard interpolation mechanism supports *only* simple string interpolation. Using a more appropriate quoting mechanism is significantly more inconvenient to the script programmer; hence it is not so often used. But here, in Python, we have an opportunity that these other languages do not -- the opportunity to design a new string interpolation mechanism that is more convenient than what we have now. If this mechanism makes application-specific quoting just as, or nearly as, easy as simple interpolation -- that is, lowering the usage barrier to the safe path -- then security considerations would actually point in *favour* of the new mechanism. Where does this design constraint take us? One possibility it suggests is that the interpolation function could be generic, allowing the formatting template to be of a custom type that supports application-specific quoting. If we go this route, that would argue in favour of solutions like a method on the template object or an infix operator, and would reject solutions like a built-in interpolated string type. Another possibility is that the interpolation function, if a built-in, could be overridable. -- ?!ng From jan.grant at bristol.ac.uk Wed Dec 6 11:24:20 2006 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 6 Dec 2006 10:24:20 +0000 (GMT) Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> Message-ID: <20061206100741.A30563@tribble.ilrt.bris.ac.uk> On Wed, 6 Dec 2006, Ka-Ping Yee wrote: [well-articulated point made] Thank-you: I agree with pretty much everything you say. It's an interesting question to ask, "can one construct a convenient interpolation mechanism that uses the appropriate application quoting mechanisms where necessary?" > If i understand you correctly, your argument is: > > Many people use simple string interpolation when they should > be using an application-specific quoting mechanism. > > Therefore string interpolation should be inconvenient. Actually, "unavailable", but yup :-) > Therefore application-specific quoting mechanisms should > be more convenient to create and use. Agreed, if this is possible whilst remaining "Pythonic". > Where does this design constraint take us? > > One possibility it suggests is that the interpolation function could > be generic, allowing the formatting template to be of a custom type > that supports application-specific quoting. If we go this route, > that would argue in favour of solutions like a method on the > template object or an infix operator, and would reject solutions > like a built-in interpolated string type. That's pretty much the conclusion I'd come to... > Another possibility is that the interpolation function, if a built-in, > could be overridable. I think the question here is, is there a safe, convenient way that Python can determine what to do (or be told what to do) when it sees: s = "some string here with {variable} in it" If you have a good approach to the second possibility then I'd be very interested. Cheers, jan PS. IF generic functions (operators) could be sensitive to return types and IF python6k supported a complex type inference mechanism (possibly including automatic coercion), then one might be able to do something like: def foo(d: sqlDriver): s = "select template goes here" % (param1, param2) r = d.query(s) where sqlDriver has a method def query(self, s: some_type_that_signals_sql_interpolation) and "%" is overloadable on the basis that the return type is determined to be compatible with "some_type_that_signals_sql_interpolation". Those are some mighty big "IF"s though, and you could still concoct cases where things would break :-) -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ The Java disclaimer: values of 'anywhere' may vary between regions. From jan.grant at bristol.ac.uk Wed Dec 6 12:44:14 2006 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 6 Dec 2006 11:44:14 +0000 (GMT) Subject: [Python-3000] Changing the % format return type Re: features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <20061206100741.A30563@tribble.ilrt.bris.ac.uk> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> <20061206100741.A30563@tribble.ilrt.bris.ac.uk> Message-ID: <20061206112943.W30563@tribble.ilrt.bris.ac.uk> Ka-Ping Yee's suggestion of overriding the interpolation builtin (or let's say, the "%" operator used for formatting) means there might be another way to do this using GFs: let's say that the following illustrative snippet is still desirable - On Wed, 6 Dec 2006, Jan Grant wrote: > def foo(d: sqlDriver): > s = "select template goes here" % (param1, param2) > r = d.query(s) # do the Right Thing Suppose for a moment that % used like this returns, not a string, but an object that encapsulates the format and the parameter set supplied. Then the sqlDriver implementation could take advantage of Phillip's GF mechanisms thus: class driver # which implements sqlDriver def query(self, s: str): ... code here to perform a query based on a stock string ... def query(self, pf: parameterised_format): ... code here to extract the format string and quote the ... parameters appropriately within it. The _cost_ of this mechanism is that all library functions potentially have to become aware of the parameterised_format type. Alternatively (and although explicit is better than implicit), a GF framework might look to coerce a parameterised_format into a str using something like adaption. Or "parameterised_format" could be a _subtype_ of "str", which behaves exactly like the str type if it is naively used in contexts which demand a plain string. In the latter case (if achievable, I have not examined this closely) a %-formatted object would "degrade" to the simple string case for most current uses. jan -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ perl -e 's?ck?t??print:perl==pants if $_="Just Another Perl Hacker\n"' From solipsis at pitrou.net Wed Dec 6 12:50:37 2006 From: solipsis at pitrou.net (Antoine) Date: Wed, 6 Dec 2006 12:50:37 +0100 (CET) Subject: [Python-3000] interpolated strings In-Reply-To: <20061206100741.A30563@tribble.ilrt.bris.ac.uk> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> <20061206100741.A30563@tribble.ilrt.bris.ac.uk> Message-ID: <48906.62.39.9.251.1165405837.squirrel@webmail.nerim.net> > On Wed, 6 Dec 2006, Ka-Ping Yee wrote: > and "%" is overloadable on the basis that the return type is determined > to be compatible with "some_type_that_signals_sql_interpolation". Those > are some mighty big "IF"s though, and you could still concoct cases > where things would break :-) Or perhaps simpler, a new kind of string literal would construct an Interpolation object: s = i"some string here with {variable} in it" equivalent to: s = Interpolation("some string here with {variable} in it", {"variable": }) The Interpolation object captures the format string, as well as a dict of the needed variables from the current locals and globals (here, the "variable"). But it does not render the interpolated string immediately, just stores the values for future use. The __str__ method on the Interpolation object would do the "obvious" thing, i.e. interpolate without any quoting. However, nothing stops a more demanding object from doing its own interpolation instead of blindly calling __str__. For example, in: d = SqlDriver(...) s = i"select password from {user}" r = d.query(s) d.query() would automatically quote all variables before interpolating. The Interpolation object could provide convenience methods to make this trivial for library implementers, e.g. a render(quote_function=None) method. So d.query() could look like the following: def query(self, s): if hasattr(s, "render"): s = s.render(quote_function=self.sql_quote_value) else: s = str(s) # do stuff From solipsis at pitrou.net Wed Dec 6 12:52:39 2006 From: solipsis at pitrou.net (Antoine) Date: Wed, 6 Dec 2006 12:52:39 +0100 (CET) Subject: [Python-3000] interpolated strings In-Reply-To: <48906.62.39.9.251.1165405837.squirrel@webmail.nerim.net> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> <20061206100741.A30563@tribble.ilrt.bris.ac.uk> <48906.62.39.9.251.1165405837.squirrel@webmail.nerim.net> Message-ID: <57364.62.39.9.251.1165405959.squirrel@webmail.nerim.net> > Or perhaps simpler, a new kind of string literal would construct an > Interpolation object: Hmm, so Jan proposed something similar in the meantime. Sorry for the noise. From fredrik at pythonware.com Wed Dec 6 13:04:27 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Wed, 6 Dec 2006 13:04:27 +0100 Subject: [Python-3000] interpolated strings References: <4573F3BA.80603@666.com><7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org><20061206092711.G30563@tribble.ilrt.bris.ac.uk><20061206100741.A30563@tribble.ilrt.bris.ac.uk> <48906.62.39.9.251.1165405837.squirrel@webmail.nerim.net> Message-ID: Antoine wrote: > s = i"some string here with {variable} in it" > > The Interpolation object captures the format string, as well as a dict of > the needed variables from the current locals and globals (here, the > "variable"). I'm not sure I can think of a way to explain to a new Python programmer that she needs to inline all her string literals... (people who do interpolation tend to expect dynamic lexical scoping, not static object binding...) btw, note that you can get the same behaviour with today's Python: s = I("some string here with ", variable, " in it") that's only a little harder to type, is properly rendered by all existing IDEs and code colorizers, is understood by existing code analysis tools, is trivial to extend with new formatting operations, and is of course straightforward to override on all levels. From barry at python.org Wed Dec 6 13:58:03 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 6 Dec 2006 07:58:03 -0500 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <20061206092711.G30563@tribble.ilrt.bris.ac.uk> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> Message-ID: <58FEFFAF-AD82-4F7A-B941-AA28EC4F44A4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 6, 2006, at 4:36 AM, Jan Grant wrote: > On Mon, 4 Dec 2006, Barry Warsaw wrote: > >> After several years of use, I'm strongly +1 for this feature in some >> form or another. > > I think it's a terrible idea in almost every form. > >> There should be little security concern about feature specifically, >> but you do need to be aware of an issue once you start hooking into >> catalogs. > > The reason _why_ I think it's such a bad idea is the above statement > just doesn't gel with practical experience. Google will find plenty of > reading material if you search for "php sql injection" or "php > cross-site scripting". That is to say: > > In many places where string interpolation is used, what you want is > NOT > a simple interpolation. Perhaps the feature was OK in the dim distant > past of Perl, but back then Perl was aimed at being a tool for > sysadmins, and sysadmins often deal with plain text files (in the Unix > context). Just like every other tool, you have to know how to use it correctly. Should we outlaw string concatenation because if you use it to generate SQL queries, you're opening yourself up to SQL injection attacks? The security of string interpolation in the context of translatable text is really not an issue with interpolation, but with trusting your translators. For example, a translator could just as easily do some damage by translating "Are you sure you want to delete this list?" into "Do you like Kung Pao Chicken?". > So unless there's a way to get strings to magically know their > intended > use, and to find a type- and use-safe way of doing interpolation that > respects the quoting requirements of the eventual use (and I don't > think > that level of type inference is going to happen in Py3k), I think this > is, on a practical level, inviting disaster. Sadly, such is the state of modern programming. The only way out is to provide the right tool for the job and educate programmers in how to use them. The answer isn't to eradicate features that can be misused, unless you want to quit this line of work and become a Kung Pao Chicken farmer. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXa+YHEjvBPtnXfVAQLcHwP9EaF/fpxv9Eg2CHprlvbNVHzESz9dbDJN DfAs9dtbku4ybmMgVVK8IC11bbdAHyJXhblx8BRWUahBAvgQndz1rRwCcAOhUb/t RMC47FxdK9QREnCr49/0sqZdnIHBSoQQa4annnouHPySvYvR+1fYMAPxqV2NRavr RF0FZAmU7ZY= =/EGx -----END PGP SIGNATURE----- From barry at python.org Wed Dec 6 14:02:06 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 6 Dec 2006 08:02:06 -0500 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> Message-ID: <3CA5A2BE-03B0-4D5E-88E5-9FF2AC2AA12A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 6, 2006, at 5:02 AM, Ka-Ping Yee wrote: I agree with where Ka-Ping is headed... > Where does this design constraint take us? > > One possibility it suggests is that the interpolation function could > be generic, allowing the formatting template to be of a custom type > that supports application-specific quoting. If we go this route, > that would argue in favour of solutions like a method on the > template object or an infix operator, and would reject solutions > like a built-in interpolated string type. I agree that it makes less sense to build string interpolation into the language and more sense to add it to the library somehow (as we've already done for $-strings). Based on that experience, I think it would be possible to build the application-specific quoting machinery into the thing supplying the interpolation values rather than the thing providing the template. You need both sides of the equation. IME, the template is the generic "machinery, not policy" part and the substitution supplier is the "policy" part. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXa/TnEjvBPtnXfVAQJK/QP9EKGZK6DeZs/Yb/xo03/hL/P3G44JJJix Y4VwP2EffmnNH1P8GqnC/YWsY2VGCWTmbqoOBy8waim5A4LS3PVCY4NikEdVb0rZ KXdBL/b1mFvgHUfasxpAv2Cm7VOwH79YSW+wAI+1j4idOq0LrWg9cKijiOT0ey7j 5AjSHbV2h5I= =0psW -----END PGP SIGNATURE----- From barry at python.org Wed Dec 6 14:04:10 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 6 Dec 2006 08:04:10 -0500 Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <20061206100741.A30563@tribble.ilrt.bris.ac.uk> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> <20061206100741.A30563@tribble.ilrt.bris.ac.uk> Message-ID: <2497484F-E2A1-4A26-8D0D-5D61913DE600@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 6, 2006, at 5:24 AM, Jan Grant wrote: > PS. IF generic functions (operators) could be sensitive to return > types > and IF python6k supported a complex type inference mechanism (possibly > including automatic coercion), then one might be able to do something > like: > > def foo(d: sqlDriver): > s = "select template goes here" % (param1, param2) > r = d.query(s) > > where sqlDriver has a method > > def query(self, s: some_type_that_signals_sql_interpolation) > > and "%" is overloadable on the basis that the return type is > determined > to be compatible with "some_type_that_signals_sql_interpolation". > Those > are some mighty big "IF"s though, and you could still concoct cases > where things would break :-) BTW, I believe that most of the SQL wrapper packages for Python already provide safe prepared statement like functionality, that is convenient to use. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXa/y3EjvBPtnXfVAQJXTAP/YjVK4fW0cuEd7FauzlDeedO2rXxfkpIg jsOg573Ce05WcrDYyKR+jJ4KQlfEKZbHgjfkWvRZtXn9U+jvbxcNZooFmKxhqGnK FLZ4t5vGTcD/n3gjIaEkumh2TdfnWIKcdstPGqrCUQdITvCacuW37zDh7dSiGOcm 4iIipl/UONA= =snz0 -----END PGP SIGNATURE----- From barry at python.org Wed Dec 6 14:07:55 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 6 Dec 2006 08:07:55 -0500 Subject: [Python-3000] interpolated strings In-Reply-To: References: <4573F3BA.80603@666.com><7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org><20061206092711.G30563@tribble.ilrt.bris.ac.uk><20061206100741.A30563@tribble.ilrt.bris.ac.uk> <48906.62.39.9.251.1165405837.squirrel@webmail.nerim.net> Message-ID: <216AB5BA-CD0D-462F-807E-055078D1ED25@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 6, 2006, at 7:04 AM, Fredrik Lundh wrote: > btw, note that you can get the same behaviour with today's Python: > > s = I("some string here with ", variable, " in it") > Or even s = I('some string here with $variable in it') or some day s = I('some string here with {variable} in it') And I() is the thing you build the application-specific quoting (or translating) machinery into. Simple to explain and easy to read and write! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRXbAq3EjvBPtnXfVAQIlIAP/dtnQR8TKIDSQZvIu4rLShqqWOx7QoSwY NSKsHu+tqIwdLDbJSXb/YatgXKp6VoDx1AMo+EorBomuYkuH5WXzA0EIa+jF3L/T 8H3hLeBe11IjUSkQ/i1V6Qfy+hWytS67iejaXDdoSPhLq5mgLbSnbp8FD3BuRTqs 3j3icjML/g8= =5QqD -----END PGP SIGNATURE----- From solipsis at pitrou.net Wed Dec 6 15:37:01 2006 From: solipsis at pitrou.net (Antoine) Date: Wed, 6 Dec 2006 15:37:01 +0100 (CET) Subject: [Python-3000] interpolated strings In-Reply-To: References: <4573F3BA.80603@666.com><7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org><20061206092711.G30563@tribble.ilrt.bris.ac.uk><20061206100741.A30563@tribble.ilrt.bris.ac.uk> <48906.62.39.9.251.1165405837.squirrel@webmail.nerim.net> Message-ID: <39736.62.39.9.251.1165415821.squirrel@webmail.nerim.net> > (people who do interpolation tend to expect dynamic lexical scoping, not > static > object binding...) After thinking a bit about it, you are right. Or more precisely, "inline" interpolation with literals is mainly useful for quick-and-dirty scripts rather than full-blown apps. (this "quick-and-dirty" vs. "full-blown" distinction has already been discussed on this list without coming to a definite conclusion, IIRC; it was in context of character set handling in file methods) From gary at zope.com Wed Dec 6 17:18:28 2006 From: gary at zope.com (Gary Poster) Date: Wed, 6 Dec 2006 11:18:28 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <20061206091118.B30563@tribble.ilrt.bris.ac.uk> References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> <20061206091118.B30563@tribble.ilrt.bris.ac.uk> Message-ID: On Dec 6, 2006, at 4:20 AM, Jan Grant wrote: > On Mon, 4 Dec 2006, Bill Janssen wrote: >> Jim Jewett writes: ... >>> (1) Interfaces can be placed on a separate (presumably light- >>> weight) >>> inheritance hierarchy. ... >> And I don't really >> see how it's lighter-weight -- can you explain that? ... > Interfaces describe types: they are contracts. Yes. I like Jan's (longer :-) reply. Compared to ABCs, interfaces can also describe contracts that can be fulfilled with any object--not just class instances, but classes themselves, and functions. Callables are an easy real-world example. If an interface describes a callable object, the contract might be fulfilled by a function, a class, or an interface. In Zope 3, we do that, often for factories, where the implementation choice might depend on simplicty of implementation, configuration options, and so on. The ability to slam on an interface to an instance is another important difference: specific instances can fulfill contracts that all instances of the same class do not necessarily fulfill. Other examples are perhaps more theoretical--with classmethods, a class can fulfill a contract that we'd normally expect to be met with an instance. I've used interfaces to describe a contract that a class fulfills via classmethods, but I can't recall I really expected the contract to be possibly fulfilled as an instance (though it certainly could). I'm not sure that Jan and I are actually addressing what Jim Jewett intended to convey in his description of a "light-weight" interface hierarchy. This is more a general flexibility of the interface approach in comparison to ABCs. But the flexibility is very useful, in my experience, and the concept of a "contract"--including the politically correct notion that you shouldn't have preconceived notions of who can fulfill the contract--describes (Zope 3) interfaces well. Gary From jcarlson at uci.edu Wed Dec 6 17:33:27 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 06 Dec 2006 08:33:27 -0800 Subject: [Python-3000] Changing the % format return type Re: features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <20061206112943.W30563@tribble.ilrt.bris.ac.uk> References: <20061206100741.A30563@tribble.ilrt.bris.ac.uk> <20061206112943.W30563@tribble.ilrt.bris.ac.uk> Message-ID: <20061206082623.975D.JCARLSON@uci.edu> Jan Grant wrote: > On Wed, 6 Dec 2006, Jan Grant wrote: > > > def foo(d: sqlDriver): > > s = "select template goes here" % (param1, param2) > > r = d.query(s) # do the Right Thing > > Suppose for a moment that % used like this returns, not a string, but an > object that encapsulates the format and the parameter set supplied. > > Then the sqlDriver implementation could take advantage of Phillip's GF > mechanisms thus: > > class driver # which implements sqlDriver > def query(self, s: str): > ... code here to perform a query based on a stock string ... > > def query(self, pf: parameterised_format): > ... code here to extract the format string and quote the > ... parameters appropriately within it. At least in the case of the few SQL drivers I've used in the past couple years, usually there is something like the following, which automatically handles escaping as necessary... d.query("UPDATE table SET value = ? WHERE rowid = ?", val, row) If your SQL driver of choice in any language you use does not have a mechanism similar to the above, then it needs a feature addition. As such, I'm not sure that this particular problem (or its solution) is necessary for Python. Do like the SQL people have been doing for years, pass *both* parts to your query/format function. - Josiah From jan.grant at bristol.ac.uk Wed Dec 6 17:53:13 2006 From: jan.grant at bristol.ac.uk (Jan Grant) Date: Wed, 6 Dec 2006 16:53:13 +0000 (GMT) Subject: [Python-3000] features i'd like [Python 3000?] ... #4: interpolated strings ala perl In-Reply-To: <2497484F-E2A1-4A26-8D0D-5D61913DE600@python.org> References: <4573F3BA.80603@666.com> <7863E7DD-F7B8-4FDE-89DA-1BEC5A7E9B34@python.org> <20061206092711.G30563@tribble.ilrt.bris.ac.uk> <20061206100741.A30563@tribble.ilrt.bris.ac.uk> <2497484F-E2A1-4A26-8D0D-5D61913DE600@python.org> Message-ID: <20061206164908.M39873@tribble.ilrt.bris.ac.uk> On Wed, 6 Dec 2006, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Dec 6, 2006, at 5:24 AM, Jan Grant wrote: > > > PS. IF generic functions (operators) could be sensitive to return types > > and IF python6k supported a complex type inference mechanism (possibly > > including automatic coercion), then one might be able to do something > > like: > > > > def foo(d: sqlDriver): > > s = "select template goes here" % (param1, param2) > > r = d.query(s) > > > > where sqlDriver has a method > > > > def query(self, s: some_type_that_signals_sql_interpolation) > > > > and "%" is overloadable on the basis that the return type is determined > > to be compatible with "some_type_that_signals_sql_interpolation". Those > > are some mighty big "IF"s though, and you could still concoct cases > > where things would break :-) > > BTW, I believe that most of the SQL wrapper packages for Python already > provide safe prepared statement like functionality, that is convenient to use. You are absolutely right, and on reflection I think I concur. Whilst some hoop-jumping *might* be possible to ensure that the formatting operator % is compatible with various application-specific quoting requirements (and without which "string interpolation" in the language is a terrible misfeature IMHO*), appropriate (and suitably "pythonic") mechanisms already exist for this in the libraries I care about. jan * by using any proposed string interpolation syntax as sugar for a %-expression -- jan grant, ISYS, University of Bristol. http://www.bris.ac.uk/ Tel +44 (0)117 3317661 http://ioctl.org/jan/ Goth isn't dead, it's just lying very still and sucking its cheeks in. From janssen at parc.com Wed Dec 6 20:09:47 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 6 Dec 2006 11:09:47 PST Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> <20061206091118.B30563@tribble.ilrt.bris.ac.uk> Message-ID: <06Dec6.110949pst."58648"@synergy1.parc.xerox.com> > Compared to ABCs, interfaces can also describe contracts that can be > fulfilled with any object--not just class instances, but classes > themselves, and functions. Sure, and I appreciate that. But classes and functions are both instances of types, and thus can have extra types mixed into them: either marker types (no methods), or ABCs, or mix-ins that actually provide extra functionality. I'm arguing that most (all?) of the functionality of a new "stand-off" interface system can already be achieved for the vast majority of cases using a system that's already designed and implemented in Python, the multiple-inheritance type system. After all, the "object" still has to fulfill the "contract", and most of the time that means it has to implement an interface of some kind. And it's going to be difficult to do that if it doesn't have some kind of "connection" to an implementation of that interface. Using types to provide that connection is a simple and already-implemented mechanism. There are some drawbacks to this approach, such as the fact that changing the type of an instance later is (currently) fairly expensive. But that's a reasonable target for optimization. Another problem is that the basic Python types are somewhat poorly understood, which is the point of the exercise at http://wiki.python.org/moin/AbstractBaseClasses. Bill From guido at python.org Wed Dec 6 21:58:23 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 6 Dec 2006 14:58:23 -0600 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> Message-ID: On 12/5/06, Phillip J. Eby wrote: > At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: > >My point is that an interface can *document* (at least in English) a > >"contract" about the invariants between operations. While I'm not into > >enforcing or verifying such contracts, I'm very interested in > >documenting them. For example, something that has "mapping" behavior > >has a very different relationship between x[y] and "y in x" than > >something that has "sequence" behavior. > > I assumed this didn't need answering. If you're using the interface solely > for documentation, then a namespace-oriented interface suffices to provide it. I don't follow. I feel awfully stupid right now -- I have a major roadblock to understanding how your proposal helps my use case and you think that it doesn't even need answering. Maybe I should just give up and accept that I'll never understand your proposal; it can be reconsidered for Python 4000 by a future BDFL. > > If you see interfaces as > >*just* combinations of operations there is no way to put this > >information. > > Ah. I've been thinking of this as there being two different 'getitem' > operations, like at the C level. But having two operations at the C level is actually a major problem -- considering e.g. that lists currently implement "mapping-getitem" just so they can handle extended slices. AFAIK it's generally accepted that it was a mistake to have separate C operations, and this is slated to be fixed in Py3k. Anyway I don't think it is right to claim that there are many different operations if the normal way of spelling them is overloaded. I see these overloaded operations as *the same* operation (a GF as you say) but with different semantics (and sometimes a different return type) based on what category of object it is (e.g. mapping, sequence or set). The "mapping" example you claim caused so much trouble due to different understanding of what it means to be a mapping can IMO be addressed quite well by adopting a hierarchy of mapping ABCs as sketched in Bill J's wiki page. A lot of the problems in the example (and probably in Zope's interfaces) are caused by the non-existence of a *standard* set of ABCs that people can use to model their classes. Thus users make do with vague notions such as "file-like" and "mapping" that encourage Humpty Dumptyism, and don't even distinguish between the most basic variants, like mutability (in terms of required API, not in terms of actual capabilities of the object) or iterability. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Dec 6 22:45:14 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed, 06 Dec 2006 16:45:14 -0500 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061206161326.01f88cf0@sparrow.telecommunity.com> At 02:58 PM 12/6/2006 -0600, Guido van Rossum wrote: >On 12/5/06, Phillip J. Eby wrote: > > At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: > > >My point is that an interface can *document* (at least in English) a > > >"contract" about the invariants between operations. While I'm not into > > >enforcing or verifying such contracts, I'm very interested in > > >documenting them. For example, something that has "mapping" behavior > > >has a very different relationship between x[y] and "y in x" than > > >something that has "sequence" behavior. > > > > If you're using the interface solely > > for documentation, then a namespace-oriented interface suffices to > provide it. > >I don't follow. I feel awfully stupid right now -- I have a major >roadblock to understanding how your proposal helps my use case and you >think that it doesn't even need answering. I was just saying that if you want interfaces for documentation, then a docstring on the class implementing the namespace should do the trick. I.e. class Mapping(Interface): """Document some invariants in this string""" Is there something else you wanted it to do besides *hold* the documentation? >Anyway I don't think it is right to claim that there are many >different operations if the normal way of spelling them is overloaded. >I see these overloaded operations as *the same* operation (a GF as you >say) but with different semantics (and sometimes a different return >type) based on what category of object it is (e.g. mapping, sequence >or set). If there isn't any other way to distinguish the operations, then I'd suggest having an 'ismapping' or 'issequence' operation whose sole purpose is to separate one from the other. This has the nice side effect of being able to use those operations for interface inspection, in the case where you decide to do this (despite it being seriously unwise, due to conflicting meanings in context). >The "mapping" example you claim caused so much trouble due to >different understanding of what it means to be a mapping can IMO be >addressed quite well by adopting a hierarchy of mapping ABCs as >sketched in Bill J's wiki page. A lot of the problems in the example >(and probably in Zope's interfaces) are caused by the non-existence of >a *standard* set of ABCs that people can use to model their classes. This happens even with interfaces (or special attributes in the Zope 2 case) that were *Zope-defined*. The stdlib examples are just that -- more examples. The thing that motivated me to ditch zope.interface and write PyProtocols in the first place was the realization that interface inspection was causing the same class of problems that attribute inspection did. And it didn't matter if the interfaces were defined by the stdlib, Zope, or PEAK, it was producing the same kind of "function A does this with IFoo, but function B does this other thing with IFoo, and I only want function A to do that" problems. In other words, it's the *if-then* itself that causes the problems. It doesn't much matter *what* the if-then tests for. What I realized then, however, was that with adaptation, A and B can both define an IFooA and IFooB, and define default adapters for IFoo. Then, if there's a conflict for a new type T, you can define an adapter from T->IFooA and/or T->IFooB and get what you want without any problems. The if-then's go away, and all is well. Of course, creating all those interfaces is a pain, but it works. Later I realized that generic functions do the same thing without needing any new interfaces; you just define overloads for A and B, or A and B just factor out their type-testing into a new GF. I would also like to point out again, though, that the mechanism I've proposed (wrapping generic functions to create interfaces) actually supports (or can be made to support, using appropriate convenience functions) all the use cases that have been put forth for interfaces, adaptation, and generic functions. At least, the use cases I understand. To support interface inspection in the "mapping vs. sequence", my proposed approach requires that you define a query operation like ismapping() or issequence(). This is indeed more work. My argument, however, is that this is a *good* thing, because interface inspection in the general case is *evil*: it leads to the unfixable contextual interpretations problem, whenever two developers write code that inspects the same interface for different purposes. So, I think it's perfectly acceptable to make people jump through an extra hoop in order to do that sort of thing, especially since it's rare that you're going to have a mapping vs. sequence type of problem. From thomas at python.org Wed Dec 6 22:59:55 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 6 Dec 2006 13:59:55 -0800 Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <20061206093755.M30563@tribble.ilrt.bris.ac.uk> References: <4573F3FD.1080800@666.com> <20061206093755.M30563@tribble.ilrt.bris.ac.uk> Message-ID: <9e804ac0612061359k17fdcc5ds8fe408b8ae734a81@mail.gmail.com> On 12/6/06, Jan Grant wrote: > > On Mon, 4 Dec 2006, Ben Wing wrote: > > > as a result, i imagine there's a strong urge to just hardcode the name > > of the parent > ^^^^^^^^^^ > > > -- super.meth(args) calls the superclass method `meth' > ^^^^^^^^^^^^^^ > > Python supports multiple inheritance, unlike Java; the design mantra is > "explicit is better than implicit" and "ambiguity should be an error". > Two! The two design mantras are... You forget that that's actually what super() is for. It does the right thing in the case of MI (and every other case, in fact :-) > -- as an alternative or in addition, super(args) would work like > > super.meth(args) if we're currently inside of `meth' in a subclass. i > > suspect that 90% of the time or more, `super' is used to chain to the > > superclass version of an overridden method, and this optimizes for the > > common case. > > Commonly you know what the method is called at the point where you're > typing "super" :-) The point is not that it's difficult to come up with the arguments to super(ThisClass, self).methodname(args), but that it's tedious and sometimes unobvious and error-prone. super should definately be enhanced so 'super(args)' and 'super.meth' 'do the right thing' too, but as Nick said, it's not that easy :) It would help if someone started on an implementation though. (It's on my TODO list, but that list is rather swamped with "new job" things.) (For an example of 'tricky things', when you do: class X(Y): def meth(self, x, y): return super(x) + y moremeth = meth -- should X().moremeth(1, 2) call Y.meth() or Y.moremeth()? It could probably be implemented either way, with enough magic in the runtime.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061206/764bb0ef/attachment-0001.htm From guido at python.org Wed Dec 6 23:11:18 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 6 Dec 2006 16:11:18 -0600 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061206161326.01f88cf0@sparrow.telecommunity.com> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <5.1.1.6.0.20061206161326.01f88cf0@sparrow.telecommunity.com> Message-ID: On 12/6/06, Phillip J. Eby wrote: > I would also like to point out again, though, that the mechanism I've > proposed (wrapping generic functions to create interfaces) actually > supports (or can be made to support, using appropriate convenience > functions) all the use cases that have been put forth for interfaces, > adaptation, and generic functions. At least, the use cases I understand. Would you mind going through the set example *in detail*? Or don't you understand that use case? > To support interface inspection in the "mapping vs. sequence", my proposed > approach requires that you define a query operation like ismapping() or > issequence(). This is indeed more work. My argument, however, is that > this is a *good* thing, because interface inspection in the general case is > *evil*: it leads to the unfixable contextual interpretations problem, > whenever two developers write code that inspects the same interface for > different purposes. So, I think it's perfectly acceptable to make people > jump through an extra hoop in order to do that sort of thing, especially > since it's rare that you're going to have a mapping vs. sequence type of > problem. Why do you think developers would abuse interfaces but they wouldn't abuse the meaning of ismapping() or issequence()? it seems that this doesn't help at all -- whether we call an inquiry function with a vague meaning or we have an interface with a vague meaning, misunderstandings are right around the corner, unless we provide as many different query operations as there are semantic variants. And we could solve that with "classic" interfaces just as well as with your proposal. And the query operation doesn't help for adaptation, *unless* you go all the way and implement essentially PyProtocols' "when ", which I'd like to keep out of the core language and library. I also don't like the idea that the query operation can return a dynamic value -- it could decide whether to be a sequence or mapping (or both) based on the time of day, or the contents of an object, or anything else. That's more than I bargained for -- I'm only interested in invariants that the *class* cares to provide. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From gary at zope.com Wed Dec 6 23:41:59 2006 From: gary at zope.com (Gary Poster) Date: Wed, 6 Dec 2006 17:41:59 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <06Dec6.110949pst."58648"@synergy1.parc.xerox.com> References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> <20061206091118.B30563@tribble.ilrt.bris.ac.uk> <06Dec6.110949pst."58648"@synergy1.parc.xerox.com> Message-ID: <7FB4FB42-11DD-430F-AC0C-1C9AEE53DC68@zope.com> On Dec 6, 2006, at 2:09 PM, Bill Janssen wrote: ...an argument that I appreciate, even though it mixes use cases for implementation and documentation in a way that I personally don't care for. That's fine. You conclude: > Another > problem is that the basic Python types are somewhat poorly understood, > which is the point of the exercise at > http://wiki.python.org/moin/AbstractBaseClasses. FWIW, Zope has a set of these things for sequences and mappings that have stood up reasonably well (done as interfaces, but same idea). See http://svn.zope.org/Zope3/trunk/src/zope/interface/common/ . Maybe that can help your effort. Gary From janssen at parc.com Thu Dec 7 01:18:11 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 6 Dec 2006 16:18:11 PST Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <7FB4FB42-11DD-430F-AC0C-1C9AEE53DC68@zope.com> References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> <06Dec4.132817pst."58648"@synergy1.parc.xerox.com> <20061206091118.B30563@tribble.ilrt.bris.ac.uk> <06Dec6.110949pst."58648"@synergy1.parc.xerox.com> <7FB4FB42-11DD-430F-AC0C-1C9AEE53DC68@zope.com> Message-ID: <06Dec6.161820pst."58648"@synergy1.parc.xerox.com> > FWIW, Zope has a set of these things for sequences and mappings that > have stood up reasonably well (done as interfaces, but same idea). > See http://svn.zope.org/Zope3/trunk/src/zope/interface/common/ . > Maybe that can help your effort. Thanks, Gary. Interesting effort. I'll study it further this weekend when I get the time. Bill From jcarlson at uci.edu Thu Dec 7 01:27:20 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 06 Dec 2006 16:27:20 -0800 Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <9e804ac0612061359k17fdcc5ds8fe408b8ae734a81@mail.gmail.com> References: <20061206093755.M30563@tribble.ilrt.bris.ac.uk> <9e804ac0612061359k17fdcc5ds8fe408b8ae734a81@mail.gmail.com> Message-ID: <20061206162219.976F.JCARLSON@uci.edu> "Thomas Wouters" wrote: > On 12/6/06, Jan Grant wrote: > > On Mon, 4 Dec 2006, Ben Wing wrote: > > > > > as a result, i imagine there's a strong urge to just hardcode the name > > > of the parent > > ^^^^^^^^^^ > > > > > -- super.meth(args) calls the superclass method `meth' > > ^^^^^^^^^^^^^^ > > > > Python supports multiple inheritance, unlike Java; the design mantra is > > "explicit is better than implicit" and "ambiguity should be an error". > > Two! The two design mantras are... > > > You forget that that's actually what super() is for. It does the right thing > in the case of MI (and every other case, in fact :-) Except for this one: >>> class foo(object): ... pass ... >>> class bar(foo): ... def method(self): ... super(bar, self).method() ... >>> bar().method() Traceback (most recent call last): File "", line 1, in ? File "", line 3, in method AttributeError: 'super' object has no attribute 'method' >>> Because of this kind of thing, I do my best to never produce code that has a diamond inheritance structure (except for object), and always use things like foo.method(self). - Josiah From thomas at python.org Thu Dec 7 01:44:18 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 6 Dec 2006 16:44:18 -0800 Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <20061206162219.976F.JCARLSON@uci.edu> References: <20061206093755.M30563@tribble.ilrt.bris.ac.uk> <9e804ac0612061359k17fdcc5ds8fe408b8ae734a81@mail.gmail.com> <20061206162219.976F.JCARLSON@uci.edu> Message-ID: <9e804ac0612061644q3f143915wf7f5566448e704ba@mail.gmail.com> On 12/6/06, Josiah Carlson wrote: > > > "Thomas Wouters" wrote: > > On 12/6/06, Jan Grant wrote: > > > On Mon, 4 Dec 2006, Ben Wing wrote: > > > > > > > as a result, i imagine there's a strong urge to just hardcode the > name > > > > of the parent > > > ^^^^^^^^^^ > > > > > > > -- super.meth(args) calls the superclass method `meth' > > > ^^^^^^^^^^^^^^ > > > > > > Python supports multiple inheritance, unlike Java; the design mantra > is > > > "explicit is better than implicit" and "ambiguity should be an error". > > > Two! The two design mantras are... > > > > > > You forget that that's actually what super() is for. It does the right > thing > > in the case of MI (and every other case, in fact :-) > > Except for this one: > > >>> class foo(object): > ... pass > ... > >>> class bar(foo): > ... def method(self): > ... super(bar, self).method() > ... > >>> bar().method() > Traceback (most recent call last): > File "", line 1, in ? > File "", line 3, in method > AttributeError: 'super' object has no attribute 'method' > >>> I'm not sure what makes you say that. There is no superclass method 'method', so 'AttributeError' seems like the Right Thing to me. Because of this kind of thing, I do my best to never produce code that > has a diamond inheritance structure (except for object), and always use > things like foo.method(self). Which would have given you the same AttributeError. If you want co-operative MI classes, you need to do more than just use super(): you need a baseclass that provides the 'interface', if you will, that you wish to co-operate in. For many of the standard __hooks__, object does this sensibly. For other methods, not so much. I personally think that this is correct behaviour; I wouldn't want 'return super(ThisClass, self).__gettiem__(item)' to decide to return, say, None for no apparent reason (if the typo was apparent, I wouldn't have made it :). AttributeError is the right thing. Avoiding MI is certainly a good option, provided you're able to enforce that policy somehow. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061206/57ba934a/attachment.htm From jcarlson at uci.edu Thu Dec 7 02:47:21 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Wed, 06 Dec 2006 17:47:21 -0800 Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <9e804ac0612061644q3f143915wf7f5566448e704ba@mail.gmail.com> References: <20061206162219.976F.JCARLSON@uci.edu> <9e804ac0612061644q3f143915wf7f5566448e704ba@mail.gmail.com> Message-ID: <20061206171958.9776.JCARLSON@uci.edu> "Thomas Wouters" wrote: > On 12/6/06, Josiah Carlson wrote: > > "Thomas Wouters" wrote: > > > You forget that that's actually what super() is for. It does the right > > thing > > > in the case of MI (and every other case, in fact :-) > > > > Except for this one: > > > > >>> class foo(object): > > ... pass > > ... > > >>> class bar(foo): > > ... def method(self): > > ... super(bar, self).method() > > ... > > >>> bar().method() > > Traceback (most recent call last): > > File "", line 1, in ? > > File "", line 3, in method > > AttributeError: 'super' object has no attribute 'method' > > >>> > > > I'm not sure what makes you say that. There is no superclass method > 'method', so 'AttributeError' seems like the Right Thing to me. I agree with you, but it makes it *very* difficult to write properly behaving cooperative super calls. In fact, if one inherits at any point from object, one must necessarily use the following pattern in every class: try: _method = super(cls, obj).method except AttributeError: pass else: result = _method(...) How is this reasonable or from an "I want to write correct code" standpoint? I would argue that it isn't reasonable, and is in fact insane. I don't want to write that, but to prevent AttributeErrors from cropping up in arbitrary locations, it is necessary. > > Because of this kind of thing, I do my best to never produce code that > > has a diamond inheritance structure (except for object), and always use > > things like foo.method(self). > > > Which would have given you the same AttributeError. No it wouldn't have, because I wouldn't have written foo.method(self) bar.method(). I was using it as an example of class.method(self) calling semantics. I am ([un]fortunately) anal-retentive enough to pre-check my calls. > If you want co-operative MI classes, you need to do more than just use > super(): you need a baseclass that provides the 'interface', if you will, > that you wish to co-operate in. For many of the standard __hooks__, object Certainly that is a solution, but it kills one of the major uses of multiple inheritance: mixins. Not all mixin classes subclass from the *one and only one base class that doesn't call super*. An example of such is in the standard library; SocketServer and its derivatives (or whatever it is, I can never remember). I think about the only thing in common is perhaps their subclassing of object. > Avoiding MI is certainly a good option, provided you're able to enforce that > policy somehow. Actually, I use MI quite often, I just verify that I never have a diamond inheritance with anything other than object. For example: class A(X): pass class B(A): pass class C(A): pass class D(B, C): pass I never do the above unless I *know* that X is object, because then I *know* that in D.meth(self) I can use B.meth(self) and C.meth(self) (assuming that D, B, and C all implement a .meth() method). But again, I find that being anal-retentive is easier and faster than fighting with super() and its semantics. - Josiah From janssen at parc.com Thu Dec 7 05:21:11 2006 From: janssen at parc.com (Bill Janssen) Date: Wed, 6 Dec 2006 20:21:11 PST Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <20061206171958.9776.JCARLSON@uci.edu> References: <20061206162219.976F.JCARLSON@uci.edu> <9e804ac0612061644q3f143915wf7f5566448e704ba@mail.gmail.com> <20061206171958.9776.JCARLSON@uci.edu> Message-ID: <06Dec6.202115pst."58648"@synergy1.parc.xerox.com> > I agree with you, but it makes it *very* difficult to write properly > behaving cooperative super calls. In fact, if one inherits at any point > from object, one must necessarily use the following pattern in every > class: > > try: > _method = super(cls, obj).method > except AttributeError: > pass > else: > result = _method(...) > > How is this reasonable or from an "I want to write correct code" > standpoint? I would argue that it isn't reasonable, and is in fact > insane. I don't want to write that, but to prevent AttributeErrors from > cropping up in arbitrary locations, it is necessary. I completely agree. Lisp has a function called "call-next-method", which is used to invoke the next method logically in the inheritance sequence. If there is no "next" method, it invokes another standard function, "no-next-method", which you can override to ignore the call, or provide a default, or whatever. And having to explicitly provide "cls" and "self" is also grotty, but necessary because the call has no handle on its context (hmmm, stack frames?). Incidentally, the Internet Archive has a nice video by Danny Bobrow up, which is a tour through the CLOS system, including some discussion of the design rationale. Anyone interested in knowing more about generic function implementations and class precedence determination might be interested in watching it. http://www.archive.org/details/DanielGB1987 Bill From python3000 at davious.org Thu Dec 7 06:52:35 2006 From: python3000 at davious.org (Dave Anderson) Date: Thu, 07 Dec 2006 00:52:35 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <06Dec3.165426pst."58648"@synergy1.parc.xerox.com> References: <457368C0.4010608@davious.org> <06Dec3.165426pst."58648"@synergy1.parc.xerox.com> Message-ID: <4577AC23.1080302@davious.org> on 12/3/2006 7:54 PM Bill Janssen wrote: >> Implementations are used to make Method Dispatching more flexible by >> allowing a class to claim that it acts like another class/type even >> though it is not derived from that class/type. > > I'm not sure just what this adds over the simple "change_class" > function that I've posted twice already. Is there an implementation > efficiency argument here? There is more to the syntax sketch than class-vouching. Duck-typing and method-subset implementation declaration/dispatching are included as equally visible in this framework. I'll explain more below. I've took a couple passes in the archives, and I haven't found the change_class posts yet. I think you are referring to lying about the __class__. Did I guess correctly? (Sorry, an archive link would help me.) Inheritance-defined Relationships ================================= I appreciate the cleanliness of the idea of relating all classes through multiple inheritance of either concrete or abstract/type classes. A subset of the syntax sketch supports this (basically as multi-methods dispatching by implementations that happen to only use types). Other-defined Relationships =========================== Still there are common, if unrefined, conventions of using python I'd like to see supported formally as well. Has-Method Contracts -------------------- The first thing I loved about python (coming from java), was this realization that I was no longer need be caught up in the inheritance game. If I wanted to use an unspoken convention that I could get a special string from every object just by defining name() in each of those objects, that's all I had to do. It didn't matter which class tree it came from. Looking at the syntax sketch, this can be formalized; but, its a low level of formality that fits the context. # (I used Generic.name in my original post, but that was a mistake. # Sorry for my confusion, Phillip) def get_name(arg: AnyClass.name): # AnyClass will magically have return arg.name() # a reference to any method name # somehow further refinement of this would be: class HasName: def name(self): ... def get_name(arg: HasName): return arg.name() but this refinement (in the syntax sketch), would force implementers to declare the following implementation: class X: implements HasName def name(self): ... Has Class-specific Method Contracts ----------------------------------- Similar to Has-Method Contracts are class-specific method contracts. This is when you use a well-defined class as a way to indicate the context of a method and separate it from homonyms. def add_key_value_pair(arg: MutableContainer.add, key, value): arg[key] = value Implement (As a) vs Inherit (Is a) ---------------------------------- This is nuanced. Basically, if I have some object Car and I want to be able to use it like a dict, I don't necessarily consider it inheriting from dict (or even UserDict). That is, I don't think a Car is a dict. To me, even if Car happens to represent a dict by being able to be used as a dict, the expression of this fact feels better expressed as merely implementing dict. compare: class Car(dict): # car... is a dict? hmm, a bad ring to it ... class Car: implements dict # oh, Cars can be used like a dict, got it ... In addition to that subjective differentiation, there is a technical argument. Without something like an implements declaration, if I ever want to express a relationship to another class, but with different internals, I am forced to first define an abstract class and then inherit from that. With an implements declaration, I can say implements dict and be free of inheriting any of its internal implementation. (I concede here that I could, in fact, inherit UserDict, override all the methods, and just not call super in my init.) Even with the nicely done ABCs, is there an easy way for a class to declare that it can be used like a dict with its own implementation of dict methods? With this syntax sketch, UserDict would remain the same except: class UserDict: implements dict # dispatch can rely on this declaration ... # rather than method-checking The Consequences of Going Outside of the Hierarchy -------------------------------------------------- For all the above reasons, I want to see python support something more flexible than type-based dispatching. We can look at the method-scoped units of Implementations as a limited version of Guido's Abilities, and we can look at the class-scoped implementations as types, or classes, or also as abstract interfaces from concrete classes. Unfortunately, I am now outside of the hierarchy tree; an implementation tree would be a superset of a class's hierarchy tree. I suppose that using dispatch this way is more complex compared to a strict class-based set of relationships (because I am asking for checks not just on the class-tree, but also on the implementations off that class-tree); but, I'd rather fold current python usage into a future of type / protocol-aware methods than force current pythonists into choosing between class or no class, to type or not to type. (Small Caveat:) Using Implementations as a Development Tool ------------------------------------------------------------ Btw, if we formally differentiate that I'm declaring an implementation (versus inherit and override), I could programatically double-check my implementation with tools... class A: implements Container def len(self) ... >>> interface_compliance(A) A implements Container: methods: len missing: get, iterator > What happens when you call an existing method from one of the > (supertypes?) of an implementation? Do you have to re-implement it? > Or do you have some way of coercing the value to that type? In the syntax sketch, an Implementation is basically a list of types/class, methods, and other Implementation lists. You wouldn't call a method from an Implementation object. (If I missed the gist of these questions. Please forgive me and clarify them for me.) > > Bill > > From hasan.diwan at gmail.com Thu Dec 7 16:06:29 2006 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Thu, 7 Dec 2006 10:06:29 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <4577AC23.1080302@davious.org> References: <457368C0.4010608@davious.org> <4577AC23.1080302@davious.org> Message-ID: <2cda2fc90612070706g5467491do45e856254d0dd880@mail.gmail.com> The parametrized version of the pass keyword could be used to indicate a formal interface. On 07/12/06, Dave Anderson wrote: > > > Has-Method Contracts > -------------------- > > The first thing I loved about python (coming from java), was this > realization that I was no longer need be caught up in the inheritance > game. If I wanted to use an unspoken convention that I could get a > special string from every object just by defining name() in each of > those objects, that's all I had to do. It didn't matter which class > tree it came from. > > Looking at the syntax sketch, this can be formalized; but, its a low > level of formality that fits the context. > > # (I used Generic.name in my original post, but that was a mistake. > # Sorry for my confusion, Phillip) > > def get_name(arg: AnyClass.name): # AnyClass will magically have > return arg.name() # a reference to any method name > # somehow > > further refinement of this would be: > > class HasName: > def name(self): > ... > > def get_name(arg: HasName): > return arg.name() > > but this refinement (in the syntax sketch), would force implementers to > declare the following implementation: > > class X: > implements HasName > > def name(self): I may be missing something here, but we already have a mechanism for defining that a object defines certain methods. Could we not keep it orthogonal and have, e. g.: class super(): def __init__(self): pass def typeOfSuper(self): pass(subclass) The way pass is defined now would remain the default. A new builtin pass() would be added to support your notion of interfaces. Has Class-specific Method Contracts > ----------------------------------- > > Similar to Has-Method Contracts are class-specific method contracts. > This is when you use a well-defined class as a way to indicate the > context of a method and separate it from homonyms. > > def add_key_value_pair(arg: MutableContainer.add, key, value): > arg[key] = value def __add__(self, onemore): MutableContainer.add (self, onemore.key, onemore.value): If you need more than one function to implement '+', you can use two classes. > Implement (As a) vs Inherit (Is a) > ---------------------------------- > > This is nuanced. > Basically, if I have some object Car and I want to be able to use it > like a dict, I don't necessarily consider it inheriting from dict (or > even UserDict). That is, I don't think a Car is a dict. > > To me, even if Car happens to represent a dict by being able to be used > as a dict, the expression of this fact feels better expressed as merely > implementing dict. > > compare: > > class Car(dict): # car... is a dict? hmm, a bad ring to it > ... > > class Car: > implements dict # oh, Cars can be used like a dict, got it > ... > > In addition to that subjective differentiation, there is a technical > argument. Without something like an implements declaration, if I ever > want to express a relationship to another class, but with different > internals, I am forced to first define an abstract class and then > inherit from that. With an implements declaration, I can say implements > dict and be free of inheriting any of its internal implementation. > > (I concede here that I could, in fact, inherit UserDict, override all > the methods, and just not call super in my init.) > > Even with the nicely done ABCs, is there an easy way for a class to > declare that it can be used like a dict with its own implementation of > dict methods? With this syntax sketch, UserDict would remain the same > except: > > class UserDict: > implements dict # dispatch can rely on this declaration > ... # rather than method-checking > > > The Consequences of Going Outside of the Hierarchy > -------------------------------------------------- > > For all the above reasons, I want to see python support something more > flexible than type-based dispatching. We can look at the method-scoped > units of Implementations as a limited version of Guido's Abilities, and > we can look at the class-scoped implementations as types, or classes, or > also as abstract interfaces from concrete classes. > > Unfortunately, I am now outside of the hierarchy tree; an implementation > tree would be a superset of a class's hierarchy tree. I suppose that > using dispatch this way is more complex compared to a strict class-based > set of relationships (because I am asking for checks not just on the > class-tree, but also on the implementations off that class-tree); but, > I'd rather fold current python usage into a future of type / > protocol-aware methods than force current pythonists into choosing > between class or no class, to type or not to type. > > > (Small Caveat:) Using Implementations as a Development Tool > ------------------------------------------------------------ > > Btw, if we formally differentiate that I'm declaring an implementation > (versus inherit and override), I could programatically double-check my > implementation with tools... > > class A: > implements Container > > def len(self) > ... > > >>> interface_compliance(A) This would just scan for a parametrised "pass" keyword and check that all subclasses inheriting from it implemented that method. (If I missed the gist of these questions. Please forgive me and clarify > them for me.) > Ditto for me. -- Cheers, Hasan Diwan -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061207/e0f01890/attachment.html From fredrik at pythonware.com Thu Dec 7 16:10:44 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 7 Dec 2006 16:10:44 +0100 Subject: [Python-3000] A better way to initialize PyTypeObject References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> <45722F87.9050903@acm.org> Message-ID: Talin wrote: > OK. Its easy to implement either way. I coded up a string-based version, > but if you guys want integer constants that's fine with me; probably > best to let Fredrik & Larry tweak their patch rather than me continuing > along this path since their stuff is further along. wouldn't it be easier to just look at Larry's stuff and steal the good things, if there are any? what worries me is that people pop up over in 2.X land, come up with new ideas and provide patches, only to find that nobody cares about what they do; the 2.X:ers finds the changes too radical for Python 2.X, while the 3.X:ers are too busy with their own ideas. From python3000 at davious.org Thu Dec 7 16:37:24 2006 From: python3000 at davious.org (Dave Anderson) Date: Thu, 07 Dec 2006 10:37:24 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <2cda2fc90612070706g5467491do45e856254d0dd880@mail.gmail.com> References: <457368C0.4010608@davious.org> <4577AC23.1080302@davious.org> <2cda2fc90612070706g5467491do45e856254d0dd880@mail.gmail.com> Message-ID: <45783534.6000201@davious.org> on 12/7/2006 10:06 AM Hasan Diwan wrote: > The parametrized version of the pass keyword could be used to indicate a > formal interface. > > I may be missing something here, but we already have a mechanism for > defining that a object defines certain methods. Could we not keep it > orthogonal and have, e. g.: > > class super(): > def __init__(self): pass > > def typeOfSuper(self): pass(subclass) > > The way pass is defined now would remain the default. A new builtin > pass() would be added to support your notion of interfaces. I feel on a different page from you. My best guess it is that def typeOfSuper(self): pass(subclass) is ceding its implementation to its subclasses, kind of like what an abstract class or interface would do but I feel my guess is wrong. Please clarify. >> >> def add_key_value_pair(arg: MutableContainer.add, key, value): >> arg[key] = value > > def __add__(self, onemore): > MutableContainer.add (self, onemore.key, onemore.value): > I'm not getting the insight you are trying to convey by contrasting these two functions. The former is an example of someone writing a function whose first parameter is expected to be any object that is derived from MutableContainer (including dict), or is derived from a class that implements MutableContainer (including those implementing dict), or is derived from a class that specifically implements MutableContainer.add using a declaration like: class CanAdd: implements MutableContainer.add def add(self, key, value): ... The later seems to be a class method which uses MutableContainer's add method to add any object that has a key and a value attribute. From hasan.diwan at gmail.com Thu Dec 7 16:57:23 2006 From: hasan.diwan at gmail.com (Hasan Diwan) Date: Thu, 7 Dec 2006 10:57:23 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <45783534.6000201@davious.org> References: <457368C0.4010608@davious.org> <4577AC23.1080302@davious.org> <2cda2fc90612070706g5467491do45e856254d0dd880@mail.gmail.com> <45783534.6000201@davious.org> Message-ID: <2cda2fc90612070757r67b2df23lc95e20e94f972114@mail.gmail.com> On 07/12/06, Dave Anderson wrote: > > > I feel on a different page from you. > > My best guess it is that > > def typeOfSuper(self): > pass(subclass) > > is ceding its implementation to its subclasses, kind of like what an > abstract class or interface would do > > but I feel my guess is wrong. Please clarify. No, that is spot-on. > The former is an example of someone writing a function whose first > parameter is expected to be any object that is derived from > MutableContainer (including dict), or is derived from a class that > implements MutableContainer (including those implementing dict), or is > derived from a class that specifically implements MutableContainer.add > using a declaration like: > > class CanAdd: > implements MutableContainer.add > > def add(self, key, value): I find you're making the whole "implements" procedure, a little too complex. -- Cheers, Hasan Diwan From python3000 at davious.org Thu Dec 7 17:22:47 2006 From: python3000 at davious.org (Dave Anderson) Date: Thu, 07 Dec 2006 11:22:47 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <2cda2fc90612070757r67b2df23lc95e20e94f972114@mail.gmail.com> References: <457368C0.4010608@davious.org> <4577AC23.1080302@davious.org> <2cda2fc90612070706g5467491do45e856254d0dd880@mail.gmail.com> <45783534.6000201@davious.org> <2cda2fc90612070757r67b2df23lc95e20e94f972114@mail.gmail.com> Message-ID: <45783FD7.4040900@davious.org> on 12/7/2006 10:57 AM Hasan Diwan wrote: > On 07/12/06, Dave Anderson wrote: >> >> >> I feel on a different page from you. >> >> My best guess it is that >> >> def typeOfSuper(self): >> pass(subclass) >> >> is ceding its implementation to its subclasses, kind of like what an >> abstract class or interface would do >> >> but I feel my guess is wrong. Please clarify. > > No, that is spot-on. The Implementations syntax sketch has no need for abstract classes, since concrete classes serve as the interfaces. Still, in the syntax, one could simply define a class with a bunch of empty methods, and it would be usable as an abstract class. >> The former is an example of someone writing a function whose first >> parameter is expected to be any object that is derived from >> MutableContainer (including dict), or is derived from a class that >> implements MutableContainer (including those implementing dict), or is >> derived from a class that specifically implements MutableContainer.add >> using a declaration like: >> >> class CanAdd: >> implements MutableContainer.add >> >> def add(self, key, value): > > I find you're making the whole "implements" procedure, a little too > complex. Ouch. Touch?. :( Simple things simple. Complex things possible. From janssen at parc.com Thu Dec 7 17:25:26 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 7 Dec 2006 08:25:26 PST Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <4577AC23.1080302@davious.org> References: <457368C0.4010608@davious.org> <06Dec3.165426pst."58648"@synergy1.parc.xerox.com> <4577AC23.1080302@davious.org> Message-ID: <06Dec7.082531pst."58648"@synergy1.parc.xerox.com> Dave, thanks. > To me, even if Car happens to represent a dict by being able to be used > as a dict, the expression of this fact feels better expressed as merely > implementing dict. > > compare: > > class Car(dict): # car... is a dict? hmm, a bad ring to it > ... > > class Car: > implements dict # oh, Cars can be used like a dict, got it > ... OK, I guess the difference is that, for me, (1) this doesn't happen a lot (ever?), and (2) I don't feel that the nuance here is important enough to add another mechanism to the language which needs to be supported. I'd rather just use type inheritance. > Without something like an implements declaration, if I ever > want to express a relationship to another class, but with different > internals, I am forced to first define an abstract class and then > inherit from that. With an implements declaration, I can say implements > dict and be free of inheriting any of its internal implementation. That's why it's important for Python to have a clear set of base interface types for people to use. > In the syntax sketch, an Implementation is basically a list of > types/class, methods, and other Implementation lists. That sounds a whole lot like a Python 2.5 class, to me. Bill From python3000 at davious.org Thu Dec 7 18:05:22 2006 From: python3000 at davious.org (Dave Anderson) Date: Thu, 07 Dec 2006 12:05:22 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <06Dec7.082531pst."58648"@synergy1.parc.xerox.com> References: <457368C0.4010608@davious.org> <06Dec3.165426pst."58648"@synergy1.parc.xerox.com> <4577AC23.1080302@davious.org> <06Dec7.082531pst."58648"@synergy1.parc.xerox.com> Message-ID: <457849D2.9010402@davious.org> on 12/7/2006 11:25 AM Bill Janssen wrote: >> To me, even if Car happens to represent a dict by being able to be used >> as a dict, the expression of this fact feels better expressed as merely >> implementing dict. >> >> compare: >> >> class Car(dict): # car... is a dict? hmm, a bad ring to it >> ... >> >> class Car: >> implements dict # oh, Cars can be used like a dict, got it >> ... > > OK, I guess the difference is that, for me, (1) this doesn't happen a > lot (ever?), and (2) I don't feel that the nuance here is important > enough to add another mechanism to the language which needs to be > supported. I'd rather just use type inheritance. I'm neither surprised nor do I frown upon your preferences. I'm just letting you know that I look at python in a different way than you do. The intent of the sketch is to bring together different camps of pythonists by presenting a framework that recognizes those that would use python in different ways. > >> Without something like an implements declaration, if I ever >> want to express a relationship to another class, but with different >> internals, I am forced to first define an abstract class and then >> inherit from that. With an implements declaration, I can say implements >> dict and be free of inheriting any of its internal implementation. > > That's why it's important for Python to have a clear set of base > interface types for people to use. Here, we are focused on base types. I like what ABCs are going to allow. And I've incorporated ABC's into some of the syntax examples. As a note, I'll repeat the point I made last post. As a regular python user, I'm well acquainted with dict. But, looking at the ABC page right now, to implement Dict, I'll need to inherit Mapping and MutableContainer. If is that Dict is an abstract class itself and dict is a concrete implementation of Dict, then I again bring up the benefit of being able to say something like "implements dict" versus class A(Dict) while in my head I'm thinking: I want A to be used just like dict. This example is coming from the ABC context of things. I'd expect, with just an inheritance-based solution, a lot of user code out there would have to deal with the same Interface/well-known class dilemma as well, where they would have a core set of classes that they'd feel obligated to copy into interfaces, just so third party code could easily inter-operate with their code-base. > >> In the syntax sketch, an Implementation is basically a list of >> types/class, methods, and other Implementation lists. > > That sounds a whole lot like a Python 2.5 class, to me. I'm surprised that your understanding of what an Implementation (as defined in the syntax sketch) is doesn't seem to be what I'm thinking of. Also, I'm not sure I understand what you mean by it sounding like a Python 2.5 class. Please explain further. > > Bill From janssen at parc.com Thu Dec 7 18:22:52 2006 From: janssen at parc.com (Bill Janssen) Date: Thu, 7 Dec 2006 09:22:52 PST Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <457849D2.9010402@davious.org> References: <457368C0.4010608@davious.org> <06Dec3.165426pst."58648"@synergy1.parc.xerox.com> <4577AC23.1080302@davious.org> <06Dec7.082531pst."58648"@synergy1.parc.xerox.com> <457849D2.9010402@davious.org> Message-ID: <06Dec7.092257pst."58648"@synergy1.parc.xerox.com> > But, looking > at the ABC page right now, to implement Dict, I'll need to inherit > Mapping and MutableContainer. Dave, I completely feel your pain. I think dict is overly complex (and Mapping). Almost nobody needs a whole "dict" -- it's become a grab bag for handy tricks to perform with mapped sets. After all, even Container implements a "mapping", via the "get" method. I think we need simpler builds on Container that are what people use (without going down Java's "twisty little maze of twisty little mazes" class library path). I also think that the interface to Sequence could be made considerably simpler by making the Slice object more explicit. Right now, there's a lot of syntactic sugar on the interfaces to "dict" and "list", and that sugar, while useful, is also confusing. Bill From talin at acm.org Thu Dec 7 18:45:45 2006 From: talin at acm.org (Talin) Date: Thu, 07 Dec 2006 09:45:45 -0800 Subject: [Python-3000] A better way to initialize PyTypeObject In-Reply-To: References: <456BFFBF.9000208@acm.org> <456C85E9.8000905@acm.org> <456D357A.8030707@acm.org> <4571D136.2080303@acm.org> <45722F87.9050903@acm.org> Message-ID: <45785349.4060207@acm.org> Fredrik Lundh wrote: > Talin wrote: > >> OK. Its easy to implement either way. I coded up a string-based version, >> but if you guys want integer constants that's fine with me; probably >> best to let Fredrik & Larry tweak their patch rather than me continuing >> along this path since their stuff is further along. > > wouldn't it be easier to just look at Larry's stuff and steal the good things, if there > are any? what worries me is that people pop up over in 2.X land, come up with > new ideas and provide patches, only to find that nobody cares about what they > do; the 2.X:ers finds the changes too radical for Python 2.X, while the 3.X:ers > are too busy with their own ideas. At this point, I'm not sure what to do, as it seems that the discussion has gone in an entirely different direction. I'm referring to the idea of using a preprocessor to write extension module code, which is something I'm not at all interested in. So I guess I'll just wait and see what happens. -- Talin From talin at acm.org Thu Dec 7 19:12:44 2006 From: talin at acm.org (Talin) Date: Thu, 07 Dec 2006 10:12:44 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> Message-ID: <4578599C.303@acm.org> Phillip J. Eby wrote: > At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: >> My point is that an interface can *document* (at least in English) a >> "contract" about the invariants between operations. While I'm not into >> enforcing or verifying such contracts, I'm very interested in >> documenting them. For example, something that has "mapping" behavior >> has a very different relationship between x[y] and "y in x" than >> something that has "sequence" behavior. > > I assumed this didn't need answering. If you're using the interface solely > for documentation, then a namespace-oriented interface suffices to provide it. I'm guessing that Guido's use of the word 'document' means something more than just conveying information to a human reader. From what I can tell, the argument boils down to this: You are saying that a class is merely the sum of its attributes and methods, and Guido is saying that it's not. For example: You cannot deduce by examining the methods of a list that the keys are required to be consecutive integers starting from 0. Lists have __getitem__, but so do maps; They support len(), but so do maps; and so on. Thus, lists embody a particular behavior contract (called 'concept' in C++ template land). (The word 'interface' and 'ability' has been used in this context, but that's somewhat misleading. [1] [2]) The concept is supported by the methods of the class, but is only incompletely derivable from them. I think what Guido is looking for is a way to signal that a class obeys a given behavioral contract, and a way to inspect the object, both at runtime *and* to a human reader, and discover what contracts are supported by the class. On the other side of the argument is the fear that such explicitness will lead to Java-style type bureaucracy, and lead to the death of implicit, a la 'duck', typing. The response to that, I think, is that we're not looking to replace duck typing but only to supplement it - in other words, this extra information would only be used to convey that which is not derivable by inspection of types and methods. [1] The reason that I think 'ability' is misleading is because what we're really talking about is a *disability* - a limitation on what inputs a class can accept. Also, one tends to think of the abilities of a class as its methods, whereas what we're talking about here is a way to describe the relationships *between* the methods. [2] The reason I think 'interface' are misleading is due to its Java connotations, in which an interface is a description of a bunch of methods as well as a signal of an abstract type, but in this case I think what they want is *only* the contract signal and not the method declarations. -- Talin From guido at python.org Thu Dec 7 20:01:22 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 7 Dec 2006 13:01:22 -0600 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <4578599C.303@acm.org> References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <4578599C.303@acm.org> Message-ID: That sounds like a reasonable summary to me. One nit below. On 12/7/06, Talin wrote: > Phillip J. Eby wrote: > > At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: > >> My point is that an interface can *document* (at least in English) a > >> "contract" about the invariants between operations. While I'm not into > >> enforcing or verifying such contracts, I'm very interested in > >> documenting them. For example, something that has "mapping" behavior > >> has a very different relationship between x[y] and "y in x" than > >> something that has "sequence" behavior. > > > > I assumed this didn't need answering. If you're using the interface solely > > for documentation, then a namespace-oriented interface suffices to provide it. > > I'm guessing that Guido's use of the word 'document' means something > more than just conveying information to a human reader. > > From what I can tell, the argument boils down to this: You are saying > that a class is merely the sum of its attributes and methods, and Guido > is saying that it's not. > > For example: You cannot deduce by examining the methods of a list that > the keys are required to be consecutive integers starting from 0. Lists > have __getitem__, but so do maps; They support len(), but so do maps; > and so on. > > Thus, lists embody a particular behavior contract (called 'concept' in > C++ template land). (The word 'interface' and 'ability' has been used in > this context, but that's somewhat misleading. [1] [2]) The concept is > supported by the methods of the class, but is only incompletely > derivable from them. > > I think what Guido is looking for is a way to signal that a class obeys > a given behavioral contract, and a way to inspect the object, both at > runtime *and* to a human reader, and discover what contracts are > supported by the class. > > On the other side of the argument is the fear that such explicitness > will lead to Java-style type bureaucracy, and lead to the death of > implicit, a la 'duck', typing. > > The response to that, I think, is that we're not looking to replace duck > typing but only to supplement it - in other words, this extra > information would only be used to convey that which is not derivable by > inspection of types and methods. > > [1] The reason that I think 'ability' is misleading is because what > we're really talking about is a *disability* - a limitation on what > inputs a class can accept. Also, one tends to think of the abilities of > a class as its methods, whereas what we're talking about here is a way > to describe the relationships *between* the methods. > > [2] The reason I think 'interface' are misleading is due to its Java > connotations, in which an interface is a description of a bunch of > methods as well as a signal of an abstract type, but in this case I > think what they want is *only* the contract signal and not the method > declarations. I don't see it this way. The methods are part of the contract -- the contract talks about specific methods *and* how they work together. I still believe this is also what Java interfaces are for, at least in the best of circumstances, like the collections framework (maybe I should ask Josh Bloch about this). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ntoronto at cs.byu.edu Thu Dec 7 20:26:45 2006 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Thu, 07 Dec 2006 12:26:45 -0700 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <4578599C.303@acm.org> Message-ID: <45786AF5.9050608@cs.byu.edu> Guido van Rossum wrote: > That sounds like a reasonable summary to me. One nit below. > Ditto. It was nice to see all the ambiguity cleared up in one fell swoop. Nicely done. > On 12/7/06, Talin wrote: > >> [2] The reason I think 'interface' are misleading is due to its Java >> connotations, in which an interface is a description of a bunch of >> methods as well as a signal of an abstract type, but in this case I >> think what they want is *only* the contract signal and not the method >> declarations. >> > > I don't see it this way. The methods are part of the contract -- the > contract talks about specific methods *and* how they work together. I > still believe this is also what Java interfaces are for, at least in > the best of circumstances, like the collections framework (maybe I > should ask Josh Bloch about this). In most circumstances that's correct. An interface generally connotes some kind of behavior, often very specific, and sometimes even spelled out in detail in the interface's Javadoc. Here's an example from interface Comparable: This interface imposes a total ordering on the objects of each class that implements it. This ordering is referred to as the class's natural ordering, and the class's compareTo method is referred to as its natural comparison method. Lists (and arrays) of objects that implement this interface can be sorted automatically by Collections.sort (and Arrays.sort). Objects that implement this interface can be used as keys in a sorted map or elements in a sorted set, without the need to specify a comparator. The natural ordering for a class C is said to be /consistent with equals/ if and only if (e1.compareTo((Object)e2) == 0) has the same boolean value as e1.equals((Object)e2) for every e1 and e2 of class C. Note that null is not an instance of any class, and e.compareTo(null) should throw a NullPointerException even though e.equals(null) returns false. It is strongly recommended (though not required) that natural orderings be consistent with equals. This is so because sorted sets (and sorted maps) without explicit comparators behave "strangely" when they are used with elements (or keys) whose natural ordering is inconsistent with equals. In particular, such a sorted set (or sorted map) violates the general contract for set (or map), which is defined in terms of the equals method. So when you write a method that takes a Comparable, you're generally free to assume that 1) its compareTo() defines a total ordering, and 2) its compareTo() and equals() are consistent with each other. Other examples of interfaces having semantic meaning are littered throughout the API. But most of the time the behavior is implicit because the author of the interface was too lazy to spell out how classes implementing an interface are expected to behave, or assumed it would be clear from the method signatures. My experience is limited mostly to collections, Swing, and database APIs, but from what I've seen, interfaces nearly *always* connote behavior. Neil From tim.hochberg at ieee.org Thu Dec 7 21:18:05 2006 From: tim.hochberg at ieee.org (Tim Hochberg) Date: Thu, 07 Dec 2006 13:18:05 -0700 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <4578599C.303@acm.org> References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <4578599C.303@acm.org> Message-ID: Talin wrote: > Phillip J. Eby wrote: >> At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: >>> My point is that an interface can *document* (at least in English) a >>> "contract" about the invariants between operations. While I'm not into >>> enforcing or verifying such contracts, I'm very interested in >>> documenting them. For example, something that has "mapping" behavior >>> has a very different relationship between x[y] and "y in x" than >>> something that has "sequence" behavior. >> I assumed this didn't need answering. If you're using the interface solely >> for documentation, then a namespace-oriented interface suffices to provide it. > > I'm guessing that Guido's use of the word 'document' means something > more than just conveying information to a human reader. > > From what I can tell, the argument boils down to this: You are saying > that a class is merely the sum of its attributes and methods, and Guido > is saying that it's not. That's not my interpretation. Let me throw out my spin on my understanding of Philip's position. Here goes: to the extent that the semantics aren't clear from the methods it is sufficient, and preferable, to attach the semantic information to the methods rather their associated class. > For example: You cannot deduce by examining the methods of a list that > the keys are required to be consecutive integers starting from 0. Lists > have __getitem__, but so do maps; They support len(), but so do maps; > and so on. No, but if you tag the methods of the class then it appears that this works. > > Thus, lists embody a particular behavior contract (called 'concept' in > C++ template land). (The word 'interface' and 'ability' has been used in > this context, but that's somewhat misleading. [1] [2]) The concept is > supported by the methods of the class, but is only incompletely > derivable from them. > > I think what Guido is looking for is a way to signal that a class obeys > a given behavioral contract, and a way to inspect the object, both at > runtime *and* to a human reader, and discover what contracts are > supported by the class. Let me throw out an examples of how this could look. @implements(sequence.__getitem__, sequence.__len__) class MySequence: #.... @implements(mapping.__getitem__, mapping.keys, mapping.__len__) class MyMapping: #.... #... # supports reads better than has_ability to me in this context. if supports(someobject, mapping.__getitem__): # do something else: # do something else Assuming that class decorators make it in, I believe that this could be implemented in terms of generic functions or a simple registry with no additional syntactic support. On the plus side, this operates at the same level of granularity as duck typing and would fit more naturally into the existing duck typing scheme IMO. Also, it avoids having to muck with bases when retrofitting classes to support a [concept/interface/ability], which gives me the heebie jeebies. Let's move on to Guido's set example: [Guido writes] > Now I'd like to introduce a standard set implementation. It implements > all those operations. But I'd like it to automatically accept operands > of other set implementations, as long as they claim the relevant > ability. For example, in Python 2.4, the built-in set implementation's > __and__ is defined roughly like this (ignoring some optimizations, > like that it's written in C :-): > > # This a method of class 'set', a concrete set implementation > > def __and__(self, other): > if not isinstance(other, baseset): > return NotImplemented > result = set() > for elem in self: > if elem in other: > result.add(elem) > return result This could be written: def __and__(self, other): if not supports(other, set.__contains__): returns NotImplemented result = set() for elem in self: if elem in other: result.add(elem) return result Where the other class has been marked as implementing set.__contains__. For example: @implements(set.__contains__) class MySet: #.... > But I'd like to be able to write it like this instead (still inside > the 'set' class): > > def __and__(self, other: MinimalSet): > result = set() > for elem in self: > if elem in other: > result.add(elem) > return result There are several ways one could translate the above. If we let a tuple of these markers denote the union of all those types, one way would be to simply define: MinimalSet = (set.__contains__,) And then the above definition could remain unchanged. Similarly: IterableSet = (set.__contains__,set.__iter__) StandardSet = (set.__contains__,set.__iter__,set.__and__,...) This gives you the ability to express the three main types of sets, but I could also indicate, for example, that I have an IterableSet that also supports '&' using (set.__contains__,set.__iter__,set.__and__). In my experience this type of partial implementation is pretty common: since I don't always need all of the pieces, I just implement those parts that I do. If I can only spell three types of set, I'm forced to do extra work to implement the other methods or lie about what the class supports, neither of which is ideal. -tim From baptiste13 at altern.org Thu Dec 7 22:34:59 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Thu, 07 Dec 2006 22:34:59 +0100 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <4578599C.303@acm.org> Message-ID: Tim Hochberg a ?crit : > > Here goes: to the extent that the semantics aren't clear from the > methods it is sufficient, and preferable, to attach the semantic > information to the methods rather their associated class. > which imho is the whole of the question: which level of granularity is desirable for defining semantics? If the granularity is in too big chunks, it will get in the way of the user the same way isinstance() would. If it is too small, lots of time and energy will be waisted in defining contracts that nobody relies on (plus said contracts might well be inconsistent if they don't have actual users). Imho finding this right level of granularity is part of the design of an API, and can only be decided be a human on a case by case basis. There is no silver bullet here: sometimes methods are the right level of granularity, sometimes classes, sometimes inbetween. Thus the language should be neutral on that point. Cheers, BC From steven.bethard at gmail.com Thu Dec 7 23:13:24 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Thu, 7 Dec 2006 15:13:24 -0700 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <4578599C.303@acm.org> Message-ID: On 12/7/06, Tim Hochberg wrote: > Let me throw out an examples of how this could look. > > @implements(sequence.__getitem__, sequence.__len__) > class MySequence: > #.... > > @implements(mapping.__getitem__, mapping.keys, mapping.__len__) > class MyMapping: > #.... > > #... > > # supports reads better than has_ability to me in this context. > if supports(someobject, mapping.__getitem__): > # do something FWLIW, I like the concept of a ``supports()`` method like this, and I think it shouldn't be too foreign to Python users syntactically. I'm sure I've seen code like this before:: if hasattr(someobject, '__getitem__'): # do something And the only real difference between the ``hasattr()`` code and the ``supports()`` code is that the ``supports()`` code not only guarantees the existence of the method, but guarantees that the method conforms to the expected semantics. For the sake of DRY, I'd prefer to see:: class MyMapping: def mapping.__getitem__(self, key): ... def mapping.__len__(self): ... instead of:: @implements(mapping.__getitem__, mapping.__len__) class MyMapping: def __getitem__(self, key): ... def __len__(self): ... but since we don't have syntactic support for either of these variants right now, I think what would be most helpful is a proof-of-concept patch that worked something like:: class MyMapping: def __getitem__(self, key): ... def __len__(self): ... implements(MyMapping, mapping.__getitem__, mapping.__len__) STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From greg.ewing at canterbury.ac.nz Fri Dec 8 00:40:07 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 08 Dec 2006 12:40:07 +1300 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <4577AC23.1080302@davious.org> References: <457368C0.4010608@davious.org> <06Dec3.165426pst.58648@synergy1.parc.xerox.com> <4577AC23.1080302@davious.org> Message-ID: <4578A657.2090105@canterbury.ac.nz> Dave Anderson wrote: > def get_name(arg: AnyClass.name): # AnyClass will magically have > return arg.name() # a reference to any method name > # somehow What if you want to say that arg has more than one such method? -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing at canterbury.ac.nz +--------------------------------------+ From python3000 at davious.org Fri Dec 8 05:19:55 2006 From: python3000 at davious.org (Dave Anderson) Date: Thu, 07 Dec 2006 23:19:55 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <4578A657.2090105@canterbury.ac.nz> References: <457368C0.4010608@davious.org> <06Dec3.165426pst.58648@synergy1.parc.xerox.com> <4577AC23.1080302@davious.org> <4578A657.2090105@canterbury.ac.nz> Message-ID: <4578E7EB.1040901@davious.org> on 12/7/2006 6:40 PM Greg Ewing wrote: > Dave Anderson wrote: > >> def get_name(arg: AnyClass.name): # AnyClass will magically have >> return arg.name() # a reference to any method name >> # somehow > > What if you want to say that arg has more > than one such method? > def get_name_height(arg: ArgClass.name, AnyClass.height): return arg.name(), arg.height() alternatively: name_height = Implementation(ArgClass.name, AnyClass.height) def get_name_height(arg: name_height): return arg.name(), arg.height() From rrr at ronadam.com Fri Dec 8 05:38:37 2006 From: rrr at ronadam.com (Ron Adam) Date: Thu, 07 Dec 2006 22:38:37 -0600 Subject: [Python-3000] Types related to Syntax Message-ID: I don't normally start threads, but I wasn't sure who to post this as a reply to. This thought or direction of discussion, seems like it may be a useful viewpoint in the type/interface topics. Python's syntax interacts strongly with a number of basic types. For example (and in general) a for loop requires something that can iterated. The * and ** require or produce lists and dictionaries when used in a function def or call signature. The math operators need numerical types, or a sequence in the case of '+'. The [:] notation indicates a slice object is used. The "", [], (), and {} return or consume specific types of objects. And of course there is the common basic "object". I'm wonder how many places in pythons syntax requires or produces specific types. And if listing and organizing these commonly used "syntactically necessary" types would be useful to determine the minimum list of ABC's. (More can always be added later.) One approach to a class tree is to list all the basic parts of possibly commonly used various objects and organize them in a tree such as how java has done. It seems like that's not what we want from the discussion so far. The approach suggested above, takes a much narrower view and would start by what is only currently necessary by the language's current syntax. These are places where you may possibly need to test for type-ness. I also wonder if there should be a distinction between a "language type" object, and a "data type" object. A language type object would express something needed by the language itself, where a data type object would be something used to express or interact with information. Cheers, Ron From talin at acm.org Fri Dec 8 10:54:34 2006 From: talin at acm.org (Talin) Date: Fri, 08 Dec 2006 01:54:34 -0800 Subject: [Python-3000] Metaclasses in Py3K Message-ID: <4579365A.3050008@acm.org> I'd like to see metaclasses have additional capabilities in Py3K, and one thing I'd like is for metaclasses to be able to have access to the order of declarations within a class. If I understand correctly, in the current Python a class definition is essentially a suite with ordinary local variables; once that suite finishes executing, the locals() dict becomes the class's __dict__. The metaclass slips in between these two phases of construction, in that it can examine or modify the dict before it becomes a class. Suppose, however, that the metaclass could hook into the get/set operations of the locals() dict itself. In other words, the __metaclass__ statement would have a side-effect of replacing the locals() dict with a mapping object supplied by the metaclass. This would allow the metaclass to intercept individual variable assignments as they occur, rather than taking the whole dict in at once. I can think of a ton of applications for this. For example, you could implement the functionality of __slots__ as a metaclass, allowing you to define the slots using regular Python syntax rather than having to pass in a list of name strings. You could define C structures using regular Python syntax. You could declare CORBA or other IDL interfaces using Python syntax and appropriate decorators. You might even be able to do things like enumerations using metaclasses. -- Talin From bingham at cenix-bioscience.com Fri Dec 8 11:15:00 2006 From: bingham at cenix-bioscience.com (Aaron Bingham) Date: Fri, 08 Dec 2006 11:15:00 +0100 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <1d85506f0611240718s364f83c2ya3e9e12e2ca7354c@mail.gmail.com> <8979205067466383560@unknownmsgid> <1d85506f0611301245x357bcf86w24b6338dd8bde716@mail.gmail.com> <8653224966967895047@unknownmsgid> <79990c6b0612010117o35d5d5eeqccfb7682b87dab66@mail.gmail.com> <-6849499535038592093@unknownmsgid> <1d85506f0612010908p4bf2d055p6cedf7123cc0e06d@mail.gmail.com> <79990c6b0612011134n27b792a0me24b9ba799b2604a@mail.gmail.com> <5.1.1.6.0.20061201154644.03d52ee8@sparrow.telecommunity.com> <5.1.1.6.0.20061201181137.0440dfc8@sparrow.telecommunity.com> <5.1.1.6.0.20061205111223.01f6d9c0@sparrow.telecommunity.com> <4578599C.303@acm.org> Message-ID: <45793B24.5080504@cenix-bioscience.com> Tim Hochberg wrote: > Talin wrote: > >> Phillip J. Eby wrote: >> >>> At 09:59 AM 12/5/2006 -0600, Guido van Rossum wrote: >>> >>>> My point is that an interface can *document* (at least in English) a >>>> "contract" about the invariants between operations. While I'm not into >>>> enforcing or verifying such contracts, I'm very interested in >>>> documenting them. For example, something that has "mapping" behavior >>>> has a very different relationship between x[y] and "y in x" than >>>> something that has "sequence" behavior. >>>> >>> I assumed this didn't need answering. If you're using the interface solely >>> for documentation, then a namespace-oriented interface suffices to provide it. >>> >> I'm guessing that Guido's use of the word 'document' means something >> more than just conveying information to a human reader. >> >> From what I can tell, the argument boils down to this: You are saying >> that a class is merely the sum of its attributes and methods, and Guido >> is saying that it's not. >> > > That's not my interpretation. Let me throw out my spin on my > understanding of Philip's position. > > Here goes: to the extent that the semantics aren't clear from the > methods it is sufficient, and preferable, to attach the semantic > information to the methods rather their associated class. > Of course we can attach the pre- and post-conditions of a method to the method itself, but there is semantic information that logically belongs with the class, namely, the class invariant. If you can't attach the invariant to the class itself, you will be forced to repeat the invariant as a postcondition for every method. Regards, -- -------------------------------------------------------------------- Aaron Bingham Senior Software Engineer Cenix BioScience GmbH -------------------------------------------------------------------- From talin at acm.org Fri Dec 8 11:15:19 2006 From: talin at acm.org (Talin) Date: Fri, 08 Dec 2006 02:15:19 -0800 Subject: [Python-3000] 'defop' syntax with no new keywords Message-ID: <45793B37.5060702@acm.org> Another mini-brainstorm, which is how to represent PJE's 'defop' syntax without adding any keywords: Alternative 1. Omitting the function name in 'def' statements. Let us define the behavior of the 'def' statement as follows: The function name in the 'def' statement is optional - if the function name is absent, then def behaves exactly as it does now, *except* that the assignment of the function object to the symbol name is skipped. In particular, any function decorators are still invoked, however the result of those decorators are thrown away. So to overload the 'len' operator: class Foo: @overload(len): def (self): ... compute len( self ) ... The idea here is that it's up to the *decorator* to store the function object somewhere - we don't automatically store it if there's no function name supplied. In this example, the argument to the 'overload' decorator is a reference to the global 'len' generic function. The 'overload' in this case takes the reference to the resulting function and adds it as a method to 'len'. The decorator can extract the type of the function's enclosing class from the various attributes of the function object (i.e. the function still knows what class defined it.) The expression 'def (args)' is a no-op unless there are decorators. Alternative 2: Function name in parens: In a def statement, putting the name of the function in parentheses indicates that the function name is an expression rather than a symbol: @overload def (len)(self): ... Decorators that are used in this fashion should expect two arguments rather than one, which are the name-expression and the function object. -- Talin From guido at python.org Fri Dec 8 15:53:34 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 8 Dec 2006 08:53:34 -0600 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4579365A.3050008@acm.org> References: <4579365A.3050008@acm.org> Message-ID: I've been thinking about this too, and I think it's reasonable to let the metaclass provide the dict to be used as locals. This is easier when the metaclass is given in the class header, e.g. by way of a base class, or through the default metaclass. Although the default default metaclass could of course have a hook to pass things on to another metaclass specified using a __metaclass__ assignment, I'd rather innovate a bit more and add keyword arguments to the class header, e.g. class C(B1, B2, metaclass=Foo): ... How can we design this feature without yet another gigantic contentious thread? I'm losing my appetite for those. I wonder if the right thing wouldn't be if you could do some serious research into the best possible semantics, syntax and implementation, and then post a fully-formed proposal (not quite a PEP). There is of course always the danger that you might go off into the deep end with that, so I'll leave it up to your judgement to decide when to post again. (Also, I'm still quite restricted in my internet access, at least until Monday.) --Guido On 12/8/06, Talin wrote: > I'd like to see metaclasses have additional capabilities in Py3K, and > one thing I'd like is for metaclasses to be able to have access to the > order of declarations within a class. > > If I understand correctly, in the current Python a class definition is > essentially a suite with ordinary local variables; once that suite > finishes executing, the locals() dict becomes the class's __dict__. The > metaclass slips in between these two phases of construction, in that it > can examine or modify the dict before it becomes a class. > > Suppose, however, that the metaclass could hook into the get/set > operations of the locals() dict itself. In other words, the > __metaclass__ statement would have a side-effect of replacing the > locals() dict with a mapping object supplied by the metaclass. > > This would allow the metaclass to intercept individual variable > assignments as they occur, rather than taking the whole dict in at once. > > I can think of a ton of applications for this. For example, you could > implement the functionality of __slots__ as a metaclass, allowing you to > define the slots using regular Python syntax rather than having to pass > in a list of name strings. You could define C structures using regular > Python syntax. You could declare CORBA or other IDL interfaces using > Python syntax and appropriate decorators. You might even be able to do > things like enumerations using metaclasses. > > -- Talin > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Fri Dec 8 18:47:30 2006 From: talin at acm.org (Talin) Date: Fri, 08 Dec 2006 09:47:30 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> Message-ID: <4579A532.50203@acm.org> How about this: Email responses/ideas to me directly, I'll summarize them and post the result. Guido van Rossum wrote: > I've been thinking about this too, and I think it's reasonable to let > the metaclass provide the dict to be used as locals. This is easier > when the metaclass is given in the class header, e.g. by way of a base > class, or through the default metaclass. Although the default default > metaclass could of course have a hook to pass things on to another > metaclass specified using a __metaclass__ assignment, I'd rather > innovate a bit more and add keyword arguments to the class header, > e.g. > > class C(B1, B2, metaclass=Foo): > ... > > How can we design this feature without yet another gigantic > contentious thread? I'm losing my appetite for those. I wonder if the > right thing wouldn't be if you could do some serious research into the > best possible semantics, syntax and implementation, and then post a > fully-formed proposal (not quite a PEP). There is of course always the > danger that you might go off into the deep end with that, so I'll > leave it up to your judgement to decide when to post again. (Also, I'm > still quite restricted in my internet access, at least until Monday.) > > --Guido > > On 12/8/06, Talin wrote: >> I'd like to see metaclasses have additional capabilities in Py3K, and >> one thing I'd like is for metaclasses to be able to have access to the >> order of declarations within a class. >> >> If I understand correctly, in the current Python a class definition is >> essentially a suite with ordinary local variables; once that suite >> finishes executing, the locals() dict becomes the class's __dict__. The >> metaclass slips in between these two phases of construction, in that it >> can examine or modify the dict before it becomes a class. >> >> Suppose, however, that the metaclass could hook into the get/set >> operations of the locals() dict itself. In other words, the >> __metaclass__ statement would have a side-effect of replacing the >> locals() dict with a mapping object supplied by the metaclass. >> >> This would allow the metaclass to intercept individual variable >> assignments as they occur, rather than taking the whole dict in at once. >> >> I can think of a ton of applications for this. For example, you could >> implement the functionality of __slots__ as a metaclass, allowing you to >> define the slots using regular Python syntax rather than having to pass >> in a list of name strings. You could define C structures using regular >> Python syntax. You could declare CORBA or other IDL interfaces using >> Python syntax and appropriate decorators. You might even be able to do >> things like enumerations using metaclasses. >> >> -- Talin >> _______________________________________________ >> Python-3000 mailing list >> Python-3000 at python.org >> http://mail.python.org/mailman/listinfo/python-3000 >> Unsubscribe: >> http://mail.python.org/mailman/options/python-3000/guido%40python.org >> > > From talin at acm.org Fri Dec 8 18:48:04 2006 From: talin at acm.org (Talin) Date: Fri, 08 Dec 2006 09:48:04 -0800 Subject: [Python-3000] Assignment decorators Message-ID: <4579A554.2070107@acm.org> One other minor brainstorm before I head off to work: I'd like function decorators to work with assignment statements as well as 'def' statements. Use case: class Foo: @private def myfunc(): pass @private selected = True Rationale: The 'def' statement is a type of assignment statement, in that it assigns a value (a function object) to a symbol. For consistency, the same mechanic ought to be available to other kinds of assignment. -- Talin From steven.bethard at gmail.com Fri Dec 8 19:11:09 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 8 Dec 2006 11:11:09 -0700 Subject: [Python-3000] 'defop' syntax with no new keywords In-Reply-To: <45793B37.5060702@acm.org> References: <45793B37.5060702@acm.org> Message-ID: On 12/8/06, Talin wrote: > Another mini-brainstorm, which is how to represent PJE's 'defop' syntax > without adding any keywords: > > Alternative 1. Omitting the function name in 'def' statements. [snip] > @overload(len): > def (self): > ... compute len( self ) ... [snip] > Alternative 2: Function name in parens: [snip] > @overload > def (len)(self): > ... Why can't we just use the regular ``def`` syntax, with an appropriate hook in ``type``, so that if you define a ``__len__`` function and ``len`` exists in an enclosing scope ``len.__addmethod__`` is called. So just like you do now, you'd write:: def __len__(self): .... Here's a proof-of-concept interactive session:: >>> import __builtin__ >>> class Type(type): ... def __init__(cls, name, bases, bodydict): ... for name, value in bodydict.items(): ... if callable(value): ... if name.startswith('__') and name.endswith('__'): ... name = name[2:-2] ... if name in globals() or hasattr(__builtin__, name): ... # in the real code, this would actually do the call ... format = '--> %s.__addmethod__(%s.%s, %s)' ... tup = name, cls.__name__, value.__name__, cls.__name__ ... print format % tup ... >>> def overloadable_function(self): ... pass ... >>> class C(object): ... __metaclass__ = Type ... def __len__(self): ... return 4 ... def overloadable_function(self): ... return 'foo' ... def regular_method(self): ... return 'bar' ... --> overloadable_function.__addmethod__(C.overloadable_function, C) --> len.__addmethod__(C.__len__, C) Note that the hook is in the metaclass so that the class exists when ``__addmethod__`` is called. The real version might need to be a little more careful about what it considers a function and how it looks in the enclosing scope, and we'd have to decide where ``__addmethod__`` gets called for things like ``__add__`` or ``__getitem__``, but hopefully the idea is clear. I suspect the one small modification you'd want to make to the ``def`` statement then would be to allow dotted names so that you could do something like:: ... def mapping.__getitem__(self): ... return 'foo' which would then call ``mapping.__getitem__.__addmethod__(C.__getitem__, C)``. If we went this route, we could then define a a plain ``__getitem__`` method as being equivalent to ``mapping.__getitem__`` or maybe ``container.__getitem__``. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From pje at telecommunity.com Fri Dec 8 19:21:54 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 08 Dec 2006 13:21:54 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> <4579365A.3050008@acm.org> Message-ID: <5.1.1.6.0.20061208131050.03fba610@sparrow.telecommunity.com> At 08:53 AM 12/8/2006 -0600, Guido van Rossum wrote: >I've been thinking about this too, and I think it's reasonable to let >the metaclass provide the dict to be used as locals. This is easier >when the metaclass is given in the class header, e.g. by way of a base >class, or through the default metaclass. Although the default default >metaclass could of course have a hook to pass things on to another >metaclass specified using a __metaclass__ assignment, I'd rather >innovate a bit more and add keyword arguments to the class header, >e.g. > >class C(B1, B2, metaclass=Foo): > ... Did you have any idea for what other keywords besides 'metaclass' might mean? Otherwise, this doesn't seem to add much over, say: class C(metaclass(Foo), B1, B2): (Except that metaclass() would then have to be a built-in, I suppose.) I definitely like the idea of the metaclass getting to control the locals() of the class suite, but then again it also seems like it would be just as easy to say that the class statement creates a class/type object (using the normal metaclass rules applied to the bases), and then the suite executes with a locals() that does getattr(cls, name) and setattr(cls, name, val) for getitem and setitem. This allows you to do a variety of interesting things, including assigning to __class__ in the body of the class to set (or change!) the metaclass directly, and allowing you to define metaclass-level properties. You can also access *inherited* methods and attributes in the body of the class (due to the use of getattr()). With those features in place, I'm not sure what we'd do with class statement keywords, unless we were going to use them to create parameterized types or something, which might be cool, but I have no idea at the moment how to go about doing it. :) From talin at acm.org Fri Dec 8 19:23:33 2006 From: talin at acm.org (Talin) Date: Fri, 08 Dec 2006 10:23:33 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> Message-ID: <4579ADA5.50704@acm.org> Guido van Rossum wrote: > I've been thinking about this too, and I think it's reasonable to let > the metaclass provide the dict to be used as locals. This is easier > when the metaclass is given in the class header, e.g. by way of a base > class, or through the default metaclass. Although the default default > metaclass could of course have a hook to pass things on to another > metaclass specified using a __metaclass__ assignment, I'd rather > innovate a bit more and add keyword arguments to the class header, > e.g. > > class C(B1, B2, metaclass=Foo): > ... > > How can we design this feature without yet another gigantic > contentious thread? I'm losing my appetite for those. I wonder if the > right thing wouldn't be if you could do some serious research into the > best possible semantics, syntax and implementation, and then post a > fully-formed proposal (not quite a PEP). There is of course always the > danger that you might go off into the deep end with that, so I'll > leave it up to your judgement to decide when to post again. (Also, I'm > still quite restricted in my internet access, at least until Monday.) Although, I'm not sure a discussion is needed - I think your syntax is fine, better than what we have now, and I have no objection to it. Here's a use case example: # Define a C-struct rectangle class Rect(metaclass=ctypes.struct): x = ctypes.int32 y = ctypes.int32 width = ctypes.int32 height = ctypes.int32 The only topic for discussion that I see is what the metaclass interface should look like. (Please don't start a discussion of flaws in my example, it's just an example, don't take it too seriously.) -- Talin From amcnabb at mcnabbs.org Fri Dec 8 19:11:17 2006 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Fri, 8 Dec 2006 11:11:17 -0700 Subject: [Python-3000] Assignment decorators In-Reply-To: <4579A554.2070107@acm.org> References: <4579A554.2070107@acm.org> Message-ID: <20061208181117.GL9618@mcnabbs.org> On Fri, Dec 08, 2006 at 09:48:04AM -0800, Talin wrote: > One other minor brainstorm before I head off to work: I'd like function= =20 > decorators to work with assignment statements as well as 'def' statements. >=20 > Use case: >=20 > class Foo: >=20 > @private > def myfunc(): > pass >=20 > @private > selected =3D True >=20 What would get passed to the decorator in the case of assignment? It would seem logical for it to be something like: def private(name, value): pass However, function decorators get something like: def private(func): pass Would these be unified, or would something completely different happen? -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 186 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20061208/2c52b705/attachment.pgp From janssen at parc.com Fri Dec 8 19:35:17 2006 From: janssen at parc.com (Bill Janssen) Date: Fri, 8 Dec 2006 10:35:17 PST Subject: [Python-3000] Types related to Syntax In-Reply-To: References: Message-ID: <06Dec8.103524pst."58648"@synergy1.parc.xerox.com> Ron, > I'm wonder how many places in pythons syntax requires or produces specific > types. And if listing and organizing these commonly used "syntactically > necessary" types would be useful to determine the minimum list of ABC's. (More > can always be added later.) I think a lot of the syntactic niceties have been developed to support various usage patterns, but not every usage pattern has been special-cased with syntax, because some are already supported by the basic language structures. But I agree that looking at the types which define special methods which are mapped to "operators" is a good idea. I just don't see any assurance that it would produce the complete set. > The approach suggested above, takes a much narrower view and would start by what > is only currently necessary by the language's current syntax. These are places > where you may possibly need to test for type-ness. To support operators. But there are other reasons to test for type-ness. Bill From theller at ctypes.org Fri Dec 8 19:37:28 2006 From: theller at ctypes.org (Thomas Heller) Date: Fri, 08 Dec 2006 19:37:28 +0100 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4579ADA5.50704@acm.org> References: <4579365A.3050008@acm.org> <4579ADA5.50704@acm.org> Message-ID: Talin schrieb: > Guido van Rossum wrote: >> I've been thinking about this too, and I think it's reasonable to let >> the metaclass provide the dict to be used as locals. This is easier >> when the metaclass is given in the class header, e.g. by way of a base >> class, or through the default metaclass. Although the default default >> metaclass could of course have a hook to pass things on to another >> metaclass specified using a __metaclass__ assignment, I'd rather >> innovate a bit more and add keyword arguments to the class header, >> e.g. >> >> class C(B1, B2, metaclass=Foo): >> ... >> > > Although, I'm not sure a discussion is needed - I think your syntax is > fine, better than what we have now, and I have no objection to it. > > Here's a use case example: > > # Define a C-struct rectangle > class Rect(metaclass=ctypes.struct): > x = ctypes.int32 > y = ctypes.int32 > width = ctypes.int32 > height = ctypes.int32 > Wouldn't it be nice also if functions called with named arguments were able to reconstruct the order in which arguments were passed? Having a way to provide the type of dict that will be used as **kw? (Since I normally dor *not* follow the discussions on this list I have no idea if this has been discussed already or is in a PEP.) Thomas From steven.bethard at gmail.com Fri Dec 8 19:48:54 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 8 Dec 2006 11:48:54 -0700 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <5.1.1.6.0.20061208131050.03fba610@sparrow.telecommunity.com> References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061208131050.03fba610@sparrow.telecommunity.com> Message-ID: At 08:53 AM 12/8/2006 -0600, Guido van Rossum wrote: > I've been thinking about this too, and I think it's reasonable to let > the metaclass provide the dict to be used as locals. [snip] > class C(B1, B2, metaclass=Foo): > ... On 12/8/06, Phillip J. Eby wrote: > I definitely like the idea of the metaclass getting to control the locals() > of the class suite, but then again it also seems like it would be just as > easy to say that the class statement creates a class/type object (using the > normal metaclass rules applied to the bases), and then the suite executes > with a locals() that does getattr(cls, name) and setattr(cls, name, val) > for getitem and setitem. The suggestion in open issues of the `make statement PEP`_ was that, given a class statement like:: class C(metaclass=metaclass): something like the following would be executed:: metadict = metaclass.__metadict__() exec in metadict C = metaclass('C', (), metadict) So if you wanted to monitor how attributes were assigned, you'd define classes like:: class my_type(type): class __metadict__(object): def __getitem__(self, name): ... def __setitem__(self, name, value): ... I know that seems overly complicated, and it seems like it would be better if you could follow Phillip's suggestion where the metaclass instance is created first and then ``getattr()`` and ``setattr()`` are called appropriately. Then you could just write:: class my_type(type): def __getattr__(self, name): ... def __setattr__(self, name, value): ... The `make statement PEP`_ didn't propose this approach because it would have broken existing metaclasses when the signatures of ``__new__`` and ``__init__`` dropped the classdict parameter, e.g.:: def __new__(meta, name, bases): ... def __init__(cls, name, bases): ... Personally, I wouldn't object to having to rewrite my metaclasses to these new APIs, but I'm still not certain how much backwards incompatibility Python 3000 is willing to accept. .. _make statement: http://www.python.org/dev/peps/pep-0359/ STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Fri Dec 8 20:58:38 2006 From: brett at python.org (Brett Cannon) Date: Fri, 8 Dec 2006 11:58:38 -0800 Subject: [Python-3000] Assignment decorators In-Reply-To: <4579A554.2070107@acm.org> References: <4579A554.2070107@acm.org> Message-ID: On 12/8/06, Talin wrote: > > One other minor brainstorm before I head off to work: I'd like function > decorators to work with assignment statements as well as 'def' statements. > > Use case: > > class Foo: > > @private > def myfunc(): > pass > > @private > selected = True > > Rationale: The 'def' statement is a type of assignment statement, in > that it assigns a value (a function object) to a symbol. For > consistency, the same mechanic ought to be available to other kinds of > assignment. That just seems ugly. Decorators are fine for functions, methods (and eventually classes) because they only appear in one place. But having them strewn about within a function body just seems messy to me.:: def fxn(): @decorator a = b @decorator c = d @decorator e = f Contrived, I know, but that just hurts my eyes. Plus if you want to change the value of something before it is assigned then pass the value to be assigned to a function. Decorators provide the ability to pre-process objects that one could not normally touch before assignment. You don't have this issue with assignment statements in a function and thus you are not solving a problem here that decorators were designed for. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061208/b101ff08/attachment.htm From greg.ewing at canterbury.ac.nz Sat Dec 9 01:20:54 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 09 Dec 2006 13:20:54 +1300 Subject: [Python-3000] Metaclasses in Py3K Message-ID: <457A0166.9060109@canterbury.ac.nz> Talin wrote: > In other words, the > __metaclass__ statement would have a side-effect of replacing the > locals() dict with a mapping object supplied by the metaclass. __metaclass__ isn't a statement, it's just an attribute that is interpreted in a special way *after* the class namespace has been populated, by which time it's too late to do what you suggest. -- Greg From pje at telecommunity.com Sat Dec 9 02:17:38 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 08 Dec 2006 20:17:38 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <457A0166.9060109@canterbury.ac.nz> Message-ID: <5.1.1.6.0.20061208201505.027e8f38@sparrow.telecommunity.com> At 01:20 PM 12/9/2006 +1300, Greg Ewing wrote: >Talin wrote: > > In other words, the > > __metaclass__ statement would have a side-effect of replacing the > > locals() dict with a mapping object supplied by the metaclass. > >__metaclass__ isn't a statement, it's just an >attribute that is interpreted in a special way >*after* the class namespace has been populated, >by which time it's too late to do what you >suggest. Note that if we simply make locals() set or get attributes on the class instance, assigning to __class__ in the body of the class would actually set the metaclass immediately. The principal hurdle that would need to be overcome to do this, is that you can't change a builtin type's __class__ currently: Python 2.5 (r25:51908, Dec 6 2006, 15:13:26) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> class X(object): pass ... >>> class Y(type): pass ... >>> X.__class__ = Y Traceback (most recent call last): File "", line 1, in TypeError: __class__ assignment: only for heap types Fixing this problem in Python 2.6 would be nice, actually, to say nothing of Py3K. From ncoghlan at gmail.com Sat Dec 9 02:19:46 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 09 Dec 2006 11:19:46 +1000 Subject: [Python-3000] Assignment decorators In-Reply-To: <4579A554.2070107@acm.org> References: <4579A554.2070107@acm.org> Message-ID: <457A0F32.4010407@gmail.com> Talin wrote: > One other minor brainstorm before I head off to work: I'd like function > decorators to work with assignment statements as well as 'def' statements. > > Use case: > > class Foo: > > @private > def myfunc(): > pass > > @private > selected = True > > Rationale: The 'def' statement is a type of assignment statement, in > that it assigns a value (a function object) to a symbol. For > consistency, the same mechanic ought to be available to other kinds of > assignment. The mechanic is available, the spelling is just different: selected = private(True) Compare to: myfunc = private(lambda: None) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From rrr at ronadam.com Sat Dec 9 04:01:07 2006 From: rrr at ronadam.com (Ron Adam) Date: Fri, 08 Dec 2006 21:01:07 -0600 Subject: [Python-3000] Types related to Syntax In-Reply-To: <06Dec8.103524pst."58648"@synergy1.parc.xerox.com> References: <06Dec8.103524pst."58648"@synergy1.parc.xerox.com> Message-ID: Bill Janssen wrote: > Ron, > >> I'm wonder how many places in pythons syntax requires or produces specific >> types. And if listing and organizing these commonly used "syntactically >> necessary" types would be useful to determine the minimum list of ABC's. (More >> can always be added later.) > > I think a lot of the syntactic niceties have been developed to support > various usage patterns, but not every usage pattern has been > special-cased with syntax, because some are already supported by the > basic language structures. But I agree that looking at the types > which define special methods which are mapped to "operators" is a good > idea. I just don't see any assurance that it would produce the complete > set. What kind of basic language structures are you thinking of? An example would be good. It's not just about types having special methods, which address's most situations where you may need to test an argument. There are also those syntax's that produce types (objects) with no arguments, and places that are implemented directly as byte codes so that no python level methods are involved. It wouldn't be a complete set. What would that be anyway? This would be the necessary subset needed to catch and handle error situations that may arise because of basic language constraints. It would be/contain all built in types and be relatively small. It would not rule out one or more importable class trees suited to general or even very specific data usage patterns. >> The approach suggested above, takes a much narrower view and would start by what >> is only currently necessary by the language's current syntax. These are places >> where you may possibly need to test for type-ness. > > To support operators. But there are other reasons to test for type-ness. I should have said... possibly need to test for type-ness due to syntax requirements. Ron From ironfroggy at gmail.com Sat Dec 9 04:40:28 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Fri, 8 Dec 2006 22:40:28 -0500 Subject: [Python-3000] Assignment decorators In-Reply-To: <4579A554.2070107@acm.org> References: <4579A554.2070107@acm.org> Message-ID: <76fd5acf0612081940o72d1ec53hbeeb9ed26599ff85@mail.gmail.com> On 12/8/06, Talin wrote: > One other minor brainstorm before I head off to work: I'd like function > decorators to work with assignment statements as well as 'def' statements. > > Use case: > > class Foo: > > @private > def myfunc(): > pass > > @private > selected = True > > Rationale: The 'def' statement is a type of assignment statement, in > that it assigns a value (a function object) to a symbol. For > consistency, the same mechanic ought to be available to other kinds of > assignment. > > -- Talin > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > Although I agree with the merits of this, to some degree (I even thought of it previously), I would have to now disagree because the overall value is probably not worth the resulting confusion. The confusion is both in the look of the code (which has been pointed out) and the lack of usefulness compared to function decorators. I'm all for class decorators, of course. Any kind of assignment that does not simple bind the name to a given expression is up for decorators, I say. Even though that includes the following, possibly terrible, example: @max for largest in list_of_pairs: print larget -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From jackdied at jackdied.com Sat Dec 9 06:38:57 2006 From: jackdied at jackdied.com (Jack Diederich) Date: Sat, 9 Dec 2006 00:38:57 -0500 Subject: [Python-3000] Assignment decorators In-Reply-To: References: <4579A554.2070107@acm.org> Message-ID: <20061209053857.GB5386@performancedrivers.com> On Fri, Dec 08, 2006 at 11:58:38AM -0800, Brett Cannon wrote: > On 12/8/06, Talin wrote: > > > >One other minor brainstorm before I head off to work: I'd like function > >decorators to work with assignment statements as well as 'def' statements. > > > >Use case: > > > > class Foo: > > > > @private > > def myfunc(): > > pass > > > > @private > > selected = True > > > >Rationale: The 'def' statement is a type of assignment statement, in > >that it assigns a value (a function object) to a symbol. For > >consistency, the same mechanic ought to be available to other kinds of > >assignment. > > > That just seems ugly. Decorators are fine for functions, methods (and > eventually classes) because they only appear in one place. But having them > strewn about within a function body just seems messy to me.:: > > def fxn(): > @decorator > a = b > @decorator > c = d > @decorator > e = f > > Contrived, I know, but that just hurts my eyes. Plus if you want to change > the value of something before it is assigned then pass the value to be > assigned to a function. Decorators provide the ability to pre-process > objects that one could not normally touch before assignment. You don't have > this issue with assignment statements in a function and thus you are not > solving a problem here that decorators were designed for. > Decorators let you announce that something is modified nearer to the definition. You get to say up front "the following is modified by @foo()" eg/ @munge def foo(): # 50 lines # instead of def foo() # 50 lines foo = munge(foo) Adding a decorator for simple assignments is at best benign and more likely confusing because now you have to read two lines instead of one. You can easilly say "this assignment is modified by foo()" by simply calling foo() during the assignment. eg/ @munge foo = 7 # instead of foo = munge(7) Function (and class) definitions are already spread out vertically and decorators just change the order to put the more important parts first. Simple assignments happen on one line so adding decorators would spread out vertically what is an otherwise concise operation. -Jack NB, may be a dup. I typo'd my email address earlier and the mail was held for moderation. From greg.ewing at canterbury.ac.nz Sat Dec 9 08:15:21 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 09 Dec 2006 20:15:21 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061208131050.03fba610@sparrow.telecommunity.com> Message-ID: <457A6289.5070500@canterbury.ac.nz> > At 08:53 AM 12/8/2006 -0600, Guido van Rossum wrote: > >>I've been thinking about this too, and I think it's reasonable to let >>the metaclass provide the dict to be used as locals. > > [snip] > >>class C(B1, B2, metaclass=Foo): If there's to be a special syntax for this, I don't think it should have the word "metaclass" in it -- it's too verbose and too jargony. How about class C(B1, B2) is M: ... This is concise, keeps the metaclass clearly separated from the list of bases, and avoids creating any new keywords. -- Greg From jcarlson at uci.edu Sat Dec 9 10:53:33 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 09 Dec 2006 01:53:33 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: Message-ID: <20061209012306.97D3.JCARLSON@uci.edu> "Steven Bethard" wrote: > For the sake of DRY, I'd prefer to see:: > > class MyMapping: > def mapping.__getitem__(self, key): > ... > def mapping.__len__(self): > ... I'd prefer to see... class MyMapping: @implements(mapping) def __getitem__(self, key): ... @implements(mapping) def __len__(self): ... Because then we don't need a new dotted name syntax proposal. The 3-argument type call would need to learn how to handle methods with @implements(interface) behavior defined, but other than that, I think it would work as-is. - Josiah P.S. Here's a 10 minute implementation of the above semantics that is *almost* backwards compatible with Python 2.3 (except for the decorator thing, which Python 2.3 has to do without). _type = type def make_interface(name, methods): return _type(name, (object,), dict([(nam, object()) for nam in methods])) mapping = make_interface('mapping', mapping_methods) sequence = make_interface('sequence', sequence_methods) #... _supports = {} def supports(cls, interface): if interface in _supports: return cls in _supports[interface] return False def implements(*interfaces): def foo(fcn): return implements_wrapper(fcn, interfaces) return foo class implements_wrapper(object): __slots__ = ['fcn', 'interfaces'] def __init__(self, fcn, interfaces): self.fcn = fcn self.interfaces = interfaces def __call__(self): return self.fcn, self.interfaces def type(*args): if len(args) != 3: return _type(*args) name, bases, dct = args dct2 = {} for nam, obj in dct.items(): if isinstance(obj, implements_wrapper): f, i = obj() dct2[nam] = i dct[nam] = f cls = _type(name, bases, dct) for nam, i in dct2.iteritems(): for j in i: #default dict that produced sets would work well here _supports.setdefault(getattr(i, nam), {})[cls] = None return cls __builtins__.type = type From ironfroggy at gmail.com Sat Dec 9 11:53:20 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 9 Dec 2006 05:53:20 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <457A6289.5070500@canterbury.ac.nz> References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061208131050.03fba610@sparrow.telecommunity.com> <457A6289.5070500@canterbury.ac.nz> Message-ID: <76fd5acf0612090253n591f6ecfq9b2d95d112c89e0e@mail.gmail.com> On 12/9/06, Greg Ewing wrote: > > At 08:53 AM 12/8/2006 -0600, Guido van Rossum wrote: > > > >>I've been thinking about this too, and I think it's reasonable to let > >>the metaclass provide the dict to be used as locals. > > > > [snip] > > > >>class C(B1, B2, metaclass=Foo): > > If there's to be a special syntax for this, I don't > think it should have the word "metaclass" in it -- > it's too verbose and too jargony. > > How about > > class C(B1, B2) is M: > ... > > This is concise, keeps the metaclass clearly > separated from the list of bases, and avoids > creating any new keywords. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ironfroggy%40gmail.com > The obvious problem with a metaclass specific syntax is that it doesnt allow for other things. Adding keyword options to class declarations means you could do things beyond just metaclasses, such as interfaces, if that happens. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From jackdiedk at jackdied.com Fri Dec 8 23:23:53 2006 From: jackdiedk at jackdied.com (Jack Diederich) Date: Fri, 8 Dec 2006 17:23:53 -0500 Subject: [Python-3000] Assignment decorators In-Reply-To: References: <4579A554.2070107@acm.org> Message-ID: <20061208222353.GF5382@performancedrivers.com> On Fri, Dec 08, 2006 at 11:58:38AM -0800, Brett Cannon wrote: > On 12/8/06, Talin wrote: > > > >One other minor brainstorm before I head off to work: I'd like function > >decorators to work with assignment statements as well as 'def' statements. > > > >Use case: > > > > class Foo: > > > > @private > > def myfunc(): > > pass > > > > @private > > selected = True > > > >Rationale: The 'def' statement is a type of assignment statement, in > >that it assigns a value (a function object) to a symbol. For > >consistency, the same mechanic ought to be available to other kinds of > >assignment. > > > That just seems ugly. Decorators are fine for functions, methods (and > eventually classes) because they only appear in one place. But having them > strewn about within a function body just seems messy to me.:: > > def fxn(): > @decorator > a = b > @decorator > c = d > @decorator > e = f > > Contrived, I know, but that just hurts my eyes. Plus if you want to change > the value of something before it is assigned then pass the value to be > assigned to a function. Decorators provide the ability to pre-process > objects that one could not normally touch before assignment. You don't have > this issue with assignment statements in a function and thus you are not > solving a problem here that decorators were designed for. Decorators let you announce that something is modified nearer to the definition. eg/ @munge def foo(): # 50 lines # instead of def foo() # 50 lines foo = munge(foo) Adding a decorator for simple assignments is at best benign and more likely confusing because now you have to read two lines instead of one. eg/ @munge foo = 7 # instead of foo = munge(7) Function (and class) definitions are already spread out vertically and decorators just change the order to put the more important parts first. Simple assignments happen on one line so adding decorators would spread out vertically what is an otherwise concise operation. -Jack From steven.bethard at gmail.com Sat Dec 9 18:11:32 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 9 Dec 2006 10:11:32 -0700 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: <20061209012306.97D3.JCARLSON@uci.edu> References: <20061209012306.97D3.JCARLSON@uci.edu> Message-ID: On 12/9/06, Josiah Carlson wrote: > > I'd prefer to see... > > class MyMapping: > @implements(mapping) > def __getitem__(self, key): > ... > > @implements(mapping) > def __len__(self): > ... > [snip] > > P.S. Here's a 10 minute implementation of the above semantics that is > *almost* backwards compatible with Python 2.3 (except for the decorator > thing, which Python 2.3 has to do without). > > > _type = type > > def make_interface(name, methods): > return _type(name, (object,), > dict([(nam, object()) for nam in methods])) > > mapping = make_interface('mapping', mapping_methods) > sequence = make_interface('sequence', sequence_methods) > #... > > _supports = {} > > def supports(cls, interface): > if interface in _supports: > return cls in _supports[interface] > return False > > def implements(*interfaces): > def foo(fcn): > return implements_wrapper(fcn, interfaces) > return foo > > class implements_wrapper(object): > __slots__ = ['fcn', 'interfaces'] > def __init__(self, fcn, interfaces): > self.fcn = fcn > self.interfaces = interfaces > def __call__(self): > return self.fcn, self.interfaces > > def type(*args): > if len(args) != 3: > return _type(*args) > name, bases, dct = args > dct2 = {} > for nam, obj in dct.items(): > if isinstance(obj, implements_wrapper): > f, i = obj() > dct2[nam] = i > dct[nam] = f > cls = _type(name, bases, dct) > for nam, i in dct2.iteritems(): > for j in i: > #default dict that produced sets would work well here > _supports.setdefault(getattr(i, nam), {})[cls] = None > return cls > > __builtins__.type = type Sorry, I don't understand this implementation. I just tried it and the following fails because ``__getitem__`` is now an ``implements_wrapper`` that accepts the wrong arguments:: class C(object): @implements(mapping) def __getitem__(self, item): return 42 C()[1] But I think the ``implements()`` decorator is a good way to approach this. Here's a different approach to it:: def supports(cls, func_or_funcs): try: iter(func_or_funcs) except TypeError: return func_or_funcs in cls.__supports__ else: return cls.__supports__.issuperset(func_or_funcs) def implements(*interfaces): def decorate(func): func.__interfaces__ = interfaces return func return decorate class interface_monitor(type): def __init__(cls, name, bases, bodydict): cls.__supports__ = set() for name, value in cls.__dict__.items(): try: interfaces = value.__interfaces__ except AttributeError: pass else: for interface in interfaces: method = getattr(interface, name) cls.__supports__.add(method) Basically, each function is identified with a set of interface namespaces, e.g. ``mapping``, and the metaclass then uses ``getattr()`` to get the specific functions, e.g. ``mapping.__getitem__``, and add them to the set of operations the class supports, i.e. ``cls.__supports__``. Here's how you might actually use this code:: class Mapping(object): def __iter__(self): pass def __getitem__(self, key): pass def get(self, key, default=None): pass minimal_mapping = Mapping.__iter__, Mapping.__getitem__ complete_mapping = minimal_mapping + (Mapping.get,) class C(object): __metaclass__ = interface_monitor @implements(Mapping) def __getitem__(self, key): return 4 @implements(Mapping) def __iter__(self): return [4, 4, 4] print supports(C, Mapping.__iter__) # prints True print supports(C, Mapping.__getitem__) # prints True print supports(C, minimal_mapping) # prints True print supports(C, complete_mapping) # prints False Note that with this code, an interface is just a set of methods, so you can use ``supports()`` equally well to check for support of individual methods or to check support for whole sets of methods. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From jcarlson at uci.edu Sat Dec 9 18:27:04 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 09 Dec 2006 09:27:04 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <76fd5acf0612090253n591f6ecfq9b2d95d112c89e0e@mail.gmail.com> References: <457A6289.5070500@canterbury.ac.nz> <76fd5acf0612090253n591f6ecfq9b2d95d112c89e0e@mail.gmail.com> Message-ID: <20061209091708.97D6.JCARLSON@uci.edu> "Calvin Spealman" wrote: > The obvious problem with a metaclass specific syntax is that it doesnt > allow for other things. Adding keyword options to class declarations > means you could do things beyond just metaclasses, such as interfaces, > if that happens. Right, but as is being discussed in another thread, perhaps one doesn't want to say, "this implements the full sequence interface", but instead "this implements these portions of the sequence interface". The proposal that I sent to Talin can be summed up as... Use keywords as suggested by Guido, but give them a different name. class FooClass(X, Y, type=FooType): ... One can then expand the keyword argument to include support for the dictionary-like object to be used during class body execution. class FooClass(X, Y, dict=FooDict): ... Then we don't have the issue with 'need to instantiate the metaclass before we can get the special dict from it'. In the other thread, they don't really bring up "implements the full interface", but that could be handled by class decorators in a similar fashion to what I proposed there, or even something like the above as... class FooClass(X, Y, implements=sequence): ... It is getting a bit unwieldy... class FooClass(X, Y, type=FooType, dict=FooDict, implements=sequence): ... But I think that any syntax that supports all three of the currently proposed "wouldn't it be nice if" can get that way. - Josiah From jcarlson at uci.edu Sat Dec 9 18:53:34 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 09 Dec 2006 09:53:34 -0800 Subject: [Python-3000] iostack and Oh Oh In-Reply-To: References: <20061209012306.97D3.JCARLSON@uci.edu> Message-ID: <20061209092741.97D9.JCARLSON@uci.edu> "Steven Bethard" wrote: > On 12/9/06, Josiah Carlson wrote: > > I'd prefer to see... > > > > class MyMapping: > > @implements(mapping) > > def __getitem__(self, key): > > ... > > > > @implements(mapping) > > def __len__(self): > > ... > > > [snip] > > > > P.S. Here's a 10 minute implementation of the above semantics that is > > *almost* backwards compatible with Python 2.3 (except for the decorator > > thing, which Python 2.3 has to do without). [snip] > > __builtins__.type = type > > > Sorry, I don't understand this implementation. I just tried it and > the following fails because ``__getitem__`` is now an > ``implements_wrapper`` that accepts the wrong arguments:: > > class C(object): > @implements(mapping) > def __getitem__(self, item): > return 42 > > C()[1] It doesn't work because we need to replace the base metaclass from object, which the above mechanism doesn't do. It half-asses it by monkey-patching __builtins__, but object.__new__ doesn't look in __builtins__, it uses CPython calls. If you want to get your above use to work all the time (I believe, I hadn't tested it), you can also add the following just after the __builtins__.type = type . class object(object): __metaclass__ = type __builtins__.object = object > But I think the ``implements()`` decorator is a good way to approach > this. Here's a different approach to it:: [snip] > class C(object): > __metaclass__ = interface_monitor > > @implements(Mapping) > def __getitem__(self, key): > return 4 > > @implements(Mapping) > def __iter__(self): > return [4, 4, 4] > > Note that with this code, an interface is just a set of methods, so > you can use ``supports()`` equally well to check for support of > individual methods or to check support for whole sets of methods. That's what I was getting at with the previous version, though instead of using methods on the various interface objects (eg Mapping), I used objects in the class namespace. Using a function does allow for doing argument list length and/or keyword name verification, but I don't know if we want to go that far. Using the monkey patching of __builtins__.object as I include in this message, makes the __metaclass__ bit that you use unnecessary. I also have a personal aversion for adding *yet another* attribute to the methods that implement interfaces, but since we are adding __signature__, etc., and no one else seems to be complaining, what's one more? - Josiah From ronaldoussoren at mac.com Sat Dec 9 21:01:40 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 9 Dec 2006 21:01:40 +0100 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <5.1.1.6.0.20061208201505.027e8f38@sparrow.telecommunity.com> References: <5.1.1.6.0.20061208201505.027e8f38@sparrow.telecommunity.com> Message-ID: <1CDF30CD-4978-473F-A945-8C86E4DE450C@mac.com> On Dec 9, 2006, at 2:17 AM, Phillip J. Eby wrote: > At 01:20 PM 12/9/2006 +1300, Greg Ewing wrote: >> Talin wrote: >>> In other words, the >>> __metaclass__ statement would have a side-effect of replacing the >>> locals() dict with a mapping object supplied by the metaclass. >> >> __metaclass__ isn't a statement, it's just an >> attribute that is interpreted in a special way >> *after* the class namespace has been populated, >> by which time it's too late to do what you >> suggest. > > Note that if we simply make locals() set or get attributes on the > class > instance, assigning to __class__ in the body of the class would > actually > set the metaclass immediately. > > The principal hurdle that would need to be overcome to do this, is > that you > can't change a builtin type's __class__ currently: Another hurdle is that you can add slots the the type object (currently only in C code), how would your scheme deal with that? PyObjC implements types that correspond with Objective-C classes and add a (hidden) slot to type objects that refers to those classes. Classic classes are a simular problem: they aren't instances of type: >>> class C: pass >>> isinstance(C, type) False >>> isinstance(object, type) True I know classic classes are going away, but this does seem to point to a problem with your scheme: metaclasses can change the type of the object created by a class statement. Ronald From pje at telecommunity.com Sat Dec 9 22:33:33 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 09 Dec 2006 16:33:33 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <1CDF30CD-4978-473F-A945-8C86E4DE450C@mac.com> References: <5.1.1.6.0.20061208201505.027e8f38@sparrow.telecommunity.com> <5.1.1.6.0.20061208201505.027e8f38@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061209163300.027c32a8@sparrow.telecommunity.com> At 09:01 PM 12/9/2006 +0100, Ronald Oussoren wrote: >On Dec 9, 2006, at 2:17 AM, Phillip J. Eby wrote: > > The principal hurdle that would need to be overcome to do this, is > > that you > > can't change a builtin type's __class__ currently: > >Another hurdle is that you can add slots the the type object >(currently only in C code), how would your scheme deal with that? That's not a different hurdle, it's the same one. :) From michel at dialnetwork.com Sat Dec 9 22:49:30 2006 From: michel at dialnetwork.com (Michel Pelletier) Date: Sat, 09 Dec 2006 13:49:30 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: Message-ID: <1165700970.2404.23.camel@michel-laptop> > From: Greg Ewing > How about > > class C(B1, B2) is M: > ... +1 -Michel PS Please excuse me for my last email's digest subject. From michel at dialnetwork.com Sat Dec 9 22:44:15 2006 From: michel at dialnetwork.com (Michel Pelletier) Date: Sat, 09 Dec 2006 13:44:15 -0800 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 22 In-Reply-To: References: Message-ID: <1165700656.2404.21.camel@michel-laptop> Guido: > > class C(B1, B2, metaclass=Foo): +1 Talin: > >> This would allow the metaclass to intercept individual variable > >> assignments as they occur, rather than taking the whole dict in at once. +10! Guido: > > How can we design this feature without yet another gigantic > > contentious thread? I'm losing my appetite for those. We have had many of these discussions already. Interfaces and ABCs were discussed in 1998, although I wasn't there to take part in it, but I did do the research on those threads in 2000-2001 and much more discussion ensued then, on ABCs, interfaces, type declarations and all kinds of stuff (sound familiar?). Forget what my opinion was then or is now because I largely have myself, but there were ideas and voices in those discussions who I don't see in the current discussion now. I am concerned that the important input from these people will be either forgotten or reinvented. Please I am not trying to start a flame or pointing fingers directly at anyone, obviously discussion along these lines is good, I just think there would be less noise and more signal if whomever was spearheading the next generation of PEPs on this subject dug back into the archives and understood the discussions that have taken place. -Michel From greg.ewing at canterbury.ac.nz Sun Dec 10 00:05:23 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 10 Dec 2006 12:05:23 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <76fd5acf0612090253n591f6ecfq9b2d95d112c89e0e@mail.gmail.com> References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061208131050.03fba610@sparrow.telecommunity.com> <457A6289.5070500@canterbury.ac.nz> <76fd5acf0612090253n591f6ecfq9b2d95d112c89e0e@mail.gmail.com> Message-ID: <457B4133.2010500@canterbury.ac.nz> Calvin Spealman wrote: > The obvious problem with a metaclass specific syntax is that it doesnt > allow for other things. Please, let's not hypergeneralise. I would hate to see the class header get cluttered up with unbounded quantities of extra stuff. -- Greg From greg.ewing at canterbury.ac.nz Sun Dec 10 00:06:15 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 10 Dec 2006 12:06:15 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061209091708.97D6.JCARLSON@uci.edu> References: <457A6289.5070500@canterbury.ac.nz> <76fd5acf0612090253n591f6ecfq9b2d95d112c89e0e@mail.gmail.com> <20061209091708.97D6.JCARLSON@uci.edu> Message-ID: <457B4167.9030403@canterbury.ac.nz> Josiah Carlson wrote: > class FooClass(X, Y, implements=sequence): No, no, no, no, no. This is just the sort of thing I'm afraid of. "Implements equals" is an abomination against the English language. -- Greg From jcarlson at uci.edu Sun Dec 10 09:53:42 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 10 Dec 2006 00:53:42 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <457B4167.9030403@canterbury.ac.nz> References: <20061209091708.97D6.JCARLSON@uci.edu> <457B4167.9030403@canterbury.ac.nz> Message-ID: <20061210004655.980C.JCARLSON@uci.edu> Greg Ewing wrote: > > Josiah Carlson wrote: > > > class FooClass(X, Y, implements=sequence): > > No, no, no, no, no. > > This is just the sort of thing I'm afraid of. > "Implements equals" is an abomination against the > English language. I'm happy to withdraw that particular version (I like @implements better anyways), but I think it is useful for type=M and dict=D arguments (for metaclass and special dictionary support). - Josiah From martin at v.loewis.de Sun Dec 10 10:13:16 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 10 Dec 2006 10:13:16 +0100 Subject: [Python-3000] Python/C++ question In-Reply-To: <44DA6C01.2040904@acm.org> References: <44DA6C01.2040904@acm.org> Message-ID: <457BCFAC.4000405@v.loewis.de> Talin schrieb: > A while back someone proposed switching to C++ as the implementation > language for CPython, and the response was that this would make ABI > compatibility too difficult, since the different C++ compilers don't > have a common way to represent things like vtables and such. This is a lesser issue than it used to be. Today, on Linux, many compilers support a standard ABI (specified originally for IA-64); this ABI is now also part of the Linux Standards Base. The issue also became lesser as the number of "life" C++ implementations got reduced over time. The issue is still real on Windows, where you either use the MS ABI or the GCC ABI. > However, I was thinking - if you remove all of the ABI-breaking features > of C++, such as virtual functions, name mangling, RTTI, exceptions, and > so on, its still a pretty nice language compared to C - you still have > things like namespaces, constructors/destructors (especially nice for > stack-local objects), overloadable type conversion, automatic > upcasting/downcasting, references, plus you don't have to keep repeating > the word 'struct' everywhere. Most of these are ABI relevant: - namespaces rely on the name mangling the compiler uses; different compilers use different name mangling - constructors also rely on name mangling, plus there are differences on how virtual bases are dealt with, and on where the memory allocation occurs (for heap objects) - overloadable type conversions rely on name mangling - automatic upcasting and downcasting relies on the object layout; for dynamic_cast, you need virtual methods References themselves likely have a uniform implementation, but reference parameters of course affect the name mangling. Also, binding to references may involve temporary objects, in which case constructors are involved. Not having to repeat struct has no ABI impact. Regards, Martin From rasky at develer.com Sun Dec 10 11:54:58 2006 From: rasky at develer.com (Giovanni Bajo) Date: Sun, 10 Dec 2006 11:54:58 +0100 Subject: [Python-3000] Python/C++ question In-Reply-To: <457BCFAC.4000405@v.loewis.de> References: <44DA6C01.2040904@acm.org> <457BCFAC.4000405@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> However, I was thinking - if you remove all of the ABI-breaking features >> of C++, such as virtual functions, name mangling, RTTI, exceptions, and >> so on, its still a pretty nice language compared to C - you still have >> things like namespaces, constructors/destructors (especially nice for >> stack-local objects), overloadable type conversion, automatic >> upcasting/downcasting, references, plus you don't have to keep repeating >> the word 'struct' everywhere. To me, the killer feature would be that in C++ you can implement a smart pointer which takes care of incref/decref automatically for 99% of the code. This would be a terrific tool for the extension/core writers. -- Giovanni Bajo From martin at v.loewis.de Sun Dec 10 23:50:10 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 10 Dec 2006 23:50:10 +0100 Subject: [Python-3000] Python/C++ question In-Reply-To: References: <44DA6C01.2040904@acm.org> <457BCFAC.4000405@v.loewis.de> Message-ID: <457C8F22.5060407@v.loewis.de> Giovanni Bajo schrieb: > To me, the killer feature would be that in C++ you can implement a smart > pointer which takes care of incref/decref automatically for 99% of the code. > This would be a terrific tool for the extension/core writers. Of course, this would also break in presence of binary-incompatible compilers, in particular when it relates to exception handling. Regards, Martin From ironfroggy at gmail.com Mon Dec 11 03:41:13 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sun, 10 Dec 2006 21:41:13 -0500 Subject: [Python-3000] Modules with Dual Python/C Implementations Message-ID: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> Has anyone considered consolidating the module pairs that have both a C and Python implementation? For example, pickle and cPickle and StingIO and cStringIO. It seems like keeping both around might be counter productive. It leads to more code for bugs, issues when there are accidental differences between the two, leading to misdiagnosed errors or hard to find bugs when people test with pickle and run with cPickle. Just seems like all around, if there was found to be a reason to make a module in C over its original Python version, the original doesn't need to stick around. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From ntoronto at cs.byu.edu Mon Dec 11 03:57:39 2006 From: ntoronto at cs.byu.edu (Neil Toronto) Date: Sun, 10 Dec 2006 19:57:39 -0700 Subject: [Python-3000] Python/C++ question In-Reply-To: <457BCFAC.4000405@v.loewis.de> References: <44DA6C01.2040904@acm.org> <457BCFAC.4000405@v.loewis.de> Message-ID: <457CC923.8050001@cs.byu.edu> Martin v. L?wis wrote: > Not having to repeat struct has no ABI impact. Yes! Switch to C++! Down with struct! No, I *don't* have anything useful to add to the discussion, in fact. Neil From barry at python.org Mon Dec 11 13:33:24 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 11 Dec 2006 07:33:24 -0500 Subject: [Python-3000] Python/C++ question In-Reply-To: <457CC923.8050001@cs.byu.edu> References: <44DA6C01.2040904@acm.org> <457BCFAC.4000405@v.loewis.de> <457CC923.8050001@cs.byu.edu> Message-ID: <6128115C-9C05-49CF-9C9A-97A58D749B1B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 10, 2006, at 9:57 PM, Neil Toronto wrote: > Martin v. L?wis wrote: >> Not having to repeat struct has no ABI impact. > > Yes! Switch to C++! Down with struct! > > No, I *don't* have anything useful to add to the discussion, in fact. I know this has been discussed many times before and I know that Guido is against the change. But I'll just add (again) that / requiring/ C++ to build Python will cause some of us pain. Our highly embedded/extended app has to twist and turn in order to link against one external C++ library that really has no need being C++. It's manageable on Windows and Linux, but some platforms (e.g. starts with "so" and ends with "is") are really quite painful. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRX1QIHEjvBPtnXfVAQIvdgP/dtRCeHuapv+SCidq1Rt2nuZyY48H/5AM qVKs/AYFGTZR8qYcEsTUb3LzLLxKDTG6DwVlld50YfEGmadIEgAgl7RUMGGfvvZ/ A7EdiD1dIkAO1pB5vXqPUS+u6fgYMERF4arY8QNMMK0/f0RSlNczl1X680M+4rR1 OSMyFCDGtow= =ZPmk -----END PGP SIGNATURE----- From gustavo at niemeyer.net Mon Dec 11 14:06:15 2006 From: gustavo at niemeyer.net (Gustavo Niemeyer) Date: Mon, 11 Dec 2006 11:06:15 -0200 Subject: [Python-3000] Assignment decorators In-Reply-To: References: <4579A554.2070107@acm.org> Message-ID: <20061211130615.GA9732@niemeyer.net> > That just seems ugly. Decorators are fine for functions, methods (and Ugly!? Come on.. normal assignments are so nineties. Think about the amazing possibilities: @str @math.log @multiply(3) @add(2) a = 2 Who needs RPN!? That's FPN.. Forward Python Notation. Python will be 30% better. (and I'll use something else) -- Gustavo Niemeyer http://niemeyer.net From brett at python.org Mon Dec 11 21:10:34 2006 From: brett at python.org (Brett Cannon) Date: Mon, 11 Dec 2006 12:10:34 -0800 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> Message-ID: On 12/10/06, Calvin Spealman wrote: > > Has anyone considered consolidating the module pairs that have both a > C and Python implementation? For example, pickle and cPickle and > StingIO and cStringIO. It seems like keeping both around might be > counter productive. It leads to more code for bugs, issues when there > are accidental differences between the two, leading to misdiagnosed > errors or hard to find bugs when people test with pickle and run with > cPickle. Just seems like all around, if there was found to be a reason > to make a module in C over its original Python version, the original > doesn't need to stick around. This has been argued about before. It has been suggested we actually ditch the C version since we only want to maintain one version and the Python version can be used by alternative Python implementations. This probably needs to be covered in a PEP that covers a stdlib reorg/renaming. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061211/2cbac386/attachment.html From jimjjewett at gmail.com Mon Dec 11 21:37:37 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 11 Dec 2006 15:37:37 -0500 Subject: [Python-3000] programmable syntax (was ... #4: interpolated strings ala perl) Message-ID: On 12/6/06, Jan Grant wrote: > Thank-you: I agree with pretty much everything you say. It's an > interesting question to ask, "can one construct a convenient > interpolation mechanism that uses the appropriate application quoting > mechanisms where necessary?" Since we don't know what environments will be needed (let alone their appropriate quoting needs), the answer for builtins (including str) is no. What we can do is make it a bit easier (or more idiomatic) to write domain-specific interpolators. Doing this without risking the chaos available to macros or programmable syntax is trickier. > I think the question here is, is there a safe, convenient way that > Python can determine what to do (or be told what to do) when it sees: > s = "some string here with {variable} in it" No, there isn't. Using "%" (or "\", or "{...}" as magic just hardcodes specific implementation choices that happen to work fairly often. What could work is s = MyDomainStr("some string here with {variable} in it") where MyDomainStr could very well be a subclass of str that just overrides % (orthe proposed format method) It *might* seem more natural if % was no longer associated with strings, but with Templates. (And I suppose str could implement/inherit from Templates as a convenience, so long as the concepts were separated.) s = Template("some string with %s in it") There really isn't any need to store the literal characters, including "%s". There is a need to store ("some string with ", , " in it"), but pretending that this object is a (conceptual) string just confuses things. > PS. IF generic functions (operators) could be sensitive to return types > and IF python6k supported a complex type inference mechanism (possibly > including automatic coercion), This really doesn't need to wait even for py3K. CPython happens to compile to bytecode, but there isn't anything in the language (except possibly threading interactions) which automatically prevents an optimizing compiler. (Generally optimizing -- messing with builtins or other modules would be worse than today.) > then one might be able to do something like: > > def foo(d: sqlDriver): > s = "select template goes here" % (param1, param2) > r = d.query(s) > > where sqlDriver has a method > > def query(self, s: some_type_that_signals_sql_interpolation) > > and "%" is overloadable on the basis that the return type is determined > to be compatible with "some_type_that_signals_sql_interpolation". Those > are some mighty big "IF"s though, and you could still concoct cases > where things would break :-) I think you want def foo(d: sqlDriver): s = SQLTemplate("select template goes here", args=(param1, param2)) r = d.query(s) The exact quoting mechanism would depend on the specific sqlDriver. Asking a SQLTemplate to store the constant portions is reasonable. Asking it to store a reference to specific parameters may be reasonable. Asking a sqlDriver to know its own quoting conventions is reasonable. Asking the compiler to secretly replace str literals with arbitrarily-classed objects which know to override the % method to something whose specification may be changed even after the object is created ... is fragile. -jJ From jimjjewett at gmail.com Mon Dec 11 22:15:20 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 11 Dec 2006 16:15:20 -0500 Subject: [Python-3000] [Fwd: features i'd like [Python 3000] ... #3: fix super()] In-Reply-To: <20061206093755.M30563@tribble.ilrt.bris.ac.uk> References: <4573F3FD.1080800@666.com> <20061206093755.M30563@tribble.ilrt.bris.ac.uk> Message-ID: On 12/6/06, Jan Grant wrote: > On Mon, 4 Dec 2006, Ben Wing wrote: > Commonly you know what the method is called at the point where you're > typing "super" :-) It still violated DRY. For me, the single most annoying part is the assymetry with regular calls. super(ClassName, self).method(args_excluding_self) I understand the reason (super needs the "self", and then returns a bound method), but it still jars. That it still doesn't quite solve the problem (e.g., what if the name of the class gets rebound?) makes it seem almost bad. What are the use cases for super? The ones I can think of are: (1) I'm writing a mixin. I want to call the next method (if there is one) with exactly my arguments. If the method doesn't exist on any superclass, then a no-op is fine. (2) I'm calling a parent class' method with exactly my arguments. If the method doesn't exist, then an AttributeError is appropriate. (But that probably means I made a typo, and shouldn't have had to repeat the method name in the first place.) (3) I'm calling a parent class with changed arguments. (Typically, to throw away some arguments that I used for my extensions, or to fill in defaults.) (4) I'm calling a different method on the parent, *and* I want to avoid calling that method from my own or a further derived class. (Is this ever a good pattern?) (5) ...? I have a feeling I'm missing some cases with different requirements, but can't think what they are. The fact that, at define time, I can't get hold of my class object yet, makes all super calls fragile. (Though perhaps not as fragile as the alternative.) Cases (1) and (2) are common enough that there should be a way to do them both easily, and it would be great if the argument list did not have to be repeated. By not repeating it, the cases that do make changes would stand out more. Cases (3) and (4) require an explicit argument list, but it would still be helpful if that list could match the signature to the extent possible (with "self", for instance). -jJ From jimjjewett at gmail.com Mon Dec 11 22:59:16 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 11 Dec 2006 16:59:16 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <-7927364766123597750@unknownmsgid> References: <457368C0.4010608@davious.org> <-3967619051774595947@unknownmsgid> <-7927364766123597750@unknownmsgid> Message-ID: On 12/4/06, Bill Janssen wrote: > Jim Jewett writes: > > On 12/3/06, Bill Janssen wrote: > > > > Implementations are used to make Method Dispatching more flexible by > > > > allowing a class to claim that it acts like another class/type even > > > > though it is not derived from that class/type. > > > I'm not sure just what this adds over the simple "change_class" > > > function that I've posted twice already. Is there an implementation > > > efficiency argument here? > > (1) Interfaces can be placed on a separate (presumably light-weight) > > inheritance hierarchy. > > I know it looks this way, but I think you'd just be changing one > inheritance mechanism for another. No one would use the one that's > already there; they'd just use this one instead. And I don't really > see how it's lighter-weight -- can you explain that? As an example (details could change later) When looking up an attribute (such as a method), look up the regular Base class inheritance chain. Changes to this chain are (as today) bad form, so the MRO results can be cached on the fully-derived type. There is an (unenforced) assumption that base classes are concrete, which will help some people to get their heads around your code. If the attribute is not found, instead of throwing an error, look in the interface chain as well. There is a fair chance that this list of interfaces will change even after class creation. There is a fair chance that attributes will be methods which just raise an error (because they should have been overridden). Note that this does mean (the logical equivalent of) tracing the MRO tree twice: Interface I: src="interface" class A: implements I y="normal" a=A() class B src="class" y="hidden" b=B() class C(A, B): pass c=C() c.y is a.y # because A is earlier on the inheritance tree, but ... c.src is b.src # because interfaces are always after "concrete" members > > (2) There is are efficiency (and security) concerns with changing > > bases in an arbitrary manner. Changing them only at the back (where > > they cannot hide existing attributes) may be safer and easier. (I'm > > not certain of this; but I'm not ready to rule it out either.) > I'm not sure how this is supposed to work, though, without changing > bases. I started to write up a sentence much like you wrote above, > then didn't, because I couldn't see how you could do this without > changing bases. Bases can be changed even today. But if changes are restricted to "interface" bases, then the same "it doesn't change often" optimizations used today can still be made for "regular" attributes. -jJ From bwinton at latte.ca Mon Dec 11 23:06:07 2006 From: bwinton at latte.ca (Blake Winton) Date: Mon, 11 Dec 2006 17:06:07 -0500 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> Message-ID: <457DD64F.5040508@latte.ca> Brett Cannon wrote: > This has been argued about before. It has been suggested we actually > ditch the C version since we only want to maintain one version and the > Python version can be used by alternative Python implementations. > > This probably needs to be covered in a PEP that covers a stdlib > reorg/renaming. If someone does that, I'ld like to suggest removing re.match, because the match/search distinction seems to be one of the things that trips newbies up on a fairly regular basis (and tripped me up just a few weeks ago after years of using Python), and re.match( pat ) is equivalent to: re.search( "^" + pat ) (or could hopefully be made equivalent). Later, Blake. From jimjjewett at gmail.com Mon Dec 11 23:33:16 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Mon, 11 Dec 2006 17:33:16 -0500 Subject: [Python-3000] Implementations: A syntax for extending method dispatching beyond type/class inheritance In-Reply-To: <4577AC23.1080302@davious.org> References: <457368C0.4010608@davious.org> <4577AC23.1080302@davious.org> Message-ID: On 12/7/06, Dave Anderson wrote: > on 12/3/2006 7:54 PM Bill Janssen wrote: > > I'm not sure just what this adds over the simple "change_class" > > function that I've posted twice already. > I've took a couple passes in the archives, and I haven't found the > change_class posts yet. I think you are referring to lying about the > __class__. Did I guess correctly? (Sorry, an archive link would help me.) No, but look at http://mail.python.org/pipermail/python-3000/2006-November/004674.html Note that this avoids creating new objects if they aren't needed, and keeps the same memory layout (so it is avoids the "can't inherit from two builtins" problem, and the "can't *really* override str methods" problems.) > Has-Method Contracts > -------------------- > The first thing I loved about python (coming from java), was this > realization that I was no longer need be caught up in the inheritance > game. If I wanted to use an unspoken convention that I could get a > special string from every object just by defining name() in each of > those objects, that's all I had to do. It didn't matter which class > tree it came from. This already works, if method name is all you care about. The people wanting interfaces want reassurance that the (same-named) method has the same meaning as the one they're looking for. You honor an even more subtle version of this distinction by saying that happening to implement every method doesn't imply implementing the entire interface. > Implement (As a) vs Inherit (Is a) > ---------------------------------- > Basically, if I have some object Car and I want to be able to use it > like a dict, I don't necessarily consider it inheriting from dict (or > even UserDict). That is, I don't think a Car is a dict. > To me, even if Car happens to represent a dict by being able to be used > as a dict, the expression of this fact feels better expressed as merely > implementing dict. This is a fair request, and is part of why just using Base classes doesn't feel entirely right. Whether such a Car should be considered a dict (or as implementing a dict) is ... something I don't feel like arguing. But the differences of opinion are part of why it sometimes seems that people are talking past each other. > class Car(dict): # car... is a dict? hmm, a bad ring to it vs > class Car: > implements dict # oh, Cars can be used like a dict, got it > In addition to that subjective differentiation, there is a technical > argument. Without something like an implements declaration, if I ever > want to express a relationship to another class, but with different > internals, I am forced to first define an abstract class and then > inherit from that. With an implements declaration, I can say implements > dict and be free of inheriting any of its internal implementation. Not really. Because dict is implemented in C, with a well-known C API, you can't really change the internals without breaking at least a few things. The number of broken things for dict keeps decreasing, but for str ... trying to write UserStr suprised me. > (I concede here that I could, in fact, inherit UserDict, override all > the methods, and just not call super in my init.) Go ahead and don't bother to override the 2nd/3rd/etc level functions -- they may be a bit less efficient, but they'll work. Or even go ahead and not bother to override the methods that you don't care about (dangerous, but no worse than lying about what you implement). Go ahead and call super in your init. > Even with the nicely done ABCs, is there an easy way for a class to > declare that it can be used like a dict with its own implementation of > dict methods? With this syntax sketch, UserDict would remain the same > except: > class UserDict: > implements dict # dispatch can rely on this declaration > ... # rather than method-checking > class UserDict(dict): ... You have to recognize for yourself that the methods are all overridden. > Btw, if we formally differentiate that I'm declaring an implementation > (versus inherit and override), I could programatically double-check my > implementation with tools... > > class A: > implements Container > > def len(self) > ... > >>> interface_compliance(A) > A implements Container: > methods: len > missing: get, iterator You can still write this with base classes. class Container: ... def check_compliance(self): return (self.__class__.__len__ is not Container.__len__) and ... -jJ From qrczak at knm.org.pl Mon Dec 11 23:55:21 2006 From: qrczak at knm.org.pl (Marcin 'Qrczak' Kowalczyk) Date: Mon, 11 Dec 2006 23:55:21 +0100 Subject: [Python-3000] Switch/Case statement PEP In-Reply-To: <20061202151503.BBB2.JCARLSON@uci.edu> (Josiah Carlson's message of "Sat, 02 Dec 2006 15:17:40 -0800") References: <340FE892-D6AA-4DC8-A8A3-27C23093E59D@gmail.com> <4571D3C7.2070602@acm.org> <20061202151503.BBB2.JCARLSON@uci.edu> Message-ID: <877iwyt546.fsf@qrnik.zagroda> Josiah Carlson writes: > After reading Thomas' post and Talin's response, I'm going to have to > agree with Talin; syntax options are moot until we can decide what is > acceptable in the case EXPR. IMHO any expression should be acceptable, it should be evaluated when being compared, and there should be other means (independent from switch/case) to make named constants which may be expanded at compile time. -- __("< Marcin Kowalczyk \__/ qrczak at knm.org.pl ^^ http://qrnik.knm.org.pl/~qrczak/ From greg.ewing at canterbury.ac.nz Tue Dec 12 00:50:01 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 12 Dec 2006 12:50:01 +1300 Subject: [Python-3000] programmable syntax (was ... #4: interpolated strings ala perl) In-Reply-To: References: Message-ID: <457DEEA9.2040108@canterbury.ac.nz> Jim Jewett wrote: > def foo(d: sqlDriver): > s = SQLTemplate("select template goes here", args=(param1, param2)) > r = d.query(s) I know it's just an example, but SQL really isn't the best thing to be talking about here, because you should be using parameterised queries instead of quoting when dealing with SQL. -- Greg From greg.ewing at canterbury.ac.nz Tue Dec 12 00:50:08 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 12 Dec 2006 12:50:08 +1300 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> Message-ID: <457DEEB0.300@canterbury.ac.nz> Brett Cannon wrote: > It has been suggested we actually > ditch the C version since we only want to maintain one version and the > Python version can be used by alternative Python implementations. Then we would lose all the speed advantages that were presumably thought important when the C version was created in the first place. I say stop worrying and ditch the Python version. An alternative Python implementation is going to have to provide implementations of all the existing C-only modules. A few more isn't going to kill it. -- Greg From thomas at python.org Tue Dec 12 01:06:55 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 11 Dec 2006 16:06:55 -0800 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: <457DEEB0.300@canterbury.ac.nz> References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> Message-ID: <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> On 12/11/06, Greg Ewing wrote: > > Brett Cannon wrote: > > It has been suggested we actually > > ditch the C version since we only want to maintain one version and the > > Python version can be used by alternative Python implementations. > > Then we would lose all the speed advantages that were > presumably thought important when the C version was > created in the first place. > > I say stop worrying and ditch the Python version. An > alternative Python implementation is going to have to > provide implementations of all the existing C-only > modules. A few more isn't going to kill it. Except, of course, when the Python versions has features the C version does not (thinking specifically of StringIO and unicode here.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061211/ff0ff23a/attachment.htm From barry at python.org Tue Dec 12 03:51:07 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 11 Dec 2006 21:51:07 -0500 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: <457DEEB0.300@canterbury.ac.nz> References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> Message-ID: <9777822F-BB89-4421-9ED8-908D88B9894B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 11, 2006, at 6:50 PM, Greg Ewing wrote: > I say stop worrying and ditch the Python version. An > alternative Python implementation is going to have to > provide implementations of all the existing C-only > modules. A few more isn't going to kill it. The C and Python versions of some modules are not always identical. There can be subtle differences and those would need to be addressed before the Python version could be ditched. E.g. subclass-able StringIO. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRX4ZIHEjvBPtnXfVAQKTwAQAuZ/BFmTdrVRR3eqfb1mFkYfug6q2YhwQ RZVIOjlWioxbMUeD6rKF29GdkvCgxmVZMOLwrQ/2FMsWS4E71gumBWW9HrfAlE3h Ze7p1hoczeNl/aYagxliFGYcdDfWyM8HOBi3ArzTOxKEz1Y3wi/mTUKiMAKDkFNS 4IEvWU92zAg= =hQte -----END PGP SIGNATURE----- From jcarlson at uci.edu Tue Dec 12 04:23:06 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 11 Dec 2006 19:23:06 -0800 Subject: [Python-3000] Switch/Case statement PEP In-Reply-To: <877iwyt546.fsf@qrnik.zagroda> References: <20061202151503.BBB2.JCARLSON@uci.edu> <877iwyt546.fsf@qrnik.zagroda> Message-ID: <20061211191830.984A.JCARLSON@uci.edu> "Marcin 'Qrczak' Kowalczyk" wrote: > > Josiah Carlson writes: > > > After reading Thomas' post and Talin's response, I'm going to have to > > agree with Talin; syntax options are moot until we can decide what is > > acceptable in the case EXPR. > > IMHO any expression should be acceptable, it should be evaluated when > being compared, and there should be other means (independent from > switch/case) to make named constants which may be expanded at compile > time. The problem with allowing *any* expression is that then switch/case statements are, strictly speaking, a new syntax for a simplified if/elif/else chain; offering precisely no advantages over what currently exists. One important point of the entire Switch/Case statement discussion was that if we couldn't make it fast with dictionary dispatch, then it doesn't really have a use. Deciding on what can be made into compile time constants, will allow for dictionary dispatch, and gives the PEP a point. - Josiah From ronaldoussoren at mac.com Tue Dec 12 11:09:37 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 12 Dec 2006 11:09:37 +0100 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> Message-ID: On Dec 11, 2006, at 9:10 PM, Brett Cannon wrote: > On 12/10/06, Calvin Spealman wrote: >> >> Has anyone considered consolidating the module pairs that have both a >> C and Python implementation? For example, pickle and cPickle and >> StingIO and cStringIO. It seems like keeping both around might be >> counter productive. It leads to more code for bugs, issues when there >> are accidental differences between the two, leading to misdiagnosed >> errors or hard to find bugs when people test with pickle and run with >> cPickle. Just seems like all around, if there was found to be a >> reason >> to make a module in C over its original Python version, the original >> doesn't need to stick around. > > > This has been argued about before. It has been suggested we > actually ditch > the C version since we only want to maintain one version and the > Python > version can be used by alternative Python implementations. You're kidding right? In most, if not all, cases with dual implementations the C version is significanty faster. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20061212/424f156e/attachment.bin From rasky at develer.com Tue Dec 12 12:04:28 2006 From: rasky at develer.com (Giovanni Bajo) Date: Tue, 12 Dec 2006 12:04:28 +0100 Subject: [Python-3000] Python/C++ question In-Reply-To: <457C8F22.5060407@v.loewis.de> References: <44DA6C01.2040904@acm.org> <457BCFAC.4000405@v.loewis.de> <457C8F22.5060407@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> To me, the killer feature would be that in C++ you can implement a smart >> pointer which takes care of incref/decref automatically for 99% of the code. >> This would be a terrific tool for the extension/core writers. > > Of course, this would also break in presence of binary-incompatible > compilers, in particular when it relates to exception handling. Details? Is that a problem, given that you can't compile Python core and extensions with different MSVC versions? We could also avoid the smart pointer, but use cleanups anyway. For example: struct ScopedIncRef { public: ScopedIncRef(PyObject *o) { Py_INCREF(o); } ~ScopedIncRef() { Py_DECREF(o); } }; #define WITH_INCREF(o) for (ScopedIncRef _sc_ = ScopedIncRef(o), \ bool _stat_ = true; \ _stat_; \ _stat_ = false) Usage: WITH_INCREF(o) // do something with o or: WITH_INCREF(o) { // do // something // with o } A variadic version of this macro is also possible, of course: WITH_INCREF(o1,o2,o3) // do something with them (even if it requires using compiler-specific preprocessor extensions, available at least on MSVC and GCC). -- Giovanni Bajo From jimjjewett at gmail.com Tue Dec 12 18:09:11 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 12 Dec 2006 12:09:11 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> Message-ID: On 12/8/06, Guido van Rossum wrote: > I've been thinking about this too, and I think it's reasonable to let > the metaclass provide the dict to be used as locals. Is this something that could be done in 2.6, so long as the metaclass does happen to be set before the class statement? > This is easier when the metaclass is given in the class header, ... > I'd rather innovate a bit more and add keyword arguments to the class header, > class C(B1, B2, metaclass=Foo): > ... Something about that seems to be begging for either confusion (when can/must I use a keyword?) or a mess (which keywords can I use? What code handles them?) Would this be a reasonable use of a with- block? Better yet, could it be done with a class decorator? @meta(Foo) class C(B1, B2): ... Typically, the decorator wouldn't be *called* until too late, but the decorator would be *created* (specialized to Foo) in time. Could it open a with-context that resets the default metaclass for the duration of the class statement? (Strangely enough, when I tried this out, a with- block that set the module-global __metaclass__ seemed to work for classic classes but not for new-style classes. I didn't investigate beyond that. Code below the .sig for those who care.) -jJ """with-statement for metaclass""" from __future__ import with_statement from contextlib import contextmanager @contextmanager def meta(mcls): """Sets mcls as the default metaclass for the duration of the with-block""" global __metaclass__ try: oldmeta = __metaclass__ except NameError: oldmeta = None __metaclass__ = mcls yield if oldmeta is None: del __metaclass__ else: __metaclass__ = oldmeta ########## testing ############ class ChattyType(type): def __new__(cls, name, bases, dct): print "Allocating memory for class", name return type.__new__(cls, name, bases, dct) def __init__(cls, name, bases, dct): print "Init'ing (configuring) class", name super(ChattyType, cls).__init__(name, bases, dct) class N1(object): pass class O1: pass with meta(ChattyType): class N2(object): pass class O2: pass class Nesting1(object): class Nesting2(N2): class NestingO(O2): class NestingOb: class NestingNdeep(object): pass class N3(object): pass class O3: pass From thomas at python.org Tue Dec 12 19:11:13 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 12 Dec 2006 10:11:13 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> Message-ID: <9e804ac0612121011i23dc04b4tdb5a9ed7bede307b@mail.gmail.com> On 12/12/06, Jim Jewett wrote: > > On 12/8/06, Guido van Rossum wrote: > > I've been thinking about this too, and I think it's reasonable to let > > the metaclass provide the dict to be used as locals. I do wonder about this though: will it require a particular type (like a dict)? The current code uses fast locals, as a class is essentially a function (and practically, too; just look at the generated bytecode.) We'd either need a new mechanism to build classes that was separate from functions, or we'd have to slow down the entire function execution case to handler the 'custom locals'. I'm sure people will want it for normal functions, too, then (debuggers, for instance.) The slowdown might be reduced to near-insignificant by providing a fast type in C that can proxy or be subclassed for specific behaviour, but falls back to the same sort of vector indexing as fast locals. I know, I know, premature optimization and all, but this is a rather fundamental part of Python's speed. (Strangely enough, when I tried this out, a with- block that set the > module-global __metaclass__ seemed to work for classic classes but not > for new-style classes. I didn't investigate beyond that. Code below > the .sig for those who care.) That's not strange; only classic classes look at the global __metaclass__ :) The lookup goes: class dict (setting __metaclass__ in the class suite), metaclass of parent classes (first one not classic, provided there is no conflict when inheriting from multiple classes), __metaclass__ module global. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061212/d407b1b8/attachment.html From pje at telecommunity.com Tue Dec 12 19:20:33 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 12 Dec 2006 13:20:33 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <9e804ac0612121011i23dc04b4tdb5a9ed7bede307b@mail.gmail.com > References: <4579365A.3050008@acm.org> Message-ID: <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> At 10:11 AM 12/12/2006 -0800, Thomas Wouters wrote: >On 12/12/06, Jim Jewett ><jimjjewett at gmail.com> wrote: >>On 12/8/06, Guido van Rossum <guido at python.org> >>wrote: >> > I've been thinking about this too, and I think it's reasonable to let >> > the metaclass provide the dict to be used as locals. > >I do wonder about this though: will it require a particular type (like a >dict)? The current code uses fast locals, Uh, wha? Did this change in Py3K? Python 2.x doesn't do this and can't do it without introducing backward incompatibility. In 2.x, class bodies do NOT use fast locals, any more than module bodies do. In any case, the performance within the class suite isn't especially important; it's rare that there are any loops within a class body, or that one creates classes within frequently-executed loops. From thomas at python.org Tue Dec 12 20:07:07 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 12 Dec 2006 11:07:07 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> Message-ID: <9e804ac0612121107t7c6abe74j76274b203df51b24@mail.gmail.com> On 12/12/06, Phillip J. Eby wrote: > > At 10:11 AM 12/12/2006 -0800, Thomas Wouters wrote: > > > >On 12/12/06, Jim Jewett > ><jimjjewett at gmail.com> wrote: > >>On 12/8/06, Guido van Rossum <guido at python.org> > >>wrote: > >> > I've been thinking about this too, and I think it's reasonable to let > >> > the metaclass provide the dict to be used as locals. > > > >I do wonder about this though: will it require a particular type (like a > >dict)? The current code uses fast locals, > > Uh, wha? Did this change in Py3K? Python 2.x doesn't do this and can't > do > it without introducing backward incompatibility. In 2.x, class bodies do > NOT use fast locals, any more than module bodies do. > > In any case, the performance within the class suite isn't especially > important; it's rare that there are any loops within a class body, or that > one creates classes within frequently-executed loops. I apologize. It doesn't use STORE_LOCAL. It is, however, executed as a normal function, and uses STORE_NAME. So the impact would be limited to that of changing a function's non-fast locals from a dict to an arbitrary mapping type. (Provided we don't specialcase class creation.) I do wonder why classes don't use fast locals, though. Building the locals dict from the fast locals at the end of the class suite execution seems more logical to me than what happens now... but I agree that it doesn't really matter. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061212/8529356b/attachment.htm From pje at telecommunity.com Tue Dec 12 20:17:15 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Tue, 12 Dec 2006 14:17:15 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <9e804ac0612121107t7c6abe74j76274b203df51b24@mail.gmail.com > References: <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> <4579365A.3050008@acm.org> <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061212141425.0271e4b8@sparrow.telecommunity.com> At 11:07 AM 12/12/2006 -0800, Thomas Wouters wrote: >I do wonder why classes don't use fast locals, though. Because fast locals were an optimization added for functions. Before that, there was no such thing as fast locals: everything used dictionaries. >Building the locals dict from the fast locals at the end of the class >suite execution seems more logical to me than what happens now... Well, code that does things like this: def some_func(an_ob, whatever): ... class Foo: some_func = some_func Wouldn't work any more if you switched to fast locals. Whether you consider that a good thing or a bad thing depends on your perspective, I suppose. Some people complain that the above pattern doesn't work in functions, due to the existence of fast locals. (I'm not one of them, however.) From greg.ewing at canterbury.ac.nz Wed Dec 13 00:42:22 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 13 Dec 2006 12:42:22 +1300 Subject: [Python-3000] Switch/Case statement PEP In-Reply-To: <877iwyt546.fsf@qrnik.zagroda> References: <340FE892-D6AA-4DC8-A8A3-27C23093E59D@gmail.com> <4571D3C7.2070602@acm.org> <20061202151503.BBB2.JCARLSON@uci.edu> <877iwyt546.fsf@qrnik.zagroda> Message-ID: <457F3E5E.9070103@canterbury.ac.nz> Marcin 'Qrczak' Kowalczyk wrote: > IMHO any expression should be acceptable, it should be evaluated when > being compared You'll have to be more precise about what you mean by "being compared". If you mean that the cases are to be tested one by one as in an if-else chain, there's little point in having a switch statement to begin with. -- Greg From greg.ewing at canterbury.ac.nz Wed Dec 13 00:47:45 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 13 Dec 2006 12:47:45 +1300 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> Message-ID: <457F3FA1.7010105@canterbury.ac.nz> Thomas Wouters wrote: > Except, of course, when the Python versions has features the C version > does not (thinking specifically of StringIO and unicode here.) Yes, I'm assuming the case where the Python and C versions are functionally equivalent. If not, then either the extra features of the Python version should be ported to the C version first, or the features should be partitioned into two non-overlapping modules. Having two almost-but-not-quite identically functioning modules is not a good idea. -- Greg From greg.ewing at canterbury.ac.nz Wed Dec 13 01:01:31 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 13 Dec 2006 13:01:31 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <5.1.1.6.0.20061212141425.0271e4b8@sparrow.telecommunity.com> References: <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> <4579365A.3050008@acm.org> <5.1.1.6.0.20061212131804.027b4600@sparrow.telecommunity.com> <5.1.1.6.0.20061212141425.0271e4b8@sparrow.telecommunity.com> Message-ID: <457F42DB.2040704@canterbury.ac.nz> Phillip J. Eby wrote: > At 11:07 AM 12/12/2006 -0800, Thomas Wouters wrote: > >>I do wonder why classes don't use fast locals, though. Probably because (a) class creation is rarely a speed bottleneck in any program, and (b) by using a dict you end up with it already in the right form for passing to the class constructor. > def some_func(an_ob, whatever): > ... > > class Foo: > some_func = some_func > > Wouldn't work any more if you switched to fast locals. Which wouldn't be a huge loss, since you can always rename the function to something different outside the class. -- Greg From thomas at python.org Wed Dec 13 05:51:07 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 12 Dec 2006 20:51:07 -0800 Subject: [Python-3000] The p3yk branch Message-ID: <9e804ac0612122051t722fb41amd9f780835ae741db@mail.gmail.com> There's a few more people working on the p3yk branch now, I think (or at least, there's been more work submitted than there used to :-), so I figure it's time to explain about the p3yk branch again. I think I already said most of these things once, anyway... it may have slipped my mind. Development of Python 3000 happens in the p3yk branch (and the usual submit-patches routine.) The p3yk branch is kept in sync with the trunk with 'svnmerge.py' (part of subversion's tools since 1.4, I think.) Usually (and up to now entirely) by me, but I don't mind if someone else wants to step up and learn the intense pain that is repeated merging in Subversion. If not, well, I'm apparently a masochist. Merging with svnmerge has a big consequence: changes that are applied to both the trunk and the p3yk branch are hell. Doubly so when there's extra changes ontop of the doubly-applied change in either or both branches. So, please, if you have such a change, please only check it in on the trunk, and have someone merge it using svnmerge. svnmerge can even be told to only merge a specific set of changes (although you may run into hidden dependencies that cause code to fail after the merge.) Merging between trunk and p3yk is quite often difficult, but it gets a lot harder if both sides are almost but not quite entirely like each other, you don't really know what is supposed to have changed, and you don't know the code all that well to begin with. Also, it's very important changes have working tests. It's always important, but even more so for the p3yk branch: if I fail to merge a change properly, most likely the *only* way we'll ever find out[1] is when code fails. It's a lot better if that code failure is during the execution of tests before the merge gets checked in, as opposed to two months after Python 3.0 is released. I haven't been merging at all, the last few months, because of trying to finish up work at XS4ALL, but I expect to be much more available for these things now that I started at Google. The trunk is less busy now, p3yk work will hopefully pick up, and a whole bunch of python developers can call me at work and tell me to do it. At least the p3yk branch is current now, as I just committed a 4-month merge. (Good thing my python-development machine is only a few meters away from the python.org subversion server :) ([1]: To be honest, considering the current state of branching/merging in subversion as well as the sanity-preserving new stuff that will be in svn 1.5, I might re-build and re-merge the entire p3yk branch before we get to actual Python 3.0 releases... it will serve to double-check my merging, and give us proper history in the p3yk branch (and eventually the trunk), which is immensely valuable.) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061212/4ea5f1fc/attachment.htm From rasky at develer.com Wed Dec 13 11:25:06 2006 From: rasky at develer.com (Giovanni Bajo) Date: Wed, 13 Dec 2006 11:25:06 +0100 Subject: [Python-3000] The p3yk branch In-Reply-To: <9e804ac0612122051t722fb41amd9f780835ae741db@mail.gmail.com> References: <9e804ac0612122051t722fb41amd9f780835ae741db@mail.gmail.com> Message-ID: Thomas Wouters wrote: > Merging with svnmerge has a big consequence: changes that are applied to > both the trunk and the p3yk branch are hell. You do know you can use svnmerge merge --record-only, cherry-picking the changes that have been applied on both branches, and then use a single svnmerge merge for everything else? You can't win when there are hidden dependencies of course (in my experience, the only way to avoid them is to merge VERY often, but "often" varies on the amount of traffic, and I have no clue how much traffic is being doing on python SVN). > So, please, if you have such a change, please only check it in > on the trunk, and have someone merge it using svnmerge. This policy is a very good thing IMO, but as I said above I don't think it's really critical. > but I don't mind if someone else wants to step up and learn the > intense pain that is repeated merging in Subversion. I can try and provide some help with that, but I'm not sure if it qualifies for "stepping up". I guess we could start with some IM sessions during the merges. Contact me in private for details, if you think my help can be useful. -- Giovanni Bajo From thomas at python.org Wed Dec 13 17:32:47 2006 From: thomas at python.org (Thomas Wouters) Date: Wed, 13 Dec 2006 17:32:47 +0100 Subject: [Python-3000] The p3yk branch In-Reply-To: References: <9e804ac0612122051t722fb41amd9f780835ae741db@mail.gmail.com> Message-ID: <9e804ac0612130832s5726ab43wdd8b430cc0e041d8@mail.gmail.com> On 12/13/06, Giovanni Bajo wrote: > > Thomas Wouters wrote: > > > Merging with svnmerge has a big consequence: changes that are applied to > > both the trunk and the p3yk branch are hell. > > You do know you can use svnmerge merge --record-only, cherry-picking the > changes that have been applied on both branches, and then use a single > svnmerge merge for everything else? Sure, but how do you tell that is has been properly applied to both branches? If they are the exact same change, it's not really a big deal unless lots of other work happened in either branch. The more common case, in p3yk, is for the changes to be subtly different (for instance, 'has_key' being replaced by 'in', in the p3yk branch, missing __future__ imports, indentation being different, etc.) You end up having to make sure there are no accidentally hidden changes in there. Either way, whether you do it by --record-only or by figuring out the result, you spend a lot of time checking individual changesets. > but I don't mind if someone else wants to step up and learn the > > intense pain that is repeated merging in Subversion. > > I can try and provide some help with that, but I'm not sure if it > qualifies > for "stepping up". I guess we could start with some IM sessions during the > merges. Contact me in private for details, if you think my help can be > useful. I'm not sure if collaborative merging makes sense, as the problem isn't tips or tricks in merging (I've had merging down pat for years,) but figuring out the code that is being merged. I'll be sure to holler if I'm merging any code ou touched, though :) And of course if I get stuck with svnmerge. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061213/718d10ed/attachment.htm From guido at python.org Fri Dec 15 06:51:25 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 14 Dec 2006 21:51:25 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) Message-ID: In the sandbox I've been working on a refactoring tool, which could form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to invite folks here to give it a try and give me a hand. It certainly needs more work, but I think that the basic infrastructure is sound. Check out sandbox/2to3/: http://svn.python.org/view/sandbox/trunk/2to3/. This message is to invite feedback, and to encourage contributions. It would be great if people tried their hands at writing new transformations! A brief description of how it works: We start with a slightly modified version of tokenize.py. The tokens are passed to a parsing engine (the Python re-implementation of pgen that I wrote at Elemental), also slightly modified, which builds up a concrete parse tree. Through some simple tricks (this is what I had to modify tokenize.py for), the parse tree contains annotations that hold on to all comments and whitespace between tokens. This allows us to reconstruct the original source *exactly*. (I tested this on all .py files in the Python distribution and every single file got reconstructed exactly.) Once we've build a parse tree for a file we can start refactoring. This is done by traversing the parse tree looking for nodes that match a given pattern; for matching nodes, we invoke a transformation method that returns a replacement node (or None if further inspection shows we can't actually do it despite the pattern matching); then this node is grafted into the tree in place of the originally matching node. Originally I created matching patterns by constructing a little tree of "pattern nodes". This quickly got tedious, and I designed a simple language for creating patterns (inspired by the language used to describe Python's grammar). The transformation routine is still just Python code; I would like to be able to use a more compact notation for this too, but it's more complicated since (in practice) there is a lot of ad-hoc testing that goes on. Well, have a look at the three example transformations in the code (in the "fixes" subdirectory). There's a driver program (refactor.py) that lets you repeat this process for any number of files or for all Python files in a given directory tree, with command line switches to select which transformations to apply and whether to actually write back the transformed files or just show the diffs that *would* be applied (the default). Let me describe how the "apply" transformation works, from a high level. This should change apply(f, x, y) into f(*x, **y) (and similar if y is missing). The pattern looks like this, in first approximation: power< 'apply' trailer< '(' arglist< any ',' any [',' any] > ')' > > This references the following rules from the Python grammar (I'm only showing the parts that matter for this example): power: atom trailer* ['**' factor] atom: NAME | ... trailer: '.' NAME | '[' subscriptlist ']' | '(' [arglist] ')' arglist: argument (',' argument)* In the pattern language, "power<...>" means "match a node of type 'power' whose subnodes match ...". In this example, the ... is 'apply' trailer<...>; then those ... are filled in with '(' arglist<...> ')' and finally those ... are: any ',' any [',' any]. Here 'any' is a special token that matches every possible node, and [...] indicates an optional part. Stuff in quotes of course means literal tokens; we use this to match specific keywords, identifiers or operators. Patterns match only the formal grammar; they completely ignore the whitespace and comments that are included in the parse tree as annotations. The pattern language has an additional useful feature: you can associate names with subpatterns, and the pattern matching process (if successful) will return a dictionary that maps those names to the tree nodes that matched those subpatterns. The syntax for this is currently to prefix the subpattern with "NAME="; I'm not sure if this is the most readable syntax but it's the best I could come up with. The final pattern language feature is negative look-ahead matches; something like (not xyzzy) forces the match to fail if the pattern xyzzy matches at this point. (I wanted the syntax to be !xyzzy but tokenizer.py doesn't like that because ! isn't a valid Python delimiter.) So now we have identified a node that matches the pattern. The transformation extracts the subnodes corresponding to f, x and y (if present), and then constructs a new node of the form f(*x, **y). There's one complication worth mentioning: when the original code is apply(f+y, a, b) then the transformation must be (f+y)(a, b). We add the parentheses only when really needed though. There are some unsolved problems. First of all, we don't have a symbol table. If apply is redefined in a particular scope, we don't notice this and still apply the transformation. And of course apply is a relatively simple case (being a free variable); has_key is more complicated, e.g. imagine x.y.z.has_key(a). Second, the transformations can sometimes lose comments. For example, if the input looks something like (x.has_key #blah (y)) the #blah comment is lost, since there is no good place to put it in the translation "(y in x)". Part of this problem is solvable by making the transformations preserve comments more carefully; but sometimes there just isn't a good place to put them. Maybe this is rare enough in practice that we could just leave such cases untranslated, instead flagging them as code that requires manual conversion. I also expect to run into problems with transformations for things like .keys(); right now the best solution I can think of is to translate xxx.keys() into list(xxx.keys()) and translate xxx.iterkeys() into iter(xxx.keys()). That's ugly; but I can't think of anything better without doing a ton of whole-program analysis, which I'd like to avoid. The .keys() transformation is also likely to make the tool non-idempotent; that's perhaps unavoidable, but still unfortunate, since it means you have to keep track of which files already have received which transformations. Finally, I have a dream: a GUI that will let you do this interactively, sort of like query-replace in Emacs. But this message is already too long, so I'll stop for now. Thanks for reading this far. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Fri Dec 15 07:56:49 2006 From: talin at acm.org (Talin) Date: Thu, 14 Dec 2006 22:56:49 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: References: Message-ID: <45824731.8010806@acm.org> Guido van Rossum wrote: > In the sandbox I've been working on a refactoring tool, which could > form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to > invite folks here to give it a try and give me a hand. It certainly > needs more work, but I think that the basic infrastructure is sound. > Check out sandbox/2to3/: > http://svn.python.org/view/sandbox/trunk/2to3/. > > This message is to invite feedback, and to encourage contributions. It > would be great if people tried their hands at writing new > transformations! And here I thought Py3K wasn't going to have programmable syntax :) I was working on a related idea recently, involving syntactical transformations, although not directly related to Python. One of the things I discovered was that it was possible in many cases to use the same parser on both the transformation patterns and the text to be translated. This allowed the transformation patterns to look a lot like the source and target patterns, instead of having to have a special syntax for the transformations. Ideally, you ought to be able to say something like: apply(?f, ?x, ?y) => f(*x, **y) You'd have to add an exception to the parser to recognize the unbound variable prefix ('?') when parsing patterns. You could also put constraints on the bound variables: apply(?(expr:f), ?x, ?y) => (f)(*x, **y) apply(?f, ?x, ?y) => f(*x, **y) In other words, if we need parens (because 'f' is an expression), then match the first rule, otherwise the second. Although a better way to handle the case of the parens is by repeated transformation: apply(?f, ?x, ?y) => paren(f)(*x, **y) paren(?(var:f)) => f paren(?f) => (f) In this case, we transform 'f' into 'paren(f)', and then transform that into either 'f' or '(f)' depending on whether it's a simple variable or an expression. This allows us to only have one version of the 'apply' rule. (Although the specific case of parenthesis logic ought to be built in, as you have already done, since it's going to clutter up every rule otherwise. Mainly I wanted to use it as an example to show constraints.) What I'm describing is exactly the same logic as an algebraic solver, which many people have written in Python already. One other thing you might want to do is look at the parser generator ANTLR (www.antlr.org), which has a syntax for tree transformations and pattern matching of tree fragments. Anyway - it looks pretty cool :) -- Talin From ajm at flonidan.dk Fri Dec 15 09:27:42 2006 From: ajm at flonidan.dk (Anders J. Munch) Date: Fri, 15 Dec 2006 09:27:42 +0100 Subject: [Python-3000] Refactoring tool available (work in progress) Message-ID: <9B1795C95533CA46A83BA1EAD4B01030031F69@flonidanmail.flonidan.net> GvR wrote: > > Finally, I have a dream: a GUI that will let you do this > interactively, sort of like query-replace in Emacs. But this message > is already too long, so I'll stop for now. Thanks for reading this > far. :-) Sounds like a job for revision control conflict resolution editors. Generate output that looks like a conflict: <<<<<<< .python2x apply(f, x, y) ======= f(*x, **y) >>>>>>> .python3 Lots of tools will conveniently edit something like that: KDiff3, TortoiseMerge etc. For Emacs, there's simple-merge-mode.el. - Anders From niki.spahiev at gmail.com Fri Dec 15 11:10:43 2006 From: niki.spahiev at gmail.com (Niki Spahiev) Date: Fri, 15 Dec 2006 12:10:43 +0200 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: References: Message-ID: <458274A3.4070003@gmail.com> Guido van Rossum wrote: > Second, the transformations can sometimes lose comments. For example, > if the input looks something like > > (x.has_key #blah > (y)) > > the #blah comment is lost, since there is no good place to put it in > the translation "(y in x)". What if result is (y in x #blah ) HTH Niki Spahiev From jason.orendorff at gmail.com Fri Dec 15 18:10:34 2006 From: jason.orendorff at gmail.com (Jason Orendorff) Date: Fri, 15 Dec 2006 12:10:34 -0500 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: <9B1795C95533CA46A83BA1EAD4B01030031F69@flonidanmail.flonidan.net> References: <9B1795C95533CA46A83BA1EAD4B01030031F69@flonidanmail.flonidan.net> Message-ID: On 12/15/06, Anders J. Munch wrote: > GvR wrote: > > > > Finally, I have a dream: a GUI that will let you do this > > interactively, sort of like query-replace in Emacs. But this message > > is already too long, so I'll stop for now. Thanks for reading this > > far. :-) > > Sounds like a job for revision control conflict resolution editors. > Generate output that looks like a conflict: > > <<<<<<< .python2x > apply(f, x, y) > ======= > f(*x, **y) > >>>>>>> .python3 Excellent idea! This leads to a funny conclusion, actually. These days I have source control everywhere. I review most changes in an edit-capable differ before checking in. So my diff tools are already all set up for reviewing changes (and making further changes during a review, like adding comments). So get this: Guido's dream is already implemented on my system! Give me the "--destructively-modify-in-place" option and I'll be all set. :) -j From guido at python.org Fri Dec 15 18:36:09 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Dec 2006 09:36:09 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: References: <9B1795C95533CA46A83BA1EAD4B01030031F69@flonidanmail.flonidan.net> Message-ID: Hm, I thought it's not a conflict unless there are *three* versions. However, Jason's suggestion is correct: instead of writing a new tool for this purpose, we should leverage existing tools. Cool! I can focus on writing transformations. On 12/15/06, Jason Orendorff wrote: > On 12/15/06, Anders J. Munch wrote: > > GvR wrote: > > > > > > Finally, I have a dream: a GUI that will let you do this > > > interactively, sort of like query-replace in Emacs. But this message > > > is already too long, so I'll stop for now. Thanks for reading this > > > far. :-) > > > > Sounds like a job for revision control conflict resolution editors. > > Generate output that looks like a conflict: > > > > <<<<<<< .python2x > > apply(f, x, y) > > ======= > > f(*x, **y) > > >>>>>>> .python3 > > Excellent idea! > > This leads to a funny conclusion, actually. These days I have source > control everywhere. I review most changes in an edit-capable differ > before checking in. So my diff tools are already all set up for > reviewing changes (and making further changes during a review, like > adding comments). > > So get this: Guido's dream is already implemented on my system! Give > me the "--destructively-modify-in-place" option and I'll be all set. > :) > > -j > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Dec 15 18:52:08 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Dec 2006 09:52:08 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: <45824731.8010806@acm.org> References: <45824731.8010806@acm.org> Message-ID: I'm taking the idea of using the same parser with a grain of salt -- by the time you have added all the pattern matching syntax IMO it will look too cluttered. Also, your concept of "variable" is too constrained -- the parenthesization should not only depend on the form of the experssion, but also on the context. E.g. "if x.has_key(a+b):" can translate to "if a+b in x:", but "3*x.has_key(a+b)" has to translate to "3*(a+b in x)". I'm not sure how thinking about it as an algebraic solver helps -- is there any existing code I can borrow? I looked at this bit of ANTLR briefly: http://www.antlr.org/doc/sor.html#_bb8; from the example it looks like their notation isn't any more compact than the same thing written in Python (despite ANTLR's over-use of line-noise characters). In fact, I can probably write that example using my tool easily: the pattern is something like arith_expr and the transformation (after setting up t and n to point to the respective subtrees) could be something like try: value = int(n) except ValueError: return None if value != 0: return None return t.clone() --Guido On 12/14/06, Talin wrote: > Guido van Rossum wrote: > > In the sandbox I've been working on a refactoring tool, which could > > form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to > > invite folks here to give it a try and give me a hand. It certainly > > needs more work, but I think that the basic infrastructure is sound. > > Check out sandbox/2to3/: > > http://svn.python.org/view/sandbox/trunk/2to3/. > > > > This message is to invite feedback, and to encourage contributions. It > > would be great if people tried their hands at writing new > > transformations! > > And here I thought Py3K wasn't going to have programmable syntax :) > > I was working on a related idea recently, involving syntactical > transformations, although not directly related to Python. One of the > things I discovered was that it was possible in many cases to use the > same parser on both the transformation patterns and the text to be > translated. This allowed the transformation patterns to look a lot like > the source and target patterns, instead of having to have a special > syntax for the transformations. > > Ideally, you ought to be able to say something like: > > apply(?f, ?x, ?y) => f(*x, **y) > > You'd have to add an exception to the parser to recognize the unbound > variable prefix ('?') when parsing patterns. > > You could also put constraints on the bound variables: > > apply(?(expr:f), ?x, ?y) => (f)(*x, **y) > apply(?f, ?x, ?y) => f(*x, **y) > > In other words, if we need parens (because 'f' is an expression), then > match the first rule, otherwise the second. Although a better way to > handle the case of the parens is by repeated transformation: > > apply(?f, ?x, ?y) => paren(f)(*x, **y) > paren(?(var:f)) => f > paren(?f) => (f) > > In this case, we transform 'f' into 'paren(f)', and then transform that > into either 'f' or '(f)' depending on whether it's a simple variable or > an expression. This allows us to only have one version of the 'apply' rule. > > (Although the specific case of parenthesis logic ought to be built in, > as you have already done, since it's going to clutter up every rule > otherwise. Mainly I wanted to use it as an example to show constraints.) > > What I'm describing is exactly the same logic as an algebraic solver, > which many people have written in Python already. > > One other thing you might want to do is look at the parser generator > ANTLR (www.antlr.org), which has a syntax for tree transformations and > pattern matching of tree fragments. > > Anyway - it looks pretty cool :) > > -- Talin > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Fri Dec 15 19:31:17 2006 From: talin at acm.org (Talin) Date: Fri, 15 Dec 2006 10:31:17 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: References: <45824731.8010806@acm.org> Message-ID: <4582E9F5.9000302@acm.org> Guido van Rossum wrote: > I'm not sure how thinking about it as an algebraic solver helps -- is > there any existing code I can borrow? I suppose it depends on how complex your rules are going to be. The matching algorithms used in solvers are specialized for some fairly difficult cases. Two examples I can think of are (a) where the 'constants' of the match expression are leaves, rather than roots of the expression being matched, and (b) where subtrees of the match expression have multiple ambiguous matches. An example of the latter is handling the associative rule, i.e.: "?a + ?b" can against: x + y + z either as: (x + y) + z or as: x + (y + z) However, if you aren't dealing with those kinds of cases, then probably you don't need to go that route. Code-wise, I wouldn't want to offer what I have, because it's pretty rough and unformed. But the general theory is that the matcher for each sub-expression returns a generator of all possible matches. So for example, in the expression '?a + ?b', the '?a' matcher returns a generator that yields the series ({a=x}, {a=x+y}). The '+' matcher then passes this generator to the '?b' matcher, which essentially filters the output of '?a' - that is, it either removes results from the list that are inconsistent with its own situation, or returns results augmented with its own bindings. In this case, it would yield ({a=x,b=y+z}, {a=x+y,b=z}). The '+' matcher then yields the output of '?b' to the caller. If the entire expression doesn't match, then the result is generator that immediately ends, yielding no values. (Also, the code I have could be greatly improved by porting to 2.5, with the new 'yield as expression' feature. It would require a total rewrite though.) -- Talin From guido at python.org Fri Dec 15 22:19:47 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 15 Dec 2006 13:19:47 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: <4582E9F5.9000302@acm.org> References: <45824731.8010806@acm.org> <4582E9F5.9000302@acm.org> Message-ID: Look at the WildcardPattern class in pytree.py. It generates all the matches like you say. I don't think it's worth trying to optimize this much; I'm just doing a very naive non-greedy expansion and so far it works fine, mostly because it only needs to match at one level in the tree, which in most cases is pretty limited. The example you give works slightly different for us because the Python grammar defines an arith_expr as term ('+' term)* (rather than the customary arith_expr ['+' term], since our poor LL(1) pgen can't handle left-recursive rules). To match an arbitrary number of terms you'd use a pattern like this: arith_expr. --Guido On 12/15/06, Talin wrote: > Guido van Rossum wrote: > > I'm not sure how thinking about it as an algebraic solver helps -- is > > there any existing code I can borrow? > > I suppose it depends on how complex your rules are going to be. The > matching algorithms used in solvers are specialized for some fairly > difficult cases. Two examples I can think of are (a) where the > 'constants' of the match expression are leaves, rather than roots of the > expression being matched, and (b) where subtrees of the match expression > have multiple ambiguous matches. > > An example of the latter is handling the associative rule, i.e.: > > "?a + ?b" > > can against: > > x + y + z > > either as: > > (x + y) + z > > or as: > > x + (y + z) > > However, if you aren't dealing with those kinds of cases, then probably > you don't need to go that route. > > Code-wise, I wouldn't want to offer what I have, because it's pretty > rough and unformed. But the general theory is that the matcher for each > sub-expression returns a generator of all possible matches. So for > example, in the expression '?a + ?b', the '?a' matcher returns a > generator that yields the series ({a=x}, {a=x+y}). The '+' matcher then > passes this generator to the '?b' matcher, which essentially filters the > output of '?a' - that is, it either removes results from the list that > are inconsistent with its own situation, or returns results augmented > with its own bindings. In this case, it would yield ({a=x,b=y+z}, > {a=x+y,b=z}). The '+' matcher then yields the output of '?b' to the > caller. If the entire expression doesn't match, then the result is > generator that immediately ends, yielding no values. > > (Also, the code I have could be greatly improved by porting to 2.5, with > the new 'yield as expression' feature. It would require a total rewrite > though.) > > -- Talin > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fperez.net at gmail.com Sat Dec 16 01:53:03 2006 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 15 Dec 2006 19:53:03 -0500 Subject: [Python-3000] how about switching to a preprocessor? (Re: A better way to initialize PyTypeObject) References: <3d2ce8cb0612021509t51ea7fc9vd5d8037a9ebe3e6a@mail.gmail.com> <457224C6.6040909@canterbury.ac.nz> <3d2ce8cb0612042132h2aa1572do633e81d7dcfed164@mail.gmail.com> <457602ED.6050703@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Does anyone have any recommendations for a site > to host the repository? I'd really like to avoid > SourceForge if possible, as it makes me cringe > every time I go there -- cluttered pages, slow > response, way too complicated UI. You could contact Enthought: http://enthought.com/ They host a number of projects under http://foo.scipy.org. While their focus is scientific projects and Pyrex has a wider audience, it also enjoys great popularity amongst the numpy/scipy gang (myself included), so I think it's a reasonable thing to try. Disclaimer: IPython, which I lead, has been hosted by them for a long time. I'm both happy and grateful for it, but I have otherwise no stake or saying in their decisions. Regards, f ps - feel free to ask me directly for contact details if you are interested. From martin at v.loewis.de Sat Dec 16 13:22:33 2006 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 16 Dec 2006 13:22:33 +0100 Subject: [Python-3000] Python/C++ question In-Reply-To: References: <44DA6C01.2040904@acm.org> <457BCFAC.4000405@v.loewis.de> <457C8F22.5060407@v.loewis.de> Message-ID: <4583E509.9050505@v.loewis.de> Giovanni Bajo schrieb: >>> To me, the killer feature would be that in C++ you can implement a smart >>> pointer which takes care of incref/decref automatically for 99% of the code. >>> This would be a terrific tool for the extension/core writers. >> Of course, this would also break in presence of binary-incompatible >> compilers, in particular when it relates to exception handling. > > Details? For the smart pointer itself, the object layout may differ between compilers, and the mangled names for the constructor and destructor will differ. For exception handling, the layout of stack frames, and the way unwinding information is represented will differ. > Is that a problem, given that you can't compile Python core and > extensions with different MSVC versions? Yes. First, we currently *can* compile extensions with gcc on Windows; that may stop working. Furthermore, we currently have compatibility across compilers on Solaris, HP-UX, AIX, etc; we will lose that when we use C++ (except perhaps for Linux, where compilers tend to be ABI compatible with g++ these days). > struct ScopedIncRef > { > public: > ScopedIncRef(PyObject *o) { Py_INCREF(o); } > ~ScopedIncRef() { Py_DECREF(o); } > }; That won't work: the variable "o" is not available in the destructor. Regards, Martin From talin at acm.org Sat Dec 16 20:30:19 2006 From: talin at acm.org (Talin) Date: Sat, 16 Dec 2006 11:30:19 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> Message-ID: <4584494B.4090201@acm.org> Guido van Rossum wrote: > I've been thinking about this too, and I think it's reasonable to let > the metaclass provide the dict to be used as locals. This is easier > when the metaclass is given in the class header, e.g. by way of a base > class, or through the default metaclass. Although the default default > metaclass could of course have a hook to pass things on to another > metaclass specified using a __metaclass__ assignment, I'd rather > innovate a bit more and add keyword arguments to the class header, > e.g. > > class C(B1, B2, metaclass=Foo): > ... > > How can we design this feature without yet another gigantic > contentious thread? I'm losing my appetite for those. I wonder if the > right thing wouldn't be if you could do some serious research into the > best possible semantics, syntax and implementation, and then post a > fully-formed proposal (not quite a PEP). There is of course always the > danger that you might go off into the deep end with that, so I'll > leave it up to your judgement to decide when to post again. (Also, I'm > still quite restricted in my internet access, at least until Monday.) Well, the discussion happened anyway, although it wasn't as bad as it could have been. So here's the summary of issues. First, I didn't see much resistance to the basic idea itself. There are enough use cases, I think, to make it a substantial win regardless of how it looks. The main controversies are: 1) How generalized the syntax should be. Several people, upon seeing your suggestion of "metaclass=Foo", saw an opportunity to use that as a hook for a whole bunch of other class-definition-related features (interface, implements, etc.). This apparently caused a counter-reaction by people who didn't want to see this kind of open-ended generalization (or who objected for other reasons), and resulted in a number of proposals to deliberately limit the syntax in ways that made this kind of expansion was impossible. There's also Josiah Carlson's proposal which separates the 'metaclass' function into two parts, the 'dict creation part' and the 'class finishing part'. (I.e. type= and dict=.) I would rather see them unified, as it makes the class declaration syntax simpler, and it's easy enough to write a metaclass that does one or the other or both. My personal feeling on this issue is that just because the "keyword=X" syntax opens up other possibilities, doesn't mean we have to use them. I think what you've proposed is intuitive and attractive, and if we don't want to allow other kinds of keywords there, we don't have to. However, I don't have a strong feeling on the issue either way. I do slightly prefer "metaclass=" or "metatype=" to just "type=", because we're talking about the type of a type, and I think it's confusing to just say "the type is X" when we're declaring a class. I think it's better to make the distinction between type and metatype explicit. 2) What should the interface on the metaclass look like. The general idea is to have the metaclass create a mapping object which is used as the 'locals' dictionary for the suite following the class statement. There would be some special-named function of the metaclass, such as '__metadict__', which would construct a new mapping object. I haven't seen many alternative proposals to this. 3) How to implement it under the hood. As was mentioned in the discussion, class bodies do not use fast locals, so it's primarily a matter of swapping one dictionary for another. (I may be wrong here - there may be more involved than just that.) (I appreciate the requirement of more concrete implementation details as a useful bozo-filter, but I'm not confident in my ability to reason correctly about the Python C source code; At least, my past experiences with it have have less than satisfactory results in terms of time spent vs. benefit gained. If that makes me an ignorable bozo, then I'm cool with that. :) ) 4) Backwards compatibility It might be possible to retain backwards compatibility if desired. In the proposed scheme, the metaclass is invoked twice - once to create the dictionary, and once to 'finish' the class. Note that the second step is identical to the operation of the current __metaclass__ feature. Thus, we can say that if the older __metaclass__ syntax is used, then only the second, finishing step is performed; If the newer syntax is used, then the metaclass also has an opportunity to create the mapping object. -- Talin From jcarlson at uci.edu Sat Dec 16 21:14:07 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 16 Dec 2006 12:14:07 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4584494B.4090201@acm.org> References: <4584494B.4090201@acm.org> Message-ID: <20061216114246.423F.JCARLSON@uci.edu> Talin wrote: [snip] > The main controversies are: Controversy is perhaps overstating it. There was disagreement. But it wasn't 300 posts and no names were called (that I can remember). > 1) How generalized the syntax should be. > > Several people, upon seeing your suggestion of "metaclass=Foo", saw an > opportunity to use that as a hook for a whole bunch of other > class-definition-related features (interface, implements, etc.). This > apparently caused a counter-reaction by people who didn't want to see > this kind of open-ended generalization (or who objected for other > reasons), and resulted in a number of proposals to deliberately limit > the syntax in ways that made this kind of expansion was impossible. > > There's also Josiah Carlson's proposal which separates the 'metaclass' > function into two parts, the 'dict creation part' and the 'class > finishing part'. (I.e. type= and dict=.) I would rather see them > unified, as it makes the class declaration syntax simpler, and it's easy > enough to write a metaclass that does one or the other or both. The reason I offered a mechanism for separating them was because I don't particularly like the idea of hitting the metaclass twice (particularly a staticmethod, or base type assignment, which is just one more line of code for people to forget, and makes metaclass/dictionary reuse more restrictive), and because metaclasses, so far, haven't needed to be classes. This particular proposal would make it so that metaclasses would either need to be a class, or would need to assign to an attribute on the function metaclass in order to overload the dictionary semantics. Seems to be an unnecessary requirement. Here's an alternate mechanism. As we currently have... class Foo(A, B): __metaclass__ = mc Why not extend it with __dict__ (or __metadict__)? Semantically, it would be a special name during the execution of the class namespace. That is, when the class namespace is being executed, when it comes upon an assignment to __dict__, it performs md(current_locals_dictionary). If you want it to capture everything (__metaclass__, __slots__, etc., assignment), then you make sure to put it at the top. If people think __metaclass__, __slots__, etc., were a mistake, then they will probably think __dict__ is also a mistake in this context. And I suppose the point of all of this is to get rid of __metaclass__, so __dict__ is probably right out (though it really is the only backwards not-breaking syntax on the table). As an alternative, what about the previously proposed 'is' syntax... class Foo(A, B) is type: ... That could be trivially extended to support a tuple syntax for handling meta dictionary support... class Foo(A, B) is type, dict: ... I personally like 'from' rather than 'is', but that's because it sounds better to me... "class Foo, subclassing from A and B, from type and dict" Of course, then we still have __slots__. Will the madness never end? > 2) What should the interface on the metaclass look like. > > The general idea is to have the metaclass create a mapping object which > is used as the 'locals' dictionary for the suite following the class > statement. There would be some special-named function of the metaclass, > such as '__metadict__', which would construct a new mapping object. I > haven't seen many alternative proposals to this. Proposal: don't make it a callable on the metaclass. It forces metaclasses into being classes, which I think is overly restrictive, or it forces people to do things like... def mymeta(...): ... mymeta.__metadict__ = mydict > 4) Backwards compatibility > > It might be possible to retain backwards compatibility if desired. In > the proposed scheme, the metaclass is invoked twice - once to create the > dictionary, and once to 'finish' the class. Note that the second step is > identical to the operation of the current __metaclass__ feature. Any syntax-based scheme is, by definition, not fully backwards compatible. The best we can hope for is for new functionality to not work as intended in previous Pythons, but still be runnable. The only syntax that has that ability so far, is to leave __metaclass__ where it is and to add metadict functionality like the following... class Foo(A, B): __dict__ = ... #or __metadict__ In Python 2.x, the above would basically be ignored. Which is fine. But if we went with either of the two serious options proposed... class Foo(A, B, metaclass=...): ... class Foo(A, B) is ...: ... Then you couldn't share the code between 2.x and 3.x. I only mention this because you brought up backwards compatability, and the syntax-based scheme breaks backwards compatability. - Josiah From pje at telecommunity.com Sat Dec 16 21:23:54 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 16 Dec 2006 15:23:54 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4584494B.4090201@acm.org> References: <4579365A.3050008@acm.org> Message-ID: <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> At 11:30 AM 12/16/2006 -0800, Talin wrote: >The general idea is to have the metaclass create a mapping object which >is used as the 'locals' dictionary for the suite following the class >statement. There would be some special-named function of the metaclass, >such as '__metadict__', which would construct a new mapping object. I >haven't seen many alternative proposals to this. There's mine, where you simply create mcls(name, bases, {}) and then map locals operations to get/set/delattr operations on the class. This would presumably be done using a simple mapping proxy, but it would be a built-in type rather than the user having to implement their own mapping type. From talin at acm.org Sat Dec 16 21:32:05 2006 From: talin at acm.org (Talin) Date: Sat, 16 Dec 2006 12:32:05 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061216114246.423F.JCARLSON@uci.edu> References: <4584494B.4090201@acm.org> <20061216114246.423F.JCARLSON@uci.edu> Message-ID: <458457C5.8040604@acm.org> Josiah Carlson wrote: > Talin wrote: >> There's also Josiah Carlson's proposal which separates the 'metaclass' >> function into two parts, the 'dict creation part' and the 'class >> finishing part'. (I.e. type= and dict=.) I would rather see them >> unified, as it makes the class declaration syntax simpler, and it's easy >> enough to write a metaclass that does one or the other or both. > > The reason I offered a mechanism for separating them was because I don't > particularly like the idea of hitting the metaclass twice (particularly > a staticmethod, or base type assignment, which is just one more line of > code for people to forget, and makes metaclass/dictionary reuse more > restrictive), and because metaclasses, so far, haven't needed to be > classes. This particular proposal would make it so that metaclasses > would either need to be a class, or would need to assign to an attribute > on the function metaclass in order to overload the dictionary semantics. > Seems to be an unnecessary requirement. Most of the time, the only reason for adding a custom dict is so that the metaclass can post-process it. The most common use case that I can think of is a dict that remembers the order in which things were defined; This by itself is not useful unless there's a metaclass that can take advantage of that ordering information. So in other words, I don't see the two as independent at all. Yes, technically they might be, but in terms of how metaclasses get used in practice, I don't think they will be separate, and thus we're exposing an unnecessary implementation detail and making class declarations more cluttered than needed, in order to preserve a distinction that most users won't care about. > Of course, then we still have __slots__. Will the madness never end? I believe that with this new system, we'll eventually be able to eliminate the need for __slots__. The metaclass can examine the dictionary and create a '__slots__' member based on the decorators or wrappers of the various values within the dict. >> 4) Backwards compatibility >> >> It might be possible to retain backwards compatibility if desired. In >> the proposed scheme, the metaclass is invoked twice - once to create the >> dictionary, and once to 'finish' the class. Note that the second step is >> identical to the operation of the current __metaclass__ feature. > > Any syntax-based scheme is, by definition, not fully backwards > compatible. The best we can hope for is for new functionality to not > work as intended in previous Pythons, but still be runnable. No, I'm talking about allowing the old __metaclass__ syntax to still continue work in the new versions of Python. It won't have all the functionality of the new syntax, but it will still work as it did before - at least, until we decide to remove it. -- Talin From talin at acm.org Sat Dec 16 21:35:07 2006 From: talin at acm.org (Talin) Date: Sat, 16 Dec 2006 12:35:07 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061216114246.423F.JCARLSON@uci.edu> References: <4584494B.4090201@acm.org> <20061216114246.423F.JCARLSON@uci.edu> Message-ID: <4584587B.6050908@acm.org> Josiah Carlson wrote: >> 2) What should the interface on the metaclass look like. >> >> The general idea is to have the metaclass create a mapping object which >> is used as the 'locals' dictionary for the suite following the class >> statement. There would be some special-named function of the metaclass, >> such as '__metadict__', which would construct a new mapping object. I >> haven't seen many alternative proposals to this. > > Proposal: don't make it a callable on the metaclass. It forces > metaclasses into being classes, which I think is overly restrictive, or > it forces people to do things like... > > def mymeta(...): > ... > > mymeta.__metadict__ = mydict I suppose you could make the new syntax require a callable that returns a tuple of a dict and a finishing function, i.e.: def mymeta( ... ): def finish( dict ): ... return finish, {} class Foo( metaclass=mymeta ): ... -- Talin From talin at acm.org Sat Dec 16 21:38:09 2006 From: talin at acm.org (Talin) Date: Sat, 16 Dec 2006 12:38:09 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> Message-ID: <45845931.2090208@acm.org> Phillip J. Eby wrote: > At 11:30 AM 12/16/2006 -0800, Talin wrote: >> The general idea is to have the metaclass create a mapping object which >> is used as the 'locals' dictionary for the suite following the class >> statement. There would be some special-named function of the metaclass, >> such as '__metadict__', which would construct a new mapping object. I >> haven't seen many alternative proposals to this. > > There's mine, where you simply create mcls(name, bases, {}) and then map > locals operations to get/set/delattr operations on the class. This > would presumably be done using a simple mapping proxy, but it would be a > built-in type rather than the user having to implement their own mapping > type. I'm not sure I entirely understand this. I did think about the idea of defining special get/set methods on the class, but everything I came up with was more complicated than just creating a special locals dict. The main issue for me is that I think that its important to distinguish between get/set operations that are done at class definition time, and get/set operations that are done later, after the class is created. -- Talin From steven.bethard at gmail.com Sat Dec 16 22:39:59 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 16 Dec 2006 14:39:59 -0700 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <45845931.2090208@acm.org> References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> <45845931.2090208@acm.org> Message-ID: Phillip J. Eby wrote: > There's mine, where you simply create mcls(name, bases, {}) and then map > locals operations to get/set/delattr operations on the class. This > would presumably be done using a simple mapping proxy, but it would be a > built-in type rather than the user having to implement their own mapping > type. I think this is probably the right way to go. So a class statement like:: class (, metaclass=): would get translated to:: cls = (, ) followed by the execution of where XXX_NAME (x) is changed to LOAD_FAST (cls) followed by XXX_ATTR (x), e.g. STORE_NAME (x) is changed to:: LOAD_FAST (cls) STORE_ATTR (x) I didn't see the point of passing in an empty dict as the third argument to the metaclass. I think we could just drop that argument in Python 3.0, and if you wanted to write metaclasses that were compatible with Python 2.X and 3.0, you'd write them like this:: class metaclass(type): def __init__(cls, name, bases, bodydict=None): # only perform operations on cls; # make no assumptions about bodydict being there If people were no longer concerned with 2.X compatibility, they could just drop the extra bodydict parameter. It gets a little messier if you need to call the super-metaclass, since you should only pass on the ``bodydict`` attribute if it's Python 2.X. But hopefully that's uncommon enough that we don't have to mess up the normal Python 3.0 API for it. If someone absolutely has to do it, and has to have their code work in both 2.X and 3.0, they could do it like:: if bodydict is None: super(metaclass, cls).__init__(name, bases) else: super(metaclass, cls).__init__(name, bases, bodydict) > The main issue for me is that I think that its important to distinguish > between get/set operations that are done at class definition time, and > get/set operations that are done later, after the class is created. Why? Can you explain your use case? Everything I'd care to do would treat later get/set operations on the class in the same way. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From jcarlson at uci.edu Sat Dec 16 23:47:10 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sat, 16 Dec 2006 14:47:10 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <458457C5.8040604@acm.org> References: <20061216114246.423F.JCARLSON@uci.edu> <458457C5.8040604@acm.org> Message-ID: <20061216141813.4245.JCARLSON@uci.edu> Talin wrote: > > Josiah Carlson wrote: > > Talin wrote: > >> There's also Josiah Carlson's proposal which separates the 'metaclass' > >> function into two parts, the 'dict creation part' and the 'class > >> finishing part'. (I.e. type= and dict=.) I would rather see them > >> unified, as it makes the class declaration syntax simpler, and it's easy > >> enough to write a metaclass that does one or the other or both. > > > > The reason I offered a mechanism for separating them was because I don't > > particularly like the idea of hitting the metaclass twice (particularly > > a staticmethod, or base type assignment, which is just one more line of > > code for people to forget, and makes metaclass/dictionary reuse more > > restrictive), and because metaclasses, so far, haven't needed to be > > classes. This particular proposal would make it so that metaclasses > > would either need to be a class, or would need to assign to an attribute > > on the function metaclass in order to overload the dictionary semantics. > > Seems to be an unnecessary requirement. > > Most of the time, the only reason for adding a custom dict is so that > the metaclass can post-process it. The most common use case that I can > think of is a dict that remembers the order in which things were > defined; This by itself is not useful unless there's a metaclass that > can take advantage of that ordering information. Maybe I'm strange, but I don't like the precreation/double calling semantic. It just seems...sloppy? It requires modification of all current metaclasses to update to the new metaclass= syntax, just to possibly support a single operation, which, according to the discussions so far have but a single use-case: taking advantage of operation ordering of assignments in the class namespace. I'd rather not see class dictionary overloading than a metaclass double-call semantic. > So in other words, I don't see the two as independent at all. Yes, > technically they might be, but in terms of how metaclasses get used in > practice, I don't think they will be separate, and thus we're exposing > an unnecessary implementation detail and making class declarations more > cluttered than needed, in order to preserve a distinction that most > users won't care about. It's not so much "to preserve a distinction that most users won't care about", it's about not burdoning people who implement metaclasses with functionality "that most users won't care about". Think of it like a doctor working on a patient; first do no harm. If people think the syntax is broken, fine, lets fix it, but lets also not break every metaclass that exists just so that we can get functionality that very few people want or need. Seems like a waste to me. > > Of course, then we still have __slots__. Will the madness never end? > > I believe that with this new system, we'll eventually be able to > eliminate the need for __slots__. The metaclass can examine the > dictionary and create a '__slots__' member based on the decorators or > wrappers of the various values within the dict. At the point of class creation, the only place where attributes of the final instance are defined is within __init__, or any other callable from __init__. The removal of __slots__ by attempting to introspect on the class namespace is either *really hard* or impossible. Note that __slots__ removal, at least as they currently exist in the Python cookbook, basically all rely on introspecting on the argument list to __init__. One looks for leading underscores __init__(self, _a, _b), others use other semantics. Many of them attempt to generalize on problems such as... class point(object): __slots__ = ['x', 'y', 'z'] def __init__(self, x, y, z): self.x = x self.y = y self.z = z What makes it not generalizable is that things like the above is for one fairly small set of use case that is not covered by any of the autoslots implementations. > >> 4) Backwards compatibility > >> > >> It might be possible to retain backwards compatibility if desired. In > >> the proposed scheme, the metaclass is invoked twice - once to create the > >> dictionary, and once to 'finish' the class. Note that the second step is > >> identical to the operation of the current __metaclass__ feature. > > > > Any syntax-based scheme is, by definition, not fully backwards > > compatible. The best we can hope for is for new functionality to not > > work as intended in previous Pythons, but still be runnable. > > No, I'm talking about allowing the old __metaclass__ syntax to still > continue work in the new versions of Python. It won't have all the > functionality of the new syntax, but it will still work as it did before > - at least, until we decide to remove it. That not backwards compatability of new features, it is forwards compatability of old features. - Josiah From pje at telecommunity.com Sat Dec 16 23:51:05 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 16 Dec 2006 17:51:05 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <45845931.2090208@acm.org> <4579365A.3050008@acm.org> <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> <45845931.2090208@acm.org> Message-ID: <5.1.1.6.0.20061216174612.029efbd0@sparrow.telecommunity.com> At 02:39 PM 12/16/2006 -0700, Steven Bethard wrote: > > The main issue for me is that I think that its important to distinguish > > between get/set operations that are done at class definition time, and > > get/set operations that are done later, after the class is created. > >Why? Can you explain your use case? Everything I'd care to do would >treat later get/set operations on the class in the same way. The above was actually Talin's statement, but I agree that there are circumstances where you want to know when the contents of the class are essentially finalized: 1. Validation -- have a point at which it's an error to have *not* set certain things 2. Setup of data structures that are expensive or complex to update incrementally during the process of class construction For both of these things you want to know when the class is "ready", so to speak, and they are easily accommodated with today's metaclass construction protocol. From pje at telecommunity.com Sat Dec 16 23:45:31 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 16 Dec 2006 17:45:31 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <45845931.2090208@acm.org> References: <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> <4579365A.3050008@acm.org> <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> Message-ID: <5.1.1.6.0.20061216173010.029f3388@sparrow.telecommunity.com> At 12:38 PM 12/16/2006 -0800, Talin wrote: >Phillip J. Eby wrote: >>At 11:30 AM 12/16/2006 -0800, Talin wrote: >>>The general idea is to have the metaclass create a mapping object which >>>is used as the 'locals' dictionary for the suite following the class >>>statement. There would be some special-named function of the metaclass, >>>such as '__metadict__', which would construct a new mapping object. I >>>haven't seen many alternative proposals to this. >>There's mine, where you simply create mcls(name, bases, {}) and then map >>locals operations to get/set/delattr operations on the class. This would >>presumably be done using a simple mapping proxy, but it would be a >>built-in type rather than the user having to implement their own mapping type. > >I'm not sure I entirely understand this. I did think about the idea of >defining special get/set methods on the class, but everything I came up >with was more complicated than just creating a special locals dict. My proposal uses a special locals dict, but it's a single standard builtin one that looks like this: class Metadict: def __init__(self, cls): self.cls = cls def __getitem__(self, key): return getattr(self.cls, key) def __setitem__(self, key, value): setattr(self.cls, key, value) def __delitem__(self, key): delattr(self.cls, key) so, the class creation process is: cls = mcls(name, bases, {}) exec suite in Metadict(cls), globals() The idea is to avoid creating some new special-purpose mechanism for this, when we already have an existing one that would work nicely, given a suitable Metadict implementation. Any get/set/delitem operations on the metadict map directly to get/set/delattr operations on the metaclass instance. >The main issue for me is that I think that its important to distinguish >between get/set operations that are done at class definition time, and >get/set operations that are done later, after the class is created. Well, we could always treat the class as a context manager, and allow the metaclass to have __enter__/__exit__ methods. e.g.: cls = mcls(name, bases, {}) if hasattr(cls,'__enter__'): with cls: exec suite in Metadict(cls), globals() else: exec suite in Metadict(cls), globals() Unfortunately, this just highlights the attribute ambiguity problem where a class's attributes may be either provided by a metaclass, or be intended for the class' instances. A class could have an __enter__ because it's inheriting it from its base class. So, I suppose you would really need to check whether mcls has __enter__, but even then, you won't *execute* that __enter__ in the with: block, because an inherited __enter__ can override it. (It'd be really nice if there was a way to generally fix this ambiguity in Py3K, as it's a problem for any method that could be on either a class or an instance, but isn't implemented via a C-level slot. But that's probably another thread altogether.) From talin at acm.org Sun Dec 17 00:41:14 2006 From: talin at acm.org (Talin) Date: Sat, 16 Dec 2006 15:41:14 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> <45845931.2090208@acm.org> Message-ID: <4584841A.8090606@acm.org> Steven Bethard wrote: >> The main issue for me is that I think that its important to distinguish >> between get/set operations that are done at class definition time, and >> get/set operations that are done later, after the class is created. > > Why? Can you explain your use case? Everything I'd care to do would > treat later get/set operations on the class in the same way. Imagine a metaclass that's being used to declare a C struct. When the class is "finished", we examine the dictionary and compute the layout of the struct and it's size in bytes, taking into account padding bytes and alignment and such. Once the struct has been computed, however, you can't go back and add new fields, unless you want to add code that automatically re-compute the layout of the C struct when this occurs. More likely, attributes which were added to the class after class creation would be treated as regular Python class variables. Now, I'm not saying that this is the way a C struct class would have to work. I'm trying to craft an example - the idea is that once the 'finishing' function has been called, there are certain kinds of class member definitions that may no longer be available. (Other than that - your general sketch of how metaclasses ought to work makes a lot of sense to me.) -- Talin From greg.ewing at canterbury.ac.nz Sun Dec 17 00:53:31 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 17 Dec 2006 12:53:31 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4584494B.4090201@acm.org> References: <4579365A.3050008@acm.org> <4584494B.4090201@acm.org> Message-ID: <458486FB.1030700@canterbury.ac.nz> Talin wrote: > My personal feeling on this issue is that just because the "keyword=X" > syntax opens up other possibilities, doesn't mean we have to use them. Even if the syntax was never used for anything else, I still don't like it. Metaclasses are an elegant concept that deserves an equally elegant syntax. To my eyes, the keyword proposal just swaps one ugly syntax for another. The main use case for all this seems to be that some metaclasses would like to know the order in which attributes were defined. So here's an alternative proposal: Leave the syntax the way it is, and *always* use a specialised dict that remembers the order of definition. The metaclass can then use that information or ignore it as it wishes. -- Greg From greg.ewing at canterbury.ac.nz Sun Dec 17 00:55:03 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 17 Dec 2006 12:55:03 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061216114246.423F.JCARLSON@uci.edu> References: <4584494B.4090201@acm.org> <20061216114246.423F.JCARLSON@uci.edu> Message-ID: <45848757.2040906@canterbury.ac.nz> Josiah Carlson wrote: > That could be trivially extended to support a tuple syntax for handling > meta dictionary support... > > class Foo(A, B) is type, dict: > ... -1. You're mangling my lovely syntax -- it no longer makes any sense! -- Greg From talin at acm.org Sun Dec 17 01:18:13 2006 From: talin at acm.org (Talin) Date: Sat, 16 Dec 2006 16:18:13 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061216141813.4245.JCARLSON@uci.edu> References: <20061216114246.423F.JCARLSON@uci.edu> <458457C5.8040604@acm.org> <20061216141813.4245.JCARLSON@uci.edu> Message-ID: <45848CC5.3000908@acm.org> Josiah Carlson wrote: > Maybe I'm strange, but I don't like the precreation/double calling > semantic. It just seems...sloppy? It requires modification of all > current metaclasses to update to the new metaclass= syntax, just to > possibly support a single operation, which, according to the discussions > so far have but a single use-case: taking advantage of operation > ordering of assignments in the class namespace. I don't see why any existing metaclasses would need to be re-written - see below. > I'd rather not see class dictionary overloading than a metaclass > double-call semantic. I see it as more of a "begin / end" kind of operation. One function gets called to begin the creation of a new class, the second gets called to finish it. I think of it as compilation - one operation creates a new template object, and the second one compiles it. >>> Of course, then we still have __slots__. Will the madness never end? >> I believe that with this new system, we'll eventually be able to >> eliminate the need for __slots__. The metaclass can examine the >> dictionary and create a '__slots__' member based on the decorators or >> wrappers of the various values within the dict. > > At the point of class creation, the only place where attributes of the > final instance are defined is within __init__, or any other callable > from __init__. The removal of __slots__ by attempting to introspect on > the class namespace is either *really hard* or impossible. > > Note that __slots__ removal, at least as they currently exist in the > Python cookbook, basically all rely on introspecting on the argument > list to __init__. One looks for leading underscores __init__(self, _a, > _b), others use other semantics. Many of them attempt to generalize on > problems such as... > > class point(object): > __slots__ = ['x', 'y', 'z'] > def __init__(self, x, y, z): > self.x = x > self.y = y > self.z = z > > What makes it not generalizable is that things like the above is for one > fairly small set of use case that is not covered by any of the autoslots > implementations. Slots can be done today, with the existing __metaclass__ syntax: class Slot: pass def slot_holder(name, bases, cdict): slots = [] newdict = dict(__slots__=slots) for key, value in cdict.iteritems(): if value is Slot: slots.append( key ) else: newdict[ key ] = value return type(name, bases, newdict) class X: __metaclass__ = slot_holder x = y = z = Slot def __init__(self, x, y, z): self.x = x self.y = y self.z = z a = X( 1, 2, 3 ) >>>> 4) Backwards compatibility >>>> >>>> It might be possible to retain backwards compatibility if desired. In >>>> the proposed scheme, the metaclass is invoked twice - once to create the >>>> dictionary, and once to 'finish' the class. Note that the second step is >>>> identical to the operation of the current __metaclass__ feature. >>> Any syntax-based scheme is, by definition, not fully backwards >>> compatible. The best we can hope for is for new functionality to not >>> work as intended in previous Pythons, but still be runnable. >> No, I'm talking about allowing the old __metaclass__ syntax to still >> continue work in the new versions of Python. It won't have all the >> functionality of the new syntax, but it will still work as it did before >> - at least, until we decide to remove it. > > That not backwards compatability of new features, it is forwards > compatability of old features. Huh? I don't understand. In my lexicon, the term "backwards compatibility" generally means that existing code will continue to run on a new version of the interpreter and/or standard libraries. As I see it, any class that uses the "__metaclass__ = foo" syntax would continue to work exactly like it does today - assuming we want it to. I'm basically saying that the new proposal and the existing system do not interfere with each other, and share some common implementation elements, that's all. I don't see why you feel that all existing metaclasses would need to be re-written. -- Talin From pje at telecommunity.com Sun Dec 17 03:10:22 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Sat, 16 Dec 2006 21:10:22 -0500 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <458486FB.1030700@canterbury.ac.nz> References: <4584494B.4090201@acm.org> <4579365A.3050008@acm.org> <4584494B.4090201@acm.org> Message-ID: <5.1.1.6.0.20061216210527.029f3408@sparrow.telecommunity.com> At 12:53 PM 12/17/2006 +1300, Greg Ewing wrote: >The main use case for all this seems to be that >some metaclasses would like to know the order in >which attributes were defined. So here's an >alternative proposal: Leave the syntax the way it >is, and *always* use a specialised dict that >remembers the order of definition. The metaclass >can then use that information or ignore it as it >wishes. Pro: ordered dicts are an often-requested feature anyway Con: doesn't allow errors to occur at the line where they really are IOW, a metaclass that validates assignments can't complain about a bad value at the point where it's assigned. This is an annoyance now because an error occurring in the metaclass __init__ or __new__ looks like it's happening at the 'class' statement currently. So, if we have to have some kind of special dictionary-like thing, I prefer using one that sets attributes on the class object, as this allows you to catch errors at the line where they occur, in the body of the class. This approach also has the added side benefit of forcing us to clean up a few quirks in the object/class model. :) From jcarlson at uci.edu Sun Dec 17 20:59:20 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 17 Dec 2006 11:59:20 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <45848CC5.3000908@acm.org> References: <20061216141813.4245.JCARLSON@uci.edu> <45848CC5.3000908@acm.org> Message-ID: <20061217030235.4248.JCARLSON@uci.edu> Talin wrote: > > > Josiah Carlson wrote: > > Maybe I'm strange, but I don't like the precreation/double calling > > semantic. It just seems...sloppy? It requires modification of all > > current metaclasses to update to the new metaclass= syntax, just to > > possibly support a single operation, which, according to the discussions > > so far have but a single use-case: taking advantage of operation > > ordering of assignments in the class namespace. > > I don't see why any existing metaclasses would need to be re-written - > see below. > > I'd rather not see class dictionary overloading than a metaclass > > double-call semantic. > > I see it as more of a "begin / end" kind of operation. One function gets > called to begin the creation of a new class, the second gets called to > finish it. I think of it as compilation - one operation creates a new > template object, and the second one compiles it. I understand what you are trying for, but I don't see it as necessary or useful (in a simialr way how you don't see splitting the metaclass and dictionary as being useful). I like Greg's idea; leave the syntax as-is and always use an ordered dictionary. > >>> Of course, then we still have __slots__. Will the madness never end? > >> I believe that with this new system, we'll eventually be able to > >> eliminate the need for __slots__. The metaclass can examine the > >> dictionary and create a '__slots__' member based on the decorators or > >> wrappers of the various values within the dict. > > > > At the point of class creation, the only place where attributes of the > > final instance are defined is within __init__, or any other callable > > from __init__. The removal of __slots__ by attempting to introspect on > > the class namespace is either *really hard* or impossible. > > > > Note that __slots__ removal, at least as they currently exist in the > > Python cookbook, basically all rely on introspecting on the argument > > list to __init__. One looks for leading underscores __init__(self, _a, > > _b), others use other semantics. Many of them attempt to generalize on > > problems such as... > > > > class point(object): > > __slots__ = ['x', 'y', 'z'] > > def __init__(self, x, y, z): > > self.x = x > > self.y = y > > self.z = z > > > > What makes it not generalizable is that things like the above is for one > > fairly small set of use case that is not covered by any of the autoslots > > implementations. > > Slots can be done today, with the existing __metaclass__ syntax: > > class Slot: > pass > > def slot_holder(name, bases, cdict): > slots = [] > newdict = dict(__slots__=slots) > for key, value in cdict.iteritems(): > if value is Slot: > slots.append( key ) > else: > newdict[ key ] = value > > return type(name, bases, newdict) > > class X: > __metaclass__ = slot_holder > x = y = z = Slot > > def __init__(self, x, y, z): > self.x = x > self.y = y > self.z = z > > a = X( 1, 2, 3 ) The above *still* has the ugliness of needing to state the name of the slots attributes as many times as in the original version, which was one of the complaints about __slots__ use. That is to say, the same issue that existed with function decorators, exists today with __slots__. There are mechanisms to get around it (using one of the autoslots and attribute initializers in the Python cookbook)... class X(object): __metaclass__ = AutoSlots def __init__(self, _x, _y, _z): InitAttrs(self, locals()) However, I've not been convinced that > >>>> 4) Backwards compatibility > >>>> > >>>> It might be possible to retain backwards compatibility if desired. In > >>>> the proposed scheme, the metaclass is invoked twice - once to create the > >>>> dictionary, and once to 'finish' the class. Note that the second step is > >>>> identical to the operation of the current __metaclass__ feature. > >>> Any syntax-based scheme is, by definition, not fully backwards > >>> compatible. The best we can hope for is for new functionality to not > >>> work as intended in previous Pythons, but still be runnable. > >> No, I'm talking about allowing the old __metaclass__ syntax to still > >> continue work in the new versions of Python. It won't have all the > >> functionality of the new syntax, but it will still work as it did before > >> - at least, until we decide to remove it. > > > > That not backwards compatability of new features, it is forwards > > compatability of old features. > > Huh? I don't understand. In my lexicon, the term "backwards > compatibility" generally means that existing code will continue to run > on a new version of the interpreter and/or standard libraries. I'm thinking of it in the other direction. That the code I write for Py3k with the ability to use new features can still be run (perhaps with slightly different semantics). In this particular case, not changing syntax, and just changing semantics, there is nothing barring us from using a __future__ import to choose between using a standard dictionary and an ordered one to have the functionality in Python 2.6 (if such is desireable). Also, if the point of this new syntax is to remove the __metaclass__ definition from the class namespace, why are we even talking about keeping it for backwards compatibility? Especially when it would offer two ways of defining what the metaclass for a class is. > As I see it, any class that uses the "__metaclass__ = foo" syntax would > continue to work exactly like it does today - assuming we want it to. > I'm basically saying that the new proposal and the existing system do > not interfere with each other, and share some common implementation > elements, that's all. > > I don't see why you feel that all existing metaclasses would need to be > re-written. Say I have a metaclass X. Say that I've been using metaclass X since Python 2.2 days. One day I hear that shortly after Py3k, __metaclass__ is going away and I need to start using some other syntax (say the metaclass= syntax for arguments sake). So what do I do? class Foo(A, B, metaclass=X): ... And I think that I have fixed my implementation for Py3k+, nevermind that it doesn't work in Python 2.x anymore, I'm a forward thinker and I like the new syntax. X implemented a class registry, so when X is called twice (once for setup, and once for finalization), now the class gets registered twice, or perhaps I get a crash because my metaclass doesn't have a default dict argument that discriminates between the two cases (using Steven's semantics). Ok, so now I need to adjust my metaclass to check to see whether it is doing the setup or finalization step. That's fine, but now I've had to modify my metaclass, something that I was told I wouldn't need to do, and with __metaclass__ going away (presumably, otherwise would we create a new syntax?), I'm going to need to rewrite *all* of my metaclasses when I switch syntaxes. But I thought you said I wouldn't need to rewrite all of my metaclasses? My arguments against a double-calling semantic with a new syntax are: 1) Given a double-calling semantic, for people moving to the new syntax, they need to update their metaclasses, whether they want or need the new functionality or not. 2) Having a new syntax, in addition to the current __metaclass__ syntax, violates TOOWTDI. 3) Forcing rewrites, just to offer a single defined use case, seems to be a waste. Using Greg's suggestion of keeping the old syntax, but making the class dictionary always be ordered, has none of these problems. For people who don't care about ordering, etc., their metaclasses keep working. For those who want to care about ordering, they write new metaclasses and get the features. If they want to use their code in both Py3k and 2.x, they can check by version, or even check to see if the dictionary argument is a dict or not (type (dct) is dict). I would also mention that Greg's suggestion of keeping the old syntax requires the smallest amount of work to make happen. - Josiah From greg.ewing at canterbury.ac.nz Mon Dec 18 01:30:10 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 18 Dec 2006 13:30:10 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061217030235.4248.JCARLSON@uci.edu> References: <20061216141813.4245.JCARLSON@uci.edu> <45848CC5.3000908@acm.org> <20061217030235.4248.JCARLSON@uci.edu> Message-ID: <4585E112.7080103@canterbury.ac.nz> A point in favour of the current syntax and semantics is that you can do things like class C: class __metaclass__(type): ... or even, if you're feeling particularly devious, class C: def __metaclass__(name, bases, namespace): ... An enhanced namespace dict would remain compatible with both of these. -- Greg From jcarlson at uci.edu Mon Dec 18 01:44:42 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 17 Dec 2006 16:44:42 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061217030235.4248.JCARLSON@uci.edu> References: <45848CC5.3000908@acm.org> <20061217030235.4248.JCARLSON@uci.edu> Message-ID: <20061217120004.4255.JCARLSON@uci.edu> (wasn't quite done with that last message, so I'll just update the parts where I wasn't finished) Josiah Carlson wrote: > Talin wrote: > > I don't see why any existing metaclasses would need to be re-written - > > see below. For people using the old syntax, yes, but the old syntax is going away (otherwise why offer new syntax?), so rewrites are necessary. [snip slots stuff] > The above *still* has the ugliness of needing to state the name of the > slots attributes as many times as in the original version, which was one > of the complaints about __slots__ use. That is to say, the same issue > that existed with function decorators, exists today with __slots__. > There are mechanisms to get around it (using one of the autoslots and > attribute initializers in the Python cookbook)... > > class X(object): > __metaclass__ = AutoSlots > def __init__(self, _x, _y, _z): > InitAttrs(self, locals()) > > However, I've not been convinced that I've not been convinced that removing __metaclass__ and __slots__ is necessarily a good idea. Yes, they are ugly. But they work. And they offer a big *HEY SOMETHING IS SPECIAL HERE* marker. If the point of the syntax change is to remove ugly, fine. But I think that the point of changing what exists is to make life easier for people. Easier to write, maintain, and understand. Removing for the sake of removing ugly, your "x = y = z = Slots" works fine, but it doesn't make understanding it any easier. It seems like magic, and the semantics smells quite a bit like the "assignment decorators" proposal. But maybe that's just me. Is __slots__ and __metaclass__ magic? Sure, but only as much as all other __meth__ definitions (which aren't going away in Py3k). The real trick is that the magic is warned about via the use of double leading and trailing underscores. Removing the magic warning isn't helpful, especially if it is only to remove ugly. [snip done parts] - Josiah From steven.bethard at gmail.com Mon Dec 18 07:52:35 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 17 Dec 2006 23:52:35 -0700 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <20061217030235.4248.JCARLSON@uci.edu> References: <20061216141813.4245.JCARLSON@uci.edu> <45848CC5.3000908@acm.org> <20061217030235.4248.JCARLSON@uci.edu> Message-ID: On 12/17/06, Josiah Carlson wrote: > Say I have a metaclass X. Say that I've been using metaclass X since > Python 2.2 days. One day I hear that shortly after Py3k, __metaclass__ > is going away and I need to start using some other syntax (say the > metaclass= syntax for arguments sake). So what do I do? > > class Foo(A, B, metaclass=X): > ... > > And I think that I have fixed my implementation for Py3k+, nevermind > that it doesn't work in Python 2.x anymore, I'm a forward thinker and I > like the new syntax. X implemented a class registry, so when X is called > twice (once for setup, and once for finalization), now the class gets > registered twice, or perhaps I get a crash because my metaclass doesn't > have a default dict argument that discriminates between the two cases > (using Steven's semantics). Ok, so now I need to adjust my metaclass to > check to see whether it is doing the setup or finalization step. That's > fine, but now I've had to modify my metaclass, something that I was told > I wouldn't need to do, and with __metaclass__ going away (presumably, > otherwise would we create a new syntax?), I'm going to need to rewrite > *all* of my metaclasses when I switch syntaxes. > > But I thought you said I wouldn't need to rewrite all of my metaclasses? FWLIW, avoiding having to rewrite code was not a goal I was striving for. You're already going to have to rewrite for 3.0 any of your code that uses, for example, dict.itervalues(), dict.iteritems(), etc., and I thought people had accepted that because there *is* a way to write that code that should work in both 2.X and 3.0 (e.g. using only iter(dict) or dict.values(), etc.) So, for me, the goal was to make sure that there was a way to write code that worked in both 2.X and 3.0. That is, I didn't mind if you had to rewrite some 2.X code to make it future compatible, as long as it was *possible* to write such code. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From ronaldoussoren at mac.com Mon Dec 18 08:17:51 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Mon, 18 Dec 2006 08:17:51 +0100 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: References: <4579365A.3050008@acm.org> <5.1.1.6.0.20061216151955.029e7d18@sparrow.telecommunity.com> <45845931.2090208@acm.org> Message-ID: <618BD018-6785-4188-B9F8-D4DBE7551916@mac.com> On 16 Dec, 2006, at 22:39, Steven Bethard wrote: > >> The main issue for me is that I think that its important to >> distinguish >> between get/set operations that are done at class definition time, >> and >> get/set operations that are done later, after the class is created. > > Why? Can you explain your use case? Everything I'd care to do would > treat later get/set operations on the class in the same way. I don't know about Phillip, but my usecase for this would be setting up C datastructures that cannot be updated later on. That is, at the end of the class definition a metaclass (possibly written in C) sets up a datastructure in C from information found in the class dict with a library that doesn't allow updates of said datastructure afterwards. My main usage for that is PyObjC: we have objc.ivar() properties that create an instance variable in the corresponding Objective-C class. Those can be added during class construction but not afterwards (because that would break existing instances). Ronald > > STeVe > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/ > ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20061218/4bb79df6/attachment-0001.bin From talin at acm.org Mon Dec 18 08:46:18 2006 From: talin at acm.org (Talin) Date: Sun, 17 Dec 2006 23:46:18 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <458486FB.1030700@canterbury.ac.nz> References: <4579365A.3050008@acm.org> <4584494B.4090201@acm.org> <458486FB.1030700@canterbury.ac.nz> Message-ID: <4586474A.8080808@acm.org> Greg Ewing wrote: > The main use case for all this seems to be that > some metaclasses would like to know the order in > which attributes were defined. So here's an > alternative proposal: Leave the syntax the way it > is, and *always* use a specialised dict that > remembers the order of definition. The metaclass > can then use that information or ignore it as it > wishes. I'm not sure that this is the *only* use case for overriding the locals() dict at class definition time, but at the moment I can't think of any others. So I won't try to struggle too hard against this idea. Do you have a specific proposal as to how the information about the ordering of definitions could be extracted? I'm assuming that there would be an extra attribute that would return a list of keys in the order that they were defined. One tricky part here is if the same key were to be defined twice (i.e. being overwritten the second time), we probably would only want it to appear once, probably at the location where it was defined last. -- Talin From guido at python.org Tue Dec 19 00:39:51 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 15:39:51 -0800 Subject: [Python-3000] Py3k release schedule worries Message-ID: I am getting worried about the Py3k release schedule. According to PEP 3000, "I hope to have a first alpha release out sometime in 2007" which would seem to give us another year at least; but in my mind I've always interpreted this (and explained it to others) as "in the first half of 2007" which would give us more like half a year at most. With few exceptions, the discussions on the python-3000 list seem more about radical redesign of the language than about the relatively modest tweaks that I had in mind when I started the project. There have been a few attempts at discussing a new string type that would be Unicode-aware without always requiring storage for 4 bytes per character, but AFAICT it fizzled; nobody has proposed to help with getting the int unification branch, which is mostly done but still has 22 failing tests last time I looked. I've received a few contributions to the bytes implementation (thanks Thomas!) and Tomer has made valiant attempts to help with the redesign of the I/O library (for which my apologies for not providing enough feedback). My general impression is that the majority of the participants on the list are more interested in free-for-all language design discussions than in getting things done. As a symptom, I received very few responses to my announcement of a refactoring tool; the ones that I got were more of a theoretical nature "maybe look at this alternative approach" rather than "how can I help" or "here's a refactoring I wrote using your tool and here's a suggestion for how you can make it easier to write such refactorings". I do realize that I have quite a bit of responsibility myself. Some of you may have seen the Google video about my internal Google project ("Mondrian") -- this has been a really fun project, but also quite time consuming, and I've definitely spent less than 50% of my time on Python over the past 6 months; especially the time from September through November felt like I had no time to work on Python (my own choice, largely, in order to accomplish some aggressive goals for Mondrian's roll-out to all of Google). And when I recently had a whole day to spend on Py3k, I foolishly engaged in a discussion about interfaces and generic functions. But I'm going to mend my ways. Think of this email as an early new year's resolution (and not of the kind that fizzles within a week). I currently have over 60 threads in my inbox with "python-3000" in the subject that I haven't read, or haven't read completely, the oldest one dating back to April. Clearly I'm not ever going to be able to catch up on all those discussions, and the wisest thing for me to do would probably to just delete (well, archive -- after all I'm using Gmail :-) those threads and start over. Or maybe at least archive everything older than a week (that would leave only 8 threads -- still about 180 messages!). I want to change the general mood in the python-3000 list to one that is more conducive to getting things done and meeting a schedule. If we aim for an alpha in June 2007, and the schedule slips (as it always will), no harm is done. If we aimed for December, however, slippage would be more serious (since the PEP currently says 2007). If we want to have an alpha by June 2007, we should aim to have the last PEP in by April. I propose that all discussions on the python-3000 list should strive to produce a PEP. This doesn't get rid of all radical redesign, but it at least sets the bar higher -- a PEP needs to be specified to a very high level of detail, and that means that discussions where nobody is interested in doing that work will automatically be cut short. Perhaps we can create a python-4000 list for folks who would like to discuss changes to the language beyond Python 3.0 -- I'm not opposed to having an outlet for such discussions, I just believe that we've come to a point where they are endangering the project's chances of ever actually reaching a milestone at all. (It seems that some participants have forgotten the mantra "Not Perl Six." :-) I'd also like to set some specific goals for some of the major tasks. For example, there are some PEPs that I'd like to see written where the basic goal has been firmly established but the details need working out: - a collection of ABCs to be used with the standard types; see Bill Janssen's wiki page - optional signature annotations (but without type checking connotations); e.g. ``def foo(a: 42) -> "hello kitty": pass'' should be fine; see my Artima blogs - make keys(), items(), values() return pseudo-sets/collections rather than lists - a spec for the new I/O library (building on Bill Janssen's wiki and Tomer's work) - a spec for the string unification (Perhaps Fredrik has done some work on it in one of those threads that I haven't opened yet?) I'd also like to see people volunteer to help with the implementation of some of these projects, and with the implementation of projects that already have a PEP or don't need one. For example: - the new string formatting (s.format(a1, a2, ...)) - finishing the int unification branch - contributing refactorings - turning list comprehensions into syntactic sugar for generator expressions - implement refactorings that convert many of the changes listed in PEP 3000 For more inspiration, see the long list of proposed changes in PEP 3100. (But do pick the hard ones first -- I'm not worried about the simple ones, and some of the simple ones are even controversial, e.g. moving id or intern, or removing find/rfind). Perhaps the most controversial issue that I'd like to tackle is a standard library reorganization. This is so controversial that I don't even know where to begin. Maybe the refactoring tool will be able to help and can automatically convert imports using old locations to imports using new locations? Then if the new locations are also made available in Python 2.6, we'd even have an upgrade path. Who wants to help out here? There's a huge amount of work and if it is left to me it won't get done. Finally, what will I do myself? Apart from redirecting certain kinds of ideas to the python-4000 list, I will take upon me a lot of the implementation work, and a lot of the PEP writing. But even if I do use my 50% Python time to the fullest for this purpose, I will need a lot of help. And if you leave the string unification to me, I will just rip out the 8-bit string object in favor of the existing Unicode implementation. So please help! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Tue Dec 19 01:33:14 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 18 Dec 2006 16:33:14 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> On 12/18/06, Guido van Rossum wrote: > > I am getting worried about the Py3k release schedule. Hah, only now? I've been worried every time I archive 20-unread-message conversations in the python-3000 list -- which it feels like I've been doing every two days for months :) I'd also like to see people volunteer to help with the implementation > of some of these projects, and with the implementation of projects > that already have a PEP or don't need one. Me, I'm all for implementing stuff. I'll happily implement physically possible, somewhat-fleshed-out ideas anyone brings up -- regardless of python-3000's prevaling opinion. I'm really not for all the yakking, especially not the ideas that seem to be relevant only to very specific cases -- but then, I really don't care how easily Python code could be used as a DSL. I just want Python, and for Python to be Python. If that means special cases have to look uglier or be harder, too bad. As you may be aware, I'm finishing up the slice-removal that I was working on at the Python sprint. Parts of it will be going into the trunk (patches all submitted, reviews welcome), and when I merge them into the p3yk branch, I'll submit the rest for p3yk inclusion. After that, I was thinking of working on the 'super' improvements that have been suggested, or removing list comprehensions as separate construct (and cleaning up the grammar and code generator to boot). I don't care for the pre-select-metaclass idea (I see no point in it, except for the DSL thing I think is harmful to generic Python), nor for generic functions or complicated solve-world-hunger ABCs or interfaces -- or anything else that's needlessly complicated. I haven't decided yet whether the new I/O system or the 'low-storage unicode' ideas fall under those categories :) However, I can pick up string formatting if no one else does, or help with the int/long unification. I should have more than enough time for all of those well before PyCon, or at PyCon -- it's not like they're really challenging; string formatting is easy, int/long unification is mostly done by Martin. Perhaps the most controversial issue that I'd like to tackle is a > standard library reorganization. This is so controversial that I don't > even know where to begin. Maybe the refactoring tool will be able to > help and can automatically convert imports using old locations to > imports using new locations? Then if the new locations are also made > available in Python 2.6, we'd even have an upgrade path. Who wants to > help out here? There's a huge amount of work and if it is left to me > it won't get done. I'm afraid that this problem just needs a decision: How do you (Guido, not python-3000) want to organize the standard library? Maybe you already made that decision and it was in one of the threads I never read -- if so, sorry :P Anyway, there's lots to be said for all approaches to organizing the standard library, so I think you should just decide between java-like (deeply structured), the current (unstructured, but with modules renamed to fit styleguide, possibly with a 'stdlib' package covering all) or inbetween. I favour second option without package; the other two options will no doubt generate much discussion about how deep the structure should be and what categories it should cover. I'd rather spend time renaming modules than archiving threads :-) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061218/1ace79f0/attachment.html From guido at python.org Tue Dec 19 01:52:28 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 16:52:28 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/18/06, Thomas Wouters wrote: > On 12/18/06, Guido van Rossum wrote: > > I am getting worried about the Py3k release schedule. > > Hah, only now? I've been worried every time I archive 20-unread-message > conversations in the python-3000 list -- which it feels like I've been doing > every two days for months :) Me too, I just didn't have the guts to bring it up before. :-) [...] > > Perhaps the most controversial issue that I'd like to tackle is a > > standard library reorganization. This is so controversial that I don't > > even know where to begin. Maybe the refactoring tool will be able to > > help and can automatically convert imports using old locations to > > imports using new locations? Then if the new locations are also made > > available in Python 2.6, we'd even have an upgrade path. Who wants to > > help out here? There's a huge amount of work and if it is left to me > > it won't get done. > > I'm afraid that this problem just needs a decision: How do you (Guido, not > python-3000) want to organize the standard library? Maybe you already made > that decision and it was in one of the threads I never read -- if so, sorry > :P > Anyway, there's lots to be said for all approaches to organizing the > standard library, so I think you should just decide between java-like > (deeply structured), the current (unstructured, but with modules renamed to > fit styleguide, possibly with a 'stdlib' package covering all) or inbetween. > I favour second option without package; the other two options will no doubt > generate much discussion about how deep the structure should be and what > categories it should cover. I'd rather spend time renaming modules than > archiving threads :-) Ok, so be it. Let this be a pronouncement -- the only stdlib reorg we're doing will be (a) deleting silly old stuff; (b) rename modules that don't conform to the current module/package naming convention, like StringIO, cPickle or UserDict. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Tue Dec 19 01:53:55 2006 From: brett at python.org (Brett Cannon) Date: Mon, 18 Dec 2006 16:53:55 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Guido van Rossum wrote: > > I am getting worried about the Py3k release schedule. According to PEP > 3000, "I hope to have a first alpha release out sometime in 2007" > which would seem to give us another year at least; but in my mind I've > always interpreted this (and explained it to others) as "in the first > half of 2007" which would give us more like half a year at most. I always wondered about that due date and when you really wanted to cut an alpha. With few exceptions, the discussions on the python-3000 list seem more > about radical redesign of the language than about the relatively > modest tweaks that I had in mind when I started the project. Oh yeah. There > have been a few attempts at discussing a new string type that would be > Unicode-aware without always requiring storage for 4 bytes per > character, but AFAICT it fizzled; nobody has proposed to help with > getting the int unification branch, which is mostly done but still has > 22 failing tests last time I looked. I've received a few contributions > to the bytes implementation (thanks Thomas!) and Tomer has made > valiant attempts to help with the redesign of the I/O library (for > which my apologies for not providing enough feedback). My general impression is that the majority of the participants on the > list are more interested in free-for-all language design discussions > than in getting things done. As a symptom, I received very few > responses to my announcement of a refactoring tool; the ones that I > got were more of a theoretical nature "maybe look at this alternative > approach" rather than "how can I help" or "here's a refactoring I > wrote using your tool and here's a suggestion for how you can make it > easier to write such refactorings". I think another big reason, though, is people are taking the view of Py3k really far in terms of it being a clean slate. I have always viewd Python 3.0 as Python 2.x cleaned up. That leaves Python 3.whatever for new additions. But I think a lot of people have skipped past the goal of getting a cleaned-up Python 3.0 and made that the whiz-bang version. I know the reason I didn't jump in on the refactoring tool is that a) I was wondering how Jeremy's tool fit into all of this (if at all), b) I am busy with the import rewrite, c) my laptop is *still* in the shop and thus I can't hack at home, and d) it doesn't interest me as much as other things I want to do for Py3K. I do realize that I have quite a bit of responsibility myself. Some of > you may have seen the Google video about my internal Google project > ("Mondrian") -- this has been a really fun project, but also quite > time consuming, and I've definitely spent less than 50% of my time on > Python over the past 6 months; especially the time from September > through November felt like I had no time to work on Python (my own > choice, largely, in order to accomplish some aggressive goals for > Mondrian's roll-out to all of Google). And when I recently had a whole > day to spend on Py3k, I foolishly engaged in a discussion about > interfaces and generic functions. > > But I'm going to mend my ways. Think of this email as an early new > year's resolution (and not of the kind that fizzles within a week). > > I currently have over 60 threads in my inbox with "python-3000" in the > subject that I haven't read, or haven't read completely, the oldest > one dating back to April. Clearly I'm not ever going to be able to > catch up on all those discussions, and the wisest thing for me to do > would probably to just delete (well, archive -- after all I'm using > Gmail :-) those threads and start over. Or maybe at least archive > everything older than a week (that would leave only 8 threads -- still > about 180 messages!). Yeah, the Py3K threads tend to grow rather quickly. I want to change the general mood in the python-3000 list to one that > is more conducive to getting things done and meeting a schedule. If we > aim for an alpha in June 2007, and the schedule slips (as it always > will), no harm is done. If we aimed for December, however, slippage > would be more serious (since the PEP currently says 2007). > > If we want to have an alpha by June 2007, we should aim to have the > last PEP in by April. I propose that all discussions on the > python-3000 list should strive to produce a PEP. This doesn't get rid > of all radical redesign, but it at least sets the bar higher -- a PEP > needs to be specified to a very high level of detail, and that means > that discussions where nobody is interested in doing that work will > automatically be cut short. > > Perhaps we can create a python-4000 list for folks who would like to > discuss changes to the language beyond Python 3.0 -- I'm not opposed > to having an outlet for such discussions, I just believe that we've > come to a point where they are endangering the project's chances of > ever actually reaching a milestone at all. (It seems that some > participants have forgotten the mantra "Not Perl Six." :-) A Python pie-in-the-sky list (python-ideas?) seems reasonable. Let's python-dev focus on maintaining the current code, python-3000 on Py3K-specific work, and the ideas list to be where new ideas are vetted out and congealed into a PEP to bring to either python-3000 or python-dev. I'd also like to set some specific goals for some of the major tasks. > > For example, there are some PEPs that I'd like to see written where > the basic goal has been firmly established but the details need > working out: > > - a collection of ABCs to be used with the standard types; see Bill > Janssen's wiki page > - optional signature annotations (but without type checking > connotations); e.g. ``def foo(a: 42) -> "hello kitty": pass'' should > be fine; see my Artima blogs I have been thinking about this one, especially after the discussions of the patch floating around that has an initial implementation. I was planning on tackling this at some point so it was nice and simple (just store strings instead of actual types so it truly just documentation initially) and tied into my signature PEP. - make keys(), items(), values() return pseudo-sets/collections rather > than lists > - a spec for the new I/O library (building on Bill Janssen's wiki and > Tomer's work) > - a spec for the string unification (Perhaps Fredrik has done some > work on it in one of those threads that I haven't opened yet?) > > I'd also like to see people volunteer to help with the implementation > of some of these projects, and with the implementation of projects > that already have a PEP or don't need one. For example: > > - the new string formatting (s.format(a1, a2, ...)) > - finishing the int unification branch > - contributing refactorings > - turning list comprehensions into syntactic sugar for generator > expressions > - implement refactorings that convert many of the changes listed in PEP > 3000 > > For more inspiration, see the long list of proposed changes in PEP > 3100. (But do pick the hard ones first -- I'm not worried about the > simple ones, and some of the simple ones are even controversial, e.g. > moving id or intern, or removing find/rfind). > > Perhaps the most controversial issue that I'd like to tackle is a > standard library reorganization. This is so controversial that I don't > even know where to begin. Maybe the refactoring tool will be able to > help and can automatically convert imports using old locations to > imports using new locations? Then if the new locations are also made > available in Python 2.6, we'd even have an upgrade path. Who wants to > help out here? There's a huge amount of work and if it is left to me > it won't get done. Well, I have tried to move this forward before and it always just leads to a huge explosion of ideas and feelings and just doesn't get anywhere. I am willing to be the guy to don the flame suit once again on this topic and try to get this PEP written for strategy for renaming, reorganizing, and deprecating modules. But I will probably need some general guidance from what you are looking for in terms of scope (e.g., if I remember correctly you don't want a 'py' top-level namespace and you didn't like the idea of having like a 'net' package, etc.) and depth (does the renaming extend to methods?). I am definitely willing to commit to PEP 352 (new-style exceptions) and PEP 362 (signature PEP, but still needs a pronouncement). After that I am up for helping with the library re-org and probably with the type annotations stuff. I am considering my import work as implementation-specific and thus can go in at almost any point (assuming backwards-compatibility issues are not considered too horrible or performance is within reason) and thus not necessarily time critical (unless you want to force it into Py3K to specify "new" semantics), in which case I would shift that up in priority. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061218/7b0b7e36/attachment-0001.htm From guido at python.org Tue Dec 19 02:14:08 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 17:14:08 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Brett Cannon wrote: > I always wondered about that due date and when you really wanted to cut an > alpha. Despite what the PEP says I've always *thought* of it as "early 2007" which I can stretch to June but not much beyond. :-) > I know the reason I didn't jump in on the refactoring tool is that a) I was > wondering how Jeremy's tool fit into all of this (if at all), I think Jeremy's version is de-facto dead. I discussed the two approaches briefly with him, when my version was about as lame as his. I never could get his version to produce valid output even from the example checked in; it seems to work for him and only for him. When I explained how I am doing it he seemed to agree that it's better to get the tokens and the tree from the same parser and in the same parsing run -- that approach just didn't appear possible to him because he wasn't aware of (or had forgotten about) the pgen reimplementation in Python donated by Elemental, that is the crucial centerpiece of my version. > b) I am busy > with the import rewrite, c) my laptop is *still* in the shop and thus I > can't hack at home, and d) it doesn't interest me as much as other things I > want to do for Py3K. Fair enough for sample of one. While I didn't mention the import rewrite, I hope that it will land in Py3k. > A Python pie-in-the-sky list (python-ideas?) seems reasonable. Let's > python-dev focus on maintaining the current code, python-3000 on > Py3K-specific work, and the ideas list to be where new ideas are vetted out > and congealed into a PEP to bring to either python-3000 or python-dev. py-in-the-sky at python.org anyone? > > - optional signature annotations (but without type checking > > connotations); e.g. ``def foo(a: 42) -> "hello kitty": pass'' should > > be fine; see my Artima blogs > > I have been thinking about this one, especially after the discussions of the > patch floating around that has an initial implementation. I was planning on > tackling this at some point so it was nice and simple (just store strings > instead of actual types so it truly just documentation initially) and tied > into my signature PEP. Oops, I must have missed that there is a patch. Where? When you say "just store strings" do you mean that the implementation would be limited to strings or just that you would only use it to store per-argument docstrings? The latter is fine with me; the former seems an unnecessary restriction. > > Perhaps the most controversial issue that I'd like to tackle is a > > standard library reorganization. This is so controversial that I don't > > even know where to begin. Maybe the refactoring tool will be able to > > help and can automatically convert imports using old locations to > > imports using new locations? Then if the new locations are also made > > available in Python 2.6, we'd even have an upgrade path. Who wants to > > help out here? There's a huge amount of work and if it is left to me > > it won't get done. > > Well, I have tried to move this forward before and it always just leads to a > huge explosion of ideas and feelings and just doesn't get anywhere. I am > willing to be the guy to don the flame suit once again on this topic and try > to get this PEP written for strategy for renaming, reorganizing, and > deprecating modules. > > But I will probably need some general guidance from what you are looking for > in terms of scope (e.g., if I remember correctly you don't want a 'py' > top-level namespace and you didn't like the idea of having like a 'net' > package, etc.) and depth (does the renaming extend to methods?). Well, what do you think of my pronouncement in response to Thomas's mail (just rename a bunch of things that don't conform to our own naming standard)? That should limit the discussion to what's the best name for StringIO etc. > I am definitely willing to commit to PEP 352 (new-style exceptions) and PEP > 362 (signature PEP, but still needs a pronouncement). I promise to look at that next. > After that I am up > for helping with the library re-org and probably with the type annotations > stuff. I am considering my import work as implementation-specific and thus Did you mean implementation-independent? > can go in at almost any point (assuming backwards-compatibility issues are > not considered too horrible or performance is within reason) and thus not > necessarily time critical (unless you want to force it into Py3K to specify > "new" semantics), in which case I would shift that up in priority. I don't want new semantics for import, I just want the existing semantics cleaned up and more easily tweaked through subclassing. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Tue Dec 19 02:11:03 2006 From: thomas at python.org (Thomas Wouters) Date: Mon, 18 Dec 2006 17:11:03 -0800 Subject: [Python-3000] Moving forward: library reorganization Message-ID: <9e804ac0612181711j12ae1b7dia5c783e605e22998@mail.gmail.com> On 12/18/06, Guido van Rossum wrote: > > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > we're doing will be (a) deleting silly old stuff; (b) rename modules > that don't conform to the current module/package naming convention, > like StringIO, cPickle or UserDict. Let's discuss some practicalities then. How about we just start renaming modules in the p3yk branch, and make a new lib-old (it's currently empt) that holds aliases for the old names -- but that raise an ImportWarning when imported. (If that warning class still exists -- something tells me we disabled again, but I don't remember the details.) The refactoring tool should be usable for code rewrites, I think, considering there won't be any implicit relative imports anymore, but we could do it by search-and-replace or spot-the-warnings, too. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061218/ee2458fd/attachment.html From guido at python.org Tue Dec 19 02:37:22 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 17:37:22 -0800 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: <457F3FA1.7010105@canterbury.ac.nz> References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> <457F3FA1.7010105@canterbury.ac.nz> Message-ID: I think a reasonable solution here is to make the C version an optional implementation detail of the Python version, such as was done for the heapq module already (the C version is _heapq and automatically imported by heapq.py if it exists). If this requires that some of the C modules need to be upgraded to be more feature compatible with the Python versions, so be it -- that would be a nice summer-of-code project for someone. Also, the specific problem with StringIO (that the .py version supports Unicode while the C version doesn't) will hopefully go away with the string unification. --Guido On 12/12/06, Greg Ewing wrote: > Thomas Wouters wrote: > > > Except, of course, when the Python versions has features the C version > > does not (thinking specifically of StringIO and unicode here.) > > Yes, I'm assuming the case where the Python and C > versions are functionally equivalent. If not, then > either the extra features of the Python version should > be ported to the C version first, or the features > should be partitioned into two non-overlapping > modules. Having two almost-but-not-quite identically > functioning modules is not a good idea. > > -- > Greg > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Tue Dec 19 02:39:36 2006 From: brett at python.org (Brett Cannon) Date: Mon, 18 Dec 2006 17:39:36 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Guido van Rossum wrote: > > On 12/18/06, Brett Cannon wrote: > > I always wondered about that due date and when you really wanted to cut > an > > alpha. > > Despite what the PEP says I've always *thought* of it as "early 2007" > which I can stretch to June but not much beyond. :-) Gotcha. > I know the reason I didn't jump in on the refactoring tool is that a) I > was > > wondering how Jeremy's tool fit into all of this (if at all), > > I think Jeremy's version is de-facto dead. I discussed the two > approaches briefly with him, when my version was about as lame as his. > I never could get his version to produce valid output even from the > example checked in; it seems to work for him and only for him. When I > explained how I am doing it he seemed to agree that it's better to get > the tokens and the tree from the same parser and in the same parsing > run -- that approach just didn't appear possible to him because he > wasn't aware of (or had forgotten about) the pgen reimplementation in > Python donated by Elemental, that is the crucial centerpiece of my > version. OK, glad to have that cleared up. > b) I am busy > > with the import rewrite, c) my laptop is *still* in the shop and thus I > > can't hack at home, and d) it doesn't interest me as much as other > things I > > want to do for Py3K. > > Fair enough for sample of one. While I didn't mention the import > rewrite, I hope that it will land in Py3k. OK. I will try to get it bootstrapped in then. > A Python pie-in-the-sky list (python-ideas?) seems reasonable. Let's > > python-dev focus on maintaining the current code, python-3000 on > > Py3K-specific work, and the ideas list to be where new ideas are vetted > out > > and congealed into a PEP to bring to either python-3000 or python-dev. > > py-in-the-sky at python.org anyone? Quaint. I can live with that. > > - optional signature annotations (but without type checking > > > connotations); e.g. ``def foo(a: 42) -> "hello kitty": pass'' should > > > be fine; see my Artima blogs > > > > I have been thinking about this one, especially after the discussions of > the > > patch floating around that has an initial implementation. I was > planning on > > tackling this at some point so it was nice and simple (just store > strings > > instead of actual types so it truly just documentation initially) and > tied > > into my signature PEP. > > Oops, I must have missed that there is a patch. Where? I mis-spoke. Tony Lownds has one but I didn't see a link in any of his emails to the code. When you say "just store strings" do you mean that the implementation > would be limited to strings or just that you would only use it to > store per-argument docstrings? The latter is fine with me; the former > seems an unnecessary restriction. So code objects would grow a co_annotations that would be a tuple of strings that stored what the annotations were as strings, with the last slot reserved for returns. Any parameter that didn't have an annotation would have None assigned to it. This means that this code:: def fxn(a:int, b, c:str='hi') -> float: pass would have a co_annotations of ``('int', None, 'str', float)``. Keeping everything as strings keeps it simple and minimizes any performance impact from them being declared. People can then easily write decorators that read the tuple (or use signature objects) to type-check as desired. Tony's implementation seems to actually fetch from globals what the annotation specifies and I just don't think that is necessary and unneeded overhead. Then again, functions are only created once and extra overhead could help discourage too much usage. But using strings and None like this cleanly solves the ambiguity issue of what value represents when no annotation was set (object, type?). > > Perhaps the most controversial issue that I'd like to tackle is a > > > standard library reorganization. This is so controversial that I don't > > > even know where to begin. Maybe the refactoring tool will be able to > > > help and can automatically convert imports using old locations to > > > imports using new locations? Then if the new locations are also made > > > available in Python 2.6, we'd even have an upgrade path. Who wants to > > > help out here? There's a huge amount of work and if it is left to me > > > it won't get done. > > > > Well, I have tried to move this forward before and it always just leads > to a > > huge explosion of ideas and feelings and just doesn't get anywhere. I > am > > willing to be the guy to don the flame suit once again on this topic and > try > > to get this PEP written for strategy for renaming, reorganizing, and > > deprecating modules. > > > > But I will probably need some general guidance from what you are looking > for > > in terms of scope (e.g., if I remember correctly you don't want a 'py' > > top-level namespace and you didn't like the idea of having like a 'net' > > package, etc.) and depth (does the renaming extend to methods?). > > Well, what do you think of my pronouncement in response to Thomas's > mail (just rename a bunch of things that don't conform to our own > naming standard)? That should limit the discussion to what's the best > name for StringIO etc. Sounds good. The only sticky point is whether we want to keep this dichotomy of C/Python implementations ala pickle and StringIO going. I personally would rather choose one (Python or C) and then work on getting the other to have the proper semantics. If we choose the C version we can stick the Python versions into lib-old or something so that if alternative Python implementations want to keep them alive and up-to-date they can for their own usage but the burden is taken off of us. > I am definitely willing to commit to PEP 352 (new-style exceptions) and > PEP > > 362 (signature PEP, but still needs a pronouncement). > > I promise to look at that next. OK. I can also bug you at PyCon about it (assuming you are not doing a PEP parade this year) and spend my sprint time on these two PEPs. > After that I am up > > for helping with the library re-org and probably with the type > annotations > > stuff. I am considering my import work as implementation-specific and > thus > > Did you mean implementation-independent? Yep. As in it could go into Py3K and/or Python 2.x. > can go in at almost any point (assuming backwards-compatibility issues are > > not considered too horrible or performance is within reason) and thus > not > > necessarily time critical (unless you want to force it into Py3K to > specify > > "new" semantics), in which case I would shift that up in priority. > > I don't want new semantics for import, I just want the existing > semantics cleaned up and more easily tweaked through subclassing. That's what I mean. I don't want to introduce new features, just maybe change stuff like saying that built-in modules are handles by a meta_path entry instead of monolithically by __import__ itself, etc. Maybe even changing its signature (don't know if globals and locals are really needed since you really only need __name__ and __path__ of the caller), etc. But that is a PEP topic and does not need to be worried about right now. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061218/2a40d689/attachment-0001.htm From brett at python.org Tue Dec 19 02:48:49 2006 From: brett at python.org (Brett Cannon) Date: Mon, 18 Dec 2006 17:48:49 -0800 Subject: [Python-3000] Moving forward: library reorganization In-Reply-To: <9e804ac0612181711j12ae1b7dia5c783e605e22998@mail.gmail.com> References: <9e804ac0612181711j12ae1b7dia5c783e605e22998@mail.gmail.com> Message-ID: On 12/18/06, Thomas Wouters wrote: > > > > On 12/18/06, Guido van Rossum wrote: > > > > > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > > we're doing will be (a) deleting silly old stuff; (b) rename modules > > that don't conform to the current module/package naming convention, > > like StringIO, cPickle or UserDict. > > > Let's discuss some practicalities then. How about we just start renaming > modules in the p3yk branch, and make a new lib-old (it's currently empt) > that holds aliases for the old names -- but that raise an ImportWarning when > imported. (If that warning class still exists -- something tells me we > disabled again, but I don't remember the details.) The refactoring tool > should be usable for code rewrites, I think, considering there won't be any > implicit relative imports anymore, but we could do it by search-and-replace > or spot-the-warnings, too. > ImportWarning is still there; it's used when you try to import a directory that doesn't have an __init_.py . It's tested in the test suite. As for using a lib-old idea, is that for Python 2.x to help transition, or did you want to do that for Py3K as well? I see the logic in the former to help transition but in the latter. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061218/a0451260/attachment.html From greg.ewing at canterbury.ac.nz Tue Dec 19 02:54:56 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 19 Dec 2006 14:54:56 +1300 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: <45874670.8030500@canterbury.ac.nz> Brett Cannon wrote: > I think another big reason, though, is people are taking the view of > Py3k really far in terms of it being a clean slate. I have always viewd > Python 3.0 as Python 2.x cleaned up. That leaves Python 3.whatever for > new additions. But I think a lot of people have skipped past the goal > of getting a cleaned-up Python 3.0 and made that the whiz-bang version. I think the reason for this is that we've been given the impression that the 2.x -> 3.0 transition is the one and only chance we'll get in our lifetimes for backward-incompatible changes -- so if we don't get them in now, we never will. But maybe this is the wrong impression. If it's going to be allowable to incorprorate more backward incompatible features as the 3.x series progresses, it would be good to make that clear to everyone. I'm sure it would do a lot to reduce the amount of radical stuff being suggested for 3.0. -- Greg From talin at acm.org Tue Dec 19 04:57:16 2006 From: talin at acm.org (Talin) Date: Mon, 18 Dec 2006 19:57:16 -0800 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4587464C.8080506@canterbury.ac.nz> References: <4579365A.3050008@acm.org> <4584494B.4090201@acm.org> <458486FB.1030700@canterbury.ac.nz> <4586474A.8080808@acm.org> <4587464C.8080506@canterbury.ac.nz> Message-ID: <4587631C.1050909@acm.org> Greg Ewing wrote: > Talin wrote: > >> Do you have a specific proposal as to how the information about the >> ordering of definitions could be extracted? I'm assuming that there >> would be an extra attribute that would return a list of keys in the >> order that they were defined. > > An extra attribute or method would be one obvious way. > > Another way would be to have the existing items() and > iteritems() methods return a list in the appropriate > order. That's a good idea. >> One tricky part here is if the same key were to be defined twice (i.e. >> being overwritten the second time), we probably would only want it to >> appear once, probably at the location where it was defined last. > > That sounds reasonable, but if anyone wanted to argue > that both definitions should be preserved, I wouldn't > object. Now that I think about it, you are right - it's probably better to let the metaclass deal with the ambiguous case if it really wants to. And that makes the dict implementation trivial. So it really sounds like all that is needed, then, is a suitable mapping class to use for class definition. It will have to be a built-in, however, since class creation depends on it. I'm not going to say anymore on this topic then, now that the marching orders have been given out. I'll move it over to the appropriate list when it gets created. :) -- Talin From talin at acm.org Tue Dec 19 05:02:04 2006 From: talin at acm.org (Talin) Date: Mon, 18 Dec 2006 20:02:04 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: <4587643C.6090700@acm.org> Thomas Wouters wrote: > On 12/18/06, Guido van Rossum wrote: > However, I can pick up string formatting if no one else does, or help with > the int/long unification. I should have more than enough time for all of > those well before PyCon, or at PyCon -- it's not like they're really > challenging; string formatting is easy, int/long unification is mostly done > by Martin. There's a Python prototype of the string formatting PEP in my Mercurial repository here: http://www.viridia.org/hg/python/string_format The main thing that needs to be done is to graft what's there in the prototype into the actual built-in string class. I don't know if that requires porting the prototype to C or not. It also needs unit tests; I can help with that part if needed. -- Talin From edcjones at comcast.net Tue Dec 19 05:37:14 2006 From: edcjones at comcast.net (Edward C. Jones) Date: Mon, 18 Dec 2006 23:37:14 -0500 Subject: [Python-3000] Refactoring Tool In-Reply-To: References: Message-ID: <45876C7A.1060308@comcast.net> Guido van Rossum" wrote: > As a symptom, I received very few > responses to my announcement of a refactoring tool; the ones that I > got were more of a theoretical nature "maybe look at this alternative > approach" rather than "how can I help" or "here's a refactoring I > wrote using your tool and here's a suggestion for how you can make it > easier to write such refactorings". Check out the ANTLR StringTemplate tool at "http://www.stringtemplate.org/". From edcjones at comcast.net Tue Dec 19 05:43:43 2006 From: edcjones at comcast.net (Edward C. Jones) Date: Mon, 18 Dec 2006 23:43:43 -0500 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 40 In-Reply-To: References: Message-ID: <45876DFF.40103@comcast.net> Guido van Rossum wrote: > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > we're doing will be (a) deleting silly old stuff; (b) rename modules > that don't conform to the current module/package naming convention, > like StringIO, cPickle or UserDict. I feel strongly that modules os, os.path, shutil, etc. that deal with files and paths need reorganizing. From greg.ewing at canterbury.ac.nz Tue Dec 19 02:54:20 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 19 Dec 2006 14:54:20 +1300 Subject: [Python-3000] Metaclasses in Py3K In-Reply-To: <4586474A.8080808@acm.org> References: <4579365A.3050008@acm.org> <4584494B.4090201@acm.org> <458486FB.1030700@canterbury.ac.nz> <4586474A.8080808@acm.org> Message-ID: <4587464C.8080506@canterbury.ac.nz> Talin wrote: > Do you have a specific proposal as to how the information about the > ordering of definitions could be extracted? I'm assuming that there > would be an extra attribute that would return a list of keys in the > order that they were defined. An extra attribute or method would be one obvious way. Another way would be to have the existing items() and iteritems() methods return a list in the appropriate order. > One tricky part here is if the same key > were to be defined twice (i.e. being overwritten the second time), we > probably would only want it to appear once, probably at the location > where it was defined last. That sounds reasonable, but if anyone wanted to argue that both definitions should be preserved, I wouldn't object. -- Greg From jcarlson at uci.edu Tue Dec 19 06:10:19 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Mon, 18 Dec 2006 21:10:19 -0800 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 40 In-Reply-To: <45876DFF.40103@comcast.net> References: <45876DFF.40103@comcast.net> Message-ID: <20061218210932.BC12.JCARLSON@uci.edu> "Edward C. Jones" wrote: > > Guido van Rossum wrote: > > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > > we're doing will be (a) deleting silly old stuff; (b) rename modules > > that don't conform to the current module/package naming convention, > > like StringIO, cPickle or UserDict. > > I feel strongly that modules os, os.path, shutil, etc. that deal with > files and paths need reorganizing. So think about what you think would make sense, perhaps propose it here, and/or write a PEP. - Josiah From guido at python.org Tue Dec 19 06:52:45 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 21:52:45 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <45874670.8030500@canterbury.ac.nz> References: <45874670.8030500@canterbury.ac.nz> Message-ID: On 12/18/06, Greg Ewing wrote: > Brett Cannon wrote: > > I think another big reason, though, is people are taking the view of > > Py3k really far in terms of it being a clean slate. I have always viewd > > Python 3.0 as Python 2.x cleaned up. That leaves Python 3.whatever for > > new additions. But I think a lot of people have skipped past the goal > > of getting a cleaned-up Python 3.0 and made that the whiz-bang version. > > I think the reason for this is that we've been given > the impression that the 2.x -> 3.0 transition is the > one and only chance we'll get in our lifetimes for > backward-incompatible changes -- so if we don't get > them in now, we never will. But many of the proposals being discussed aren't backwards compatible at all! They could be added to 2.6 just as easily as to 3.0. (Not all, just many.) Even if 3.0 *was* the only chance we ever get, and even if we were only discussing changes that *were* incompatible, I *still* would like to limit such discussions because otherwise we might as well set the release date for the year 3000. I'm not interested in waiting until it's perfect -- "good enough" is good enough for me. > But maybe this is the wrong impression. If it's > going to be allowable to incorprorate more backward > incompatible features as the 3.x series progresses, > it would be good to make that clear to everyone. > I'm sure it would do a lot to reduce the amount of > radical stuff being suggested for 3.0. No, after 3.2 or so we'll be as backward compatible as we ever were, probably more so if we are successful -- the more users we have the more compatible we must be. We could try for another incompatibility step at 4.0 but it's not clear that that will be possible. I'm allowing for small incompatibilities between 3.0, 3.1 and 3.2 only because I expect that 3.0, when it comes out, will be the first version that most users will try, and we will surely have to release a few quick upgrades to address the most egregious mistakes. But by 3.2 we should be stable. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 06:55:22 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 21:55:22 -0800 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 40 In-Reply-To: <20061218210932.BC12.JCARLSON@uci.edu> References: <45876DFF.40103@comcast.net> <20061218210932.BC12.JCARLSON@uci.edu> Message-ID: On 12/18/06, Josiah Carlson wrote: > > "Edward C. Jones" wrote: > > > > Guido van Rossum wrote: > > > > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > > > we're doing will be (a) deleting silly old stuff; (b) rename modules > > > that don't conform to the current module/package naming convention, > > > like StringIO, cPickle or UserDict. > > > > I feel strongly that modules os, os.path, shutil, etc. that deal with > > files and paths need reorganizing. > > So think about what you think would make sense, perhaps propose it here, > and/or write a PEP. Right. If you feel strongly about it, *do* it. Or shut up. The time to tell others what to do is past. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 06:57:30 2006 From: guido at python.org (Guido van Rossum) Date: Mon, 18 Dec 2006 21:57:30 -0800 Subject: [Python-3000] Refactoring Tool In-Reply-To: <45876C7A.1060308@comcast.net> References: <45876C7A.1060308@comcast.net> Message-ID: On 12/18/06, Edward C. Jones wrote: > Guido van Rossum" wrote: > > As a symptom, I received very few > > responses to my announcement of a refactoring tool; the ones that I > > got were more of a theoretical nature "maybe look at this alternative > > approach" rather than "how can I help" or "here's a refactoring I > > wrote using your tool and here's a suggestion for how you can make it > > easier to write such refactorings". > > Check out the ANTLR StringTemplate tool at "http://www.stringtemplate.org/". Sorry, that's precisely the kind of response I *don't* want to here. I'm sure I could learn something from ANTLR (I already looked at some other part of it) but I need your help in telling me what you know about it. If you count on me doing all the research, it's faster for me to write something from scratch than to wade through all the links on that page and find out what they mean. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas at python.org Tue Dec 19 08:59:07 2006 From: thomas at python.org (Thomas Wouters) Date: Tue, 19 Dec 2006 08:59:07 +0100 Subject: [Python-3000] Moving forward: library reorganization In-Reply-To: References: <9e804ac0612181711j12ae1b7dia5c783e605e22998@mail.gmail.com> Message-ID: <9e804ac0612182359o40b9feb9vfa1c827896d40827@mail.gmail.com> On 12/19/06, Brett Cannon wrote: > > As for using a lib-old idea, is that for Python 2.x to help transition, or > did you want to do that for Py3K as well? I see the logic in the former to > help transition but in the latter. > Development-transition only. I want to start doing it now (or next week, or at some point in the future) without having to think about the actual transition from py2x->py3k. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/e15c3fee/attachment.html From mail at manuzhai.nl Tue Dec 19 09:00:02 2006 From: mail at manuzhai.nl (Manuzhai) Date: Tue, 19 Dec 2006 09:00:02 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Guido van Rossum wrote: > - make keys(), items(), values() return pseudo-sets/collections rather > than lists Great to see you're back on the Py3k thing (but Mondrian looks really cool!). I got the impression before that keys(), items() and values() would become generators (or iterators) that don't return the whole thing at once. Is that still correct, or is there some other thing you mean by pseudo-sets/collections? Sorry if I am the only one confused here. Regards, Manuzhai From fredrik at pythonware.com Tue Dec 19 09:09:57 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 19 Dec 2006 09:09:57 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Guido van Rossum wrote: > - a spec for the string unification (Perhaps Fredrik has done some > work on it in one of those threads that I haven't opened yet?) I seem to remember that Google developers have access to other developer's home directories, but I thought that only applied to Google employees ;-) (I haven't abandoned this, but it hasn't been a top priority; partially because Larry Hastings work on smarter concatenation has showed that "lazy evaluation" can work in today's Python, and partially due to the schedule/discussion issues you write about.) From ncoghlan at gmail.com Tue Dec 19 10:07:50 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Dec 2006 19:07:50 +1000 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: <4587ABE6.6020608@gmail.com> Manuzhai wrote: > Guido van Rossum wrote: >> - make keys(), items(), values() return pseudo-sets/collections rather >> than lists > > Great to see you're back on the Py3k thing (but Mondrian looks really > cool!). > > I got the impression before that keys(), items() and values() would > become generators (or iterators) that don't return the whole thing at > once. Is that still correct, or is there some other thing you mean by > pseudo-sets/collections? Sorry if I am the only one confused here. A (long) while back we considered the implications of making them return pure iterators and didn't like the answer we got (a *lot* of broken and hard-to-fix code). The answer we came up with was to return a set-style view of the original collection rather than an actual iterator. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Tue Dec 19 10:27:13 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 19 Dec 2006 19:27:13 +1000 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> <457F3FA1.7010105@canterbury.ac.nz> Message-ID: <4587B071.5030703@gmail.com> Guido van Rossum wrote: > I think a reasonable solution here is to make the C version an > optional implementation detail of the Python version, such as was done > for the heapq module already (the C version is _heapq and > automatically imported by heapq.py if it exists). If this requires > that some of the C modules need to be upgraded to be more feature > compatible with the Python versions, so be it -- that would be a nice > summer-of-code project for someone. Also, the specific problem with > StringIO (that the .py version supports Unicode while the C version > doesn't) will hopefully go away with the string unification. Note that this approach has the additional benefit of being friendlier to a hybrid implementation where only some of the module functionality is provided by the C version (ala functools/_functools - one of the key goals for partial is to be faster than the equivalent function, so it's in C, while the rest of that module isn't likely to be performance critical, so it's left in Python). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From p.f.moore at gmail.com Tue Dec 19 11:25:44 2006 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 19 Dec 2006 10:25:44 +0000 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> On 12/19/06, Brett Cannon wrote: > > py-in-the-sky at python.org anyone? > > Quaint. I can live with that. While I'm not against it, python-ideas may be a better name, simply because it doesn't have a connotation that any conclusions reached on the list are unlikely to be implemented :-) Paul From fredrik at pythonware.com Tue Dec 19 13:12:51 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 19 Dec 2006 13:12:51 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Guido van Rossum wrote: > Perhaps we can create a python-4000 list for folks who would like to > discuss changes to the language beyond Python 3.0 an alternative would be to move concrete 3.0-related implementation discussions back to python-dev, and keep this list for Python 3.0 PEP work and "year 3000" stuff. From g.brandl at gmx.net Tue Dec 19 13:24:06 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 19 Dec 2006 13:24:06 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Guido van Rossum schrieb: [snip... full ACK] > For example, there are some PEPs that I'd like to see written where > the basic goal has been firmly established but the details need > working out: > > - a collection of ABCs to be used with the standard types; see Bill > Janssen's wiki page > - optional signature annotations (but without type checking > connotations); e.g. ``def foo(a: 42) -> "hello kitty": pass'' should > be fine; see my Artima blogs > - make keys(), items(), values() return pseudo-sets/collections rather > than lists > - a spec for the new I/O library (building on Bill Janssen's wiki and > Tomer's work) > - a spec for the string unification (Perhaps Fredrik has done some > work on it in one of those threads that I haven't opened yet?) > > I'd also like to see people volunteer to help with the implementation > of some of these projects, and with the implementation of projects > that already have a PEP or don't need one. For example: > > - the new string formatting (s.format(a1, a2, ...)) > - finishing the int unification branch > - contributing refactorings > - turning list comprehensions into syntactic sugar for generator expressions I'd like to point out that there is already my patch for that, which implements set comprehensions and list comprehensions exactly as syntactic sugar for GEs. This, however, affects performance greatly as LCs are executed in their own function scope, which isn't necessary. A better implementation would therefore leave the LC implementation as is, only preventing the name leaking into the enclosing scope. > - implement refactorings that convert many of the changes listed in PEP 3000 > > For more inspiration, see the long list of proposed changes in PEP > 3100. (But do pick the hard ones first -- I'm not worried about the > simple ones, and some of the simple ones are even controversial, e.g. > moving id or intern, or removing find/rfind). Oh, bad luck for me. I had intended to go over more of the easier ones and implement them... so what about id() and intern(). Care to pronounce? > Perhaps the most controversial issue that I'd like to tackle is a > standard library reorganization. This is so controversial that I don't > even know where to begin. Maybe the refactoring tool will be able to > help and can automatically convert imports using old locations to > imports using new locations? Then if the new locations are also made > available in Python 2.6, we'd even have an upgrade path. Who wants to > help out here? There's a huge amount of work and if it is left to me > it won't get done. Aside from reorganization, a large thing would be improvement of existing modules in the inside (see PEP 3001). I know some people who'd like to help by e.g. making legacy modules PEP 8 compatible. Would that be accepted, or is it too low-priority to care? cheers, Georg From fredrik at pythonware.com Tue Dec 19 13:31:36 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 19 Dec 2006 13:31:36 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Georg Brandl wrote: > so what about id() and intern(). Care to pronounce? moving/removing id()? please don't do that; id(obj) and type(obj) are essential tools for learning object semantics, and should be trivial to access and use. From barry at python.org Tue Dec 19 13:44:37 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 19 Dec 2006 07:44:37 -0500 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 40 In-Reply-To: <20061218210932.BC12.JCARLSON@uci.edu> References: <45876DFF.40103@comcast.net> <20061218210932.BC12.JCARLSON@uci.edu> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 19, 2006, at 12:10 AM, Josiah Carlson wrote: >> I feel strongly that modules os, os.path, shutil, etc. that deal with >> files and paths need reorganizing. > > So think about what you think would make sense, perhaps propose it > here, > and/or write a PEP. The nice thing about stdlib reorg is that you can do it now (or at least get pretty close to the py3k version). Distutils/setuptools, etc. Implement it now and release it to the Cheeseshop so we can all see it and try it out. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRYfeunEjvBPtnXfVAQIZAgP8DU+MBADOroTl/FjtW28NhdUL69BkC6hm HV9TTfc1/lhNq4RQXHACRwT4oV08hSBZovslnHnKSgFkow3CPY3hU9V5D98SvdKL OifyVGL6pVznmgmYDCVFpIQKjexYUqv6uQDSrQwGrATJ+VIXJ3QnCCHA5Y4uAV19 VDbvp4eYqV8= =0YZb -----END PGP SIGNATURE----- From fredrik at pythonware.com Tue Dec 19 14:03:20 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 19 Dec 2006 14:03:20 +0100 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 40 In-Reply-To: <20061218210932.BC12.JCARLSON@uci.edu> References: <45876DFF.40103@comcast.net> <20061218210932.BC12.JCARLSON@uci.edu> Message-ID: Josiah Carlson wrote: > So think about what you think would make sense, perhaps propose it here, > and/or write a PEP. here's the PEP: http://mail.python.org/pipermail/python-3000/2006-November/004147.html also see mike's concretization here: http://mail.python.org/pipermail/python-dev/2006-November/069747.html any reason this cannot go into 2.6? From jimjjewett at gmail.com Tue Dec 19 17:10:32 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Tue, 19 Dec 2006 11:10:32 -0500 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/18/06, Guido van Rossum wrote: > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > we're doing will be (a) deleting silly old stuff; (b) rename modules > that don't conform to the current module/package naming convention, > like StringIO, cPickle or UserDict. Clarification request: Is this a statement that anything beyond PEP8 renaming will be rejected, or only that proposals will be ignored until they come with a complete map inside a PEP? In other words, will we definately have the largely mechanical UserDict -> userdict cPickle -> _pickle or is there still an outside chance of UserDict -> collections.userdict or even cPickle -> formats._pickle -jJ From tony at PageDNA.com Tue Dec 19 17:44:10 2006 From: tony at PageDNA.com (Tony Lownds) Date: Tue, 19 Dec 2006 08:44:10 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: <3CFF6689-8F5A-4E59-9218-4DC5417A7497@PageDNA.com> On Dec 18, 2006, at 5:14 PM, Guido van Rossum wrote: > On 12/18/06, Brett Cannon wrote: > >>> - optional signature annotations (but without type checking >>> connotations); e.g. ``def foo(a: 42) -> "hello kitty": pass'' should >>> be fine; see my Artima blogs >> >> I have been thinking about this one, especially after the >> discussions of the >> patch floating around that has an initial implementation. I was >> planning on >> tackling this at some point so it was nice and simple (just store >> strings >> instead of actual types so it truly just documentation initially) >> and tied >> into my signature PEP. > > Oops, I must have missed that there is a patch. Where? > I did post a patch... #1607548. I'd be grateful for some review. I've asked Colin Winter to be co-author of the PEP. I'll post a new version of that in a couple of days when I have more downtime. Thanks -Tony From guido at python.org Tue Dec 19 18:57:54 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 09:57:54 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> References: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> Message-ID: On 12/19/06, Paul Moore wrote: > On 12/19/06, Brett Cannon wrote: > > > py-in-the-sky at python.org anyone? > > > > Quaint. I can live with that. > > While I'm not against it, python-ideas may be a better name, simply > because it doesn't have a connotation that any conclusions reached on > the list are unlikely to be implemented :-) Um, that was tongue-in-cheek. My serious proposal was python-4000, but python-ideas sounds better to me because it won't eventually outdate itself. Effbot wrote: > an alternative would be to move concrete 3.0-related implementation > discussions back to python-dev, and keep this list for Python 3.0 PEP > work and "year 3000" stuff. But that's confusing since the 3.0 PEP work is *also* concrete implementation related (at least in my mind it is supposed to be -- no proposal will be accepted unless it's clear how it can be implemented). And there are plenty of developers on python-dev who prefer to stick to python 2.6 work. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 19:00:49 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 10:00:49 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > - a spec for the string unification (Perhaps Fredrik has done some > > work on it in one of those threads that I haven't opened yet?) > > I seem to remember that Google developers have access to other > developer's home directories, but I thought that only applied to > Google employees ;-) You haven't been running Google desktop have you? :-) > (I haven't abandoned this, but it hasn't been a top priority; partially > because Larry Hastings work on smarter concatenation has showed that > "lazy evaluation" can work in today's Python, and partially due to the > schedule/discussion issues you write about.) Now's the time for that work to come out of the closet. Should I hold off on the basic unification-Guido-style (discarding str and renaming unicode to str, essentially) or can I start that now and will the new work be able to build on top of that? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 19:01:51 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 10:01:51 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/19/06, Jim Jewett wrote: > On 12/18/06, Guido van Rossum wrote: > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > > we're doing will be (a) deleting silly old stuff; (b) rename modules > > that don't conform to the current module/package naming convention, > > like StringIO, cPickle or UserDict. > > Clarification request: > > Is this a statement that anything beyond PEP8 renaming will be > rejected, or only that proposals will be ignored until they come with > a complete map inside a PEP? > > In other words, will we definately have the largely mechanical > > UserDict -> userdict > cPickle -> _pickle > > or is there still an outside chance of > > UserDict -> collections.userdict > or even > cPickle -> formats._pickle I thought I was pretty clear that I favor the former. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 19:03:36 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 10:03:36 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Fredrik Lundh wrote: > Georg Brandl wrote: > > > so what about id() and intern(). Care to pronounce? > > moving/removing id()? please don't do that; id(obj) and type(obj) are > essential tools for learning object semantics, and should be trivial to > access and use. Right. I tend to agree with Fredrik about id(). I'm not so sure about intern() -- IMO its use is rare enough that the move would affect very little code; I'm seeking validation of this. If nobody speaks up about it, moving intern() would be okay with me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 19:04:25 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 10:04:25 -0800 Subject: [Python-3000] Python-3000 Digest, Vol 10, Issue 40 In-Reply-To: References: <45876DFF.40103@comcast.net> <20061218210932.BC12.JCARLSON@uci.edu> Message-ID: Yes, please move this to python-dev. On 12/19/06, Fredrik Lundh wrote: > Josiah Carlson wrote: > > > So think about what you think would make sense, perhaps propose it here, > > and/or write a PEP. > > here's the PEP: > > http://mail.python.org/pipermail/python-3000/2006-November/004147.html > > also see mike's concretization here: > > http://mail.python.org/pipermail/python-dev/2006-November/069747.html > > any reason this cannot go into 2.6? > > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 19:06:15 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 10:06:15 -0800 Subject: [Python-3000] Modules with Dual Python/C Implementations In-Reply-To: <4587B071.5030703@gmail.com> References: <76fd5acf0612101841i4a063b78ne1048db01b8e651f@mail.gmail.com> <457DEEB0.300@canterbury.ac.nz> <9e804ac0612111606q558f2fb7v1526c5790c9b53ab@mail.gmail.com> <457F3FA1.7010105@canterbury.ac.nz> <4587B071.5030703@gmail.com> Message-ID: On 12/19/06, Nick Coghlan wrote: > Guido van Rossum wrote: > > I think a reasonable solution here is to make the C version an > > optional implementation detail of the Python version, such as was done > > for the heapq module already (the C version is _heapq and > > automatically imported by heapq.py if it exists). If this requires > > that some of the C modules need to be upgraded to be more feature > > compatible with the Python versions, so be it -- that would be a nice > > summer-of-code project for someone. Also, the specific problem with > > StringIO (that the .py version supports Unicode while the C version > > doesn't) will hopefully go away with the string unification. > > Note that this approach has the additional benefit of being friendlier to a > hybrid implementation where only some of the module functionality is provided > by the C version (ala functools/_functools - one of the key goals for partial > is to be faster than the equivalent function, so it's in C, while the rest of > that module isn't likely to be performance critical, so it's left in Python). *And* it's friendlier to Jython/IronPython/PyPy etc. Everybody wins. I suspect this can be done in 2.6 even. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Tue Dec 19 19:24:05 2006 From: talin at acm.org (Talin) Date: Tue, 19 Dec 2006 10:24:05 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> Message-ID: <45882E45.4080709@acm.org> Guido van Rossum wrote: > Um, that was tongue-in-cheek. My serious proposal was python-4000, but > python-ideas sounds better to me because it won't eventually outdate > itself. python-future. (python-ideas is a little too ambiguous, as it could also overlap with python-users.) -- Talin From g.brandl at gmx.net Tue Dec 19 19:45:36 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 19 Dec 2006 19:45:36 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > On 12/19/06, Fredrik Lundh wrote: >> Georg Brandl wrote: >> >> > so what about id() and intern(). Care to pronounce? >> >> moving/removing id()? please don't do that; id(obj) and type(obj) are >> essential tools for learning object semantics, and should be trivial to >> access and use. > > Right. I tend to agree with Fredrik about id(). I'm not so sure about > intern() -- IMO its use is rare enough that the move would affect very > little code; I'm seeking validation of this. If nobody speaks up about > it, moving intern() would be okay with me. Okay, I updated the patch at SF. While you're at it, in PEP 3100 there's "compile(): put in sys (or perhaps in a module of its own)". I guess that isn't really necessary either... really-longing-for-mondrian-pythondev-edition-ly y'rs Georg From guido at python.org Tue Dec 19 19:49:04 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 10:49:04 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <45882E45.4080709@acm.org> References: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> <45882E45.4080709@acm.org> Message-ID: On 12/19/06, Talin wrote: > Guido van Rossum wrote: > > Um, that was tongue-in-cheek. My serious proposal was python-4000, but > > python-ideas sounds better to me because it won't eventually outdate > > itself. > > python-future. (python-ideas is a little too ambiguous, as it could also > overlap with python-users.) The first one to convince postmaster at python.org to create the list wins. I'm outta there. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Tue Dec 19 19:52:54 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Tue, 19 Dec 2006 19:52:54 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> Message-ID: Guido van Rossum wrote: >> an alternative would be to move concrete 3.0-related implementation >> discussions back to python-dev, and keep this list for Python 3.0 PEP >> work and "year 3000" stuff. > > But that's confusing since the 3.0 PEP work is *also* concrete > implementation related (at least in my mind it is supposed to be -- no > proposal will be accepted unless it's clear how it can be > implemented). so keep it here while the details are being hammered out, and move it over to python-dev when it's time to start working on it (and move it back if it turns out that it wasn't ready, after all). after all, lots of 3.X stuff can go into 2.X anyway (especially after your pronouncements earlier today), and I think it's better to keep code-related activities in one place. > And there are plenty of developers on python-dev who > prefer to stick to python 2.6 work. yeah, but there are also plenty of people who want to contribute concrete things to Python, but don't have a clue where to start. the current situation is something of a mess. I fear that adding yet another forum will make things even messier. From barry at python.org Tue Dec 19 19:54:03 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 19 Dec 2006 13:54:03 -0500 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 19, 2006, at 1:03 PM, Guido van Rossum wrote: > On 12/19/06, Fredrik Lundh wrote: >> Georg Brandl wrote: >> >>> so what about id() and intern(). Care to pronounce? >> >> moving/removing id()? please don't do that; id(obj) and type(obj) >> are >> essential tools for learning object semantics, and should be >> trivial to >> access and use. > > Right. I tend to agree with Fredrik about id(). I'm not so sure about > intern() -- IMO its use is rare enough that the move would affect very > little code; I'm seeking validation of this. If nobody speaks up about > it, moving intern() would be okay with me. You don't want to get rid of 'interning' (e.g. PyString_Intern*()) but just the intern() built-in? We actually use both, but we use the C API call much more frequently than the built-in call. I don't think it would be too painful for us if intern() was removed, but OTOH if it was stuck in some Python internals module (sys?) and not kept as a built-in, that would be fine too. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRYg1T3EjvBPtnXfVAQJYagQAkW3/oJ8B5xE9hDN+36p8yp50BOMIUS8D r9gNT/fUiaHLlrGKpLTbyGZFMEb0AEPl6PXf3MU8d/8zZwRmDLRQqy3QBFKD7BCC uaBQHV41DZ7g7OHvy1p0GY2sVwxp4m54QDoC6gCu/2niTVXcOmOHp9OoRuf7pZBO KhdOMUHxRbk= =Othv -----END PGP SIGNATURE----- From g.brandl at gmx.net Tue Dec 19 20:02:05 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 19 Dec 2006 20:02:05 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Barry Warsaw schrieb: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Dec 19, 2006, at 1:03 PM, Guido van Rossum wrote: > >> On 12/19/06, Fredrik Lundh wrote: >>> Georg Brandl wrote: >>> >>>> so what about id() and intern(). Care to pronounce? >>> >>> moving/removing id()? please don't do that; id(obj) and type(obj) >>> are >>> essential tools for learning object semantics, and should be >>> trivial to >>> access and use. >> >> Right. I tend to agree with Fredrik about id(). I'm not so sure about >> intern() -- IMO its use is rare enough that the move would affect very >> little code; I'm seeking validation of this. If nobody speaks up about >> it, moving intern() would be okay with me. > > You don't want to get rid of 'interning' (e.g. PyString_Intern*()) > but just the intern() built-in? We actually use both, but we use the > C API call much more frequently than the built-in call. I don't > think it would be too painful for us if intern() was removed, but > OTOH if it was stuck in some Python internals module (sys?) and not > kept as a built-in, that would be fine too. Yes, exactly that is proposed: sys.intern() instead of intern(). Georg From g.brandl at gmx.net Tue Dec 19 20:02:37 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 19 Dec 2006 20:02:37 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> <45882E45.4080709@acm.org> Message-ID: Guido van Rossum schrieb: > On 12/19/06, Talin wrote: >> Guido van Rossum wrote: >> > Um, that was tongue-in-cheek. My serious proposal was python-4000, but >> > python-ideas sounds better to me because it won't eventually outdate >> > itself. >> >> python-future. (python-ideas is a little too ambiguous, as it could also >> overlap with python-users.) > > The first one to convince postmaster at python.org to create the list > wins. I'm outta there. (And please, convince gmane to mirror it too...) Georg From brett at python.org Tue Dec 19 19:58:39 2006 From: brett at python.org (Brett Cannon) Date: Tue, 19 Dec 2006 10:58:39 -0800 Subject: [Python-3000] Moving forward: library reorganization In-Reply-To: <9e804ac0612182359o40b9feb9vfa1c827896d40827@mail.gmail.com> References: <9e804ac0612181711j12ae1b7dia5c783e605e22998@mail.gmail.com> <9e804ac0612182359o40b9feb9vfa1c827896d40827@mail.gmail.com> Message-ID: On 12/18/06, Thomas Wouters wrote: > > > > On 12/19/06, Brett Cannon wrote: > > > > > As for using a lib-old idea, is that for Python 2.x to help transition, > > or did you want to do that for Py3K as well? I see the logic in the former > > to help transition but in the latter. > > > > Development-transition only. I want to start doing it now (or next week, > or at some point in the future) without having to think about the actual > transition from py2x->py3k. > Gotcha. Well, I am sticking my neck out there and saying I am willing to write the PEP listing what names need to be changed and what modules need to go. Do you (or anyone else) want to co-author it with me? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/358fe1fa/attachment.htm From nnorwitz at gmail.com Tue Dec 19 22:03:57 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 19 Dec 2006 13:03:57 -0800 Subject: [Python-3000] versionadded/versionchanged in doc Message-ID: What do we want to do with the current versionadded/versionchanged markups in the doc for 3k? Should we remove all references to 1.x changes? all 2.x changes? Keep them? all of them? Given we are trying to clean things up, I think I'd prefer to remove all the old references to 1.x/2.x changes. I'm not sure how to handle any of the 3.x changes. I expect those will be in a separate porting doc or some such. So my first thought is that it would be nice to have all the doc be clean. My only concern is that it might be confusing for people. Though my first guess is that it won't be any more confusing than anything else we do, so I'd prefer to see it cleaned up. Thoughts? n From greg.ewing at canterbury.ac.nz Tue Dec 19 21:31:34 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 20 Dec 2006 09:31:34 +1300 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <45874670.8030500@canterbury.ac.nz> Message-ID: <45884C26.5030707@canterbury.ac.nz> Guido van Rossum wrote: > I'm allowing for small incompatibilities between 3.0, 3.1 and 3.2 only > because I expect that 3.0, when it comes out, will be the first > version that most users will try, and we will surely have to release a > few quick upgrades to address the most egregious mistakes. But by 3.2 > we should be stable. Okay, that clears things up, thanks. -- Greg From fdrake at acm.org Tue Dec 19 22:17:11 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 19 Dec 2006 16:17:11 -0500 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: References: Message-ID: <200612191617.11363.fdrake@acm.org> On Tuesday 19 December 2006 16:03, Neal Norwitz wrote: > have all the doc be clean. My only concern is that it might be > confusing for people. Though my first guess is that it won't be any > more confusing than anything else we do, so I'd prefer to see it > cleaned up. Remove them from the documentation source or remove them from the formatted output? I'd be happy to see 1.x and 2.x notes removed from the formatting, but people who work with several versions find this helpful (I do!). -Fred -- Fred L. Drake, Jr. From guido at python.org Tue Dec 19 22:28:52 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 13:28:52 -0800 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: <200612191617.11363.fdrake@acm.org> References: <200612191617.11363.fdrake@acm.org> Message-ID: On 12/19/06, Fred L. Drake, Jr. wrote: > On Tuesday 19 December 2006 16:03, Neal Norwitz wrote: > > have all the doc be clean. My only concern is that it might be > > confusing for people. Though my first guess is that it won't be any > > more confusing than anything else we do, so I'd prefer to see it > > cleaned up. > > Remove them from the documentation source or remove them from the formatted > output? I'd be happy to see 1.x and 2.x notes removed from the formatting, > but people who work with several versions find this helpful (I do!). I think that it would be useful to have "new in 3.0" or "changed in 3.0" annotations visible in the 3.0 docs. I don't think it would be useful to have the exact 1/2.x version where something was added visible; someone interested in that information should consult the latest 2.x docs. I'm fine with keeping those annotations in the source though. For people who really want to write code that runs unchanged in 2.x and 3.0 the constraints will be so tight (e.g. can't use print, can't use iterkeys) that their best bet will be to use the most recent 2.x version available (2.6 or 2.7) as their only 2.x target -- those will likely have some special features that will make it easier to write forwards compatible code. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 22:42:40 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 13:42:40 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Brett Cannon wrote: > On 12/18/06, Guido van Rossum wrote: > > I am getting worried about the Py3k release schedule. According to PEP > > 3000, "I hope to have a first alpha release out sometime in 2007" > > which would seem to give us another year at least; but in my mind I've > > always interpreted this (and explained it to others) as "in the first > > half of 2007" which would give us more like half a year at most. > > I always wondered about that due date and when you really wanted to cut an > alpha. I've now updated PEP 3000 to put 3.0a1 in the first half of '07, PEP 3* submission (not approval!) by the end of April, and the 3.0final release at most (instead of at least) a year after 3.0a1. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Tue Dec 19 22:47:04 2006 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 19 Dec 2006 16:47:04 -0500 Subject: [Python-3000] versionadded/versionchanged in doc References: Message-ID: "Neal Norwitz" wrote in message news:ee2a432c0612191303n100aba92v8a07535daa3da0d at mail.gmail.com... > What do we want to do with the current versionadded/versionchanged > markups in the doc for 3k? Should we remove all references to 1.x > changes? all 2.x changes? Keep them? all of them? > > Given we are trying to clean things up, I think I'd prefer to remove > all the old references to 1.x/2.x changes. I'm not sure how to handle > any of the 3.x changes. I expect those will be in a separate porting > doc or some such. So my first thought is that it would be nice to > have all the doc be clean. I am pretty sure that a large majority of people learning Python for the first time with Python 3 would prefer this. If Python3 is as successful as we might hope, new Pythoneers will become the majority. > My only concern is that it might be confusing for people. Perhaps for some, but I would expect that docs cluttered with 'New/changed in 3.0' to be just as confusing for others. Perhaps we need two versions of the docs: a Python x.y set and a Python forever set. Terry Jan Reedy From guido at python.org Tue Dec 19 22:58:29 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 13:58:29 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Brett Cannon wrote: > On 12/18/06, Guido van Rossum wrote: > > When you say "just store strings" do you mean that the implementation > > would be limited to strings or just that you would only use it to > > store per-argument docstrings? The latter is fine with me; the former > > seems an unnecessary restriction. > > So code objects would grow a co_annotations that would be a tuple of strings > that stored what the annotations were as strings, with the last slot > reserved for returns. Any parameter that didn't have an annotation would > have None assigned to it. This means that this code:: > > def fxn(a:int, b, c:str='hi') -> float: pass > > would have a co_annotations of ``('int', None, 'str', float)``. Keeping > everything as strings keeps it simple and minimizes any performance impact > from them being declared. People can then easily write decorators that read > the tuple (or use signature objects) to type-check as desired. Sorry, that's not at all how I am envisioning it. Apart from being stored in a different place I want these to work exactly the same way as defaults -- they can be expressions with any value and whatever value they evaluate to at def time will be stored there. (Another reason is that it's far from trivial for the parser to get the actual text as a string, unless you very severely limit the syntax, which I also don't want to see -- I want to be able to use List[int] for example. > Tony's implementation seems to actually fetch from globals what the > annotation specifies and I just don't think that is necessary and unneeded > overhead. Why? I think it's the right thing to do. I'll have to look at his patch. > Then again, functions are only created once and extra overhead > could help discourage too much usage. But using strings and None like this > cleanly solves the ambiguity issue of what value represents when no > annotation was set (object, type?). I think None is fine. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Dec 19 23:01:46 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 14:01:46 -0800 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: References: Message-ID: On 12/19/06, Terry Reedy wrote: > "Neal Norwitz" wrote in message > news:ee2a432c0612191303n100aba92v8a07535daa3da0d at mail.gmail.com... > > What do we want to do with the current versionadded/versionchanged > > markups in the doc for 3k? Should we remove all references to 1.x > > changes? all 2.x changes? Keep them? all of them? > > > > Given we are trying to clean things up, I think I'd prefer to remove > > all the old references to 1.x/2.x changes. I'm not sure how to handle > > any of the 3.x changes. I expect those will be in a separate porting > > doc or some such. So my first thought is that it would be nice to > > have all the doc be clean. > > I am pretty sure that a large majority of people learning Python for the > first time with Python 3 would prefer this. If Python3 is as successful as > we might hope, new Pythoneers will become the majority. Realistically, people won't be starting to learn 3.x until it's widely accepted, since most people learn so that they can use other code that's around. A bit of a chicken/egg problem, admittedly, but we'll address that by providing a good 2->3 refactoring tool and doing parallel development of 2.6/7 and 3.0/1/2. > > My only concern is that it might be confusing for people. > > Perhaps for some, but I would expect that docs cluttered with 'New/changed > in 3.0' to be just as confusing for others. Perhaps we need two versions > of the docs: a Python x.y set and a Python forever set. Once 3.0 is mainstream we can drop all the "added in 3.0/1/2" annotations but keep the ones for 3.3 and beyond. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake at acm.org Tue Dec 19 23:02:06 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 19 Dec 2006 17:02:06 -0500 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: References: Message-ID: <200612191702.06999.fdrake@acm.org> On Tuesday 19 December 2006 16:47, Terry Reedy wrote: > I am pretty sure that a large majority of people learning Python for the > first time with Python 3 would prefer this. If Python3 is as successful > as we might hope, new Pythoneers will become the majority. Agreed. Those who are interested in the historical notes are a smaller audience, and somewhat specialized. > Perhaps for some, but I would expect that docs cluttered with 'New/changed > in 3.0' to be just as confusing for others. Perhaps we need two versions > of the docs: a Python x.y set and a Python forever set. Hmm. I think a little bit of JavaScript would go a long way. ;-) After looking at the code for the HTML generation, I don't think that would be difficult to do at all. I'm not sure how to deal with the printed version yet; perhaps the best thing for non-HTML versions is to simply drop the older version notes. Most people use the HTML version anyway. -Fred -- Fred L. Drake, Jr. From brett at python.org Tue Dec 19 23:10:38 2006 From: brett at python.org (Brett Cannon) Date: Tue, 19 Dec 2006 14:10:38 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > > On 12/18/06, Brett Cannon wrote: > > On 12/18/06, Guido van Rossum wrote: > > > When you say "just store strings" do you mean that the implementation > > > would be limited to strings or just that you would only use it to > > > store per-argument docstrings? The latter is fine with me; the former > > > seems an unnecessary restriction. > > > > So code objects would grow a co_annotations that would be a tuple of > strings > > that stored what the annotations were as strings, with the last slot > > reserved for returns. Any parameter that didn't have an annotation > would > > have None assigned to it. This means that this code:: > > > > def fxn(a:int, b, c:str='hi') -> float: pass > > > > would have a co_annotations of ``('int', None, 'str', float)``. Keeping > > everything as strings keeps it simple and minimizes any performance > impact > > from them being declared. People can then easily write decorators that > read > > the tuple (or use signature objects) to type-check as desired. > > Sorry, that's not at all how I am envisioning it. Apart from being > stored in a different place I want these to work exactly the same way > as defaults -- they can be expressions with any value and whatever > value they evaluate to at def time will be stored there. > > (Another reason is that it's far from trivial for the parser to get > the actual text as a string, unless you very severely limit the > syntax, which I also don't want to see -- I want to be able to use > List[int] for example. Ah, OK. Then I will stay out of Tony's way since he seems to be doing it the way you want. > Tony's implementation seems to actually fetch from globals what the > > annotation specifies and I just don't think that is necessary and > unneeded > > overhead. > > Why? I think it's the right thing to do. I'll have to look at his patch. Because I was viewing the whole thing from a different perspective than you. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/9dd09248/attachment.html From rhamph at gmail.com Tue Dec 19 23:52:28 2006 From: rhamph at gmail.com (Adam Olsen) Date: Tue, 19 Dec 2006 15:52:28 -0700 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Guido van Rossum wrote: > character, but AFAICT it fizzled; nobody has proposed to help with > getting the int unification branch, which is mostly done but still has > 22 failing tests last time I looked. I've received a few contributions > I'd also like to see people volunteer to help with the implementation > of some of these projects, and with the implementation of projects > that already have a PEP or don't need one. For example: > > - finishing the int unification branch Who should I conspire with to get this done? I've already tracked down one of the bugs: convert_to_double in floatobject.c prefers PyInt_AS_LONG, assuming it can't fail, which is no longer true. >>> 5.0 * 9999999999999 -5.0 >>> 1.0 + 999999999999 0.0 What is the policy on working on the p3yk branch? I thought I read that we should skip bug reports, but I've been unable to find the email saying so. -- Adam Olsen, aka Rhamphoryncus From brett at python.org Wed Dec 20 00:01:23 2006 From: brett at python.org (Brett Cannon) Date: Tue, 19 Dec 2006 15:01:23 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Adam Olsen wrote: > > On 12/18/06, Guido van Rossum wrote: > > character, but AFAICT it fizzled; nobody has proposed to help with > > getting the int unification branch, which is mostly done but still has > > 22 failing tests last time I looked. I've received a few contributions > > > I'd also like to see people volunteer to help with the implementation > > of some of these projects, and with the implementation of projects > > that already have a PEP or don't need one. For example: > > > > - finishing the int unification branch > > Who should I conspire with to get this done? Martin v. Loewis did all of the initial work on the int unification branch. He would be the best person to directly talk with. I am pretty sure he reads this mailing list regularly. I've already tracked > down one of the bugs: convert_to_double in floatobject.c prefers > PyInt_AS_LONG, assuming it can't fail, which is no longer true. > > >>> 5.0 * 9999999999999 > -5.0 > >>> 1.0 + 999999999999 > 0.0 > > What is the policy on working on the p3yk branch? It's considered just like any other branch in Python. I thought I read > that we should skip bug reports, but I've been unable to find the > email saying so. Not that I know of. There is a Python 3000 group on SF. We just don't want people reporting bugs saying that there code doesn't work on that branch. But reporting bugs like this where new stuff is not working as it should can be reported. It just doesn't hurt (for now) to say here you found the bug as well as not that many people are hacking on the p3yk branch. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/b70e1ec8/attachment.htm From guido at python.org Wed Dec 20 01:17:10 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 16:17:10 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Brett Cannon wrote: > On 12/19/06, Adam Olsen wrote: > > On 12/18/06, Guido van Rossum wrote: > > > character, but AFAICT it fizzled; nobody has proposed to help with > > > getting the int unification branch, which is mostly done but still has > > > 22 failing tests last time I looked. I've received a few contributions > > > > > I'd also like to see people volunteer to help with the implementation > > > of some of these projects, and with the implementation of projects > > > that already have a PEP or don't need one. For example: > > > > > > - finishing the int unification branch > > > > Who should I conspire with to get this done? > > Martin v. Loewis did all of the initial work on the int unification branch. > He would be the best person to directly talk with. I am pretty sure he > reads this mailing list regularly. Alternatively, if you have a patch, use the SF patch manager and assign it to me, and I'll take it on right away. > > I've already tracked > > down one of the bugs: convert_to_double in floatobject.c prefers > > PyInt_AS_LONG, assuming it can't fail, which is no longer true. > > > > >>> 5.0 * 9999999999999 > > -5.0 > > >>> 1.0 + 999999999999 > > 0.0 > > > > What is the policy on working on the p3yk branch? > > It's considered just like any other branch in Python. > > > I thought I read > > that we should skip bug reports, but I've been unable to find the > > email saying so. > > Not that I know of. There is a Python 3000 group on SF. We just don't want > people reporting bugs saying that there code doesn't work on that branch. > But reporting bugs like this where new stuff is not working as it should can > be reported. It just doesn't hurt (for now) to say here you found the bug > as well as not that many people are hacking on the p3yk branch. > > -Brett -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 01:18:21 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 16:18:21 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Georg Brandl wrote: > Okay, I updated the patch at SF. While you're at it, in PEP 3100 there's > "compile(): put in sys (or perhaps in a module of its own)". I guess that > isn't really necessary either... Hm, I think it would be fine to move, it's pretty specialized. What do others here think? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 01:19:21 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 16:19:21 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Georg Brandl wrote: > > - turning list comprehensions into syntactic sugar for generator expressions > > I'd like to point out that there is already my patch for that, which implements > set comprehensions and list comprehensions exactly as syntactic sugar for > GEs. This, however, affects performance greatly as LCs are executed in their own > function scope, which isn't necessary. A better implementation would therefore > leave the LC implementation as is, only preventing the name leaking into the > enclosing scope. Do you think you have it in you to tackle this? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 01:20:53 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 16:20:53 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <4587643C.6090700@acm.org> References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> <4587643C.6090700@acm.org> Message-ID: On 12/18/06, Talin wrote: > There's a Python prototype of the string formatting PEP in my Mercurial > repository here: > > http://www.viridia.org/hg/python/string_format > > The main thing that needs to be done is to graft what's there in the > prototype into the actual built-in string class. I don't know if that > requires porting the prototype to C or not. I would say so, yes. Any volunteers? > It also needs unit tests; I can help with that part if needed. Yes, please. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 01:23:25 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 16:23:25 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/18/06, Brett Cannon wrote: [me] > > Well, what do you think of my pronouncement in response to Thomas's > > mail (just rename a bunch of things that don't conform to our own > > naming standard)? That should limit the discussion to what's the best > > name for StringIO etc. > > Sounds good. The only sticky point is whether we want to keep this > dichotomy of C/Python implementations ala pickle and StringIO going. I > personally would rather choose one (Python or C) and then work on getting > the other to have the proper semantics. If we choose the C version we can > stick the Python versions into lib-old or something so that if alternative > Python implementations want to keep them alive and up-to-date they can for > their own usage but the burden is taken off of us. Just to clairfy, I think this should all be done the way heapq.py / _heapq.c work. Python users should only import the "python" module which should contain a complete implementation in Python but which replaces selective parts (or everything) with faster versions from the C module if available. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Dec 20 02:10:43 2006 From: brett at python.org (Brett Cannon) Date: Tue, 19 Dec 2006 17:10:43 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > > On 12/19/06, Georg Brandl wrote: > > Okay, I updated the patch at SF. While you're at it, in PEP 3100 there's > > "compile(): put in sys (or perhaps in a module of its own)". I guess > that > > isn't really necessary either... > > Hm, I think it would be fine to move, it's pretty specialized. What do > others here think? Move it, I just wonder if there is a better place for it than sys. Putting intern() in sys makes sense since it is a specific thing to the interpreter. id could go there or maybe inspect. But compile (along wth exec, but I don't think you wanted to move that) are just plain different. Dynamic code execution is its own thing that I do not directly associate with the specifics of the interpreter. Maybe a module named 'dynamic' or 'execution'? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/f22717e4/attachment.html From guido at python.org Wed Dec 20 02:19:25 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 17:19:25 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Brett Cannon wrote: > > > On 12/19/06, Guido van Rossum wrote: > > On 12/19/06, Georg Brandl wrote: > > > Okay, I updated the patch at SF. While you're at it, in PEP 3100 there's > > > "compile(): put in sys (or perhaps in a module of its own)". I guess > that > > > isn't really necessary either... > > > > Hm, I think it would be fine to move, it's pretty specialized. What do > > others here think? > > Move it, I just wonder if there is a better place for it than sys. Putting > intern() in sys makes sense since it is a specific thing to the interpreter. > id could go there or maybe inspect. > > But compile (along wth exec, but I don't think you wanted to move that) are > just plain different. Dynamic code execution is its own thing that I do not > directly associate with the specifics of the interpreter. Maybe a module > named 'dynamic' or 'execution'? Well, sys is pretty much a grab-bag. And you can't tell me that compile() isn't a hook into system internals. :-) (The compiler in this case.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Wed Dec 20 03:55:52 2006 From: brett at python.org (Brett Cannon) Date: Tue, 19 Dec 2006 18:55:52 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > > On 12/19/06, Brett Cannon wrote: > > > > > > On 12/19/06, Guido van Rossum wrote: > > > On 12/19/06, Georg Brandl wrote: > > > > Okay, I updated the patch at SF. While you're at it, in PEP 3100 > there's > > > > "compile(): put in sys (or perhaps in a module of its own)". I guess > > that > > > > isn't really necessary either... > > > > > > Hm, I think it would be fine to move, it's pretty specialized. What do > > > others here think? > > > > Move it, I just wonder if there is a better place for it than > sys. Putting > > intern() in sys makes sense since it is a specific thing to the > interpreter. > > id could go there or maybe inspect. > > > > But compile (along wth exec, but I don't think you wanted to move that) > are > > just plain different. Dynamic code execution is its own thing that I do > not > > directly associate with the specifics of the interpreter. Maybe a > module > > named 'dynamic' or 'execution'? > > Well, sys is pretty much a grab-bag. That is very true. =) I talked about breaking it up. Still want to see a PEP on that someday? And you can't tell me that > compile() isn't a hook into system internals. :-) (The compiler in > this case.) No, but I view sys as place to put stuff that influences the running of the interpreter and I don't put compile() in that category. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/2019b70c/attachment.html From brett at python.org Wed Dec 20 03:57:39 2006 From: brett at python.org (Brett Cannon) Date: Tue, 19 Dec 2006 18:57:39 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <79990c6b0612190225o58ab1f38j1f2e179468f54fb2@mail.gmail.com> <45882E45.4080709@acm.org> Message-ID: On 12/19/06, Guido van Rossum wrote: > > On 12/19/06, Talin wrote: > > Guido van Rossum wrote: > > > Um, that was tongue-in-cheek. My serious proposal was python-4000, but > > > python-ideas sounds better to me because it won't eventually outdate > > > itself. > > > > python-future. (python-ideas is a little too ambiguous, as it could also > > overlap with python-users.) > > The first one to convince postmaster at python.org to create the list > wins. I'm outta there. I win! =) Anyone who is willing to be an admin for the site, email me personally. I will go through the motions of setting the permissions, description, etc. tomorrow along with a more proper announcement to python-dev, here and c.l.py.a . -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061219/e5b81ad6/attachment.htm From guido at python.org Wed Dec 20 03:59:41 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 18:59:41 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Brett Cannon wrote: > > Well, sys is pretty much a grab-bag. > > That is very true. =) I talked about breaking it up. Still want to see a > PEP on that someday? > > > And you can't tell me that > > compile() isn't a hook into system internals. :-) (The compiler in > > this case.) > > No, but I view sys as place to put stuff that influences the running of the > interpreter and I don't put compile() in that category. Nor stdin/out/err, not argv, not getframe, nor copyright. Yes, copyright!? I'm not concerned, really. There are bigger fish to fry. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 06:47:23 2006 From: guido at python.org (Guido van Rossum) Date: Tue, 19 Dec 2006 21:47:23 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() Message-ID: I've written a quick version of PEP 3106, which expresses my ideas about how the dict methods to access keys, values and items should be redone. The text is in svn: http://svn.python.org/view/peps/trunk/pep-3106.txt?rev=53096&view=markup At some point it will appear on python.org: http://python.org/dev/peps/pep-3106/ Comments please? (Or we can skip the comments and go straight to the implementation stage. Patch anyone?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Wed Dec 20 07:39:07 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 19 Dec 2006 22:39:07 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > I've written a quick version of PEP 3106, which expresses my ideas > about how the dict methods to access keys, values and items should be > redone. > > The text is in svn: > http://svn.python.org/view/peps/trunk/pep-3106.txt?rev=53096&view=markup > > At some point it will appear on python.org: http://python.org/dev/peps/pep-3106/ """ The idea is that code that currently (in 2.x) reads:: for x in d.iterkeys(): ... should be rewritten as:: for x in d.keys(): ... """ Is this really the best example? Wouldn't this be better written as for x in d: ... ? Maybe replace keys with items or values? I think I know the answer to this question, but I just want it explicit: """ def copy(self): return set(self) """" Should that be set or frozenset (in all the classes)? Also, should self.__d be passed to set()? In d_keys: def discard(self, key): if key in self: self.remove(key) You could use self.__d here. It's an optimization and doesn't really apply to the others so I'm not sure the change is worth it. It's interesting that you add a discard method. Would you also like a similar method added to lists? (I know these are conceptual sets, but in the past I think I've wanted discard functionality more than remove. I wonder if this is somewhat similar to str.find vs str.index.) In d_values: """" def __contains__(self, value): # Slow! Do we even want to implement this? """" KISS. I'd rather keep the interface as small as possible at first, especially with something like this. Let the interface grow if we find it lacking. """ I'm soliciting better names than d_keys, d_values and d_items; these classes will be public so that their implementations may be reused by the .keys(), .values() and .items() methods of other mappings. (Or should they?) """ I'm not sure if they should be public or not, but what about just: dict.Keys dict.Items dict.Values If we have a mapping ABC, that would be a more appropriate place. Have you decided if there will be ABCs like: mapping, sequence, number, etc.? These seem to make more sense if type annotations would be provided as part of the syntax. Or perhaps there could be a mapping module? n From talin at acm.org Wed Dec 20 08:16:53 2006 From: talin at acm.org (Talin) Date: Tue, 19 Dec 2006 23:16:53 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: <4588E365.6090509@acm.org> Guido van Rossum wrote: > I've written a quick version of PEP 3106, which expresses my ideas > about how the dict methods to access keys, values and items should be > redone. > > The text is in svn: > http://svn.python.org/view/peps/trunk/pep-3106.txt?rev=53096&view=markup > > At some point it will appear on python.org: http://python.org/dev/peps/pep-3106/ > > Comments please? (Or we can skip the comments and go straight to the > implementation stage. Patch anyone?) Comments: 1) values() should be documented as list-like rather than set-like. (Or better yet bag-like, but Python doesn't have bags. In other words, it's really an unordered collection of items with no special indexing semantics or uniqueness constraint.) My reasoning is that generally people don't expect to be able to do set membership tests of the values of a mapping; Mostly, values() is used for iteration. And given that all of the set-like operations are slow, let's just cut to the chase and say that values() isn't a set at all. 2) Names for d_keys, et al: -- map_keys_view -- map_values_view -- map_keys_values_view or map_items_view Essentially, its a class that can take any map, and provide a view of its keys / values / etc. Alternatively, we could stick them in a module called 'mapview' and call them: -- mapview.keys -- mapview.values -- mapview.items -- Talin From nnorwitz at gmail.com Wed Dec 20 10:12:18 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 20 Dec 2006 01:12:18 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > On 12/19/06, Georg Brandl wrote: > > Okay, I updated the patch at SF. While you're at it, in PEP 3100 there's > > "compile(): put in sys (or perhaps in a module of its own)". I guess that > > isn't really necessary either... > > Hm, I think it would be fine to move, it's pretty specialized. What do > others here think? Move it. n From nnorwitz at gmail.com Wed Dec 20 10:19:03 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 20 Dec 2006 01:19:03 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > On 12/18/06, Brett Cannon wrote: > [me] > > > Well, what do you think of my pronouncement in response to Thomas's > > > mail (just rename a bunch of things that don't conform to our own > > > naming standard)? That should limit the discussion to what's the best > > > name for StringIO etc. > > > > Sounds good. The only sticky point is whether we want to keep this > > dichotomy of C/Python implementations ala pickle and StringIO going. I > > personally would rather choose one (Python or C) and then work on getting > > the other to have the proper semantics. If we choose the C version we can > > stick the Python versions into lib-old or something so that if alternative > > Python implementations want to keep them alive and up-to-date they can for > > their own usage but the burden is taken off of us. > > Just to clairfy, I think this should all be done the way heapq.py / > _heapq.c work. Python users should only import the "python" module > which should contain a complete implementation in Python but which > replaces selective parts (or everything) with faster versions from the > C module if available. To further clarify, stuff like this should be done on HEAD for 2.6. This is not a backwards incompatible change. We can forward port to the 3k branch (see Thomas Wouters msg in a diff thread). These changes mean we need to ensure the C impl provides a consistent interface with the Python version (or falls back when appropriate). n From greg.ewing at canterbury.ac.nz Wed Dec 20 11:32:09 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 20 Dec 2006 23:32:09 +1300 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: <45891129.5010605@canterbury.ac.nz> Brett Cannon wrote: > No, but I view sys as place to put stuff that influences the running of > the interpreter and I don't put compile() in that category. So what about sys.argv, sys.std{in,out,err}, sys.exc_info, etc. etc.? They're not exactly about influencing the running of the interpreter, either. -- Greg From greg.ewing at canterbury.ac.nz Wed Dec 20 11:46:20 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 20 Dec 2006 23:46:20 +1300 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: <4588E365.6090509@acm.org> References: <4588E365.6090509@acm.org> Message-ID: <4589147C.3010101@canterbury.ac.nz> Talin wrote: > 2) Names for d_keys, et al: > > -- map_keys_view > -- map_values_view > -- map_keys_values_view or map_items_view I didn't think we were going to attempt to implement any of these generically. I thought each type would provide its own implementations tuned to the details of its internals. -- Greg From g.brandl at gmx.net Wed Dec 20 11:51:07 2006 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 20 Dec 2006 11:51:07 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Guido van Rossum schrieb: > On 12/19/06, Georg Brandl wrote: >> > - turning list comprehensions into syntactic sugar for generator expressions >> >> I'd like to point out that there is already my patch for that, which implements >> set comprehensions and list comprehensions exactly as syntactic sugar for >> GEs. This, however, affects performance greatly as LCs are executed in their own >> function scope, which isn't necessary. A better implementation would therefore >> leave the LC implementation as is, only preventing the name leaking into the >> enclosing scope. > > Do you think you have it in you to tackle this? I'll implement it if only someone can point me in the right direction how to do it. Georg From ncoghlan at gmail.com Wed Dec 20 12:22:29 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Dec 2006 21:22:29 +1000 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: <45891CF5.20109@gmail.com> Guido van Rossum wrote: > I've written a quick version of PEP 3106, which expresses my ideas > about how the dict methods to access keys, values and items should be > redone. > > The text is in svn: > http://svn.python.org/view/peps/trunk/pep-3106.txt?rev=53096&view=markup > > At some point it will appear on python.org: http://python.org/dev/peps/pep-3106/ > > Comments please? (Or we can skip the comments and go straight to the > implementation stage. Patch anyone?) Looks pretty good. Some specific comments: 1. There's no reason to implement __contains__ on d.values() - the pseudocode currently in the PEP is what the interpreter already falls back to doing if __iter__ is provided but __contains__ isn't. 2. At least initially, I'd prefer to make the immutability of d.values() total: no pop(), no clear(). I think this rule (d.values() doesn't let you modify the underlying dict at all) is easier to remember than remembering that pop() & clear() are permitted, but everything else that d.keys() and d.items() permit is disallowed. 3. The definition of d.items().copy() as set(self) may have a problem: the values in the original dict may not be hashable, so set(d.items()) may fail with a TypeError. If we want d.items().copy() and d.values().copy() to work without potentially raising TypeError we would need two new data types (probably in the collections module): a. A keyed set: something that behaves like a set, but accepts a callable that describes how to retrieve the key value from the items in the set. Then 'd.items().copy()' would be equivalent to 'collections.keyedset(d.items(), key=operator.itemgetter(0))'. b. A non-hashing multiset: dict values aren't guaranteed to be hashable, so a defaultdict-based bag implementation wouldn't help. 4. Given the previous point, perhaps we should just drop the idea of providing a copy() method at all on any of d.keys(), d.items() or d.values()? If the dict's values are hashable (and we include a multiset implementation in collections), then 'set(d.keys())', 'set(d.items())' and 'multiset(d.values())' will be just as clear as a copy() method, and the TypeError you get if the values in the original dictionary aren't hashable will likely be less surprising. This approach would also be closer to the copying approach that tolerates non-hashable values in the original dict: 'set(d.keys())', 'list(d.items())' and 'list(d.values())'. I'm personally somewhat ambivalent regarding points 3 & 4 - I like the idea of having a corresponding concrete container class with similar semantics for each of the three alternate dictionary views, and once we have those then there's little reason not to provide the copy methods. OTOH, the growing prevalence of copy methods on standard library container types would then start to make me wonder whether or not copy.copy() (or a less forgiving equivalent designed specifically to copy containers) should simply be made a builtin, such that the OOW to do a shallow copy of any container would be 'copy(x)'. Option 4 ducks that particular issue by making it easy to avoid adding the two new container types. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Dec 20 12:43:53 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Dec 2006 21:43:53 +1000 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: <458921F9.7040504@gmail.com> Guido van Rossum wrote: > I've written a quick version of PEP 3106, which expresses my ideas > about how the dict methods to access keys, values and items should be > redone. > > The text is in svn: > http://svn.python.org/view/peps/trunk/pep-3106.txt?rev=53096&view=markup > > At some point it will appear on python.org: http://python.org/dev/peps/pep-3106/ > > Comments please? (Or we can skip the comments and go straight to the > implementation stage. Patch anyone?) > And one thing I left out of my other reply: The UserDict and UserList modules are going to need to be renamed in order to provide PEP compliant homes for ListMixin and DictMixin (as per PEP 3100, the other classes in those modules go away completely). Wherever DictMixin ends up, it is going to need classes defined that it can return from keys(), items() and values(). These may either be alternate implementations written in Python, or else the actual C versions used by the builtin dict type. For names, something simple like DictKeysView, DictItemsView and DictValuesView should be fine. Cheers, Nick. [1] 'containertools'?, 'collectiontools'?, 'collections.utils'? I suspect a BDFL name pronouncement will be needed if we pursue this idea :) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Dec 20 12:55:23 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Dec 2006 21:55:23 +1000 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: <458921F9.7040504@gmail.com> References: <458921F9.7040504@gmail.com> Message-ID: <458924AB.7010704@gmail.com> Nick Coghlan wrote: > [1] 'containertools'?, 'collectiontools'?, 'collections.utils'? I suspect a > BDFL name pronouncement will be needed if we pursue this idea :) Oops - deleted the idea, but forgot to delete the related footnote. Anyway, the idea was to move ListMixin and DictMixin into a single module dedicated to utilities that make it easier to write your own containers instead of keeping them in separate 'userlist' and 'userdict' modules. I deleted the idea because it wasn't particularly relevant to the discussion at hand. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Dec 20 13:40:51 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Dec 2006 22:40:51 +1000 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: References: Message-ID: <45892F53.4090601@gmail.com> Georg Brandl wrote: > Guido van Rossum schrieb: >> On 12/19/06, Georg Brandl wrote: >>>> - turning list comprehensions into syntactic sugar for generator expressions >>> I'd like to point out that there is already my patch for that, which implements >>> set comprehensions and list comprehensions exactly as syntactic sugar for >>> GEs. This, however, affects performance greatly as LCs are executed in their own >>> function scope, which isn't necessary. A better implementation would therefore >>> leave the LC implementation as is, only preventing the name leaking into the >>> enclosing scope. >> Do you think you have it in you to tackle this? > > I'll implement it if only someone can point me in the right direction > how to do it. One idea I've had on that front is to persuade the compiler to replace the names used for the iteration variables in the source code either with the compiler's own hidden variable names (like the ones it already uses for various internal results it can't leave on the stack), or else to add the idea of a 'scope prefix' that assigns names in the existing function scope with a numeric prefix in front of each of the symbols. I haven't actually tried to implement that, though, and I suspect things will get a little tricky when it comes to dealing with nested scopes like '[(lambda i=i: i) for i in range(10)]'. Another alternative might be to flag each list comprehension iteration variable during the symtable pass as being either deleted or restored when the comprehension is complete. Then before the LC emit code to save any variables to be restored to a hidden compiler variable and code after the LC to restore them to their original values. For deleted variables, emit the relevant deletion opcode after the LC. This second approach, however, still has issues dealing with nested scopes (in particular closure variables will refer to the variable at function scope instead of the final value of the iteration variable). In addition, it needs to be able to handle conditionally defined variables: ones which may or may not be defined when the LC executes depending on which path was followed through earlier parts of the function. So I think the first approach I mentioned (particularly the 'scope prefix' concept) is the one most likely to prove workable. But YMMV, since I haven't actually *tried* any of this. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Wed Dec 20 14:08:55 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 20 Dec 2006 23:08:55 +1000 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: References: Message-ID: <458935E7.2060509@gmail.com> Guido van Rossum wrote: > In the sandbox I've been working on a refactoring tool, which could > form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to > invite folks here to give it a try and give me a hand. It certainly > needs more work, but I think that the basic infrastructure is sound. > Check out sandbox/2to3/: > http://svn.python.org/view/sandbox/trunk/2to3/. > > This message is to invite feedback, and to encourage contributions. It > would be great if people tried their hands at writing new > transformations! Looking at a fix Georg just checked in for intern() - is there a way for the fixer to indicate to the refactoring tool that a particular module needs to be imported if the module being refactored triggers a match on the fix? In the intern() case, an entire module could be fixed just by putting "from sys import intern" at the start of the module, instead of fixing it at each invocation point. This would also have the virtue of not breaking code that overrides intern() - because the new import would be at the start of the module, it would still be overridden, just like the current builtin. For example, if a fix could provide a 'module_prefix' method which returned something to be inserted at the beginning of the module if the fix was triggered at least once, then the intern fix might look like: class FixIntern(object): # No change to __init__ or match from Georg's checked in version def module_prefix(self): # Add 'from sys import intern' to the top of the module return pytree.Node(syms.import_from, (pytree.Leaf(token.NAME, "sys"), pytree.Leaf(token.NAME, "intern") )) # Is that the right pytree incantation? def transform(self, node): # Actual occurences are left alone return node Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Wed Dec 20 16:45:06 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 20 Dec 2006 07:45:06 -0800 Subject: [Python-3000] Refactoring tool available (work in progress) In-Reply-To: <458935E7.2060509@gmail.com> References: <458935E7.2060509@gmail.com> Message-ID: On 12/20/06, Nick Coghlan wrote: > Looking at a fix Georg just checked in for intern() - is there a way for the > fixer to indicate to the refactoring tool that a particular module needs to be > imported if the module being refactored triggers a match on the fix? Not yet; I realize we need this, and I'm hoping for patches. Inserting statements isn't as easy as you'd like using the current state of the refactoring tool. The right place to pick is tricky: it has to be after the docstring, and after any "from __future__ import ..." lines, but otherwise you'd like to put it as early as possible. OTOH you might want to look for a row of familiar imports (os, re, ...) and insert or append it there. And of course it may already be imported, in which case you don't need to do anything. > In the intern() case, an entire module could be fixed just by putting "from > sys import intern" at the start of the module, instead of fixing it at each > invocation point. Although in a sense that defeats the purpose. I'm not fond of littering my code with "from M import F" statements because it doesn't scale, namespace-wise. So I ould *rather* see the current solution with the proper import statement added. > This would also have the virtue of not breaking code that overrides intern() - > because the new import would be at the start of the module, it would still be > overridden, just like the current builtin. Yes, this is a general problem -- I would like to add some kind of namespace and scope analysis to the refactoring tool, so at least we can tell where any particular name is defined (unless, of course, the source is "from ... import *", in which case I respectfully give up :-). > For example, if a fix could provide a 'module_prefix' method which returned > something to be inserted at the beginning of the module if the fix was > triggered at least once, then the intern fix might look like: > > class FixIntern(object): > # No change to __init__ or match from Georg's checked in version > > def module_prefix(self): > # Add 'from sys import intern' to the top of the module > return pytree.Node(syms.import_from, > (pytree.Leaf(token.NAME, "sys"), > pytree.Leaf(token.NAME, "intern") > )) # Is that the right pytree incantation? # Almost; remember it's a (very) concrete syntax tree, so you also have to provide leaf nodes for the 'import' and 'from' keywords. > def transform(self, node): > # Actual occurences are left alone > return node The driver currently ignores it if you return the original node unchanged (or even an unchanged copy). But we can probably easily find a slightly different way -- perhaps the driver calls a begin() method at the start of a file, the transform() code can set a flag, and ad the end of a file the driver will call an end() method which inserts the new import if the flag was set. Or perhaps it should return an import node and the driver should have some generic machinery that inserts additional imports at the right place, using various heuristics. Care to run with this idea? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 17:37:32 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 20 Dec 2006 08:37:32 -0800 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: <45892F53.4090601@gmail.com> References: <45892F53.4090601@gmail.com> Message-ID: On 12/20/06, Nick Coghlan wrote: > Georg Brandl wrote: > > I'll implement it if only someone can point me in the right direction > > how to do it. > > One idea I've had on that front is to persuade the compiler to replace the > names used for the iteration variables in the source code either with the > compiler's own hidden variable names (like the ones it already uses for > various internal results it can't leave on the stack), or else to add the idea > of a 'scope prefix' that assigns names in the existing function scope with a > numeric prefix in front of each of the symbols. > > I haven't actually tried to implement that, though, and I suspect things will > get a little tricky when it comes to dealing with nested scopes like '[(lambda > i=i: i) for i in range(10)]'. That doesn't strike me as tricky; I expect it'll happen automatically if you do the right thing. Trickier seems to be the possibility of writing a = {} for a[0] in ...: ... We should probably just leave this and other similar anomalies alone. > Another alternative might be to flag each list comprehension iteration > variable during the symtable pass as being either deleted or restored when the > comprehension is complete. Then before the LC emit code to save any variables > to be restored to a hidden compiler variable and code after the LC to restore > them to their original values. For deleted variables, emit the relevant > deletion opcode after the LC. Delete/restore seems utterly wrong with me -- it wouldn't work if one of the variables affected is a cell referenced by a nested function that was previously defined and that happens to be called at any time during the iteration. I think that the systematic renaming should work though, and be relatively easy -- you just have to modify all of the expression compiler to be aware of the potential renaming. > This second approach, however, still has issues dealing with nested scopes (in > particular closure variables will refer to the variable at function scope > instead of the final value of the iteration variable). In addition, it needs > to be able to handle conditionally defined variables: ones which may or may > not be defined when the LC executes depending on which path was followed > through earlier parts of the function. > > So I think the first approach I mentioned (particularly the 'scope prefix' > concept) is the one most likely to prove workable. But YMMV, since I haven't > actually *tried* any of this. Neither have I. But experiments should be easy enough. (Thanks Nick for the helpful suggestions!) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Dec 20 18:32:21 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 20 Dec 2006 09:32:21 -0800 Subject: [Python-3000] Need volunteer to update patch 1607548 (optional argument annotations) Message-ID: Tony Lownds created the original patch but (probably due to Thomas Wouters' recent merge) the patch doesn't apply cleanly any more. Tony hasn't responded on SF yet. Neal Norwitz added a lot of useful commentary to the SF item too. Perhaps someone can run with this? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tony at PageDNA.com Wed Dec 20 18:45:25 2006 From: tony at PageDNA.com (Tony Lownds) Date: Wed, 20 Dec 2006 09:45:25 -0800 Subject: [Python-3000] Need volunteer to update patch 1607548 (optional argument annotations) In-Reply-To: References: Message-ID: <78F74E4F-538F-4221-8A27-7DA0F8AA1365@PageDNA.com> I can get the patch clean again and work on the Neal's comments. I'm happy for the feedback. Thanks -Tony On Dec 20, 2006, at 9:32 AM, Guido van Rossum wrote: > Tony Lownds created the original patch but (probably due to Thomas > Wouters' recent merge) the patch doesn't apply cleanly any more. Tony > hasn't responded on SF yet. Neal Norwitz added a lot of useful > commentary to the SF item too. Perhaps someone can run with this? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/tony > %40printra.net > From brett at python.org Wed Dec 20 18:55:12 2006 From: brett at python.org (Brett Cannon) Date: Wed, 20 Dec 2006 09:55:12 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <45891129.5010605@canterbury.ac.nz> References: <45891129.5010605@canterbury.ac.nz> Message-ID: On 12/20/06, Greg Ewing wrote: > > Brett Cannon wrote: > > > No, but I view sys as place to put stuff that influences the running of > > the interpreter and I don't put compile() in that category. > > So what about sys.argv, sys.std{in,out,err}, > sys.exc_info, etc. etc.? They're not exactly > about influencing the running of the interpreter, > either. Oh, I know. I have proposed in the past to get 'sys' cleaned up. But Guido said to let this one go so I am. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061220/ccf77a7a/attachment.html From brett at python.org Wed Dec 20 19:23:16 2006 From: brett at python.org (Brett Cannon) Date: Wed, 20 Dec 2006 10:23:16 -0800 Subject: [Python-3000] New Python-Ideas mailing list Message-ID: At Guido's suggestion, a new mailing list has been created named Python-Ideas (http://mail.python.org/mailman/listinfo/python-ideas). This list is meant as a place for speculative, pie-in-the-sky language design ideas to be discussed and honed to the point of practically being a PEP before being presented to python-dev or python-3000. This allows both python-dev and python-3000 to focus more on implementation work or final approval/denial of ideas instead of being flooded with long threads where people discuss ideas that are too nebulous to be considered for inclusion into Python. Like python-dev and python-3000, Python-Ideas requires you subscribe before you can post, but there is no moderator approval required. If you are interested in helping me out by being an administrator or moderator for the list, please let me know. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061220/c8919f70/attachment.html From guido at python.org Wed Dec 20 21:19:13 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 20 Dec 2006 12:19:13 -0800 Subject: [Python-3000] Need volunteer to update patch 1607548 (optional argument annotations) In-Reply-To: <78F74E4F-538F-4221-8A27-7DA0F8AA1365@PageDNA.com> References: <78F74E4F-538F-4221-8A27-7DA0F8AA1365@PageDNA.com> Message-ID: OK, cool, never mind then. Sorry for being impatient! On 12/20/06, Tony Lownds wrote: > I can get the patch clean again and work on the Neal's comments. I'm > happy > for the feedback. > > Thanks > -Tony > > On Dec 20, 2006, at 9:32 AM, Guido van Rossum wrote: > > > Tony Lownds created the original patch but (probably due to Thomas > > Wouters' recent merge) the patch doesn't apply cleanly any more. Tony > > hasn't responded on SF yet. Neal Norwitz added a lot of useful > > commentary to the SF item too. Perhaps someone can run with this? > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: http://mail.python.org/mailman/options/python-3000/tony > > %40printra.net > > > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Wed Dec 20 21:38:48 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 20 Dec 2006 15:38:48 -0500 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: References: <45892F53.4090601@gmail.com> Message-ID: On 12/20/06, Guido van Rossum wrote: > On 12/20/06, Nick Coghlan wrote: > > I haven't actually tried to implement that, though, and I suspect things will > > get a little tricky when it comes to dealing with nested scopes like '[(lambda > > i=i: i) for i in range(10)]'. > That doesn't strike me as tricky; I expect it'll happen automatically > if you do the right thing. [(lambda i=i: i) for i in range(10)] => [(lambda i=0_i: i) for 0_i in range(10)] The argument to lambda can't be renamed, because it could be called as a keyword. Knowing which i's to rename strikes *me* as tricky, to the point where I would suggest at least considering a "works as implemented" pronouncement. This (along with further nesting, closure variables, conditionally defined variables, etc) is obscure enough that backwards incompatibility could be justified, if it made the explanation (or implementation) simpler. -jJ From tdelaney at avaya.com Wed Dec 20 21:40:01 2006 From: tdelaney at avaya.com (Delaney, Timothy (Tim)) Date: Thu, 21 Dec 2006 07:40:01 +1100 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() Message-ID: <2773CAC687FD5F4689F526998C7E4E5F07445B@au3010avexu1.global.avaya.com> Talin wrote: > 1) values() should be documented as list-like rather than set-like. > (Or > better yet bag-like, but Python doesn't have bags. In other words, > it's > really an unordered collection of items with no special indexing > semantics or uniqueness constraint.) > > My reasoning is that generally people don't expect to be able to do > set > membership tests of the values of a mapping; Mostly, values() is used > for iteration. And given that all of the set-like operations are slow, > let's just cut to the chase and say that values() isn't a set at all. I almost agree here, but the property that multiset(1, 2, 2) == multiset(2, 1, 2) seems useful for a dict values view. OTOH, how many people have complained that given d1 == d2, d1.values() may not equal d2.values() in 2.x? I've definitely never had any problems with it, but then again, I find it rare that I've needed values(). Tim Delaney From guido at python.org Wed Dec 20 22:01:31 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 20 Dec 2006 13:01:31 -0800 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: References: <45892F53.4090601@gmail.com> Message-ID: On 12/20/06, Jim Jewett wrote: > On 12/20/06, Guido van Rossum wrote: > > On 12/20/06, Nick Coghlan wrote: > > > I haven't actually tried to implement that, though, and I suspect things will > > > get a little tricky when it comes to dealing with nested scopes like '[(lambda > > > i=i: i) for i in range(10)]'. > > > That doesn't strike me as tricky; I expect it'll happen automatically > > if you do the right thing. > > [(lambda i=i: i) for i in range(10)] > => > [(lambda i=0_i: i) for 0_i in range(10)] > > The argument to lambda can't be renamed, because it could be called as > a keyword. Well duh, the renaming must observe the scope of the variable. Butthat's no big deal since we observe the scope when generating code for it anyway (references to locals need to use different bytecode than globals, and references to variables from outer nested scopes need to use cell references). Perhaps you were thinking of a textual substitution? That's not at all how one would do this. > Knowing which i's to rename strikes *me* as tricky, to the point where > I would suggest at least considering a "works as implemented" > pronouncement. This (along with further nesting, closure variables, > conditionally defined variables, etc) is obscure enough that backwards > incompatibility could be justified, if it made the explanation (or > implementation) simpler. I have no idea what you mean by "works as implemented". Do you propose *not* to fix the scope bleed of list comprehension loop control variables? That's not an option. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jimjjewett at gmail.com Wed Dec 20 22:31:03 2006 From: jimjjewett at gmail.com (Jim Jewett) Date: Wed, 20 Dec 2006 16:31:03 -0500 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: References: <45892F53.4090601@gmail.com> Message-ID: On 12/20/06, Guido van Rossum wrote: > On 12/20/06, Jim Jewett wrote: > > [(lambda i=i: i) for i in range(10)] > > => > > [(lambda i=0_i: i) for 0_i in range(10)] > > The argument to lambda can't be renamed, because it could be called as > > a keyword. > > Perhaps you were thinking of a textual > substitution? That's not at all how one would do this. Yes; I think I was still thinking of the Py2->Py3 translator as well. > I have no idea what you mean by "works as implemented". Do you propose > *not* to fix the scope bleed of list comprehension loop control > variables? That's not an option. I propose that if fixing the scope bleed *in a straightforward manner* entails other changes -- but those changes are restricted to expressions inside the the list comprehension -- then those other changes should be considered. Changes to scope resolution aren't great, but there will be one here anyhow (the "scope bleed" itself), and if something even more obscure also changes -- that would be a reasonable price to pay for a simpler solution. -jJ From brett at python.org Wed Dec 20 23:11:25 2006 From: brett at python.org (Brett Cannon) Date: Wed, 20 Dec 2006 14:11:25 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > > I've written a quick version of PEP 3106, which expresses my ideas > about how the dict methods to access keys, values and items should be > redone. > > The text is in svn: > http://svn.python.org/view/peps/trunk/pep-3106.txt?rev=53096&view=markup > > At some point it will appear on python.org: > http://python.org/dev/peps/pep-3106/ > > Comments please? (Or we can skip the comments and go straight to the > implementation stage. Patch anyone?) To comment on the open issues, I don't see why they can't be exposed and be subclassable. Couldn't they go in the collections module? And as for subclassing, I don't see how that would hurt anything. As for the names, could go with MappingKeysView or something. It's wordy, but you know what the classes are by name alone. Otherwise I would go with dict_keys or something so that the repr of these things is makes it easy to know what they are. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061220/d203c0b4/attachment.htm From aahz at pythoncraft.com Thu Dec 21 01:17:28 2006 From: aahz at pythoncraft.com (Aahz) Date: Wed, 20 Dec 2006 16:17:28 -0800 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: <200612191702.06999.fdrake@acm.org> References: <200612191702.06999.fdrake@acm.org> Message-ID: <20061221001728.GA5541@panix.com> On Tue, Dec 19, 2006, Fred L. Drake, Jr. wrote: > On Tuesday 19 December 2006 16:47, Terry Reedy wrote: >> >> Perhaps for some, but I would expect that docs cluttered with 'New/changed >> in 3.0' to be just as confusing for others. Perhaps we need two versions >> of the docs: a Python x.y set and a Python forever set. > > Hmm. I think a little bit of JavaScript would go a long way. ;-) After > looking at the code for the HTML generation, I don't think that would be > difficult to do at all. Die, evil scum. Signed, Lynx user. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Don't tell me you didn't KNOW that was coming From greg.ewing at canterbury.ac.nz Thu Dec 21 01:19:16 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 21 Dec 2006 13:19:16 +1300 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: <20061221001728.GA5541@panix.com> References: <200612191702.06999.fdrake@acm.org> <20061221001728.GA5541@panix.com> Message-ID: <4589D304.8030808@canterbury.ac.nz> > On Tue, Dec 19, 2006, Fred L. Drake, Jr. wrote: >>Hmm. I think a little bit of JavaScript would go a long way. ;-) Noooooo! Javascript in the Python docs would be *insane*. -- Greg From greg.ewing at canterbury.ac.nz Thu Dec 21 01:21:37 2006 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 21 Dec 2006 13:21:37 +1300 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: References: <45892F53.4090601@gmail.com> Message-ID: <4589D391.9070408@canterbury.ac.nz> Jim Jewett wrote: > Knowing which i's to rename strikes *me* as tricky, It shouldn't be too hard if you approach it the right way. One way would be to temporarily change the symbol table entry for the loop var when compiling the loop body and any nested functions so that it refers to a different slot in the locals. -- Greg From talin at acm.org Thu Dec 21 04:52:44 2006 From: talin at acm.org (Talin) Date: Wed, 20 Dec 2006 19:52:44 -0800 Subject: [Python-3000] A couple of PEP 3101 related questions Message-ID: <458A050C.1060602@acm.org> 1) What's the process for editing a PEP after it has been reviewed, but before its been accepted? I've recently had some requests to make some edits to the PEP. However, I don't want to do these in "stealth mode", like a congressional staffer who inserts changes into a bill right before it's voted on. 2) What's a good way to prototype a change to a built-in class? Is there a way that you can write a function in Python, and add it as a method to a built-in class? Specifically, what I'd like to be able to do is to add a method to the standard string class, and this should affect all strings, even ones created via literal quotes. Specifically what I'm aiming for is to be able to write unit tests for 3101 without needing to have the C code already in place. That way, whoever volunteers to do the C code can have the unit tests already working before they begin. -- Talin From brett at python.org Thu Dec 21 05:20:22 2006 From: brett at python.org (Brett Cannon) Date: Wed, 20 Dec 2006 20:20:22 -0800 Subject: [Python-3000] A couple of PEP 3101 related questions In-Reply-To: <458A050C.1060602@acm.org> References: <458A050C.1060602@acm.org> Message-ID: On 12/20/06, Talin wrote: > > 1) What's the process for editing a PEP after it has been reviewed, but > before its been accepted? If you are the original author, just edit it. If you are not, either get the original author to change it or email python-dev/python-3000 your issue and someone can make the edit. I've recently had some requests to make some edits to the PEP. However, > I don't want to do these in "stealth mode", like a congressional staffer > who inserts changes into a bill right before it's voted on. Just post them here and someone will add them if appropriate. 2) What's a good way to prototype a change to a built-in class? Is there > a way that you can write a function in Python, and add it as a method to > a built-in class? Nope. Could write it in Python and then in the built-in type have it import the Python module and delegate to the Python function. Specifically, what I'd like to be able to do is to add a method to the > standard string class, and this should affect all strings, even ones > created via literal quotes. > > Specifically what I'm aiming for is to be able to write unit tests for > 3101 without needing to have the C code already in place. That way, > whoever volunteers to do the C code can have the unit tests already > working before they begin. The above idea should do it. Otherwise you will need a local copy that does not have the C version be read-only. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061220/03ec0ad1/attachment.html From brett at python.org Thu Dec 21 05:43:39 2006 From: brett at python.org (Brett Cannon) Date: Wed, 20 Dec 2006 20:43:39 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/18/06, Guido van Rossum wrote: [SNIP] Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > we're doing will be (a) deleting silly old stuff; (b) rename modules > that don't conform to the current module/package naming convention, > like StringIO, cPickle or UserDict. Care to give a more concrete rule on (a)? For instance, is the AL/al modules worth keeping around, or any of the IRIX modules? What about modules that still lack documentation? How about modules that have not been updated since a certain version like 1.5.2 or a certain amount of time (like 3 or 4 years)? I just want to get a rough idea so that a separate thread can be started to discuss modules that should go. We can do svn log checks on code and documentation to try to automatically find out what modules have no love. We can also do Google Code Search queries on import statements to see how much various modules are used. As for (b), does this also extend to modules within a package? For instance, wsgi.simple_server or a bunch of the distutils submodules have underscores in them and PEP 8 says underscores are bad for modules and packages. Similar issue goes for xml.etree.ElementTree. But there is no mention in the PEP about modules within a package. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061220/0a8ff0b1/attachment.htm From fdrake at acm.org Thu Dec 21 07:57:28 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 21 Dec 2006 01:57:28 -0500 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: <20061221001728.GA5541@panix.com> References: <200612191702.06999.fdrake@acm.org> <20061221001728.GA5541@panix.com> Message-ID: <200612210157.28856.fdrake@acm.org> On Wednesday 20 December 2006 19:17, Aahz wrote: > Die, evil scum. :-) Actually, I don't think what I had in mind would prevent Lynx users from being able to access everything in the docs, except the ability to hide the older version notes. There wouldn't be information that you couldn't access. These days, that seems more than reasonable. There's no reason Lynx couldn't implement support for everything involved; it's not the GUI that important here. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Thu Dec 21 07:58:22 2006 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 21 Dec 2006 01:58:22 -0500 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: <4589D304.8030808@canterbury.ac.nz> References: <20061221001728.GA5541@panix.com> <4589D304.8030808@canterbury.ac.nz> Message-ID: <200612210158.22600.fdrake@acm.org> On Wednesday 20 December 2006 19:19, Greg Ewing wrote: > Noooooo! Javascript in the Python docs > would be *insane*. I can probably be convinced to use Python when it's ubiquitous in browsers. ;-) -Fred -- Fred L. Drake, Jr. From ncoghlan at gmail.com Thu Dec 21 09:56:29 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Dec 2006 18:56:29 +1000 Subject: [Python-3000] Efficient LC's which don't leak the iteration variables (was Re: Py3k release schedule worries) In-Reply-To: <4589D391.9070408@canterbury.ac.nz> References: <45892F53.4090601@gmail.com> <4589D391.9070408@canterbury.ac.nz> Message-ID: <458A4C3D.6030403@gmail.com> Greg Ewing wrote: > Jim Jewett wrote: > >> Knowing which i's to rename strikes *me* as tricky, > > It shouldn't be too hard if you approach it the > right way. One way would be to temporarily change > the symbol table entry for the loop var when > compiling the loop body and any nested functions > so that it refers to a different slot in the > locals. The more I think about it, the more I think it will be simplest to tackle the list comprehension as a 'pseudo-scope'. These would still effectively create new scopes, but because they're executed immediately, they can get away with borrowing the local namespace and execution frame of the current function rather than running independently. The compiler already has to keep track of which scope it is in order to handle closure variables - the idea would be to add a mechanism to the scope stack whereby each entry could either be a 'real' scope (actual separate local namespace) or a 'virtual' scope (names in the scope are remapped to start with an appropriate numeric prefix in the current local namespace). Georg, if you want to tackle this, go ahead. Otherwise, I'll look into it over the next couple of weeks. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From ncoghlan at gmail.com Thu Dec 21 10:09:43 2006 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 21 Dec 2006 19:09:43 +1000 Subject: [Python-3000] A couple of PEP 3101 related questions In-Reply-To: <458A050C.1060602@acm.org> References: <458A050C.1060602@acm.org> Message-ID: <458A4F57.5070008@gmail.com> Talin wrote: > 1) What's the process for editing a PEP after it has been reviewed, but > before its been accepted? > > I've recently had some requests to make some edits to the PEP. However, > I don't want to do these in "stealth mode", like a congressional staffer > who inserts changes into a bill right before it's voted on. A message to the relevant mailing list giving the gist of the changes that were made should be enough notification (and don't forget that anyone that wants the exact diff can always look it up on svn.python.org). > 2) What's a good way to prototype a change to a built-in class? Is there > a way that you can write a function in Python, and add it as a method to > a built-in class? > > Specifically, what I'd like to be able to do is to add a method to the > standard string class, and this should affect all strings, even ones > created via literal quotes. As Brett suggested, you can add a method to the builtin class that imports the relevant Python module and then calls the function you want to use. You still need to write some C code, but not much of it. main.c uses this trick to bootstrap itself into runpy.run_module() when someone uses the '-m' switch, so that should provide a decent model for your interim string method implementation. > Specifically what I'm aiming for is to be able to write unit tests for > 3101 without needing to have the C code already in place. That way, > whoever volunteers to do the C code can have the unit tests already > working before they begin. Sounds like a good plan. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org From guido at python.org Thu Dec 21 21:59:11 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Dec 2006 12:59:11 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/20/06, Brett Cannon wrote: > On 12/18/06, Guido van Rossum wrote: > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > > we're doing will be (a) deleting silly old stuff; (b) rename modules > > that don't conform to the current module/package naming convention, > > like StringIO, cPickle or UserDict. > > Care to give a more concrete rule on (a)? For instance, is the AL/al > modules worth keeping around, or any of the IRIX modules? What about > modules that still lack documentation? How about modules that have not been > updated since a certain version like 1.5.2 or a certain amount of time (like > 3 or 4 years)? No, I don't want to give a blanket rule. Come up with a list of modules *and* reasons why they should be deleted and I'll review it. > I just want to get a rough idea so that a separate thread can be started to > discuss modules that should go. We can do svn log checks on code and > documentation to try to automatically find out what modules have no love. > We can also do Google Code Search queries on import statements to see how > much various modules are used. I don't think this is something that needs to be automated. It needs actual thought. > As for (b), does this also extend to modules within a package? For > instance, wsgi.simple_server or a bunch of the distutils submodules have > underscores in them and PEP 8 says underscores are bad for modules and > packages. Similar issue goes for xml.etree.ElementTree. But there is no > mention in the PEP about modules within a package. The email package renamed all its internal modules to conform. But since the packages you mention here are owned by external contributors, this should be negotiated on a case-by-case basis. I personally find ElementTree.py a worse offender than simple_server.py (and I'm not sure I still agree 100% with the rule against underscores). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Thu Dec 21 22:21:33 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Thu, 21 Dec 2006 22:21:33 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: Guido van Rossum wrote: > I personally find ElementTree.py a worse offender than simple_server.py if you want consistency, you shouldn't change the style guide quite as often ;-) (when ET was written, the official style guide said "Module names can be either MixedCase or lowercase. /.../ Modules that export a single class (or a number of closely related classes, plus some additional support) are often named in MixedCase, with the module name being the same as the class name (e.g. the standard StringIO module).") for ET 1.3, I'll probably make more stuff available via __init__.py, so you don't have to explicitly import the ElementTree module in new code. > (and I'm not sure I still agree 100% with the rule against > underscores). not using underscores in package names makes a certain sense, but not allowing them in module names strikes me as misguided. what's the rationale? From baptiste13 at altern.org Thu Dec 21 22:35:53 2006 From: baptiste13 at altern.org (Baptiste Carvello) Date: Thu, 21 Dec 2006 22:35:53 +0100 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: Message-ID: Brett Cannon a ?crit : > But compile (along wth exec, but I don't think you wanted to move that) > are just plain different. Dynamic code execution is its own thing that > I do not directly associate with the specifics of the interpreter. > Maybe a module named 'dynamic' or 'execution'? > btw, we already have a "compiler" and a "parser" module. Maybe they should all be gathered in your "execution" module ? BC From guido at python.org Thu Dec 21 22:49:46 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Dec 2006 13:49:46 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/21/06, Fredrik Lundh wrote: > not using underscores in package names makes a certain sense, but > not allowing them in module names strikes me as misguided. what's > the rationale? See, I don't remember. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From sluggoster at gmail.com Thu Dec 21 23:53:27 2006 From: sluggoster at gmail.com (Mike Orr) Date: Thu, 21 Dec 2006 14:53:27 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: <6e9196d20612211453o6a447cbcj51981c1c76743722@mail.gmail.com> On 12/21/06, Guido van Rossum wrote: > On 12/21/06, Fredrik Lundh wrote: > > not using underscores in package names makes a certain sense, but > > not allowing them in module names strikes me as misguided. what's > > the rationale? > > See, I don't remember. :-) Maybe to make it easy to see it's a module and not a function or local variable? +1 for allowing underscores in module names. -- Mike Orr From barry at python.org Fri Dec 22 00:07:09 2006 From: barry at python.org (Barry Warsaw) Date: Thu, 21 Dec 2006 18:07:09 -0500 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: <6e9196d20612211453o6a447cbcj51981c1c76743722@mail.gmail.com> References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> <6e9196d20612211453o6a447cbcj51981c1c76743722@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Dec 21, 2006, at 5:53 PM, Mike Orr wrote: > On 12/21/06, Guido van Rossum wrote: >> On 12/21/06, Fredrik Lundh wrote: >>> not using underscores in package names makes a certain sense, but >>> not allowing them in module names strikes me as misguided. what's >>> the rationale? >> >> See, I don't remember. :-) > > Maybe to make it easy to see it's a module and not a function or > local variable? > > +1 for allowing underscores in module names. I think we just made that rule because we though they looked ugly. I still think they do, but sometimes they're less ugly than not having them, especially when you're consistent with the no-mixed-case rule. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRYsTonEjvBPtnXfVAQK3+wQArW4TlkzemUYAQ4YxUcUGEEKTZEWIFaTQ p5VIDte8vIerRD+6blPfTrKA3rLiKxUADiqGvnUjeDsyLc3SuioL504abeFP9JCk bNmvn01R+vK/RQE4iHvORrlRbbQsU8nh/jyO1YFsrA5r9aVNLHkyFvwe1KPbKWYO 4k7p+DMm3rg= =XGKy -----END PGP SIGNATURE----- From brett at python.org Fri Dec 22 00:34:38 2006 From: brett at python.org (Brett Cannon) Date: Thu, 21 Dec 2006 15:34:38 -0800 Subject: [Python-3000] Py3k release schedule worries In-Reply-To: References: <9e804ac0612181633p4a5b614an54dcd5b753d780eb@mail.gmail.com> Message-ID: On 12/21/06, Guido van Rossum wrote: > On 12/20/06, Brett Cannon wrote: > > On 12/18/06, Guido van Rossum wrote: > > > Ok, so be it. Let this be a pronouncement -- the only stdlib reorg > > > we're doing will be (a) deleting silly old stuff; (b) rename modules > > > that don't conform to the current module/package naming convention, > > > like StringIO, cPickle or UserDict. > > > > Care to give a more concrete rule on (a)? For instance, is the AL/al > > modules worth keeping around, or any of the IRIX modules? What about > > modules that still lack documentation? How about modules that have not been > > updated since a certain version like 1.5.2 or a certain amount of time (like > > 3 or 4 years)? > > No, I don't want to give a blanket rule. Come up with a list of > modules *and* reasons why they should be deleted and I'll review it. > OK, I will come up with an initial list and start a thread where people can either throw a fit or add to it. > > I just want to get a rough idea so that a separate thread can be started to > > discuss modules that should go. We can do svn log checks on code and > > documentation to try to automatically find out what modules have no love. > > We can also do Google Code Search queries on import statements to see how > > much various modules are used. > > I don't think this is something that needs to be automated. It needs > actual thought. > > > As for (b), does this also extend to modules within a package? For > > instance, wsgi.simple_server or a bunch of the distutils submodules have > > underscores in them and PEP 8 says underscores are bad for modules and > > packages. Similar issue goes for xml.etree.ElementTree. But there is no > > mention in the PEP about modules within a package. > > The email package renamed all its internal modules to conform. But > since the packages you mention here are owned by external > contributors, this should be negotiated on a case-by-case basis. I > personally find ElementTree.py a worse offender than simple_server.py > (and I'm not sure I still agree 100% with the rule against > underscores). > OK. -Brett From brett at python.org Fri Dec 22 03:09:00 2006 From: brett at python.org (Brett Cannon) Date: Thu, 21 Dec 2006 18:09:00 -0800 Subject: [Python-3000] Any platforms we want to drop for Py3K? Message-ID: I am starting to compile the list of modules to suggest for removal and I noticed that there are still a large number for platform-specific directories in Lib/. I was wondering if we should consider some platforms for removal. The reason I am not sending this to python-dev is I figure we could possibly be a little bit more aggressive with the removal of support here. Anyway, I would think we could remove the following platforms: * AtheOS * BeOS * FreeBSD 2 (maybe more?) * IRIX * NeXT * OS/2 * SunOS 5 * UnixWare If any of these are not considered contraversial at all we can also deprecate for removal in Python 2.x. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061221/939458fd/attachment.html From guido at python.org Fri Dec 22 03:40:47 2006 From: guido at python.org (Guido van Rossum) Date: Thu, 21 Dec 2006 18:40:47 -0800 Subject: [Python-3000] Any platforms we want to drop for Py3K? In-Reply-To: References: Message-ID: Actually you *should* post this to python-dev. Let's be aggressive. All of those look like we should've stopped supporting them years ago... :-) On 12/21/06, Brett Cannon wrote: > I am starting to compile the list of modules to suggest for removal and I > noticed that there are still a large number for platform-specific > directories in Lib/. I was wondering if we should consider some platforms > for removal. > > The reason I am not sending this to python-dev is I figure we could possibly > be a little bit more aggressive with the removal of support here. > > Anyway, I would think we could remove the following platforms: > > * AtheOS > * BeOS > * FreeBSD 2 (maybe more?) > * IRIX > * NeXT > * OS/2 > * SunOS 5 > * UnixWare > > If any of these are not considered contraversial at all we can also > deprecate for removal in Python 2.x. > > -Brett > > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ronaldoussoren at mac.com Fri Dec 22 07:54:14 2006 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Fri, 22 Dec 2006 07:54:14 +0100 Subject: [Python-3000] Any platforms we want to drop for Py3K? In-Reply-To: References: Message-ID: <34C2D237-24B7-44B6-B18E-5537B476AE7C@mac.com> On 22 Dec, 2006, at 3:09, Brett Cannon wrote: > I am starting to compile the list of modules to suggest for removal > and I > noticed that there are still a large number for platform-specific > directories in Lib/. I was wondering if we should consider some > platforms > for removal. > > The reason I am not sending this to python-dev is I figure we could > possibly > be a little bit more aggressive with the removal of support here. > > Anyway, I would think we could remove the following platforms: > > * AtheOS This one probably shouldn't be dropped, it is a fairly new OS. > * SunOS 5 AKA Solaris from version 2 upto at least 9 (I haven't played with Solaris 10 so can't say if that still uses a sunos 5 kernel). Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3562 bytes Desc: not available Url : http://mail.python.org/pipermail/python-3000/attachments/20061222/2ad5f7a0/attachment.bin From fredrik at pythonware.com Fri Dec 22 09:55:00 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 22 Dec 2006 09:55:00 +0100 Subject: [Python-3000] versionadded/versionchanged in doc In-Reply-To: References: Message-ID: Neal Norwitz wrote: > What do we want to do with the current versionadded/versionchanged > markups in the doc for 3k? Should we remove all references to 1.x > changes? all 2.x changes? Keep them? all of them? the right answer is of course to move the documentation over to an environment that can track such changes, and render things dynamically based on whatever version/platform the user's interested in. moving the reference portions of the documentation into the source files might also be a pretty good idea (DRY!). I'd say a suitable requirement for 3.0's standard library is that you should be able to do this: import module import docgenerator model = docgenerator.getinfomodel(module) and get an information model that contains enough information to generate a manual page that's at least as good as your average library reference latex page. I'll prepare a draft PEP when I find the time. From andre.roberge at gmail.com Fri Dec 22 14:45:41 2006 From: andre.roberge at gmail.com (Andre Roberge) Date: Fri, 22 Dec 2006 09:45:41 -0400 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 Message-ID: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> A few months ago, there was an active discussion on edu-sig regarding the proposed fate of raw_input(). The text below is an attempt at summarizing the discussion in the form of a tentative PEP. It is respectfully submitted for your consideration. If it is to be considered, in some form, as an official PEP, I have absolutely no objection for a regular python-dev contributor to take over the ownership/authorship. Andr? Roberge ----------------------------------------------------------------- PEP: XXX Title: Simple input built-in in Python 3000 Version: $Revision: 0.2 $ Last-Modified: $Date: 2006/12/22 10:00:00 $ Author: Andr? Roberge Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Sep-2006 Python-Version: 3.0 Post-History: Abstract ======== Input and output are core features of computer programs. Currently, Python provides a simple means of output through the print keyword and two simple means of interactive input through the input() and raw_input() built-in functions. Python 3.0 will introduces various incompatible changes with previous Python versions[1]. Among the proposed changes, print will become a built-in function, print(), while input() and raw_input() would be removed completely from the built-in namespace, requiring importing some module to provide even the most basic input capability. This PEP proposes that Python 3.0 retains some simple interactive user input capability, equivalent to raw_input(), within the built-in namespace. Motivation ========== With its easy readability and its support for many programming styles (e.g. procedural, object-oriented, etc.) among others, Python is perhaps the best computer language to use in introductory programming classes. Simple programs often need to provide information to the user (output) and to obtain information from the user (interactive input). Any computer language intended to be used in an educational setting should provide straightforward methods for both output and interactive input. The current proposals for Python 3.0 [1] include a simple output pathway via a built-in function named print(), but a more complicated method for input [e.g. via sys.stdin.readline()], one that requires importing an external module. Current versions of Python (pre-3.0) include raw_input() as a built-in function. With the availability of such a function, programs that require simple input/output can be written from day one, without requiring discussions of importing modules, streams, etc. Rationale ========= Current built-in functions, like input() and raw_input(), are found to be extremely useful in traditional teaching settings. (For more details, see [2] and the discussion that followed.) While the BDFL has clearly stated [3] that input() was not to be kept in Python 3000, he has also stated that he was not against revising the decision of killing raw_input(). raw_input() provides a simple mean to ask a question and obtain a response from a user. The proposed plans for Python 3.0 would require the replacement of the single statement name = raw_input("What is your name?") by the more complicated import sys print("What is your name?") same = sys.stdin.readline() However, from the point of view of many Python beginners and educators, the use of sys.stdin.readline() presents the following problems: 1. Compared to the name "raw_input", the name "sys.stdin.readline()" is clunky and inelegant. 2. The names "sys" and "stdin" have no meaning for most beginners, who are mainly interested in *what* the function does, and not *where* in the package structure it is located. The lack of meaning also makes it difficult to remember: is it "sys.stdin.readline()", or "stdin.sys.readline()"? To a programming novice, there is not any obvious reason to prefer one over the other. In contrast, functions simple and direct names like print, input, and raw_input, and open are easier to remember. 3. The use of "." notation is unmotivated and confusing to many beginners. For example, it may lead some beginners to think "." is a standard character that could be used in any identifier. 4. There is an asymmetry with the print function: why is print not called sys.stdout.print()? Specification ============= The built-in input function should be totally equivalent to the existing raw_input() function. Open issues =========== With input() effectively removed from the language, the name raw_input() makes much less sense and alternatives should be considered. The various possibilities mentioned in various forums include: ask() ask_user() get_string() input() # rejected by BDFL prompt() read() user_input() get_response() While it has bee rejected by the BDFL, it has been suggested that the most direct solution would be to rename "raw_input" to "input" in Python 3000. The main objection is that Python 2.x already has a function named "input", and, even though it is not going to be included in Python 3000, having a built-in function with the same name but different semantics may confuse programmers migrating from 2.x to 3000. Certainly, this is no problem for beginners, and the scope of the problem is unclear for more experienced programmers, since raw_input(), while popular with many, is not in universal use. In this instance, the good it does for beginners could be seen to outweigh the harm it does to experienced programmers - although it could cause confusion for people reading older books or tutorials. References ========== .. [1] PEP 3100, Miscellaneous Python 3.0 Plans, Kuchling, Cannon (http://www.python.org/dev/peps/pep-3100/) .. [2] The fate of raw_input() in Python 3000 (http://mail.python.org/pipermail/edu-sig/2006-September/006967.html) .. [3] Educational aspects of Python 3000 (http://mail.python.org/pipermail/python-3000/2006-September/003589.html) Copyright ========= This document has been placed in the public domain. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061222/2b90afe5/attachment.htm From fredrik at pythonware.com Fri Dec 22 16:44:41 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Fri, 22 Dec 2006 16:44:41 +0100 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: Andre Roberge wrote: > The various possibilities mentioned in various forums include: > > ask() > ask_user() > get_string() > input() # rejected by BDFL > prompt() > read() > user_input() > get_response() why not call it "readline", and define it as import sys def readline(): return sys.stdin.readline() ? if you include the definition in the docstring, you get a nice little lead-in to a discussion about modules and object access syntax. From steven.bethard at gmail.com Fri Dec 22 18:17:25 2006 From: steven.bethard at gmail.com (Steven Bethard) Date: Fri, 22 Dec 2006 10:17:25 -0700 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: On 12/22/06, Fredrik Lundh wrote: > Andre Roberge wrote: > > > The various possibilities mentioned in various forums include: > > > > ask() > > ask_user() > > get_string() > > input() # rejected by BDFL > > prompt() > > read() > > user_input() > > get_response() > > why not call it "readline", and define it as > > import sys > > def readline(): > return sys.stdin.readline() > > ? > > if you include the definition in the docstring, you get a nice little > lead-in to a discussion about modules and object access syntax. +1. This also makes it clearer that a whole line is read in. So if you want to, say, read a single character at a time (a frequent c.l.py question) you'll know you have to do something else. STeVe -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From tony at PageDNA.com Fri Dec 22 18:51:20 2006 From: tony at PageDNA.com (Tony Lownds) Date: Fri, 22 Dec 2006 09:51:20 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: <756012D2-A98A-46EE-832D-720694B3C88D@PageDNA.com> On Dec 22, 2006, at 5:45 AM, Andre Roberge wrote: > raw_input() provides a simple mean to ask a question and obtain a > response > from a user. The proposed plans for Python 3.0 would require the > replacement > of the single statement > > name = raw_input("What is your name?") > > by the more complicated > > import sys > print("What is your name?") > same = sys.stdin.readline() > It is even more complicated: import sys print("What is your name?", end='') name = sys.stdin.readline().strip('\r\n') Another reason that sys.stdin.readline() is not quite as good as raw_input is the fact that it does not use readline. +1 from me, I've missed raw_input() while playing with the p3yk branch already. -Tony From brett at python.org Fri Dec 22 21:07:25 2006 From: brett at python.org (Brett Cannon) Date: Fri, 22 Dec 2006 12:07:25 -0800 Subject: [Python-3000] Any platforms we want to drop for Py3K? In-Reply-To: References: Message-ID: On 12/21/06, Guido van Rossum wrote: > Actually you *should* post this to python-dev. Let's be aggressive. > All of those look like we should've stopped supporting them years > ago... :-) > OK, moved it over along with the two comments that came in on this thread. -Brett > On 12/21/06, Brett Cannon wrote: > > I am starting to compile the list of modules to suggest for removal and I > > noticed that there are still a large number for platform-specific > > directories in Lib/. I was wondering if we should consider some platforms > > for removal. > > > > The reason I am not sending this to python-dev is I figure we could possibly > > be a little bit more aggressive with the removal of support here. > > > > Anyway, I would think we could remove the following platforms: > > > > * AtheOS > > * BeOS > > * FreeBSD 2 (maybe more?) > > * IRIX > > * NeXT > > * OS/2 > > * SunOS 5 > > * UnixWare > > > > If any of these are not considered contraversial at all we can also > > deprecate for removal in Python 2.x. > > > > -Brett > > > > > > _______________________________________________ > > Python-3000 mailing list > > Python-3000 at python.org > > http://mail.python.org/mailman/listinfo/python-3000 > > Unsubscribe: > > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > > > > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From brett at python.org Fri Dec 22 21:20:21 2006 From: brett at python.org (Brett Cannon) Date: Fri, 22 Dec 2006 12:20:21 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: On 12/22/06, Steven Bethard wrote: > On 12/22/06, Fredrik Lundh wrote: > > Andre Roberge wrote: > > > > > The various possibilities mentioned in various forums include: > > > > > > ask() > > > ask_user() > > > get_string() > > > input() # rejected by BDFL > > > prompt() > > > read() > > > user_input() > > > get_response() > > > > why not call it "readline", and define it as > > > > import sys > > > > def readline(): > > return sys.stdin.readline() > > > > ? > > > > if you include the definition in the docstring, you get a nice little > > lead-in to a discussion about modules and object access syntax. > > +1. This also makes it clearer that a whole line is read in. So if > you want to, say, read a single character at a time (a frequent c.l.py > question) you'll know you have to do something else. > +1 from me as well, although as Ron suggested, you probably want to strip off the newline if we are targetting this to new programmers. -Brett From pje at telecommunity.com Fri Dec 22 21:52:02 2006 From: pje at telecommunity.com (Phillip J. Eby) Date: Fri, 22 Dec 2006 15:52:02 -0500 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: <5.1.1.6.0.20061222154851.04311ac8@sparrow.telecommunity.com> At 12:20 PM 12/22/2006 -0800, Brett Cannon wrote: >+1 from me as well, although as Ron suggested, you probably want to >strip off the newline if we are targetting this to new programmers. In which case, why not go all the way and use say() and ask()? E.g.: say('Hello,', ask("What's your name?")) One benefit of using 'say' instead of 'print' is that 'say' could then be added to the 2.x line, allowing people to prepare for the switch. (Whereas print's privileged meaning in 2.x prevents it being used as a function.) (I don't think calling the function readline() is a good idea if it's going to strip newlines, btw.) From guido at python.org Fri Dec 22 22:47:35 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Dec 2006 13:47:35 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: I like the exact proposal made here better than any of the alternatives mentioned so far. - Against naming it readline(): the "real" readline doesn't strip the \n and returns an empty string for EOF instead of raising EOFError; I believe the latter is more helpful for true beginners' code. - Against naming it ask() and renaming print() to say(): I find those rather silly names that belong in toy or AI languages. Changing print from statement to function maintains Pythonicity; renaming it say() does not. - I don't expect there will be much potential confusion with the 2.x input(); that function is used extremely rarely. It will be trivial to add rules to the refactoring tool (sandbox/2to3/) that replace input() with eval(input()) and replace raw_input() with input(). --Guido On 12/22/06, Andre Roberge wrote: > A few months ago, there was an active discussion on edu-sig regarding > the proposed fate of raw_input(). The text below is an attempt at > summarizing the discussion in the form of a tentative PEP. > It is respectfully submitted for your consideration. > > If it is to be considered, in some form, as an official PEP, I have > absolutely no objection for a regular python-dev contributor to take over > the > ownership/authorship. > > Andr? Roberge > > ----------------------------------------------------------------- > PEP: XXX > Title: Simple input built-in in Python 3000 > Version: $Revision: 0.2 $ > Last-Modified: $Date: 2006/12/22 10:00:00 $ > Author: Andr? Roberge > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 13-Sep-2006 > Python-Version: 3.0 > Post-History: > > Abstract > ======== > > Input and output are core features of computer programs. Currently, > Python provides a simple means of output through the print keyword > and two simple means of interactive input through the input() > and raw_input() built-in functions. > > Python 3.0 will introduces various incompatible changes with previous > Python versions[1]. Among the proposed changes, print will become a > built-in > function, print(), while input() and raw_input() would be removed completely > from the built-in namespace, requiring importing some module to provide > even the most basic input capability. > > This PEP proposes that Python 3.0 retains some simple interactive user > input capability, equivalent to raw_input(), within the built-in namespace. > > Motivation > ========== > > With its easy readability and its support for many programming styles > (e.g. procedural, object-oriented, etc.) among others, Python is perhaps > the best computer language to use in introductory programming classes. > Simple programs often need to provide information to the user (output) > and to obtain information from the user (interactive input). > Any computer language intended to be used in an educational setting should > provide straightforward methods for both output and interactive input. > > The current proposals for Python 3.0 [1] include a simple output pathway > via a built-in function named print(), but a more complicated method for > input [e.g. via sys.stdin.readline()], one that requires importing an > external > module. Current versions of Python (pre-3.0) include raw_input() as a > built-in function. With the availability of such a function, programs that > require simple input/output can be written from day one, without requiring > discussions of importing modules, streams, etc. > > Rationale > ========= > > Current built-in functions, like input() and raw_input(), are found to be > extremely useful in traditional teaching settings. (For more details, > see [2] and the discussion that followed.) > While the BDFL has clearly stated [3] that input() was not to be kept in > Python 3000, he has also stated that he was not against revising the > decision of killing raw_input(). > > raw_input() provides a simple mean to ask a question and obtain a response > from a user. The proposed plans for Python 3.0 would require the > replacement > of the single statement > > name = raw_input("What is your name?") > > by the more complicated > > import sys > print("What is your name?") > same = sys.stdin.readline() > > However, from the point of view of many Python beginners and educators, the > use of sys.stdin.readline() presents the following problems: > > 1. Compared to the name "raw_input", the name "sys.stdin.readline()" > is clunky and inelegant. > > 2. The names "sys" and "stdin" have no meaning for most beginners, > who are mainly interested in *what* the function does, and not *where* > in the package structure it is located. The lack of meaning also makes > it difficult to remember: > is it "sys.stdin.readline()", or " stdin.sys.readline()"? > To a programming novice, there is not any obvious reason to prefer > one over the other. In contrast, functions simple and direct names like > print, input, and raw_input, and open are easier to remember. > > 3. The use of "." notation is unmotivated and confusing to many beginners. > For example, it may lead some beginners to think "." is a standard > character that could be used in any identifier. > > 4. There is an asymmetry with the print function: why is print not called > sys.stdout.print()? > > > Specification > ============= > > The built-in input function should be totally equivalent to the existing > raw_input() function. > > Open issues > =========== > > With input() effectively removed from the language, the name raw_input() > makes much less sense and alternatives should be considered. The > various possibilities mentioned in various forums include: > > ask() > ask_user() > get_string() > input() # rejected by BDFL > prompt() > read() > user_input() > get_response() > > While it has bee rejected by the BDFL, it has been suggested that the most > direct solution would be to rename "raw_input" to "input" in Python 3000. > The main objection is that Python 2.x already has a function named "input", > and, even though it is not going to be included in Python 3000, > having a built-in function with the same name but different semantics may > confuse programmers migrating from 2.x to 3000. Certainly, this is no > problem > for beginners, and the scope of the problem is unclear for more experienced > programmers, since raw_input(), while popular with many, is not in > universal use. In this instance, the good it does for beginners could be > seen to outweigh the harm it does to experienced programmers - > although it could cause confusion for people reading older books or > tutorials. > > > References > ========== > > .. [1] PEP 3100, Miscellaneous Python 3.0 Plans, Kuchling, Cannon > (http://www.python.org/dev/peps/pep-3100/) > .. [2] The fate of raw_input() in Python 3000 > (http://mail.python.org/pipermail/edu-sig/2006-September/006967.html) > .. [3] Educational aspects of Python 3000 > ( > http://mail.python.org/pipermail/python-3000/2006-September/003589.html) > > > Copyright > ========= > > This document has been placed in the public domain. > > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: > http://mail.python.org/mailman/options/python-3000/guido%40python.org > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Sat Dec 23 00:28:23 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 23 Dec 2006 00:28:23 +0100 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: Guido van Rossum wrote: > I like the exact proposal made here better than any of the > alternatives mentioned so far. > > - Against naming it readline(): the "real" readline doesn't strip the > \n and returns an empty string for EOF instead of raising EOFError; I > believe the latter is more helpful for true beginners' code. given how much trouble true beginners have with exceptions, that's far from obvious. From guido at python.org Sat Dec 23 03:14:09 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Dec 2006 18:14:09 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: On 12/22/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > I like the exact proposal made here better than any of the > > alternatives mentioned so far. > > > > - Against naming it readline(): the "real" readline doesn't strip the > > \n and returns an empty string for EOF instead of raising EOFError; I > > believe the latter is more helpful for true beginners' code. > > given how much trouble true beginners have with exceptions, that's far > from obvious. But they typically aren't used to entering EOF either; EOF is not exactly a typical input in an interactive program, and having to enter it typically means you're talking to a Unix utility that's not really designed for interactive use. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Dec 23 06:49:08 2006 From: guido at python.org (Guido van Rossum) Date: Fri, 22 Dec 2006 21:49:08 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: On 12/19/06, Guido van Rossum wrote: > I've written a quick version of PEP 3106, which expresses my ideas > about how the dict methods to access keys, values and items should be > redone. Thanks for all the comments. The new version is in svn: http://svn.python.org/view/peps/trunk/pep-3106.txt?view=markup At some point it will appear on python.org: http://python.org/dev/peps/pep-3106/ Most issues brought up are either resolved or clarified in the new PEP, or explicitly added as open issues. I have strawman answers for most open issues except for the class names, which I'll leave up to whoever wants to implement this (except I *don't* like Neal's suggestion to name them dict.Keys etc.; that's a rather Javaesque use of a class as a namespace that feels unpythonic to me. OTOH the collections module seems a fine resting place.) Some specific answers to issues brought up that you may not easily find back in the new PEP, and some new (perhaps off-topic) thoughts: - Adding list.discard() is a separate issue and doesn't have to wait for Py3k. I think it's a reasonable suggestion. - I am beginning to wonder if copy() methods should be discouraged in general in favor of explicitly naming the result type you want; e.g. to copy a set x you would write set(x) instead of x.copy(). This makes "generic" copying a bit harder, e.g. suppose you have an x that could be either a set or a frozenset and you want a copy that has the same type as x; but I'm not actually convinced that there are a lot of use cases for that outside the copy module. But this is also off-topic. I believe Alex Martelli may have some ideas on this, since I believe he dislikes using x[:] to copy a list and prefers list(x). - OTOH, d_items.copy() could easily be implemented as self.__d.copy().items(). (I forgot to add this to the PEP.) I wouldn't want to do something like this for copying keys or values, but for items it seems to be fast and space-efficient, since the dict copy doesn't use actual tuples for its items. - I expect that some refinements to this PEP will become apparent once we have a solid proposal for ABCs (which I fully intend to support). - Talin asked for values to be list-like rather than set-like, and proposes bags. Well, I thought bags and multisets are the same thing (unordered lists that may contain duplicates). I really don't want them to be list-like since that implies an ordering (albeit arbitrary) and an O(1) indexing (v[i]) operation. I believe the Java Collections Framework calls this a Collection -- it is a little bit more than an iterable since it has a length and a membership test, but it doesn't promise that these are efficient. That's pretty close to what d_values does in the new PEP, except I also added an implementation of equality (albeit an inefficient one due to the potential unhashability of values). - Again off-topic: Java's add() method returns true if the collection was changed; its remove() method acts like our discard() but again returns true if the collection was changed. Would it be worth to copy this API style? If I left your specific suggestion or question unaddressed or unanswered, please send it again (after re-reading the PEP, of course). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fredrik at pythonware.com Sat Dec 23 11:03:15 2006 From: fredrik at pythonware.com (Fredrik Lundh) Date: Sat, 23 Dec 2006 11:03:15 +0100 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: Guido van Rossum wrote: > But they typically aren't used to entering EOF either; EOF is not > exactly a typical input in an interactive program, and having to enter > it typically means you're talking to a Unix utility that's not really > designed for interactive use. the progression I had in mind was 1. literals, simple operations, printing the result 2. reading input from the console, simple operations, printing the result 3. reading input from a file, simple operations, printing the result the jump between 2 and 3 is a bit too large in today's Python. another way to address that would be to add standard input and output *objects* (which delegates to sys.stdin/stdout) to the builtin name- space; after all: input.readline() is pretty self-documenting, even if you don't really understand dotted notation just yet. From guido at python.org Sat Dec 23 16:46:18 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Dec 2006 07:46:18 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: On 12/23/06, Fredrik Lundh wrote: > Guido van Rossum wrote: > > > But they typically aren't used to entering EOF either; EOF is not > > exactly a typical input in an interactive program, and having to enter > > it typically means you're talking to a Unix utility that's not really > > designed for interactive use. > > the progression I had in mind was > > 1. literals, simple operations, printing the result > 2. reading input from the console, simple operations, printing > the result > 3. reading input from a file, simple operations, printing the > result > > the jump between 2 and 3 is a bit too large in today's Python. > > another way to address that would be to add standard input and output > *objects* (which delegates to sys.stdin/stdout) to the builtin name- > space; after all: > > input.readline() > > is pretty self-documenting, even if you don't really understand dotted > notation just yet. I don't see a strong connection between reading from interactive input and reading from a file. Reading from input interactively is more similar to accepting user input from a GUI than reading data from a file. One of the first interactive sessions in a beginner's program is the guess-a-number game ("Is it larger than 10?" etc.). This just doesn't have an analogy with reading files. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Sat Dec 23 16:54:03 2006 From: aahz at pythoncraft.com (Aahz) Date: Sat, 23 Dec 2006 07:54:03 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) Message-ID: <20061223155403.GA13959@panix.com> On Tue, Dec 19, 2006, Guido van Rossum wrote: > On 12/19/06, Fredrik Lundh wrote: >> Guido van Rossum wrote: >>> >>> - a spec for the string unification (Perhaps Fredrik has done some >>> work on it in one of those threads that I haven't opened yet?) >> >> (I haven't abandoned this, but it hasn't been a top priority; partially >> because Larry Hastings work on smarter concatenation has showed that >> "lazy evaluation" can work in today's Python, and partially due to the >> schedule/discussion issues you write about.) > > Now's the time for that work to come out of the closet. Should I hold > off on the basic unification-Guido-style (discarding str and renaming > unicode to str, essentially) or can I start that now and will the new > work be able to build on top of that? My recollection is that Larry said he could make it work with Unicode with little difficulty -- I suspect he's either on vacation or ignoring this thread, so I'm renaming and cc'ing him. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "I support family values -- Addams family values" --www.nancybuttons.com From gsakkis at rutgers.edu Sat Dec 23 17:01:16 2006 From: gsakkis at rutgers.edu (George Sakkis) Date: Sat, 23 Dec 2006 11:01:16 -0500 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: <91ad5bf80612230801h74eb9ecdl831f43bca0716f2@mail.gmail.com> On 12/23/06, Guido van Rossum wrote: > If I left your specific suggestion or question unaddressed or > unanswered, please send it again (after re-reading the PEP, of > course). Shouldn't there be a motivation section ? If nothing else, it should discuss (or at least link to a previous discussion) why the original plan of using plain iterators was rejected. Iteration is probably the most common view operation by far, and it's trivial for the user to populate any other container if necessary. Is the complexity of adding three new types with non obvious semantics justified ? If so, some more compelling use cases would help; I don't think the shown example is really an improvement in readability or performance over for k, v in d.items(): ... for k, v in d.items(): .. Regards, George From guido at python.org Sat Dec 23 17:08:45 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Dec 2006 08:08:45 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: BTW, can someone clean up and check in the proto-PEP and start working on an implementation or patch? Should be really simple. I'd like to see a patch for the refactoring tool (sandbox/2to3) as well. --Guido -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tony at PageDNA.com Sat Dec 23 18:05:51 2006 From: tony at PageDNA.com (Tony Lownds) Date: Sat, 23 Dec 2006 09:05:51 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: References: Message-ID: <5856283A-E35D-460D-B78A-5212FE82478F@PageDNA.com> On Dec 22, 2006, at 9:49 PM, Guido van Rossum wrote: > - Talin asked for values to be list-like ... I really don't want > them to be list-like since that implies an ordering Will iter(keys()) and iter(values()) have the same relative order, as they do today? Ie, d == dict(zip(d.keys(), d.values())) -Tony From larry at hastings.org Sat Dec 23 21:08:16 2006 From: larry at hastings.org (Larry Hastings) Date: Sat, 23 Dec 2006 12:08:16 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) In-Reply-To: <20061223155403.GA13959@panix.com> References: <20061223155403.GA13959@panix.com> Message-ID: <458D8CB0.3030909@hastings.org> Aahz wrote: > On Tue, Dec 19, 2006, Guido van Rossum wrote: > >> On 12/19/06, Fredrik Lundh wrote: >> >>> (I haven't abandoned this, but it hasn't been a top priority; partially >>> because Larry Hastings work on smarter concatenation has showed that >>> "lazy evaluation" can work in today's Python, and partially due to the >>> schedule/discussion issues you write about.) >>> >> Now's the time for that work to come out of the closet. Should I hold >> off on the basic unification-Guido-style (discarding str and renaming >> unicode to str, essentially) or can I start that now and will the new >> work be able to build on top of that? >> > My recollection is that Larry said he could make it work with Unicode > with little difficulty -- I suspect he's either on vacation or ignoring > this thread, so I'm renaming and cc'ing him. > (Howdy folks. I wasn't reading the Python 3000 mailing list, but here I am.) The "lazy strings" patch can be adapted to Unicode strings, you bet. And if it has a chance of going in I'm happy to do it. I'm guessing there's more interest in the lazy concatenation than in the lazy slices, so I'll file them as separate patches. Gimme, oh, two weeks. Cheers, /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061223/87ce5de8/attachment.htm From guido at python.org Sun Dec 24 02:17:52 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Dec 2006 17:17:52 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: <5856283A-E35D-460D-B78A-5212FE82478F@PageDNA.com> References: <5856283A-E35D-460D-B78A-5212FE82478F@PageDNA.com> Message-ID: On 12/23/06, Tony Lownds wrote: > > On Dec 22, 2006, at 9:49 PM, Guido van Rossum wrote: > > - Talin asked for values to be list-like ... I really don't want > > them to be list-like since that implies an ordering > > Will iter(keys()) and iter(values()) have the same relative order, as > they do > today? > > Ie, d == dict(zip(d.keys(), d.values())) Yes. This is implied by the implementation shown in the PEP, which derives all iterations from the iteration over the dict. I'll make it explicit though. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ironfroggy at gmail.com Sun Dec 24 02:24:28 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 23 Dec 2006 20:24:28 -0500 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> Message-ID: <76fd5acf0612231724k4e52c377m8f7ff56c1da7fc3a@mail.gmail.com> On 12/23/06, Guido van Rossum wrote: > On 12/23/06, Fredrik Lundh wrote: > > Guido van Rossum wrote: > > > > > But they typically aren't used to entering EOF either; EOF is not > > > exactly a typical input in an interactive program, and having to enter > > > it typically means you're talking to a Unix utility that's not really > > > designed for interactive use. > > > > the progression I had in mind was > > > > 1. literals, simple operations, printing the result > > 2. reading input from the console, simple operations, printing > > the result > > 3. reading input from a file, simple operations, printing the > > result > > > > the jump between 2 and 3 is a bit too large in today's Python. > > > > another way to address that would be to add standard input and output > > *objects* (which delegates to sys.stdin/stdout) to the builtin name- > > space; after all: > > > > input.readline() > > > > is pretty self-documenting, even if you don't really understand dotted > > notation just yet. > > I don't see a strong connection between reading from interactive input > and reading from a file. Reading from input interactively is more > similar to accepting user input from a GUI than reading data from a > file. One of the first interactive sessions in a beginner's program is > the guess-a-number game ("Is it larger than 10?" etc.). This just > doesn't have an analogy with reading files. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) I was reading the thread and wanted to make a comment about the "input from a GUI". It should be noted that whatever the names of these functions, or their allowed parameters, etc., the idea of them being more than console IO should be allowed for. The builtins should be easily redefinable when you run a script (at least optionally, in the future) to use a simply prompt GUI and dialogs, etc. Even if the script was written without such in mind. Is this too much to consider? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From guido at python.org Sun Dec 24 02:37:16 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Dec 2006 17:37:16 -0800 Subject: [Python-3000] PEP 3106: Revamping dict.keys(), .values() and .items() In-Reply-To: <91ad5bf80612230801h74eb9ecdl831f43bca0716f2@mail.gmail.com> References: <91ad5bf80612230801h74eb9ecdl831f43bca0716f2@mail.gmail.com> Message-ID: On 12/23/06, George Sakkis wrote: > On 12/23/06, Guido van Rossum wrote: > > > If I left your specific suggestion or question unaddressed or > > unanswered, please send it again (after re-reading the PEP, of > > course). > > Shouldn't there be a motivation section ? If nothing else, it should > discuss (or at least link to a previous discussion) why the original > plan of using plain iterators was rejected. Iteration is probably the > most common view operation by far, and it's trivial for the user to > populate any other container if necessary. Is the complexity of adding > three new types with non obvious semantics justified ? If so, some > more compelling use cases would help; I don't think the shown example > is really an improvement in readability or performance over > for k, v in d.items(): ... > for k, v in d.items(): .. I thought that being able to use keys and items as sets without having to copy them is cool enough to motivate the proposal. I'm sure when this was first brought up (and it was brought up and AFAIK agreed to approximately a year ago) plenty of motivation was discussed. Maybe you can suggest some words for a motivational section, if you think one is needed? At this point I'm personally more interested in getting the specification as tight as possible. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Dec 24 02:41:44 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Dec 2006 17:41:44 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: <76fd5acf0612231724k4e52c377m8f7ff56c1da7fc3a@mail.gmail.com> References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> <76fd5acf0612231724k4e52c377m8f7ff56c1da7fc3a@mail.gmail.com> Message-ID: On 12/23/06, Calvin Spealman wrote: > I was reading the thread and wanted to make a comment about the "input > from a GUI". It should be noted that whatever the names of these > functions, or their allowed parameters, etc., the idea of them being > more than console IO should be allowed for. The builtins should be > easily redefinable when you run a script (at least optionally, in the > future) to use a simply prompt GUI and dialogs, etc. Even if the > script was written without such in mind. > > Is this too much to consider? It is already easy enough to replace any built-in in a specific environment. I don't think we need to make this part of the spec for input(). For example, it should be trivial to write a little driver program that takes a script file and further arguments (for the script) and runs the script in an environment where stdout/stderr go to a Tkinter text window (or Gtk, or whatever) and input() pops up some kind of dialog, for example. IDLE does something like this; its input() reads from the console window. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ironfroggy at gmail.com Sun Dec 24 03:35:27 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 23 Dec 2006 21:35:27 -0500 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> <76fd5acf0612231724k4e52c377m8f7ff56c1da7fc3a@mail.gmail.com> Message-ID: <76fd5acf0612231835n6c38bf3ck1d0fa29e8a1603ce@mail.gmail.com> On 12/23/06, Guido van Rossum wrote: > On 12/23/06, Calvin Spealman wrote: > > I was reading the thread and wanted to make a comment about the "input > > from a GUI". It should be noted that whatever the names of these > > functions, or their allowed parameters, etc., the idea of them being > > more than console IO should be allowed for. The builtins should be > > easily redefinable when you run a script (at least optionally, in the > > future) to use a simply prompt GUI and dialogs, etc. Even if the > > script was written without such in mind. > > > > Is this too much to consider? > > It is already easy enough to replace any built-in in a specific > environment. I don't think we need to make this part of the spec for > input(). For example, it should be trivial to write a little driver > program that takes a script file and further arguments (for the > script) and runs the script in an environment where stdout/stderr go > to a Tkinter text window (or Gtk, or whatever) and input() pops up > some kind of dialog, for example. IDLE does something like this; its > input() reads from the console window. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > I was meaning to say that the functions be defined such, just that the idea is kept in mind throughout the discussions about them and thus nothing is done to make such usages less nice. For example, this would be additional reason to avoid newlines in the input. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From guido at python.org Sun Dec 24 03:43:37 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 23 Dec 2006 18:43:37 -0800 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: <76fd5acf0612231835n6c38bf3ck1d0fa29e8a1603ce@mail.gmail.com> References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> <76fd5acf0612231724k4e52c377m8f7ff56c1da7fc3a@mail.gmail.com> <76fd5acf0612231835n6c38bf3ck1d0fa29e8a1603ce@mail.gmail.com> Message-ID: On 12/23/06, Calvin Spealman wrote: > On 12/23/06, Guido van Rossum wrote: > > On 12/23/06, Calvin Spealman wrote: > > > I was reading the thread and wanted to make a comment about the "input > > > from a GUI". It should be noted that whatever the names of these > > > functions, or their allowed parameters, etc., the idea of them being > > > more than console IO should be allowed for. The builtins should be > > > easily redefinable when you run a script (at least optionally, in the > > > future) to use a simply prompt GUI and dialogs, etc. Even if the > > > script was written without such in mind. > > > > > > Is this too much to consider? > > > > It is already easy enough to replace any built-in in a specific > > environment. I don't think we need to make this part of the spec for > > input(). For example, it should be trivial to write a little driver > > program that takes a script file and further arguments (for the > > script) and runs the script in an environment where stdout/stderr go > > to a Tkinter text window (or Gtk, or whatever) and input() pops up > > some kind of dialog, for example. IDLE does something like this; its > > input() reads from the console window. > > I was meaning to say that the functions be defined such, just that the > idea is kept in mind throughout the discussions about them and thus > nothing is done to make such usages less nice. For example, this would > be additional reason to avoid newlines in the input. Oh, don't worry. I'm not about to let input() return a string ending in a newline; that would just add an unnecessary hurdle to one's first program. I suppose that when input() is implemented as a dialog box, there could be a Cancel button that causes it to raise EOFError. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ironfroggy at gmail.com Sun Dec 24 03:48:52 2006 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 23 Dec 2006 21:48:52 -0500 Subject: [Python-3000] Pre-PEP: Simple input built-in in Python 3000 In-Reply-To: References: <7528bcdd0612220545u147f07a4gb476dd43733dfe46@mail.gmail.com> <76fd5acf0612231724k4e52c377m8f7ff56c1da7fc3a@mail.gmail.com> <76fd5acf0612231835n6c38bf3ck1d0fa29e8a1603ce@mail.gmail.com> Message-ID: <76fd5acf0612231848w4b297f48s20eb3142938f5eb3@mail.gmail.com> On 12/23/06, Guido van Rossum wrote: > On 12/23/06, Calvin Spealman wrote: > > On 12/23/06, Guido van Rossum wrote: > > > On 12/23/06, Calvin Spealman wrote: > > > > I was reading the thread and wanted to make a comment about the "input > > > > from a GUI". It should be noted that whatever the names of these > > > > functions, or their allowed parameters, etc., the idea of them being > > > > more than console IO should be allowed for. The builtins should be > > > > easily redefinable when you run a script (at least optionally, in the > > > > future) to use a simply prompt GUI and dialogs, etc. Even if the > > > > script was written without such in mind. > > > > > > > > Is this too much to consider? > > > > > > It is already easy enough to replace any built-in in a specific > > > environment. I don't think we need to make this part of the spec for > > > input(). For example, it should be trivial to write a little driver > > > program that takes a script file and further arguments (for the > > > script) and runs the script in an environment where stdout/stderr go > > > to a Tkinter text window (or Gtk, or whatever) and input() pops up > > > some kind of dialog, for example. IDLE does something like this; its > > > input() reads from the console window. > > > > I was meaning to say that the functions be defined such, just that the > > idea is kept in mind throughout the discussions about them and thus > > nothing is done to make such usages less nice. For example, this would > > be additional reason to avoid newlines in the input. > > Oh, don't worry. I'm not about to let input() return a string ending > in a newline; that would just add an unnecessary hurdle to one's first > program. I suppose that when input() is implemented as a dialog box, > there could be a Cancel button that causes it to raise EOFError. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) I brought it up mostly because I'm always excited about ways to have one codebase and muitliple interfaces. the anygui prototype, for example, is a fantastic concept. I'd love to see even the tip of the iceberg in that direction in python. -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://ironfroggy-code.blogspot.com/ From j_r_fonseca at yahoo.co.uk Mon Dec 25 11:43:21 2006 From: j_r_fonseca at yahoo.co.uk (=?iso-8859-1?b?Sm9z6Q==?= Fonseca) Date: Mon, 25 Dec 2006 10:43:21 +0000 (UTC) Subject: [Python-3000] Refactoring tool available (work in progress) References: Message-ID: Hi, On Thu, 14 Dec 2006 21:51:25 -0800, Guido van Rossum wrote: > In the sandbox I've been working on a refactoring tool, which could > form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to > invite folks here to give it a try and give me a hand. It certainly > needs more work, but I think that the basic infrastructure is sound. > Check out sandbox/2to3/: > http://svn.python.org/view/sandbox/trunk/2to3/. > > This message is to invite feedback, and to encourage contributions. It > would be great if people tried their hands at writing new > transformations! > > A brief description of how it works: > > [...] > > Finally, I have a dream: a GUI that will let you do this > interactively, sort of like query-replace in Emacs. But this message > is already too long, so I'll stop for now. Thanks for reading this > far. :-) FWIW, I've written an interactive (GUI) refactoring tool in Python. The subject language is not Python but C, and the purpose is to reverse engineer C programs. A video of the tool running is available at http://paginas.fe.up.pt/~mei04010/viva.avi . See http://paginas.fe.up.pt/~mei04010/idc/ for more info. Internally the program is also stored as an AST (there is no means no preserve white-space yet), and the refactorings are implemented in a transformation language inspired on the Stratego language ( http://www.stratego.org/ ). Regards, Jos? Fonseca From guido at python.org Thu Dec 28 02:31:55 2006 From: guido at python.org (Guido van Rossum) Date: Wed, 27 Dec 2006 17:31:55 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations Message-ID: I just noticed that PEP 3107 has quietly been checked in. Thanks Collin and Tony! After skimming it, I have one observation: the grammar in the PEP doesn't match that implemented by Tony's patch. The difference is only apparent for tuple-unpacking parameters (e.g. b and c in "def foo(a, (b, c), d): pass"). The PEP supports this syntax: def foo((a, b): "something"): ... while the patch supports this instead: def foo((a: "something", b: "something_else")): ... (I have to say that I like the patch version better. :-) I also note that the PEP uses foo.__signature__.annotations to access the annotations dict, while the patch uses foo.func_annotations. This is reasonable since we don't have the __signature__ API yet (it's PEP 362, but I don't know its status). Finally, the PEP uses Number, Mapping and Sequence as example annotations. I'd rather not use those since they could incorrectly convey the notion that annotations imply type checking semantics, which thety don't (at least not without some kind of decorator). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Thu Dec 28 02:49:11 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 27 Dec 2006 17:49:11 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: References: Message-ID: Some of these have already been fixed in a version sent to me. I need to check that version in. I haven't reviewed it, but I believe it addresses the __signature__ issue. I should have time to get this in tonight. n -- On 12/27/06, Guido van Rossum wrote: > I just noticed that PEP 3107 has quietly been checked in. Thanks > Collin and Tony! > > After skimming it, I have one observation: the grammar in the PEP > doesn't match that implemented by Tony's patch. The difference is only > apparent for tuple-unpacking parameters (e.g. b and c in "def foo(a, > (b, c), d): pass"). The PEP supports this syntax: > > def foo((a, b): "something"): > ... > > while the patch supports this instead: > > def foo((a: "something", b: "something_else")): > ... > > (I have to say that I like the patch version better. :-) > > I also note that the PEP uses foo.__signature__.annotations to access > the annotations dict, while the patch uses foo.func_annotations. This > is reasonable since we don't have the __signature__ API yet (it's PEP > 362, but I don't know its status). > > Finally, the PEP uses Number, Mapping and Sequence as example > annotations. I'd rather not use those since they could incorrectly > convey the notion that annotations imply type checking semantics, > which thety don't (at least not without some kind of decorator). > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-3000 mailing list > Python-3000 at python.org > http://mail.python.org/mailman/listinfo/python-3000 > Unsubscribe: http://mail.python.org/mailman/options/python-3000/nnorwitz%40gmail.com > From tony at PageDNA.com Thu Dec 28 03:48:03 2006 From: tony at PageDNA.com (Tony Lownds) Date: Wed, 27 Dec 2006 18:48:03 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: References: Message-ID: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> On Dec 27, 2006, at 5:31 PM, Guido van Rossum wrote: > I just noticed that PEP 3107 has quietly been checked in. Thanks > Collin and Tony! > > After skimming it, I have one observation: the grammar in the PEP > doesn't match that implemented by Tony's patch. The difference is only > apparent for tuple-unpacking parameters (e.g. b and c in "def foo(a, > (b, c), d): pass"). The PEP supports this syntax: > > def foo((a, b): "something"): > ... > > while the patch supports this instead: > > def foo((a: "something", b: "something_else")): > ... > > (I have to say that I like the patch version better. :-) > Ok, updated (and sent to Neal for checkin, thanks Neal!) > I also note that the PEP uses foo.__signature__.annotations to access > the annotations dict, while the patch uses foo.func_annotations. This > is reasonable since we don't have the __signature__ API yet (it's PEP > 362, but I don't know its status). > I have changed the language related to PEP 362 so that the __signature__ API isn't restricted unduly. > Finally, the PEP uses Number, Mapping and Sequence as example > annotations. I'd rather not use those since they could incorrectly > convey the notion that annotations imply type checking semantics, > which thety don't (at least not without some kind of decorator). > I've replaced with builtins, but that probably isn't enough of a change. I can work on replacing with arbitrary class names (like A, B, etc). Here's what I have currently. Thanks -Tony ------------------------------------------------------------------------ --------------------------- PEP: 3107 Title: Function Annotations Version: $Revision: 53144 $ Last-Modified: $Date: 2006-12-22 07:46:01 -0800 (Fri, 22 Dec 2006) $ Author: Collin Winter , Tony Lownds Status: Draft Type: Standards Track Requires: 362 Content-Type: text/x-rst Created: 2-Dec-2006 Python-Version: 3.0 Post-History: Abstract ======== This PEP introduces a syntax for adding arbitrary metadata annotations to Python functions [#functerm]_. Rationale ========= Because Python's 2.x series lacks a standard way of annotating a function's parameters and return values (e.g., with information about a what type a function's return value should be), a variety of tools and libraries have appeared to fill this gap [#tailexamp]_. Some utilise the decorators introduced in "PEP 318", while others parse a function's docstring, looking for annotations there. This PEP aims to provide a single, standard way of specifying this information, reducing the confusion caused by the wide variation in mechanism and syntax that has existed until this point. Fundamentals of Function Annotations ==================================== Before launching into a discussion of the precise ins and outs of Python 3.0's function annotations, let's first talk broadly about what annotations are and are not: 1. Function annotations, both for parameters and return values, are completely optional. 2. Function annotations are nothing more than a way of associating arbitrary Python expressions with various parts of a function at compile-time. By itself, Python does not attach any particular meaning or significance to annotations. Left to its own, Python simply makes these expressions available as described in `Accessing Function Annotations`_ below. The only way that annotations take on meaning is when they are interpreted by third-party libraries. These annotation consumers can do anything they want with a function's annotations. For example, one library might use string-based annotations to provide improved help messages, like so:: def compile(source: "something compilable", filename: "where the compilable thing comes from", mode: "is this a single statement or a suite?"): ... Another library might be used to provide typechecking for Python functions and methods. This library could use annotations to indicate the function's expected input and return types, possibly something like :: def haul(item: Haulable, *vargs: PackAnimal) -> Distance: ... However, neither the strings in the first example nor the type information in the second example have any meaning on their own; meaning comes from third-party libraries alone. 3. Following from point 2, this PEP makes no attempt to introduce any kind of standard semantics, even for the built-in types. This work will be left to third-party libraries. There is no worry that these libraries will assign semantics at random, or that a variety of libraries will appear, each with varying semantics and interpretations of what, say, a tuple of strings means. The difficulty inherent in writing annotation interpreting libraries will keep their number low and their authorship in the hands of people who, frankly, know what they're doing. Syntax ====== Parameters ---------- Annotations for parameters take the form of optional expressions that follow the parameter name. This example indicates that parameters 'a' and 'c' should both be a ``int``, while parameter 'b' should both be a ``dict``:: def foo(a: int, b: dict, c: int = 5): ... In pseudo-grammar, parameters now look like ``identifier [: expression] [= expression]``. That is, annotations always precede a parameter's default value and both annotations and default values are optional. Just like how equal signs are used to indicate a default value, colons are used to mark annotations. All annotation expressions are evaluated when the function definition is executed. Annotations for excess parameters (i.e., ``*args`` and ``**kwargs``) are indicated similarly. In the following function definition, ``*args`` is flagged as a tuple of ``int``, and ``**kwargs`` is marked as a dict whose keys are strings and whose values are of type ``str``. :: def foo(*args: int, **kwargs: str): ... Note that, depending on what annotation-interpreting library you're using, the following might also be a valid spelling of the above:: def foo(*args: [int], **kwargs: {str: str}): ... Only the first, however, has the BDFL's blessing [#blessedexcess]_ as the One Obvious Way. Return Values ------------- The examples thus far have omitted examples of how to annotate the type of a function's return value. This is done like so:: def sum(*args: int) -> int: ... The parameter list can now be followed by a literal ``->`` and a Python expression. Like the annotations for parameters, this expression will be evaluated when the function definition is executed. The grammar for function definitions [#grammar]_ is now:: decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE decorators: decorator+ funcdef: [decorators] 'def' NAME parameters ['->' test] ':' suite parameters: '(' [typedargslist] ')' typedargslist: ((tfpdef ['=' test] ',')* ('*' [tname] (',' tname ['=' test])* [',' '**' tname] | '**' tname) | tfpdef ['=' test] (',' tfpdef ['=' test])* [',']) tname: NAME [':' test] tfpdef: tname | '(' tfplist ')' tfplist: tfpdef (',' tfpdef)* [','] Lambda ------ ``lambda``'s syntax does not support annotations. The syntax of ``lambda`` could be changed to support annotations, by requiring parentheses around the parameter list. However it was decided [#lambda]_ not to make this change because: 1. It would be an incompatible change. 2. Lambda's are neutered anyway. 3. The lambda can always be changed to a function. Accessing Function Annotations ============================== Once compiled, a function's annotations are available via the function's ``func_annotations`` attribute. This attribute is a dictionary, mapping parameter names to an object representing the evaluated annotation expression There is a special key in the ``func_annotations`` mapping, ``"return"``. This key is present only if an annotation was supplied for the function's return value. For example, the following annotation:: def foo(a: 'x', b: 5 + 6, c: list) -> str: ... would result in a ``func_annotation`` mapping of :: {'a': 'x', 'b': 11, 'c': list, 'return': str} The ``return`` key was chosen because it cannot conflict with the name of a parameter; any attempt to use ``return`` as a parameter name would result in a ``SyntaxError``. ``func_annotations`` is an empty dictionary if no there are no annotations on the function. ``func_annotations`` is always an empty dictionary for functions created from ``lambda`` expressions. Standard Library ================ pydoc and inspect ----------------- The ``pydoc`` module should display the function annotations when displaying help for a function. The ``inspect`` module should change to support annotations. Relation to Other PEPs ====================== Function Signature Objects [#pep-362]_ -------------------------------------- Function Signature Objects should expose the function's annotations. The ``Parameter`` object may change or other changes may be warranted. Implementation ============== A sample implementation for the syntax changes has been provided [#implementation]_ by Tony Lownds. Rejected Proposals ================== + The BDFL rejected the author's idea for a special syntax for adding annotations to generators as being "too ugly" [#rejectgensyn]_. + Though discussed early on ([#threadgen]_, [#threadhof]_), including special objects in the stdlib for annotating generator functions and higher-order functions was ultimately rejected as being more appropriate for third-party libraries; including them in the standard library raised too many thorny issues. + Despite considerable discussion about a standard type parameterisation syntax, it was decided that this should also be left to third-party libraries. ([#threadimmlist]_, [#threadmixing]_, [#emphasistpls]_) References and Footnotes ======================== .. [#functerm] Unless specifically stated, "function" is generally used as a synonym for "callable" throughout this document. .. [#tailexamp] The author's typecheck_ library makes use of decorators, while `Maxime Bourget's own typechecker`_ utilises parsed docstrings. .. [#blessedexcess] http://mail.python.org/pipermail/python-3000/2006-May/002173.html .. [#rejectgensyn] http://mail.python.org/pipermail/python-3000/2006-May/002103.html .. _typecheck: http://oakwinter.com/code/typecheck/ .. _Maxime Bourget's own typechecker: http://maxrepo.info/taxonomy/term/3,6/all .. [#threadgen] http://mail.python.org/pipermail/python-3000/2006-May/002091.html .. [#threadhof] http://mail.python.org/pipermail/python-3000/2006-May/001972.html .. [#threadimmlist] http://mail.python.org/pipermail/python-3000/2006-May/002105.html .. [#threadmixing] http://mail.python.org/pipermail/python-3000/2006-May/002209.html .. [#emphasistpls] http://mail.python.org/pipermail/python-3000/2006-June/002438.html .. [#implementation] http://python.org/sf/1607548 .. _numeric: http://docs.python.org/lib/typesnumeric.html .. _mapping: http://docs.python.org/lib/typesmapping.html .. _sequence protocols: http://docs.python.org/lib/typesseq.html .. [#grammar] http://www.python.org/doc/current/ref/function.html .. [#lambda] http://mail.python.org/pipermail/python-3000/2006-May/001613.html .. [#pep-362] http://www.python.org/dev/peps/pep-0362/ Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From nnorwitz at gmail.com Thu Dec 28 06:03:23 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 27 Dec 2006 21:03:23 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> Message-ID: On 12/27/06, Tony Lownds wrote: > > On Dec 27, 2006, at 5:31 PM, Guido van Rossum wrote: > > > I just noticed that PEP 3107 has quietly been checked in. Thanks > > Collin and Tony! > > Ok, updated (and sent to Neal for checkin, thanks Neal!) I just checked in a slightly modified version from what you sent me in private mail. Hopefully that was the same version as what was sent to the list. I made a few grammatical changes. That was all. For the next update can you pull down the changes from svn. Also verify I checked in the correct version. I'll be looking at the patch and hopefully check that in soon. Was I supposed to give you svn access? Sorry if I forgot. Did you send you ssh keys to pydotorg? I'm not sure I know how to give access to the PEPs, but I can try to give you access if you send your keys (as an attachment). n From tony at pagedna.com Thu Dec 28 06:40:33 2006 From: tony at pagedna.com (Tony Lownds) Date: Wed, 27 Dec 2006 21:40:33 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> Message-ID: <194630D4-38E5-4CDF-BA5F-3AD048676D3A@pagedna.com> On Dec 27, 2006, at 9:03 PM, Neal Norwitz wrote: > On 12/27/06, Tony Lownds wrote: >> >> On Dec 27, 2006, at 5:31 PM, Guido van Rossum wrote: >> >>> I just noticed that PEP 3107 has quietly been checked in. Thanks >>> Collin and Tony! >> >> Ok, updated (and sent to Neal for checkin, thanks Neal!) > > I just checked in a slightly modified version from what you sent me in > private mail. Hopefully that was the same version as what was sent to > the list. > > I made a few grammatical changes. That was all. For the next update > can you pull down the changes from svn. Also verify I checked in the > correct version. > > I'll be looking at the patch and hopefully check that in soon. > > Was I supposed to give you svn access? Sorry if I forgot. Did you > send you ssh keys to pydotorg? I'm not sure I know how to give access > to the PEPs, but I can try to give you access if you send your keys > (as an attachment). Thanks, checked in version looks good. I sent my keys to the PEP editors based on their instructions. Now that the PEP is posted, I don't think there is any rush to have edit access. I will base my future edits off the SVN version. Do Collin or I need to post to lists besides this one to gather comments? Thanks -Tony From nnorwitz at gmail.com Thu Dec 28 07:53:26 2006 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 27 Dec 2006 22:53:26 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> Message-ID: I checked in a modified version of the patch (rev 53170). Mostly I: * added error checking * updated N_TOKENS in Include/token.h (and Lib/token.py) since a token was added * Changed some code in Lib/compiler/pycodegen.py (I couldn't let Guido see a 2-space indent or he might think Google had corrupted you :-) * Made "return" an interned string Some outstanding questions I have: * Should annotations be allowed to be a subclass of dict? (currently subclasses of dicts are allowed, but generic mappings are not) * With the modification of MAKE_FUNCTION to be a 32-bit value, this means that EXTENDED_ARG is now used. This means that the peephole optimizer won't work for the outer function. I think we should correct this. * I don't understand why there was a new annotations (flag) parameter added in symtable. My guess was that this was to support lambdas without annotations. If this is true, it would be nicer if we didn't have to special case. I think there was talk about requiring parens around lambda params. I don't recall the outcome though. * We should update doc Tony, you cleaned up quite a few places in the AST, care to do more cleanup? I liked your refactorings. I also don't love all the names in Grammar. Guido had mentioned about cleaning up the Grammar. I don't know if he still wants to or if it can be improved. It would be nice once all the basic features are in if we could make a cleanup pass for things like Grammar, obsolete APIs, etc. n On 12/27/06, Neal Norwitz wrote: > On 12/27/06, Tony Lownds wrote: > > > > On Dec 27, 2006, at 5:31 PM, Guido van Rossum wrote: > > > > > I just noticed that PEP 3107 has quietly been checked in. Thanks > > > Collin and Tony! > > > > Ok, updated (and sent to Neal for checkin, thanks Neal!) > > I just checked in a slightly modified version from what you sent me in > private mail. Hopefully that was the same version as what was sent to > the list. > > I made a few grammatical changes. That was all. For the next update > can you pull down the changes from svn. Also verify I checked in the > correct version. > > I'll be looking at the patch and hopefully check that in soon. > > Was I supposed to give you svn access? Sorry if I forgot. Did you > send you ssh keys to pydotorg? I'm not sure I know how to give access > to the PEPs, but I can try to give you access if you send your keys > (as an attachment). > > n > From thomas at python.org Thu Dec 28 10:08:41 2006 From: thomas at python.org (Thomas Wouters) Date: Thu, 28 Dec 2006 10:08:41 +0100 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> Message-ID: <9e804ac0612280108j28e3f202mb66bf50f7ad4910@mail.gmail.com> On 12/28/06, Neal Norwitz wrote: > > It would be nice once all the basic features are in > if we could make a cleanup pass for things like Grammar, obsolete > APIs, etc. FWIW, that was my intention, too. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061228/500c3290/attachment.html From aahz at pythoncraft.com Thu Dec 28 16:19:25 2006 From: aahz at pythoncraft.com (Aahz) Date: Thu, 28 Dec 2006 07:19:25 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: <194630D4-38E5-4CDF-BA5F-3AD048676D3A@pagedna.com> References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> <194630D4-38E5-4CDF-BA5F-3AD048676D3A@pagedna.com> Message-ID: <20061228151925.GA24848@panix.com> On Wed, Dec 27, 2006, Tony Lownds wrote: > > Do Collin or I need to post to lists besides this one to gather > comments? You should post to comp.lang.python, although honestly, anyone who cares about Python 3.0 and isn't on this list is being foolish. I suppose it's a good opportunity to remind people where the action is. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "I support family values -- Addams family values" --www.nancybuttons.com From tony at pagedna.com Thu Dec 28 17:13:05 2006 From: tony at pagedna.com (Tony Lownds) Date: Thu, 28 Dec 2006 08:13:05 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> Message-ID: <582BC187-758E-4367-B8E3-3755BD5F75D9@pagedna.com> On Dec 27, 2006, at 10:53 PM, Neal Norwitz wrote: > I checked in a modified version of the patch (rev 53170). Mostly I: > * added error checking > * updated N_TOKENS in Include/token.h (and Lib/token.py) since a > token was added > * Changed some code in Lib/compiler/pycodegen.py (I couldn't let > Guido see a 2-space indent or he might think Google had corrupted you > :-) > * Made "return" an interned string Thanks > Some outstanding questions I have: > > * Should annotations be allowed to be a subclass of dict? > (currently subclasses of dicts are allowed, but generic mappings > are not) A subclass should be fine. There is no need to check for exactly a dict. > * With the modification of MAKE_FUNCTION to be a 32-bit value, this > means that EXTENDED_ARG is now used. This means that the peephole > optimizer won't work for the outer function. I think we should > correct this. Ok. Something like this should fix it in peephople.c. I can post a patch. @@ -535,8 +535,13 @@ break; case EXTENDED_ARG: - goto exitUnchanged; + if (codestr[i+3] != MAKE_FUNCTION) + goto exitUnchanged; + /* don't visit MAKE_FUNCTION as GETARG will be wrong */ + i += 3; + break; + > * I don't understand why there was a new annotations (flag) parameter > added in symtable. My guess was that this was to support lambdas > without annotations. If this is true, it would be nicer if we didn't > have to special case. I think there was talk about requiring parens > around lambda params. I don't recall the outcome though. > The flag is there to re-use the logic for visiting parameters, once for the annotations, once for the parameter names. Those need to be separate passes because the parameter names are added to the functions symbol table, annotations to the outer symbol table. The thread about requiring parens was here, I took the outcome to be YAGNI on lambda annotations. http://mail.python.org/pipermail/python-3000/2006-May/001613.html > * We should update doc I will start looking at the doc changes that are needed. > Tony, you cleaned up quite a few places in the AST, care to do more > cleanup? I liked your refactorings. > Sure, anything in particular? > I also don't love all the names in Grammar. Guido had mentioned about > cleaning up the Grammar. I don't know if he still wants to or if it > can be improved. It would be nice once all the basic features are in > if we could make a cleanup pass for things like Grammar, obsolete > APIs, etc. The only name changes I made in Grammar were made to allow the same compiler code to process either lambda's arguments without annotations or def's arguments with annotations. fpdef became tfpdef and vfpdef, fplist became tfplist and vfplist. tname is a new nonterminal that holds the annotation itself, and vname is the corresponding version without an annotation. "tname" replaced NAME in many places. Maybe there is a better naming convention that "t" and "v" prefixes? Or perhaps "tname/vname" could be improved? Thanks -Tony From tony at PageDNA.com Sat Dec 30 00:29:09 2006 From: tony at PageDNA.com (Tony Lownds) Date: Fri, 29 Dec 2006 15:29:09 -0800 Subject: [Python-3000] PEP 3107 - Function Annotations In-Reply-To: <20061228151925.GA24848@panix.com> References: <6B99BFD7-F305-4A69-B7F3-04BAE9A8BC82@PageDNA.com> <194630D4-38E5-4CDF-BA5F-3AD048676D3A@pagedna.com> <20061228151925.GA24848@panix.com> Message-ID: <9CE4723A-A5C9-4D77-A16E-BF9FFF50393C@PageDNA.com> On Dec 28, 2006, at 7:19 AM, Aahz wrote: > On Wed, Dec 27, 2006, Tony Lownds wrote: >> >> Do Collin or I need to post to lists besides this one to gather >> comments? > > You should post to comp.lang.python, although honestly, anyone who > cares > about Python 3.0 and isn't on this list is being foolish. I > suppose it's > a good opportunity to remind people where the action is. I have posted to python-list. -Tony From larry at hastings.org Sun Dec 31 00:24:15 2006 From: larry at hastings.org (Larry Hastings) Date: Sat, 30 Dec 2006 15:24:15 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) In-Reply-To: <20061223155403.GA13959@panix.com> References: <20061223155403.GA13959@panix.com> Message-ID: <4596F51F.3010701@hastings.org> On Tue, Dec 19, 2006, Guido van Rossum wrote: > On 12/19/06, Fredrik Lundh wrote: >> (I haven't abandoned this, but it hasn't been a top priority; >> partially because Larry Hastings work on smarter concatenation has >> showed that "lazy evaluation" can work in today's Python, and >> partially due to the schedule/discussion issues you write about.) > Now's the time for that work to come out of the closet. At last, I can take a crack at this, now that Christmas is over. Just double-checking: 1. I know you want it for what are currently "unicode strings" (Objects/unicodeobject.c). Do you /also/ want it for single-byte strings (Objects/stringobject.c)? 2. I should generate my diffs against http://svn.python.org/projects/python/branches/p3yk ? Cheers, /larry/ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-3000/attachments/20061230/11282e49/attachment.html From guido at python.org Sun Dec 31 03:32:33 2006 From: guido at python.org (Guido van Rossum) Date: Sat, 30 Dec 2006 18:32:33 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) In-Reply-To: <4596F51F.3010701@hastings.org> References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org> Message-ID: On 12/30/06, Larry Hastings wrote: > > > On Tue, Dec 19, 2006, Guido van Rossum wrote: > > On 12/19/06, Fredrik Lundh wrote: > > (I haven't abandoned this, but it hasn't been a top priority; partially > because Larry Hastings work on smarter concatenation has showed that "lazy > evaluation" can work in today's Python, and partially due to the > schedule/discussion issues you write about.) Now's the time for that work to > come out of the closet. > At last, I can take a crack at this, now that Christmas is over. Just > double-checking: > > > I know you want it for what are currently "unicode strings" > (Objects/unicodeobject.c). Do you also want it for single-byte strings > (Objects/stringobject.c)? No, the 8-bit strings will die in Py3k anyway. > I should generate my diffs against > http://svn.python.org/projects/python/branches/p3yk ? Yes, please. Thanks for doing this! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From talin at acm.org Sun Dec 31 04:47:27 2006 From: talin at acm.org (Talin) Date: Sat, 30 Dec 2006 19:47:27 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) In-Reply-To: References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org> Message-ID: <459732CF.3020803@acm.org> Guido van Rossum wrote: > On 12/30/06, Larry Hastings wrote: >> >> On Tue, Dec 19, 2006, Guido van Rossum wrote: >> >> On 12/19/06, Fredrik Lundh wrote: >> >> (I haven't abandoned this, but it hasn't been a top priority; partially >> because Larry Hastings work on smarter concatenation has showed that "lazy >> evaluation" can work in today's Python, and partially due to the >> schedule/discussion issues you write about.) Now's the time for that work to >> come out of the closet. >> At last, I can take a crack at this, now that Christmas is over. Just >> double-checking: >> >> >> I know you want it for what are currently "unicode strings" >> (Objects/unicodeobject.c). Do you also want it for single-byte strings >> (Objects/stringobject.c)? > > No, the 8-bit strings will die in Py3k anyway. > >> I should generate my diffs against >> http://svn.python.org/projects/python/branches/p3yk ? > > Yes, please. > > Thanks for doing this! Maybe this would be a good time to review, or at least restate, the specific plans for strings in Py3K? I know that there's been a great deal of discussion on this, but a lot of that discussion took place *before* Larry's work (specifically, before a number of people in this group drank the lazy-strings KoolAid.) I'm specifically concerned about avoiding confusion over the "lazy" aspect of strings, because there's two kinds of "laziness" that has been discussed here: Lazy string manipulation (slice and join), and lazy format conversion (8-bit, 16-bit, 32-bit.) Both are, I think, desirable. They are also inter-related, in that the design of one likely affects the other, so I don't think it makes sense to discuss these issues in isolation. Is there a PEP which defines what is going to happen? I specifically refer to issues of: -- Internal representation of varying-width string encodings -- On-the-fly encoding changes -- C API changes -- String 'views' -- Lazy slicing and concatenation. -- Performance expectations for all of the above. -- Talin From guido at python.org Sun Dec 31 19:43:07 2006 From: guido at python.org (Guido van Rossum) Date: Sun, 31 Dec 2006 10:43:07 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) In-Reply-To: <459732CF.3020803@acm.org> References: <20061223155403.GA13959@panix.com> <4596F51F.3010701@hastings.org> <459732CF.3020803@acm.org> Message-ID: On 12/30/06, Talin wrote: > Maybe this would be a good time to review, or at least restate, the > specific plans for strings in Py3K? I know that there's been a great > deal of discussion on this, but a lot of that discussion took place > *before* Larry's work (specifically, before a number of people in this > group drank the lazy-strings KoolAid.) The *only* think that I care about is that we end up with a single string type named 'str' that has the same semantics (and some of the same performance) as the current Unicode strings. (I mention performance because s[i] should remain an O(1) operation.) I don't particularly care about preserving the C API except from a practical standpoint -- changing all uses of strings would require modifying nearly every other line of the interpreter code, so a decent amount of backward compatibility will help us meet the deadline (of an alpha release by the end of Q2 '07). > I'm specifically concerned about avoiding confusion over the "lazy" > aspect of strings, because there's two kinds of "laziness" that has been > discussed here: Lazy string manipulation (slice and join), and lazy > format conversion (8-bit, 16-bit, 32-bit.) Both are, I think, desirable. > They are also inter-related, in that the design of one likely affects > the other, so I don't think it makes sense to discuss these issues in > isolation. > > Is there a PEP which defines what is going to happen? I specifically > refer to issues of: > > -- Internal representation of varying-width string encodings > -- On-the-fly encoding changes > -- C API changes > -- String 'views' > -- Lazy slicing and concatenation. > -- Performance expectations for all of the above. I tuned out the endless discussions about this, and I doubt that anyone is really eager to repeat them. At this point I prefer to let code speak. If someone thinks they can do better, let them produce a competing patch, and a benchmark that we can agree on. It seems unlikely that we'll be able to address all the issues you mention without doing a lot of implementation work; while I don't recommend jumping in without doing some serious thinking, I really don't want to get back in the mode of endless discussion without a basis in fact. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jcarlson at uci.edu Sun Dec 31 21:10:13 2006 From: jcarlson at uci.edu (Josiah Carlson) Date: Sun, 31 Dec 2006 12:10:13 -0800 Subject: [Python-3000] Lazy strings (was Re: Py3k release schedule worries) In-Reply-To: References: <459732CF.3020803@acm.org> Message-ID: <20061231115800.82E3.JCARLSON@uci.edu> "Guido van Rossum" wrote: > On 12/30/06, Talin wrote: > > Maybe this would be a good time to review, or at least restate, the > > specific plans for strings in Py3K? I know that there's been a great > > deal of discussion on this, but a lot of that discussion took place > > *before* Larry's work (specifically, before a number of people in this > > group drank the lazy-strings KoolAid.) > > The *only* think that I care about is that we end up with a single > string type named 'str' that has the same semantics (and some of the > same performance) as the current Unicode strings. (I mention > performance because s[i] should remain an O(1) operation.) If I understood the lazy string stuff, this requirement more or less kills them (at least with the rope-based implementation that I last paid attention to), unless we consider ammortization arguments. It also seems to kill any attempts to make the underlying representation always ucs-2, with surrogate pairs working the way they were intended (pairs are seen as one character, not two), as the algorithm/structure involved got O(log n) performance for s[i] on strings with surrogate pairs, O(1) otherwise. This does not, however, preclude the construction of a wrapper type that implements lazy strings, views, etc., or its packaging as a 3rd party module for Python 2.x and/or 3.x . - Josiah