From zac256 at gmail.com Sat Aug 1 20:28:05 2009 From: zac256 at gmail.com (Zac Burns) Date: Sat, 1 Aug 2009 11:28:05 -0700 Subject: [Python-ideas] os.listdir with current working directory as default In-Reply-To: References: <94bdd2610905221551k5e4ab70eue973ffdb2d84fd9f@mail.gmail.com> <94bdd2610905221619m6a4b7e79m4dcf2ce407fb301a@mail.gmail.com> <94bdd2610905221701v96a1b59h5aeada22bc0a092b@mail.gmail.com> <91ad5bf80905221709r23bcd712k6814886233ef48b5@mail.gmail.com> <94bdd2610905231708o1222551axf7ef449fc1329621@mail.gmail.com> <333edbe80906091713ma098b0bwd20491725c9cac71@mail.gmail.com> Message-ID: <333edbe80908011128l51897880x9d5adaf519ca63fe@mail.gmail.com> Yes, I always do use the full path. -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games On Tue, Jun 9, 2009 at 11:25 PM, Lie Ryan wrote: > Zac Burns wrote: >>> You'd better be careful with it. The "current directory" is process-wide and so >>> modifying it in a thread will affect all other threads in your program. I'm not >>> saying all programs have this characteristic, but relying on the current >>> directory rather than explicit paths isn't always a good idea... >> >> I agree with Antoine, but would go further to say that making it a >> default argument is an enablement - just making it easier for people >> to follow a bad pattern. >> >> This doesn't apply to all default arguments of course, just that the >> working directory is "considered harmful" and we probably shouldn't >> make changes that make it easier for people to use. > > When you open() files do you always use full path? Why don't we > eliminate all modules and built-in function that uses relative path by > default then? That way we will always be safe? Making open(), etc raise > RelativePathError if using relative path. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From digitalxero at gmail.com Mon Aug 3 07:44:54 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Sun, 2 Aug 2009 23:44:54 -0600 Subject: [Python-ideas] etree improvements Message-ID: Well I find myself converting a project from an OLD version of minidom (from back 1999 or so) to etree. And I find myself wishing there was something like the Element(Tree).findall / Element(Tree).getiterator pair for Element(Tree).find and Element(Tree).findtext, like a Element(Tree).iter_find / Element(Tree).iter_findtext that would recurse till the whole tree is walked or they found the tag / path you specified. Something like http://dpaste.com/hold/74590/ From sridharr at activestate.com Tue Aug 4 18:11:42 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 04 Aug 2009 09:11:42 -0700 Subject: [Python-ideas] Meta idea - monthly digest of python-ideas Message-ID: 1. Create a blog 2. Create a summary of each thread 3. On every month, post the summaries in the blog The summary of a thread may adhere to a standard template, for instance, definition of the idea, its merits, and criticisms. -srid From phd at phd.pp.ru Tue Aug 4 18:25:14 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 4 Aug 2009 20:25:14 +0400 Subject: [Python-ideas] Meta idea - monthly digest of python-ideas In-Reply-To: References: Message-ID: <20090804162514.GB24468@phd.pp.ru> On Tue, Aug 04, 2009 at 09:11:42AM -0700, Sridhar Ratnakumar wrote: > 2. Create a summary of each thread > 3. On every month, post the summaries Wanna work on this? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From tlesher at gmail.com Tue Aug 4 18:49:37 2009 From: tlesher at gmail.com (Tim Lesher) Date: Tue, 4 Aug 2009 12:49:37 -0400 Subject: [Python-ideas] Meta idea - monthly digest of python-ideas In-Reply-To: References: Message-ID: <9613db600908040949j67da04b0y524ba28786f6c1e5@mail.gmail.com> On Tue, Aug 4, 2009 at 12:11, Sridhar Ratnakumar wrote: > 1. Create a blog > 2. Create a summary of each thread > 3. On every month, post the summaries in the blog You mean like the Python-Dev summaries? http://www.python.org/dev/summary/ -- Tim Lesher From SridharR at activestate.com Tue Aug 4 19:53:57 2009 From: SridharR at activestate.com (Sridhar Ratnakumar) Date: Tue, 04 Aug 2009 10:53:57 -0700 Subject: [Python-ideas] Meta idea - monthly digest of python-ideas In-Reply-To: <20090804162514.GB24468@phd.pp.ru> References: <20090804162514.GB24468@phd.pp.ru> Message-ID: On Tue, 04 Aug 2009 09:25:14 -0700, Oleg Broytmann wrote: > On Tue, Aug 04, 2009 at 09:11:42AM -0700, Sridhar Ratnakumar wrote: >> 2. Create a summary of each thread >> 3. On every month, post the summaries > Wanna work on this? No - which is why I posted this as an idea here. :-) Maybe someone will take that responsibility .. or not. -srid From fuzzyman at gmail.com Tue Aug 4 19:58:29 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Tue, 4 Aug 2009 18:58:29 +0100 Subject: [Python-ideas] Meta idea - monthly digest of python-ideas In-Reply-To: References: Message-ID: <6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com> 2009/8/4 Sridhar Ratnakumar > 1. Create a blog > 2. Create a summary of each thread > 3. On every month, post the summaries in the blog > > The summary of a thread may adhere to a standard template, for instance, > definition of the idea, its merits, and criticisms. It's a *great* idea, but without someone (or preferably plural) to take it on it won't go anywhere. Michael > > > -srid > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristjan at ccpgames.com Tue Aug 4 23:39:37 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Tue, 4 Aug 2009 21:39:37 +0000 Subject: [Python-ideas] adding a "path" component to XMLRPC dispatch objects Message-ID: <930F189C8A437347B80DF2C156F7EC7F098493C5F3@exchis.ccp.ad.local> I've created the following on Rietveld: http://codereview.appspot.com/100046 Here's the descriptions: by passing the "path" component of the xmlrpc request to the dispatch method, it becomes possible to dispatch differently according to this. This patch provides that addition. Additionally, it provides an MultiPathXMLRPCDispatcher mixin class and a MultiPathXMLRPCServer that uses it, to have multiple dispatchers for different paths. This allows a single server port to serve different XMLRPC servers as differentiated by the HTTP path. A test is also preovided. I intend this as an addition to the standard lib. Any thoughts? K -------------- next part -------------- An HTML attachment was scrubbed... URL: From tleeuwenburg at gmail.com Thu Aug 6 03:01:20 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Thu, 6 Aug 2009 11:01:20 +1000 Subject: [Python-ideas] What about allowing '?' in method names? Message-ID: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> this could be nicer than prefixing things with is, and makes things clearer in some cases, e.g. def shouldSomethingHappenFlag(self, context): if 'relevant' in context: return True return False vs def shouldSomethingHappen?(self, context): if 'relevant' in context: return True return False I just think it looks nicer, and is a good hint that a true/false or other kind of flag is likely to come back. Cheers, -T -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at xavierho.com Thu Aug 6 09:50:34 2009 From: contact at xavierho.com (Xavier Ho) Date: Thu, 6 Aug 2009 17:50:34 +1000 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> Message-ID: <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com> On Thu, Aug 6, 2009 at 11:01 AM, Tennessee Leeuwenburg < tleeuwenburg at gmail.com> wrote: > this could be nicer than prefixing things with is, and makes things clearer > in some cases, e.g. > > def shouldSomethingHappenFlag(self, context): > if 'relevant' in context: > return True > > return False > > vs > > def shouldSomethingHappen?(self, context): > if 'relevant' in context: > return True > return False > > I just think it looks nicer, and is a good hint that a true/false or other > kind of flag is likely to come back. > What about hasSomethingHappened(self, context) or isRelevant(self, context) Either way.. I think ? has better uses though. Regards, Ching-Yun "Xavier" Ho, Technical Artist Contact Information Mobile: (+61) 04 3335 4748 Skype ID: SpaXe85 Email: contact at xavierho.com Website: http://xavierho.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From tleeuwenburg at gmail.com Thu Aug 6 10:10:25 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Thu, 6 Aug 2009 18:10:25 +1000 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com> Message-ID: <43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com> On Thu, Aug 6, 2009 at 5:50 PM, Xavier Ho wrote: > On Thu, Aug 6, 2009 at 11:01 AM, Tennessee Leeuwenburg < > tleeuwenburg at gmail.com> wrote: > >> this could be nicer than prefixing things with is, and makes things >> clearer in some cases, e.g. >> >> def shouldSomethingHappenFlag(self, context): >> if 'relevant' in context: >> return True >> >> return False >> >> vs >> >> def shouldSomethingHappen?(self, context): >> if 'relevant' in context: >> return True >> return False >> >> I just think it looks nicer, and is a good hint that a true/false or other >> kind of flag is likely to come back. >> > > What about > > hasSomethingHappened(self, context) > > or > > isRelevant(self, context) > > Either way.. I think ? has better uses though. > Well, probably everyone will want to do things differently. Not all method names work with an 'is' stuck on the front, or a 'Flag' stuck on the end... I'm trying out an idiom of ending True/False returners with 'Q', i.e. self.isSomethingTheCaseQ() or self.shouldSomethingHappenQ() or self.getSomeFlagQ()... I find that's reasonably aesthetically pleasing, and is a good hint about what the method is for. Also, when working with someone else's code, they may already has isFoo() style methods which don't return flags. So Q is good enough for me, but ? would be even better, for me. Cheers, -T -------------- next part -------------- An HTML attachment was scrubbed... URL: From contact at xavierho.com Thu Aug 6 10:30:14 2009 From: contact at xavierho.com (Xavier Ho) Date: Thu, 6 Aug 2009 18:30:14 +1000 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com> <43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com> Message-ID: <2d56febf0908060130q375994ej87bf51f665e6bc27@mail.gmail.com> On Thu, Aug 6, 2009 at 6:10 PM, Tennessee Leeuwenburg < tleeuwenburg at gmail.com> wrote: > Well, probably everyone will want to do things differently. Not all method > names work with an 'is' stuck on the front, or a 'Flag' stuck on the end... > I'm trying out an idiom of ending True/False returners with 'Q', i.e. > self.isSomethingTheCaseQ() or self.shouldSomethingHappenQ() or > self.getSomeFlagQ()... I find that's reasonably aesthetically pleasing, and > is a good hint about what the method is for. > Also, when working with someone else's code, they may already has isFoo() > style methods which don't return flags. > > So Q is good enough for me, but ? would be even better, for me. > > Cheers, > -T > It's cool, I'd like to know what others think about it. Although I don't think many would agree with you, as this is more of a personal style thing. Q looks good, and obvious. Or you could resort to the traditional style where b_ prefix seemed to be loved by many C programmers. Hmm. isXX and hasXXX still work for me well though. Take it easy, -Xav -------------- next part -------------- An HTML attachment was scrubbed... URL: From ben+python at benfinney.id.au Thu Aug 6 11:40:51 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 06 Aug 2009 19:40:51 +1000 Subject: [Python-ideas] What about allowing '?' in method names? References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <2d56febf0908060050g292b76f7y9a3584d38f6254a9@mail.gmail.com> <43c8685c0908060110l730ea810tafd9cffca4235423@mail.gmail.com> Message-ID: <877hxhnuu4.fsf@benfinney.id.au> Tennessee Leeuwenburg writes: > I'm trying out an idiom of ending True/False returners with 'Q', i.e. > self.isSomethingTheCaseQ() or self.shouldSomethingHappenQ() or > self.getSomeFlagQ()... I find that's reasonably aesthetically > pleasing, and is a good hint about what the method is for. If you want it to be more likely to be recognisable to programmers (and I would advise you to conform these names to PEP 8), you could name it as ?foo_p? which is computer-science shortcut for ?foo as a predicate?. class Foo(object): def should_frobnicate_the_widget_p(self): if warble(self): return True return False def content_modified_p(self): if wibble(self): return False return True -- \ ?When I get real bored, I like to drive downtown and get a | `\ great parking spot, then sit in my car and count how many | _o__) people ask me if I'm leaving.? ?Steven Wright | Ben Finney From ncoghlan at gmail.com Thu Aug 6 14:29:31 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 06 Aug 2009 22:29:31 +1000 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: References: Message-ID: <4A7ACCAB.3030005@gmail.com> (moving to python-ideas where this thread belongs) ilya wrote: > and I think this can be done just as easily with existing syntax: > > x = try_1(float, string, except_ = float('nan'), if_ = ValueError) > y = try_2(float, string, { ValueError: float('nan') }) As with PEP 308, the thing missing from function based solutions is lazy evaluation of the operands. They also tend be seriously ugly, as is the case with the above suggestions. Given Guido's reluctance in adding the far more common conditional expressions, I suspect this idea isn't ever going to get anywhere, but even if that is the case it might be nice to have a rejected PEP that makes this explicit. If we were going to do it, I would definitely favour the "EXPR except ALT_EXPR if EXCEPTION" format, but I doubt it is actually going to get that far. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Aug 6 14:32:01 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 06 Aug 2009 22:32:01 +1000 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> Message-ID: <4A7ACD41.6030605@gmail.com> (moving to python-ideas) Dj Gilcrease wrote: > On Thu, Aug 6, 2009 at 4:47 AM, Nick Coghlan wrote: >> Option 2: >> x = float(string) except ValueError: float('nan') >> op(float(string) except ValueError: float('nan')) >> >> This has the virtue of closely matching the statement syntax, but >> embedding colons inside expressions is somewhat ugly. Yes, lambda >> already does it, but lambda can hardly be put forward as a paragon of >> beauty. > > +1 on this option as it resembles the standard try/except block enough > it would be a quick edit to convert it to one if later you realize you > need to catch more exceptions* > > * I recommend NOT allowing multiple exceptions in this form eg > x = float(string)/var except ValueError, ZeroDivisionError, ...: float('nan') > > as it will start to reduce readability quickly All 3 components would just be ordinary expressions. The exception definition would be allowed to resolve to a single exception or a tuple of exceptions, just as it is in a normal try/except statement. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From digitalxero at gmail.com Thu Aug 6 15:21:33 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Thu, 6 Aug 2009 07:21:33 -0600 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: <4A7ACD41.6030605@gmail.com> References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> Message-ID: I figure I would write up the PEP draft, I have never tried writing a pep before, but i did read PEP 1 and tried to follow it's formating guides. If there are no additions to the idea, then it seems there just needs to be a consensus on the syntax before submitting it to the peps list PEP: Title: try-except conditional expressions Version: Last-Modified: Author: Jeff McAninch , Dj Gilcrease Discussions-To: python-ideas at python.org Status: Draft Type: Standards Track Content-Type: text/plain Created: 06-Aug-2009 Python-Version: 2.7/3.2 Post-History: Abstract: I very often want something like a try-except conditional expression similar to the if-else conditional instead of resorting to a multi-line try-except block. Design Goals: The new syntax should * Be simple to read * Be intuitive so people who may use it infrequently dont need to go lookup the format every time * Make it obvious what is happening Modivation: Often when doing calculations or string recasting (to int, float, etc) it is required to wrap the section in a simple try-except where the exception just assigns a default value. It would be more readable and consise if these type of try-excepts could be written on a single line. Issues: Unknown Specification: All 3 components would just be ordinary expressions. The exception definition would be allowed to resolve to a single exception or a tuple of exceptions, just as it is in a normal try/except statement. Syntax Ideas: Option 1: x = float(string) except float('nan') if ValueError op(float(string) except float('nan') if ValueError) Option 2: x = float(string) except ValueError: float('nan') op(float(string) except ValueError: float('nan')) Option 3: x = float(string) except ValueError else float('nan') op(float(string) except ValueError else float('nan')) From steve at pearwood.info Thu Aug 6 17:14:15 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 7 Aug 2009 01:14:15 +1000 Subject: [Python-ideas] [Python-Dev] (try-except) conditional expression similar =?iso-8859-1?q?to=09?=(if-else) conditional (PEP 308) In-Reply-To: References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> Message-ID: <200908070114.15798.steve@pearwood.info> Moved to python-ideas. On Thu, 6 Aug 2009 09:32:16 pm Antoine Pitrou wrote: > I have to say that the original example: > x = float(string) except ValueError else float('nan') > > looks artificial. I don't see how it's adequate behaviour to return a > NaN when presented with a string which doesn't represent a float > number. By memory, the IEEE standard allows for 254 different NANs. Back in the 80s or 90s, SANE (Standard Apple Numerics Environment) supported different NANs for different errors, and I'm pretty sure that one of them was for failed string-to-float conversions. In any case, the idea seems perfectly sensible to me. As for the suggestion, I'm not convinced by the examples given either. I'm not a huge fan of the `x if y else z` construct -- I thought I would be, but in practice I nearly never use it -- and I think this is just more of the same. Seems good in principle, but in practice, I'm not sure it is actually that readable, and there are usually perfectly adequate alternatives. -0 from me. -- Steven D'Aprano From veloso at verylowsodium.com Thu Aug 6 17:16:34 2009 From: veloso at verylowsodium.com (Greg Falcon) Date: Thu, 6 Aug 2009 11:16:34 -0400 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> Message-ID: <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> On Wed, Aug 5, 2009 at 9:01 PM, Tennessee Leeuwenburg wrote: > def shouldSomethingHappen?(self, context): > I just think it looks nicer, and is a good hint that a true/false or other > kind of flag is likely to come back. Two problems I immediately see with this: 1) '?' is punctuation, and uninitiated readers are likely to guess that it is some sort of postfix operator. 2) IPython gives a special meaning to ?, and it would be a shame to collide with that. Greg F From dickinsm at gmail.com Thu Aug 6 17:41:52 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Thu, 6 Aug 2009 16:41:52 +0100 Subject: [Python-ideas] [Python-Dev] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: <200908070114.15798.steve@pearwood.info> References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <200908070114.15798.steve@pearwood.info> Message-ID: <5c6f2a5d0908060841p22de821cj98cc6d8c3eee7741@mail.gmail.com> On Thu, Aug 6, 2009 at 4:14 PM, Steven D'Aprano wrote: > By memory, the IEEE standard allows for 254 different NANs. Did you mean 2**54 here? For IEEE 754 binary64 format, I count 2**53-2 bit patterns representing NaNs: 2**52 quiet NaNs and 2**52-2 signaling NaNs. (The 11 exponent bits must all be 1s; the sign bit and the 52 significand bits can be chosen arbitrarily, except that two bit patterns are reserved for +/-infinity; the top significand bit determines whether you've got a signaling or quiet NaN). I agree that that particular example isn't particularly compelling, but I've definitely encountered other cases where I would have used this construct if it were available. Not that that necessarily makes it a good idea. :-) Mark From lucaspradomelo at gmail.com Thu Aug 6 20:11:44 2009 From: lucaspradomelo at gmail.com (Lucas Prado Melo) Date: Thu, 6 Aug 2009 15:11:44 -0300 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> Message-ID: <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon wrote: > On Wed, Aug 5, 2009 at 9:01 PM, Tennessee > Leeuwenburg wrote: > > def shouldSomethingHappen?(self, context): > > > I just think it looks nicer, and is a good hint that a true/false or > other > > kind of flag is likely to come back. > > Two problems I immediately see with this: > > 1) '?' is punctuation, and uninitiated readers are likely to guess > that it is some sort of postfix operator. Actually, Ruby allows this notation and, until now, I have not seen many complains about it. Despite that, its use would be pretty clear for the beginner, since it would be associated with meaningful identifiers: oil_tank.isItFull?() The worst problem could be the useless parenthesis after the '?' sign. > 2) IPython gives a special meaning to ?, and it would be a shame to > collide with that. IMHO, IPython might not be mature enough to interfere with Python design decisions yet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Aug 6 20:57:25 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 6 Aug 2009 11:57:25 -0700 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> Message-ID: Adding '?' to identifiers in Python ain't gonna happen. On Thu, Aug 6, 2009 at 11:11 AM, Lucas Prado Melo wrote: > On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon > wrote: >> >> On Wed, Aug 5, 2009 at 9:01 PM, Tennessee >> Leeuwenburg wrote: >> > def shouldSomethingHappen?(self, context): >> >> > I just think it looks nicer, and is a good hint that a true/false or >> > other >> > kind of flag is likely to come back. >> >> Two problems I immediately see with this: >> >> 1) '?' is punctuation, and uninitiated readers are likely to guess >> that it is some sort of postfix operator. > > Actually, Ruby allows this notation and, until now, I have not seen many > complains about it. > Despite that, its use would be pretty clear for the beginner, since it would > be associated with meaningful identifiers: > > oil_tank.isItFull?() > > The worst problem could be the useless parenthesis after the '?' sign. > >> >> 2) IPython gives a special meaning to ?, and it would be a shame to >> collide with that. > > ?IMHO, IPython might not be mature enough to interfere with Python design > decisions yet. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jnoller at gmail.com Thu Aug 6 23:07:28 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 6 Aug 2009 17:07:28 -0400 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> Message-ID: <4222a8490908061407r2975f565tfebfce38acbf7d6f@mail.gmail.com> +100 On Thu, Aug 6, 2009 at 2:57 PM, Guido van Rossum wrote: > Adding '?' to identifiers in Python ain't gonna happen. > > On Thu, Aug 6, 2009 at 11:11 AM, Lucas Prado > Melo wrote: >> On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon >> wrote: >>> >>> On Wed, Aug 5, 2009 at 9:01 PM, Tennessee >>> Leeuwenburg wrote: >>> > def shouldSomethingHappen?(self, context): >>> >>> > I just think it looks nicer, and is a good hint that a true/false or >>> > other >>> > kind of flag is likely to come back. >>> >>> Two problems I immediately see with this: >>> >>> 1) '?' is punctuation, and uninitiated readers are likely to guess >>> that it is some sort of postfix operator. >> >> Actually, Ruby allows this notation and, until now, I have not seen many >> complains about it. >> Despite that, its use would be pretty clear for the beginner, since it would >> be associated with meaningful identifiers: >> >> oil_tank.isItFull?() >> >> The worst problem could be the useless parenthesis after the '?' sign. >> >>> >>> 2) IPython gives a special meaning to ?, and it would be a shame to >>> collide with that. >> >> ?IMHO, IPython might not be mature enough to interfere with Python design >> decisions yet. >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From zuo at chopin.edu.pl Fri Aug 7 02:15:56 2009 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Fri, 07 Aug 2009 02:15:56 +0200 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> Message-ID: > Syntax Ideas: > Option 1: > x = float(string) except float('nan') if ValueError > op(float(string) except float('nan') if ValueError) -0.5 I wouldn't mix up 'if' word with 'except' -- there are different concepts. > Option 2: > x = float(string) except ValueError: float('nan') > op(float(string) except ValueError: float('nan')) +0.5 IMHO seems to be the best of the propositions. I'd add to it optional as (like in present 'try:... except Exc as exc:...') > Option 3: > x = float(string) except ValueError else float('nan') > op(float(string) except ValueError else float('nan')) -1 IMHO it's illogical. 'else' has completely opposite role in present 'try...except...' clauses. It'd be terribly missleading, not only for newbies. Cheers, *j -- Jan Kaliszewski (zuo) From jkwon.work at gmail.com Fri Aug 7 02:28:23 2009 From: jkwon.work at gmail.com (Jae Kwon) Date: Thu, 6 Aug 2009 17:28:23 -0700 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> Message-ID: <19461FAF-6637-4769-9415-68CA60BDE059@gmail.com> On Aug 6, 2009, at 5:15 PM, Jan Kaliszewski wrote: >> Syntax Ideas: >> Option 1: >> x = float(string) except float('nan') if ValueError >> op(float(string) except float('nan') if ValueError) > > -0.5 > I wouldn't mix up 'if' word with 'except' -- there are different > concepts. > >> Option 2: >> x = float(string) except ValueError: float('nan') >> op(float(string) except ValueError: float('nan')) > > +0.5 > IMHO seems to be the best of the propositions. I'd add to it optional > as (like in present 'try:... except Exc as exc:...') > +0.5 the colon implies EOL. how about 'then'? x = float(string) except ValueError [, e] then float('nan') [, e] optional - Jae >> Option 3: >> x = float(string) except ValueError else float('nan') >> op(float(string) except ValueError else float('nan')) > > -1 > IMHO it's illogical. 'else' has completely opposite role in present > 'try...except...' clauses. It'd be terribly missleading, not only > for newbies. > > > Cheers, > *j > > -- > Jan Kaliszewski (zuo) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From python at mrabarnett.plus.com Fri Aug 7 02:41:08 2009 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 07 Aug 2009 01:41:08 +0100 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> Message-ID: <4A7B7824.2060504@mrabarnett.plus.com> Jan Kaliszewski wrote: >> Syntax Ideas: >> Option 1: >> x = float(string) except float('nan') if ValueError >> op(float(string) except float('nan') if ValueError) > > -0.5 > I wouldn't mix up 'if' word with 'except' -- there are different concepts. > >> Option 2: >> x = float(string) except ValueError: float('nan') >> op(float(string) except ValueError: float('nan')) > > +0.5 > IMHO seems to be the best of the propositions. I'd add to it optional > as (like in present 'try:... except Exc as exc:...') > Can there be multiple 'except's? reciprocal = 1.0 / float(string) except ValueError: float('nan') except ZeroDivisionError: float('inf') (We can already chain '==', etc.) >> Option 3: >> x = float(string) except ValueError else float('nan') >> op(float(string) except ValueError else float('nan')) > > -1 > IMHO it's illogical. 'else' has completely opposite role in present > 'try...except...' clauses. It'd be terribly missleading, not only > for newbies. > From ben+python at benfinney.id.au Fri Aug 7 03:39:50 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 07 Aug 2009 11:39:50 +1000 Subject: [Python-ideas] What about allowing '?' in method names? References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> Message-ID: <87bpmsmmft.fsf@benfinney.id.au> Guido van Rossum writes: > Adding '?' to identifiers in Python ain't gonna happen. Thank you. (Both for making the decision clearly, and the outcome of that decision.) -- \ ?Earth gets its price for what Earth gives us.? ?James Russell | `\ Lowell | _o__) | Ben Finney From stephen at xemacs.org Fri Aug 7 05:07:09 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 07 Aug 2009 12:07:09 +0900 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> Message-ID: <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> Lucas Prado Melo writes: > On Thu, Aug 6, 2009 at 12:16 PM, Greg Falcon wrote: > > > On Wed, Aug 5, 2009 at 9:01 PM, Tennessee > > Leeuwenburg wrote: > > > def shouldSomethingHappen?(self, context): > Actually, Ruby allows this notation and, until now, I have not seen many > complains about it. So does Lisp. But those languages simply allow ? as part of an identifier. Python, for one thing, envisions syntax extensions. Sometimes these are done with keywords, of course, but sometimes it makes sense to use an operator (as with decorators' use of "@"). Reserving all punctuation for this purpose is a reasonable heuristic. I suspect that the BDFL pronounced simply on the basis that there are plenty of alternatives (trailing "Q" as Tennessee suggested, trailing "_p" per Ben) so the benefit is minute, while the costs would be substantial. > > 2) IPython gives a special meaning to ?, and it would be a shame to > > collide with that. > > IMHO, IPython might not be mature enough to interfere with Python design > decisions yet. It already has, IIRC. ISTR IPython was a consideration, though not a showstopper, in the choice of "@" for decorators. From lucaspradomelo at gmail.com Fri Aug 7 05:07:13 2009 From: lucaspradomelo at gmail.com (Lucas Prado Melo) Date: Fri, 7 Aug 2009 00:07:13 -0300 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <6217e0a0908062007x21d0a2afu72fe0c5747778e02@mail.gmail.com> On Fri, Aug 7, 2009 at 12:07 AM, Stephen J. Turnbull wrote: > I suspect that the BDFL pronounced simply on the basis that there are > plenty of alternatives (trailing "Q" as Tennessee suggested, trailing > "_p" per Ben) so the benefit is minute, while the costs would be > substantial. I guess so. I really didn't like the interrogation mark followed by parenthesis anyway. > > > 2) IPython gives a special meaning to ?, and it would be a shame to > > > collide with that. > > > > IMHO, IPython might not be mature enough to interfere with Python > design > > decisions yet. > > It already has, IIRC. ISTR IPython was a consideration, though not a > showstopper, in the choice of "@" for decorators. > > Oh. Nevermind, then. I thought it wasn't mature enough because the current stable version has such a low value as 0.10. -------------- next part -------------- An HTML attachment was scrubbed... URL: From debatem1 at gmail.com Fri Aug 7 04:57:53 2009 From: debatem1 at gmail.com (CTO) Date: Thu, 6 Aug 2009 19:57:53 -0700 (PDT) Subject: [Python-ideas] Meta idea - monthly digest of python-ideas In-Reply-To: <6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com> References: <6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com> Message-ID: On Aug 4, 1:58?pm, Michael Foord wrote: > 2009/8/4 Sridhar Ratnakumar > > > 1. Create a blog > > 2. Create a summary of each thread > > 3. On every month, post the summaries in the blog > > > The summary of a thread may adhere to a standard template, for instance, > > definition of the idea, its merits, and criticisms. > > It's a *great* idea, but without someone (or preferably plural) to take it > on it won't go anywhere. > > Michael I'm happy to contribute. Where do we want the site? Geremy Condra From g.brandl at gmx.net Fri Aug 7 08:26:47 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 07 Aug 2009 08:26:47 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <6217e0a0908062007x21d0a2afu72fe0c5747778e02@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> <6217e0a0908062007x21d0a2afu72fe0c5747778e02@mail.gmail.com> Message-ID: Lucas Prado Melo schrieb: > > > 2) IPython gives a special meaning to ?, and it would be a shame to > > > collide with that. > > > > IMHO, IPython might not be mature enough to interfere with > Python design > > decisions yet. > > It already has, IIRC. ISTR IPython was a consideration, though not a > showstopper, in the choice of "@" for decorators. > > Oh. Nevermind, then. I thought it wasn't mature enough because the > current stable version has such a low value as 0.10. Don't judge open source projects' maturity by their version number. For some reason, authors have a tendency to be over-modest in switching to something post-0.x, even if the project is stable and well-used for years. It's probably just a reaction to the orthogonal practice of some corporate software vendors :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From g.brandl at gmx.net Fri Aug 7 08:30:14 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 07 Aug 2009 08:30:14 +0200 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: <4A7B7824.2060504@mrabarnett.plus.com> References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> <4A7B7824.2060504@mrabarnett.plus.com> Message-ID: MRAB schrieb: > Jan Kaliszewski wrote: >>> Syntax Ideas: >>> Option 1: >>> x = float(string) except float('nan') if ValueError >>> op(float(string) except float('nan') if ValueError) >> >> -0.5 >> I wouldn't mix up 'if' word with 'except' -- there are different concepts. >> >>> Option 2: >>> x = float(string) except ValueError: float('nan') >>> op(float(string) except ValueError: float('nan')) >> >> +0.5 >> IMHO seems to be the best of the propositions. I'd add to it optional >> as (like in present 'try:... except Exc as exc:...') >> > Can there be multiple 'except's? > > reciprocal = 1.0 / float(string) except ValueError: float('nan') > except ZeroDivisionError: float('inf') Don't over-complicate. This proposal has a slim chance of success as it is; piling on top isn't going to help. IMO, any syntax that puts something other than the exception class after "except" has zero chance. The "except-then" is the most attractive to me, but still doesn't read as naturally as "if-else". Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From mwm-keyword-python.b4bdba at mired.org Fri Aug 7 09:21:20 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Fri, 7 Aug 2009 03:21:20 -0400 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> <4A7B7824.2060504@mrabarnett.plus.com> Message-ID: <20090807032120.37476799@bhuda.mired.org> On Fri, 07 Aug 2009 08:30:14 +0200 Georg Brandl wrote: > MRAB schrieb: > > Jan Kaliszewski wrote: > >>> Syntax Ideas: > >>> Option 1: > >>> x = float(string) except float('nan') if ValueError > >>> op(float(string) except float('nan') if ValueError) > >> > >> -0.5 > >> I wouldn't mix up 'if' word with 'except' -- there are different concepts. > >> > >>> Option 2: > >>> x = float(string) except ValueError: float('nan') > >>> op(float(string) except ValueError: float('nan')) > >> > >> +0.5 > >> IMHO seems to be the best of the propositions. I'd add to it optional > >> as (like in present 'try:... except Exc as exc:...') > >> > > Can there be multiple 'except's? > > > > reciprocal = 1.0 / float(string) except ValueError: float('nan') > > except ZeroDivisionError: float('inf') > > Don't over-complicate. This proposal has a slim chance of success as it > is; piling on top isn't going to help. Assuming you avoid the ':' version, shouldn't multiple excepts fall out of the grammar naturally? x = ((1.0 / float(string) except float('nan') if ValueError) except float('inf') if ZeroDivisionError) parses as "expr1 except float('inf') if ValueError", where expr1 is "(1.0 / float(string) except float('nan') if ValueError)". Yes, it's as ugly as nesting if/else expressions, but those work, so these should. FWIW, I'm +.5 on this. I don't see a lot of use for it, but can see that where it is useful, it's easier to read than a try:/except:. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From jkwon.work at gmail.com Fri Aug 7 10:03:02 2009 From: jkwon.work at gmail.com (Jae Kwon) Date: Fri, 7 Aug 2009 01:03:02 -0700 Subject: [Python-ideas] (try-except) conditional expression similar to (if-else) conditional (PEP 308) In-Reply-To: <20090807032120.37476799@bhuda.mired.org> References: <4A7A0626.5050303@lanl.gov> <592033416A4F4C20A60ACB28E40F84DF@RaymondLaptop1> <20090806012101.B42493A406B@sparrow.telecommunity.com> <4A7AB4D1.6070501@gmail.com> <4A7ACD41.6030605@gmail.com> <4A7B7824.2060504@mrabarnett.plus.com> <20090807032120.37476799@bhuda.mired.org> Message-ID: <1BF02135-CF7D-4541-83F4-7093131704B2@gmail.com> How about making this work? foo = try: float(string) except SomethingError, e: float(NAN) This would imply that try blocks have a value. Maybe even allow this: foo = try: float(string) except Something Error, e: float(NAN) Sure, it introduces a couple new grammar tweaks, but it's easy to understand and is backwards compatible. - Jae On Aug 7, 2009, at 12:21 AM, Mike Meyer wrote: > On Fri, 07 Aug 2009 08:30:14 +0200 > Georg Brandl wrote: > >> MRAB schrieb: >>> Jan Kaliszewski wrote: >>>>> Syntax Ideas: >>>>> Option 1: >>>>> x = float(string) except float('nan') if ValueError >>>>> op(float(string) except float('nan') if ValueError) >>>> >>>> -0.5 >>>> I wouldn't mix up 'if' word with 'except' -- there are different >>>> concepts. >>>> >>>>> Option 2: >>>>> x = float(string) except ValueError: float('nan') >>>>> op(float(string) except ValueError: float('nan')) >>>> >>>> +0.5 >>>> IMHO seems to be the best of the propositions. I'd add to it >>>> optional >>>> as (like in present 'try:... except Exc as exc:...') >>>> >>> Can there be multiple 'except's? >>> >>> reciprocal = 1.0 / float(string) except ValueError: float('nan') >>> except ZeroDivisionError: float('inf') >> >> Don't over-complicate. This proposal has a slim chance of success >> as it >> is; piling on top isn't going to help. > > Assuming you avoid the ':' version, shouldn't multiple excepts fall > out of the grammar naturally? > > x = ((1.0 / float(string) > except float('nan') if ValueError) > except float('inf') if ZeroDivisionError) > > parses as "expr1 except float('inf') if ValueError", where expr1 is > "(1.0 / float(string) except float('nan') if ValueError)". > > Yes, it's as ugly as nesting if/else expressions, but those work, so > these should. > > FWIW, I'm +.5 on this. I don't see a lot of use for it, but can see > that where it is useful, it's easier to read than a try:/except:. > > -- > Mike Meyer http://www.mired.org/consulting.html > Independent Network/Unix/Perforce consultant, email for more > information. > > O< ascii ribbon campaign - stop html mail - www.asciiribbon.org > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From greg.ewing at canterbury.ac.nz Fri Aug 7 11:10:24 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 07 Aug 2009 21:10:24 +1200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4A7BEF80.1090505@canterbury.ac.nz> Stephen J. Turnbull wrote: > So does Lisp. But those languages simply allow ? as part of an identifier. Lisp allows almost *anything* as part of an identifier. Python doesn't, and I like it that way. Making exceptions for just a few characters would make the rules for what is allowed in an identifier more complicated and harder to remember, for little benefit that I can see. -- Greg From ede at mit.edu Fri Aug 7 12:04:47 2009 From: ede at mit.edu (Eric Eisner) Date: Fri, 7 Aug 2009 19:04:47 +0900 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> Message-ID: <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> Hello, As previously mentioned on python-ideas [1] (circa 2006), it would make sense to be able to perform bitwise operations on bytes/bytearray. Stealing the example from the original suggestion: Suppose I have a string (say, read in from a binary file) that has a 4-byte field in it. Bit 11 indicates whether the accompanying data is in glorped form (something I'm just making up for this example). For example, if the field has 0x000480c4, the data is not glorped. If the data is glorped, bit 11 would be on, i.e., 0x001480c4. Let's say I want to turn on the glorp bit; what I have to do now: GLORPED = 0x10 newchar = flags[1] | GLORPED flags = flags[0] + newchar + flags[2:] What I'd like to be able to do is something like: GLORPED = b"\x00\x10\x00\x00" flags |= GLORPED # test if the glorped bit is on any(flags & GLORPED) I have run into this a few times, at least when reading/writing binary formats etc. This approach is more intuitive than the typical/archaic way of converting to an integer, performing a bitwise operation on the integer, converting back to bytes. Arguably, bitwise operations on a high-level integer type don't make sense, as base 2 is an implementation detail. At the very least, bytes and bytearray should be usable with the ~ ^ | & operators Example behavior: >>> ~b'\x55\xff' b'\xaa\x00' >>> b'\xf5\x60' & b'\xa9\x3c' b'\xa1\x20' Unresolved problems: If the two arguments are of different length, either it could either raise a ValueError or mimic the behavior of ints. Xoring an int to a byte seems less than well defined in general, due to endianness ambiguity of the int and size ambiguity. I would think this should not be allowed. Also conceivable is using the shift operators >> and << on bytes, but I personally would use that less often, and the result of such an operation is ambiguous due to endianness. -Eric Eisner [1] http://mail.python.org/pipermail/python-ideas/2006-December/000001.html From taleinat at gmail.com Fri Aug 7 13:43:54 2009 From: taleinat at gmail.com (Tal Einat) Date: Fri, 7 Aug 2009 14:43:54 +0300 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> Message-ID: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> Eric Eisner wrote: > Hello, > > As previously mentioned on python-ideas [1] (circa 2006), it would > make sense to be able to perform bitwise operations on > bytes/bytearray. > > Stealing the example from the original suggestion: > > Suppose I have a string (say, read in from a binary file) that has a > 4-byte field in it. Bit 11 indicates whether the accompanying data is > in glorped form (something I'm just making up for this example). > > For example, if the field has 0x000480c4, the data is not glorped. If > the data is glorped, bit 11 would be on, i.e., 0x001480c4. > > Let's say I want to turn on the glorp bit; what I have to do now: > > GLORPED = 0x10 > newchar = flags[1] | GLORPED > flags = flags[0] + newchar + flags[2:] > > What I'd like to be able to do is something like: > > GLORPED = b"\x00\x10\x00\x00" > flags |= GLORPED You can already do bitwise operations on bytes, and retrieving the relevant byte and using single-byte operations on it is simple enough. As I see it, the major underlying problem here is that byte-arrays are immutable, since what you really want is to be able to change a single byte of the array in-place. In general, Python's byte arrays and strings are very ill suited for dealing with data which isn't byte-based, and definitely horrible at building or modifying data streams. I recommend using an external library for working with data structures and/or data streams. Construct[1] is my personal favorite, since it's especially Pythonic and easy to use. In any case, if you wish to propose a mutable byte/bit array which supports array-wise binary operators, I can say I would certainly be glad to have such a class at my disposal. - Tal [1] http://construct.wikispaces.com/ From phd at phd.pp.ru Fri Aug 7 13:51:08 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 7 Aug 2009 15:51:08 +0400 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> Message-ID: <20090807115108.GA32694@phd.pp.ru> On Fri, Aug 07, 2009 at 02:43:54PM +0300, Tal Einat wrote: > In any case, if you wish to propose a mutable byte/bit array which > supports array-wise binary operators, I can say I would certainly be > glad to have such a class at my disposal. Like this http://pypi.python.org/pypi/BitVector/ ? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From ede at mit.edu Fri Aug 7 13:57:54 2009 From: ede at mit.edu (Eric Eisner) Date: Fri, 7 Aug 2009 20:57:54 +0900 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <20090807115108.GA32694@phd.pp.ru> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> <20090807115108.GA32694@phd.pp.ru> Message-ID: <42ef4ee0908070457j32ff805fy3a4ef6dd350e9c39@mail.gmail.com> On Fri, Aug 7, 2009 at 20:51, Oleg Broytmann wrote: > On Fri, Aug 07, 2009 at 02:43:54PM +0300, Tal Einat wrote: >> In any case, if you wish to propose a mutable byte/bit array which >> supports array-wise binary operators, I can say I would certainly be >> glad to have such a class at my disposal. > > ? Like this http://pypi.python.org/pypi/BitVector/ ? BitVector does seem to nicely handle every conceivable usecase I had in mind, but I think some of that functionality would be useful to have available in the standard implementation of bytes. -Eric From ncoghlan at gmail.com Fri Aug 7 14:24:01 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 07 Aug 2009 22:24:01 +1000 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> Message-ID: <4A7C1CE1.4080307@gmail.com> Tal Einat wrote: > You can already do bitwise operations on bytes, and retrieving the > relevant byte and using single-byte operations on it is simple enough. > As I see it, the major underlying problem here is that byte-arrays are > immutable, since what you really want is to be able to change a single > byte of the array in-place. In general, Python's byte arrays and > strings are very ill suited for dealing with data which isn't > byte-based, and definitely horrible at building or modifying data > streams. Careful with your terminology here. In Python 3.x, the following two types both exist: bytes: immutable sequence of integers between 0 and 255 inclusive bytearray: similar to bytes, but mutable The operations ~x, x&y, x|y and x^y make sense for both bytes and bytearray. The augmented assignment operators only make sense for bytearray. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ilya.nikokoshev at gmail.com Fri Aug 7 14:46:40 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Fri, 7 Aug 2009 16:46:40 +0400 Subject: [Python-ideas] pep 312 - implicit lambdas idea Message-ID: I was thinking about a good syntax for implicit lambdas for a while and today I had this idea: make ``_:`` a shortcut for ``lambda _=None:`` For example: map( _: _ + 5, some_list) register_callback( _: True) def apply_transform(..., transform = _:_, ... ): but still addition = lamba x, y: x + y The rationale is that you only want to get rid of lambda keyword to create a *very* simple function, the one that will be called either without parameters or with only one parameter. For everything more complicated, you really should go and write the explicit function signature using lambda. Even though ``:`` could theoretically denote implicit lambda, it's too easy to miss it. The combination ``_:`` is much easier to notice. It also makes explicit that there is at most one parameter and it's name is ``_``. Since it's very short, it can easily be used in a long function call or as a default parameter, as above Your thoughts? From gerald.britton at gmail.com Fri Aug 7 14:58:28 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Fri, 7 Aug 2009 08:58:28 -0400 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: Message-ID: <5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com> The underscore is already a viable identifier: >>> _ = lambda x:x*2 >>> _(3) 6 and >>> x = lambda _:_*2 >>> x(3) 6 >>> So I don't think you can use it the way you're proposing without breaking existing programs On Fri, Aug 7, 2009 at 8:46 AM, ilya wrote: > I was thinking about a good syntax for implicit lambdas for a while > and today I had this idea: make ``_:`` a shortcut for ``lambda > _=None:`` > > For example: > > map( _: _ + 5, some_list) > register_callback( _: True) > def apply_transform(..., transform = _:_, ... ): > > but still > > addition = lamba x, y: x + y > > The rationale is that you only want to get rid of lambda keyword to > create a *very* simple function, the one that will be called either > without parameters or with only one parameter. For everything more > complicated, you really should go and write the explicit function > signature using lambda. > > Even though ``:`` could theoretically denote implicit lambda, it's too > easy to miss it. The combination ``_:`` is much easier to notice. It > also makes explicit that there is at most one parameter and it's name > is ``_``. Since it's very short, it can easily be used in a long > function call or as a default parameter, as above > > Your thoughts? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilya.nikokoshev at gmail.com Fri Aug 7 15:08:38 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Fri, 7 Aug 2009 17:08:38 +0400 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com> References: <5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com> Message-ID: I'd like to use this syntax only in situations of PEP 312, that is, where a colon is prohibited by current grammar. In those situations, writing _: is also prohibited by current grammar so it will be parsed unambiguously. The examples you quoted will not change under my idea, but ``x = (_:_*2)`` would be possible as an alternative. Note that I would be against being able to use ``x = _:_*2``. I think it's better to require either lambda keyword or parentheses for the RHS of the assignment as colon has a well-defined meaning as starting an indented block unless it's inside some brackets. On Fri, Aug 7, 2009 at 4:58 PM, Gerald Britton wrote: > The underscore is already a viable identifier: >>>> _ = lambda x:x*2 >>>> _(3) > 6 > and >>>> x = lambda _:_*2 >>>> x(3) > 6 >>>> > > So I don't think you can use it the way you're proposing without breaking > existing programs > > On Fri, Aug 7, 2009 at 8:46 AM, ilya wrote: >> >> I was thinking about a good syntax for implicit lambdas for a while >> and today I had this idea: make ``_:`` a shortcut for ``lambda >> _=None:`` >> >> For example: >> >> ? ?map( _: _ + 5, some_list) >> ? ?register_callback( _: True) >> ? ?def apply_transform(..., transform = _:_, ... ): >> >> but still >> >> ? ?addition = lamba x, y: x + y >> >> The rationale is that you only want to get rid of lambda keyword to >> create a *very* simple function, the one that will be called either >> without parameters or with only one parameter. For everything more >> complicated, you really should go and write the explicit function >> signature using lambda. >> >> Even though ``:`` could theoretically denote implicit lambda, it's too >> easy to miss it. The combination ``_:`` is much easier to notice. It >> also makes explicit that there is at most one parameter and it's name >> is ``_``. Since it's very short, it can easily be used in a long >> function call or as a default parameter, as above >> >> Your thoughts? >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas > > > > -- > Gerald Britton > From dickinsm at gmail.com Fri Aug 7 15:19:20 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 7 Aug 2009 14:19:20 +0100 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> Message-ID: <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> On Fri, Aug 7, 2009 at 11:04 AM, Eric Eisner wrote: > Hello, > > As previously mentioned on python-ideas [1] (circa 2006), it would > make sense to be able to perform bitwise operations on > bytes/bytearray. +1 from me. I'd say that it makes just as much sense to do bit operations on bytes or bytearrays as it does on integers. I've always felt a little bit dirty when I've abused ints as bitstrings in the past; byte strings seem better suited to this task, especially where input and output are also involved. > Unresolved problems: > If the two arguments are of different length, either it could either > raise a ValueError or mimic the behavior of ints. I'd say ValueError, since it's not really clear what 'mimic the behaviour of ints' means here. E.g., should bytes([1,2,3,4]) | bytes([16, 32, 64]) be equal to bytes([17, 34, 67, 64]) or bytes([1, 18, 35, 68]). It seems better to restrict the use to the cases where there's a single obvious interpretation. > Xoring an int to a byte seems less than well defined in general, due > to endianness ambiguity of the int and size ambiguity. I would think > this should not be allowed. Agreed. > Also conceivable is using the shift operators >> and << on bytes, but > I personally would use that less often, and the result of such an > operation is ambiguous due to endianness. Agreed. To make sense of the shift operators you effectively have to give 'position' interpretations for the individual bits, and there's no single obvious way of doing this; for the plain bitwise operations this isn't necessary. One other question: if a has type bytes and b has type bytearray, what types would you propose that a & b and b & a should have? Mark From masklinn at masklinn.net Fri Aug 7 15:20:19 2009 From: masklinn at masklinn.net (Masklinn) Date: Fri, 7 Aug 2009 15:20:19 +0200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: <5d1a32000908070558q37068830yb8a37f958b5219b0@mail.gmail.com> Message-ID: <2E804908-D98C-4935-8EC2-B07CF10EF1EB@masklinn.net> On 7 Aug 2009, at 15:08 , ilya wrote: > I'd like to use this syntax only in situations of PEP 312, that is, > where a colon is prohibited by current grammar. In those situations, > writing _: is also prohibited by current grammar so it will be parsed > unambiguously. > > The examples you quoted will not change under my idea, but ``x = > (_:_*2)`` would be possible as an alternative. Note that I would be > against being able to use ``x = _:_*2``. I think it's better to > require either lambda keyword or parentheses for the RHS of the > assignment as colon has a well-defined meaning as starting an indented > block unless it's inside some brackets. It's not much of a gain though, some kind of currying for operators might be a more general idea for those cases. Or simply a better lambda syntax (Haskell has a pretty cool one for what it's worth, but it might conflict with the EOL escape thing), but as far as I know previous discussions on the subject have either failed to come up with a viable syntax or failed to get the approval of the community or BDFL. From ede at mit.edu Fri Aug 7 15:31:27 2009 From: ede at mit.edu (Eric Eisner) Date: Fri, 7 Aug 2009 22:31:27 +0900 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> Message-ID: <42ef4ee0908070631o3f751ff0t59e31ea503dfbbff@mail.gmail.com> > One other question: ?if a has type bytes and b has type bytearray, > what types would you propose that a & b and b & a should have? For consistency they should probably do what they currently do with concatenation: the type of the first expression is the type of the result: >>> b'a' + bytearray(b'b') b'ab' >>> bytearray(b'b') + b'a' bytearray(b'ba') -Eric From ben+python at benfinney.id.au Fri Aug 7 16:06:59 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Sat, 08 Aug 2009 00:06:59 +1000 Subject: [Python-ideas] pep 312 - implicit lambdas idea References: Message-ID: <87d477lnuk.fsf@benfinney.id.au> ilya writes: > I was thinking about a good syntax for implicit lambdas for a while Why were you thinking of this, and why is it important enough to override ?explicit is better than implicit?? > and today I had this idea: make ``_:`` a shortcut for ``lambda > _=None:`` -1. The name ?_? already sees a lot of use as an identifier. It is in common use as a conventional shortcut to the ?gettext.gettext? function. It is also commonly used as a name to bind to a value that is returned from some process but not actually needed. Overloading existing conventions of that valid identifier with this special construct is too much, in my view. > Even though ``:`` could theoretically denote implicit lambda, it's too > easy to miss it. The combination ``_:`` is much easier to notice. I disagree; it is too easy to miss. -- \ ?Last year I went fishing with Salvador Dali. He was using a | `\ dotted line. He caught every other fish.? ?Steven Wright | _o__) | Ben Finney From debatem1 at gmail.com Fri Aug 7 16:46:13 2009 From: debatem1 at gmail.com (CTO) Date: Fri, 7 Aug 2009 07:46:13 -0700 (PDT) Subject: [Python-ideas] Meta idea - monthly digest of python-ideas In-Reply-To: References: <6f4025010908041058t5bd5fb87h22c2d71b1b72c3bd@mail.gmail.com> Message-ID: <74efd982-7ec4-48c2-a02b-b72faf214915@r24g2000vbn.googlegroups.com> On Aug 6, 10:57?pm, CTO wrote: > On Aug 4, 1:58?pm, Michael Foord wrote: > > > 2009/8/4 Sridhar Ratnakumar > > > > 1. Create a blog > > > 2. Create a summary of each thread > > > 3. On every month, post the summaries in the blog > > > > The summary of a thread may adhere to a standard template, for instance, > > > definition of the idea, its merits, and criticisms. > > > It's a *great* idea, but without someone (or preferably plural) to take it > > on it won't go anywhere. > > > Michael Went ahead and created a temporary site for us over at squarespace. If we need to move it it shouldn't be hard. Will need at least one other contributor to make this work, though. Geremy Condra From steve at pearwood.info Sat Aug 8 02:21:15 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Aug 2009 10:21:15 +1000 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: Message-ID: <200908081021.15580.steve@pearwood.info> On Fri, 7 Aug 2009 10:46:40 pm ilya wrote: > I was thinking about a good syntax for implicit lambdas for a while > and today I had this idea: make ``_:`` a shortcut for ``lambda > _=None:`` [...] > The rationale is that you only want to get rid of lambda keyword to > create a *very* simple function, the one that will be called either > without parameters or with only one parameter. For everything more > complicated, you really should go and write the explicit function > signature using lambda. Why would you want to get rid of the lambda keyword? What's the benefit? Is this about saving twelve keystrokes? lambda _=None: versus _: Just how often do you want, or need, to write such a lambda? It seems to me that not only is it a special case you want to break the rules for, which goes against the Zen, but it's an incredibly rare special case. _ as an identifier already has three conventional meanings: (1) In the interactive interpreter, _ is the value of the last expression typed. (2) It is commonly used to mean "I don't care about this value", e.g. t = ("Fred Smith", "123 Fourth Street", "New York", "dog") name, _, _, pet = t (3) It is also often used in internationalization. You want to give it the extra meaning "a default parameter name for lambda when I can't be bothered typing even a single letter name". Because _ is already a valid identifier, this will break code that does this: while _: process() _ = function() if _: print "something" Not the best choice of names, but somebody, somewhere, is doing that, and your suggestion will break their code. Looking at the three examples you gave: map( _: _ + 5, some_list) register_callback( _: True) def apply_transform(..., transform = _:_, ... ): In the first case, I wouldn't use the short-cut form even if it were available. I'd write a lambda that used a more meaningful name. In this case, I'm expecting an int, so I would use n, or a float, so I'd use x. I'd also avoid setting the pointless default: map(lambda x: x+5, some_list) vs map(_: _+5, some_list) Since your suggestion doesn't do precisely what I want, the only reason I would have for using your construct is to save seven keystrokes. Encouraging laziness on the behalf of the programmer is not a good reason for special-casing rare cases. Second case: register_callback( _: True) I assume you're implying that the callback function must take a single argument. In this example, using _ as the parameter name to the lambda makes sense, because it is a "don't care" argument. But if the callback function is documented as always being given a single argument, I would want to know if it was being called without any arguments, so the default value of None is inappropriate and I would avoid using it. Third case: def apply_transform(..., transform = _:_, ... ): I don't think I'd write a function called apply_transform() which made the transformation function optional, let alone buried deep in the middle of a whole lot of extra parameters. (I presume that's what the "..."s are meant to imply.) But putting that aside, I see your intention: a default do-nothing function which appears in a very long parameter list. The problem is that instead of trying to shrink the default value so you can fit all the parameters on a single line, you should make such a complicated function signature more readable by spreading it out: def apply_transform( obj, start, end, # start (inc) and end (exc) positions to apply another_arg, # does something very important I'm sure x=0, y=1, z=2, # more very important arguments transform=( # default null transformation lambda obj=None: obj), frotz=False, # if true, frotz the hymangirator with spangule hymangirator=None, spangule=None, magic=12345, # this needs no documentation namespace={}, flibbertigibbet=None, _private_magic=[] # the caller shouldn't supply this ): (Even better is to avoid such complicated function signatures, but sometimes that's not an option.) So again I'd be very unlikely to use your suggested construct except out of simple can't-be-bothered-to-type-a-dozen-letters laziness. Pandering to that sort of laziness is probably not a good thing. Fundamentally, this suggestion doesn't add expressability to the language, or power. Laziness on it's own is not a good reason for special casing rare cases. If it was a very common case, then *perhaps* you would have an argument for special casing needless verbiage: conciseness (up to a point) is a virtue in a language. That's partly why we have lambdas in the first place, so we can write this: reduce(lambda a,b: (a+b)/2.0, somelist) instead of this: def average(a, b): return (a+b)/2.0 reduce(average, somelist) But this isn't a common case, it's a rare case, and the case you're hoping to replace is pretty concise already. -- Steven D'Aprano From greg.ewing at canterbury.ac.nz Sat Aug 8 02:49:24 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 08 Aug 2009 12:49:24 +1200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <87d477lnuk.fsf@benfinney.id.au> References: <87d477lnuk.fsf@benfinney.id.au> Message-ID: <4A7CCB94.8020408@canterbury.ac.nz> My suggestion for a lightweight lambda syntax is args -> expr Examples: map(x -> x + 5, something) d = defauldict(->[]) -- Greg From tleeuwenburg at gmail.com Sat Aug 8 03:07:30 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Sat, 8 Aug 2009 11:07:30 +1000 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <4A7BEF80.1090505@canterbury.ac.nz> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <3cdcefb80908060816n3124ca44kf0ae0738332bdd0e@mail.gmail.com> <6217e0a0908061111u5970fbc3p9803e72b8b3e53a0@mail.gmail.com> <871vnob9uq.fsf@uwakimon.sk.tsukuba.ac.jp> <4A7BEF80.1090505@canterbury.ac.nz> Message-ID: <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> Hi all, Well, the thread's basically concluded, but I'm not sure if it's appropriate to just 'leave it hanging' seeing as I instigated the thread. So thanks all for the discussion points and I think we can wrap things up now. Please feel free to comment on the summary below if you think it doesn't have all the relevant details. To summarise the discussion for posterity: On the Plus Side: ? is part of English grammar, and is an intuitive clue-in that a yes/no, true/false response is expected as a result. Some people (myself included) would find this an aesthetically pleasing way to label methods which will come back with a flag. On the Negative Side: ? looks like an operator, not part of the method name, and might confuse people It wasn't mentioned much, but of course there is all the work involved to change the Python grammar! It could collide with external dependencies such as IPython Outcome: It was decided this would not be adopted. Secondary outcomes: A couple of idioms were suggested, such as a trailing capital Q instead (in my mind, Q for question) of the ?, or _p for predicate. It's possible people may find it useful to adopt or further discuss such idioms. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sat Aug 8 03:17:39 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 08 Aug 2009 13:17:39 +1200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <200908081021.15580.steve@pearwood.info> References: <200908081021.15580.steve@pearwood.info> Message-ID: <4A7CD233.7060905@canterbury.ac.nz> Steven D'Aprano wrote: > Why would you want to get rid of the lambda keyword? What's the benefit? > > Is this about saving twelve keystrokes? It's about conciseness. The only time it makes sense to write a function in-line is when the body is extremely short -- but then it gets swamped by the lambda keyword itself. But I agree that the OP's particular solution is restricted to a case that's too special. My version is completely general -- it can express anything that the existing lambda can express. -- Greg From steve at pearwood.info Sat Aug 8 03:53:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Aug 2009 11:53:38 +1000 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> Message-ID: <200908081153.39137.steve@pearwood.info> On Sat, 8 Aug 2009 11:07:30 am Tennessee Leeuwenburg wrote: > Hi all, > Well, the thread's basically concluded, but I'm not sure if it's > appropriate to just 'leave it hanging' seeing as I instigated the > thread. So thanks all for the discussion points and I think we can > wrap things up now. Please feel free to comment on the summary below > if you think it doesn't have all the relevant details. > > To summarise the discussion for posterity: > > On the Plus Side: > ? is part of English grammar, and is an intuitive clue-in that a > yes/no, true/false response is expected as a result. What makes you think that the question mark is a clue-in that a yes/no answer is expected? When did you come up with the idea? Who would agree with you? Where can I see some evidence for that suggestion? Excepting rhetorical questions, ? is an intuitive clue-in that a response is required, with no limitation on the nature of the response. Who/what/why/how/when questions expect open-ended answers. It seems to me that the clue-in that a yes/no answer is expected comes from existence words and intention words, those related to be and will: Is there a doctor in the house? Would you pass me the spanner? Can I do this? In any case, I don't think that "returns a boolean flag" versus "returns a non-flag" is an important distinction. Flags can be used as arbitrary objects, and non-flags can be used as flags. -- Steven D'Aprano From gerald.britton at gmail.com Sat Aug 8 07:10:15 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sat, 8 Aug 2009 01:10:15 -0400 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <4A7CD233.7060905@canterbury.ac.nz> References: <200908081021.15580.steve@pearwood.info> <4A7CD233.7060905@canterbury.ac.nz> Message-ID: <5d1a32000908072210m493dffd5k1a6ba73fd03999bc@mail.gmail.com> +1 On Fri, Aug 7, 2009 at 9:17 PM, Greg Ewing wrote: > Steven D'Aprano wrote: > > Why would you want to get rid of the lambda keyword? What's the benefit? >> >> Is this about saving twelve keystrokes? >> > > It's about conciseness. The only time it makes sense to > write a function in-line is when the body is extremely > short -- but then it gets swamped by the lambda keyword > itself. > > But I agree that the OP's particular solution is restricted > to a case that's too special. My version is completely > general -- it can express anything that the existing lambda > can express. > > -- > Greg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Aug 8 07:51:26 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Aug 2009 15:51:26 +1000 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <7afdee2f0908070443j3d269ceco897aaa129a5b5b49@mail.gmail.com> Message-ID: <200908081551.26421.steve@pearwood.info> On Fri, 7 Aug 2009 09:43:54 pm Tal Einat wrote: > You can already do bitwise operations on bytes, Are you sure about that? I tried looking it up in the docs, but python.org is down at the moment, so excuse me if I've missed something obvious. In Python 3.0: >>> b'ABCD' & 255 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for &: 'bytes' and 'int' >>> >>> b'ABCD' & b'\xff' Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for &: 'bytes' and 'bytes' Have I missed something? > As I see it, the major underlying problem here is that > byte-arrays are immutable, since what you really want is to be able > to change a single byte of the array in-place. I'm not speaking for the original poster, but for myself, not necessarily. While it would be nice to be able to flip bits directly, I'd be happy with bitwise operators to return new instances, as they currently do for ints. Something like: # not actual code >>> bb = bytes("ABCD") >>> bb &= b"\xff\0" >>> bb b"C\0" This could work for bit-flipping operations, although I'd welcome an easier to use API: # still not actual code >>> bb = bytes("ABCD") >>> bb |= (2**5)*256 # set the 5th bit of the 2nd byte >>> bb b"ABcD" -- Steven D'Aprano From steve at pearwood.info Sat Aug 8 07:54:36 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 8 Aug 2009 15:54:36 +1000 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> Message-ID: <200908081554.36358.steve@pearwood.info> On Fri, 7 Aug 2009 11:19:20 pm Mark Dickinson wrote: > > Also conceivable is using the shift operators >> and << on bytes, > > but I personally would use that less often, and the result of such > > an operation is ambiguous due to endianness. > > Agreed. To make sense of the shift operators you effectively have > to give 'position' interpretations for the individual bits, and > there's no single obvious way of doing this; for the plain bitwise > operations this isn't necessary. To me, the single obvious meaning of left- and right-shift is to shift to the left and the right :) E.g. b"abcd" >> 8 => "abc" b"abcd" << 8 => "abcd\0" which would have the benefit of matching what ints already do: >>> [hex(ord(c)) for c in "ABCD"] ['0x41', '0x42', '0x43', '0x44'] >>> n = 0x41424344 >>> hex(n >> 8) '0x414243' >>> hex(n << 8) '0x4142434400' I'm not sure what other "obvious" meanings you could give them. Have I missed something? -- Steven D'Aprano From bruce at leapyear.org Sat Aug 8 08:55:43 2009 From: bruce at leapyear.org (Bruce Leban) Date: Fri, 7 Aug 2009 23:55:43 -0700 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <4A7CCB94.8020408@canterbury.ac.nz> References: <87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz> Message-ID: I notice you said "args" not "arg". How would you parse foo(x, y -> x + y)? Even if the compiler can parse it, how about the human reading the code? If lambda wasn't already there, I could see using (args) -> expr syntax but I think there shouldn't be two ways to do something so simple. If you really want a shorter syntax, the obvious choice is foo(?x, y: x + y). --- Bruce On Fri, Aug 7, 2009 at 5:49 PM, Greg Ewing wrote: > My suggestion for a lightweight lambda syntax is > > args -> expr > > Examples: > > map(x -> x + 5, something) > > d = defauldict(->[]) > > -- > Greg > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn at masklinn.net Sat Aug 8 10:18:39 2009 From: masklinn at masklinn.net (Masklinn) Date: Sat, 8 Aug 2009 10:18:39 +0200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: <87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz> Message-ID: <9129F740-971E-4188-A946-A80A38313B31@masklinn.net> On 8 Aug 2009, at 08:55 , Bruce Leban wrote: > If you really want a shorter syntax, the obvious choice is foo(?x, > y: x + > y). > > --- Bruce +1 From greg.ewing at canterbury.ac.nz Sat Aug 8 13:00:15 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 08 Aug 2009 23:00:15 +1200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: <87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz> Message-ID: <4A7D5ABF.7020205@canterbury.ac.nz> Bruce Leban wrote: > I notice you said "args" not "arg". How would you parse foo(x, y -> x + > y)? Function foo has 2 arguments there. The other way would be foo((x, y) -> x + y) -- Greg From zuo at chopin.edu.pl Sat Aug 8 13:19:07 2009 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 08 Aug 2009 13:19:07 +0200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <4A7CCB94.8020408@canterbury.ac.nz> References: <87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz> Message-ID: 08-08-2009 Greg Ewing wrote: > My suggestion for a lightweight lambda syntax is > > args -> expr > > Examples: > > map(x -> x + 5, something) > > d = defauldict(->[]) -1 The idea would be nice, but "->" has already another meaning (in a different context -- function annotations -- but it's not a good idea to use the same "operator" for two completely different things, both connected with functions). *j From zuo at chopin.edu.pl Sat Aug 8 13:22:58 2009 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Sat, 08 Aug 2009 13:22:58 +0200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: Message-ID: > I was thinking about a good syntax for implicit lambdas for a while > and today I had this idea: make ``_:`` a shortcut for ``lambda > _=None:`` -1 I personally feel it as ugly and not readable (visually) at first sight... And, as others noted, '_' is already being used for a few completely different things. *j From solipsis at pitrou.net Sat Aug 8 16:51:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 8 Aug 2009 14:51:28 +0000 (UTC) Subject: [Python-ideas] bitwise operations on bytes and buffer objects References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> Message-ID: Mark Dickinson writes: > > +1 from me. I'd say that it makes just as much sense to do bit > operations on bytes or bytearrays as it does on integers. We should also make it work on memoryviews. Regards Antoine. From guido at python.org Sat Aug 8 23:31:32 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Aug 2009 14:31:32 -0700 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <200908081554.36358.steve@pearwood.info> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: On Fri, Aug 7, 2009 at 10:54 PM, Steven D'Aprano wrote: > On Fri, 7 Aug 2009 11:19:20 pm Mark Dickinson wrote: > >> > Also conceivable is using the shift operators >> and << on bytes, >> > but I personally would use that less often, and the result of such >> > an operation is ambiguous due to endianness. >> >> Agreed. ?To make sense of the shift operators you effectively have >> to give 'position' interpretations for the individual bits, and >> there's no single obvious way of doing this; ?for the plain bitwise >> operations this isn't necessary. > > To me, the single obvious meaning of left- and right-shift is to shift > to the left and the right :) > > E.g. > > b"abcd" >> 8 > => "abc" > > b"abcd" << 8 > => "abcd\0" > > which would have the benefit of matching what ints already do: > >>>> [hex(ord(c)) for c in "ABCD"] > ['0x41', '0x42', '0x43', '0x44'] >>>> n = 0x41424344 >>>> hex(n >> 8) > '0x414243' >>>> hex(n << 8) > '0x4142434400' > > > I'm not sure what other "obvious" meanings you could give them. Have I > missed something? Yes, byte order. It is not at all "obvious" whether the lowest-order byte is on the left or on the right. Your interpretation is big-endian. But mathematically speaking, a little-endian interpretation is somewhat easier, because the value in byte number i corresponds to that value multiplied by 256**i. Another way to look at is, is b'abc' supposed to be equal to b'\0abc' (big-endian) or b'abc\0' (little-endian) ? I find ignoring trailing nulls more logical than ignoring leading nulls, since the indexes of the significant digits are the same in the little-endian case. In the grander scheme of things, I worry that interpreting byte strings as integers and implementing bitwise operators on them is going to cause more confusion and isn't generally useful enough to warrant the extra code. I'd be okay with a standard API to transform a byte array into an integer and vice versa -- there you can be explicit about byte order and what to do about negative numbers. I can't remember right now if we already have such an API for arbitrary sizes -- the struct module only handles sizes 2, 4 and 8. I can hack it by going via a hex representation: i = 10**100 b = bytes.fromhex(hex(i)[2:]) import binascii j = int(binascii.hexlify(b), 16) assert j == i but this is a pretty gross hack. Still, most likely faster than writing out a loop in Python. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From g.brandl at gmx.net Sun Aug 9 00:09:42 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 09 Aug 2009 00:09:42 +0200 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: Guido van Rossum schrieb: > In the grander scheme of things, I worry that interpreting byte > strings as integers and implementing bitwise operators on them is > going to cause more confusion and isn't generally useful enough to > warrant the extra code. What about operations that don't require the bytes to be interpreted as anything else? The OP proposed bytes_a | bytes_b to mean bytes(b_a | b_b for (b_a, b_b) in zip(bytes_a, bytes_b)) except that (probably) an equal length would have to be asserted. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Sun Aug 9 00:27:46 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 8 Aug 2009 15:27:46 -0700 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: I think it's a very rarely used feature that is more likely to baffle the casual reader. The question of what to do with byte strings of unequal length is just the first issue that crops up. The (still legitimate) question why we would support | and & but not << and >> is another. It's a slippery slope... --Guido On Sat, Aug 8, 2009 at 3:09 PM, Georg Brandl wrote: > Guido van Rossum schrieb: > >> In the grander scheme of things, I worry that interpreting byte >> strings as integers and implementing bitwise operators on them is >> going to cause more confusion and isn't generally useful enough to >> warrant the extra code. > > What about operations that don't require the bytes to be interpreted > as anything else? > > The OP proposed > > bytes_a | bytes_b > > to mean > > bytes(b_a | b_b for (b_a, b_b) in zip(bytes_a, bytes_b)) > > except that (probably) an equal length would have to be asserted. > > Georg > > -- > Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. > Four shall be the number of spaces thou shalt indent, and the number of thy > indenting shall be four. Eight shalt thou not indent, nor either indent thou > two, excepting that thou then proceed to four. Tabs are right out. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Sun Aug 9 00:41:55 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 09 Aug 2009 10:41:55 +1200 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: <4A7DFF33.8050003@canterbury.ac.nz> Guido van Rossum wrote: > I think it's a very rarely used feature that is more likely to baffle > the casual reader. However there's currently no way to efficiently do bitwise operations en masse on raw bytes without converting to and from long integers, which doesn't seem very satisfactory. Refusing to provide this ability for the above reason sounds like purity beating practicality to me. > The question of what to do with byte strings of > unequal length is just the first issue that crops up. I would raise an exception. > The (still > legitimate) question why we would support | and & but not << and >> is > another. Ambiguity due to byte order seems like a good enough reason not to implement shifts as operators. Maybe provide methods that specify a byte order explicitly? -- Greg From mwm-keyword-python.b4bdba at mired.org Sun Aug 9 01:22:07 2009 From: mwm-keyword-python.b4bdba at mired.org (Mike Meyer) Date: Sat, 8 Aug 2009 19:22:07 -0400 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <4A7DFF33.8050003@canterbury.ac.nz> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <4A7DFF33.8050003@canterbury.ac.nz> Message-ID: <20090808192207.31c36098@bhuda.mired.org> On Sun, 09 Aug 2009 10:41:55 +1200 Greg Ewing wrote: > Guido van Rossum wrote: > > I think it's a very rarely used feature that is more likely to baffle > > the casual reader. > > However there's currently no way to efficiently do > bitwise operations en masse on raw bytes without > converting to and from long integers, which doesn't > seem very satisfactory. Refusing to provide this > ability for the above reason sounds like purity > beating practicality to me. It seems to me that bytes are a container type. How many builtin operators are there that operate on every element in any kind of container (other than queries, many of which already work on bytes)? It seems to me that what you really want is a high-performance array module of some flavor. Isn't that part of what NumPy provides? http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From facundobatista at gmail.com Sun Aug 9 07:29:37 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Sun, 9 Aug 2009 02:29:37 -0300 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: On Sat, Aug 8, 2009 at 6:31 PM, Guido van Rossum wrote: > In the grander scheme of things, I worry that interpreting byte > strings as integers and implementing bitwise operators on them is > going to cause more confusion and isn't generally useful enough to Indeed. But let me rephrase this. I don't know Python, I'm just starting with Py3k, and I see a bytes object. I don't know what a "byte string" is (and I even feel that the term is strange), but I understand that b"b3\xdd" is an array of three bytes, which, of course, is an array of 24 bits (unless I know that bytes are not always octets, but I think it's the same for this case). So, I think that providing bit semantics is a must for bytes objects. I don't care (or I don't know enough to know that this exist) about big/little endian, I just assume that this "just works" (it's is the same case with line ending bytes: I may not know that different OSs finish lines with different marks). *I* (me as Facundo), don't really know enough to be able to propose "what could work and keep surprise to a minimum", but just wanted to present an "end user case" about this. > warrant the extra code. I'd be okay with a standard API to transform a > byte array into an integer and vice versa -- there you can be explicit > about byte order and what to do about negative numbers. I can't Yes. Bit arrays and integers suffer (suffer?) from the same issue that Unicode. An integer, and an Unicode character, are encoded into bits... and if you have bits, you need to know how to decode them to get again your Unicode character, or your integer. Maybe we could use the same names? What about something like b"....". decode("little_endian__two's_complement") --> int? (please, very please, better encoding names) Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From ede at mit.edu Sun Aug 9 09:06:32 2009 From: ede at mit.edu (Eric Eisner) Date: Sun, 9 Aug 2009 16:06:32 +0900 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: <42ef4ee0908090006q774923b5p17fc6ae7b1bff32b@mail.gmail.com> > In the grander scheme of things, I worry that interpreting byte > strings as integers and implementing bitwise operators on them is > going to cause more confusion and isn't generally useful enough to > warrant the extra code. I'd be okay with a standard API to transform a > byte array into an integer and vice versa -- there you can be explicit > about byte order and what to do about negative numbers. For some background, I think the separation of text and data is a powerful idea, and now that I have my nice builtin object for data (and its mutable cousin), I want to be able to manipulate the raw data directly. Currently, all of the bitwise manipulations live in the int type (for what I like to think of as historical reasons). Currently, when manipulating a byte at a time, the bytes type does the right thing and shows the elements as ints, giving me access to these bitwise manipulations. However there currently lacks builtin functionality to manipulate multi-bytes chunks. In the grander scheme of things, this is what I would like to see addressed. The most general solution, which Guido mentions, and which I have also been thinking about, is a builtin conversion from bytes to ints that explicitly requires resolving ambiguities (eg endianness, negatives, data length). I personally use those pretty gross hacks to fake the functionality now, but they assumes big-endian, two's complement etc. I proposed direct bitwise operators (~ & ^ |) for bytes because these seemed to be the subset of all bitwise manipulations that were (mostly) unambiguous for raw bytes. Even if python gains conversions between bytes and ints, I still think these unambiguous operations would be useful to have (especially ~, the most unambiguous among them). -Eric From ilya.nikokoshev at gmail.com Sun Aug 9 09:29:37 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Sun, 9 Aug 2009 11:29:37 +0400 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <200908081021.15580.steve@pearwood.info> References: <200908081021.15580.steve@pearwood.info> Message-ID: Thank you and everyone else for insightful posts detailing why my examples don't make a good argument for the syntax. Even though my original suggestion, similarly pep 312, wouldn't break any existing programs and would not lead to ambiguity in 'if _:', I rescind it. However, another reason for implicit lambdas is lazy evaluation. For example, in another thread people discuss "... except ... if/else" conditional statement --- one reason being control expressions evaluate lazily. A function call passing callables currently looks ugly and unreadable: lazy_cond(expr, lambda: expensive(5), lambda: factorial(10**5)) and here 6 keystrokes of 'lambda' word *do* matter. Therefore I hope my unsuccessful proposal will encourage people to find something that works. On Sat, Aug 8, 2009 at 4:21 AM, Steven D'Aprano wrote: > On Fri, 7 Aug 2009 10:46:40 pm ilya wrote: >> I was thinking about a good syntax for implicit lambdas for a while >> and today I had this idea: make ``_:`` a shortcut for ``lambda >> _=None:`` > > [...] > >> The rationale is that you only want to get rid of lambda keyword to >> create a *very* simple function, the one that will be called either >> without parameters or with only one parameter. For everything more >> complicated, you really should go and write the explicit function >> signature using lambda. > > Why would you want to get rid of the lambda keyword? What's the benefit? > > Is this about saving twelve keystrokes? > > lambda _=None: > ?versus > _: > > Just how often do you want, or need, to write such a lambda? It seems to > me that not only is it a special case you want to break the rules for, > which goes against the Zen, but it's an incredibly rare special case. > > _ as an identifier already has three conventional meanings: > > (1) In the interactive interpreter, _ is the value of the last > expression typed. > > (2) It is commonly used to mean "I don't care about this value", e.g. > > t = ("Fred Smith", "123 Fourth Street", "New York", "dog") > name, _, _, pet = t > > (3) It is also often used in internationalization. > > You want to give it the extra meaning "a default parameter name for > lambda when I can't be bothered typing even a single letter name". > > Because _ is already a valid identifier, this will break code that does > this: > > while _: > ? ?process() > ? ?_ = function() > > if _: > ? ?print "something" > > > Not the best choice of names, but somebody, somewhere, is doing that, > and your suggestion will break their code. > > > Looking at the three examples you gave: > > map( _: _ + 5, some_list) > register_callback( _: True) > def apply_transform(..., transform = _:_, ... ): > > In the first case, I wouldn't use the short-cut form even if it were > available. I'd write a lambda that used a more meaningful name. In this > case, I'm expecting an int, so I would use n, or a float, so I'd use x. > I'd also avoid setting the pointless default: > > map(lambda x: x+5, some_list) > ?vs > map(_: _+5, some_list) > > Since your suggestion doesn't do precisely what I want, the only reason > I would have for using your construct is to save seven keystrokes. > Encouraging laziness on the behalf of the programmer is not a good > reason for special-casing rare cases. > > Second case: register_callback( _: True) > > I assume you're implying that the callback function must take a single > argument. In this example, using _ as the parameter name to the lambda > makes sense, because it is a "don't care" argument. But if the callback > function is documented as always being given a single argument, I would > want to know if it was being called without any arguments, so the > default value of None is inappropriate and I would avoid using it. > > Third case: def apply_transform(..., transform = _:_, ... ): > > I don't think I'd write a function called apply_transform() which made > the transformation function optional, let alone buried deep in the > middle of a whole lot of extra parameters. (I presume that's what > the "..."s are meant to imply.) But putting that aside, I see your > intention: a default do-nothing function which appears in a very long > parameter list. The problem is that instead of trying to shrink the > default value so you can fit all the parameters on a single line, you > should make such a complicated function signature more readable by > spreading it out: > > def apply_transform( > ? ?obj, > ? ?start, end, ? ? ?# start (inc) and end (exc) positions to apply > ? ?another_arg, ? ? # does something very important I'm sure > ? ?x=0, y=1, z=2, ? # more very important arguments > ? ?transform=( ? ? ?# default null transformation > ? ? ? ? ? ? ?lambda obj=None: obj), > ? ?frotz=False, ? ? # if true, frotz the hymangirator with spangule > ? ?hymangirator=None, > ? ?spangule=None, > ? ?magic=12345, ? ? # this needs no documentation > ? ?namespace={}, > ? ?flibbertigibbet=None, > ? ?_private_magic=[] ?# the caller shouldn't supply this > ? ?): > > (Even better is to avoid such complicated function signatures, but > sometimes that's not an option.) > > So again I'd be very unlikely to use your suggested construct except out > of simple can't-be-bothered-to-type-a-dozen-letters laziness. Pandering > to that sort of laziness is probably not a good thing. > > Fundamentally, this suggestion doesn't add expressability to the > language, or power. Laziness on it's own is not a good reason for > special casing rare cases. If it was a very common case, then *perhaps* > you would have an argument for special casing needless verbiage: > conciseness (up to a point) is a virtue in a language. That's partly > why we have lambdas in the first place, so we can write this: > > reduce(lambda a,b: (a+b)/2.0, somelist) > > instead of this: > > def average(a, b): > ? ?return (a+b)/2.0 > reduce(average, somelist) > > But this isn't a common case, it's a rare case, and the case you're > hoping to replace is pretty concise already. > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From gerald.britton at gmail.com Sun Aug 9 13:03:43 2009 From: gerald.britton at gmail.com (Gerald Britton) Date: Sun, 9 Aug 2009 07:03:43 -0400 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: References: <200908081021.15580.steve@pearwood.info> Message-ID: <5d1a32000908090403h53d1d25ai5b7e72e06cb591fa@mail.gmail.com> Your proposal certainly would lead to ambiguity in reading: _ = myfunc if _:_(_:_, ...) The "if _:_" means: if '_' evaluates to boolean true, call it The second _:_ uses your new lambda construct. Could a compiler parse it? Probably. Can a human understand it? Maybe, with difficulty, but it would surely lead to hard-to-find errors. On Sun, Aug 9, 2009 at 3:29 AM, ilya wrote: > Thank you and everyone else for insightful posts detailing why my > examples don't make a good argument for the syntax. > > Even though my original suggestion, similarly pep 312, wouldn't break > any existing programs and would not lead to ambiguity in 'if _:', I > rescind it. > > However, another reason for implicit lambdas is lazy evaluation. For > example, in another thread people discuss "... except ... if/else" > conditional statement --- one reason being control expressions > evaluate lazily. A function call passing callables currently looks > ugly and unreadable: > > lazy_cond(expr, lambda: expensive(5), lambda: factorial(10**5)) > > and here 6 keystrokes of 'lambda' word *do* matter. > > Therefore I hope my unsuccessful proposal will encourage people to > find something that works. > > > On Sat, Aug 8, 2009 at 4:21 AM, Steven D'Aprano > wrote: > > On Fri, 7 Aug 2009 10:46:40 pm ilya wrote: > >> I was thinking about a good syntax for implicit lambdas for a while > >> and today I had this idea: make ``_:`` a shortcut for ``lambda > >> _=None:`` > > > > [...] > > > >> The rationale is that you only want to get rid of lambda keyword to > >> create a *very* simple function, the one that will be called either > >> without parameters or with only one parameter. For everything more > >> complicated, you really should go and write the explicit function > >> signature using lambda. > > > > Why would you want to get rid of the lambda keyword? What's the benefit? > > > > Is this about saving twelve keystrokes? > > > > lambda _=None: > > versus > > _: > > > > Just how often do you want, or need, to write such a lambda? It seems to > > me that not only is it a special case you want to break the rules for, > > which goes against the Zen, but it's an incredibly rare special case. > > > > _ as an identifier already has three conventional meanings: > > > > (1) In the interactive interpreter, _ is the value of the last > > expression typed. > > > > (2) It is commonly used to mean "I don't care about this value", e.g. > > > > t = ("Fred Smith", "123 Fourth Street", "New York", "dog") > > name, _, _, pet = t > > > > (3) It is also often used in internationalization. > > > > You want to give it the extra meaning "a default parameter name for > > lambda when I can't be bothered typing even a single letter name". > > > > Because _ is already a valid identifier, this will break code that does > > this: > > > > while _: > > process() > > _ = function() > > > > if _: > > print "something" > > > > > > Not the best choice of names, but somebody, somewhere, is doing that, > > and your suggestion will break their code. > > > > > > Looking at the three examples you gave: > > > > map( _: _ + 5, some_list) > > register_callback( _: True) > > def apply_transform(..., transform = _:_, ... ): > > > > In the first case, I wouldn't use the short-cut form even if it were > > available. I'd write a lambda that used a more meaningful name. In this > > case, I'm expecting an int, so I would use n, or a float, so I'd use x. > > I'd also avoid setting the pointless default: > > > > map(lambda x: x+5, some_list) > > vs > > map(_: _+5, some_list) > > > > Since your suggestion doesn't do precisely what I want, the only reason > > I would have for using your construct is to save seven keystrokes. > > Encouraging laziness on the behalf of the programmer is not a good > > reason for special-casing rare cases. > > > > Second case: register_callback( _: True) > > > > I assume you're implying that the callback function must take a single > > argument. In this example, using _ as the parameter name to the lambda > > makes sense, because it is a "don't care" argument. But if the callback > > function is documented as always being given a single argument, I would > > want to know if it was being called without any arguments, so the > > default value of None is inappropriate and I would avoid using it. > > > > Third case: def apply_transform(..., transform = _:_, ... ): > > > > I don't think I'd write a function called apply_transform() which made > > the transformation function optional, let alone buried deep in the > > middle of a whole lot of extra parameters. (I presume that's what > > the "..."s are meant to imply.) But putting that aside, I see your > > intention: a default do-nothing function which appears in a very long > > parameter list. The problem is that instead of trying to shrink the > > default value so you can fit all the parameters on a single line, you > > should make such a complicated function signature more readable by > > spreading it out: > > > > def apply_transform( > > obj, > > start, end, # start (inc) and end (exc) positions to apply > > another_arg, # does something very important I'm sure > > x=0, y=1, z=2, # more very important arguments > > transform=( # default null transformation > > lambda obj=None: obj), > > frotz=False, # if true, frotz the hymangirator with spangule > > hymangirator=None, > > spangule=None, > > magic=12345, # this needs no documentation > > namespace={}, > > flibbertigibbet=None, > > _private_magic=[] # the caller shouldn't supply this > > ): > > > > (Even better is to avoid such complicated function signatures, but > > sometimes that's not an option.) > > > > So again I'd be very unlikely to use your suggested construct except out > > of simple can't-be-bothered-to-type-a-dozen-letters laziness. Pandering > > to that sort of laziness is probably not a good thing. > > > > Fundamentally, this suggestion doesn't add expressability to the > > language, or power. Laziness on it's own is not a good reason for > > special casing rare cases. If it was a very common case, then *perhaps* > > you would have an argument for special casing needless verbiage: > > conciseness (up to a point) is a virtue in a language. That's partly > > why we have lambdas in the first place, so we can write this: > > > > reduce(lambda a,b: (a+b)/2.0, somelist) > > > > instead of this: > > > > def average(a, b): > > return (a+b)/2.0 > > reduce(average, somelist) > > > > But this isn't a common case, it's a rare case, and the case you're > > hoping to replace is pretty concise already. > > > > > > > > -- > > Steven D'Aprano > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > http://mail.python.org/mailman/listinfo/python-ideas > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- Gerald Britton -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn at masklinn.net Sun Aug 9 16:40:15 2009 From: masklinn at masklinn.net (Masklinn) Date: Sun, 9 Aug 2009 16:40:15 +0200 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <5d1a32000908090403h53d1d25ai5b7e72e06cb591fa@mail.gmail.com> References: <200908081021.15580.steve@pearwood.info> <5d1a32000908090403h53d1d25ai5b7e72e06cb591fa@mail.gmail.com> Message-ID: <9226C8D2-0C02-4D63-BA19-66A210D005A9@masklinn.net> On 9 Aug 2009, at 13:03 , Gerald Britton wrote: > Your proposal certainly would lead to ambiguity in reading: > _ = myfunc > if _:_(_:_, ...) > > The "if _:_" means: > > if '_' evaluates to boolean true, call it > > The second _:_ uses your new lambda construct. Could a compiler > parse it? > Probably. Can a human understand it? Maybe, with difficulty, but it > would > surely lead to hard-to-find errors. And from the point of view of someone who'd really like a "better lambda", the "implicit lambda" idea is pretty much worthless. As others have said, it saves a few keystrokes and that's pretty much it. If work is done towards a better/more useable lambda, it should at least encompass full-blown anonymous expressions, not limit itself to the current restricted lambda. From guido at python.org Sun Aug 9 22:57:26 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 9 Aug 2009 13:57:26 -0700 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: On Sat, Aug 8, 2009 at 10:29 PM, Facundo Batista wrote: > On Sat, Aug 8, 2009 at 6:31 PM, Guido van Rossum wrote: > >> In the grander scheme of things, I worry that interpreting byte >> strings as integers and implementing bitwise operators on them is >> going to cause more confusion and isn't generally useful enough to > > Indeed. ?But let me rephrase this. > > I don't know Python, I'm just starting with Py3k, and I see a bytes > object. I don't know what a "byte string" is ?(and I even feel that > the term is strange), but I understand that b"b3\xdd" is an array of > three bytes, which, of course, is an array of 24 bits (unless I know > that bytes are not always octets, but I think it's the same for this > case). > > So, I think that providing bit semantics is a must for bytes objects. > > I don't care (or I don't know enough to know that this exist) about > big/little endian, I just assume that this "just works" (it's is the > same case with line ending bytes: I may not know that different OSs > finish lines with different marks). Sounds like a non-sequitur if anything. >From the perspective of "I don't know Python" you cannot expect to draw valid conclusions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at mrabarnett.plus.com Sun Aug 9 23:32:14 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 09 Aug 2009 22:32:14 +0100 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: <4A7F405E.90409@mrabarnett.plus.com> Guido van Rossum wrote: > On Sat, Aug 8, 2009 at 10:29 PM, Facundo > Batista wrote: >> On Sat, Aug 8, 2009 at 6:31 PM, Guido van Rossum wrote: >> >>> In the grander scheme of things, I worry that interpreting byte >>> strings as integers and implementing bitwise operators on them is >>> going to cause more confusion and isn't generally useful enough to >> Indeed. But let me rephrase this. >> >> I don't know Python, I'm just starting with Py3k, and I see a bytes >> object. I don't know what a "byte string" is (and I even feel that >> the term is strange), but I understand that b"b3\xdd" is an array of >> three bytes, which, of course, is an array of 24 bits (unless I know >> that bytes are not always octets, but I think it's the same for this >> case). >> >> So, I think that providing bit semantics is a must for bytes objects. >> >> I don't care (or I don't know enough to know that this exist) about >> big/little endian, I just assume that this "just works" (it's is the >> same case with line ending bytes: I may not know that different OSs >> finish lines with different marks). > > Sounds like a non-sequitur if anything. > >>From the perspective of "I don't know Python" you cannot expect to > draw valid conclusions. > I think that bitstrings would be better handled by a distinct bitstring class. From dickinsm at gmail.com Mon Aug 10 10:05:30 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Aug 2009 09:05:30 +0100 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum wrote: > [...] I'd be okay with a standard API to transform a > byte array into an integer and vice versa -- there you can be explicit > about byte order and what to do about negative numbers. [...] That would also be a welcome addition. It's been requested on bugs.python.org at least a couple of times[1][2], and the C code to do the conversions already exists (_PyLong_{As,From}_ByteArray in longobject.c), so it wouldn't be too much work to implement. The main problem would be deciding exactly what the API should be and where to put it. Mark [1] http://bugs.python.org/issue1023290 [2] http://bugs.python.org/issue923643 From dickinsm at gmail.com Mon Aug 10 11:23:13 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 10 Aug 2009 10:23:13 +0100 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> Message-ID: <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum wrote: > -- the struct module only handles sizes 2, 4 and 8. I can hack it by > going via a hex representation: > > ?i = 10**100 > ?b = bytes.fromhex(hex(i)[2:]) > ?import binascii > ?j = int(binascii.hexlify(b), 16) > ?assert j == i > > but this is a pretty gross hack. The first part also doesn't work if hex(i) has odd length. [py3k]: >>> bytes.fromhex(hex(10**99)[2:]) Traceback (most recent call last): File "", line 1, in ValueError: non-hexadecimal number found in fromhex() arg at position 82 I think the fact that it's non-trivial to get this right first time is further evidence that it would be useful to have built-in int <-> bytes conversions somewhere. Mark From ncoghlan at gmail.com Mon Aug 10 12:30:38 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Aug 2009 20:30:38 +1000 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> Message-ID: <4A7FF6CE.7070903@gmail.com> Mark Dickinson wrote: > On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum wrote: >> [...] I'd be okay with a standard API to transform a >> byte array into an integer and vice versa -- there you can be explicit >> about byte order and what to do about negative numbers. [...] > > That would also be a welcome addition. It's been requested on > bugs.python.org at least a couple of times[1][2], and the C code > to do the conversions already exists (_PyLong_{As,From}_ByteArray > in longobject.c), so it wouldn't be too much work to implement. > > The main problem would be deciding exactly what the API should be > and where to put it. My suggestion would be to provide the relevant constructors as class methods on int, bytes and bytearray: bytes.from_int bytearray.from_int int.from_bytes Alternatively, the int.from_bytes classmethod could be replaced with a "to_int" instance method on bytes and bytearray. The method signatures would need to closely resemble the C API. In particular, for the conversion from int to bytes being able to state a desired size would both allow detection of cases where the value is too large as well as proper padding of the two's complement sign bit. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ede at mit.edu Mon Aug 10 13:02:08 2009 From: ede at mit.edu (Eric Eisner) Date: Mon, 10 Aug 2009 20:02:08 +0900 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <4A7FF6CE.7070903@gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> Message-ID: <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> On Mon, Aug 10, 2009 at 19:30, Nick Coghlan wrote: > Mark Dickinson wrote: >> The main problem would be deciding exactly what the API should be >> and where to put it. > > My suggestion would be to provide the relevant constructors as class > methods on int, bytes and bytearray: > > bytes.from_int > bytearray.from_int > int.from_bytes > > Alternatively, the int.from_bytes classmethod could be replaced with a > "to_int" instance method on bytes and bytearray. > > The method signatures would need to closely resemble the C API. In > particular, for the conversion from int to bytes being able to state a > desired size would both allow detection of cases where the value is too > large as well as proper padding of the two's complement sign bit. For completeness, any function converting from int to bytes needs to accept the arguments size, endianness, and two's complement handling. By default, size and two's complement could be inferred from the int's value. Any function converting from bytes to int needs to accept the arguments endianness and two's complement. The function cannot really make a reasonable assumption about either. Having this much to specify in bytes.from_int or bytes.to_int seems a little overwhelming (to me anyway). It is already more complicated than the analogous bytes.decode for strings. What about a putting this in module support, and putting each of these options in its own named method? Some dummy examples that may need better names: base256.encode_le(0x123456, size=5) # returns b'\x56\x34\x12\x00\x00' base256.decode_le_positive(b'\xff\xce') # returns 0xceff -Eric From ncoghlan at gmail.com Mon Aug 10 15:13:48 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Aug 2009 23:13:48 +1000 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> Message-ID: <4A801D0C.2000502@gmail.com> Eric Eisner wrote: > For completeness, any function converting from int to bytes needs to > accept the arguments size, endianness, and two's complement handling. > By default, size and two's complement could be inferred from the int's > value. Keep in mind that Mark's suggestion isn't to completely reinvent the wheel - it is more about exposing an existing internal C API to Python code. Since that C API is provided by the int object, we should also consider the option of providing it in Python purely as methods of that object. Bytes to int: bytes pointer: pointer to first byte to be converted size: number of bytes to be converted little_endian: flag indicating MSB is at offset 0 signed: flag indicating whether or not the value is signed For a Python version of this conversion, the size argument is unnecessary, reducing the required parameters to a bytes/bytearray/memoryview reference, an endianness marker and a 'signed' flag (to indicate whether the buffer contains an unsigned value or a signed two's complement value). One slight quirk of the C API that probably shouldn't be replicated is a size of 0 translating to an integer result of zero. For the Python API, passing in an empty byte sequence should trigger a ValueError. In the C API, int to bytes takes the same parameters as the bytes to int conversion, but the meaning is slightly different. Int to bytes: bytes pointer: pointer to first byte to be converted size: number of bytes in result little_endian: flag indicating to write MSB to offset 0 signed: flag indicating negative value can be converted In this case, the size matters even for a Python version as an OverflowError will be raised if the integer won't fit in the specified size and sufficient sign bits are added to pad the result out to the requested size otherwise. That suggests to me the following signatures for the conversion functions (regardless of the names they might be given): int.from_bytes(data, *, little_endian=None, signed=True) little_endian would become a three-valued parameter for the Python version: None = native; False = little-endian; True = big-endian. The three valued parameter is necessary since Python code can't get at the "IS_LITTLE_ENDIAN" macro that the PyLong code uses to determine the endianness of the system when calling the C API functions. signed would just be an ordinary boolean flag int.to_bytes(data, *, size=0, little_endian=None, signed=True) A size <= 0 would mean to produce as many bytes as are needed to represent the integer and no more. Otherwise it would represent the maximum number of bytes allowed in the response (raising OverflowError if the value won't fit). little_endian and signed would be interpreted as for the conversion from bytes to an integer Sure, these could be moved into a module of their own, but I'm not sure what would be gained by doing so. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Mon Aug 10 15:38:36 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 10 Aug 2009 13:38:36 +0000 (UTC) Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> Message-ID: Nick Coghlan writes: > > int.to_bytes(data, *, size=0, little_endian=None, signed=True) It would be better IMO if this method belonged to the target type, so that you can write `bytearray.from_int()` without going through an intermediate bytes object. What if `data` is equal to 0? Does it return an empty bytes object? It doesn't look dangerous to do so, so I'd say go for it. From ncoghlan at gmail.com Mon Aug 10 15:53:53 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 10 Aug 2009 23:53:53 +1000 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> Message-ID: <4A802671.9040002@gmail.com> Antoine Pitrou wrote: > Nick Coghlan writes: >> int.to_bytes(data, *, size=0, little_endian=None, signed=True) > > It would be better IMO if this method belonged to the target type, so that you > can write `bytearray.from_int()` without going through an intermediate bytes > object. > > What if `data` is equal to 0? Does it return an empty bytes object? It doesn't > look dangerous to do so, so I'd say go for it. Actually, that's just a typo after copying the from_bytes signature line when writing my message. int.to_bytes would just be a normal instance method. If the value was 0, then the result would be a single byte with the value zero (or all zeroes if a size was specified). As far as bytearray goes, my original post did suggest that approach (i.e. class methods on the target types), but that then gets tedious as every new type that wants to support conversion from an integer needs to either go through one of the existing types or else have it's own C extension call the underlying C API directly. Something analagous to the file.readinto() method seems like a more appropriate solution for populating the mutable types. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From rrr at ronadam.com Mon Aug 10 19:16:18 2009 From: rrr at ronadam.com (Ron Adam) Date: Mon, 10 Aug 2009 12:16:18 -0500 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> Message-ID: <4A8055E2.2040001@ronadam.com> Mark Dickinson wrote: > On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum wrote: >> -- the struct module only handles sizes 2, 4 and 8. I can hack it by >> going via a hex representation: >> >> i = 10**100 >> b = bytes.fromhex(hex(i)[2:]) >> import binascii >> j = int(binascii.hexlify(b), 16) >> assert j == i >> >> but this is a pretty gross hack. > > The first part also doesn't work if hex(i) has odd length. > [py3k]: > >>>> bytes.fromhex(hex(10**99)[2:]) > Traceback (most recent call last): > File "", line 1, in > ValueError: non-hexadecimal number found in fromhex() arg at position 82 > > I think the fact that it's non-trivial to get this right first > time is further evidence that it would be useful to have > built-in int <-> bytes conversions somewhere. Are there going to possibly be other conversions to bytes and back? (float, string, struct, ...) It seems to me the type conversion to and from bytes should be on the encoded non-byte type, and other types including user created ones could follow that pattern. That may allow bytes to work with any type that has the required special methods to do the conversions. Then most of the methods on bytes would be for manipulating bytes in various ways. The constructor for the int type already does base and string conversions, extending it to bytes seems like it would be natural. int(bytes) # just like int(string) bytes = bytes(int) # calls int.__to_bytes__() to do the actual work. Ron From guido at python.org Mon Aug 10 19:36:26 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 10 Aug 2009 10:36:26 -0700 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <4A8055E2.2040001@ronadam.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> <4A8055E2.2040001@ronadam.com> Message-ID: The struct module already handles all those -- long ints are pretty much the only common type that it doesn't cover, because it only deals with fixed-length values. On Mon, Aug 10, 2009 at 10:16 AM, Ron Adam wrote: > > > Mark Dickinson wrote: >> >> On Sat, Aug 8, 2009 at 10:31 PM, Guido van Rossum wrote: >>> >>> -- the struct module only handles sizes 2, 4 and 8. I can hack it by >>> going via a hex representation: >>> >>> ?i = 10**100 >>> ?b = bytes.fromhex(hex(i)[2:]) >>> ?import binascii >>> ?j = int(binascii.hexlify(b), 16) >>> ?assert j == i >>> >>> but this is a pretty gross hack. >> >> The first part also doesn't work if hex(i) has odd length. >> [py3k]: >> >>>>> bytes.fromhex(hex(10**99)[2:]) >> >> Traceback (most recent call last): >> ?File "", line 1, in >> ValueError: non-hexadecimal number found in fromhex() arg at position 82 >> >> I think the fact that it's non-trivial to get this right first >> time is further evidence that it would be useful to have >> built-in int <-> bytes conversions somewhere. > > Are there going to possibly be other conversions to bytes and back? (float, > string, struct, ...) > > It seems to me the type conversion to and from bytes should be on the > encoded non-byte type, and other types including user created ones could > follow that pattern. That may allow bytes to work with any type that has the > required special methods to do the conversions. > > Then most of the methods on bytes would be for manipulating bytes in various > ways. > > > The constructor for the int type already does base and string conversions, > extending it to bytes seems like it would be natural. > > ? int(bytes) ? ? ? ? ? ?# just like int(string) > ? bytes = bytes(int) ? ?# calls int.__to_bytes__() to do the actual work. > > > Ron > > > > > > > > > > > > > > > > > > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ede at mit.edu Tue Aug 11 00:38:51 2009 From: ede at mit.edu (Eric Eisner) Date: Tue, 11 Aug 2009 07:38:51 +0900 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <4A8055E2.2040001@ronadam.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> <4A8055E2.2040001@ronadam.com> Message-ID: <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com> On Tue, Aug 11, 2009 at 02:16, Ron Adam wrote: > The constructor for the int type already does base and string conversions, > extending it to bytes seems like it would be natural. > > ? int(bytes) ? ? ? ? ? ?# just like int(string) > ? bytes = bytes(int) ? ?# calls int.__to_bytes__() to do the actual work. The constructor for bytes currently accepts a single int as an argument, producing that many zero bytes. As far as I can tell this behavior is undocumented, but I don't know if that can be changed easily... The first thing I did when I tried out the bytes type in 3.x was to try to convert an int to a byte via the constructor, and the current behavior surprised me. -Eric From greg.ewing at canterbury.ac.nz Tue Aug 11 03:05:26 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Aug 2009 13:05:26 +1200 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <4A801D0C.2000502@gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> Message-ID: <4A80C3D6.4030202@canterbury.ac.nz> Nick Coghlan wrote: > One slight quirk of the C API that probably shouldn't be replicated is a > size of 0 translating to an integer result of zero. For the Python API, > passing in an empty byte sequence should trigger a ValueError. Why? This seems like a perfectly logical limiting case to me. > int.from_bytes(data, *, little_endian=None, signed=True) > little_endian would become a three-valued parameter for the Python > version: None = native; False = little-endian; True = big-endian. I don't like the idea of a three-valued boolean. I also don't like boolean parameters whose sense is abritrary (why is it called "little_endian" and not "big_endian", and how do I remember which convention was chosen?) My suggestion would be to use the same characters that the struct module uses to represent endianness (">" for big-endian, "<" for little-endian, etc.) -- Greg From greg.ewing at canterbury.ac.nz Tue Aug 11 03:08:57 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 11 Aug 2009 13:08:57 +1200 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <4A802671.9040002@gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <4A802671.9040002@gmail.com> Message-ID: <4A80C4A9.7010300@canterbury.ac.nz> Nick Coghlan wrote: > As far as bytearray goes, my original post did suggest that approach > (i.e. class methods on the target types), but that then gets tedious as > every new type that wants to support conversion from an integer needs to > either go through one of the existing types or else have it's own C > extension call the underlying C API directly. I'd suggest that int.to_bytes should take an optional destination parameter, and it should accept any object having a mutable buffer interface. That would keep all the int-related methods with the int class, without restricting the source or destination to any particular type. -- Greg From rrr at ronadam.com Tue Aug 11 05:26:59 2009 From: rrr at ronadam.com (Ron Adam) Date: Mon, 10 Aug 2009 22:26:59 -0500 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> <4A8055E2.2040001@ronadam.com> <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com> Message-ID: <4A80E503.7090407@ronadam.com> Eric Eisner wrote: > On Tue, Aug 11, 2009 at 02:16, Ron Adam wrote: >> The constructor for the int type already does base and string conversions, >> extending it to bytes seems like it would be natural. >> >> int(bytes) # just like int(string) >> bytes = bytes(int) # calls int.__to_bytes__() to do the actual work. > > The constructor for bytes currently accepts a single int as an > argument, producing that many zero bytes. As far as I can tell this > behavior is undocumented, but I don't know if that can be changed > easily... > > The first thing I did when I tried out the bytes type in 3.x was to > try to convert an int to a byte via the constructor, and the current > behavior surprised me. It seems that bytes are quite different here in 2.6 and 3.0. In 2.6 and bytes is an alias for string(), so the bytes constructor behavior just converts the int to a string. ra at Gutsy:~$ python Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> bytes(123) '123' >>> bytes >>> help(bytes) class str(basestring) | str(object) -> string | | Return a nice string representation of the object. | If the argument is a string, the return value is the same object. | (clipped) Ron From g.brandl at gmx.net Tue Aug 11 08:01:55 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 11 Aug 2009 08:01:55 +0200 Subject: [Python-ideas] bitwise operations on bytes In-Reply-To: <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100223l74f2833dha2e45c1b55ac5c84@mail.gmail.com> <4A8055E2.2040001@ronadam.com> <42ef4ee0908101538m7a689cd1ldec143b62bd1b10e@mail.gmail.com> Message-ID: Eric Eisner schrieb: > On Tue, Aug 11, 2009 at 02:16, Ron Adam wrote: >> The constructor for the int type already does base and string conversions, >> extending it to bytes seems like it would be natural. >> >> int(bytes) # just like int(string) >> bytes = bytes(int) # calls int.__to_bytes__() to do the actual work. > > The constructor for bytes currently accepts a single int as an > argument, producing that many zero bytes. As far as I can tell this > behavior is undocumented, but I don't know if that can be changed > easily... It is documented, see http://docs.python.org/py3k/library/functions#bytearray. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Tue Aug 11 12:43:23 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 11 Aug 2009 10:43:23 +0000 (UTC) Subject: [Python-ideas] What about allowing '?' in method names? References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: Steven D'Aprano writes: > > What makes you think that the question mark is a clue-in that a yes/no > answer is expected? AFAIK it is a widely-used convention in the Ruby world. I'd even go as far as saying that it's quite pretty, as a typographical convention (not that other Ruby conventions are :-)). > In any case, I don't think that "returns a boolean flag" versus "returns > a non-flag" is an important distinction. I don't think this is a valid argument here. If '?' were to be allowed in identifier names (or at least at their end), people would be free to choose whatever convention they like. From arnodel at googlemail.com Tue Aug 11 15:25:15 2009 From: arnodel at googlemail.com (Arnaud Delobelle) Date: Tue, 11 Aug 2009 14:25:15 +0100 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: > Steven D'Aprano writes: >> >> What makes you think that the question mark is a clue-in that a yes/ >> no >> answer is expected? > > AFAIK it is a widely-used convention in the Ruby world. > I'd even go as far as saying that it's quite pretty, as a > typographical > convention (not that other Ruby conventions are :-)) Also in Scheme. (I think the question mark more or less replaces the 'p' suffix used in LISP). A noun, adjective or verb followed by a question mark is pretty obviously a predicate: number? empty? restart? I expect them all to return a boolean. -- Arnaud From masklinn at masklinn.net Tue Aug 11 15:55:04 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 11 Aug 2009 15:55:04 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote: > On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: > Steven D'Aprano writes: >>> >>> What makes you think that the question mark is a clue-in that a >>> yes/no >>> answer is expected? >> >> AFAIK it is a widely-used convention in the Ruby world. >> I'd even go as far as saying that it's quite pretty, as a >> typographical >> convention (not that other Ruby conventions are :-)) > > Also in Scheme. (I think the question mark more or less replaces the > 'p' suffix used in LISP). It does, and Ruby's idea of using the "?" postfix for boolean query (instead of an `is` or `is_` prefix) comes from there. From g.brandl at gmx.net Tue Aug 11 19:33:13 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 11 Aug 2009 19:33:13 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: Masklinn schrieb: > On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote: >> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: >> Steven D'Aprano writes: >>>> >>>> What makes you think that the question mark is a clue-in that a >>>> yes/no >>>> answer is expected? >>> >>> AFAIK it is a widely-used convention in the Ruby world. >>> I'd even go as far as saying that it's quite pretty, as a >>> typographical >>> convention (not that other Ruby conventions are :-)) >> >> Also in Scheme. (I think the question mark more or less replaces the >> 'p' suffix used in LISP). > It does, and Ruby's idea of using the "?" postfix for boolean query > (instead of an `is` or `is_` prefix) comes from there. But, and I believe that was Steven's point, it is no more than a convention. Omitting the "is_" prefix makes for a "question" that isn't unambiguously a yes/no question anymore in many cases. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From aaron.rubin at 4dtechnology.com Tue Aug 11 20:08:53 2009 From: aaron.rubin at 4dtechnology.com (Aaron Rubin) Date: Tue, 11 Aug 2009 11:08:53 -0700 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <200908081153.39137.steve@pearwood.info> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: <985db6080908111108l4dce84f4k5ef77afc2afde176@mail.gmail.com> On Fri, Aug 7, 2009 at 6:53 PM, Steven D'Aprano wrote: > On Sat, 8 Aug 2009 11:07:30 am Tennessee Leeuwenburg wrote: > > Hi all, > > Well, the thread's basically concluded, but I'm not sure if it's > > appropriate to just 'leave it hanging' seeing as I instigated the > > thread. So thanks all for the discussion points and I think we can > > wrap things up now. Please feel free to comment on the summary below > > if you think it doesn't have all the relevant details. > > > > To summarise the discussion for posterity: > > > > On the Plus Side: > > ? is part of English grammar, and is an intuitive clue-in that a > > yes/no, true/false response is expected as a result. > > What makes you think that the question mark is a clue-in that a yes/no > answer is expected? I think was the poster implied is that when a question mark is appended to any statement (or word for that matter), it becomes an existence or yes/no question. There are exceptions, but they need the qualifiers "when, who, where, why, etc." prepended (as you have proven below). Otherwise, they are Boolean. Method or function names (the subject of this thread) are typically nouns or verbs, making the point more poignant. > > When did you come up with the idea? > > Who would agree with you? > > Where can I see some evidence for that suggestion? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn at masklinn.net Tue Aug 11 21:05:54 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 11 Aug 2009 21:05:54 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: On 11 Aug 2009, at 19:33 , Georg Brandl wrote: > Masklinn schrieb: >> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote: >>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: >>> Steven D'Aprano writes: >>>>> >>>>> What makes you think that the question mark is a clue-in that a >>>>> yes/no >>>>> answer is expected? >>>> >>>> AFAIK it is a widely-used convention in the Ruby world. >>>> I'd even go as far as saying that it's quite pretty, as a >>>> typographical >>>> convention (not that other Ruby conventions are :-)) >>> >>> Also in Scheme. (I think the question mark more or less replaces the >>> 'p' suffix used in LISP). >> It does, and Ruby's idea of using the "?" postfix for boolean query >> (instead of an `is` or `is_` prefix) comes from there. > > But, and I believe that was Steven's point, it is no more than a > convention. It isn't, any more than Python's `_` prefix convention, or its `self` argument for that matter. I'd heard the Python community was pretty big on smart conventions but I might be wrong. From g.brandl at gmx.net Tue Aug 11 21:09:07 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 11 Aug 2009 21:09:07 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: Masklinn schrieb: > On 11 Aug 2009, at 19:33 , Georg Brandl wrote: >> Masklinn schrieb: >>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote: >>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: >>>> Steven D'Aprano writes: >>>>>> >>>>>> What makes you think that the question mark is a clue-in that a >>>>>> yes/no >>>>>> answer is expected? >>>>> >>>>> AFAIK it is a widely-used convention in the Ruby world. >>>>> I'd even go as far as saying that it's quite pretty, as a >>>>> typographical >>>>> convention (not that other Ruby conventions are :-)) >>>> >>>> Also in Scheme. (I think the question mark more or less replaces the >>>> 'p' suffix used in LISP). >>> It does, and Ruby's idea of using the "?" postfix for boolean query >>> (instead of an `is` or `is_` prefix) comes from there. >> >> But, and I believe that was Steven's point, it is no more than a >> convention. > It isn't, any more than Python's `_` prefix convention, or its `self` > argument for that matter. I'd heard the Python community was pretty > big on smart conventions but I might be wrong. But what is the advantage of "?" to "is_" then? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From masklinn at masklinn.net Tue Aug 11 21:56:24 2009 From: masklinn at masklinn.net (Masklinn) Date: Tue, 11 Aug 2009 21:56:24 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> On 11 Aug 2009, at 21:09 , Georg Brandl wrote: > Masklinn schrieb: >> On 11 Aug 2009, at 19:33 , Georg Brandl wrote: >>> Masklinn schrieb: >>>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote: >>>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: >>>>> Steven D'Aprano writes: >>>>>>> >>>>>>> What makes you think that the question mark is a clue-in that a >>>>>>> yes/no >>>>>>> answer is expected? >>>>>> >>>>>> AFAIK it is a widely-used convention in the Ruby world. >>>>>> I'd even go as far as saying that it's quite pretty, as a >>>>>> typographical >>>>>> convention (not that other Ruby conventions are :-)) >>>>> >>>>> Also in Scheme. (I think the question mark more or less replaces >>>>> the >>>>> 'p' suffix used in LISP). >>>> It does, and Ruby's idea of using the "?" postfix for boolean query >>>> (instead of an `is` or `is_` prefix) comes from there. >>> >>> But, and I believe that was Steven's point, it is no more than a >>> convention. >> It isn't, any more than Python's `_` prefix convention, or its `self` >> argument for that matter. I'd heard the Python community was pretty >> big on smart conventions but I might be wrong. > > But what is the advantage of "?" to "is_" then? > I think it's easier to spot due to 1. being a fairly rare character in programs (in languages where the ?: ternary doesn't exist anyway) and 2. being at the end of the call (though on Python method the () operator makes that much less interesting, if you aren't using a property). is_ is less easy to spot as it's a pretty unremarkable sequence of characters and at the middle of the call/line. From jnoller at gmail.com Wed Aug 12 01:50:17 2009 From: jnoller at gmail.com (Jesse Noller) Date: Tue, 11 Aug 2009 19:50:17 -0400 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> Message-ID: <4222a8490908111650i339f2a1bwba98cbb3f7652134@mail.gmail.com> On Tue, Aug 11, 2009 at 3:56 PM, Masklinn wrote: > On 11 Aug 2009, at 21:09 , Georg Brandl wrote: >> >> Masklinn schrieb: >>> >>> On 11 Aug 2009, at 19:33 , Georg Brandl wrote: >>>> >>>> Masklinn schrieb: >>>>> >>>>> On 11 Aug 2009, at 15:25 , Arnaud Delobelle wrote: >>>>>> >>>>>> On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: >>>>>> Steven D'Aprano writes: >>>>>>>> >>>>>>>> What makes you think that the question mark is a clue-in that a >>>>>>>> yes/no >>>>>>>> answer is expected? >>>>>>> >>>>>>> AFAIK it is a widely-used convention in the Ruby world. >>>>>>> I'd even go as far as saying that it's quite pretty, as a >>>>>>> typographical >>>>>>> convention (not that other Ruby conventions are :-)) >>>>>> >>>>>> Also in Scheme. (I think the question mark more or less replaces the >>>>>> 'p' suffix used in LISP). >>>>> >>>>> It does, and Ruby's idea of using the "?" postfix for boolean query >>>>> (instead of an `is` or `is_` prefix) comes from there. >>>> >>>> But, and I believe that was Steven's point, it is no more than a >>>> convention. >>> >>> It isn't, any more than Python's `_` prefix convention, or its `self` >>> argument for that matter. I'd heard the Python community was pretty >>> big on smart conventions but I might be wrong. >> >> But what is the advantage of "?" to "is_" then? >> > I think it's easier to spot due to 1. being a fairly rare character in > programs (in languages where the ?: ternary doesn't exist anyway) and 2. > being at the end of the call (though on Python method the () operator makes > that much less interesting, if you aren't using a property). > > is_ is less easy to spot as it's a pretty unremarkable sequence of > characters and at the middle of the call/line. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > Isn't this discussion moot? Guido already took it out back and sent it on it's way. jesse From steve at pearwood.info Wed Aug 12 02:21:19 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 12 Aug 2009 10:21:19 +1000 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> Message-ID: <200908121021.19273.steve@pearwood.info> On Wed, 12 Aug 2009 05:56:24 am Masklinn wrote: > >> I'd heard the Python community > >> was pretty big on smart conventions but I might be wrong. And would accepting or rejecting this proposal prove that we're big on smart conventions? > > But what is the advantage of "?" to "is_" then? > > I think it's easier to spot due to 1. being a fairly rare character > in programs (in languages where the ?: ternary doesn't exist anyway) It's certainly rare in Python, because it's illegal (except in strings and comments). It won't be rare if this convention is allowed. > and 2. being at the end of the call (though on Python method the () > operator makes that much less interesting, if you aren't using a > property). A thought experiment -- what if this convention had been in place in Python? Here is literally the first piece of code I looked at from doctest.py with a Yes/No test in it: def _normalize_module(module, depth=2): if inspect.module?(module): return module elif instance?(module, (str, unicode)): return __import__(module, globals(), locals(), ["*"]) elif module is None: return sys.modules[sys._getframe(depth).f_globals['__name__']] else: raise TypeError("Expected a module, string, or None") Notice the inconsistency -- method and function calls which return a bool have a question mark, but operators do not. I hope you're not going to suggest changing `is` to `is?`. But in any case, I don't think instance?() is any more readable than isinstance(), or module?() than ismodule(). Perhaps one advantage is that you can run your spell checker over your code with fewer false negatives. Deeper in doctest.py: def _failure_header(self, test, example): out = [self.DIVIDER] if test.filename: if test.lineno is not None and example.lineno is not None: lineno = test.lineno + example.lineno + 1 ... Allowing ? in identifiers won't change this method, but it illustrates another characteristic of Python (and Pythonic) code: the distinction between Yes/No booleans and Everything Else simply isn't that important. There's a method with two tests, one (the inner) is a conventional True or False test, but the other (the outer) is a Something or Nothing style test. For example, str.startswith(prefix) and str.endswith(suffix) could be modified to return the prefix/suffix found, or the empty string, and virtually all code using them should continue to work correctly. The exceptions being code that did something like this: if "a string".startswith("a") is True: ... but in my opinion, code that bad deserves to be broken. Elevating functions that return a Yes/No answer over other functions with extra punctuation is an unnecessary, and dare I say, un-pythonic, distinction. > is_ is less easy to spot as it's a pretty unremarkable sequence of > characters and at the middle of the call/line. The same argument could be made for len, str, dict, or any other word. You can't expect to understand code without reading the words. Adding another punctuation character identifiers isn't going to change that. -- Steven D'Aprano From masklinn at masklinn.net Wed Aug 12 09:32:01 2009 From: masklinn at masklinn.net (Masklinn) Date: Wed, 12 Aug 2009 09:32:01 +0200 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <200908121021.19273.steve@pearwood.info> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> <200908121021.19273.steve@pearwood.info> Message-ID: <7ED7005D-BDE5-4504-BE49-891A27686225@masklinn.net> On 12 Aug 2009, at 02:21 , Steven D'Aprano wrote: > On Wed, 12 Aug 2009 05:56:24 am Masklinn wrote: > >>>> I'd heard the Python community >>>> was pretty big on smart conventions but I might be wrong. > > And would accepting or rejecting this proposal prove that we're big on > smart conventions? > That's not what I wrote. >>> But what is the advantage of "?" to "is_" then? >> >> I think it's easier to spot due to 1. being a fairly rare character >> in programs (in languages where the ?: ternary doesn't exist anyway) > > It's certainly rare in Python, because it's illegal (except in strings > and comments). Which is exactly what I said, but thanks for repeating it. > Notice the inconsistency -- method and function calls which return a > bool have a question mark, but operators do not. I hope you're not > going to suggest changing `is` to `is?`. > That's an interesting point. From larry at hastings.org Wed Aug 12 16:32:01 2009 From: larry at hastings.org (Larry Hastings) Date: Wed, 12 Aug 2009 07:32:01 -0700 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <4A7BEF80.1090505@canterbury.ac.nz> <43c8685c0908071807w2ad02aebke54188ff3984b9b7@mail.gmail.com> <200908081153.39137.steve@pearwood.info> Message-ID: <4A82D261.5020404@hastings.org> Arnaud Delobelle wrote: > On 11 Aug 2009, at 11:43, Antoine Pitrou wrote: >> AFAIK it is a widely-used convention in the Ruby world. > Also in Scheme. And also in FORTH. See page 174 of the "Thinking FORTH" PDF, online version of the 1984 book. Or on this classic bumper sticker: FORTH LOVE? IF HONK THEN /larry/ From ceronman at gmail.com Wed Aug 12 20:02:22 2009 From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=) Date: Wed, 12 Aug 2009 13:02:22 -0500 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects Message-ID: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> Hello. There are some functions in the standard library that take an iterator/sequence as parameter and return an iterator. Most of them are in the itertools module and some are built in functions. I think they should be added as methods of the iterator objects as well. For example: itertools.takewhile(pred, seq) --> seq.takewhile(pred) sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, reverse) Rationale: ======= First, I know the rationale behind standalone functions like len as opposed to methods, but I think some iterator functions are special cases. I believe it is a common pattern to arrange these kind of functions in a pipe-filter system to perform complex queries over collections. The current system of standalone functions creates code difficult to read with nested parenthesis: ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))... It is very hard to see the pipe-filter flow in this code. The case is even worse because in some functions the order of the sequence argument and other parameters vary. For example: sorted takes the sequence first and then the key and reverse parameters while itertools.takewhile takes the predicate first and then the sequence. A few months ago, Donald 'Paddy' McCarthy suggested a pipe function [0] in the itertools module. But I believe using methods creates a better work flow, for example: seq.fun1(param).fun2(param).fun3(param).fun4(param) [0] http://mail.python.org/pipermail/python-ideas/2009-May/004877.html Examples: ======== Example 1. I want two groups of employees with the two best salaries: Using current functions: groups = itertools.islice(itertools.groupby(sorted(employees, key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2) Using methods: groups = employees.sorted(lambda e: e.salary, reverse=True).groupby(lambda e: e.salary).slice(None, 2) Example 2. I want the pairs of programmers assigned by task: Using current functions: pairs_tasks = itertools.izip(itertools.cycle(itertools.combinations(programmers, 2)), tasks) Using methods: pars_tasks = programmers.combinations(2).cycle().izip(tasks) Probably is better to keep izip as a standalone function: pars_tasks = itertools.izip(programmers.combinations(2).cycle(), tasks) Precedent: ======== There is another case where the pipe-filter pattern is seen in Python: strings. There are a lot of functions in the string module that take strings as argument and returns a string. Those functions could be arranged in a pipe-filter system. Python has a history of adding functions from the string module to the string objects. I think the same could be done with iterator functions. Example: We can use: parts = text.lower().strip().split() As opposed to: parts = string.split(string.strip(string.lower(text))) That's all for now. If you think this is a good idea we could elaborate on which methods should be added. Hope to see your comments. Manuel Cer?n. From guido at python.org Wed Aug 12 20:05:26 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 12 Aug 2009 11:05:26 -0700 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> Message-ID: No, no, no! This is intentional. The iterator protocol is intentionally minimal -- there are many implementations of it, and those implementations don't share code. Adding a large number of methods to it would require every iterator implementation to provide all those methods. This has been discussed before. Maybe it's time to write a negative PEP? Or add a paragraph explaining this to the original iterator protocol PEP? Or to the itertools module docs? --Guido 2009/8/12 Manuel Alejandro Cer?n Estrada : > Hello. > > There are some functions in the standard library that take an > iterator/sequence as parameter and return an iterator. Most of them > are in the itertools module and some are built in functions. I think > they should be added as methods of the iterator objects as well. For > example: > > itertools.takewhile(pred, seq) --> seq.takewhile(pred) > > sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, reverse) > > > Rationale: > ======= > > First, I know the rationale behind standalone functions like len as > opposed to methods, but I think some iterator functions are special > cases. I believe it is a common pattern to arrange these kind of > functions in a pipe-filter system to perform complex queries over > collections. The current system of standalone functions creates code > difficult to read with nested parenthesis: > > ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))... > > It is very hard to see the pipe-filter flow in this code. The case is > even worse because in some functions the order of the sequence > argument and other parameters vary. For example: sorted takes the > sequence first and then the key and reverse parameters while > itertools.takewhile takes the predicate first and then the sequence. > > A few months ago, Donald 'Paddy' McCarthy suggested a pipe function > [0] in the itertools module. But I believe using methods creates a > better work flow, for example: > > seq.fun1(param).fun2(param).fun3(param).fun4(param) > > [0] http://mail.python.org/pipermail/python-ideas/2009-May/004877.html > > > Examples: > ======== > > Example 1. I want two groups of employees with the two best salaries: > > Using current functions: > > groups = itertools.islice(itertools.groupby(sorted(employees, > key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2) > > Using methods: > > groups = employees.sorted(lambda e: e.salary, > reverse=True).groupby(lambda e: e.salary).slice(None, 2) > > Example 2. I want the pairs of programmers assigned by task: > > Using current functions: > > pairs_tasks = itertools.izip(itertools.cycle(itertools.combinations(programmers, > 2)), tasks) > > Using methods: > > pars_tasks = programmers.combinations(2).cycle().izip(tasks) > > Probably is better to keep izip as a standalone function: > > pars_tasks = itertools.izip(programmers.combinations(2).cycle(), tasks) > > > Precedent: > ======== > > There is another case where the pipe-filter pattern is seen in Python: > strings. There are a lot of functions in the string module that take > strings as argument and returns a string. Those functions could be > arranged in a pipe-filter system. Python has a history of adding > functions from the string module to the string objects. I think the > same could be done with iterator functions. > > Example: > > We can use: > > parts = text.lower().strip().split() > > As opposed to: > > parts = string.split(string.strip(string.lower(text))) > > > That's all for now. If you think this is a good idea we could > elaborate on which methods should be added. > > Hope to see your comments. > > Manuel Cer?n. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jkwon.work at gmail.com Wed Aug 12 22:11:47 2009 From: jkwon.work at gmail.com (Jae Kwon) Date: Wed, 12 Aug 2009 13:11:47 -0700 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> Message-ID: <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> these methods should be available in a common superclass (not the iterator protocol), and all or most python native iterators could subclass it. On Aug 12, 2009, at 11:05 AM, Guido van Rossum wrote: > No, no, no! > > This is intentional. The iterator protocol is intentionally minimal -- > there are many implementations of it, and those implementations don't > share code. Adding a large number of methods to it would require every > iterator implementation to provide all those methods. > > This has been discussed before. Maybe it's time to write a negative > PEP? Or add a paragraph explaining this to the original iterator > protocol PEP? Or to the itertools module docs? > > --Guido > > 2009/8/12 Manuel Alejandro Cer?n Estrada : >> Hello. >> >> There are some functions in the standard library that take an >> iterator/sequence as parameter and return an iterator. Most of them >> are in the itertools module and some are built in functions. I think >> they should be added as methods of the iterator objects as well. For >> example: >> >> itertools.takewhile(pred, seq) --> seq.takewhile(pred) >> >> sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, >> reverse) >> >> >> Rationale: >> ======= >> >> First, I know the rationale behind standalone functions like len as >> opposed to methods, but I think some iterator functions are special >> cases. I believe it is a common pattern to arrange these kind of >> functions in a pipe-filter system to perform complex queries over >> collections. The current system of standalone functions creates code >> difficult to read with nested parenthesis: >> >> ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))... >> >> It is very hard to see the pipe-filter flow in this code. The case is >> even worse because in some functions the order of the sequence >> argument and other parameters vary. For example: sorted takes the >> sequence first and then the key and reverse parameters while >> itertools.takewhile takes the predicate first and then the sequence. >> >> A few months ago, Donald 'Paddy' McCarthy suggested a pipe function >> [0] in the itertools module. But I believe using methods creates a >> better work flow, for example: >> >> seq.fun1(param).fun2(param).fun3(param).fun4(param) >> >> [0] http://mail.python.org/pipermail/python-ideas/2009-May/ >> 004877.html >> >> >> Examples: >> ======== >> >> Example 1. I want two groups of employees with the two best salaries: >> >> Using current functions: >> >> groups = itertools.islice(itertools.groupby(sorted(employees, >> key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2) >> >> Using methods: >> >> groups = employees.sorted(lambda e: e.salary, >> reverse=True).groupby(lambda e: e.salary).slice(None, 2) >> >> Example 2. I want the pairs of programmers assigned by task: >> >> Using current functions: >> >> pairs_tasks = >> itertools.izip(itertools.cycle(itertools.combinations(programmers, >> 2)), tasks) >> >> Using methods: >> >> pars_tasks = programmers.combinations(2).cycle().izip(tasks) >> >> Probably is better to keep izip as a standalone function: >> >> pars_tasks = itertools.izip(programmers.combinations(2).cycle(), >> tasks) >> >> >> Precedent: >> ======== >> >> There is another case where the pipe-filter pattern is seen in >> Python: >> strings. There are a lot of functions in the string module that take >> strings as argument and returns a string. Those functions could be >> arranged in a pipe-filter system. Python has a history of adding >> functions from the string module to the string objects. I think the >> same could be done with iterator functions. >> >> Example: >> >> We can use: >> >> parts = text.lower().strip().split() >> >> As opposed to: >> >> parts = string.split(string.strip(string.lower(text))) >> >> >> That's all for now. If you think this is a good idea we could >> elaborate on which methods should be added. >> >> Hope to see your comments. >> >> Manuel Cer?n. >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From guido at python.org Wed Aug 12 22:12:43 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 12 Aug 2009 13:12:43 -0700 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> Message-ID: That idea has also been considered and rejected, because of Python's commitment to duck typing. Really, please find the old discussion and read it rather than rehashing old broken ideas. 2009/8/12 Jae Kwon : > these methods should be available in a common superclass (not the iterator > protocol), and > all or most python native iterators could subclass it. > > On Aug 12, 2009, at 11:05 AM, Guido van Rossum wrote: > >> No, no, no! >> >> This is intentional. The iterator protocol is intentionally minimal -- >> there are many implementations of it, and those implementations don't >> share code. Adding a large number of methods to it would require every >> iterator implementation to provide all those methods. >> >> This has been discussed before. Maybe it's time to write a negative >> PEP? Or add a paragraph explaining this to the original iterator >> protocol PEP? Or to the itertools module docs? >> >> --Guido >> >> 2009/8/12 Manuel Alejandro Cer?n Estrada : >>> >>> Hello. >>> >>> There are some functions in the standard library that take an >>> iterator/sequence as parameter and return an iterator. Most of them >>> are in the itertools module and some are built in functions. I think >>> they should be added as methods of the iterator objects as well. For >>> example: >>> >>> itertools.takewhile(pred, seq) --> seq.takewhile(pred) >>> >>> sorted(seq, key=keyfun, reverse=reverse) --> seq.sorted(keyfun, reverse) >>> >>> >>> Rationale: >>> ======= >>> >>> First, I know the rationale behind standalone functions like len as >>> opposed to methods, but I think some iterator functions are special >>> cases. I believe it is a common pattern to arrange these kind of >>> functions in a pipe-filter system to perform complex queries over >>> collections. The current system of standalone functions creates code >>> difficult to read with nested parenthesis: >>> >>> ...fun4(param, fun3(param, fun2(param, fun1(param, seq))))... >>> >>> It is very hard to see the pipe-filter flow in this code. The case is >>> even worse because in some functions the order of the sequence >>> argument and other parameters vary. For example: sorted takes the >>> sequence first and then the key and reverse parameters while >>> itertools.takewhile takes the predicate first and then the sequence. >>> >>> A few months ago, Donald 'Paddy' McCarthy suggested a pipe function >>> [0] in the itertools module. But I believe using methods creates a >>> better work flow, for example: >>> >>> seq.fun1(param).fun2(param).fun3(param).fun4(param) >>> >>> [0] http://mail.python.org/pipermail/python-ideas/2009-May/004877.html >>> >>> >>> Examples: >>> ======== >>> >>> Example 1. I want two groups of employees with the two best salaries: >>> >>> Using current functions: >>> >>> groups = itertools.islice(itertools.groupby(sorted(employees, >>> key=lambda e: e.salary, reverse=True), lambda e: e.salary), None, 2) >>> >>> Using methods: >>> >>> groups = employees.sorted(lambda e: e.salary, >>> reverse=True).groupby(lambda e: e.salary).slice(None, 2) >>> >>> Example 2. I want the pairs of programmers assigned by task: >>> >>> Using current functions: >>> >>> pairs_tasks = >>> itertools.izip(itertools.cycle(itertools.combinations(programmers, >>> 2)), tasks) >>> >>> Using methods: >>> >>> pars_tasks = programmers.combinations(2).cycle().izip(tasks) >>> >>> Probably is better to keep izip as a standalone function: >>> >>> pars_tasks = itertools.izip(programmers.combinations(2).cycle(), tasks) >>> >>> >>> Precedent: >>> ======== >>> >>> There is another case where the pipe-filter pattern is seen in Python: >>> strings. There are a lot of functions in the string module that take >>> strings as argument and returns a string. Those functions could be >>> arranged in a pipe-filter system. Python has a history of adding >>> functions from the string module to the string objects. I think the >>> same could be done with iterator functions. >>> >>> Example: >>> >>> We can use: >>> >>> parts = text.lower().strip().split() >>> >>> As opposed to: >>> >>> parts = string.split(string.strip(string.lower(text))) >>> >>> >>> That's all for now. If you think this is a good idea we could >>> elaborate on which methods should be added. >>> >>> Hope to see your comments. >>> >>> Manuel Cer?n. >>> _______________________________________________ >>> Python-ideas mailing list >>> Python-ideas at python.org >>> http://mail.python.org/mailman/listinfo/python-ideas >>> >> >> >> >> -- >> --Guido van Rossum (home page: http://www.python.org/~guido/) >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ceronman at gmail.com Wed Aug 12 22:48:29 2009 From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=) Date: Wed, 12 Aug 2009 15:48:29 -0500 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> Message-ID: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> Hello again. El 12 de agosto de 2009 15:12, Guido van Rossum escribi?: > That idea has also been considered and rejected, because of Python's > commitment to duck typing. > > Really, please find the old discussion and read it rather than > rehashing old broken ideas. Sorry for getting back already discussed ideas. Finding old discussions can be very hard since the Python mailing lists have thousands of mails and some time you just don't use the correct keywords while searching. I understand why this has been rejected. However, I still think that the core idea of making the pipe-filter pattern and queries more readable is valuable. I'm thinking on implementing this as a third party wrapper module for itertools and others. An QueryIter wrapper class could help to this purpose and It would be very easy to implement. Could be something like this: pairs_tasks = IterQuery(programmers).combinations(2).cycle().izip(tasks) Methods would be simple wrappers: def combinations(self, repeat): return IterQuery(itertools.combinations(self, repeat)) Manuel. -- Manuel Alejandro Cer?n Estrada http://ceronman.freaks-unidos.net From jkwon.work at gmail.com Wed Aug 12 23:04:32 2009 From: jkwon.work at gmail.com (Jae Kwon) Date: Wed, 12 Aug 2009 14:04:32 -0700 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> Message-ID: <6BB6141B-3E1C-44EF-A3E7-C2825759B85A@gmail.com> +1 i tried to google "site:http://mail.python.org/pipermail/python-ideas/ itertools" and could not find anything. also, i like this idea. - Jae On Aug 12, 2009, at 1:48 PM, Manuel Alejandro Cer?n Estrada wrote: > Hello again. > > El 12 de agosto de 2009 15:12, Guido van Rossum > escribi?: >> That idea has also been considered and rejected, because of Python's >> commitment to duck typing. >> >> Really, please find the old discussion and read it rather than >> rehashing old broken ideas. > > Sorry for getting back already discussed ideas. Finding old > discussions can be very hard since the Python mailing lists have > thousands of mails and some time you just don't use the correct > keywords while searching. > > I understand why this has been rejected. However, I still think that > the core idea of making the pipe-filter pattern and queries more > readable is valuable. > > I'm thinking on implementing this as a third party wrapper module for > itertools and others. An QueryIter wrapper class could help to this > purpose and It would be very easy to implement. Could be something > like this: > > pairs_tasks = > IterQuery(programmers).combinations(2).cycle().izip(tasks) > > Methods would be simple wrappers: > > def combinations(self, repeat): > return IterQuery(itertools.combinations(self, repeat)) > > Manuel. > > -- > Manuel Alejandro Cer?n Estrada > http://ceronman.freaks-unidos.net From zuo at chopin.edu.pl Thu Aug 13 01:32:19 2009 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 13 Aug 2009 01:32:19 +0200 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> Message-ID: 12-08-2009 Manuel Alejandro Cer?n Estrada wrote: > I understand why this has been rejected. However, I still think that > the core idea of making the pipe-filter pattern and queries more > readable is valuable. > > I'm thinking on implementing this as a third party wrapper module for > itertools and others. An QueryIter wrapper class could help to this > purpose and It would be very easy to implement. Could be something > like this: > > pairs_tasks = IterQuery(programmers).combinations(2).cycle().izip(tasks) > > Methods would be simple wrappers: > > def combinations(self, repeat): > return IterQuery(itertools.combinations(self, repeat)) Maybe it could be generalized? Imagine funpipe() factory function (or with another name, doesn't matter at the moment) which could produce proxy objects with such a feature that e.g.: funpipe().fun1().fun2().fun3() -- would be effectively equivalent to: fun3(fun2(fun1(, ), ), ) -- where means 0 or more positional arguments, and means 0 or more positional and/or kw. arguments. I hope I'm clear about what's the idea. It could be in functools, itertools or as a built-in function... What do you think? *j -- Jan Kaliszewski (zuo) From zuo at chopin.edu.pl Thu Aug 13 01:40:14 2009 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 13 Aug 2009 01:40:14 +0200 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> Message-ID: 13-08-2009 Jan Kaliszewski wrote: > Maybe it could be generalized? Imagine funpipe() factory function (or > with another name, doesn't matter at the moment) which could produce > proxy objects with such a feature that e.g.: > > funpipe().fun1().fun2().fun3() PS. Maybe '|' operator would be here more easy to implement and more logical -- e.g.: funpipe().fun1() | fun2() | fun3() Treat it as a 'loud thinking'. *j -- Jan Kaliszewski (zuo) From george.sakkis at gmail.com Thu Aug 13 02:18:27 2009 From: george.sakkis at gmail.com (George Sakkis) Date: Wed, 12 Aug 2009 20:18:27 -0400 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> Message-ID: <91ad5bf80908121718u473ece31o93e56ea080557a6c@mail.gmail.com> 2009/8/12 Manuel Alejandro Cer?n Estrada : > Sorry for getting back already discussed ideas. Finding old > discussions can be very hard since the Python mailing lists have > thousands of mails and some time you just don't use the correct > keywords while searching. Check out the (long) "Builtin iterator type" thread at http://mail.python.org/pipermail/python-3000/2006-November/; I don't know if there are more. > I'm thinking on implementing this as a third party wrapper module for > itertools and others. An QueryIter wrapper class could help to this > purpose and It would be very easy to implement. Could be something > like this: > > pairs_tasks = IterQuery(programmers).combinations(2).cycle().izip(tasks) You can start from the recipe spawned off the 2006 thread: http://code.activestate.com/recipes/498272/. George From ede at mit.edu Thu Aug 13 03:25:24 2009 From: ede at mit.edu (Eric Eisner) Date: Thu, 13 Aug 2009 10:25:24 +0900 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> Message-ID: <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com> 2009/8/13 Manuel Alejandro Cer?n Estrada : > Using current functions: > > pairs_tasks = itertools.izip(itertools.cycle(itertools.combinations(programmers, > 2)), tasks) > > Using methods: > > pars_tasks = programmers.combinations(2).cycle().izip(tasks) As an aside, I've always thought that Python tends to use functions because Guido's native language is a head-initial language, where the action precedes the object. On the other hand, Ruby tends to use methods because its creator, Yukihiro Matsumoto, speaks a head-final language, where the object preceds the action. So in conclusion, functions are more pythonic. -Eric From tjreedy at udel.edu Thu Aug 13 07:08:11 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 13 Aug 2009 14:08:11 +0900 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> Message-ID: Manuel Alejandro Cer?n Estrada wrote: > I'm thinking on implementing this as a third party wrapper module for > itertools and others. For something that is longer than a recipe, that is definitely the way to go. You are free to do it the way you want to and free to try variations. People who agree with you can give feedback based on real experience. tjr From ncoghlan at gmail.com Thu Aug 13 11:31:59 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 13 Aug 2009 19:31:59 +1000 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <6BB6141B-3E1C-44EF-A3E7-C2825759B85A@gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <3C46DA38-E01A-42AD-8A0C-70144A8473DA@gmail.com> <796874fc0908121348u2e31d466w701c1550931a05c6@mail.gmail.com> <6BB6141B-3E1C-44EF-A3E7-C2825759B85A@gmail.com> Message-ID: <4A83DD8F.5000808@gmail.com> Jae Kwon wrote: > +1 > > i tried to google "site:http://mail.python.org/pipermail/python-ideas/ > itertools" and could not find anything. python-ideas is a relatively young list - older discussions are all on python-dev. It's usually best to search both lists anyway, since less "out there" suggestions are often brought up on python-dev rather than here. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From janssen at parc.com Thu Aug 13 18:23:08 2009 From: janssen at parc.com (Bill Janssen) Date: Thu, 13 Aug 2009 09:23:08 PDT Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com> Message-ID: <10531.1250180588@parc.com> Interesting thread. Is Python procedural (function-oriented)? My opinion is that it was, when it started out, but isn't really any more, due to the growth of an object-oriented style, in which data, state, is hidden behind an RPC-like interface, finally officially "blessed" in Python 3 with the appearance of abstract base classes. In an alternate history, a more procedural idea of Python could have grown generic functions, a la Common Lisp. Should the iterator interface be "wide" (lots of utility methods), either by extending it directly or by producing a wide superclass? Probably not. Most languages with wide interfaces need them because they only have "types", or they have objects, but only single inheritance. Java is only half-way there, with its notion of interfaces (most of which are very wide (too wide), as well). But in Python, you can produce mixins, which are fully implemented partial classes, intended to be mixed together with other classes for actual use. So you can produce lots of "narrow" interfaces, and mix them in as needed. Keeping individual interfaces narrow also makes it easier to do duck-typing. Unfortunately, it seems to me that Python currently makes mixins somewhat clumsy to use. We could use a low-weight way of making a class which is a mix of three or four mixins, much as "lambda" provides a low-weight way of producing a small function. Something like a = class(Mixin1, Mixin2, Mixin3)() a.mixin1_method_foo() a.mixin3_method_bar() and so on. Bill From zuo at chopin.edu.pl Thu Aug 13 20:32:36 2009 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Thu, 13 Aug 2009 20:32:36 +0200 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <10531.1250180588@parc.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com> <10531.1250180588@parc.com> Message-ID: 13-08-2009 Bill Janssen wrote: > a = class(Mixin1, Mixin2, Mixin3)() > a.mixin1_method_foo() > a.mixin3_method_bar() Something similar already is possible out of the box: a = type('a', (Mixin1, Mixin2, Mixin3), {})() Mixin1.foo(a) Mixin3.bar(a) You could also make creating of such objects more compact by using prepared factory function, e.g. def makeclass(*base_classes, **kwargs): cls_name = kwargs.get('name', 'noname') cls_attrs = kwargs.get('attrs', {}) return type(cls_name, base_classes, cls_attrs) Then you have exactly what you wanted: a = makeclass(Mixin1, Mixin2, Mixin3)() Regards, *j -- Jan Kaliszewski (zuo) From dreamingforward at gmail.com Fri Aug 14 03:04:16 2009 From: dreamingforward at gmail.com (average) Date: Thu, 13 Aug 2009 20:04:16 -0500 Subject: [Python-ideas] What about allowing '?' in method names? In-Reply-To: <200908121021.19273.steve@pearwood.info> References: <43c8685c0908051801o5555b08ye1e295c26b1fe06a@mail.gmail.com> <27A72DFD-D503-4BB7-B7C5-CA1FE046F644@masklinn.net> <200908121021.19273.steve@pearwood.info> Message-ID: <913f9f570908131804x3f11e443ie4fa8b10b511eee9@mail.gmail.com> > And would accepting or rejecting this proposal prove that we're big on > smart conventions? > >> > But what is the advantage of "?" to "is_" then? >> >> I think it's easier to spot due to 1. being a fairly rare character >> in programs (in languages where the ?: ternary doesn't exist anyway) I know for myself, it would help me mentally organize all the different methods within classes. To me its the sort of syntactical convention similar to using [] brackets to set off a conceptual item as opposed to using parens and making a standard, more generic function call. Or like the elation of when the user finally gets the point of whitespace and case conventions in python -- they've just improved conceptual separation (and thus readability, maintainability) by several-fold by simply agreeing to a widely-applicable convention. The argument "Really, blah blah how difficult is it to just use is_ ?" is insufficient. To me it goes along with what's been learned with independant component analysis (ICA)--elongate the distance between information-independant items while minimizing the effort to keep related information grouped together. (ICA is the same tech that allows one to separate conversations at a cocktail party.) This pretty much encapsulates the evolution of programming languages. \> A thought experiment -- what if this convention had been in place in > Python? Here is literally the first piece of code I looked at from > doctest.py with a Yes/No test in it: > > def _normalize_module(module, depth=2): > ? ?if inspect.module?(module): > ? ? ? ?return module > ? ?elif instance?(module, (str, unicode)): > ? ? ? ?return __import__(module, globals(), locals(), ["*"]) > ? ?elif module is None: > ? ? ? ?return sys.modules[sys._getframe(depth).f_globals['__name__']] > ? ?else: > ? ? ? ?raise TypeError("Expected a module, string, or None") Note that any example on existing code will only be partially useful as the names chosen were chosen before an option that provided greater conceptual separation was available. > Notice the inconsistency -- method and function calls which return a > bool have a question mark, but operators do not. I hope you're not > going to suggest changing `is` to `is?`. That would be redundant. In any case, I would take this suggestion for an additional convention even further by suggesting that methods which have side effects or mutate the object (list.sorted) should have an exclaimation point, getting rid of ambiguous usages like list.sort and list.sorted which are counter-intuitive--keep list.sort! for an in-place, mutating method name, for example, and list.sort to return a sorted list. Marcos From prozacgod at gmail.com Fri Aug 14 08:22:07 2009 From: prozacgod at gmail.com (Prozacgod) Date: Fri, 14 Aug 2009 01:22:07 -0500 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept Message-ID: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> A while back i had learned about the 'with' statement and at first I naively thought that it worked similarly to a with statement I was familiar with, from delphi/pascal - obviously it doesn't and I was instantly hit with the idea of how useful it would be to have a keyword that could use a namespace from an object, hitting the familiar __getattr__ functions and related. the keyword I proposed would be 'interrogate' (which is rather long, could use another one like 'using' or something) but.. thats not really important to me, the idea is, so class test(): def __init__(self): self.x = 42 foo = test() bar = test() x = 33 bar.x = 22 interrogate foo, bar: print x ----- output is 42 since x is found inside of foo before bar, and locals would be interrogated last, but a more usefull example would be .. class test(): def __getattr__(self, name): if len(name) == 1 and (ord(name) in range(ord('a'), ord('z'))) : return ord(name) test_ns = test() interrogate test_ns: print a ---- output is 97 using the above example lexical closures may be awkward to implement in the language itself, this is a bit of a contrived example.. but.. eh.. # if I understand python enough, then I believe that when we enter this block, and create variables, they go out of scope, # and there are no exceptions to this rule, ever so .. in order to implement mine obviously scope would be the same # mechanism so I have oh_bother declared in the local namespace first oh_bother = None interrogate test_ns: def geronimo(): return z oh_bother = geronimo print oh_bother() --- output is 122 -- -Prozacgod "Prozac may heal the mind, but friends can mend the soul" -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri Aug 14 09:54:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Aug 2009 17:54:03 +1000 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> Message-ID: <200908141754.03678.steve@pearwood.info> On Fri, 14 Aug 2009 04:22:07 pm Prozacgod wrote: > A while back i had learned about the 'with' statement and at first I > naively thought that it worked similarly to a with statement I was > familiar with, from delphi/pascal - obviously it doesn't and I was > instantly hit with the idea of how useful it would be to have a > keyword that could use a namespace from an object, hitting the > familiar __getattr__ functions and related. What is the motivation? Is it just to reduce typing? Being an ex-Pascal coder myself, I originally missed having a Pascal-like 'with' statement too. But now I'm not so sure it is needed. I know this proposal (under the name 'with') has been discussed before, but I can't find a PEP for it... ah, here you go, it was in the FAQs: http://www.python.org/doc/faq/general/#why-doesn-t-python-have-a-with-statement-for-attribute-assignments C-# doesn't have one either: http://msdn.microsoft.com/en-us/vcsharp/aa336816.aspx I don't think there are any problems with a Pascal-style 'with' statement that couldn't be overcome, but I don't think the benefit is great enough to create a new keyword for it. Can you explain in more detail why this proposed feature is useful? -- Steven D'Aprano From steve at pearwood.info Fri Aug 14 09:58:51 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Aug 2009 17:58:51 +1000 Subject: [Python-ideas] Adding some standalone iterator/sequence functions as methods of the iterator objects In-Reply-To: <10531.1250180588@parc.com> References: <796874fc0908121102h6581c496x9c9dc1d987b0a331@mail.gmail.com> <42ef4ee0908121825t4c60c73fqbe67e23ed8615c7e@mail.gmail.com> <10531.1250180588@parc.com> Message-ID: <200908141758.51760.steve@pearwood.info> On Fri, 14 Aug 2009 02:23:08 am Bill Janssen wrote: > Interesting thread. > > Is Python procedural (function-oriented)? My opinion is that it was, > when it started out, but isn't really any more, Python has always been object-oriented. I don't have direct experience with the releases before version 1.5, but all data in Python were objects then, and they remain objects now. Python has *also* always been procedural: Python supports a mix of OO and procedural programming very naturally. It would be difficult to write a non-trivial program using "pure" OO or "pure" procedural styles alone. Python has also always supported (albeit weakly) a functional style. Although it doesn't make the guarantee of "no side-effects" that purely functional languages do, Python has provided functional tools like filter(), map() and reduce() since the 1.x series, and with more and more iterator-based code, Python is arguably becoming more functional rather than less. [...] > Unfortunately, it seems to me that Python currently makes mixins > somewhat clumsy to use. We could use a low-weight way of making a > class which is a mix of three or four mixins, much as "lambda" > provides a low-weight way of producing a small function. Something > like > > a = class(Mixin1, Mixin2, Mixin3)() > a.mixin1_method_foo() > a.mixin3_method_bar() What's wrong with a.foo() a.bar() ? As far as I know, the caller shouldn't need to know or care where a inherits the method `foo` from, or even if it is inherited. On the rare occasion the caller does want to specify which base class' method to call, you can do it with: Mixin1.foo(a) Mixin3.bar(a) -- Steven D'Aprano From ncoghlan at gmail.com Fri Aug 14 10:11:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 14 Aug 2009 18:11:09 +1000 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <200908141754.03678.steve@pearwood.info> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908141754.03678.steve@pearwood.info> Message-ID: <4A851C1D.5020507@gmail.com> Steven D'Aprano wrote: > I don't think there are any problems with a Pascal-style 'with' > statement that couldn't be overcome, but I don't think the benefit is > great enough to create a new keyword for it. Can you explain in more > detail why this proposed feature is useful? Also, if you just want to be able to chain multiple namespaces together, you can do that by implementing an appropriate class with a custom __getattr__ method. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ilya.nikokoshev at gmail.com Fri Aug 14 11:25:53 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Fri, 14 Aug 2009 13:25:53 +0400 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <4A851C1D.5020507@gmail.com> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908141754.03678.steve@pearwood.info> <4A851C1D.5020507@gmail.com> Message-ID: Even if for some reason you needed a statement similar to Pascal's "with" (and no, I don't know any example where this could be useful since you'll lose access to anything other then the class), you could implement it with existing syntax: class Test: '''Class to be interrogated.''' def __init__(self, value): self.value = value test = Test(10) class getattrs(dict): '''An auxiliary class.''' def __init__(self, instance): self.instance = instance def __getitem__(self, name): return getattr(self.instance, name) def __setitem__(self, name, value): return setattr(self.instance, name, value) # interrogate test: value += 5 exec('value += 5', getattrs(test)) print(test.value) # prints 15. On Fri, Aug 14, 2009 at 12:11 PM, Nick Coghlan wrote: > Steven D'Aprano wrote: >> I don't think there are any problems with a Pascal-style 'with' >> statement that couldn't be overcome, but I don't think the benefit is >> great enough to create a new keyword for it. Can you explain in more >> detail why this proposed feature is useful? > > Also, if you just want to be able to chain multiple namespaces together, > you can do that by implementing an appropriate class with a custom > __getattr__ method. > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From ilya.nikokoshev at gmail.com Fri Aug 14 11:35:55 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Fri, 14 Aug 2009 13:35:55 +0400 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908141754.03678.steve@pearwood.info> <4A851C1D.5020507@gmail.com> Message-ID: Sorry, it's actually even easier; interrogate() is a one-liner: class Test: '''Class to be interrogated.''' def __init__(self, value): self.value = value test = Test(10) def interrogate(what, how): exec(how, what.__dict__) # interrogate test: value += 5 interrogate(test, 'value += 5') # interrogate test: print(value) interrogate(test, 'print(value)') On Fri, Aug 14, 2009 at 1:25 PM, ilya wrote: > Even if for some reason you needed a statement similar to Pascal's > "with" (and no, I don't know any example where this could be useful > since you'll lose access to anything other then the class), you could > implement it with existing syntax: > > class Test: > ? ?'''Class to be interrogated.''' > ? ?def __init__(self, value): > ? ? ? ?self.value = value > > test = Test(10) > > class getattrs(dict): > ? ?'''An auxiliary class.''' > ? ?def __init__(self, instance): > ? ? ? ?self.instance = instance > ? ?def __getitem__(self, name): > ? ? ? ?return getattr(self.instance, name) > ? ?def __setitem__(self, name, value): > ? ? ? ?return setattr(self.instance, name, value) > > # interrogate test: value += 5 > exec('value += 5', getattrs(test)) > > print(test.value) > # prints 15. > > > On Fri, Aug 14, 2009 at 12:11 PM, Nick Coghlan wrote: >> Steven D'Aprano wrote: >>> I don't think there are any problems with a Pascal-style 'with' >>> statement that couldn't be overcome, but I don't think the benefit is >>> great enough to create a new keyword for it. Can you explain in more >>> detail why this proposed feature is useful? >> >> Also, if you just want to be able to chain multiple namespaces together, >> you can do that by implementing an appropriate class with a custom >> __getattr__ method. >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia >> --------------------------------------------------------------- >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > From steve at pearwood.info Fri Aug 14 13:17:23 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 14 Aug 2009 21:17:23 +1000 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> Message-ID: <200908142117.24072.steve@pearwood.info> On Fri, 14 Aug 2009 07:35:55 pm ilya wrote: > Sorry, it's actually even easier; interrogate() is a one-liner: > > class Test: > '''Class to be interrogated.''' > def __init__(self, value): > self.value = value > > test = Test(10) > > def interrogate(what, how): > exec(how, what.__dict__) Apart from the security implications of exec(), it also takes a fairly hefty performance hit. In Python 2.6: >>> from timeit import Timer >>> setup = 'from __main__ import test, interrogate' >>> Timer("interrogate(test, 'value += 5')", setup).repeat() [18.503479957580566, 18.218451023101807, 18.218581914901733] >>> Timer("test.value += 5", setup).repeat() [0.33056807518005371, 0.33118104934692383, 0.33114814758300781] That's a factor of 55 times slower -- not precisely an optimization. -- Steven D'Aprano From prozacgod at gmail.com Fri Aug 14 21:25:29 2009 From: prozacgod at gmail.com (Prozacgod) Date: Fri, 14 Aug 2009 14:25:29 -0500 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <200908141754.03678.steve@pearwood.info> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908141754.03678.steve@pearwood.info> Message-ID: <90df837e0908141225n2ed9c6f3v68ff3cab55b58e87@mail.gmail.com> The motivation came up in this thread on python-forum.org http://www.python-forum.org/pythonforum/viewtopic.php?f=2&t=11606 My main motivation most likely is just that it would open up some very interesting meta-programing or dynamic programming opportunities, variables that exists entirely programmatically without the prefix of object_instance.foo, surely just syntactical sugar to some (or even most) - I have found it to be valuable syntax. just like the current with statement bypasses some syntactical nusances of try: except: .. this would bypass the nusance of inst.x inst.y inst.z ... After posting last night I realized A pitfall of implementing this - would be interrogate(foo, bar): xylophone = 1024 using this reference - .. http://docs.python.org/reference/datamodel.html#customizing-attribute-access __setattr__ has no current way to fall through, so foo would get the setattr event, and not be able to allow interrogate to hand it to bar. I posit that raising the AttributeError exception would be sufficient, for the hand off to occur, and if foo, then bar both did it, then one of two ways to handle it would be the current scope variables would get it, (eg it would be __del__ after the end of the block) or putting the new variable in the parent namespace, since it is somewhat assumed that locals would follow after bar On Fri, Aug 14, 2009 at 2:54 AM, Steven D'Aprano wrote: > On Fri, 14 Aug 2009 04:22:07 pm Prozacgod wrote: > > A while back i had learned about the 'with' statement and at first I > > naively thought that it worked similarly to a with statement I was > > familiar with, from delphi/pascal - obviously it doesn't and I was > > instantly hit with the idea of how useful it would be to have a > > keyword that could use a namespace from an object, hitting the > > familiar __getattr__ functions and related. > > What is the motivation? Is it just to reduce typing? > > Being an ex-Pascal coder myself, I originally missed having a > Pascal-like 'with' statement too. But now I'm not so sure it is needed. > > I know this proposal (under the name 'with') has been discussed before, > but I can't find a PEP for it... ah, here you go, it was in the FAQs: > > > http://www.python.org/doc/faq/general/#why-doesn-t-python-have-a-with-statement-for-attribute-assignments > > C-# doesn't have one either: > http://msdn.microsoft.com/en-us/vcsharp/aa336816.aspx > > I don't think there are any problems with a Pascal-style 'with' > statement that couldn't be overcome, but I don't think the benefit is > great enough to create a new keyword for it. Can you explain in more > detail why this proposed feature is useful? > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- -Prozacgod "Prozac may heal the mind, but friends can mend the soul" -------------- next part -------------- An HTML attachment was scrubbed... URL: From prozacgod at gmail.com Fri Aug 14 21:46:17 2009 From: prozacgod at gmail.com (Prozacgod) Date: Fri, 14 Aug 2009 14:46:17 -0500 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> Message-ID: <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com> I just read this inside of the C# reasoning, and this pretty much kinda says, look its to difficult... This is a good reason "Another approach is to push a scope and make the property hide the local variable, but then there's no way to refer to the local without adding some escape syntax." bascially, they added the dot syntax to acces variables that are intended to be directed towards object namespace. Which I do agree helps with disambiguation, foo = 32 interrogate(test_ns): print .foo print foo But don't find all that paletable. My thoughts on this would be to cover the parent local namespace, and even globals with your object, the object gets it first everytime, and if it raised an exception, then the normal method would be implemented. If it were important to have a variable that was named the same, then access it outide the interrogate block or otherwise.. But if you were a developer and working on writing a program, knowing you wanted to use interrogate, then wouldn't you be wise to choose different variable names? I think I would. Also doing this is kinda kludgy feeling to me.. function(args).dict[index][index].a = 21 function(args).dict[index][index].b = 42 function(args).dict[index][index].c = 63 write this: ref = function(args).dict[index][index] ref.a = 21 ref.b = 42 ref.c = 63 even though it is optimised, and works well, this feels more like the programmer adapting to the environment instead of the environment working for the programmer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ilya.nikokoshev at gmail.com Fri Aug 14 22:15:19 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Sat, 15 Aug 2009 00:15:19 +0400 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com> Message-ID: Well, > foo = 32 > interrogate(test_ns): > print .foo > print foo > can be changed to foo = 32 _ = test_ns print _.foo print foo which actually has less symbols and doesn't have any ambiguity as for the meaning of the new statement! > But don't find all that paletable. My thoughts on this would be to cover > the parent local namespace, and even globals with your object, the object If a suggestion is to resolve names into several dictionaries, I don't think this is practical in any way! As the source you cited correctly explains, there is an irreparable problem. Suppose I write foo = 32 interrogate(test_ns): print bar # test_ns.bar print foo # global foo Then some time later I add global variable bar somewhere in the module and the program is mysteriously broken! The same thing happens if you first consult test_ns.__dict__ -- somebody can always add foo to test_ns, its parent or *even object class*. So I don't think there's a good way to define semantics of 'interrogate' without some kind of dot-syntax, which is debated for Python for quite a long time without success. On Fri, Aug 14, 2009 at 11:46 PM, Prozacgod wrote: > I just read this inside of the C# reasoning, and this pretty much kinda > says, look its to difficult... > > This is a good reason "Another approach is to push a scope and make the > property hide the local variable, but then there's no way to refer to the > local without adding some escape syntax." > > bascially, they added the dot syntax to acces variables that are intended to > be directed towards object namespace.? Which I do agree helps with > disambiguation, > > foo = 32 > interrogate(test_ns): > ? print .foo > ? print foo > > But don't find all that paletable.? My thoughts on this would be to cover > the parent local namespace, and even globals with your object, the object > gets it first everytime, and if it raised an exception, then the normal > method would be implemented.? If it were important to have a variable that > was named the same, then access it outide the interrogate block or > otherwise..? But if you were a developer and working on writing a program, > knowing you wanted to use interrogate, then wouldn't you be wise to choose > different variable names?? I think I would. > > Also doing this is kinda kludgy feeling to me.. > > function(args).dict[index][index].a = 21 > function(args).dict[index][index].b = 42 > function(args).dict[index][index].c = 63 > > write this: > > ref = function(args).dict[index][index] > ref.a = 21 > ref.b = 42 > ref.c = 63 > > even though it is optimised, and works well, this feels more like the > programmer adapting to the environment instead of the environment working > for the programmer. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From ilya.nikokoshev at gmail.com Fri Aug 14 22:29:11 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Sat, 15 Aug 2009 00:29:11 +0400 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <200908142117.24072.steve@pearwood.info> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908142117.24072.steve@pearwood.info> Message-ID: That's a useful statistics, but the bottleneck is **only** because of parsing 'value +=5'. Here's how I time it: # insert my old program here... from timeit import Timer from codeop import compile_command def timing(something): setup = 'from __main__ import test, interrogate, command, inc5' best = sorted(Timer(something, setup).repeat(3, 1000))[0] print('{0!r} -> {1:.3} ms'.format(something, best)) print('# test.value =', test.value) command = ''' for i in range(10): value += 1 ''' inc5 = compile_command(command) timing("interrogate(test, command)") timing(command.replace('value', 'test.value')) timing("interrogate(test, inc5)") Result: 15 'interrogate(test, command)' -> 0.0908 ms # test.value = 30015 '\nfor i in range(10):\n test.value += 1\n' -> 0.00408 ms # test.value = 60015 'interrogate(test, inc5)' -> 0.00469 ms # test.value = 90015 so interrogate() with additional precompiling introduces very little overhead. Though I agree it's inconvenient to write functions as strings; I think someone smarter than me can find a way to do it like a regular function call. On Fri, Aug 14, 2009 at 3:17 PM, Steven D'Aprano wrote: > On Fri, 14 Aug 2009 07:35:55 pm ilya wrote: >> Sorry, it's actually even easier; interrogate() is a one-liner: >> >> class Test: >> ? ?'''Class to be interrogated.''' >> ? ?def __init__(self, value): >> ? ? ? ?self.value = value >> >> test = Test(10) >> >> def interrogate(what, how): >> ? ?exec(how, what.__dict__) > > Apart from the security implications of exec(), it also takes a fairly > hefty performance hit. In Python 2.6: > >>>> from timeit import Timer >>>> setup = 'from __main__ import test, interrogate' >>>> Timer("interrogate(test, 'value += 5')", setup).repeat() > [18.503479957580566, 18.218451023101807, 18.218581914901733] >>>> Timer("test.value += 5", setup).repeat() > [0.33056807518005371, 0.33118104934692383, 0.33114814758300781] > > That's a factor of 55 times slower -- not precisely an optimization. > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From prozacgod at gmail.com Sat Aug 15 01:53:41 2009 From: prozacgod at gmail.com (Prozacgod) Date: Fri, 14 Aug 2009 18:53:41 -0500 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908142117.24072.steve@pearwood.info> Message-ID: <90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com> This makes sense, obviously you should compile it first, which when executed should execute exactly the same as the original context python. On Fri, Aug 14, 2009 at 3:29 PM, ilya wrote: > That's a useful statistics, but the bottleneck is **only** because of > parsing 'value +=5'. > > Here's how I time it: > > # insert my old program here... > > > from timeit import Timer > from codeop import compile_command > > def timing(something): > setup = 'from __main__ import test, interrogate, command, inc5' > best = sorted(Timer(something, setup).repeat(3, 1000))[0] > print('{0!r} -> {1:.3} ms'.format(something, best)) > print('# test.value =', test.value) > > command = ''' > for i in range(10): > value += 1 > ''' > > inc5 = compile_command(command) > > timing("interrogate(test, command)") > timing(command.replace('value', 'test.value')) > timing("interrogate(test, inc5)") > > Result: > > 15 > 'interrogate(test, command)' -> 0.0908 ms > # test.value = 30015 > '\nfor i in range(10):\n test.value += 1\n' -> 0.00408 ms > # test.value = 60015 > 'interrogate(test, inc5)' -> 0.00469 ms > # test.value = 90015 > > so interrogate() with additional precompiling introduces very little > overhead. Though I agree it's inconvenient to write functions as > strings; I think someone smarter than me can find a way to do it like > a regular function call. > > > > On Fri, Aug 14, 2009 at 3:17 PM, Steven D'Aprano > wrote: > > On Fri, 14 Aug 2009 07:35:55 pm ilya wrote: > >> Sorry, it's actually even easier; interrogate() is a one-liner: > >> > >> class Test: > >> '''Class to be interrogated.''' > >> def __init__(self, value): > >> self.value = value > >> > >> test = Test(10) > >> > >> def interrogate(what, how): > >> exec(how, what.__dict__) > > > > Apart from the security implications of exec(), it also takes a fairly > > hefty performance hit. In Python 2.6: > > > >>>> from timeit import Timer > >>>> setup = 'from __main__ import test, interrogate' > >>>> Timer("interrogate(test, 'value += 5')", setup).repeat() > > [18.503479957580566, 18.218451023101807, 18.218581914901733] > >>>> Timer("test.value += 5", setup).repeat() > > [0.33056807518005371, 0.33118104934692383, 0.33114814758300781] > > > > That's a factor of 55 times slower -- not precisely an optimization. > > > > > > > > -- > > Steven D'Aprano > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > http://mail.python.org/mailman/listinfo/python-ideas > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- -Prozacgod "Prozac may heal the mind, but friends can mend the soul" -------------- next part -------------- An HTML attachment was scrubbed... URL: From prozacgod at gmail.com Sat Aug 15 01:59:11 2009 From: prozacgod at gmail.com (Prozacgod) Date: Fri, 14 Aug 2009 18:59:11 -0500 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com> Message-ID: <90df837e0908141659k32dffb8fod69bf647736193f4@mail.gmail.com> Oh .. good call, you right, this would be difficult to observe, find, and correct >foo = 32 >interrogate(test_ns): > print bar # test_ns.bar > print foo # global foo >Then some time later I add global variable bar somewhere in the module >and the program is mysteriously broken! The same thing happens if you >first consult test_ns.__dict__ -- somebody can always add foo to >test_ns, its parent or *even object class*. In delphi/pascal they left this up to the developer, so you're supposed to divine that a new property was made on a class, to me that is a reasonable trade-off, I could choose to only use this syntax on code that I'm writing and supporting, or realize something could break - BUT I also understand the mentality of - it shouldn't randomly break ever, which is a reasonable supposition of code you write. After this discussion I'm wondering why it ever ended up in any language to begin with? Seems like it opens up to many issues to the developers using the language and the language implementors. On Fri, Aug 14, 2009 at 3:15 PM, ilya wrote: > Well, > > > foo = 32 > > interrogate(test_ns): > > print .foo > > print foo > > > > can be changed to > > foo = 32 > _ = test_ns > print _.foo > print foo > > which actually has less symbols and doesn't have any ambiguity as for > the meaning of the new statement! > > > But don't find all that paletable. My thoughts on this would be to cover > > the parent local namespace, and even globals with your object, the object > > If a suggestion is to resolve names into several dictionaries, I don't > think this is practical in any way! As the source you cited correctly > explains, there is an irreparable problem. Suppose I write > > foo = 32 > interrogate(test_ns): > print bar # test_ns.bar > print foo # global foo > > Then some time later I add global variable bar somewhere in the module > and the program is mysteriously broken! The same thing happens if you > first consult test_ns.__dict__ -- somebody can always add foo to > test_ns, its parent or *even object class*. > > So I don't think there's a good way to define semantics of > 'interrogate' without some kind of dot-syntax, which is debated for > Python for quite a long time without success. > > > On Fri, Aug 14, 2009 at 11:46 PM, Prozacgod wrote: > > I just read this inside of the C# reasoning, and this pretty much kinda > > says, look its to difficult... > > > > This is a good reason "Another approach is to push a scope and make the > > property hide the local variable, but then there's no way to refer to the > > local without adding some escape syntax." > > > > bascially, they added the dot syntax to acces variables that are intended > to > > be directed towards object namespace. Which I do agree helps with > > disambiguation, > > > > foo = 32 > > interrogate(test_ns): > > print .foo > > print foo > > > > But don't find all that paletable. My thoughts on this would be to cover > > the parent local namespace, and even globals with your object, the object > > gets it first everytime, and if it raised an exception, then the normal > > method would be implemented. If it were important to have a variable > that > > was named the same, then access it outide the interrogate block or > > otherwise.. But if you were a developer and working on writing a > program, > > knowing you wanted to use interrogate, then wouldn't you be wise to > choose > > different variable names? I think I would. > > > > Also doing this is kinda kludgy feeling to me.. > > > > function(args).dict[index][index].a = 21 > > function(args).dict[index][index].b = 42 > > function(args).dict[index][index].c = 63 > > > > write this: > > > > ref = function(args).dict[index][index] > > ref.a = 21 > > ref.b = 42 > > ref.c = 63 > > > > even though it is optimised, and works well, this feels more like the > > programmer adapting to the environment instead of the environment working > > for the programmer. > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas at python.org > > http://mail.python.org/mailman/listinfo/python-ideas > > > > > -- -Prozacgod "Prozac may heal the mind, but friends can mend the soul" -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Aug 15 04:02:29 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Aug 2009 12:02:29 +1000 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908142117.24072.steve@pearwood.info> Message-ID: <200908151202.29948.steve@pearwood.info> On Sat, 15 Aug 2009 06:29:11 am ilya wrote: > That's a useful statistics, but the bottleneck is **only** because of > parsing 'value +=5'. > > Here's how I time it: > > # insert my old program here... > > > from timeit import Timer > from codeop import compile_command > > def timing(something): > setup = 'from __main__ import test, interrogate, command, inc5' > best = sorted(Timer(something, setup).repeat(3, 1000))[0] min(alist) is a more-direct, simpler, faster, easier to read way of calculating sorted(alist)[0]. > print('{0!r} -> {1:.3} ms'.format(something, best)) > print('# test.value =', test.value) > > command = ''' > for i in range(10): > value += 1 > ''' You're muddying the water by including a for-loop and call to range() inside the code snippet being tested. We're trying to compare your function interrogate(test, 'value += 1') with the standard call to test.value += 1. Why include the time required to generate a range() object, and iterate over it ten times, as part of the code snippet? All that does is mix up the time required to execute common code and the time required to execute the code we care about. If it's not obvious why your approach is flawed, consider this extreme example: Timer("time.sleep(1000); interrogate(test, 'value += 1')", ...) Timer("time.sleep(1000); test.value += 1", ...) The differences in speed between the interrogate call (using exec) and the direct access to test.value will be swamped by the time used by the common code. A more accurate measurement is to remove the "for i in..." part from command, and increase the number=1000 argument to Timer.repeat() to 10000. > inc5 = compile_command(command) This is an unfair test. We're comparing directly accessing test.value versus indirectly accessing test.value using exec. Regardless of whether the caller compiles the statement manually before passing it to exec, or just passes it to exec to compile it automatically, the cost of that compilation has to be payed. Pulling that outside of the timing code just hides the true cost. Pre-compiling before passing to exec is a good optimization for the cases where you need to exec the same code snippet over and over again. If you want to execute "for i in range(1000): exec(s)" then it makes sense to pull out the compilation of s outside of the loop. But generally when timing code snippets, the only purpose of the loop is to minimize errors and give more accurate results, so pulling out the compilation just hides some of the real cost. > timing("interrogate(test, command)") > timing(command.replace('value', 'test.value')) > timing("interrogate(test, inc5)") > > Result: > > 15 Where does the 15 come from? > 'interrogate(test, command)' -> 0.0908 ms > # test.value = 30015 > '\nfor i in range(10):\n test.value += 1\n' -> 0.00408 ms > # test.value = 60015 > 'interrogate(test, inc5)' -> 0.00469 ms > # test.value = 90015 > > so interrogate() with additional precompiling introduces very little > overhead. Only because you're ignoring the overhead of pre-compiling. A more accurate test would be: timing("inc5 = compile_command(command); interrogate(test, inc5)") > Though I agree it's inconvenient to write functions as > strings; If you think that's inconvenient, just try writing functions as code objects without calling compile :) -- Steven D'Aprano From steve at pearwood.info Sat Aug 15 04:13:22 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 15 Aug 2009 12:13:22 +1000 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com> Message-ID: <200908151213.23078.steve@pearwood.info> On Sat, 15 Aug 2009 09:53:41 am Prozacgod wrote: > This makes sense, obviously you should compile it first, which when > executed should execute exactly the same as the original context > python. We want to compare the speed of the following statement: test.value += 1 with the equivalent: code = compile("value += 1", '', 'exec') exec code in test.__dict__ Why do you think it is appropriate to ignore the time taken by the compile() when comparing the two? Put it this way: if you want to know how long it takes to make fresh stir-fried chicken, you have to include the ten minutes it takes to chop up the vegetables and chicken, and not just the two minutes it takes to stir-fry them. -- Steven D'Aprano From solipsis at pitrou.net Sat Aug 15 13:19:37 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 15 Aug 2009 11:19:37 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Interrogate_alternate_namespace_keyword_?= =?utf-8?q?and=09concept?= References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908141754.03678.steve@pearwood.info> <90df837e0908141225n2ed9c6f3v68ff3cab55b58e87@mail.gmail.com> Message-ID: Prozacgod writes: > > My main motivation most likely is just that it would open up some very > interesting meta-programing or dynamic programming opportunities, variables > that exists entirely programmatically without the prefix of > object_instance.foo, surely just syntactical sugar to some (or even most) - This sounds very PHP-like (doing indirections like $$foo because there isn't any powerful introspection or OO system, and many things (e.g. functions, classes) aren't first-class objects and can't be passed as parameters). I don't think Python should grow something like that; instead people should learn more structured ways of accessing data. getattr() and friends are powerful enough for playing access tricks. (in other words: -1 from me) From dickinsm at gmail.com Sat Aug 15 13:23:17 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 15 Aug 2009 12:23:17 +0100 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <4A801D0C.2000502@gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> Message-ID: <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> On Mon, Aug 10, 2009 at 2:13 PM, Nick Coghlan wrote: > Bytes to int: > > [...] > Int to bytes: > > [...] Alexandre Vassalotti has posted a patch at http://bugs.python.org/issue1023290 that implements methods very much like the ones that Nick describes. Mark From ede at mit.edu Sat Aug 15 14:13:51 2009 From: ede at mit.edu (Eric Eisner) Date: Sat, 15 Aug 2009 21:13:51 +0900 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> Message-ID: <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> This patch seems very complete, with only the API to hammered out. Here is my summary of some options: method names: patch behavior: int.as_bytes / int.frombytes int.as_bytes / int.from_bytes int.asbytes / int.frombytes Endianness: patch behavior: default flag: little_endian=False byteorder option accepting 'big' or 'little', this can also accept sys.byteorder sign: patch behavior: default flag: signed=True maybe unsigned as the default? byte length: patch behavior: fixed_length=None other names: length, bytelength As for my own opinions: I think the method names should have consistent underscore usage. I think it is important to have all of big, little, and native as byteorder options, but I would be against having native as the default. I think it is unclean for core functionality to be platform dependent (there may be some examples of this that I'm not thinking of though). One option would be to have the defaults of int.as_bytes mirror the hex builtin function, eg big endian and unsigned. This way it could be more consistent with related functionality already in the core. Thoughts? -Eric On Sat, Aug 15, 2009 at 20:23, Mark Dickinson wrote: > On Mon, Aug 10, 2009 at 2:13 PM, Nick Coghlan wrote: >> Bytes to int: >> >> [...] >> Int to bytes: >> >> [...] > > Alexandre Vassalotti has posted a patch at > > http://bugs.python.org/issue1023290 > > that implements methods very much like the ones that Nick describes. > > Mark > From dickinsm at gmail.com Sat Aug 15 14:14:40 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 15 Aug 2009 13:14:40 +0100 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <4A801D0C.2000502@gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> Message-ID: <5c6f2a5d0908150514l31d3532an382d6372db30c410@mail.gmail.com> (I'm repeating some of the comments already made in the bug-tracker; as Antoine pointed out, discussion should probably remain here until the API is settled.) On Mon, Aug 10, 2009 at 2:13 PM, Nick Coghlan wrote: > [...] > > That suggests to me the following signatures for the conversion > functions (regardless of the names they might be given): > > int.from_bytes(data, *, little_endian=None, signed=True) > ?little_endian would become a three-valued parameter for the Python > version: ?None = native; False = little-endian; True = big-endian. The > three valued parameter is necessary since Python code can't get at the > "IS_LITTLE_ENDIAN" macro that the PyLong code uses to determine the > endianness of the system when calling the C API functions. > ?signed would just be an ordinary boolean flag Sounds good to me. I'm not sure about the 'signed=True' default; to me, a default of unsigned seems more natural. But this is bikeshedding, and I'd happily accept either default. I agree with other posters that there seems little reason not to accept the empty string. It's a natural end-case for unsigned input; whether it's natural for signed input (where there should really be at least one 'sign bit', and hence at least one byte) is arguable, but I can't see the harm in accepting it. > int.to_bytes(data, *, size=0, little_endian=None, signed=True) > ?A size <= 0 would mean to produce as many bytes as are needed to > represent the integer and no more. Otherwise it would represent the > maximum number of bytes allowed in the response (raising OverflowError > if the value won't fit). > ?little_endian and signed would be interpreted as for the conversion > from bytes to an integer I'm not convinced that it's valuable to a have a variable-size version of this; I'd make size a required argument. The problem with the variable size version is that the choice of byte-length for the output for a given integer input is a little bit arbitrary. For a particular requirement (producing code to conform with some existing serialization protocol, for example) it seems likely that the choice Python makes will disagree with what's required by that protocol, so that size still has to be given explicitly. On the other hand, if a user just wants a quick and easy way to serialize ints, without caring about the exact form of the serialization, then there are number of solutions already available within Python. +1 on raising OverflowError for out-of-range inputs, instead of wrapping modulo 2**whatever. This also fits with the way that the struct module currently behaves. Does anyone see other use-cases for variable-size conversion? [Greg Ewing] > I don't like the idea of a three-valued boolean. I also > don't like boolean parameters whose sense is abritrary > (why is it called "little_endian" and not "big_endian", > and how do I remember which convention was chosen?) > My suggestion would be to use the same characters that > the struct module uses to represent endianness (">" > for big-endian, "<" for little-endian, etc.) How about a parameter byteorder=None, accepting values 'big' and 'little'? Then one could use byteorder=sys.byteorder to explicitly specify native byteorder. Mark From ncoghlan at gmail.com Sat Aug 15 17:08:26 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Aug 2009 01:08:26 +1000 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> Message-ID: <4A86CF6A.7070409@gmail.com> Eric Eisner wrote: > One option would be to have the defaults of int.as_bytes mirror the > hex builtin function, eg big endian and unsigned. This way it could be > more consistent with related functionality already in the core. Having the byte conversion mirror the hex builtin is probably the least arbitrary style guide we could come up with for the defaults. The other option would be to not *have* defaults for either of these settings and force the programmer to make a deliberate choice. Then once it has been out in the field for a while and we have evidence about the way people use it, add sensible defaults in the following release. I would prefer either of those two options to attempting to just guess what the appropriate defaults would be in the absence of a wide selection of use cases. I also like the idea of using sys.byteorder as the model for the byteorder selection option (i.e. byteorder=sys.byteorder to select native layout and 'big'/'little' to choose a specific one) I think Mark also makes a good point that in cases where the size doesn't matter, marshal or pickle is probably a better choice than this direct conversion so +0 on requiring an explicit size on the conversion to bytes. Again, this is a case where *adding* a default later would be easy, but changing a default or removing a variable size feature would be difficult. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From alexandre at peadrop.com Sat Aug 15 17:45:37 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sat, 15 Aug 2009 11:45:37 -0400 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> Message-ID: [Oops, I forgot to CC python-ideas. Sorry Eric for the double post.] On Sat, Aug 15, 2009 at 8:13 AM, Eric Eisner wrote: > This patch seems very complete, with only the API to hammered out. > Here is my summary of some options: > > method names: > patch behavior: int.as_bytes / int.frombytes > int.as_bytes / int.from_bytes > int.asbytes / int.frombytes > In my patch, I chosen to use the name 'as_bytes' because it was consistent with 'float.as_integer_ratio'. Similarly, I chosen the name 'frombytes' because it was consistent with 'float.fromhex'. I don't mind the inconsistent use of the underscore in the names, but I admit there is room for improvement. So, what do you think of `int.frombytes` and `int.tobytes`? > Endianness: > patch behavior: default flag: little_endian=False > byteorder option accepting 'big' or 'little', this can also accept sys.byteorder > I like the byteorder option better. I believe the byteorder option shouldn't default to use the native byte-order however. As you mentioned, it would be a bad choice to encourage the default behaviour to be platform-dependent. And since the primary purpose of the API is long serialization, it would be short-sighted to choose the option that cannot be used for serialization as the default. Whether it should default to 'little' or 'big' is pretty much an arbitrary choice. In my patch, I choose to default big-endian since it is the standard network byte-order. But maybe the option should default to little-endian instead since it more widely used. In addition, the patch is slightly more efficient with little-endian. > sign: > patch behavior: default flag: signed=True > maybe unsigned as the default? > Either is fine by me. The advantage with 'signed' as the default is 'signed' works with all longs (and not only with non-negative ones). > byte length: > patch behavior: fixed_length=None > other names: length, bytelength > I still like `fixed_length` better than proposed alternatives. The name `fixed_length` makes it clear that the returned object has a fixed and constant length. And, I find `fixed_length=None` is more telling than `length=None`. -- Alexandre From dickinsm at gmail.com Sat Aug 15 18:25:02 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 15 Aug 2009 17:25:02 +0100 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> Message-ID: <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> On Sat, Aug 15, 2009 at 4:45 PM, Alexandre Vassalotti wrote: > I believe the byteorder option shouldn't default to use the native > byte-order however. As you mentioned, it would be a bad choice to > encourage the default behaviour to be platform-dependent. And since > the primary purpose of the API is long serialization, it would be > short-sighted to choose the option that cannot be used for > serialization as the default. > > Whether it should default to 'little' or 'big' is pretty much an > arbitrary choice. In my patch, I choose to default big-endian since it > is the standard network byte-order. But maybe the option should > default to little-endian instead since it more widely used. In > addition, the patch is slightly more efficient with little-endian. Given all this, it sounds like byteorder should be a required argument. 'In the face of ambiguity... ' and all that. As Nick pointed out, we can always add a default later when a larger set of use-cases has emerged. You say that the 'primary purpose of the API is long serialization'. I'd argue that that's not quite true. That is, I see two separate uses: (1) fixed-size conversions: e.g., interpreting a three-byte sequence as an integer for the purposes of bit operations, or converting an int generated by random.getrandbits(k) to a random byte sequence. (See http://bugs.python.org/msg69285 and http://bugs.python.org/msg54262.) (2) Provide a primitive operation that's useful for serialization protocols. Here I'd guess that the details of e.g., how a negative integer is serialized would vary from protocol to protocol, so that the serialization code would in most cases still end up having to specify the fixed_length argument. I *don't* see int.tobytes and int.frombytes (or whatever the names turn out to be) as providing integer serialization by themselves. There's no need for this, since pickle and marshal already do this job. Incidentally, the commenters in http://bugs.python.org/issue467384 have quite a lot to say on this subject. It's on this basis that I'm suggesting that the size argument should be required for int.tobytes. > On Sat, Aug 15, 2009 at 8:13 AM, Eric Eisner wrote: > > sign: > > patch behavior: default flag: signed=True > > maybe unsigned as the default? > Either is fine by me. The advantage with 'signed' as the default is > 'signed' works with all longs (and not only with non-negative ones). Agreed. Of course, this advantage disappears if the size argument is mandatory. I don't have any strong opinions about the method and parameter names, so I'll keep quiet on that subject. :) Mark From alexandre at peadrop.com Sat Aug 15 18:34:47 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sat, 15 Aug 2009 12:34:47 -0400 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908150514l31d3532an382d6372db30c410@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908070304x29051d2w934c67a9a0e30f4@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150514l31d3532an382d6372db30c410@mail.gmail.com> Message-ID: On Sat, Aug 15, 2009 at 8:14 AM, Mark Dickinson wrote: > I'm not convinced that it's valuable to a have a variable-size > version of this; ?I'd make size a required argument. > > The problem with the variable size version is that the choice of > byte-length for the output for a given integer input is a little bit > arbitrary. ?For a particular requirement (producing code to conform > with some existing serialization protocol, for example) it seems > likely that the choice Python makes will disagree with what's > required by that protocol, so that size still has to be given explicitly. > On the other hand, if a user just wants a quick and easy way > to serialize ints, without caring about the exact form of the > serialization, then there are number of solutions already > available within Python. > Well, the only use-case in the standard library I found (i.e., simplifying encode_long() and decode_long() in pickle.py) needed the variable-length version. However, unlike I originally thought, the variable length version is not difficult to emulate using `int.bit_length()`. For example, with my patch I can rewrite: def encode_long(x): if x == 0: return b"" return x.as_bytes(little_endian=True) as: def encode_long(x) if x == 0: return b"" nbytes = (x.bit_length() >> 3) + 1 result = x.as_bytes(nbytes, little_endian=True) if x < 0 and nbytes > 1: if result[-1] == 0xff and (result[-2] & 0x80) != 0: result = result[:-1] return result I usually hate with passion APIs that requires you to know the length of the result in advance. But this doesn't look bad. The only use-case for the variable-length version I have is the encode_long() function in pickle.py. In addition, it sounds reasonable to leave the duty of long serialization to pickle. So, +1 from me. -- Alexandre From alexandre at peadrop.com Sat Aug 15 18:48:12 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sat, 15 Aug 2009 12:48:12 -0400 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> Message-ID: On Sat, Aug 15, 2009 at 12:25 PM, Mark Dickinson wrote: > On Sat, Aug 15, 2009 at 4:45 PM, Alexandre > Vassalotti wrote: >> I believe the byteorder option shouldn't default to use the native >> byte-order however. As you mentioned, it would be a bad choice to >> encourage the default behaviour to be platform-dependent. And since >> the primary purpose of the API is long serialization, it would be >> short-sighted to choose the option that cannot be used for >> serialization as the default. >> >> Whether it should default to 'little' or 'big' is pretty much an >> arbitrary choice. In my patch, I choose to default big-endian since it >> is the standard network byte-order. But maybe the option should >> default to little-endian instead since it more widely used. In >> addition, the patch is slightly more efficient with little-endian. > > Given all this, it sounds like byteorder should be a required argument. > 'In the face of ambiguity... ' and all that. ?As Nick pointed out, we > can always add a default later when a larger set of use-cases has > emerged. > > You say that the 'primary purpose of the API is long serialization'. > I'd argue that that's not quite true. And you are totally right. Honestly, the only reason I was thinking about long serialization is because I have my hand full with pickle guts presently. :-) -- Alexandre From prozacgod at gmail.com Sat Aug 15 19:57:58 2009 From: prozacgod at gmail.com (Prozacgod) Date: Sat, 15 Aug 2009 12:57:58 -0500 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908141754.03678.steve@pearwood.info> <90df837e0908141225n2ed9c6f3v68ff3cab55b58e87@mail.gmail.com> Message-ID: <90df837e0908151057g4096advee80bac742cb1649@mail.gmail.com> I'm leaning towards a -0.5 right now myself, simply because of the ambiguities that have been mentioned, python is anything but ambiguous, and that's a distinction I like about it, in every case I can think of python does what you'd expect without random oh, but there is this qualifier in this context .. (*cough* perl *cough* ruby *cough*).. even if I think I like the syntax, it doesn't really make sense to implement it at this time, or likely ever. I got the inspiration for the idea from delphi/pascal, but in retrospect it turns out that the primary large (~1million lines) commercial app I worked on where we used that syntax (heavily I might add), they started to phase out the use of the with statement all over the code because of these same abiguities creating obscure difficult to track down bugs. On Sat, Aug 15, 2009 at 6:19 AM, Antoine Pitrou wrote: > Prozacgod writes: > > > > My main motivation most likely is just that it would open up some very > > interesting meta-programing or dynamic programming opportunities, > variables > > that exists entirely programmatically without the prefix of > > object_instance.foo, surely just syntactical sugar to some (or even most) > - > > This sounds very PHP-like (doing indirections like $$foo because there > isn't any > powerful introspection or OO system, and many things (e.g. functions, > classes) > aren't first-class objects and can't be passed as parameters). I don't > think > Python should grow something like that; instead people should learn more > structured ways of accessing data. getattr() and friends are powerful > enough for > playing access tricks. > > (in other words: -1 from me) > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- -Prozacgod "Prozac may heal the mind, but friends can mend the soul" -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Sat Aug 15 20:42:23 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 15 Aug 2009 19:42:23 +0100 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> Message-ID: <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> On Sat, Aug 15, 2009 at 5:48 PM, Alexandre Vassalotti wrote: > On Sat, Aug 15, 2009 at 12:25 PM, Mark Dickinson wrote: >> You say that the 'primary purpose of the API is long serialization'. >> I'd argue that that's not quite true. > > And you are totally right. [...] Eh? But I came here for an argument! Isn't this room 12? A couple of other things: If these additions to int go in, then presumably the _PyLong_AsBytes and _PyLong_FromBytes functions should be documented and made public (and have their leading underscores removed, too). Those functions have been stable for a good while, and are well-used within the Python source; I think they're robust enough for public consumption. There may be some additional argument validation required; I'll take a look at this. In the issue tracker, Josiah Carlson asked about the possibility of backporting to 2.7. I can't see any problem with this, though there would be some small extra work involved in making things work for int as well as long. Does anyone else see any issues with this? Mark From greg.ewing at canterbury.ac.nz Sun Aug 16 03:06:35 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Aug 2009 13:06:35 +1200 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <90df837e0908141659k32dffb8fod69bf647736193f4@mail.gmail.com> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <90df837e0908141246y2c6ca0ddse34bf876249e3be9@mail.gmail.com> <90df837e0908141659k32dffb8fod69bf647736193f4@mail.gmail.com> Message-ID: <4A875B9B.7000001@canterbury.ac.nz> Prozacgod wrote: > After this discussion I'm wondering why it ever ended up in any language > to begin with? Pascal has it because it doesn't have any general way of binding a local name to some nested part of a data structure. You can always do this in Python, and also in C using the & operator, so those languages don't need a Pascal-style with-statement. -- Greg From greg.ewing at canterbury.ac.nz Sun Aug 16 03:30:05 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Aug 2009 13:30:05 +1200 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <5c6f2a5d0908070619n44c7d39cua75cec88c5dea11e@mail.gmail.com> <200908081554.36358.steve@pearwood.info> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> Message-ID: <4A87611D.7080903@canterbury.ac.nz> Alexandre Vassalotti wrote: > Whether it should default to 'little' or 'big' is pretty much an > arbitrary choice. In my patch, I choose to default big-endian since it > is the standard network byte-order. But maybe the option should > default to little-endian instead since it more widely used. This all sounds to me like an argument for not having a default at all. -- Greg From ncoghlan at gmail.com Sun Aug 16 03:23:39 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 16 Aug 2009 11:23:39 +1000 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <5c6f2a5d0908100105g267d90c6j8927f8395f0c771@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> Message-ID: <4A875F9B.5070701@gmail.com> Mark Dickinson wrote: > In the issue tracker, Josiah Carlson asked about the possibility of > backporting to 2.7. I can't see any problem with this, though there > would be some small extra work involved in making things work > for int as well as long. Does anyone else see any issues with this? What would we be converting them to in that case? 2.x strings? (I don't have a problem with that, just pointing out there may be some additional work due to changing the target type). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From greg.ewing at canterbury.ac.nz Sun Aug 16 03:09:59 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 16 Aug 2009 13:09:59 +1200 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <200908151213.23078.steve@pearwood.info> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <90df837e0908141653gcfe8c4eme6b0657fc47d5dea@mail.gmail.com> <200908151213.23078.steve@pearwood.info> Message-ID: <4A875C67.2080501@canterbury.ac.nz> Steven D'Aprano wrote: > if you want to know > how long it takes to make fresh stir-fried chicken, you have to include > the ten minutes it takes to chop up the vegetables and chicken, Don't forget to include the time required to catch and kill the chicken as well! -- Greg From steve at pearwood.info Sun Aug 16 08:17:52 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 16 Aug 2009 16:17:52 +1000 Subject: [Python-ideas] =?iso-8859-1?q?Interrogate_alternate_namespace_key?= =?iso-8859-1?q?word_and=09concept?= In-Reply-To: <4A875C67.2080501@canterbury.ac.nz> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908151213.23078.steve@pearwood.info> <4A875C67.2080501@canterbury.ac.nz> Message-ID: <200908161617.53682.steve@pearwood.info> On Sun, 16 Aug 2009 11:09:59 am Greg Ewing wrote: > Steven D'Aprano wrote: > > if you want to know > > how long it takes to make fresh stir-fried chicken, you have to > > include the ten minutes it takes to chop up the vegetables and > > chicken, > > Don't forget to include the time required to catch and kill > the chicken as well! "In order to make an apple pie from scratch, you must first create the universe." -- Carl Sagan Of course all timing tests make certain assumptions, and the art of optimization is partly to recognise when those assumptions are violated. If you do have a situation where you are executing the exact same code repeatedly, then pre-compiling is a good optimization over calling exec on the string directly. But you can't exclude the time for pre-compiling in the general case. -- Steven D'Aprano From alexandre at peadrop.com Sun Aug 16 20:44:30 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Sun, 16 Aug 2009 14:44:30 -0400 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <4A7FF6CE.7070903@gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> Message-ID: On Sat, Aug 15, 2009 at 2:42 PM, Mark Dickinson wrote: > On Sat, Aug 15, 2009 at 5:48 PM, Alexandre > Vassalotti wrote: >> On Sat, Aug 15, 2009 at 12:25 PM, Mark Dickinson wrote: >>> You say that the 'primary purpose of the API is long serialization'. >>> I'd argue that that's not quite true. >> >> And you are totally right. [...] > > Eh? ?But I came here for an argument! ?Isn't this room 12? > Oh, oh, I am sorry. This is agreement. You want 12A, next door. =) > A couple of other things: > > If these additions to int go in, then presumably the _PyLong_AsBytes > and _PyLong_FromBytes functions should be documented and made public > (and have their leading underscores removed, too). You are referring to _PyLong_FromByteArray and _PyLong_AsByteArray, right? -- Alexandre From tleeuwenburg at gmail.com Mon Aug 17 05:05:15 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Mon, 17 Aug 2009 13:05:15 +1000 Subject: [Python-ideas] Python 3.1 'api tracker' idea Message-ID: <43c8685c0908162005h5d028d0r114f6a82b95c1e1@mail.gmail.com> I use Python at work, but I depend on Numeric, Scientific and (and some others). It would be neat if there were a website I could use to check whether all my external libraries were supported under Python 3.x. If I could tick boxes to quickly check whether Python 3 could meet my needs -- or better yet register with an email daemon which would email me when Python 3.x became useful for me -- that would be awesome. Cheers, -T -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Mon Aug 17 15:04:02 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 17 Aug 2009 15:04:02 +0200 Subject: [Python-ideas] Python 3.1 'api tracker' idea In-Reply-To: <43c8685c0908162005h5d028d0r114f6a82b95c1e1@mail.gmail.com> References: <43c8685c0908162005h5d028d0r114f6a82b95c1e1@mail.gmail.com> Message-ID: Tennessee Leeuwenburg wrote: > I use Python at work, but I depend on Numeric, Scientific and (and some > others). > > It would be neat if there were a website I could use to check whether all my > external libraries were supported under Python 3.x. If I could tick boxes to > quickly check whether Python 3 could meet my needs -- or better yet register > with an email daemon which would email me when Python 3.x became useful for > me -- that would be awesome. Have you searched PyPI for Python 3 compliant packages? Everything that doesn't list itself there as supporting Py3 likely doesn't support it. Stefan From mcaninch at lanl.gov Thu Aug 20 01:07:54 2009 From: mcaninch at lanl.gov (Jeff McAninch) Date: Wed, 19 Aug 2009 17:07:54 -0600 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression Message-ID: <4A8C85CA.6010306@lanl.gov> I would like to propose an expression, similar to the if-else expression, that responds to exceptions. I had originally posted this (probably mistakenly) on py-dev. This current posting is a cleaned up version of the idea, based on responses I got on from the earlier posting. _*Abstract: *_Proposal for a conditional expression, similar to the if-else expression, that responds to exceptions. _*Motivation: *_An expression syntax that responds to exceptions, and which reproduces the readability and conciseness of the if-else conditional expression, would simplify some exception-handling cases, especially within list comprehensions. _*Very Simple Example - type coercion: *_Current approach: try: x = float(string) except: x = float('nan') Proposed solution using exception-based conditional expression: x = float(string) except ValueError: float('nan') _*Simple Example - type coercion in a list comprehension: *_Current approach: def safe_float(string): try: x = float(string) except ValueError: x = float('nan') return x ... xs = (safe(float(string)) for string in strings) Proposed solution using exception-based conditional expression: xs = ((float(string) except ValueError: float('nan')) for string in strings) _*Discussion: *_In my own python coding, I find I make common use of the if-else conditional expression, especially within list comprehensions. (In one of my packages, which has ~5800 lines of code, I found if-else expressions in ~1% of the lines.) Here is a slightly more involved example than the examples presented above. In data processing, I often string together a sequence of iterable list comprehensions, corresponding to a sequence of operations on a given dataset "ys" to produce a processed dataset "x": xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in xs if filter_C(x)) # final, explicit list of values xs = [ x for x in xs ] This is often a flexible way for me to define processing and filtering sequences which also seems to have good performance on very large datasets. One advantage is that I can quickly mix-and-match from existing processes like this to make a new process. An exception-based conditional would go nicely into many of these process sequences, keeping them both robust and flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) I also often have object classes which have some common method or attribute. For instance, some of my objects have scope-dependent values: x = y.evaluate(scope)) where scope is typically locals(), globals(), or some other dictionary-like container. But, to keep my code modular, I want to handle, in the same lines of code, objects which do not have some particular method, which leads me to lines of code like: x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y This seems not very "Pythonic", similar to using type-testing instead of try-except. (My impression was that there was a long-standing trend in the evolution of Python to remove tests like this, and I thought that was the original motivation for the try-except syntax.) I would much rather write: x = y.evaluate(locals()) except AttributeError: y or, in the list comprehension example: xs = (y.evaluate(locals()) except AttributeError: y for y in ys) Clearly this can be handled in several ways with the language as it is. One way is to define a new function, as in the second simple example above: def safe_evaluate(y,scope): try: x = y.evaluate(scope) except AttributeError: x = y return x ... xs = (safe_evaluate(y,locals()) for y in ys) but this quickly (in my packages at least) leads to an annoying proliferation of "safe_" functions. Again, this seems not to be in the "Pythonic" spirit, and is also less concise, less readable. (I also suspect, but have not verified, that this is in general less efficient than in-line expressions -- wasn't that part of the original motivation for list comprehensions?). In the thread of my previous post to py-dev, there were comments, questions, and suggestions concerning the details of the syntax. Having reflected on this for a couple weeks, I am now most strongly supportive of what is essentially just an inline compression of the current try-except syntax. So the following examples would be allowed: x = expression0 except: default_expression x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression Or, more generally: x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression In this last example, the behaviour would be as follows: - evaluate expression0. If no exception is encountered, return the result. - if an exception is encountered, search for the matching exception in the except clauses. - if a matching exception ("exceptionI") is found, evaluate the corresponding expression ("expressionI"), and return the result. - if no matching exception is found, and a default except: clause (i.e., one without and exception) is given, evaluate default_expression, and return the result. - if no matching exception is found, and no default except clause if given, pass the exception on to the caller. - if a new exception is encountered while evaluating an an except expression ("expressionI"), pass the exception on to the caller. I hope I have made a convincing case here. This seems to me to be a natural ("Pythonic") addition to the language. Jeff McAninch -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch at lanl.gov ========================== -------------- next part -------------- An HTML attachment was scrubbed... URL: From tleeuwenburg at gmail.com Thu Aug 20 01:59:31 2009 From: tleeuwenburg at gmail.com (Tennessee Leeuwenburg) Date: Thu, 20 Aug 2009 09:59:31 +1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: <43c8685c0908191659h73567060h2075cd4390b19eb0@mail.gmail.com> I like it. I'm not fully competent to comment on the details, but a big +1 for something like this, if not this. -T On Thu, Aug 20, 2009 at 9:07 AM, Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else expression, > that responds to exceptions. > > I had originally posted this (probably mistakenly) on py-dev. This current > posting is a cleaned up > version of the idea, based on responses I got on from the earlier posting. > > *Abstract: > *Proposal for a conditional expression, similar to the if-else expression, > that responds to exceptions. > > *Motivation: > *An expression syntax that responds to exceptions, and which reproduces > the readability and conciseness of the if-else conditional expression, would > simplify some exception-handling cases, especially within list > comprehensions. > > *Very Simple Example - type coercion: > *Current approach: > try: > x = float(string) > except: > x = float('nan') > > Proposed solution using exception-based conditional expression: > x = float(string) except ValueError: float('nan') > > > *Simple Example - type coercion in a list comprehension: > *Current approach: > def safe_float(string): > try: > x = float(string) > except ValueError: > x = float('nan') > return x > ... > xs = (safe(float(string)) for string in strings) > > Proposed solution using exception-based conditional expression: > xs = ((float(string) except ValueError: float('nan')) for string in > strings) > > *Discussion: > *In my own python coding, I find I make common use of the if-else > conditional expression, especially within list comprehensions. (In one of > my packages, which has ~5800 lines of code, I found if-else expressions in > ~1% of the lines.) > > Here is a slightly more involved example than the examples presented > above. In data processing, I often string together a sequence of iterable > list comprehensions, corresponding to a sequence of operations on a given > dataset "ys" to produce a processed dataset "x": > xs = (operation_A(x) for x in ys) > xs = (operation_B(x) for x in xs if filter_B(x)) > xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x > in xs if filter_C(x)) > # final, explicit list of values > xs = [ x for x in xs ] > This is often a flexible way for me to define processing and filtering > sequences which also seems > to have good performance on very large datasets. One advantage is that I > can quickly mix-and-match from existing processes like this to make a new > process. An exception-based conditional would go nicely > into many of these process sequences, keeping them both robust and > flexible. > xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in > xs) > > I also often have object classes which have some common method or > attribute. For instance, some of my objects have scope-dependent values: > x = y.evaluate(scope)) > where scope is typically locals(), globals(), or some other dictionary-like > container. But, to keep my code modular, I want to handle, in the same > lines of code, objects which do not have some particular method, which leads > me to lines of code like: > x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y > This seems not very "Pythonic", similar to using type-testing instead of > try-except. (My impression was that there was a long-standing trend in the > evolution of Python to remove tests like this, and I thought that was the > original motivation for the try-except syntax.) > > I would much rather write: > x = y.evaluate(locals()) except AttributeError: y > or, in the list comprehension example: > xs = (y.evaluate(locals()) except AttributeError: y for y in ys) > > Clearly this can be handled in several ways with the language as it is. > One way is to define a new function, as in the second simple example above: > def safe_evaluate(y,scope): > try: > x = y.evaluate(scope) > except AttributeError: > x = y > return x > ... > xs = (safe_evaluate(y,locals()) for y in ys) > but this quickly (in my packages at least) leads to an annoying > proliferation of "safe_" functions. > Again, this seems not to be in the "Pythonic" spirit, and is also less > concise, less readable. (I also suspect, but have not verified, that this > is in general less efficient than in-line expressions -- wasn't that part of > the original motivation for list comprehensions?). > > In the thread of my previous post to py-dev, there were comments, > questions, and suggestions concerning the details of the syntax. Having > reflected on this for a couple weeks, I am now most strongly supportive of > what is essentially just an inline compression of the current try-except > syntax. So the following examples would be allowed: > x = expression0 except: default_expression > x = expression0 except exception1: expression1 except exception2: > expression2 except: default_expression > > Or, more generally: > x = expression0\ > except exception1: expression1\ > except exception2: expression2\ > ... > except exceptionI: expressionI\ > ... > except: default_expression > In this last example, the behaviour would be as follows: > - evaluate expression0. > If no exception is encountered, return the result. > - if an exception is encountered, > search for the matching exception in the except clauses. > - if a matching exception ("exceptionI") is found, > evaluate the corresponding expression ("expressionI"), and > return the result. > - if no matching exception is found, and a default except: clause > (i.e., one without and exception) > is given, evaluate default_expression, and return the result. > - if no matching exception is found, and no default except clause if > given, > pass the exception on to the caller. > - if a new exception is encountered while evaluating an an except > expression ("expressionI"), > pass the exception on to the caller. > > I hope I have made a convincing case here. This seems to me to be a > natural ("Pythonic") addition to the language. > > Jeff McAninch > > -- > ========================== > Jeffrey E. McAninch, PhD > Physicist, X-2-IFD > Los Alamos National Laboratory > Phone: 505-667-0374 > Email: mcaninch at lanl.gov > ========================== > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think" -------------- next part -------------- An HTML attachment was scrubbed... URL: From ironfroggy at gmail.com Thu Aug 20 03:59:07 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Wed, 19 Aug 2009 21:59:07 -0400 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: <76fd5acf0908191859t44f77d93ke909d8059b939d69@mail.gmail.com> -1 on colons in the expression like that. I like the idea of being able to handle an exception in generator expressions and the like, but I've never seen a syntax I liked. I think I've favored the idea of something like `float(x) except float('nan') if ValueError` thinking it reads more naturally as an expression, puts the real logic ("convert x to a float or get a NaN float") together, which I think makes sense. I fear being able to "express" too much. That is, we're going to remove all that indentation gives us if eventually everything is given an expression form. Let's add an import expression, next. Maybe this? sys.exit(1) from sys Joke, obviously. On Wed, Aug 19, 2009 at 7:07 PM, Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else expression, > that responds to exceptions. > > I had originally posted this (probably mistakenly) on py-dev.? This current > posting is a cleaned up > version of the idea, based on responses I got on from the earlier posting. > > Abstract: > Proposal for a conditional expression, similar to the if-else expression, > that responds to exceptions. > > Motivation: > An expression syntax that responds to exceptions, and which reproduces the > readability and conciseness of the if-else conditional expression, would > simplify some exception-handling cases, especially within list > comprehensions. > > Very Simple Example - type coercion: > Current approach: > ??? try: > ??????? x = float(string) > ??? except: > ??????? x = float('nan') > > Proposed solution using exception-based conditional expression: > ??? x = float(string) except ValueError: float('nan') > > > Simple Example - type coercion in a list comprehension: > Current approach: > ??? def safe_float(string): > ??????? try: > ??????????? x = float(string) > ??????? except ValueError: > ??????????? x = float('nan') > ??????? return x > ??? ... > ??? xs = (safe(float(string)) for string in strings) > > Proposed solution using exception-based conditional expression: > ??? xs = ((float(string) except ValueError: float('nan')) for string in > strings) > > Discussion: > In my own python coding, I find I make common use of the if-else conditional > expression, especially within list comprehensions.? (In one of my packages, > which has ~5800 lines of code, I found if-else expressions in ~1% of the > lines.) > > Here is a slightly more involved example than the examples presented above. > In data processing, I often string together a sequence of iterable list > comprehensions, corresponding to a sequence of operations on a given dataset > "ys" to produce a processed dataset "x": > ??? xs = (operation_A(x) for x in ys) > ??? xs = (operation_B(x) for x in xs if filter_B(x)) > ??? xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in > xs if filter_C(x)) > ??? # final, explicit list of values > ??? xs = [ x for x in xs ] > This is often a flexible way for me to define processing and filtering > sequences which also seems > to have good performance on very large datasets.? One advantage is that I > can quickly mix-and-match from existing processes like this to make a new > process.? An exception-based conditional would go nicely > into many of these process sequences, keeping them both robust and flexible. > ??? xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) > > I also often have object classes which have some common method or > attribute.? For instance, some of my objects have scope-dependent values: > ??? x = y.evaluate(scope)) > where scope is typically locals(), globals(), or some other dictionary-like > container.? But, to keep my code modular, I want to handle, in the same > lines of code, objects which do not have some particular method, which leads > me to lines of code like: > ??? x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y > This seems not very "Pythonic", similar to using type-testing instead of > try-except.? (My impression was that there was a long-standing trend in the > evolution of Python to remove tests like this, and I thought that was the > original motivation for the try-except syntax.) > > I would much rather write: > ??? x = y.evaluate(locals()) except AttributeError: y > or, in the list comprehension example: > ??? xs = (y.evaluate(locals()) except AttributeError: y for y in ys) > > Clearly this can be handled in several ways with the language as it is.? One > way is to define a new function, as in the second simple example above: > ??? def safe_evaluate(y,scope): > ??? ?? try: > ??? ?? ?? x = y.evaluate(scope) > ??? ?? except AttributeError: > ??? ?? ?? x = y > ??? ?? return x > ??? ... > ??? xs = (safe_evaluate(y,locals()) for y in ys) > but this quickly (in my packages at least) leads to an annoying > proliferation of "safe_" functions. > Again, this seems not to be in the "Pythonic" spirit, and is also less > concise, less readable.? (I also suspect, but have not verified, that this > is in general less efficient than in-line expressions -- wasn't that part of > the original motivation for list comprehensions?). > > In the thread of my previous post to py-dev, there were comments, questions, > and suggestions concerning the details of the syntax.? Having reflected on > this for a couple weeks, I am now most strongly supportive of what is > essentially just an inline compression of the current try-except syntax.? So > the following examples would be allowed: > ??? x = expression0 except: default_expression > ??? x = expression0 except exception1: expression1 except exception2: > expression2 except: default_expression > > Or, more generally: > ??? x = expression0\ > ? ? ? ? ? ? except exception1: expression1\ > ??????????? except exception2: expression2\ > ??????????? ... > ??????????? except exceptionI: expressionI\ > ??????????? ... > ??????????? except: default_expression > In this last example, the behaviour would be as follows: > ??? - evaluate expression0. > ??????????? If no exception is encountered, return the result. > ??? - if an exception is encountered, > ??????????? search for the matching exception in the except clauses. > ??? - if a matching exception ("exceptionI") is found, > ??????????? evaluate the corresponding expression ("expressionI"), and > return the result. > ??? - if no matching exception is found, and a default except: clause (i.e., > one without and exception) > ? ? ? ??? is given, evaluate default_expression, and return the result. > ??? - if no matching exception is found, and no default except clause if > given, > ????????? pass the exception on to the caller. > ??? - if a new exception is encountered while evaluating an an except > expression ("expressionI"), > ????????? pass the exception on to the caller. > > I hope I have made a convincing case here.? This seems to me to be a natural > ("Pythonic") addition to the language. > > Jeff McAninch > > -- > ========================== > Jeffrey E. McAninch, PhD > Physicist, X-2-IFD > Los Alamos National Laboratory > Phone: 505-667-0374 > Email: mcaninch at lanl.gov > ========================== > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From stephen at xemacs.org Thu Aug 20 04:31:20 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 20 Aug 2009 11:31:20 +0900 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: <87eir75i87.fsf@uwakimon.sk.tsukuba.ac.jp> Jeff McAninch writes: > _*Very Simple Example - type coercion: > *_Current approach: > try: > x = float(string) > except: > x = float('nan') > > Proposed solution using exception-based conditional expression: > x = float(string) except ValueError: float('nan') -1 I really don't like the colon in the middle of the expression. It just looks like broken syntax because everywhere else in Python colon introduces a suite. I wonder if you really need it? I generally don't like a general conditional expression. For reasons presented below, I don't think it's as flexible as think it can be. OTOH, I fund it much less readable than the existing statement-based construction. > _*Simple Example - type coercion in a list comprehension: > *_Current approach: > def safe_float(string): > try: > x = float(string) > except ValueError: > x = float('nan') > return x > ... > xs = (safe(float(string)) for string in strings) I don't have a problem with the above. But I can see that for > In data processing, I often string together a sequence of iterable > list comprehensions, corresponding to a sequence of operations on a > given dataset "ys" to produce a processed dataset "x": an exception-handling clause in comprehensions (and generator expressions?) might be useful, to make the exceptions handled explicit. For the cases I can imagine using myself, I would generally prefer the approach of defining functions to handle the exceptions, because they'd all be similar (eg, coercing exceptions to float("nan") as above), though. > try-except syntax. So the following examples would be allowed: > x = expression0 except: default_expression > x = expression0 except exception1: expression1 except exception2: expression2 except: default_expression But it seems to me the ez-read interpretation of x = expression0 except exception1: expression1 except: default_expression is x = expression0 except exception1: (expression1 except: default_expression) Ie, your parser resolves the "dangling except" ambiguity in the opposite way to the conventional resolution of the "dangling else" ambiguity. And what if excepts are mixed with conditional expressions? x = exp0 except exc1: exp1 if exp2 else exp3 except: exp4 Does the bare except bind to exp0, the if expression, or exp3? I'm sure you can define rules to disambiguate. However, I suspect that it will be hard to get agreement that any given set of rules is the appropriate way to resolve the ambiguity. From dstanek at dstanek.com Thu Aug 20 04:28:35 2009 From: dstanek at dstanek.com (David Stanek) Date: Wed, 19 Aug 2009 22:28:35 -0400 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: On Wed, Aug 19, 2009 at 7:07 PM, Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else expression, > that responds to exceptions. This is a pretty good idea. I can definitely see where I would use it in my own code. The only thing I would caution is in your simple example. I would choose to create a module full of conversion functions. It violates DRY to have the 'try float otherwise Nan' statement all over the place. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek From mcaninch at lanl.gov Thu Aug 20 12:15:28 2009 From: mcaninch at lanl.gov (Jeff McAninch) Date: Thu, 20 Aug 2009 04:15:28 -0600 Subject: [Python-ideas] [Fwd: Re: exception based conditional expression, similar to if-else conditional expression] Message-ID: <4A8D2240.6080107@lanl.gov> Calvin Spealman wrote: > -1 on colons in the expression like that. I like the idea of being > able to handle an exception in generator expressions and the like, but > I've never seen a syntax I liked. I think I've favored the idea of > something like `float(x) except float('nan') if ValueError` thinking > it reads more naturally as an expression, puts the real logic > ("convert x to a float or get a NaN float") together, which I think > makes sense. > > Yes, I agree about the colons. They have no purpose. I was just blindly following the try-except. (Duh! on my part) So, in the simple example: x = float(string) except ValueError float('nan') But possibly the exception tuples now have to be explicitly tuples? x = float(string) except (ValueError,) float('nan') So the general case would be something like exception_expression :== nominal_value {except exception_tuple exception_value}* {except default_value} Jeff McAninch -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch at lanl.gov ========================== From mcaninch at lanl.gov Thu Aug 20 12:30:02 2009 From: mcaninch at lanl.gov (Jeff McAninch) Date: Thu, 20 Aug 2009 04:30:02 -0600 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <87eir75i87.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A8C85CA.6010306@lanl.gov> <87eir75i87.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4A8D25AA.4080502@lanl.gov> Stephen J. Turnbull wrote: > Ie, your parser resolves the "dangling except" ambiguity in the > opposite way to the conventional resolution of the "dangling else" > ambiguity. And what if excepts are mixed with conditional > expressions? > > x = exp0 except exc1: exp1 if exp2 else exp3 except: exp4 > > Does the bare except bind to exp0, the if expression, or exp3? I'm > sure you can define rules to disambiguate. However, I suspect that it > will be hard to get agreement that any given set of rules is the > appropriate way to resolve the ambiguity. > Following up on my previous post, I think the general syntax would be something like: exception_expression :== nominal_value {except exception_tuple exception_value}* {except default_value} Hopefully this disambiguates the issue, at least from the parser point of view. Requiring an explicit tuple of exceptions (ie., requiring the parens around the tuple of exceptions) makes sense too, and I thought I had seen other discussions of requiring explicit tuples where currently parens could be implied. With the definition above, I think the combinations of exception expressions with if-else conditional expressions is probably also unambiguous (though not necessarily easily read). I haven't sat down to verify this mathematically yet though. Allowing parens around the except clauses might help to make it more readable? x = float(string) (except (ValueError,) float('nan')) (except (SomeOtherException,) someOtherValue) Okay, maybe not? (BTW: thanks for the cool new verb -- disambiguate -- though my friends and family may not be so happy when I start using it at every opportunity!) -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch at lanl.gov ========================== From pyideas at rebertia.com Thu Aug 20 12:45:31 2009 From: pyideas at rebertia.com (Chris Rebert) Date: Thu, 20 Aug 2009 03:45:31 -0700 Subject: [Python-ideas] [Fwd: Re: exception based conditional expression, similar to if-else conditional expression] In-Reply-To: <4A8D2240.6080107@lanl.gov> References: <4A8D2240.6080107@lanl.gov> Message-ID: <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> On Thu, Aug 20, 2009 at 3:15 AM, Jeff McAninch wrote: > Calvin Spealman wrote: > >> -1 on colons in the expression like that. I like the idea of being >> able to handle an exception in generator expressions and the like, but >> I've never seen a syntax I liked. I think I've favored the idea of >> something like `float(x) except float('nan') if ValueError` thinking >> it reads more naturally as an expression, puts the real logic >> ("convert x to a float or get a NaN float") together, which I think >> makes sense. >> >> > > Yes, I agree about the colons. ?They have no purpose. ?I was just blindly > following the try-except. ?(Duh! on my part) > > So, in the simple example: > ?x = float(string) except ValueError float('nan') I would feel more comfortable with another keyword in there. Perhaps: x = float(string) except ValueError then float('nan') Which reads like: "x is the float value of string, except in the event of ValueError, then it's float('nan')" Which I find rather pleasing. > But possibly the exception tuples now have to be explicitly tuples? > ?x = float(string) except (ValueError,) float('nan') Yech. That doesn't look so great. At any rate, I do seriously question the bang-for-the-buck of adding this construct. Yes, it makes such conversions shorter, but couldn't one just define a `float_or_nan` conversion function that defaults to NaN in case of error to essentially the same effect? I can see how the construct might help in comparatively quick-and-dirty scripts, but should Python actively encourage comparatively cavalier error-handling? Cheers, Chris -- http://blog.rebertia.com From masklinn at masklinn.net Thu Aug 20 13:17:08 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 20 Aug 2009 13:17:08 +0200 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: <93EF2F8E-3BD1-4243-83DE-5FC0E6FD58D2@masklinn.net> On 20 Aug 2009, at 01:07 , Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else > expression, > that responds to exceptions. -1, if this goes through it will be extended to all other compound statements over time and we'll end up with 2 or 3 ways to do every thing. A much better idea would be to find a way to make all compound statements into expressions, future-proofing the decision and avoiding the redundancy between "compound statement statement" and "compound statement expression". From mcaninch at lanl.gov Thu Aug 20 13:18:42 2009 From: mcaninch at lanl.gov (Jeff McAninch) Date: Thu, 20 Aug 2009 05:18:42 -0600 Subject: [Python-ideas] [Fwd: Re: exception based conditional expression, similar to if-else conditional expression] In-Reply-To: <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> References: <4A8D2240.6080107@lanl.gov> <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> Message-ID: <4A8D3112.7030905@lanl.gov> Chris Rebert wrote: > Yech. That doesn't look so great. > I agree about the parens, but thought I should put it out there. The parens around the exception clauses should not be required, but should be allowed to reduce ambiguity, In my own case, I would tend to use continuation lines to make multi-exception expressions more readable. > At any rate, I do seriously question the bang-for-the-buck of adding > this construct. Yes, it makes such conversions shorter, but couldn't > one just define a `float_or_nan` conversion function that defaults to > NaN in case of error to essentially the same effect? > I can see how the construct might help in comparatively > quick-and-dirty scripts, but should Python actively encourage > comparatively cavalier error-handling? > > One of the surprising things about Python is the impressive performance that can be achieved, even though it is an interpreted langauge. In my experience, this is one of the aspects of Python that sets it apart from other interpreted environments, and why I can apply Python to heavy duty scientific computing in the same environment where I pop up a gui or parse a text file. This performance is achieved as one increases the ratio of work done in compiled code, to the amount of work being done by the interpreter. Hence, list comprehensions and iterator expressions, in place of for and while loops. The bang-for-the-buck in the exception expression is in performance. This is not in any way just a quick-and-dirty issue. It's exactly the same motivation as the if-else expression. Please correct me if I am in error, but doesn't a call to a python function (as opposed to a compiled C-function) substantially slow down a list comprehension? (I will endeavor to generate a test case this weekend and post the results, but this is consistent with everything I've read about optimizing Python code.) And remember, this is called "exception-handling" rather than "error-handling" for a reason. In many practical cases, the exceptions can be a significant, if not dominant, fraction of the executions of the code. That, I thought, was the logic behind try:except in the first place: treating exceptions as a natural part of the coding, rather than a collection of special cases. I see this as encouraging more robust, rather than more cavalier, exception handling, and doing it in a more concise syntax (and I assert, more efficient from a performance perspective -- I'll try to put this assertion to a quantitative test this weekend.). -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch at lanl.gov ========================== From p.f.moore at gmail.com Thu Aug 20 16:02:28 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 20 Aug 2009 15:02:28 +0100 Subject: [Python-ideas] pep 312 - implicit lambdas idea In-Reply-To: <4A7CCB94.8020408@canterbury.ac.nz> References: <87d477lnuk.fsf@benfinney.id.au> <4A7CCB94.8020408@canterbury.ac.nz> Message-ID: <79990c6b0908200702l5aaf6b06kd799760c289c818c@mail.gmail.com> 2009/8/8 Greg Ewing : > My suggestion for a lightweight lambda syntax is > > ?args -> expr > > Examples: > > ?map(x -> x + 5, something) > > ?d = defauldict(->[]) +1 Paul. From jimjjewett at gmail.com Thu Aug 20 16:25:06 2009 From: jimjjewett at gmail.com (Jim Jewett) Date: Thu, 20 Aug 2009 10:25:06 -0400 Subject: [Python-ideas] [Fwd: Re: exception based conditional expression, similar to if-else conditional expression] In-Reply-To: <4A8D3112.7030905@lanl.gov> References: <4A8D2240.6080107@lanl.gov> <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> <4A8D3112.7030905@lanl.gov> Message-ID: On Thu, Aug 20, 2009 at 7:18 AM, Jeff McAninch wrote: > The bang-for-the-buck in the exception expression > is in performance. ... > ... doesn't a call to a python function > (as opposed to a compiled C-function) substantially > slow down a list comprehension? Probably, but you don't have to use a comprehension. def g(seq): for e in seq: try: yield float(e) except: pass # whatever, even skipping the element If you don't need it in generator form, then just collect the results into a list and return that instead of yielding. (And obviously, that for loop doesn't even have to be in a separate (generator or) function at all.) -jJ From steve at pearwood.info Thu Aug 20 16:25:51 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Aug 2009 00:25:51 +1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: <200908210025.51640.steve@pearwood.info> On Thu, 20 Aug 2009 09:07:54 am Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else > expression, that responds to exceptions. ... > _*Simple Example - type coercion in a list comprehension: > *_Current approach: > def safe_float(string): > try: > x = float(string) > except ValueError: > x = float('nan') > return x > ... > xs = (safe(float(string)) for string in strings) Personally, I think the above is likely to be the approach in a serious application. For lightweight applications, an expression-based solution would be acceptable, but it has the disadvantage of not being able to be documented or tested independently. > Proposed solution using exception-based conditional expression: > xs = ((float(string) except ValueError: float('nan')) for string > in strings) Others have suggested that the colon should be dropped. I dislike that idea, because there's nothing but whitespace delimiting the list of exceptions from the except-expression: EXPR except EXCEPT-LIST EXCEPT-EXPR As far as I can tell, that would be unprecedented in Python syntax. ... > Here is a slightly more involved example than the examples presented > above. In data processing, I often string together a sequence of > iterable list comprehensions, corresponding to a sequence of > operations on a given dataset "ys" to produce a processed dataset > "x": > xs = (operation_A(x) for x in ys) > xs = (operation_B(x) for x in xs if filter_B(x)) > xs = (operation_C(x) if (some_condition(x)) else operation_D(x) > for x in xs if filter_C(x)) > # final, explicit list of values > xs = [ x for x in xs ] Ouch! That last one is better written as: xs = list(xs) > This is often a flexible way for me to define processing and > filtering sequences which also seems > to have good performance on very large datasets. Chaining iterators together is a powerful technique, and you don't have to be limited to generator expressions, you can use regular generators too: def operate(xs): for x in xs: try: yield operation_C(x) except ValueError: yield operation_D(x) xs = (operation_A(x) for x in ys) xs = (operation_B(x) for x in xs if filter_B(x)) xs = operate(xs) # finally convert to a list xs = list(xs) > One advantage is > that I can quickly mix-and-match from existing processes like this to > make a new process. An exception-based conditional would go nicely > into many of these process sequences, keeping them both robust and > flexible. xs = (operation_N(x) except exceptionN: operation_Nprime(x) > for x in xs) So far this is an argument for iterator-based processing, not for exception-based conditionals :) You can use exceptions in generator exceptions, you just have to put them into a function first. > I also often have object classes which have some common method or > attribute. For instance, some of my objects have scope-dependent > values: x = y.evaluate(scope)) > where scope is typically locals(), globals(), or some other > dictionary-like container. But, to keep my code modular, I want to > handle, in the same lines of code, objects which do not have some > particular method, which leads me to lines of code like: > x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y Is that real code? I don't think it works the way you think. Given a reasonable definition for y (an instance of a class with no special tricks) the if test will never succeed: >>> class MyObject(object): ... def evaluate(self, arg): ... return arg ... >>> y = MyObject() >>> y.evaluate(42) 42 >>> 'evaluate' in y.__dict__ False A better way of writing that would be: x = y.evaluate(locals()) if hasattr(y, 'evaluate') else y which have the bonus of working correctly even if y gets its evaluate() method via inheritance. Another way, two lines instead of one: try: x = y.evaluate(locals()) except AttributeError: x = y A third method: make sure all objects you pass have an evaluate() method, even if they are a null-operation that just return self. > This seems not very "Pythonic", similar to using type-testing instead > of try-except. You're talking about the Look Before You Leap technique rather than Easier To Ask Forgiveness Than Permission. Both techniques are valid, and both have advantages. ETAFTP is advantageous when exceptions are rare and hard to predict; LBYL is better if exceptions are common and easy to predict. > In the thread of my previous post to py-dev, there were comments, > questions, and suggestions concerning the details of the syntax. > Having reflected on this for a couple weeks, I am now most strongly > supportive of what is essentially just an inline compression of the > current try-except syntax. So the following examples would be > allowed: > x = expression0 except: default_expression I would argue strongly that allowing a bare except is a SERIOUS mistake. Bare excepts have very few legitimate uses, and now that string exceptions are deprecated and very rare, it's no hardship to write: except Exception: if you want to catch everything except KeyboardInterrupt and one or two other things, or: except BaseException: for those rare times you actually do want to catch everything. -- Steven D'Aprano From mcaninch at lanl.gov Thu Aug 20 16:27:26 2009 From: mcaninch at lanl.gov (Jeff McAninch) Date: Thu, 20 Aug 2009 08:27:26 -0600 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression Message-ID: <4A8D5D4E.5090108@lanl.gov> > > -1, if this goes through it will be extended to all other compound > statements over time and we'll end up with 2 or 3 ways to do every > thing. > > A much better idea would be to find a way to make all compound > statements into expressions, future-proofing the decision and avoiding > the redundancy between "compound statement statement" and "compound > statement expression". > Python's list of compound statements is pretty thin (just 7), and 4 of these already have expression forms. for ==> list comprehension, iterator expression while ==> list comprehension, iterator if ==> if-else conditional expression def ==> already have lambda expressions try ==> my proposal class ==> Okay, so this one doesn't have a compound statement, but where would you use it? with ==> I haven't had an occasion to use with, so can't think of an example where it could be useful as a compound expression. Seems to me try-except is kind of sticking out here, and I've tried to show what I think are natural and common places where a try-except expression would be appropriate. -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch at lanl.gov ========================== From steve at pearwood.info Thu Aug 20 16:46:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 21 Aug 2009 00:46:38 +1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8D5D4E.5090108@lanl.gov> References: <4A8D5D4E.5090108@lanl.gov> Message-ID: <200908210046.39235.steve@pearwood.info> On Fri, 21 Aug 2009 12:27:26 am Jeff McAninch wrote: > Python's list of compound statements is pretty thin (just 7), > and 4 of these already have expression forms. > > for ==> list comprehension, iterator expression > while ==> list comprehension, iterator > if ==> if-else conditional expression But that one is ugly (that is, I don't think anyone *likes* the syntax, it's just considered the least worst), and some of us consider that its usefulness is dubious, or at least weak. I think the main reason that Python has a ternary if statement is to stop people writing the error-prone: cond and x or y as a short-circuit ternary if. > def ==> already have lambda expressions > > try ==> my proposal > > class ==> Okay, so this one doesn't have a compound statement, > but where would you use it? But it does have a function: >>> type("MyClass", (object,), {}) Or rather, class is syntactic sugar for the type known as type(), which isn't actually a function, but you call it as if it were one. > with ==> I haven't had an occasion to use with, > so can't think of an example where it could > be useful as a compound expression. > > Seems to me try-except is kind of sticking out here, and I've > tried to show what I think are natural and common places where > a try-except expression would be appropriate. At the moment, I would vote: -0 for the general idea -1 if it allows a bare except I could possibly be convinced to change my first vote to +0 if I could see some real examples of code where this would be useful. (However, I don't think there's any power in the universe which could convince me that allowing bare excepts here was a good thing :) Has anyone considered binding to an exception instance? E.g: x = f(x) except (ValueError, TypeError) as e: extract_from(e) -- Steven D'Aprano From masklinn at masklinn.net Thu Aug 20 16:50:25 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 20 Aug 2009 16:50:25 +0200 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8D5D4E.5090108@lanl.gov> References: <4A8D5D4E.5090108@lanl.gov> Message-ID: On 20 Aug 2009, at 16:27 , Jeff McAninch wrote: > Seems to me try-except is kind of sticking out here, and I've > tried to show what I think are natural and common places where > a try-except expression would be appropriate. Let me be clear (apparently I wasn't clear enough): I'm not arguing against a try/except expression (or, indeed, any expression). I'm arguing against creating a brand new expression instead of trying to find a way to make the current statements into expressions, which I think would be both clearer and better for the language as a whole. From mcaninch at lanl.gov Thu Aug 20 16:52:27 2009 From: mcaninch at lanl.gov (Jeff McAninch) Date: Thu, 20 Aug 2009 08:52:27 -0600 Subject: [Python-ideas] [Fwd: Re: exception based conditional expression, similar to if-else conditional expression] In-Reply-To: References: <4A8D2240.6080107@lanl.gov> <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> <4A8D3112.7030905@lanl.gov> Message-ID: <4A8D632B.9010608@lanl.gov> Jim Jewett wrote: > > Probably, but you don't have to use a comprehension. > > def g(seq): > for e in seq: > try: > yield float(e) > except: > pass # whatever, even skipping the element > > If you don't need it in generator form, then just collect the results > into a list and return that instead of yielding. (And obviously, that > for loop doesn't even have to be in a separate (generator or) function > at all.) > > -jJ > Certainly there are a number of ways to address this without using an exception conditional expression. But it seems each of these would lead to a proliferation of these special functions, as one tries to flexibly address the different exception-value pairs that would be applied to different operations. I could for instance define the function Except: def Except ( seq, nominal_function, function_lookup ): for e in seq: try: yield nominal_function(e) except: (exception_type,exception_message) = sys.exc_info()[:2] if (exception_type in function_lookup): yield function_lookup[exception_type](e) else: raise exception_type, exception_message Then I could write my simple example as: xs = Except( ys, (lambda x: float(x)), {ValueError: (lamba x: float('nan'))} ) So I agree, the behaviour can be produced with the language as-is. But many of the evolutions in the language were not put in to address things that "can't be done", but rather to let them be done in a more concise, robust, and/or elegant way. -- ========================== Jeffrey E. McAninch, PhD Physicist, X-2-IFD Los Alamos National Laboratory Phone: 505-667-0374 Email: mcaninch at lanl.gov ========================== From masklinn at masklinn.net Thu Aug 20 16:54:13 2009 From: masklinn at masklinn.net (Masklinn) Date: Thu, 20 Aug 2009 16:54:13 +0200 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <200908210025.51640.steve@pearwood.info> References: <4A8C85CA.6010306@lanl.gov> <200908210025.51640.steve@pearwood.info> Message-ID: On 20 Aug 2009, at 16:25 , Steven D'Aprano wrote: > > A better way of writing that would be: > > x = y.evaluate(locals()) if hasattr(y, 'evaluate') else y > > which have the bonus of working correctly even if y gets its > evaluate() > method via inheritance. > > Another way, two lines instead of one: > > try: x = y.evaluate(locals()) > except AttributeError: x = y > > > A third method: make sure all objects you pass have an evaluate() > method, even if they are a null-operation that just return self. > Fourth method: x = getattr(y, 'evaluate', lambda _: y)(locals()) From greg.ewing at canterbury.ac.nz Fri Aug 21 00:12:45 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Aug 2009 10:12:45 +1200 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <200908210025.51640.steve@pearwood.info> References: <4A8C85CA.6010306@lanl.gov> <200908210025.51640.steve@pearwood.info> Message-ID: <4A8DCA5D.6080807@canterbury.ac.nz> Steven D'Aprano wrote: > Others have suggested that the colon should be dropped. I dislike that > idea, because there's nothing but whitespace delimiting the list of > exceptions from the except-expression: > > EXPR except EXCEPT-LIST EXCEPT-EXPR I agree. Even if it can be parsed unambiguously, it's going to seem weird and confusing to a human. So far I haven't seen anything I like better than except if despite the fact that it uses the words 'except' and 'if' in a different way than elsewhere. -- Greg From greg.ewing at canterbury.ac.nz Fri Aug 21 00:19:41 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 21 Aug 2009 10:19:41 +1200 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: References: <4A8D5D4E.5090108@lanl.gov> Message-ID: <4A8DCBFD.9050109@canterbury.ac.nz> Masklinn wrote: > I'm > arguing against creating a brand new expression instead of trying to > find a way to make the current statements into expressions, which I > think would be both clearer and better for the language as a whole. Many people have tried, and dismally failed to convince Guido that their suggested solution would be of benefit to the language. -- Greg From ncoghlan at gmail.com Fri Aug 21 11:49:51 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 21 Aug 2009 19:49:51 +1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <200908210046.39235.steve@pearwood.info> References: <4A8D5D4E.5090108@lanl.gov> <200908210046.39235.steve@pearwood.info> Message-ID: <4A8E6DBF.1010902@gmail.com> Steven D'Aprano wrote: > But that one is ugly (that is, I don't think anyone *likes* the syntax, > it's just considered the least worst), and some of us consider that its > usefulness is dubious, or at least weak. I think the main reason that > Python has a ternary if statement is to stop people writing the > error-prone: > > cond and x or y > > as a short-circuit ternary if. That's actually a good point - eliminating the persistent use of buggy workarounds was definitely one of the major motivations behind accepting PEP 308. In this case, there *are* no buggy workarounds: if you want to trap exception, you have to use a real statement. Without that additional motivation of bug prevention, I really doubt this proposal is going to go anywhere. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stefan_ml at behnel.de Fri Aug 21 14:16:53 2009 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 21 Aug 2009 14:16:53 +0200 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> References: <4A8D2240.6080107@lanl.gov> <50697b2c0908200345y4384daa9lf9dcc6e51dade77c@mail.gmail.com> Message-ID: Chris Rebert wrote: > On Thu, Aug 20, 2009 at 3:15 AM, Jeff McAninch wrote: >> Calvin Spealman wrote: >> >>> -1 on colons in the expression like that. I like the idea of being >>> able to handle an exception in generator expressions and the like, but >>> I've never seen a syntax I liked. I think I've favored the idea of >>> something like `float(x) except float('nan') if ValueError` thinking >>> it reads more naturally as an expression, puts the real logic >>> ("convert x to a float or get a NaN float") together, which I think >>> makes sense. >>> >>> >> Yes, I agree about the colons. They have no purpose. I was just blindly >> following the try-except. (Duh! on my part) >> >> So, in the simple example: >> x = float(string) except ValueError float('nan') > > I would feel more comfortable with another keyword in there. Perhaps: > > x = float(string) except ValueError then float('nan') > > Which reads like: > > "x is the float value of string, except in the event of ValueError, > then it's float('nan')" > > Which I find rather pleasing. I'm -1 on this. I think it's of much rarer use than even conditional expressions, has the disadvantage of only handling one special use case without being easily extensible to more than one exception, and it's easily served by a simple function that does the same thing (and potentially much more) but gives it a name. Stefan From ilya.nikokoshev at gmail.com Fri Aug 21 21:51:11 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Fri, 21 Aug 2009 23:51:11 +0400 Subject: [Python-ideas] Interrogate alternate namespace keyword and concept In-Reply-To: <200908151202.29948.steve@pearwood.info> References: <90df837e0908132322r6cbc9337v4d853599eae9ee57@mail.gmail.com> <200908142117.24072.steve@pearwood.info> <200908151202.29948.steve@pearwood.info> Message-ID: Thanks for code tips! I appreciate them. Ok, I kind of not sure I correctly remember why this topic was raised, so let me remember. You said that using a separate function has a performance hit. The purpose of my program was to establish that timing of function interrogate() == statement interrogate + compile time It doesn't appear to me that you dispute this claim, so let's accept it. Since a typical program shouldn't compile lots of code snippets (not in number of millions certainly) this means the performance of function and statement is the same for out purposes. I believe this closes performance question. > min(alist) is a more-direct, simpler, faster, easier to read way of > calculating sorted(alist)[0]. > You're muddying the water by including a for-loop and call to range() > inside the code snippet being tested. We're trying to compare your > function interrogate(test, 'value += 1') with the standard call to > test.value += 1. Why include the time required to generate a range() > object, and iterate over it ten times, as part of the code snippet? All > that does is mix up the time required to execute common code and the > time required to execute the code we care about. > > If it's not obvious why your approach is flawed, consider this extreme > example: > > Timer("time.sleep(1000); interrogate(test, 'value += 1')", ...) > Timer("time.sleep(1000); test.value += 1", ...) > > The differences in speed between the interrogate call (using exec) and > the direct access to test.value will be swamped by the time used by the > common code. > > A more accurate measurement is to remove the "for i in..." part from > command, and increase the number=1000 argument to Timer.repeat() to > 10000. > > >> inc5 = compile_command(command) > > This is an unfair test. We're comparing directly accessing test.value > versus indirectly accessing test.value using exec. Regardless of > whether the caller compiles the statement manually before passing it to > exec, or just passes it to exec to compile it automatically, the cost > of that compilation has to be payed. Pulling that outside of the timing > code just hides the true cost. > > Pre-compiling before passing to exec is a good optimization for the > cases where you need to exec the same code snippet over and over again. > If you want to execute "for i in range(1000): exec(s)" then it makes > sense to pull out the compilation of s outside of the loop. But > generally when timing code snippets, the only purpose of the loop is to > minimize errors and give more accurate results, so pulling out the > compilation just hides some of the real cost. > > >> timing("interrogate(test, command)") >> timing(command.replace('value', 'test.value')) >> timing("interrogate(test, inc5)") >> >> Result: >> >> 15 > > Where does the 15 come from? > >> 'interrogate(test, command)' -> 0.0908 ms >> # test.value = 30015 >> '\nfor i in range(10):\n ? ?test.value += 1\n' -> 0.00408 ms >> # test.value = 60015 >> 'interrogate(test, inc5)' -> 0.00469 ms >> # test.value = 90015 >> >> so interrogate() with additional precompiling introduces very little >> overhead. > > Only because you're ignoring the overhead of pre-compiling. A more > accurate test would be: > > timing("inc5 = compile_command(command); interrogate(test, inc5)") > > >> Though I agree it's inconvenient to write functions as >> strings; > > If you think that's inconvenient, just try writing functions as code > objects without calling compile :) > > > > -- > Steven D'Aprano > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From ilya.nikokoshev at gmail.com Fri Aug 21 22:34:06 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Sat, 22 Aug 2009 00:34:06 +0400 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: First, I'd like to thank you for taking time to write this as a very concise post. Your idea sounds interesting. But I don't think I saw any reasonable (= readable) syntax. Anything with bare semicolon is clearly confusing because bare semicolon has a well-defined meaning as ending a line and starting new block. Anything with bare except clause isn't good because catching all exceptions is a wrong style. Anything that encourages the use of \ line continuation character clearly contradicts the well-established guideline that this character should be used as a last resort. -1 on either of above suggestions. But there are deeper problems even in the "simple example". To see them, let me show how this example should look in programming style I strive for: def safe_float(string: 'any string', invalid = float('nan')) -> 'float': '''Wrapper around float() that converts all invalid values to given param. Use with care --- this will also catch invalid inputs and other cases where notifying the user with exception is a better idea. ''' try: return float(string) except ValueError: return invalid ... class tests: def testSafeFloat(self): assert safe_float('5.7') == float('5.7') assert safe_float('invalid') == float('nan') ... # Here string comes from an experiment and I know it's either empty or should # be valid. xs1 = (float(string or 'nan') for string in strings) # Here string comes from a user input. Tell him if he types something wrong. for string in strings: try: xs2[i] = float(string) catch ValueError: string = input('Please retype string #{}'.format(i)) ... # Here strings are a result of series of experiments. Unfortunately we don't # know what format the researchers used for 'no data' or 'invalid data'. So # for now we'll just change all of these values to 'nan'. xs3 = (safe_float(string) for string in strings) Your proposed solution using exception-based conditional expression: # Here strings are a result of series of experiments. Unfortunately we don't # know what format the researchers used for 'no data' or 'invalid data'. So # for now we'll just change all of these values to 'nan'. # But, unfortunately, if in the future we'll decide to use some different # function to process data, we'll have to rewrite this whole cycle... xs = ((float(string) except ValueError: float('nan')) for string in strings) I think from my examples you get what I'm trying to say -- when you're thinking about easily catching exceptions, you're either ok with something simpler or you're actually trying to apply a function. So why not do it explicitely? Ilya. On Thu, Aug 20, 2009 at 3:07 AM, Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else expression, > that responds to exceptions. > > I had originally posted this (probably mistakenly) on py-dev.? This current > posting is a cleaned up > version of the idea, based on responses I got on from the earlier posting. > > Abstract: > Proposal for a conditional expression, similar to the if-else expression, > that responds to exceptions. > > Motivation: > An expression syntax that responds to exceptions, and which reproduces the > readability and conciseness of the if-else conditional expression, would > simplify some exception-handling cases, especially within list > comprehensions. > > Very Simple Example - type coercion: > Current approach: > ??? try: > ??????? x = float(string) > ??? except: > ??????? x = float('nan') > > Proposed solution using exception-based conditional expression: > ??? x = float(string) except ValueError: float('nan') > > > Simple Example - type coercion in a list comprehension: > Current approach: > ??? def safe_float(string): > ??????? try: > ??????????? x = float(string) > ??????? except ValueError: > ??????????? x = float('nan') > ??????? return x > ??? ... > ??? xs = (safe(float(string)) for string in strings) > > Proposed solution using exception-based conditional expression: > ??? xs = ((float(string) except ValueError: float('nan')) for string in > strings) > > Discussion: > In my own python coding, I find I make common use of the if-else conditional > expression, especially within list comprehensions.? (In one of my packages, > which has ~5800 lines of code, I found if-else expressions in ~1% of the > lines.) > > Here is a slightly more involved example than the examples presented above. > In data processing, I often string together a sequence of iterable list > comprehensions, corresponding to a sequence of operations on a given dataset > "ys" to produce a processed dataset "x": > ??? xs = (operation_A(x) for x in ys) > ??? xs = (operation_B(x) for x in xs if filter_B(x)) > ??? xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in > xs if filter_C(x)) > ??? # final, explicit list of values > ??? xs = [ x for x in xs ] > This is often a flexible way for me to define processing and filtering > sequences which also seems > to have good performance on very large datasets.? One advantage is that I > can quickly mix-and-match from existing processes like this to make a new > process.? An exception-based conditional would go nicely > into many of these process sequences, keeping them both robust and flexible. > ??? xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) > > I also often have object classes which have some common method or > attribute.? For instance, some of my objects have scope-dependent values: > ??? x = y.evaluate(scope)) > where scope is typically locals(), globals(), or some other dictionary-like > container.? But, to keep my code modular, I want to handle, in the same > lines of code, objects which do not have some particular method, which leads > me to lines of code like: > ??? x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y > This seems not very "Pythonic", similar to using type-testing instead of > try-except.? (My impression was that there was a long-standing trend in the > evolution of Python to remove tests like this, and I thought that was the > original motivation for the try-except syntax.) > > I would much rather write: > ??? x = y.evaluate(locals()) except AttributeError: y > or, in the list comprehension example: > ??? xs = (y.evaluate(locals()) except AttributeError: y for y in ys) > > Clearly this can be handled in several ways with the language as it is.? One > way is to define a new function, as in the second simple example above: > ??? def safe_evaluate(y,scope): > ??? ?? try: > ??? ?? ?? x = y.evaluate(scope) > ??? ?? except AttributeError: > ??? ?? ?? x = y > ??? ?? return x > ??? ... > ??? xs = (safe_evaluate(y,locals()) for y in ys) > but this quickly (in my packages at least) leads to an annoying > proliferation of "safe_" functions. > Again, this seems not to be in the "Pythonic" spirit, and is also less > concise, less readable.? (I also suspect, but have not verified, that this > is in general less efficient than in-line expressions -- wasn't that part of > the original motivation for list comprehensions?). > > In the thread of my previous post to py-dev, there were comments, questions, > and suggestions concerning the details of the syntax.? Having reflected on > this for a couple weeks, I am now most strongly supportive of what is > essentially just an inline compression of the current try-except syntax.? So > the following examples would be allowed: > ??? x = expression0 except: default_expression > ??? x = expression0 except exception1: expression1 except exception2: > expression2 except: default_expression > > Or, more generally: > ??? x = expression0\ > ? ? ? ? ? ? except exception1: expression1\ > ??????????? except exception2: expression2\ > ??????????? ... > ??????????? except exceptionI: expressionI\ > ??????????? ... > ??????????? except: default_expression > In this last example, the behaviour would be as follows: > ??? - evaluate expression0. > ??????????? If no exception is encountered, return the result. > ??? - if an exception is encountered, > ??????????? search for the matching exception in the except clauses. > ??? - if a matching exception ("exceptionI") is found, > ??????????? evaluate the corresponding expression ("expressionI"), and > return the result. > ??? - if no matching exception is found, and a default except: clause (i.e., > one without and exception) > ? ? ? ??? is given, evaluate default_expression, and return the result. > ??? - if no matching exception is found, and no default except clause if > given, > ????????? pass the exception on to the caller. > ??? - if a new exception is encountered while evaluating an an except > expression ("expressionI"), > ????????? pass the exception on to the caller. > > I hope I have made a convincing case here.? This seems to me to be a natural > ("Pythonic") addition to the language. > > Jeff McAninch > > -- > ========================== > Jeffrey E. McAninch, PhD > Physicist, X-2-IFD > Los Alamos National Laboratory > Phone: 505-667-0374 > Email: mcaninch at lanl.gov > ========================== > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From rrr at ronadam.com Sat Aug 22 03:28:04 2009 From: rrr at ronadam.com (Ron Adam) Date: Fri, 21 Aug 2009 20:28:04 -0500 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8DCA5D.6080807@canterbury.ac.nz> References: <4A8C85CA.6010306@lanl.gov> <200908210025.51640.steve@pearwood.info> <4A8DCA5D.6080807@canterbury.ac.nz> Message-ID: <4A8F49A4.4000805@ronadam.com> Greg Ewing wrote: > Steven D'Aprano wrote: > >> Others have suggested that the colon should be dropped. I dislike that >> idea, because there's nothing but whitespace delimiting the list of >> exceptions from the except-expression: >> >> EXPR except EXCEPT-LIST EXCEPT-EXPR > > I agree. Even if it can be parsed unambiguously, it's > going to seem weird and confusing to a human. > > So far I haven't seen anything I like better than > > except if > > despite the fact that it uses the words 'except' > and 'if' in a different way than elsewhere. Possibly the only way to do this within those constraints is a purely syntactic. value = (expr1 :exception: expr2) Sense most people (and editors) recognize exceptions names easily, there really isn't a strong need to use the key word "except" in an expression like this. The colons in this case can be considered to be similar to slice syntax. And if the colons are still considered bad, possibly some other symbol could be used. Ron From steve at pearwood.info Sat Aug 22 07:32:14 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 22 Aug 2009 15:32:14 +1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8F49A4.4000805@ronadam.com> References: <4A8C85CA.6010306@lanl.gov> <4A8DCA5D.6080807@canterbury.ac.nz> <4A8F49A4.4000805@ronadam.com> Message-ID: <200908221532.15272.steve@pearwood.info> On Sat, 22 Aug 2009 11:28:04 am Ron Adam wrote: > Greg Ewing wrote: > > Steven D'Aprano wrote: > >> Others have suggested that the colon should be dropped. I dislike > >> that idea, because there's nothing but whitespace delimiting the > >> list of exceptions from the except-expression: > >> > >> EXPR except EXCEPT-LIST EXCEPT-EXPR > > > > I agree. Even if it can be parsed unambiguously, it's > > going to seem weird and confusing to a human. > > > > So far I haven't seen anything I like better than > > > > except if > > > > despite the fact that it uses the words 'except' > > and 'if' in a different way than elsewhere. > > Possibly the only way to do this within those constraints Which constraints? Are you referring to the fact that any proposed syntax must be capable of being unambiguously parsed? > is a purely syntactic. I'm not sure I understand. Can you give an example of a proposal which *isn't* syntactic? > value = (expr1 :exception: expr2) Are the brackets intended to be part of the syntax, or are they there just for grouping? In other words, would this be allowed? value = function(expr1 :exception: expr2) Or would we have to write this? value = function((expr1 :exception: expr2)) > Sense most people (and editors) recognize exceptions names easily, > there really isn't a strong need to use the key word "except" in an > expression like this. The word "except" cues in the reader that they are seeing an exception-expression, rather than some sort of data structure like {key:value} or (a, tuple). For the same reason, list comprehensions use "for": [x+1 for x in alist] rather than mathematical notation: {x+1 ? x ? alist} (if that doesn't display for you, the first odd character is an upside down A, "for all", and the second is a curved E, "element of"). If you don't like unicode-based source code, we could replace the "for all" and "element of" symbols with colons: [x+1 : x : alist] but we don't because the words "for" and "in" give the reader hints as to what is being done, which a colon would not. Words are just symbols, so anytime you have syntax including a keyword, you could replace the keyword with a symbol. That way leads to APL. As a general rule, keywords are more readable, punctuation is more concise. -- Steven D'Aprano From ilya.nikokoshev at gmail.com Sat Aug 22 11:36:26 2009 From: ilya.nikokoshev at gmail.com (ilya) Date: Sat, 22 Aug 2009 13:36:26 +0400 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A8C85CA.6010306@lanl.gov> References: <4A8C85CA.6010306@lanl.gov> Message-ID: I like your use case, but it appears to me a processing task that repeats a lot deserves its own function, so: xs = apply(operation_N, {exceptionN: operation_Nprime}, xs) which also happens to be shorter than xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) Similarly, multiline try/except as in your last example x = expression0\ except exception1: expression1\ except exception2: expression2\ ... except exceptionI: expressionI\ ... except: default_expression is certainly useful. But we're lucky, nearly identical syntax already exists out of the box! try: _ = expression0 except exception1: _ = expression1 except exception2: _ = expression2 ... except exceptionI: _ = expressionI ... except: _ = default_expression x = _ What's so good about the your snippet that is bad about mine? Anyway, instead of either of those I would prefer # In most cases expression0 returns well-formed foo. try: _ = expression0 except exception1: # Take care of the case of bar being not frobnicable. _ = expression1 except exception2: # This looks like magic, but is isn't. See documentation for # exception2 at http:// some_url for explanation. _ = expression2 ... except exceptionI: _ = expressionI ... except exceptionLast: _ = expressionLast # All cases I could think of are exhausted! Of course, anything can # happen so any other exception will propagate up. x = _ On Thu, Aug 20, 2009 at 3:07 AM, Jeff McAninch wrote: > I would like to propose an expression, similar to the if-else expression, > that responds to exceptions. > > I had originally posted this (probably mistakenly) on py-dev.? This current > posting is a cleaned up > version of the idea, based on responses I got on from the earlier posting. > > Abstract: > Proposal for a conditional expression, similar to the if-else expression, > that responds to exceptions. > > Motivation: > An expression syntax that responds to exceptions, and which reproduces the > readability and conciseness of the if-else conditional expression, would > simplify some exception-handling cases, especially within list > comprehensions. > > Very Simple Example - type coercion: > Current approach: > ??? try: > ??????? x = float(string) > ??? except: > ??????? x = float('nan') > > Proposed solution using exception-based conditional expression: > ??? x = float(string) except ValueError: float('nan') > > > Simple Example - type coercion in a list comprehension: > Current approach: > ??? def safe_float(string): > ??????? try: > ??????????? x = float(string) > ??????? except ValueError: > ??????????? x = float('nan') > ??????? return x > ??? ... > ??? xs = (safe(float(string)) for string in strings) > > Proposed solution using exception-based conditional expression: > ??? xs = ((float(string) except ValueError: float('nan')) for string in > strings) > > Discussion: > In my own python coding, I find I make common use of the if-else conditional > expression, especially within list comprehensions.? (In one of my packages, > which has ~5800 lines of code, I found if-else expressions in ~1% of the > lines.) > > Here is a slightly more involved example than the examples presented above. > In data processing, I often string together a sequence of iterable list > comprehensions, corresponding to a sequence of operations on a given dataset > "ys" to produce a processed dataset "x": > ??? xs = (operation_A(x) for x in ys) > ??? xs = (operation_B(x) for x in xs if filter_B(x)) > ??? xs = (operation_C(x) if (some_condition(x)) else operation_D(x) for x in > xs if filter_C(x)) > ??? # final, explicit list of values > ??? xs = [ x for x in xs ] > This is often a flexible way for me to define processing and filtering > sequences which also seems > to have good performance on very large datasets.? One advantage is that I > can quickly mix-and-match from existing processes like this to make a new > process.? An exception-based conditional would go nicely > into many of these process sequences, keeping them both robust and flexible. > ??? xs = (operation_N(x) except exceptionN: operation_Nprime(x) for x in xs) > > I also often have object classes which have some common method or > attribute.? For instance, some of my objects have scope-dependent values: > ??? x = y.evaluate(scope)) > where scope is typically locals(), globals(), or some other dictionary-like > container.? But, to keep my code modular, I want to handle, in the same > lines of code, objects which do not have some particular method, which leads > me to lines of code like: > ??? x = y.evaluate(locals()) if ('evaluate' in y.__dict__) else y > This seems not very "Pythonic", similar to using type-testing instead of > try-except.? (My impression was that there was a long-standing trend in the > evolution of Python to remove tests like this, and I thought that was the > original motivation for the try-except syntax.) > > I would much rather write: > ??? x = y.evaluate(locals()) except AttributeError: y > or, in the list comprehension example: > ??? xs = (y.evaluate(locals()) except AttributeError: y for y in ys) > > Clearly this can be handled in several ways with the language as it is.? One > way is to define a new function, as in the second simple example above: > ??? def safe_evaluate(y,scope): > ??? ?? try: > ??? ?? ?? x = y.evaluate(scope) > ??? ?? except AttributeError: > ??? ?? ?? x = y > ??? ?? return x > ??? ... > ??? xs = (safe_evaluate(y,locals()) for y in ys) > but this quickly (in my packages at least) leads to an annoying > proliferation of "safe_" functions. > Again, this seems not to be in the "Pythonic" spirit, and is also less > concise, less readable.? (I also suspect, but have not verified, that this > is in general less efficient than in-line expressions -- wasn't that part of > the original motivation for list comprehensions?). > > In the thread of my previous post to py-dev, there were comments, questions, > and suggestions concerning the details of the syntax.? Having reflected on > this for a couple weeks, I am now most strongly supportive of what is > essentially just an inline compression of the current try-except syntax.? So > the following examples would be allowed: > ??? x = expression0 except: default_expression > ??? x = expression0 except exception1: expression1 except exception2: > expression2 except: default_expression > > Or, more generally: > ??? x = expression0\ > ? ? ? ? ? ? except exception1: expression1\ > ??????????? except exception2: expression2\ > ??????????? ... > ??????????? except exceptionI: expressionI\ > ??????????? ... > ??????????? except: default_expression > In this last example, the behaviour would be as follows: > ??? - evaluate expression0. > ??????????? If no exception is encountered, return the result. > ??? - if an exception is encountered, > ??????????? search for the matching exception in the except clauses. > ??? - if a matching exception ("exceptionI") is found, > ??????????? evaluate the corresponding expression ("expressionI"), and > return the result. > ??? - if no matching exception is found, and a default except: clause (i.e., > one without and exception) > ? ? ? ??? is given, evaluate default_expression, and return the result. > ??? - if no matching exception is found, and no default except clause if > given, > ????????? pass the exception on to the caller. > ??? - if a new exception is encountered while evaluating an an except > expression ("expressionI"), > ????????? pass the exception on to the caller. > > I hope I have made a convincing case here.? This seems to me to be a natural > ("Pythonic") addition to the language. > > Jeff McAninch > > -- > ========================== > Jeffrey E. McAninch, PhD > Physicist, X-2-IFD > Los Alamos National Laboratory > Phone: 505-667-0374 > Email: mcaninch at lanl.gov > ========================== > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From rrr at ronadam.com Sat Aug 22 18:29:19 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 22 Aug 2009 11:29:19 -0500 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <200908221532.15272.steve@pearwood.info> References: <4A8C85CA.6010306@lanl.gov> <4A8DCA5D.6080807@canterbury.ac.nz> <4A8F49A4.4000805@ronadam.com> <200908221532.15272.steve@pearwood.info> Message-ID: <4A901CDF.8080908@ronadam.com> Steven D'Aprano wrote: > On Sat, 22 Aug 2009 11:28:04 am Ron Adam wrote: >> Greg Ewing wrote: >>> Steven D'Aprano wrote: >>>> Others have suggested that the colon should be dropped. I dislike >>>> that idea, because there's nothing but whitespace delimiting the >>>> list of exceptions from the except-expression: >>>> >>>> EXPR except EXCEPT-LIST EXCEPT-EXPR >>> I agree. Even if it can be parsed unambiguously, it's >>> going to seem weird and confusing to a human. >>> >>> So far I haven't seen anything I like better than >>> >>> except if >>> >>> despite the fact that it uses the words 'except' >>> and 'if' in a different way than elsewhere. >> Possibly the only way to do this within those constraints > > Which constraints? Are you referring to the fact that any proposed > syntax must be capable of being unambiguously parsed? And also to not using 'except' and 'if' in a different way than elsewhere. >> is a purely syntactic. > > I'm not sure I understand. Can you give an example of a proposal which > *isn't* syntactic? Technically you are correct, but I was trying to refer to patterns that do not use keywords to identify the operation. For example slice operations use a pattern rather than a unique symbol to identify the operation. So this is an example of a pattern that could work. >> value = (expr1 :exception: expr2) > Are the brackets intended to be part of the syntax, or are they there > just for grouping? In other words, would this be allowed? Yes, I intended the brackets to be part of the pattern. > value = function(expr1 :exception: expr2) > > Or would we have to write this? > > value = function((expr1 :exception: expr2)) I supose it could work like generator expressions do. Then both of these could be equivalent. >> Sense most people (and editors) recognize exceptions names easily, >> there really isn't a strong need to use the key word "except" in an >> expression like this. > > The word "except" cues in the reader that they are seeing an > exception-expression, rather than some sort of data structure like > {key:value} or (a, tuple). True, it needs to be unique enough to be easily identifiable or it would not be worth it. > For the same reason, list comprehensions use "for": > > [x+1 for x in alist] > > rather than mathematical notation: > > {x+1 ? x ? alist} It displayed fine. :-) > (if that doesn't display for you, the first odd character is an upside > down A, "for all", and the second is a curved E, "element of"). If you > don't like unicode-based source code, we could replace the "for all" > and "element of" symbols with colons: > > [x+1 : x : alist] I think having the square brackets with the colons would be a bit too close to slice notation here. ie... it's not unique enough to be easily identifiable. > but we don't because the words "for" and "in" give the reader hints as > to what is being done, which a colon would not. This is especially true for more complex operations such as looping and repeating. > Words are just symbols, so anytime you have syntax including a keyword, > you could replace the keyword with a symbol. That way leads to APL. As > a general rule, keywords are more readable, punctuation is more > concise. Yes, I agree, but python isn't that strict about everything. To me a well defined exception expression that All the previous suggestions view the term as needing three values, so you need two separators along with unique symbols (or keywords) to come up with something that fits the need in a clear and concise way. I was thinking what if we could use the exception object itself as an operator, then the term becomes even simpler and more concise. But the only way I can think to do that in a clear way is to use syntax to identify the exception, such as putting colons, or some other symbol around the exception. Ron From rrr at ronadam.com Sat Aug 22 21:53:33 2009 From: rrr at ronadam.com (Ron Adam) Date: Sat, 22 Aug 2009 14:53:33 -0500 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A901CDF.8080908@ronadam.com> References: <4A8C85CA.6010306@lanl.gov> <4A8DCA5D.6080807@canterbury.ac.nz> <4A8F49A4.4000805@ronadam.com> <200908221532.15272.steve@pearwood.info> <4A901CDF.8080908@ronadam.com> Message-ID: <4A904CBD.6080601@ronadam.com> Ron Adam wrote: > > > Steven D'Aprano wrote: >> Words are just symbols, so anytime you have syntax including a >> keyword, you could replace the keyword with a symbol. That way leads >> to APL. As a general rule, keywords are more readable, punctuation is >> more concise. > > Yes, I agree, but python isn't that strict about everything. To me a > well defined exception expression that Editing fragment... Disregard the incomplete sentence. > All the previous suggestions view the term as needing three values, so > you need two separators along with unique symbols (or keywords) to come > up with something that fits the need in a clear and concise way. I was > thinking what if we could use the exception object itself as an > operator, then the term becomes even simpler and more concise. But the > only way I can think to do that in a clear way is to use syntax to > identify the exception, such as putting colons, or some other symbol > around the exception. An additional note, the when you view an exception object in the console it is bracketed by the less than and and greater than signs. Maybe there is a way to use those in a manner that is consistent with the string repr() returns. Ron From cmjohnson.mailinglist at gmail.com Sun Aug 23 00:23:04 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 22 Aug 2009 12:23:04 -1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A904CBD.6080601@ronadam.com> References: <4A8C85CA.6010306@lanl.gov> <4A8DCA5D.6080807@canterbury.ac.nz> <4A8F49A4.4000805@ronadam.com> <200908221532.15272.steve@pearwood.info> <4A901CDF.8080908@ronadam.com> <4A904CBD.6080601@ronadam.com> Message-ID: <3bdda690908221523o68fcae55n9a9f47ac536e8ea6@mail.gmail.com> Ron Adam wrote: > An additional note, the when you view an exception object in the console it > is bracketed by the less than and and greater than signs. ?Maybe there is a > way to use those in a manner that is consistent with the string repr() > returns. There won't be unless you rule out using "x > y" as an expression that gives the default value. But doing that will make Python's grammar way too complicated, and you're still left with the problem of "x, y = a < b, b > c" which is legal today. It's my understanding that Ruby has run into a lot of ambiguity with its use of | as both a bitwise operator and a part of a block definition. This seems worse. From steve at pearwood.info Sun Aug 23 03:39:54 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 23 Aug 2009 11:39:54 +1000 Subject: [Python-ideas] exception based conditional expression, similar to if-else conditional expression In-Reply-To: <4A904CBD.6080601@ronadam.com> References: <4A8C85CA.6010306@lanl.gov> <4A901CDF.8080908@ronadam.com> <4A904CBD.6080601@ronadam.com> Message-ID: <200908231139.55627.steve@pearwood.info> On Sun, 23 Aug 2009 05:53:33 am Ron Adam wrote: > An additional note, the when you view an exception object in the > console it is bracketed by the less than and and greater than signs. > ?Maybe there is a way to use those in a manner that is consistent > with the string repr() returns. Lots of things have a repr() which includes < and > signs, including repr() itself. >>> print repr Also new style classes and instances, types, functions, methods, etc. Exceptions are printed with < > signs because exceptions are subclasses of object. There's nothing specifically "exception-like" about the use of < and > in the repr of exceptions. -- Steven D'Aprano From travis+ml-python-ideas at subspacefield.org Mon Aug 24 22:14:58 2009 From: travis+ml-python-ideas at subspacefield.org (travis+ml-python-ideas at subspacefield.org) Date: Mon, 24 Aug 2009 15:14:58 -0500 Subject: [Python-ideas] expose {set,get}res{uid,gid} in posixmodule.c Message-ID: <20090824201458.GA6647@subspacefield.org> So there has been a little discussion on the python mailing list regarding a request I had to expose {set,get}res{uid,gid}. The general consensus there is that even though it isn't a POSIX standard, it is shared among many Unix OSes and belongs in the same module as setuid and friends, namely os, which maps to posixmodule.c Previous discusion and background as to why this is desirable: http://bugs.python.org/issue6508 If this is the Right Way to do it, I'll brush up on autotools and see about submitting a patch... -- Obama Nation | My emails do not have attachments; it's a digital signature that your mail program doesn't understand. | http://www.subspacefield.org/~travis/ If you are a spammer, please email john at subspacefield.org to get blacklisted. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From dickinsm at gmail.com Tue Aug 25 16:25:23 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 25 Aug 2009 15:25:23 +0100 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> Message-ID: <5c6f2a5d0908250725yc1d1808p60ae4711f47ffe1b@mail.gmail.com> [Nick Coghlan] > What would we be converting them to in that case? 2.x strings? (I don't > have a problem with that, just pointing out there may be some additional > work due to changing the target type). Yes, the 2.x 'str' type. I haven't thought about whether this could cause 2-to-3 translation problems for people using this in 2.x. (I don't immediately see why it would.) Might there be political reasons not to backport this to 2.x? I seem to recall it being suggested at the PyCon language summit that we should consider making new features 3.x only, but I don't entirely remember what the rationale for this was. [Alexandre Vassalotti] > On Sat, Aug 15, 2009 at 2:42 PM, Mark Dickinson wrote: >> If these additions to int go in, then presumably the _PyLong_AsBytes >> and _PyLong_FromBytes functions should be documented and made public >> (and have their leading underscores removed, too). > > You are referring to _PyLong_FromByteArray and _PyLong_AsByteArray, right? Whoops! Yes, that's what I meant. Thanks. :) Sorry for my silence on the issue tracker. I'll try to find time to look at your new patch this weekend. Mark From ncoghlan at gmail.com Wed Aug 26 13:13:33 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 26 Aug 2009 21:13:33 +1000 Subject: [Python-ideas] Direct byte<->int conversions (was Re: bitwise operations on bytes) In-Reply-To: <5c6f2a5d0908250725yc1d1808p60ae4711f47ffe1b@mail.gmail.com> References: <42ef4ee0908070254m6779d191g1c545bf16106a456@mail.gmail.com> <42ef4ee0908100402h25df9a5j2a45c67ccb264050@mail.gmail.com> <4A801D0C.2000502@gmail.com> <5c6f2a5d0908150423l4f595574q64c521d2398443d8@mail.gmail.com> <42ef4ee0908150513r6fb4dddaicaa4942edd231532@mail.gmail.com> <5c6f2a5d0908150925l19e13f1bg4bd0c4c18ea8be1b@mail.gmail.com> <5c6f2a5d0908151142q782740e2j6d64dfc705e3d612@mail.gmail.com> <5c6f2a5d0908250725yc1d1808p60ae4711f47ffe1b@mail.gmail.com> Message-ID: <4A9518DD.8010506@gmail.com> Mark Dickinson wrote: > [Nick Coghlan] >> What would we be converting them to in that case? 2.x strings? (I don't >> have a problem with that, just pointing out there may be some additional >> work due to changing the target type). > > Yes, the 2.x 'str' type. I haven't thought about whether this could cause > 2-to-3 translation problems for people using this in 2.x. (I don't > immediately see why it would.) > > Might there be political reasons not to backport this to 2.x? I seem to > recall it being suggested at the PyCon language summit that we > should consider making new features 3.x only, but I don't entirely > remember what the rationale for this was. I wasn't there, but there were rumbles about having a few nice carrots in 3.x to help people think it was worthwhile to switch. That said, I think the final consensus was that new features *must* go into 3.x, but if they can be backported and someone is happy to do the work then backporting isn't an issue. And in this case, the fact that the methods will produce/consume strings in 2.x shouldn't cause any more problems than any other cases that involve 2.x storing binary data in text strings. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Thu Aug 27 06:13:02 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 27 Aug 2009 04:13:02 +0000 (UTC) Subject: [Python-ideas] =?utf-8?b?ZXhwb3NlIHtzZXQsZ2V0fXJlc3t1aWQsZ2lkfSBp?= =?utf-8?q?n_posixmodule=2Ec?= References: <20090824201458.GA6647@subspacefield.org> Message-ID: writes: > > So there has been a little discussion on the python mailing list regarding > a request I had to expose {set,get}res{uid,gid}. > > The general consensus there is that even though it isn't a POSIX standard, > it is shared among many Unix OSes and belongs in the same module as setuid > and friends, namely os, which maps to posixmodule.c It seems reasonable indeed. > Previous discusion and background as to why this is desirable: > http://bugs.python.org/issue6508 > > If this is the Right Way to do it, I'll brush up on autotools and see > about submitting a patch... Please do! Also, once a patch has been submitted, further discussion and review should take place on the aforementioned tracker issue. Regards Antoine. From ubershmekel at gmail.com Mon Aug 31 02:45:51 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Mon, 31 Aug 2009 03:45:51 +0300 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion Message-ID: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> I believe int(s, base) needs an inverse function to allow string representation with different bases. An example use case is 'hashing' a counter like video ID's on youtube, you could use a regular int internally and publish a shorter base-62 id for links. This subject was discussed 2.5 years ago: http://mail.python.org/pipermail/python-dev/2006-January/059789.html I opened a feature request ticket: http://bugs.python.org/issue6783 Some of the questions that remain: 1. Whether this should be a method for int or a regular function in a standard library module like math. 2. What should the method/function be called? (base_convert, radix, etc) What do you guys think? RunThePun -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Mon Aug 31 06:49:53 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sun, 30 Aug 2009 18:49:53 -1000 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> Message-ID: <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> How would it work for bases beyond 10? OK, so base-16 has a well known encoding, but what about base-100? Base-16 is case insensitive, but would Base-37 be case insensitive?? ISTM that the common encodings of bin, oct, and hex are already covered by built-in functions and for bases beyond that, you have decide what system to use for encoding symbols that are out of range. Do people really need trinary through septary that often? Is there a single, best-practices way of doing base-1013? Feels like YAGNI to me, but I could be wrong. ? Carl Johnson From cmjohnson.mailinglist at gmail.com Mon Aug 31 06:55:03 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sun, 30 Aug 2009 18:55:03 -1000 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> Message-ID: <3bdda690908302155v145d7578ua9e994db208327a7@mail.gmail.com> Doing some testing with int, I found: ValueError: int() base must be >= 2 and <= 36 That resolves a lot of questions (it has to be case insensitive), but takes away ability to do the motivating use case: writing out video ids in Base-62. ?Carl On Sun, Aug 30, 2009 at 6:49 PM, Carl Johnson wrote: > How would it work for bases beyond 10? OK, so base-16 has a well known > encoding, but what about base-100? Base-16 is case insensitive, but > would Base-37 be case insensitive?? > > ISTM that the common encodings of bin, oct, and hex are already > covered by built-in functions and for bases beyond that, you have > decide what system to use for encoding symbols that are out of range. > Do people really need trinary through septary that often? Is there a > single, best-practices way of doing base-1013? Feels like YAGNI to me, > but I could be wrong. > > ? Carl Johnson > From greg at krypto.org Mon Aug 31 08:53:12 2009 From: greg at krypto.org (Gregory P. Smith) Date: Sun, 30 Aug 2009 23:53:12 -0700 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <3bdda690908302155v145d7578ua9e994db208327a7@mail.gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> <3bdda690908302155v145d7578ua9e994db208327a7@mail.gmail.com> Message-ID: <52dc1c820908302353t5ddf8012u98dd6ee2b784bfc5@mail.gmail.com> On Sun, Aug 30, 2009 at 9:55 PM, Carl Johnson < cmjohnson.mailinglist at gmail.com> wrote: > Doing some testing with int, I found: > > ValueError: int() base must be >= 2 and <= 36 > > That resolves a lot of questions (it has to be case insensitive), but > takes away ability to do the motivating use case: writing out video > ids in Base-62. > If there are widely accepted and used non-controversial standard character -> digit mappings for bases greater than that I don't immediately see any reasons not to support them for ints but I do not think we should go beyond base 255. If someone wants base 256 they should use struct.pack() and beyond that I expect to find people fighting over encoding schemes. ;) Since this is the ideas list... Consider adding a digit character sequence or map passed to the conversion function defining which characters map to which digit number. > > ?Carl > > On Sun, Aug 30, 2009 at 6:49 PM, Carl > Johnson wrote: > > How would it work for bases beyond 10? OK, so base-16 has a well known > > encoding, but what about base-100? Base-16 is case insensitive, but > > would Base-37 be case insensitive?? > > > > ISTM that the common encodings of bin, oct, and hex are already > > covered by built-in functions and for bases beyond that, you have > > decide what system to use for encoding symbols that are out of range. > > Do people really need trinary through septary that often? Is there a > > single, best-practices way of doing base-1013? Feels like YAGNI to me, > > but I could be wrong. > > > > ? Carl Johnson > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Mon Aug 31 10:21:45 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 31 Aug 2009 20:21:45 +1200 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> <3bdda690908302149m2cae05fdwd7bb399f34cd1cd3@mail.gmail.com> Message-ID: <4A9B8819.6000107@canterbury.ac.nz> Carl Johnson wrote: > How would it work for bases beyond 10? If you keep going through the alphabet you can get up to base 36. If anyone cares beyond that, it could take a string of characters to use for digits. With Unicode that would provide quite a wide range. :-) -- Greg From masklinn at masklinn.net Mon Aug 31 10:32:44 2009 From: masklinn at masklinn.net (Masklinn) Date: Mon, 31 Aug 2009 10:32:44 +0200 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> Message-ID: <31909350-5278-4B75-B9B6-081D6364FF94@masklinn.net> On 31 Aug 2009, at 02:45 , Yuvgoog Greenle wrote: > Some of the questions that remain: > 1. Whether this should be a method for int or a regular function in a > standard library module like math. > 2. What should the method/function be called? (base_convert, radix, > etc) > > What do you guys think? The format mini-language already has presentation types for binary, octal, dec and hex outputs. Maybe this could be expanded in some way to get arbitrary base output (between 2 and 36 to match int()) From ncoghlan at gmail.com Mon Aug 31 15:00:27 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 31 Aug 2009 23:00:27 +1000 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> Message-ID: <4A9BC96B.2010207@gmail.com> Yuvgoog Greenle wrote: > I believe int(s, base) needs an inverse function to allow string > representation with different bases. An example use case is 'hashing' a > counter like video ID's on youtube, you could use a regular int > internally and publish a shorter base-62 id > for links. > > This subject was discussed 2.5 years ago: > http://mail.python.org/pipermail/python-dev/2006-January/059789.html > > I opened a feature request ticket: > http://bugs.python.org/issue6783 > > Some of the questions that remain: > 1. Whether this should be a method for int or a regular function in a > standard library module like math. > 2. What should the method/function be called? (base_convert, radix, etc) > > What do you guys think? This has been coming up for years and always gets bogged down in a spelling argument (a method on int, a function in the math module and an update to the str.format mini language would be the current contenders). However, most of the actual real use cases for bases between 2 and 36 were dealt with by the addition of binary and octal output to string formatting so the impetus to do anything about it is now a lot lower. As far as bases between 37 and 62 go, that would involve first getting agreement on extending int() to handle those bases by allowing case sensitive digit parsing. Presumably that would use string lexical ordering so that int('a', 37) > int('A', 37) and int('b', 37) would raise an exception. That would only be intuitive to someone that knows how ASCII based alphanumeric ordering works though. Probably not an impossible sell, but you would want a compelling use case (converting long strings of digits into shorter link URLs is a pretty good one though - I've seen that in places other than youtube). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From masklinn at masklinn.net Mon Aug 31 15:58:53 2009 From: masklinn at masklinn.net (Masklinn) Date: Mon, 31 Aug 2009 15:58:53 +0200 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <4A9BC96B.2010207@gmail.com> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> <4A9BC96B.2010207@gmail.com> Message-ID: <1AD8D421-A250-421D-A6ED-4992234A5B96@masklinn.net> On 31 Aug 2009, at 15:00 , Nick Coghlan wrote: Yuvgoog Greenle wrote: >> I believe int(s, base) needs an inverse function to allow string >> representation with different bases. An example use case is >> 'hashing' a >> counter like video ID's on youtube, you could use a regular int >> internally and publish a shorter base-62 id >> for links. >> >> This subject was discussed 2.5 years ago: >> http://mail.python.org/pipermail/python-dev/2006-January/059789.html >> >> I opened a feature request ticket: >> http://bugs.python.org/issue6783 >> >> Some of the questions that remain: >> 1. Whether this should be a method for int or a regular function in a >> standard library module like math. >> 2. What should the method/function be called? (base_convert, radix, >> etc) >> >> What do you guys think? > > This has been coming up for years and always gets bogged down in a > spelling argument (a method on int, a function in the math module > and an > update to the str.format mini language would be the current > contenders). > > However, most of the actual real use cases for bases between 2 and 36 > were dealt with by the addition of binary and octal output to string > formatting so the impetus to do anything about it is now a lot lower. > > As far as bases between 37 and 62 go, that would involve first getting > agreement on extending int() to handle those bases by allowing case > sensitive digit parsing. Presumably that would use string lexical > ordering so that int('a', 37) > int('A', 37) and int('b', 37) would > raise an exception. > > That would only be intuitive to someone that knows how ASCII based > alphanumeric ordering works though. Or it could be handled via a translation table (needed both ways of course) mapping n indexes to n characters (with n the base you're working with), defaulting to something sane. Though I'm not sure this is of much interest really: even Erlang (which provides pretty good base conversion tools: it supports literal integers of any base between 2 and 36) doesn't natively support bases beyond 36. A library would probably be better for those more conflictual (or less intuitive) ranges. From python at mrabarnett.plus.com Mon Aug 31 17:47:24 2009 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 31 Aug 2009 16:47:24 +0100 Subject: [Python-ideas] Add a builtin method to 'int' for base/radix conversion In-Reply-To: <1AD8D421-A250-421D-A6ED-4992234A5B96@masklinn.net> References: <9d153b7c0908301745n51a9751avbaf16d0e155fec7e@mail.gmail.com> <4A9BC96B.2010207@gmail.com> <1AD8D421-A250-421D-A6ED-4992234A5B96@masklinn.net> Message-ID: <4A9BF08C.9060607@mrabarnett.plus.com> Masklinn wrote: > On 31 Aug 2009, at 15:00 , Nick Coghlan wrote: > Yuvgoog Greenle wrote: >>> I believe int(s, base) needs an inverse function to allow string >>> representation with different bases. An example use case is 'hashing' a >>> counter like video ID's on youtube, you could use a regular int >>> internally and publish a shorter base-62 id >>> for links. >>> >>> This subject was discussed 2.5 years ago: >>> http://mail.python.org/pipermail/python-dev/2006-January/059789.html >>> >>> I opened a feature request ticket: >>> http://bugs.python.org/issue6783 >>> >>> Some of the questions that remain: >>> 1. Whether this should be a method for int or a regular function in a >>> standard library module like math. >>> 2. What should the method/function be called? (base_convert, radix, etc) >>> >>> What do you guys think? >> >> This has been coming up for years and always gets bogged down in a >> spelling argument (a method on int, a function in the math module and an >> update to the str.format mini language would be the current contenders). >> >> However, most of the actual real use cases for bases between 2 and 36 >> were dealt with by the addition of binary and octal output to string >> formatting so the impetus to do anything about it is now a lot lower. >> >> As far as bases between 37 and 62 go, that would involve first getting >> agreement on extending int() to handle those bases by allowing case >> sensitive digit parsing. Presumably that would use string lexical >> ordering so that int('a', 37) > int('A', 37) and int('b', 37) would >> raise an exception. >> >> That would only be intuitive to someone that knows how ASCII based >> alphanumeric ordering works though. ASCII? Surely it should be Unicode! :-) > Or it could be handled via a translation table (needed both ways of > course) mapping n indexes to n characters (with n the base you're > working with), defaulting to something sane. > The default could cover only bases 2 to 36. Any base > 36 would require a user-supplied translation table. > Though I'm not sure this is of much interest really: even Erlang (which > provides pretty good base conversion tools: it supports literal integers > of any base between 2 and 36) doesn't natively support bases beyond 36. > A library would probably be better for those more conflictual (or less > intuitive) ranges. > It could permit a dict as the translation table when 'decoding' so that both 'A' and 'a' could be mapped to 10, if necessary.