From cmjohnson.mailinglist at gmail.com Wed Dec 1 01:49:29 2010 From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson) Date: Tue, 30 Nov 2010 14:49:29 -1000 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: This has been discussed before. A couple points: - Chinese Python already exists (existed), see http://www.chinesepython.org/cgi_bin/cgb.cgi/english/english.html but it looks like the project has been dead for several years. Why? Because no one really wants a completely localized Python - The consensus seems to be that while it is a good idea to translate the docstrings/Python docs, there's no point in translating keywords or module names. For one thing, if you ever want to use an outside module, there's going to have be some way of saying that the chrono module = the time module. As a programmer, you can't get away with not knowing that when things go wrong. For another thing, what happens if in english.py I have a variable named "por" and in spanish.py I have a variable named "for"? How can these modules work together? - The comment that tends to come up in these threads is that "non-native English speakers are going to have to learn 'a new language' anyway, so what's the difference between learning to map a word in their own language to a Python concept to learning to map an arbitrary English word to a Python concept. In either case, it doesn't make any sense to write, "for the sake of justice:" in English or "??????" in Japanese. Python's "for" is not the English "for." It's its own thing. It means "process this iterator in a loop." That's different from the English "for" and it's going to be different from any existing word in any other language either. - That said, letting people use unicode variable names is basically a good idea and has already been implemented in Python 3 and can be turned on in Python 2 with the right declarations. Hope that helps, -- Carl Johnson -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Dec 1 01:57:38 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 1 Dec 2010 10:57:38 +1000 Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module In-Reply-To: <5214C323-9A3C-40ED-B018-D4C20F0C44BB@masklinn.net> References: <4CEC234D.6020301@netwok.org> <20101123215458.53eed0b0@pitrou.net> <4CEC4F1F.2090007@canterbury.ac.nz> <4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net> <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net> <4CF54D1A.6060106@netwok.org> <5214C323-9A3C-40ED-B018-D4C20F0C44BB@masklinn.net> Message-ID: On Wed, Dec 1, 2010 at 8:08 AM, Masklinn wrote: >> ?That?s why isintance+ABCs is the right test to use here. ?See >> http://docs.python.org/dev/reference/datamodel#special-method-names > Due to Python's lookup rule, looking up most (if not all) of the special method on the instance is going to give the same result. It's the "most" in that sentence that makes the ABC the more accurate check of the examples given. Checking the instance does work most of the time (since most instances don't have magic methods attached) but fails abysmally when the instance in question is a class and the intent is to invoke the magic methods of the metaclass (usually 'type'). Deliberately bypassing the instance with either type(x) or x.__class__ is the closest pure Python code can get to correctly following the special method lookup rules: http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From stephen at xemacs.org Wed Dec 1 03:27:05 2010 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 01 Dec 2010 11:27:05 +0900 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: <87lj4a4492.fsf@uwakimon.sk.tsukuba.ac.jp> Dima Tisnek writes: > Let's recall Guido's old Computer Programming for Everybody (CP4E) proposal. > > Nowadays that Python is established, it's high time to push Python > into education, especially first programming language education. I > think, in the modern world it means pre-school. > > Now the larger part of the world's children doesn't learn English > before school, therefore we need to have truly localized Python. Not really. Python is a conventional programming language; you need to learn its syntax and semantics, which can't be changed when you change the strings that denote the keywords and operators. It's programming semantics that are foreign to almost all people, whether native English speakers or not. That's the difficult hurdle to overcome, not memorizing a few dozen keywords, builtins, and module names in an unfamiliar alphabet. *Especially* for preschoolers. Put language in front of them, and they *will* learn it (if it interests them at all, and sometimes even if not). At least for Japanese children, they start learning the alphabet as early as they learn even kana (the Japanese "alphabet"), let alone kanji (the thousands of Chinese ideographs). How else can a kid recognize "Coke" or "Pringles" in the grocery store or at a vending machine? Surely that's far more important (to the kid) than learning Python! Localized identifiers, on the other hand, *are* useful because they help the programmer in the task of design by activating connotations of domain knowledge in the real world. But Python already has those. From dimaqq at gmail.com Wed Dec 1 03:47:56 2010 From: dimaqq at gmail.com (Dima Tisnek) Date: Tue, 30 Nov 2010 19:47:56 -0700 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: 2010/11/30 Carl M. Johnson : > This has been discussed before. A couple points: > - Chinese Python already exists (existed), > see?http://www.chinesepython.org/cgi_bin/cgb.cgi/english/english.html but it > looks like the project has been dead for several years. Why? Because no one > really wants a completely localized Python Well it is clear that professional programmers will master Python syntax in no time, and if someone writes a module, its functions as well as docs better be in English or at least a widely spoken language for the module be used by someone. I'm not trying to put a Chinese Room Argument here ;-) I'm trying to address a very specific case, that is young programmer education. If a young programmer were to, say, name their variables in arabic, why should they be forced to use necessarily awkwards writing direction changes and why should they need to learn a foreign language concept "[... for ... in ... if ...]" or understnad concatenation "elif" or memorize a few hundred standard type methods. My point with young programmers is that they are not learning Python per se, but rather the concept of programming. For example, quite often learning Boolean logic happens during programming classes. Surely it would go better if learner can use native words for "and", "or", "... if ... else ...", etc It would go better if the young mind was not cluttered with unfamiliar conventions and words on top of semantics. Also, the way I see it, the best thing about Python is its libraries. Quite often I just write a statement that uses something from a library I'm not familiar with and it works. Why is that? Because Python libs are designed to be intuitive, however they can only be if the words, method names, etc are already in programmer's head. As far as nobody really wants goes, I think, professional programmers don't need this, sure, But teachers won't even get to think if they want it or not unless it's there. > - The consensus seems to be that while it is a good idea to translate the > docstrings/Python docs, there's no point in translating keywords or module > names. For one thing, if you ever want to use an outside module, there's > going to have be some way of saying that the chrono module = the time > module. As a programmer, you can't get away with not knowing that when > things go wrong. For another thing, what happens if in english.py I have a > variable named "por" and in spanish.py I have a variable named "for"? How > can these modules work together? I think they can, as long as only one language is used within a given module. Python bytecode doesn't store "for" anywhere after all. Module API would need to be adapted of course. Also debugging would be tough. Yet if Python gains thrice as many users at the expense of some bad code out there, it's worth it. > - The comment that tends to come up in these threads is that "non-native > English speakers are going to have to learn 'a new language' anyway, so > what's the difference between learning to map a word in their own language > to a Python concept to learning to map an arbitrary English word to a Python > concept. In either case, it doesn't make any sense to write, "for the sake > of justice:" in English or "??????" in Japanese. Python's "for" is not the > English "for." It's its own thing. It means "process this iterator in a > loop." That's different from the English "for" and it's going to be > different from any existing word in any other language either. I don't propose to translate python keywords literally, as a matter of fact, for some languages even syntax might change slightly*. Basically some languages might require a different parser and/or "front-end". Actually perhaps the educational part could then be a completely different front-end with much simplified syntax? *) Western world uss 'three fifths" for fractions, while Asia uses "fifths three" in spoken language. Similar differences could pop up with "a and b or c" or "b if a else c" types of expressions. I say it doesn't matter much for core Python as long as same bytecode is generated. > - That said, letting people use unicode variable names is basically a good > idea and has already been implemented in Python 3 and can be turned on in > Python 2 with the right declarations. > Hope that helps, > -- Carl Johnson > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > From greg.ewing at canterbury.ac.nz Wed Dec 1 06:20:46 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 01 Dec 2010 18:20:46 +1300 Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module In-Reply-To: <4CF54D1A.6060106@netwok.org> References: <4CEC234D.6020301@netwok.org> <20101123215458.53eed0b0@pitrou.net> <4CEC4F1F.2090007@canterbury.ac.nz> <4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net> <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net> <4CF54D1A.6060106@netwok.org> Message-ID: <4CF5DB2E.4030504@canterbury.ac.nz> ?ric Araujo wrote: > Not exactly: magic methods are looked up on the class, not on the > instance. That?s why isintance+ABCs is the right test to use here. That seems to be an exceedingly fine straw to split here. I'd be very surprised if someone deliberately gave an instance an attribute called "__len__" and relied on it both not being recognised as a special method and not being picked up by LYBL tests. -- Greg From denis.spir at gmail.com Wed Dec 1 11:28:20 2010 From: denis.spir at gmail.com (spir) Date: Wed, 1 Dec 2010 11:28:20 +0100 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: <20101201112820.4f111929@o> On Tue, 30 Nov 2010 14:49:29 -1000 "Carl M. Johnson" wrote: > - The comment that tends to come up in these threads is that "non-native > English speakers are going to have to learn 'a new language' anyway, so > what's the difference between learning to map a word in their own language > to a Python concept to learning to map an arbitrary English word to a Python > concept. In either case, it doesn't make any sense to write, "for the sake > of justice:" in English or "??????" in Japanese. Python's "for" is not the > English "for." It's its own thing. It means "process this iterator in a > loop." That's different from the English "for" and it's going to be (All of this is just personal opinion.) I have been a proponent of "python, international", now I agree with all what is generally said about the difficulty to create non-english Python-s, naming problems that arise from that, and doubts about whether it's worth it at all. Except for pegagogic-only intents. But, imo, what you state above greatly underestimates the power of words. I think that our brains intensively use all kinds of helps & hints available for thinking -- words, or signs or symbols in general, being its preferred food ;-) I think you use the common sense of "for" for programming, at least you used it as long as its sense in python was not fully integrated. Reason why unproper naming or "symboling" is so harmful, esp for newcomers. Then, we develop new values (in the linguistic sense) for existing words, just like with natural langages when getting used to a new acception of an already known word or idiom. But original senses remain and interact. Wrong symbols still disturb our brains. The more directly a programming word maps to the usual sense, the better. There are 2 syntax errors I constantly do after years & years of python -- and I consider them as revealing its only 2 syntactic defaults: using "=" instead of "==" (because the sense is wrong) and forgetting ":" (because there is no sense at all). Probably my brain is not flexible enough, and/or I'm too old ;-). Well, those are not words, but I think you understand what I mean, anyway. denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From denis.spir at gmail.com Wed Dec 1 11:38:41 2010 From: denis.spir at gmail.com (spir) Date: Wed, 1 Dec 2010 11:38:41 +0100 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: <20101201113841.6685bf41@o> On Tue, 30 Nov 2010 19:47:56 -0700 Dima Tisnek wrote: > Basically some languages might require a different parser and/or "front-end". I think you are right here. What is needed is "Pedon", a pedagogical variant, also natural-language independant. I first thought this was precisely the point of the dedicated community (CP4E, EDU-SIG...). This could be done via a dedicated editor introducing custom lexical and syntactic variant, then transcoded into standard python for storage (and transcoded back according to config at module load time into the editor). Indeed, interaction with other modules can work fine in the general case only when using the same set of customizations, because of keyword (and possible key-sign) conflicts. But for pedagogic use, it seems fine. Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From ncoghlan at gmail.com Wed Dec 1 11:49:19 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 1 Dec 2010 20:49:19 +1000 Subject: [Python-ideas] Truly international Python In-Reply-To: <20101201113841.6685bf41@o> References: <20101201113841.6685bf41@o> Message-ID: On Wed, Dec 1, 2010 at 8:38 PM, spir wrote: > On Tue, 30 Nov 2010 19:47:56 -0700 > Dima Tisnek wrote: > >> Basically some languages might require a different parser and/or "front-end". > > I think you are right here. What is needed is "Pedon", a pedagogical variant, also natural-language independant. I first thought this was precisely the point of the dedicated community (CP4E, EDU-SIG...). > This could be done via a dedicated editor introducing custom lexical and syntactic variant, then transcoded into standard python for storage (and transcoded back according to config at module load time into the editor). Indeed, interaction with other modules can work fine in the general case only when using the same set of customizations, because of keyword (and possible key-sign) conflicts. But for pedagogic use, it seems fine. If you're going to put a different front-end on Python, it makes far more sense to target the ASDL than the concrete syntax. PyPy is also almost certainly a more fruitful platform for this kind of experimentation than CPython :) Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From alexander.belopolsky at gmail.com Wed Dec 1 18:21:00 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 1 Dec 2010 12:21:00 -0500 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: On Tue, Nov 30, 2010 at 12:46 PM, Dima Tisnek wrote: .. > Now the larger part of the world's children doesn't learn English > before school, therefore we need to have truly localized Python. > As Russians say, "All 'new' is old, but well-forgotten" (<>.) In the 80's, when I went to school in the USSR , we studied a programming language called RAPIRA (<>.) You can see what it looked like on Wikipedia. [1] It was a traditional Algol-like language with some features (such as '::' delimiter for the loop and conditional statement bodies) similar to Python. RAPIRA used Russian exclusively. In fact, many popular computers at that time used a 7-bit character set and displays had a physical switch that controlled whether user will see Latin or Cyrillic glyphs for the same 7-bit codes, so you could not effectively mix Russian and English in the same document. Russian is a relatively easy natural language for an adaptation of a computer language designed in English. Many, particularly Latin or Arabic origin words are the same in English and Russian: "algorithm", "procedure", "function", "cycle" all sound very close to their Russian translations. Punctuation and grammar of English and Russian are not that dissimilar, so a word-by-word translation of a simple English sentence may not be perfect, but can usually be understood by a Russian speaker. Still, in its early versions RAPIRA evaluated assignment statements left to right and used '->' operator. (You would write "1 -> x" instead of "x = 1".) That order was considered more natural. Nevertheless, RAPIRA designers had to meet some challenges. For example, many common keywords had Russian translations that were longer than English prototypes: "if" is "????" and "end" is "?????". It appears that "end" was considered too common to use a 5-letter word "?????" for, so it was abbreviated to "???". RAPIRA had limited success and many schools switched to teaching Pascal instead. I note, however that RAPIRA was designed for high-school level students, not for pre-K. What I observed in young students, was that they prefer unfamiliar but shorter words to longer but meaningful ones. When children learn that in turtle they can write "fd" and "lt" for "forward" and "left", they start using these forms exclusively. This phenomenon is not limited to abbreviations. Children who grow up in multi-lingual environment often mix up words from different languages in the same sentence picking shorter words. I suggest that those interested in teaching young children programming in their native language start by localizing the turtle module. It already has hooks for translating docstrings, but I don't think translating function names was considered. I predict that children will still prefer "fd" and "lt" to their local equivalents simply because they are easier to type. The the bigger barrier in my experience was not the meaning of the words, but Python syntax. For many young students, "degrees" is a new word regardless of the native language used to express it. The need to remember that arguments need to be comma separated and placed inside parentheses is often a bigger challenge. In this respect a Tcl- or Logo-like language may be more suitable for this age. See also a simple command interpreter for turtle presented as a cmd module example. [2] [1] http://ru.wikipedia.org/wiki/????:RAPIRA_example.png [2] http://docs.python.org/dev/library/cmd.html#cmd-example From dimaqq at gmail.com Wed Dec 1 23:08:28 2010 From: dimaqq at gmail.com (Dima Tisnek) Date: Wed, 1 Dec 2010 15:08:28 -0700 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: Thanks for the comments, it is indeed a valid observation that shorte syntax is preferred. Perhaps, what is needed for education is not a clone of Python with native syntax, but rather a learner-friendly Python editor that supplants native Python syntax with short explanations and examples in native language. A decent editor already distinguishes between keywords, standard types, etc, and already displays phpdoc/javadoc/etc, similar context-aware hooks for keywords and translated docstrings for standard modules, perhaps aliases for standard functions, auto-correction for local punctuation and perhaps auto-correction between scripts should be enough. It doesn't help with writing systems radically different from latin, but then again Python use of punctuation may be completely incompatible in this case anyway. 2010/12/1 Alexander Belopolsky : > On Tue, Nov 30, 2010 at 12:46 PM, Dima Tisnek wrote: > .. >> Now the larger part of the world's children doesn't learn English >> before school, therefore we need to have truly localized Python. >> > > As Russians say, "All 'new' is old, but well-forgotten" (< ??? ?????? ??????? ??????>>.) ?In the 80's, when I went to school in > the USSR , we studied a programming language called RAPIRA (<>.) > ?You can see what it looked like on Wikipedia. [1] ?It was a > traditional Algol-like language with some features (such as '::' > delimiter for the loop and conditional statement bodies) similar to > Python. RAPIRA used Russian exclusively. ?In fact, many popular > computers at that time used a 7-bit character set and displays had a > physical switch that controlled whether user will see Latin or > Cyrillic glyphs for the same 7-bit codes, so you could not effectively > mix Russian and English in the same document. > > Russian is a relatively easy natural language for an adaptation of a > computer language designed in English. ?Many, particularly Latin or > Arabic origin words are the same in English and Russian: "algorithm", > "procedure", "function", "cycle" all sound very close to their Russian > translations. ? Punctuation and grammar of English and Russian are not > that dissimilar, so a word-by-word translation of a simple English > sentence may not be perfect, but can usually be understood by a > Russian speaker. ?Still, in its early versions RAPIRA evaluated > assignment statements left to right and used '->' operator. ?(You > would write "1 -> x" instead of "x = 1".) ? That order was considered > more natural. > > Nevertheless, ?RAPIRA designers had to meet some challenges. ? ?For > example, many common keywords had Russian translations that were > longer than English prototypes: ?"if" is "????" and "end" is "?????". > ?It appears that "end" was considered too common to use a 5-letter > word "?????" for, so it was abbreviated to "???". > > RAPIRA had limited success and many schools switched to teaching Pascal instead. > > I note, however that RAPIRA was designed for high-school level > students, not for pre-K. ?What I observed in young students, was that > they prefer unfamiliar but shorter words to longer but meaningful > ones. ? When children learn that in turtle they can write "fd" and > "lt" for "forward" and "left", they start using these forms > exclusively. ?This phenomenon is not limited to abbreviations. > Children who grow up in multi-lingual environment often mix up words > from different languages in the same sentence picking shorter words. > > I suggest that those interested in teaching young children programming > in their native language start by localizing the turtle module. ?It > already has hooks for translating docstrings, but I don't think > translating function names was considered. ?I predict that children > will still prefer "fd" and "lt" to their local equivalents simply > because they are easier to type. ? The the bigger barrier in my > experience was not the meaning of the words, but Python syntax. ?For > many young students, "degrees" is a new word regardless of the native > language used to express it. ?The need to remember that arguments need > to be comma separated and placed inside parentheses is often a bigger > challenge. ? In this respect a Tcl- or Logo-like language may be more > suitable for this age. ?See also a simple command interpreter for > turtle presented as a cmd module example. [2] > > > [1] http://ru.wikipedia.org/wiki/????:RAPIRA_example.png > [2] http://docs.python.org/dev/library/cmd.html#cmd-example > From tjreedy at udel.edu Thu Dec 2 06:59:21 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 02 Dec 2010 00:59:21 -0500 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: On 12/1/2010 5:08 PM, Dima Tisnek wrote: > Thanks for the comments, it is indeed a valid observation that shorte > syntax is preferred. > > Perhaps, what is needed for education is not a clone of Python with > native syntax, but rather a learner-friendly Python editor that > supplants native Python syntax with short explanations and examples in > native language. A decent editor already distinguishes between > keywords, standard types, etc, and already displays > phpdoc/javadoc/etc, similar context-aware hooks for keywords and > translated docstrings for standard modules, perhaps aliases for > standard functions, auto-correction for local punctuation and perhaps > auto-correction between scripts should be enough. IDLE has a plugin extension system which I was unaware of until recently. All I know is that some extensions come pre-installed. Perhaps an 'international' extension plus language specific extensions could be developed to do some of the above. In any case, somebody has to volunteer time or money to make anything happen. Terry Jan Reedy From bruce at leapyear.org Thu Dec 2 07:20:24 2010 From: bruce at leapyear.org (Bruce Leban) Date: Wed, 1 Dec 2010 22:20:24 -0800 Subject: [Python-ideas] Truly international Python In-Reply-To: References: Message-ID: On Tue, Nov 30, 2010 at 9:46 AM, Dima Tisnek wrote: > > Now the larger part of the world's children doesn't learn English > before school, therefore we need to have truly localized Python. > > Of course there are concerns for many languages: > Each language needs to establish stable translations for keywords, > basic types, standard modules, methods in standard modules, etc. > Some languages don't support word spaces natively > Some languages have different punctuation rules, e.g. comma for decimal > point > Some languages use different quotes > RTL languages spell words RTL yet (some/all?) spell numbers LTR > Hopefully none has to recreate 10,000-separator system ;-) > This is not a new idea. Algol 68 had keywords that could be in different languages as they were marked in a way that ensured that they didn't conflict with variable names. See http://en.wikipedia.org/wiki/Stropping_(programming). At least Algol limited this to keywords. The idea that we can't share code unless we both have the same version of python and the right set of localized extensions seems like a compatibility nightmare. On Tue, Nov 30, 2010 at 6:47 PM, Dima Tisnek wrote: > > *) Western world uss 'three fifths" for fractions, while Asia uses > "fifths three" in spoken language. > Similar differences could pop up with "a and b or c" or "b if a > else c" types of expressions. > I say it doesn't matter much for core Python as long as same > bytecode is generated. > > 1- I think that translating other syntax (e.g., writing 5 \ 3 instead of 3 / 5) is asking for trouble. The syntax of mathematics doesn't change in other languages. --- Bruce Latest blog post: http://www.vroospeak.com/2010/11/enduring-joe-barton.html Learn about security: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From cool-rr at cool-rr.com Sat Dec 4 12:14:28 2010 From: cool-rr at cool-rr.com (Ram Rachum) Date: Sat, 4 Dec 2010 11:14:28 +0000 (UTC) Subject: [Python-ideas] =?utf-8?q?Adding=60Unpicklable=60_to_the_=60collec?= =?utf-8?q?tions=60_module?= References: <4CEC234D.6020301@netwok.org> <20101123215458.53eed0b0@pitrou.net> <4CEC4F1F.2090007@canterbury.ac.nz> <4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net> <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net> Message-ID: Ram Rachum <4CEC234D.6020301@netwok.org> <20101123215458.53eed0b0@pitrou.net> <4CEC4F1F.2090007@canterbury.ac.nz> <4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net> <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net> Message-ID: <20101204124900.7e6815b7@pitrou.net> On Sat, 4 Dec 2010 11:14:28 +0000 (UTC) Ram Rachum wrote: > Ram Rachum | | So I think it will be best to have both a `pickle.Incompatible` and a > | | `pickle.Compatible`. The reason to have the negative is to let people > | | inherit from it, the reason to have the positive is to make > | | `isinstance` calls more natural. (i.e. avoid a double negative.) > | | > | | What do you think? > | | > | | Ram. > | | > | > | So... are people in favor of this idea? > | > | Ram. > | > > I see a lot of discussion in this thread about use of `isinstance` in general. > What about the `pickle.(In)compatible` proposal? Is it something I should write > a PEP for? You don't need to write a PEP. Just open an issue on http://bugs.python.org and propose a patch. Regards Antoine. From cs at zip.com.au Sat Dec 4 23:00:54 2010 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 5 Dec 2010 09:00:54 +1100 Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for non-type inputs In-Reply-To: <28819.1291083724@parc.com> References: <28819.1291083724@parc.com> Message-ID: <20101204220054.GA16042@cskk.homeip.net> On 29Nov2010 18:22, Bill Janssen wrote: | Nick Coghlan wrote: | > As far as the original post's request goes, no. issubclass expects to | > be given two types. If you have an insanely polymorphic variable that | > can contain a wide variety of objects, only some of which are type | > instances, then either use isinstance to check first as you are | > already doing, or a try-except block to convert the TypeError to a | > False result (and bundle the combined test into a helper function if | > you're doing it a lot). | | Right. Use isinstance instead; that's what it's there for. | | But handling exceptions properly really requires thinking about which | might occur, and what to do if they do occur. If you don't do that | preemptive thinking about exceptionality, you might as well not bother | catching them at all. My problem with try/except (which leads me to be reluctant to use it except around really really simple things) is that you don't know what threw the exception. Consider: if obj.foo == 0: # handle 0 else: x = y / obj.foo The try/except verision goes like this: try: x = y / obj.foo except ZeroDivisionError: # handle 0 Now, the reason I'm using "obj.foo" here is that obj.foo may be a property or otherwise inplemented by __getattr__; arbitrarily complex code may be happening to obtain the value it returns. The upshot of that is that even this very simple looking code may be an unhandled ZeroDivisionError from deeper in the call stack - I don't know it came from the division visible in the code example above. For this reason, I will usually prefer to go the if/else route, and if I go the try/except route I will often have a lot of leading gumph to get everything into purely local variables just to avoid the risk shown above: z = obj.foo # gumph try: x = y / z except ZeroDivisionError: # handle 0 The obfuscation from the leading gumph can often outweigh the brazen pleasures of the try/except leap into the unknown:-) Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Truly, the ultimate demonstration of the computer's utility is that it continues to be indispensable in spite of those who run the things. - Steve Glass, in e-mail From merwok at netwok.org Sat Dec 4 23:05:34 2010 From: merwok at netwok.org (=?UTF-8?B?w4lyaWMgQXJhdWpv?=) Date: Sat, 04 Dec 2010 23:05:34 +0100 Subject: [Python-ideas] Adding`Unpicklable` to the `collections` module In-Reply-To: <5214C323-9A3C-40ED-B018-D4C20F0C44BB@masklinn.net> References: <4CEC234D.6020301@netwok.org> <20101123215458.53eed0b0@pitrou.net> <4CEC4F1F.2090007@canterbury.ac.nz> <4CF1BEE6.7080605@scottdial.com> <20101128101139.77ccf7e7@pitrou.net> <9061EFCB-2801-47F8-AD51-BAAB77FF03CD@masklinn.net> <4CF54D1A.6060106@netwok.org> <5214C323-9A3C-40ED-B018-D4C20F0C44BB@masklinn.net> Message-ID: <4CFABB2E.1050905@netwok.org> Le 30/11/2010 23:08, Masklinn a ?crit : > On 2010-11-30, at 20:14 , ?ric Araujo wrote: >>> For instance, as of Python 2.7 these blocks of code are roughly >>> equivalent (the first two are in fact identical in effect): >>> >>> if isinstance(obj, collections.Sized): >>> doSomethingWith(len(obj)) >>> >>> if hasattr(obj, '__len__'): >>> doSomethingWith(len(obj)) >>> >>> try: >>> doSomethingWith(len(obj)) >>> except TypeError: >>> pass >>> >>> They all test for the same thing: that you can call len() on the object. >> >> Not exactly: magic methods are looked up on the class, not on the >> instance. > So? So it?s possible that an object has a __len__ attribute but that calling len on it raises a TypeError. See also Nick?s reply. > May I know why you're replying that to *my* mail? Because I quoted a bit of your message to reply to it. (I?m not sure I understand the question.) Regards From steve at pearwood.info Sun Dec 5 02:16:37 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 05 Dec 2010 12:16:37 +1100 Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for non-type inputs In-Reply-To: <20101204220054.GA16042@cskk.homeip.net> References: <28819.1291083724@parc.com> <20101204220054.GA16042@cskk.homeip.net> Message-ID: <4CFAE7F5.4070801@pearwood.info> Cameron Simpson wrote: > The try/except verision goes like this: > > try: > x = y / obj.foo > except ZeroDivisionError: > # handle 0 > > Now, the reason I'm using "obj.foo" here is that obj.foo may be a property > or otherwise inplemented by __getattr__; arbitrarily complex code may > be happening to obtain the value it returns. The upshot of that is that > even this very simple looking code may be an unhandled ZeroDivisionError > from deeper in the call stack - I don't know it came from the division > visible in the code example above. That's true. But if obj.foo raises ZeroDivisionError, perhaps you should be treating this as equivalent to x/0 and continue, particularly if you have no control over the obj.foo. What else are you going to do (assuming that letting the exception propagate is not an option)? In any case, if you want to guard against such bugs, you can inspect the traceback and decide what to do. This is particularly easy in Python3.x, but possible in 2.x as well: >>> class K: ... def __getattr__(self, name): ... raise ZeroDivisionError ... >>> >>> try: ... 1/K().spam ... except ZeroDivisionError as e: ... if e.__traceback__.tb_next is None: ... print("handle division by zero") ... else: ... print("handle bug in __getattr__") ... handle bug in __getattr__ And it's not like the if test catches all possible errors. There are many potential error conditions that aren't caught by something like: if obj.foo != 0: print(x/obj.foo) else: # handle 0 You still have no guarantee that the division will succeed. Perhaps one or other of x and obj.foo define __truediv__ or __rtruediv__ in such a way that it raises ZeroDivisionError even when obj.foo is not zero. That's not necessarily a bug -- one or the other could be an interval quantity that straddles zero, but isn't equal to zero. If you fear that obj.foo could contain arbitrarily complex code that may accidentally raise ZeroDivisionError (or anything else!), the same holds for __ne__. Once you stop trusting your values, you can't trust *anything* -- maybe object.foo has the __eq__ and __ne__ reversed. Who knows? How defensively do you code? Here's an error condition from the Bad Old Days before IEEE floats: perhaps division and equality are done to different precisions, such that y != 0 but x/y still attempts division by zero. (This really used to happen, on some mainframes.) Obviously this can't happen with Python floats -- or can it? are IEEE semantics guaranteed? -- but it could happen with some custom numeric type. You're right that the try...except idiom is subject to false positives. It might, under some (presumably rare) circumstances, catch an exception which should be treated as a bug. But the if...else idiom is subject to both false positives and false negatives: your test may be too strict, or it may be not strict enough. Or both at the same time. Or the thing you are testing may be subject to race conditions: if os.exists(pathname): # This is NOT safe! fp = open(pathname) else: print("no such file") vs. try: fp = open(pathname) except IOError: print("no such file") It seems to me that the LBYL idiom is actually *less* safe than the exception handling idiom. -- Steven From cs at zip.com.au Sun Dec 5 03:14:05 2010 From: cs at zip.com.au (Cameron Simpson) Date: Sun, 5 Dec 2010 13:14:05 +1100 Subject: [Python-ideas] `issubclass` shouldn't be raising exceptions for non-type inputs In-Reply-To: <4CFAE7F5.4070801@pearwood.info> References: <4CFAE7F5.4070801@pearwood.info> Message-ID: <20101205021405.GA13140@cskk.homeip.net> On 05Dec2010 12:16, Steven D'Aprano wrote: | Cameron Simpson wrote: | >The try/except verision goes like this: | > | > try: | > x = y / obj.foo | > except ZeroDivisionError: | > # handle 0 | > | >Now, the reason I'm using "obj.foo" here is that obj.foo may be a property | >or otherwise inplemented by __getattr__; arbitrarily complex code may | >be happening to obtain the value it returns. The upshot of that is that | >even this very simple looking code may be an unhandled ZeroDivisionError | >from deeper in the call stack - I don't know it came from the division | >visible in the code example above. | | That's true. But if obj.foo raises ZeroDivisionError, perhaps you | should be treating this as equivalent to x/0 and continue, | particularly if you have no control over the obj.foo. What else are | you going to do (assuming that letting the exception propagate is | not an option)? Well, that's the point: if the ZeroDivision came from my divide presumably I have a strategy to handle that because it is expected. But if it came from inside the implementation of obj.foo then more likely I should let the exception propagate, because it is _not_ the cricumstance for which my try/except was accomodating. | In any case, if you want to guard against such bugs, you can inspect | the traceback and decide what to do. This is particularly easy in | Python3.x, but possible in 2.x as well: | | >>> class K: | ... def __getattr__(self, name): | ... raise ZeroDivisionError | ... | >>> | >>> try: | ... 1/K().spam | ... except ZeroDivisionError as e: | ... if e.__traceback__.tb_next is None: | ... print("handle division by zero") | ... else: | ... print("handle bug in __getattr__") | ... | handle bug in __getattr__ Interesting. But this is easier to read and maintain than the if/else form how? | And it's not like the if test catches all possible errors. There are | many potential error conditions that aren't caught by something | like: | | if obj.foo != 0: | print(x/obj.foo) | else: | # handle 0 Of course there are; it's an example to illustrate the ambiguity of the received exception versus the concreteness of a failed if-test. [...] | If you fear that obj.foo could contain arbitrarily complex code that | may accidentally raise ZeroDivisionError (or anything else!), the | same holds for __ne__. Once you stop trusting your values, you can't | trust *anything* -- maybe object.foo has the __eq__ and __ne__ | reversed. Who knows? Sure, but this is the realm of the _unhandled_ circumstance. With the try/except I will easily try to handle that circumstance in the _mistaken_ belief that it is a specific failure I was ready for. | How defensively do you code? More defensively than many, in my experience. In particular, I generally want to be sure that an "exceptional" circumstance I'm handling really is the anticipated circumstance, and not something totally else. With exceptions you don't know that without cumbersome inspection of the thrown exception. The point of the example is that one can write code for an anticipated circumstance, but that code should _not_ run for the unanticipated circumstance. This is easy with the if/else because the test is specific. [...] | You're right that the try...except idiom is subject to false | positives. It might, under some (presumably rare) circumstances, | catch an exception which should be treated as a bug. But the | if...else idiom is subject to both false positives and false | negatives: your test may be too strict, or it may be not strict | enough. My test may be accurate or not, sure. But inaccuracy is a coding bug on my part, not an exceptional circumstance that might legitimately happen at any runtime. | Or both at the same time. Or the thing you are testing may | be subject to race conditions: | | if os.exists(pathname): | # This is NOT safe! | fp = open(pathname) | else: | print("no such file") | | vs. | | try: | fp = open(pathname) | except IOError: | print("no such file") | | It seems to me that the LBYL idiom is actually *less* safe than the | exception handling idiom. Not as written. I take your point about the race and in fact _do_ go for try/except there because I'm after "did the open succeed?", something that can't be tested in advance without time travel. But I would be writing it like this: try: fp = open(pathname) except: # disaster! and feeling awful about the bare except. But the fact of th matter is that an open may fail for many reasons, not just I/O. And in fact your example is a classic case of my concern with try/except - you're reporting "no such file" when in fact there may be other reasons eg a permission failure. You're doing just what I wish to avoid: _misdiagnosing_ an exception and acting on that diagnosis. It sounds like we're both aware of the pros and cons but differ in what risks to accept. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ "What do you want to reinstall today?" - Bob O`Bob From zac256 at gmail.com Sun Dec 5 03:31:38 2010 From: zac256 at gmail.com (Zac Burns) Date: Sun, 5 Dec 2010 10:31:38 +0800 Subject: [Python-ideas] Extend docstring construction Message-ID: Greetings, I would like to see python's doc-string creation extended to allow for % style string substitution. Rationale: This allows for patterns that keep doc-strings up-to-date without certain types of boilerplate maintenance. For instance: Including additional information from the module: class Colors(Enum): RED = (255, 0, 0) def acceptsColor(color): """ color could be any: %s """ % '\n\t'.join('Color.' + color for color in publicDir(Colors)) Including docs of other functions def _localFunction(): "Keeping the docstring with the function that does the work, so it's more likely to keep up to date" def publicFunction(): "%s" % _localFunction.__doc__ Drawbacks: Executing code to retrieve a doc-string is considered a security vulnerability. (Right now only the abstract syntax tree is used to get docstrings) In theory this could be backwards incompatible with existing code... but that code would have to be relying on the side effect of some string substitution line without doing anything with the new string - yuck. Thoughts? -- Zachary Burns (407)590-4814 Aim - Zac256FL -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg at krypto.org Sun Dec 5 05:37:46 2010 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 4 Dec 2010 20:37:46 -0800 Subject: [Python-ideas] 3to3 tool for 3.x to 3.x+n version migrations Message-ID: Now that we've developed 2to3 and 3to2 I've started wondering "what next?" and realized that we could use a 3to3 tool to apply the relatively few and relatively simple sets of code changes to deal with library reorganizations, renaming, parameter changes, default behavior changes, other deprecations that have an equivalent, etc. The list of migrations to go between any given 3.x to another 3.x release would likely be pretty small. Just tossing the idea out there incase someone is interested. I expect the hard work is already done. thoughts? -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Sun Dec 5 06:45:11 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 05 Dec 2010 00:45:11 -0500 Subject: [Python-ideas] Extend docstring construction In-Reply-To: References: Message-ID: On 12/4/2010 9:31 PM, Zac Burns wrote: > Greetings, > > I would like to see python's doc-string creation extended to allow for % > style string substitution. > > Rationale: > This allows for patterns that keep doc-strings up-to-date without > certain types of boilerplate maintenance. For instance: I am sympathetic to the purpose, having recently run into such a problem. But... > > Including additional information from the module: > > class Colors(Enum): > RED = (255, 0, 0) > > def acceptsColor(color): > """ > color could be any: > %s > """ % '\n\t'.join('Color.' + color for color in publicDir(Colors)) > > Including docs of other functions > def _localFunction(): > "Keeping the docstring with the function that does the work, so it's > more likely to keep up to date" > > def publicFunction(): > "%s" % _localFunction.__doc__ > > > Drawbacks: 1, The need to reliably recognize that an expression is meant to define a docstring rather than that be part of the runtime code. A string expression is senseless as runtime code. However, other expression statements are executed for their side-effects, and most any expression have have such. 2. You are proposing to use the semi-deprecated %-as-format operator. > Executing code to retrieve a doc-string is considered a security > vulnerability. The string would have to be computed along with the object it gets attached to. -- Terry Jan Reedy From tjreedy at udel.edu Sun Dec 5 06:51:40 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 05 Dec 2010 00:51:40 -0500 Subject: [Python-ideas] 3to3 tool for 3.x to 3.x+n version migrations In-Reply-To: References: Message-ID: On 12/4/2010 11:37 PM, Gregory P. Smith wrote: > Now that we've developed 2to3 and 3to2 I've started wondering "what > next?" and realized that we could use a 3to3 tool to apply the > relatively few and relatively simple sets of code changes to deal with > library reorganizations, renaming, parameter changes, default behavior > changes, other deprecations that have an equivalent, etc. The list of > migrations to go between any given 3.x to another 3.x release would > likely be pretty small. I had a vague thought like this recently. I am glad you formulated it as a coherent thought and posted it. Such a tool should also make note of deprecated construct that it recognizes but cannot fix. One problem with runtime DeprecationWarnings is that test suites do not always have 100% coverage, and code not executed will not issue a warning. -- Terry Jan Reedy From ncoghlan at gmail.com Sun Dec 5 10:18:34 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 5 Dec 2010 19:18:34 +1000 Subject: [Python-ideas] Extend docstring construction In-Reply-To: References: Message-ID: On Sun, Dec 5, 2010 at 12:31 PM, Zac Burns wrote: > Including additional information from the module: > > class Colors(Enum): > RED = (255, 0, 0) > > def acceptsColor(color): > """ > color could be any: > %s > """ % '\n\t'.join('Color.' + color for color in publicDir(Colors)) Just roll your own decorator if you want to do something like this: def format_docstring(*args, **kwds): def docstringFormatter(f): fmt = f.__doc__ f.__doc__ = fmt.format(*args, **kwds) return f return docstringFormatter @format_docstring("Any text you like") def sample(): """Include a supplied string in the docstring: {} """ help(sample) ============================= Help on function sample in module __main__: sample() Include a supplied string in the docstring: Any text you like ============================= Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From cmjohnson.mailinglist at gmail.com Sun Dec 5 10:46:34 2010 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 4 Dec 2010 23:46:34 -1000 Subject: [Python-ideas] 3to3 tool for 3.x to 3.x+n version migrations In-Reply-To: References: Message-ID: <02987033BECD47B185833324B37AA4D7@gmail.com> When we were discussing yield from, Guido mentioned that it would be nice to have some way to test out things like that before committing all of Python to it. One way to do that might be if someone adapted it into a "4-to-3" to let you rewrite yield from, for example, in its full Python 3.2 expansion form. In other words, the n-to-n engine could be used as a general purpose macro system. Perhaps it could even useful for the internationalization project mentioned recently? The advantage of having the macro engine separate from Python proper would be that this way these extensions never get popular, and so 99% of Pythonistas never have to worry that maybe someone redefined "for item in items:" in their module using C-style #macros. It would exist solely as a way of testing out the various proposals that are made on Python-ideas and elsewhere before having to commit to them or of translating Python programs into a native language for younger, non-English native programmers. -- Carl -------------- next part -------------- An HTML attachment was scrubbed... URL: From zac256 at gmail.com Mon Dec 6 02:59:50 2010 From: zac256 at gmail.com (Zac Burns) Date: Mon, 6 Dec 2010 09:59:50 +0800 Subject: [Python-ideas] 3to3 tool for 3.x to 3.x+n version migrations In-Reply-To: <02987033BECD47B185833324B37AA4D7@gmail.com> References: <02987033BECD47B185833324B37AA4D7@gmail.com> Message-ID: On Sun, Dec 5, 2010 at 5:46 PM, Carl Johnson < cmjohnson.mailinglist at gmail.com> wrote: > When we were discussing yield from, Guido mentioned that it would be nice > to have some way to test out things like that before committing all of > Python to it. One way to do that might be if someone adapted it into a > "4-to-3" to let you rewrite yield from, for example, in its full Python 3.2 > expansion form. In other words, the n-to-n engine could be used as a general > purpose macro system. Perhaps it could even useful for the > internationalization project mentioned recently? > > The advantage of having the macro engine separate from Python proper would > be that this way these extensions never get popular, and so 99% of > Pythonistas never have to worry that maybe someone redefined "for item in > items:" in their module using C-style #macros. It would exist solely as a > way of testing out the various proposals that are made on Python-ideas and > elsewhere before having to commit to them or of translating Python programs > into a native language for younger, non-English native programmers. > > -- Carl > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > I am in favor of keeping this in some form as well (but would like to see some changes made, it's design reflects a sort of one-offness). I have used it in the past for broad codebase manipulations at a sort of package compile time. Using it for yield from seems brilliant as well. -- Zachary Burns (407)590-4814 Aim - Zac256FL -------------- next part -------------- An HTML attachment was scrubbed... URL: From cool-rr at cool-rr.com Mon Dec 6 19:25:58 2010 From: cool-rr at cool-rr.com (Ram Rachum) Date: Mon, 6 Dec 2010 18:25:58 +0000 (UTC) Subject: [Python-ideas] Detecting circular imports ? References: <20101118162236.680ddca7@pitrou.net> <4CE5B1AF.6080502@canterbury.ac.nz> <4CE5B338.1010607@egenix.com> Message-ID: Tarek Ziad? writes: > > On Fri, Nov 19, 2010 at 12:21 AM, Michael Foord > wrote: > > > > If you use the "from module import something" form it will be an > > ImportError. If you do "import module" and then "module.something" you will > > see an AttributeError. These are the ones that can be confusing. > > Yes that's the case I was thinking of. After years of Python I can > still get trapped on those. But maybe that's just me :) It's not just you, I get that too and it's really annoying. I have a one-module- per-class approach so it happens a lot to me. I would love to have more informative error messages when this happens. Ram. From rrr at ronadam.com Sat Dec 11 09:06:53 2010 From: rrr at ronadam.com (Ron Adam) Date: Sat, 11 Dec 2010 02:06:53 -0600 Subject: [Python-ideas] Linking generators In-Reply-To: <4CC858F3.4000602@improva.dk> References: <4CC63065.9040507@improva.dk> <4CC858F3.4000602@improva.dk> Message-ID: <4D03311D.9060103@ronadam.com> On 10/27/2010 11:53 AM, Jacob Holm wrote: > On 2010-10-26 18:56, Guido van Rossum wrote: >> Now, if I may temporarily go into wild-and-crazy mode (this *is* >> python-ideas after all :-), we could invent some ad-hoc syntax for >> this pattern, e.g.: > Hmm. This got me thinking. One thing I'd really like to see in python > is something like the "channel" object from the go language > (http://golang.org/). > > Based on PEP 380 or Gregs new cofunctions PEP (or perhaps even without > any of them) it is possible to write a trampoline-based implementation > of a channel object with "send" and "next" methods that work as > expected. One thing that is *not* possible (I think) is to make that > object iterable. Your wild idea above gave me a similar wild idea of my > own. An extension to the cofunctions PEP that would make that possible. > [clipped] > All this to make it possible to write a code like this: > > def consumer(ch): > for val in ch: > cocall print(val) # XXX need a cocall somewhere > > def producer(ch): > for val in range(10): > cocall ch.send(val) > > def main() > sched = scheduler() > ch = channel() > sched.add(consumer(ch)) > sched.add(producer(ch)) > sched.run() > > > Thoughts? The following isn't quite the same as channels, but you might be able to use the technique below to get something like it. These examples show how linking generators using nonlocal can simplify some problems. The nice thing about these is the inner loops are very simple and there is no try-except blocks, exceptions, or if statements involved. ;-) Ron def averager(): def collector(): nonlocal tally, count while 1: tally += yield count += 1 def emitter(): nonlocal tally, count while 1: yield tally / count count = tally = 0 coll = collector() next(coll) return coll, emitter() coll, emit = averager() for x in range(100): coll.send(x) print('average: %s' % next(emit)) def parallel_reduce(iterable, funcs): def reduce_collector(func): def collector(func): nonlocal outcome outcome = yield while 1: outcome = func(outcome, (yield)) def emitter(): nonlocal outcome while 1: yield outcome outcome = None coll = collector(func) next(coll) return coll, emitter() collectors = [reduce_collector(func) for func in funcs] for val in iterable: for coll, _ in collectors: coll.send(val) return [next(emit) for _, emit in collectors] print(parallel_reduce(range(5, 10), [min, max])) From greg at krypto.org Mon Dec 13 23:35:34 2010 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 13 Dec 2010 14:35:34 -0800 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] Message-ID: The number of times I've seen code written that reads well and passes less trained code reviews easily such as: if check_num and mystring and mystring.isdigit: ... has increased beyond what I can count on one hand. The code executes and even does the right thing part of the time. Proposal: any method that returns a boolean on builtins like bytes and unicode objects be turned into a property in a future python version. That way the above mistake would happen less often. mystring.isdigit would be a boolean rather than a method reference that is always True. There are many ways of detecting the above beyond code review such as making a pylink or pychecker type thing flag them, having proper unittest coverage that tests both conditions, etc. But those are not always in place. A hack could be done today to have these behave as both functions and properties by having the property return a callable bool derived class that returns itself. But that seems a bit too cool yet gross... -gps -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Dec 14 00:57:03 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 14 Dec 2010 10:57:03 +1100 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: References: Message-ID: <4D06B2CF.4050500@pearwood.info> Gregory P. Smith wrote: > The number of times I've seen code written that reads well and passes less > trained code reviews easily such as: > > if check_num and mystring and mystring.isdigit: > ... > > has increased beyond what I can count on one hand. The code executes and > even does the right thing part of the time. You can simplify that code by writing: if check_num and mystring.isdigit(): ... since ''.isdigit() correctly returns False. Being simpler, there's less to read and it is less likely for the reader to miss any errors. > Proposal: any method that returns a boolean on builtins like bytes and > unicode objects be turned into a property in a future python version. Why limit this to booleans? Similar errors could apply to any function or method that takes no explicit arguments. > That > way the above mistake would happen less often. Making it more difficult and error prone and slower to do things like: if all(str.isdigit, list_of_strings): ... or similar. -0 on the idea. -- Steven From cmjohnson.mailinglist at gmail.com Tue Dec 14 01:08:42 2010 From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson) Date: Mon, 13 Dec 2010 14:08:42 -1000 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: <4D06B2CF.4050500@pearwood.info> References: <4D06B2CF.4050500@pearwood.info> Message-ID: On Mon, Dec 13, 2010 at 1:57 PM, Steven D'Aprano wrote: > Making it more difficult and error prone and slower to do things like: > > if all(str.isdigit, list_of_strings): > ? ?... > > or similar. Technical note: all only takes an iterable. Its signature is not like map's. The correct way to write that is if all(s.isdigit() for s in list_of_strings) (minus the calling parentheses if this proposal were accepted). From ncoghlan at gmail.com Tue Dec 14 01:32:52 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 14 Dec 2010 10:32:52 +1000 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: References: <4D06B2CF.4050500@pearwood.info> Message-ID: On Tue, Dec 14, 2010 at 10:08 AM, Carl M. Johnson < cmjohnson.mailinglist at gmail.com> wrote: > On Mon, Dec 13, 2010 at 1:57 PM, Steven D'Aprano > wrote: > > > Making it more difficult and error prone and slower to do things like: > > > > if all(str.isdigit, list_of_strings): > > ... > > > > or similar. > > Technical note: all only takes an iterable. Its signature is not like > map's. The correct way to write that is if all(s.isdigit() for s in > list_of_strings) (minus the calling parentheses if this proposal were > accepted). > It's a typo. Given the point he was making, Steven presumably meant the call to be "all(map(str.isdigit, list_of_strings))". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Dec 14 12:16:27 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 14 Dec 2010 22:16:27 +1100 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: References: <4D06B2CF.4050500@pearwood.info> Message-ID: <4D07520B.2010001@pearwood.info> Carl M. Johnson wrote: > On Mon, Dec 13, 2010 at 1:57 PM, Steven D'Aprano wrote: > >> Making it more difficult and error prone and slower to do things like: >> >> if all(str.isdigit, list_of_strings): >> ... >> >> or similar. > > Technical note: all only takes an iterable. Its signature is not like > map's. The correct way to write that is if all(s.isdigit() for s in > list_of_strings) (minus the calling parentheses if this proposal were > accepted). Oops! Of course you're right. This is what happens when you test the concept with filter, then change it at the last second to something that would be more commonly used than filter :( -- Steven From masklinn at masklinn.net Tue Dec 14 12:31:56 2010 From: masklinn at masklinn.net (Masklinn) Date: Tue, 14 Dec 2010 12:31:56 +0100 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: <4D06B2CF.4050500@pearwood.info> References: <4D06B2CF.4050500@pearwood.info> Message-ID: On 2010-12-14, at 00:57 , Steven D'Aprano wrote: > > That >> way the above mistake would happen less often. > > Making it more difficult and error prone and slower to do things like: > > if all(str.isdigit, list_of_strings): > ... > > or similar. Though I am ?0 on the idea as well, this might be solveable by making properties callable and equivalent to unbound method aliases to their fget (e.g. __call__(self, instance) would forward to self.__get__). In that case, `filter(str.isdigit, list_of_strings)` would work correctly I believe. From fuzzyman at voidspace.org.uk Tue Dec 14 14:19:24 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 14 Dec 2010 13:19:24 +0000 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: References: Message-ID: On 13 December 2010 22:35, Gregory P. Smith wrote: > The number of times I've seen code written that reads well and passes less > trained code reviews easily such as: > > if check_num and mystring and mystring.isdigit: > ... > > has increased beyond what I can count on one hand. The code executes and > even does the right thing part of the time. > > Proposal: any method that returns a boolean on builtins like bytes and > unicode objects be turned into a property in a future python version. That > way the above mistake would happen less often. mystring.isdigit would be a > boolean rather than a method reference that is always True. > > There are many ways of detecting the above beyond code review such as > making a pylink or pychecker type thing flag them, having proper unittest > coverage that tests both conditions, etc. But those are not always in > place. > > A hack could be done today to have these behave as both functions and > properties by having the property return a callable bool derived class that > returns itself. But that seems a bit too cool yet gross... > > bool derived class... good luck with that. :-) Michael > -gps > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > > -- http://www.voidspace.org.uk -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Tue Dec 14 19:42:29 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Tue, 14 Dec 2010 13:42:29 -0500 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 5:35 PM, Gregory P. Smith wrote: > The number of times I've seen code written that reads well and passes less > trained code reviews easily such as: > if check_num and mystring and mystring.isdigit: > ?? ... > has increased beyond what I can count on one hand. ?The code executes and > even does the right thing part of the time. Missing parens is only one of the issues with the above code. If you rely on the above check to make sure that mystring will be accepted by str(), you should use isdecimal instead: >>> mystring = '\xB2\xB3' >>> mystring.isdigit() and int(mystring) Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '??' >>> mystring.isdecimal() and int(mystring) False > Proposal: any method that returns a boolean on builtins like bytes and unicode objects be turned into a property -1 These methods invoke fairly non-trivial calculations that are not appropriate for properties. However, the situation can be improved if methods and functions would stop supporting implicit conversion to bool. In my experience, checking the bool value of a method or function is either an outright mistake or can be appropriately replaced with an is not None check. This would have to wait for py4k, though. In py3k we have enough trouble with None becoming unorderable to introduce yet another breakage. From zuo at chopin.edu.pl Wed Dec 15 11:04:45 2010 From: zuo at chopin.edu.pl (Jan Kaliszewski) Date: Wed, 15 Dec 2010 11:04:45 +0100 Subject: [Python-ideas] replace boolean methods on builtin types with properties [py4k?] In-Reply-To: References: <4D06B2CF.4050500@pearwood.info> Message-ID: <20101215100445.GA3244@chopin.edu.pl> 1. Masklinn dixit (2010-12-14, 12:31): > Though I am ?0 on the idea as well, this might be solveable by making > properties callable and equivalent to unbound method aliases to their > fget There are no unbound methods in Py3k, but pure function instead. That would make the issue even easier. There could be @callableproperty decorator for such cases. I'd +0 on such an idea. 2. Maybe better idea (if we want to avoid missed-parenthesis-mistakes) would be to raise an error when trying to test such functions/methods for truth. I could be achieved with a subclass of function type with __bool__() method raising TypeError or NotImplementedError. Cheers, *j From m_daoust at hotmail.com Fri Dec 17 01:12:39 2010 From: m_daoust at hotmail.com (M Daoust) Date: Thu, 16 Dec 2010 19:12:39 -0500 Subject: [Python-ideas] Python-ideas Digest, Vol 49, Issue 10 In-Reply-To: References: Message-ID: > Date: Tue, 14 Dec 2010 13:19:24 +0000 > From: Michael Foord > To: "Gregory P. Smith" > Cc: Python-Ideas > Subject: Re: [Python-ideas] replace boolean methods on builtin types > with properties [py4k?] > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > > On 13 December 2010 22:35, Gregory P. Smith wrote: > > > > A hack could be done today to have these behave as both functions and > > properties by having the property return a callable bool derived class that > > returns itself. But that seems a bit too cool yet gross... > > > > > bool derived class... good luck with that. :-) > > Michael Well it it doesn't sound that odd to me. This it's one of the (few) things I miss from Matlab. Not that parentheses are optional for function calls, but that 'true', 'false', 'Inf' and 'NaN' act both as their respective values, and as functions. Like 'ones' and 'zeros' they can be called with a size tuple to get an array of nd-array of that value. -Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Fri Dec 17 02:31:57 2010 From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson) Date: Thu, 16 Dec 2010 15:31:57 -1000 Subject: [Python-ideas] Python-ideas Digest, Vol 49, Issue 10 In-Reply-To: References: Message-ID: On Thu, Dec 16, 2010 at 2:12 PM, M Daoust wrote: >> > A hack could be done today to have these behave as both functions and >> > properties by having the property return a callable bool derived class >> > that >> > returns itself. But that seems a bit too cool yet gross... That could break existing code. Suppose someone has something like f = random.choice([s.isalpha, None, s.isdigit]); truthiness = f() if f else f. OK, the random choice part is dumb, but I can see someone doing a test where they assume that methods like .isdigit will always return True and using that as a replacement for callable(). > Well it it doesn't sound that odd to me. > This it's one of the (few) things I miss from Matlab. > Not that parentheses are optional for function calls, but that 'true', > 'false', 'Inf' and 'NaN' act both > as their respective values,? and as functions. Like 'ones' and 'zeros' they > can be called with a size > tuple to get an array of nd-array of that value. What's the advantage over [True] * n? Seems like a limited usecase, especially since the people who are really serious about arrays (vs. lists) are going to be using one of the numpy array classes. -- Carl From eike.welk at gmx.net Sat Dec 18 00:21:35 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sat, 18 Dec 2010 00:21:35 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python Message-ID: <201012180021.37232.eike.welk@gmx.net> After learning a bit of Ocaml I started to like its pattern matching features. Since then I want to have a "match" statement in Python. I wonder if anybody else would like this too. ML style pattern matching is syntactic sugar, that combines "if" statements with tuple unpacking, access to object attributes, and assignments. It is a compact, yet very readable syntax for algorithms, that would otherwise require nested "if" statements. It is especially useful for writing interpreters, and processing complex trees. Instead of a specification in BNF, here is a function written with the proposed pattern matching syntax. It demonstrates the features that I find most important. The comments and the print statements explain what is done. Proposed Syntax --------------- def foo(x): match x with | 1 -> # Equality print("x is equal to 1") | a:int -> # Type check print("x has type int: %s" % a) | (a, b) -> # Tuple unpacking print("x is a tuple with length 2: (%s, %s)" % (a, b)) | {| a, b |} -> # Attribute existence and access print("x is an object with attributes 'a' and 'b'.") print("a=%s, b=%s" % (a, b)) # Additional condition | (a, b, c) with a > b -> print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) print("The first element is greater than the second element.") # Complex case | {| c:int, d=1 |}:Foo -> print("x has type Foo") print("x is an object with attributes 'c' and 'd'.") print("'c' has type 'int', 'd' is equal to 1.") print("c=%s, d=%s" % (c, d)) # Default case | _ -> print("x can be anything") Equivalent Current Python ------------------------- The first four cases could be handled more simply, but handling all cases in the same way leads IMHO to more simple code overall. def foo(x): while True: # Equality if x == 1: print("x is equal to 1") break # Type check if isinstance(x, int): a = x print("x is an integer: %s" % a) break # Tuple unpacking if isinstance(x, tuple) and len(x) == 2: a, b = x print("x is a tuple with length 2: (%s, %s)" % (a, b)) break # Attribute existence testing and access if hasattr(x, "a") and hasattr(x, "b"): a, b = x.a, x.b print("x is an object with attributes 'a' and 'b'.") print("a=%s, b=%s" % (a, b)) break # Additional condition if isinstance(x, tuple) and len(x) == 3: a, b, c = x if a > b : print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) print("The first element is greater than the second " "element.") break # Complex case if isinstance(x, Foo) and hasattr(x, "c") and hasattr(x, "d"): c, d = x.c, x.d if isinstance(c, int) and d == 1: print("x has type Foo") print("x is an object with attributes 'c' and 'd'.") print("'c' has type 'int', 'd' is equal to 1.") print("c=%s, d=%s" % (c, d)) break # Default case print("x can be anything") break Additional Code to Run Function "foo" ------------------------------------- class Bar(object): def __init__(self, a, b): self.a = a self.b = b class Foo(object): def __init__(self, c, d): self.c = c self.d = d foo(1) # Equality foo(2) # Type check foo((1, 2)) # Tuple unpacking foo(Bar(1, 2)) # Attribute existence testing and access foo((2, 1, 3)) # Additional condition foo(Foo(2, 1)) # Complex case foo("hello") # Default case I left out dict and set, because I'm not sure how they should be handled. I think list should be handled like tuples. Probably there should be a universal matching syntax for all sequences, similarly to the already existing syntax: a, b, *c = s I don't really like the "->" digraph at the end of each match case. A colon would be much more consistent, but I use colons already for type checking (a:int). I generally think that Python should acquire more features from functional languages. In analogy to "RPython" it should ultimately lead to "MLPython", a subset of the Python language that can be type checked and reasoned about by external tools, similarly to what is possible with Ocaml. Eike. From ethan at stoneleaf.us Sat Dec 18 01:13:07 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 17 Dec 2010 16:13:07 -0800 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012180021.37232.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> Message-ID: <4D0BFC93.5000503@stoneleaf.us> Eike Welk wrote: > Instead of a specification in BNF, here is a function written with the > proposed pattern matching syntax. My BNF is weak -- thanks for the code! > Proposed Syntax > --------------- > > def foo(x): > match x with > | 1 -> # Equality > print("x is equal to 1") > | a:int -> # Type check > print("x has type int: %s" % a) > | (a, b) -> # Tuple unpacking > print("x is a tuple with length 2: (%s, %s)" % (a, b)) > | {| a, b |} -> # Attribute existence and access > print("x is an object with attributes 'a' and 'b'.") > print("a=%s, b=%s" % (a, b)) > > # Additional condition > | (a, b, c) with a > b -> > print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) > print("The first element is greater than the second element.") > > # Complex case > | {| c:int, d=1 |}:Foo -> > print("x has type Foo") > print("x is an object with attributes 'c' and 'd'.") > print("'c' has type 'int', 'd' is equal to 1.") > print("c=%s, d=%s" % (c, d)) > > # Default case > | _ -> > print("x can be anything") I am unfamiliar with OCAML -- if x can match more than one condition, will it match all possible, or just the first one? If just the first one, the python code below can be simplified by ditching the while loop, removing the breaks, and using elif and else. def foo(x): # Equality if x == 1: print("x is equal to 1") # Type check elif isinstance(x, int): a = x print("x is an integer: %s" % a) # Tuple unpacking elif isinstance(x, tuple) and len(x) == 2: a, b = x print("x is a tuple with length 2: (%s, %s)" % (a, b)) # Attribute existence testing and access elif hasattr(x, "a") and hasattr(x, "b"): a, b = x.a, x.b print("x is an object with attributes 'a' and 'b'.") print("a=%s, b=%s" % (a, b)) # Additional condition elif isinstance(x, tuple) and len(x) == 3: a, b, c = x if a > b : print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) print("The first element is greater than the second " "element.") # Complex case elif isinstance(x, Foo) and hasattr(x, "c") and hasattr(x, "d"): c, d = x.c, x.d if isinstance(c, int) and d == 1: print("x has type Foo") print("x is an object with attributes 'c' and 'd'.") print("'c' has type 'int', 'd' is equal to 1.") print("c=%s, d=%s" % (c, d)) # Default case else: print("x can be anything") One of the things I like about Python is it's readability. While the OCAML inspired version is much more concise, I don't see it as significantly easier to read -- and at this point I don't see any place where I would make use of it myself. Here's a bit of code that might be convertable: result = [] for i, piece in enumerate(pieces): if '-' in piece: piece = piece.replace('-',' ') piece = '-'.join(NameCase(piece).split()) elif alpha_num(piece) in ('i', 'ii', 'iii', 'iv', 'v', 'vi', \ 'vii', 'viii', 'ix', 'x', 'pc', 'llc') \ or piece.upper() in job_titles \ or i and piece.upper() in deg_suffixi: piece = piece.upper() elif piece in ('and', 'de', 'del', 'der', 'el', 'la', 'van', ): pass elif piece[:2] == 'mc': piece = 'Mc' + piece[2:].title() else: possible = mixed_case_names.get(piece, None) if possible is not None: piece = possible else: piece = piece.title() if piece[-2:].startswith("'"): piece = piece[:-1] + piece[-1].lower() result.append(piece) Ugly, I know -- but the question is: how would 'match' handle things like '-' in piece piece.upper() in .... piece in (....) piece[:2] = 'mc' in other words, would match save me much in this circumstance, and would it be easier to read? -1 ~Ethan~ From tjreedy at udel.edu Sat Dec 18 01:14:07 2010 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 17 Dec 2010 19:14:07 -0500 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012180021.37232.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> Message-ID: On 12/17/2010 6:21 PM, Eike Welk wrote: > After learning a bit of Ocaml I started to like its pattern matching features. > Since then I want to have a "match" statement in Python. I wonder if anybody > else would like this too. -1 on adding anything like this to core Python. In your example, it is 1. completely redundant with respect to current Python syntax; 2. look like chicken scratches; 3. limited by the chars availables to just a few of the infinity of tests one might want to run. 4. gives special prominence to tuples, which is hardly appropriate to list or iterable-oriented code. The third point is why Python does not try to everything with syntax symbols. How test that the input is a positive number?How test that the input is a positive number? Type testing is somewhat contrary to duck-typing. For instance, how would you test that the input is an iterable? > ML style pattern matching is syntactic sugar, that combines "if" statements > with tuple unpacking, access to object attributes, and assignments. It is a > compact, yet very readable syntax for algorithms, that would otherwise require > nested "if" statements. Your example does not use nesting. > It is especially useful for writing interpreters, and > processing complex trees. Sounds like better suited to a special-purpose 3rd party module, like pyparsing. > Instead of a specification in BNF, here is a function written with the > proposed pattern matching syntax. It demonstrates the features that I find > most important. The comments and the print statements explain what is done. > > > Proposed Syntax > --------------- > > def foo(x): > match x with > | 1 -> # Equality > print("x is equal to 1") > | a:int -> # Type check > print("x has type int: %s" % a) > | (a, b) -> # Tuple unpacking > print("x is a tuple with length 2: (%s, %s)" % (a, b)) > | {| a, b |} -> # Attribute existence and access > print("x is an object with attributes 'a' and 'b'.") > print("a=%s, b=%s" % (a, b)) > > # Additional condition > | (a, b, c) with a> b -> > print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) > print("The first element is greater than the second element.") > > # Complex case > | {| c:int, d=1 |}:Foo -> > print("x has type Foo") > print("x is an object with attributes 'c' and 'd'.") > print("'c' has type 'int', 'd' is equal to 1.") > print("c=%s, d=%s" % (c, d)) > > # Default case > | _ -> > print("x can be anything") > > > Equivalent Current Python > ------------------------- > > The first four cases could be handled more simply, but handling all cases in > the same way leads IMHO to more simple code overall. > > def foo(x): > while True: > # Equality > if x == 1: > print("x is equal to 1") > break > > # Type check > if isinstance(x, int): > a = x > print("x is an integer: %s" % a) > break > > # Tuple unpacking > if isinstance(x, tuple) and len(x) == 2: > a, b = x > print("x is a tuple with length 2: (%s, %s)" % (a, b)) > break > > # Attribute existence testing and access > if hasattr(x, "a") and hasattr(x, "b"): > a, b = x.a, x.b > print("x is an object with attributes 'a' and 'b'.") > print("a=%s, b=%s" % (a, b)) > break > > # Additional condition > if isinstance(x, tuple) and len(x) == 3: > a, b, c = x > if a> b : > print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) > print("The first element is greater than the second " > "element.") > break > > # Complex case > if isinstance(x, Foo) and hasattr(x, "c") and hasattr(x, "d"): > c, d = x.c, x.d > if isinstance(c, int) and d == 1: > print("x has type Foo") > print("x is an object with attributes 'c' and 'd'.") > print("'c' has type 'int', 'd' is equal to 1.") > print("c=%s, d=%s" % (c, d)) > break > > # Default case > print("x can be anything") > break > > > Additional Code to Run Function "foo" > ------------------------------------- > > class Bar(object): > def __init__(self, a, b): > self.a = a > self.b = b > > class Foo(object): > def __init__(self, c, d): > self.c = c > self.d = d > > > foo(1) # Equality > foo(2) # Type check > foo((1, 2)) # Tuple unpacking > foo(Bar(1, 2)) # Attribute existence testing and access > foo((2, 1, 3)) # Additional condition > foo(Foo(2, 1)) # Complex case > foo("hello") # Default case > > > > I left out dict and set, because I'm not sure how they should be handled. I > think list should be handled like tuples. Probably there should be a universal > matching syntax for all sequences, similarly to the already existing syntax: > a, b, *c = s > > I don't really like the "->" digraph at the end of each match case. A colon > would be much more consistent, but I use colons already for type checking > (a:int). > > I generally think that Python should acquire more features from functional > languages. In analogy to "RPython" it should ultimately lead to "MLPython", a > subset of the Python language that can be type checked and reasoned about by > external tools, similarly to what is possible with Ocaml. > > > > Eike. -- Terry Jan Reedy From cmjohnson.mailinglist at gmail.com Sat Dec 18 02:25:54 2010 From: cmjohnson.mailinglist at gmail.com (Carl M. Johnson) Date: Fri, 17 Dec 2010 15:25:54 -1000 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: References: <201012180021.37232.eike.welk@gmx.net> Message-ID: It's already trivial to make a decorator to do pattern matching, and if PJE ever finished generic functions, it will get trivial-er. You just need to create a simple decorator to do something like this: @basecase def fac(n): return n * fac(n -1) @f.equals(1): def f(n): return 1 @f.lessthan(1) def f(n): raise Error("n less than 1, operation not defined.") It's not that pretty, but I don't see any need for a whole new syntax for what is basically a glorified elif. On Fri, Dec 17, 2010 at 2:14 PM, Terry Reedy wrote: > On 12/17/2010 6:21 PM, Eike Welk wrote: >> >> After learning a bit of Ocaml I started to like its pattern matching >> features. >> Since then I want to have a "match" statement in Python. I wonder if >> anybody >> else would like this too. > > -1 on adding anything like this to core Python. > In your example, it is > 1. completely redundant with respect to current Python syntax; > 2. look like chicken scratches; > 3. limited by the chars availables to just a few of the infinity of tests > one might want to run. > 4. gives special prominence to tuples, which is hardly appropriate to list > or iterable-oriented code. > > The third point is why Python does not try to everything with syntax > symbols. How test that the input is a positive number?How test that the > input is a positive number? Type testing is somewhat contrary to > duck-typing. For instance, how would you test that the input is an iterable? > >> ML style pattern matching is syntactic sugar, that combines "if" >> statements >> with tuple unpacking, access to object attributes, and assignments. It is >> a >> compact, yet very readable syntax for algorithms, that would otherwise >> require >> nested "if" statements. > > Your example does not use nesting. > >> It is especially useful for writing interpreters, and >> >> processing complex trees. > > Sounds like better suited to a special-purpose 3rd party module, like > pyparsing. > >> Instead of a specification in BNF, here is a function written with the >> proposed pattern matching syntax. It demonstrates the features that I find >> most important. The comments and the print statements explain what is >> done. >> >> >> Proposed Syntax >> --------------- >> >> def foo(x): >> ? ? match x with >> ? ? | 1 -> ? ? ? ? ? ? ? ? ? ? ? # Equality >> ? ? ? ? print("x is equal to 1") >> ? ? | a:int -> ? ? ? ? ? ? ? ? ? # Type check >> ? ? ? ? print("x has type int: %s" % a) >> ? ? | (a, b) -> ? ? ? ? ? ? ? ? ?# Tuple unpacking >> ? ? ? ? print("x is a tuple with length 2: (%s, %s)" % (a, b)) >> ? ? | {| a, b |} -> ? ? ? ? ? ? ?# Attribute existence and access >> ? ? ? ? print("x is an object with attributes 'a' and 'b'.") >> ? ? ? ? print("a=%s, b=%s" % (a, b)) >> >> ? ? # Additional condition >> ? ? | (a, b, c) with a> ?b -> >> ? ? ? ? print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) >> ? ? ? ? print("The first element is greater than the second element.") >> >> ? ? # Complex case >> ? ? | {| c:int, d=1 |}:Foo -> >> ? ? ? ? print("x has type Foo") >> ? ? ? ? print("x is an object with attributes 'c' and 'd'.") >> ? ? ? ? print("'c' has type 'int', 'd' is equal to 1.") >> ? ? ? ? print("c=%s, d=%s" % (c, d)) >> >> ? ? # Default case >> ? ? | _ -> >> ? ? ? ? print("x can be anything") >> >> >> Equivalent Current Python >> ------------------------- >> >> The first four cases could be handled more simply, but handling all cases >> in >> the same way leads IMHO to more simple code overall. >> >> def foo(x): >> ? ? while True: >> ? ? ? ? # Equality >> ? ? ? ? if x == 1: >> ? ? ? ? ? ? print("x is equal to 1") >> ? ? ? ? ? ? break >> >> ? ? ? ? # Type check >> ? ? ? ? if isinstance(x, int): >> ? ? ? ? ? ? a = x >> ? ? ? ? ? ? print("x is an integer: %s" % a) >> ? ? ? ? ? ? break >> >> ? ? ? ? # Tuple unpacking >> ? ? ? ? if isinstance(x, tuple) and len(x) == 2: >> ? ? ? ? ? ? a, b = x >> ? ? ? ? ? ? print("x is a tuple with length 2: (%s, %s)" % (a, b)) >> ? ? ? ? ? ? break >> >> ? ? ? ? # Attribute existence testing and access >> ? ? ? ? if hasattr(x, "a") and hasattr(x, "b"): >> ? ? ? ? ? ? a, b = x.a, x.b >> ? ? ? ? ? ? print("x is an object with attributes 'a' and 'b'.") >> ? ? ? ? ? ? print("a=%s, b=%s" % (a, b)) >> ? ? ? ? ? ? break >> >> ? ? ? ? # Additional condition >> ? ? ? ? if isinstance(x, tuple) and len(x) == 3: >> ? ? ? ? ? ? a, b, c = x >> ? ? ? ? ? ? if a> ?b : >> ? ? ? ? ? ? ? ? print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, >> c)) >> ? ? ? ? ? ? ? ? print("The first element is greater than the second " >> ? ? ? ? ? ? ? ? ? ? ? ? "element.") >> ? ? ? ? ? ? ? ? break >> >> ? ? ? ? # Complex case >> ? ? ? ? if isinstance(x, Foo) and hasattr(x, "c") and hasattr(x, "d"): >> ? ? ? ? ? ? c, d = x.c, x.d >> ? ? ? ? ? ? if isinstance(c, int) and d == 1: >> ? ? ? ? ? ? ? ? print("x has type Foo") >> ? ? ? ? ? ? ? ? print("x is an object with attributes 'c' and 'd'.") >> ? ? ? ? ? ? ? ? print("'c' has type 'int', 'd' is equal to 1.") >> ? ? ? ? ? ? ? ? print("c=%s, d=%s" % (c, d)) >> ? ? ? ? ? ? ? ? break >> >> ? ? ? ? # Default case >> ? ? ? ? print("x can be anything") >> ? ? ? ? break >> >> >> Additional Code to Run Function "foo" >> ------------------------------------- >> >> class Bar(object): >> ? ? def __init__(self, a, b): >> ? ? ? ? self.a = a >> ? ? ? ? self.b = b >> >> class Foo(object): >> ? ? def __init__(self, c, d): >> ? ? ? ? self.c = c >> ? ? ? ? self.d = d >> >> >> foo(1) ? ? ? ? ?# Equality >> foo(2) ? ? ? ? ?# Type check >> foo((1, 2)) ? ? # Tuple unpacking >> foo(Bar(1, 2)) ?# Attribute existence testing and access >> foo((2, 1, 3)) ?# Additional condition >> foo(Foo(2, 1)) ?# Complex case >> foo("hello") ? ?# Default case >> >> >> >> I left out dict and set, because I'm not sure how they should be handled. >> I >> think list should be handled like tuples. Probably there should be a >> universal >> matching syntax for all sequences, similarly to the already existing >> syntax: >> ? ? a, b, *c = s >> >> I don't really like the "->" digraph at the end of each match case. A >> colon >> would be much more consistent, but I use colons already for type checking >> (a:int). >> >> I generally think that Python should acquire more features from functional >> languages. In analogy to "RPython" it should ultimately lead to >> "MLPython", a >> subset of the Python language that can be type checked and reasoned about >> by >> external tools, similarly to what is possible with Ocaml. >> >> >> >> Eike. > > > -- > Terry Jan Reedy > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > From eike.welk at gmx.net Sat Dec 18 02:31:26 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sat, 18 Dec 2010 02:31:26 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0BFC93.5000503@stoneleaf.us> References: <201012180021.37232.eike.welk@gmx.net> <4D0BFC93.5000503@stoneleaf.us> Message-ID: <201012180231.27348.eike.welk@gmx.net> On Saturday 18.12.2010 01:13:07 Ethan Furman wrote: > I am unfamiliar with OCAML -- if x can match more than one condition, > will it match all possible, or just the first one? If just the first > one, the python code below can be simplified by ditching the while loop, > removing the breaks, and using elif and else. The match statement should indeed match (and execute) only the first condition. However you can't simplify the code the way you do: If one of the inner "if" statements fails, the algorithm should try the next match case. > > def foo(x): > # Equality > if x == 1: > print("x is equal to 1") > > # Type check > elif isinstance(x, int): > a = x > print("x is an integer: %s" % a) > > # Tuple unpacking > elif isinstance(x, tuple) and len(x) == 2: > a, b = x > print("x is a tuple with length 2: (%s, %s)" % (a, b)) > > # Attribute existence testing and access > elif hasattr(x, "a") and hasattr(x, "b"): > a, b = x.a, x.b > print("x is an object with attributes 'a' and 'b'.") > print("a=%s, b=%s" % (a, b)) > > # Additional condition > elif isinstance(x, tuple) and len(x) == 3: > a, b, c = x > if a > b : > print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) > print("The first element is greater than the second " > "element.") > > # Complex case > elif isinstance(x, Foo) and hasattr(x, "c") and hasattr(x, "d"): > c, d = x.c, x.d > if isinstance(c, int) and d == 1: > print("x has type Foo") > print("x is an object with attributes 'c' and 'd'.") > print("'c' has type 'int', 'd' is equal to 1.") > print("c=%s, d=%s" % (c, d)) > > # Default case > else: > print("x can be anything") > > > One of the things I like about Python is it's readability. While the > OCAML inspired version is much more concise, I don't see it as > significantly easier to read -- and at this point I don't see any place > where I would make use of it myself. > > Here's a bit of code that might be convertable: > > result = [] > for i, piece in enumerate(pieces): > if '-' in piece: > piece = piece.replace('-',' ') > piece = '-'.join(NameCase(piece).split()) > elif alpha_num(piece) in ('i', 'ii', 'iii', 'iv', 'v', 'vi', \ > 'vii', 'viii', 'ix', 'x', 'pc', 'llc') \ > or piece.upper() in job_titles \ > or i and piece.upper() in deg_suffixi: > piece = piece.upper() > elif piece in ('and', 'de', 'del', 'der', 'el', 'la', 'van', ): > pass > elif piece[:2] == 'mc': > piece = 'Mc' + piece[2:].title() > else: > possible = mixed_case_names.get(piece, None) > if possible is not None: > piece = possible > else: > piece = piece.title() > if piece[-2:].startswith("'"): > piece = piece[:-1] + piece[-1].lower() > result.append(piece) > > Ugly, I know -- but the question is: how would 'match' handle things like > '-' in piece > piece.upper() in .... > piece in (....) > piece[:2] = 'mc' > > in other words, would match save me much in this circumstance, and would > it be easier to read? Your example can't be easily simplified with the match statement. The match statement isn't meant for string processing, but rather for nested tuples, or objects. Additionally I didn't yet think about set operations (for "in"). The strength of the match statement is taking complex objects apart, and testing some conditions on the pieces, at the same time. Eike. From steve at pearwood.info Sat Dec 18 02:35:30 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 18 Dec 2010 12:35:30 +1100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012180021.37232.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> Message-ID: <4D0C0FE2.6000609@pearwood.info> Eike Welk wrote: > After learning a bit of Ocaml I started to like its pattern matching features. > Since then I want to have a "match" statement in Python. I wonder if anybody > else would like this too. I've heard of pattern matching in other languages, but never had the opportunity to play around with them. It seems to me though, that it's just an enhanced case or switch statement. Python already has had a proposal to add a case/switch to the language. Unfortunately, due to lack of consensus on functionality and syntax, and lack of any pressing need, it hasn't gone anywhere. > ML style pattern matching is syntactic sugar, that combines "if" statements > with tuple unpacking, access to object attributes, and assignments. It is a > compact, yet very readable syntax for algorithms, that would otherwise require > nested "if" statements. It is especially useful for writing interpreters, and > processing complex trees. It doesn't seem very readable to me. It looks like it is heavy on "magic" characters... for example, what is the purpose of the leading | character and the trailing -> digraph in this, and how does (| |) imply attribute access given the rest of Python's syntax? match x with | {| a, b |} -> # Attribute existence and access print("x is an object with attributes 'a' and 'b'.") print("a=%s, b=%s" % (a, b)) It would require a new keyword "match", and overloading an existing keyword "with" to have a second meaning. Python is *very* resistant to adding new keywords. You would need to demonstrate that pattern matching leads to significant benefits over if...elif...else in order to compensate for the pain you cause to those who use "match" as a variable name. This includes Python's own standard library, which has match functions and methods in the re module. It also uses special characters in a way that looks more like Perl than Python. With few exceptions -- iterable slicing comes to mind -- Python tends to avoid magic characters like that. If you are serious about pursuing this idea, I suggest you need to demonstrate some non-trivial gains from pattern matching. Perhaps you should also look at how Haskell does it, and how functions in Haskell can be written concisely. Here's a toy example: factorial :: Integer -> Integer factorial 0 = 1 factorial n = n * factorial (n - 1) I'm interested in pattern matching, but I think the suggested syntax is completely inappropriate for Python. I think that for this to have any hope, its best bet would be as an enhancement of the case/switch PEP, and it would need to look like Python code, not like Haskell or OCAML. -- Steven From greg.ewing at canterbury.ac.nz Sat Dec 18 03:03:06 2010 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 18 Dec 2010 15:03:06 +1300 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012180021.37232.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> Message-ID: <4D0C165A.60609@canterbury.ac.nz> Eike Welk wrote: > After learning a bit of Ocaml I started to like its pattern matching features. > Since then I want to have a "match" statement in Python. I wonder if anybody > else would like this too. I don't object to the general idea, but syntax such as > | {| a, b |} -> # Attribute existence and access is far too cryptic and does nothing to improve readability. -- Greg From ncoghlan at gmail.com Sat Dec 18 07:43:34 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 18 Dec 2010 17:43:34 +1100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012180021.37232.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> Message-ID: Rather than such a large, monolithic construct, perhaps consider pursuing smaller tweaks that could improve the following. def _check_call(f, x): ??? try: ??????? return f(*x) ??? except TypeError: ??????? return False if x ==1: ? ? pass elif isinstance(x, int): ? ? pass elif isinstance(x, tuple) and len(x) == 2: ? ? pass elif hasattr(x, 'a') and hasattr(x, 'b'): ??? pass elif _check_call((lambda a, b: a > b), x): ??? pass elif isinstance(x, Foo) and isinstance(getattr(x, 'c', None), int) and getattr(x, 'd', None) == 1: pass else: pass Cheers. Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From denis.spir at gmail.com Sat Dec 18 11:23:38 2010 From: denis.spir at gmail.com (spir) Date: Sat, 18 Dec 2010 11:23:38 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012180021.37232.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> Message-ID: <20101218112338.73c77517@o> On Sat, 18 Dec 2010 00:21:35 +0100 Eike Welk wrote: > After learning a bit of Ocaml I started to like its pattern matching features. > Since then I want to have a "match" statement in Python. I wonder if anybody > else would like this too. > > ML style pattern matching is syntactic sugar, that combines "if" statements > with tuple unpacking, access to object attributes, and assignments. It is a > compact, yet very readable syntax for algorithms, that would otherwise require > nested "if" statements. It is especially useful for writing interpreters, and > processing complex trees. While I generally like pattern matching in FP languages, I think it is not appropriate and not necessary for a general application language like Python. It is an important feature in FP because (1) it matches (!) FP's general point of view or paradigm (2) FP language example application are very much about algorithmics --far less about general designing/modelling. > Instead of a specification in BNF, here is a function written with the > proposed pattern matching syntax. It demonstrates the features that I find > most important. The comments and the print statements explain what is done. If ever pattern matching would be considered for inclusion in Python, it would have to fit Python's style, which your proposed syntax really does not. Actually, I find your current-python translation far more readable (except for the enclosing while true hack). The syntax should look switch/case, probably. A keyword seems necessary to tell apart ordinary switch conditions from pattern matching ones. But it cannot be "match", so let's say "matching". Other changes inline below: > Proposed Syntax > --------------- > def foo(x): > match x with matching x: > | 1 -> # Equality case 1: > print("x is equal to 1") > | a:int -> # Type check case int: # <=> isinstance(x, int) or x.__type__==int ??? > print("x has type int: %s" % a) > | (a, b) -> # Tuple unpacking case (a, b): > print("x is a tuple with length 2: (%s, %s)" % (a, b)) > | {| a, b |} -> # Attribute existence and access case (.a, .b): > print("x is an object with attributes 'a' and 'b'.") > print("a=%s, b=%s" % (a, b)) > > # Additional condition > | (a, b, c) with a > b -> case (.a, .b, .c) and a > b: > print("x is a tuple with length 3: (%s, %s, %s)" % (a, b, c)) > print("The first element is greater than the second element.") > > # Complex case > | {| c:int, d=1 |}:Foo -> case Foo(.c, .d) and isinstance(c,int) and d == 1: case Foo(.c, .d) and c.__type__ == int and d == 1: > print("x has type Foo") > print("x is an object with attributes 'c' and 'd'.") > print("'c' has type 'int', 'd' is equal to 1.") > print("c=%s, d=%s" % (c, d)) > > # Default case > | _ -> else: > print("x can be anything") > Equivalent Current Python > ------------------------- > > The first four cases could be handled more simply, but handling all cases in > the same way leads IMHO to more simple code overall. > [...] Seems the only annoying aspect in current python is expressing secondary conditions. For instance, if x can be int and have either .a or .b, with different actions in sub-cases. Expressing this "naturally" in python is wrong because if x has neither .a nore .b other cases won't be matched. The only workaround is to have several top-level cases, repeating the top condition about x beeing an int. I guess. > I left out dict and set, because I'm not sure how they should be handled. I > think list should be handled like tuples. Probably there should be a universal > matching syntax for all sequences, similarly to the already existing syntax: > a, b, *c = s This cannot be! Even less lists. Proposal: case [] # check list case [n] # ditto; n must be int telling length case {a, b, c} # check set / elements case {a:, b:, c:} # check dict / keys An annoying thing is python has no syntax for structured objects. I find acceptable the above used workaround: (.a, .b, .c) to tell about attributes and T(.a, .b, .c) to additionally check the type. > I don't really like the "->" digraph at the end of each match case. A colon > would be much more consistent, but I use colons already for type checking > (a:int). Does a:int means isinstance(a,int) or a.__type__==int ? I would like python to have an 'isa' operator for the latter. > I generally think that Python should acquire more features from functional > languages. In analogy to "RPython" it should ultimately lead to "MLPython", a > subset of the Python language that can be type checked and reasoned about by > external tools, similarly to what is possible with Ocaml. I doubt about that for a handful of reasons. ref to related feature, FWIW: OMeta general structural pattern matching http://tinlizzie.org/ometa/ (with implementation for python) Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From eike.welk at gmx.net Sat Dec 18 12:23:45 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sat, 18 Dec 2010 12:23:45 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0C165A.60609@canterbury.ac.nz> References: <201012180021.37232.eike.welk@gmx.net> <4D0C165A.60609@canterbury.ac.nz> Message-ID: <201012181223.46790.eike.welk@gmx.net> On Saturday 18.12.2010 03:03:06 Greg Ewing wrote: > Eike Welk wrote: > > | {| a, b |} -> # Attribute existence and access > > is far too cryptic and does nothing to improve readability. List comprehensions are also cryptic if you see them for the first time. The curly brackets with bars "{| |}" should symbolize that Python object are glorified dicts. It is also quite close to Ocaml's syntax for records. My secret agenda was however, to later introduce a new class and object syntax for Python. I just didn't dare to propose a completely revised Python. But here we go: :-) Class Creation -------------- Foo = class {| a, b |} This should be equivalent to: class Foo(object): def __init__(self, a, b): self.a = a self.b = b Inheritance is not so important in the context of classes that have no methods. It could be expressed with a new method ("inherits") of the metaclass. Like this: Foo = class {| a, b |}.inherits(Bar, Baz) How one would create methods, and if it should be possible to create method at all, needs to be discussed. Instance Creation ----------------- foo = {| a=1, b=2 |}:Foo This should be equivalent to: foo = Foo(a=1, b=2) In the usual case, it should not be necessary to specify the class name: foo = {| a=1, b=2 |} The run-time should search for a class with a matching "__init__" method. In case of ambiguities an exception would be raised. You have to name object attributes unambiguously for this to work. However it is more flexible than Ocaml because you can disambiguate it by specifying the class name. These constructions are expression, and can therefore be nested. The next nice idea from Ocaml would be extending the class system ... I'm getting a bit off topic though. I'll present a revised syntax for object access later; generalized constructors. Eike. From denis.spir at gmail.com Sat Dec 18 12:45:49 2010 From: denis.spir at gmail.com (spir) Date: Sat, 18 Dec 2010 12:45:49 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012181223.46790.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <4D0C165A.60609@canterbury.ac.nz> <201012181223.46790.eike.welk@gmx.net> Message-ID: <20101218124549.35a0d1c9@o> On Sat, 18 Dec 2010 12:23:45 +0100 Eike Welk wrote: > Class Creation > -------------- > > Foo = class {| a, b |} > > This should be equivalent to: > > class Foo(object): > def __init__(self, a, b): > self.a = a > self.b = b > > Inheritance is not so important in the context of classes that have no > methods. It could be expressed with a new method ("inherits") of the > metaclass. Like this: > > Foo = class {| a, b |}.inherits(Bar, Baz) > > How one would create methods, and if it should be possible to create method at > all, needs to be discussed. What advantage? > Instance Creation > ----------------- > > foo = {| a=1, b=2 |}:Foo > > This should be equivalent to: > > foo = Foo(a=1, b=2) > > In the usual case, it should not be necessary to specify the class name: > > foo = {| a=1, b=2 |} I want composite object literal notation as well. But certainly not {| a=1, b=2 |}. Rather (a=1, b=2) or (a:1, b:2). Untyped case would created an instance of Object (but since as of now they can't have attrs, there should be another modif), or of a new FreeObject subtype of Object. > The run-time should search for a class with a matching "__init__" method. ? conflicts? Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From eike.welk at gmx.net Sat Dec 18 14:31:12 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sat, 18 Dec 2010 14:31:12 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <20101218112338.73c77517@o> References: <201012180021.37232.eike.welk@gmx.net> <20101218112338.73c77517@o> Message-ID: <201012181431.13689.eike.welk@gmx.net> Great that you thought about a syntax for dicts and sets. One other point that must be solved, is how to distinguish symbols that should be assigned, from symbols that are specify conditions. Ocamls solution is that all symbols are assigned. Conditions must be specified with literals. Here is an example to illustrate it: This is a simple function that compares its argument "x" to the integer one: def foo(x): match x with | 1 -> print("x is equal to 1") | _ -> print("x is not equal to 1") In the code below we want to parametrize the value that we use for comparison. The parameter "a" is introduced for this purpose. Now we need a way to tell the compiler that "x" should be compared to "a"; and not assigned to "a". In Ocaml this is impossible. The compiler would issue a warning ("Redundant case in a pattern matching."). def foo(x, a): match x with | a -> print("x is equal to %s" % a) | _ -> print("x is not equal to %s" % a) Maybe you come up with a nice notation for this problem. By the way, why does no one like those "|" characters? They look good and take up few space! Eike. From denis.spir at gmail.com Sat Dec 18 15:07:38 2010 From: denis.spir at gmail.com (spir) Date: Sat, 18 Dec 2010 15:07:38 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012181431.13689.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <20101218112338.73c77517@o> <201012181431.13689.eike.welk@gmx.net> Message-ID: <20101218150738.2e838cef@o> On Sat, 18 Dec 2010 14:31:12 +0100 Eike Welk wrote: > Great that you thought about a syntax for dicts and sets. > > > One other point that must be solved, is how to distinguish symbols that should > be assigned, from symbols that are specify conditions. Ocamls solution is that > all symbols are assigned. Conditions must be specified with literals. Here is > an example to illustrate it: > > > This is a simple function that compares its argument "x" to the integer one: > > def foo(x): > match x with > | 1 -> > print("x is equal to 1") > | _ -> > print("x is not equal to 1") > > > In the code below we want to parametrize the value that we use for comparison. > The parameter "a" is introduced for this purpose. Now we need a way to tell > the compiler that "x" should be compared to "a"; and not assigned to "a". In > Ocaml this is impossible. The compiler would issue a warning ("Redundant case > in a pattern matching."). > > def foo(x, a): > match x with > | a -> > print("x is equal to %s" % a) > | _ -> > print("x is not equal to %s" % a) > > Maybe you come up with a nice notation for this problem. No, I would do it like in OCaml, I think (but may be wrong) it fits the python-way, like having no formally qualified readonly or private slots. > By the way, why does no one like those "|" characters? They look good and take > up few space! ;-) It does not fit Python overall syntax look. And requires a magic id '_' for default case. And requires adding '->'. And we already have 'case x:' (and 'else') which fit perfectly. And are far more readable (no need to guess what given symbols mean, including the 'sense' of an arrow ;-). > Eike. Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From ethan at stoneleaf.us Sat Dec 18 18:27:53 2010 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 18 Dec 2010 09:27:53 -0800 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012181223.46790.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <4D0C165A.60609@canterbury.ac.nz> <201012181223.46790.eike.welk@gmx.net> Message-ID: <4D0CEF19.7010505@stoneleaf.us> Eike Welk wrote: > My secret agenda was however, to later introduce a new class and object syntax > for Python. I just didn't dare to propose a completely revised Python. But > here we go: :-) > > > Class Creation > -------------- > > Foo = class {| a, b |} > > This should be equivalent to: > > class Foo(object): > def __init__(self, a, b): > self.a = a > self.b = b Really? So Foobar = class {| 17, 'yellow' |} means class Foobar() def __init__(self, 17, 'yellow'): self.17 = 17 self.'yellow' = 'yellow' Looks like I'm either stuck with always using a, b, c, etc for attribute names, or I get compile errors. This is *definitely* not Pythonic -- offers nothing for readability, requires new strange symbols... yuck. If you want a one-liner, make a function: def Class(*args, **kwargs): obj = type('Simple', (object, ), dict()) for i, arg in enumerate(args): attr = chr(ord('a') + i) setattr(obj, attr, arg) for kw in kwargs: setattr(obj, kw, kwargs[kw]) return obj then Foobar = Class( 17, 'yellow', this='that' ) and season to taste. ~Ethan~ From g.brandl at gmx.net Sat Dec 18 18:34:57 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 18 Dec 2010 18:34:57 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012181223.46790.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <4D0C165A.60609@canterbury.ac.nz> <201012181223.46790.eike.welk@gmx.net> Message-ID: Am 18.12.2010 12:23, schrieb Eike Welk: > Instance Creation > ----------------- > > foo = {| a=1, b=2 |}:Foo > > This should be equivalent to: > > foo = Foo(a=1, b=2) > > In the usual case, it should not be necessary to specify the class name: > > foo = {| a=1, b=2 |} > > The run-time should search for a class with a matching "__init__" method. I'm beginning to suspect this is an elaborate trolling attempt... From eike.welk at gmx.net Sat Dec 18 19:29:55 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sat, 18 Dec 2010 19:29:55 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> Message-ID: <201012181929.56841.eike.welk@gmx.net> On Saturday 18.12.2010 18:34:57 Georg Brandl wrote: > Am 18.12.2010 12:23, schrieb Eike Welk: > > Instance Creation > > ----------------- > > > > foo = {| a=1, b=2 |}:Foo > > > > This should be equivalent to: > > > > foo = Foo(a=1, b=2) > > > > In the usual case, it should not be necessary to specify the class name: > > > > foo = {| a=1, b=2 |} > > > > The run-time should search for a class with a matching "__init__" method. > > I'm beginning to suspect this is an elaborate trolling attempt... Yes, it's a bit of fun. This would never get into Python anyway, but I'd like it nevertheless. (It is Ocaml's record syntax with some small tweaks.) It would be possible to realize it though. With "inspect.getfullargspec" you can see the a function's signature. There might be problems with efficiency, because you'd have to search through globals and locals repeatedly. You you could however store which class matched, somewhere in the code object. Storing this kind of information would go against Python's nature as a dynamic language, even though it would work as expected in 99% of the use cases. Oh well ... Eike From ianb at colorstudy.com Sat Dec 18 20:18:59 2010 From: ianb at colorstudy.com (Ian Bicking) Date: Sat, 18 Dec 2010 11:18:59 -0800 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012181929.56841.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <201012181929.56841.eike.welk@gmx.net> Message-ID: I don't have any real opinion on this, but this old experiment seems related to the discussion: http://svn.colorstudy.com/home/ianb/recipes/patmatch.py -- Sent from a phone On Dec 18, 2010 12:30 PM, "Eike Welk" wrote: On Saturday 18.12.2010 18:34:57 Georg Brandl wrote: > Am 18.12.2010 12:23, schrieb Eike Welk: > > In... Yes, it's a bit of fun. This would never get into Python anyway, but I'd like it nevertheless. (It is Ocaml's record syntax with some small tweaks.) It would be possible to realize it though. With "inspect.getfullargspec" you can see the a function's signature. There might be problems with efficiency, because you'd have to search through globals and locals repeatedly. You you could however store which class matched, somewhere in the code object. Storing this kind of information would go against Python's nature as a dynamic language, even though it would work as expected in 99% of the use cases. Oh well ... Eike _______________________________________________ Python-ideas mailing list Python-ideas at python.org ht... -------------- next part -------------- An HTML attachment was scrubbed... URL: From m_daoust at hotmail.com Sat Dec 18 21:41:06 2010 From: m_daoust at hotmail.com (M Daoust) Date: Sat, 18 Dec 2010 15:41:06 -0500 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: References: Message-ID: > Date: Sat, 18 Dec 2010 12:45:49 +0100 > From: spir > To: python-ideas at python.org > Subject: Re: [Python-ideas] ML Style Pattern Matching for Python > Message-ID: <20101218124549.35a0d1c9 at o> > Content-Type: text/plain; charset=UTF-8 > > On Sat, 18 Dec 2010 12:23:45 +0100 > Eike Welk wrote: > > ... > > I want composite object literal notation as well. But certainly not {| a=1, b=2 |}. Rather (a=1, b=2) or (a:1, b:2). > Untyped case would created an instance of Object (but since as of now they can't have attrs, there should be another modif), or of a new FreeObject subtype of Object. > ... > > Denis "(a=1, b=2) or (a:1, b:2)" These look to me like they should produce a named-tuple instance. I usually just use: ... class Simple(object): ... def __init__(self, **kwargs): ... self.__dict__.update(kwargs) ... Simple(a=1,b=2) -Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From bruce at leapyear.org Sat Dec 18 22:26:05 2010 From: bruce at leapyear.org (Bruce Leban) Date: Sat, 18 Dec 2010 13:26:05 -0800 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: References: <201012180021.37232.eike.welk@gmx.net> Message-ID: On 12/17/2010 6:21 PM, Eike Welk wrote: > >> After learning a bit of Ocaml I started to like its pattern matching >> features. >> Since then I want to have a "match" statement in Python. I wonder if >> anybody >> else would like this too. > > On Fri, Dec 17, 2010 at 4:14 PM, Terry Reedy wrote: > 1. completely redundant with respect to current Python syntax; > 2. look like chicken scratches; > 3. limited by the chars availables to just a few of the infinity of tests > one might want to run. > 4. gives special prominence to tuples, which is hardly appropriate to list > or iterable-oriented code. > > I'm skeptical that anything is worth adding here, but here's an alternative. I'm only looking at the decomposition part. I'm using a matches keyword and "&" and "*". (The choice here doesn't matter but I have to use something for the examples, so please don't pick on that aspect.) x = (3, 4) if x matches (&a, &b): print(a, b) is equivalent to x = (3, 4) if isinstance(x, tuple) and len(x) == 2: a, b = x print(a, b) Or a bit more complicated: x = [3, 4] if x matches [3, &b, *]: print(b) is equivalent to x = [3, 4] if isinstance(x, list) and x[0] == 3: _, b, *_ = x print(b) and if x matches {j : &k}: print(k) means if isinstance(x, dict) and j in x and len(x) == 1: k = a[j] print(k) Here's a more complex example: x matches { 'account': (&username, &domain), 'type': 'administrator', 'password': &hash, 'friends': (&best_friend, *&other_friends), * } Whether or not this is useful at all is a big question, but I think it's at least more interesting. This isn't perfect. For example x matches Foo(a=&alpha, b=&beta) could mean checking x.hasattr('a') but there's no guarantee that Foo(a=1, b=2) will produce that result. Maybe that's OK. Also, any values on the right hand side not marked by & are matched for equality, so you could write x matches (3 * 14 + big_complicated_expression, &y) which is pretty ugly. And that suggests: x matches (f(&y), g(&z)) which either doesn't make sense or is messy. We could add a __matches__ attribute to functions so they could support this, but now this is getting pretty complicated for unclear benefit. --- Bruce Latest blog post: http://www.vroospeak.com/2010/11/enduring-joe-barton.html Learn about security: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From eike.welk at gmx.net Sun Dec 19 01:40:53 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sun, 19 Dec 2010 01:40:53 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: References: <201012180021.37232.eike.welk@gmx.net> Message-ID: <201012190140.55231.eike.welk@gmx.net> I would love to see this implemented! Off course I like my syntax better. :-) On Saturday 18.12.2010 22:26:05 Bruce Leban wrote: > x = (3, 4) > if x matches (&a, &b): > print(a, b) > > > is equivalent to > > x = (3, 4) > if isinstance(x, tuple) and len(x) == 2: > a, b = x > print(a, b) ... > Whether or not this is useful at all is a big question, but I think it's at > least more interesting. This isn't perfect. For example > > x matches Foo(a=&alpha, b=&beta) > For this case I prose that the class "Foo" should implement a special function "__rinit__" that breaks an instance apart, into components that can be matched. These components should be chosen so that they could be given to "__init__" as its arguments. "__rinit__" should return the following objects: * a tuple of objects, corresponding to the regular arguments of "__init__". * a tuple of strings that contains the argument names. * a dict of string:object that represents the keyword only attributes. A simple class would look like this: class Foo(object): def __init__(self, c, d): self.c = c self.d = d def __rinit__(self): return (self.c, self.d), ("c", "d"), {} The example from above: x matches Foo(c=&gamma, d=&delta) would work like this: if isinstance(x, Foo): vals, names, kwargs = x.__rinit__() gamma = vals[names.index("c")] delta = vals[names.index("d")] Positional arguments are also possible: x matches Foo(&gamma, &delta) would work like this: if isinstance(x, Foo): vals, names, kwargs = x.__rinit__() gamma = vals[0] delta = vals[0] A type check would be expressed like this: x matches Foo(&*_) > could mean checking x.hasattr('a') but there's no guarantee that Foo(a=1, > b=2) will produce that result. Maybe that's OK. > > Also, any values on the right hand side not marked by & are matched for > equality, so you could write > > x matches (3 * 14 + big_complicated_expression, &y) It's really neat, but maybe too much visual clutter. It should be discussed if the following looks better: x matches (x, y) and x == 3 * 14 + big_complicated_expression I'm unsure about it. > which is pretty ugly. And that suggests: > > x matches (f(&y), g(&z)) It can only work with the few functions that are bijective. Many algorithms destroy information and can therefore not be run backwards. > which either doesn't make sense or is messy. We could add a __matches__ > attribute to functions so they could support this, but now this is getting > pretty complicated for unclear benefit. Eike From steve at pearwood.info Sun Dec 19 02:03:43 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 19 Dec 2010 12:03:43 +1100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012181223.46790.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <4D0C165A.60609@canterbury.ac.nz> <201012181223.46790.eike.welk@gmx.net> Message-ID: <4D0D59EF.3070900@pearwood.info> Eike Welk wrote: > On Saturday 18.12.2010 03:03:06 Greg Ewing wrote: >> Eike Welk wrote: > >> > | {| a, b |} -> # Attribute existence and access >> >> is far too cryptic and does nothing to improve readability. > > List comprehensions are also cryptic if you see them for the first time. The > curly brackets with bars "{| |}" should symbolize that Python object are > glorified dicts. It is also quite close to Ocaml's syntax for records. Not really. A list comp looks like a cross between a list and a for-loop, which is exactly what a list comp is. [2*x for x in sequence] I believe that people -- at least some people -- could intuit the meaning of this from context. But even if they can't, it's a simple extension to existing Python syntax they should already know. And best of all, you can easily experiment on it by copying and pasting a list comp into the interactive interpreter and seeing what it does. Your proposed syntax {| |} could be related to sets, or dicts, or both. Or it could be related to the | operator. If you don't know which, it's hard to guess. It looks more like a set than a dict: {a, b} # set {a: value, b: value} # dict {|a, b|} # looks like a set with extra symbols and there's probably nothing you can do with it in isolation from a full pattern match block. > My secret agenda was however, to later introduce a new class and object syntax > for Python. I just didn't dare to propose a completely revised Python. But > here we go: :-) > > > Class Creation > -------------- > > Foo = class {| a, b |} > > This should be equivalent to: > > class Foo(object): > def __init__(self, a, b): > self.a = a > self.b = b What's so special about Foo that it needs special syntax just to make it easier? In any case, these days I'd suggest that's probably best written as a namedtuple: >>> from collections import namedtuple >>> Spam = namedtuple('Spam', 'a b c') >>> x = Spam(a=23, b=42, c=None) >>> x Spam(a=23, b=42, c=None) > Inheritance is not so important in the context of classes that have no > methods. It could be expressed with a new method ("inherits") of the > metaclass. Like this: > > Foo = class {| a, b |}.inherits(Bar, Baz) > > How one would create methods, and if it should be possible to create method at > all, needs to be discussed. I don't think it does :) I think you're falling into the trap of thinking that everything needs to be a one-liner. It doesn't. > Instance Creation > ----------------- > > foo = {| a=1, b=2 |}:Foo > > This should be equivalent to: > > foo = Foo(a=1, b=2) Why do you think you need two ways of spelling Foo(a=1, b=2)? What's wrong with the Python syntax for it? It seems to me that you want the obfuscation of OCAML's syntax with the slowness of Python, a strange choice... > In the usual case, it should not be necessary to specify the class name: > > foo = {| a=1, b=2 |} > > The run-time should search for a class with a matching "__init__" method. In Oh wow. Just ... wow. You want the Python virtual machine to do a search through N objects in K scopes, checking each one to see if it is a class, then checking the signature of the __init__ method, just to allow the caller to *implicitly* specify the class of an instance instead of explicitly. -1000 on that. > case of ambiguities an exception would be raised. You have to name object > attributes unambiguously for this to work. However it is more flexible than > Ocaml because you can disambiguate it by specifying the class name. > > These constructions are expression, and can therefore be nested. *cries* -- Steven From bruce at leapyear.org Sun Dec 19 10:21:42 2010 From: bruce at leapyear.org (Bruce Leban) Date: Sun, 19 Dec 2010 01:21:42 -0800 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012190140.55231.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <201012190140.55231.eike.welk@gmx.net> Message-ID: On Sat, Dec 18, 2010 at 4:40 PM, Eike Welk wrote: > I would love to see this implemented! Off course I like my syntax better. > :-) > > well, I have to agree with Terry that your syntax looks like chicken scratches. Terseness is not a synonym for elegant. In this proposal, I'm using a terse syntax in two places where I think a less terse syntax would be too verbose. For references, I used & which mirrors the familiar C reference syntax, and for extra arguments, I used * which mirrors the use of that in function parameter lists. Arguably, I should be using * when the result is a list and ** when the result is a dict, but that's a detail not worth obsessing about at this point, since this is pretty unlikely to get added to the language. :-) > On Saturday 18.12.2010 22:26:05 Bruce Leban wrote: > > Whether or not this is useful at all is a big question, but I think it's > at > > least more interesting. This isn't perfect. For example > > > > x matches Foo(a=&alpha, b=&beta) > > For this case I prose that the class "Foo" should implement a special > function > "__rinit__" that breaks an instance apart, into components that can be > matched. These components should be chosen so that they could be given to > "__init__" as its arguments. > > "__rinit__" should return the following objects: > * a tuple of objects, corresponding to the regular arguments of "__init__". > * a tuple of strings that contains the argument names. > * a dict of string:object that represents the keyword only attributes. > This doesn't work. Positional and keyword arguments are not mutually exclusive. Furthermore, it may be a significant amount of work to unconstruct an object just to find out if it has an attribute. And some classes aren't "unconstructable". > > A type check would be expressed like this: > > x matches Foo(&*_) That would just be x matches Foo(*) There's no need to bind the results to something. > could mean checking x.hasattr('a') but there's no guarantee that Foo(a=1, > > b=2) will produce that result. Maybe that's OK. > > > > Also, any values on the right hand side not marked by & are matched for > > equality, so you could write > > > > x matches (3 * 14 + big_complicated_expression, &y) > > It's really neat, but maybe too much visual clutter. It should be discussed > if > the following looks better: > > x matches (x, y) and x == 3 * 14 + big_complicated_expression > Um, this was a *bad* example, indicating how the feature can be misused. There's no need to "discuss" the alternative you give though. The matches expression could be combined just like any other expression. So you could write: x matches (&k, &y) and k == some_expression or k = some_expression x matches (k, &y) I'm unsure about it. > > > > which is pretty ugly. And that suggests: > > > > x matches (f(&y), g(&z)) > > It can only work with the few functions that are bijective. Many algorithms > destroy information and can therefore not be run backwards. > > Again, this was a *bad* example. > > --- Bruce Latest blog post: http://www.vroospeak.com/2010/11/enduring-joe-barton.html Learn about security: http://j.mp/gruyere-security -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Sun Dec 19 13:05:02 2010 From: denis.spir at gmail.com (spir) Date: Sun, 19 Dec 2010 13:05:02 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0D59EF.3070900@pearwood.info> References: <201012180021.37232.eike.welk@gmx.net> <4D0C165A.60609@canterbury.ac.nz> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> Message-ID: <20101219130502.73095979@o> On Sun, 19 Dec 2010 12:03:43 +1100 Steven D'Aprano wrote: > > List comprehensions are also cryptic if you see them for the first time. The > > curly brackets with bars "{| |}" should symbolize that Python object are > > glorified dicts. It is also quite close to Ocaml's syntax for records. > > Not really. A list comp looks like a cross between a list and a > for-loop, which is exactly what a list comp is. And actually python (composite) objects, conceptually, are closer to named tuples than to dicts; composite objects are _not_ collections. That they are based on dicts is, imo, an implementation detail. For this reason, I'm pleased their dict attr is weirdly called "__dict__". Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From eike.welk at gmx.net Sun Dec 19 19:52:28 2010 From: eike.welk at gmx.net (Eike Welk) Date: Sun, 19 Dec 2010 19:52:28 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0D59EF.3070900@pearwood.info> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> Message-ID: <201012191952.30430.eike.welk@gmx.net> On Sunday 19.12.2010 02:03:43 Steven D'Aprano wrote: > Eike Welk wrote: > Your proposed syntax {| |} could be related to sets, or dicts, or both. > Or it could be related to the | operator. If you don't know which, it's > hard to guess. It looks more like a set than a dict: > > {a, b} # set > {a: value, b: value} # dict > {|a, b|} # looks like a set with extra symbols > > and there's probably nothing you can do with it in isolation from a full > pattern match block. I like this argument. This might lead to a design decision for the pattern matching syntax: A pattern that matches an object should look closely like the code that creates that object. > > Class Creation > > -------------- > > > > Foo = class {| a, b |} > What's so special about Foo that it needs special syntax just to make it > easier? In any case, these days I'd suggest that's probably best written > > as a namedtuple: > >>> from collections import namedtuple > >>> Spam = namedtuple('Spam', 'a b c') > >>> x = Spam(a=23, b=42, c=None) > >>> x > > Spam(a=23, b=42, c=None) Yes you are right. namedtuple is rely a perfect substitute for for Ocaml's records. The only thing which it is missing, is optional type checking in the constructor. Not only as a consistency test, but also as a terse form of documentation. > > > Inheritance is not so important in the context of classes that have no > > methods. It could be expressed with a new method ("inherits") of the > > metaclass. Like this: > > > > Foo = class {| a, b |}.inherits(Bar, Baz) > > > > How one would create methods, and if it should be possible to create > > method at all, needs to be discussed. > > I don't think it does :) > > I think you're falling into the trap of thinking that everything needs > to be a one-liner. It doesn't. Yes, in a "real" program you would want to document the attributes, and soon the class definition would be no longer a one liner. On the other hand: If the code is short (and readable), you can get an overview much more easily. My positive attitude towards this syntax comes from the only weakness that Python IMHO has: You can't easily see which data attributes an instance has. This information is hidden in __init__, and sometimes elsewhere. I think a mechanism like slots should be the norm, and dynamism the exception. > > > Instance Creation > > ----------------- > > > > foo = {| a=1, b=2 |}:Foo > > > > This should be equivalent to: > > > > foo = Foo(a=1, b=2) > > Why do you think you need two ways of spelling Foo(a=1, b=2)? What's > wrong with the Python syntax for it? It seems to me that you want the > obfuscation of OCAML's syntax with the slowness of Python, a strange > choice... I would like to combine the nice aspects of Ocaml, with the unproblematic nature of Python. In Ocaml I like the syntax for records, the match expression, and some aspects of the file system. Most other things are IMHO rather horrible. Eike. From denis.spir at gmail.com Sun Dec 19 20:44:24 2010 From: denis.spir at gmail.com (spir) Date: Sun, 19 Dec 2010 20:44:24 +0100 Subject: [Python-ideas] object construction (was: Re: ML Style Pattern Matching for Python) In-Reply-To: <201012191952.30430.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> <201012191952.30430.eike.welk@gmx.net> Message-ID: <20101219204424.3630e62e@o> On Sun, 19 Dec 2010 19:52:28 +0100 Eike Welk wrote: > My positive attitude towards this syntax comes from the only weakness that > Python IMHO has: You can't easily see which data attributes an instance has. > This information is hidden in __init__, and sometimes elsewhere. Agreed. Rather commonly elsewhere, I guess. > I think a > mechanism like slots should be the norm, and dynamism the exception. I wish we could put in front place the set of intended data attributes, including ones w/o defaults and optional ones. Even better, instanciation with the same param names would automagically set those attributes. class Point(Object): x = 0 y = 0 d color def __init__(self, HLSColor=None): if color is not None: self.color = toRGB(color) self.d = self.x + self.y p = Point(x=1, y=2, color=HLS(0,0,50) assert (p.d == 3) (untested) I find those loads of "self.x=x" in constructors sooo stupid --I want the machine to do it for me. __init__ should only define the essential part of obj construction; while the final constructor would do some mechanical job in addition. Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From steve at pearwood.info Sun Dec 19 22:26:26 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 20 Dec 2010 08:26:26 +1100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <201012191952.30430.eike.welk@gmx.net> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> <201012191952.30430.eike.welk@gmx.net> Message-ID: <4D0E7882.1090206@pearwood.info> Eike Welk wrote: > My positive attitude towards this syntax comes from the only weakness that > Python IMHO has: You can't easily see which data attributes an instance has. What's wrong with dir(obj) and vars(obj)? >>> class Spam: ... x = 1 ... def __init__(self): ... self.y = 2 ... >>> obj = Spam() >>> vars(obj) {'y': 2} >>> dir(obj) ['__doc__', '__init__', '__module__', 'x', 'y'] Python has *awesome* self-inspection abilities -- there's very little you can't easily find out about an object, so much so that people sometimes complain that you can't really hide information from the caller in Python -- there are no truly private attributes, only private by convention. See also the inspect module. > This information is hidden in __init__, and sometimes elsewhere. I think a > mechanism like slots should be the norm, and dynamism the exception. Such a language would no longer be Python. Perhaps it will be a better language, perhaps a worse one, but it won't be Python. By the way, __slots__ is intended as a memory optimization, not as a mechanism for defeating Python's dynamic nature. -- Steven From fuzzyman at gmail.com Mon Dec 20 00:14:07 2010 From: fuzzyman at gmail.com (Michael) Date: Sun, 19 Dec 2010 23:14:07 +0000 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0E7882.1090206@pearwood.info> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> <201012191952.30430.eike.welk@gmx.net> <4D0E7882.1090206@pearwood.info> Message-ID: <971414FA-AD93-4BCF-A37D-9F179F9822FE@gmail.com> On 19 Dec 2010, at 21:26, Steven D'Aprano wrote: > Eike Welk wrote: > >> My positive attitude towards this syntax comes from the only weakness that Python IMHO has: You can't easily see which data attributes an instance has. > > What's wrong with dir(obj) and vars(obj)? > > > >>> class Spam: > ... x = 1 > ... def __init__(self): > ... self.y = 2 > ... > >>> obj = Spam() > >>> vars(obj) > {'y': 2} > >>> dir(obj) > ['__doc__', '__init__', '__module__', 'x', 'y'] > I think the issue that Elke is pointing out is that dir(Spam) knows nothing about y (whereas it would if you used __slots__). One answer is to have class member defaults for all instance members. Michael > Python has *awesome* self-inspection abilities -- there's very little you can't easily find out about an object, so much so that people sometimes complain that you can't really hide information from the caller in Python -- there are no truly private attributes, only private by convention. See also the inspect module. > > >> This information is hidden in __init__, and sometimes elsewhere. I think a mechanism like slots should be the norm, and dynamism the exception. > > Such a language would no longer be Python. Perhaps it will be a better language, perhaps a worse one, but it won't be Python. > > > By the way, __slots__ is intended as a memory optimization, not as a mechanism for defeating Python's dynamic nature. > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas From eike.welk at gmx.net Mon Dec 20 02:04:10 2010 From: eike.welk at gmx.net (Eike Welk) Date: Mon, 20 Dec 2010 02:04:10 +0100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0E7882.1090206@pearwood.info> References: <201012180021.37232.eike.welk@gmx.net> <201012191952.30430.eike.welk@gmx.net> <4D0E7882.1090206@pearwood.info> Message-ID: <201012200204.11302.eike.welk@gmx.net> On Sunday 19.12.2010 22:26:26 Steven D'Aprano wrote: > Eike Welk wrote: > > My positive attitude towards this syntax comes from the only weakness > > that Python IMHO has: You can't easily see which data attributes an > > instance has. > > What's wrong with dir(obj) and vars(obj)? I should have expressed it more clearly, I was referring to the source text. In the declaration of a class in C++ you can see the data attributes that its instances use. In a Python class you have to look at the "__init__" method, and possibly into other methods, to see which data attributes are used. It's much harder to get an overview over the data attributes in Python. Python's introspection features are off course great. Eike P.S.: In my last mail, I also wanted to express that "I like Ocaml's type system", not its "file system". From ncoghlan at gmail.com Mon Dec 20 04:18:52 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 20 Dec 2010 14:18:52 +1100 Subject: [Python-ideas] object construction (was: Re: ML Style Pattern Matching for Python) In-Reply-To: <20101219204424.3630e62e@o> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> <201012191952.30430.eike.welk@gmx.net> <20101219204424.3630e62e@o> Message-ID: On Mon, Dec 20, 2010 at 6:44 AM, spir wrote: > On Sun, 19 Dec 2010 19:52:28 +0100 > Eike Welk wrote: > >> My positive attitude towards this syntax comes from the only weakness that >> Python IMHO has: You can't easily see which data attributes an instance has. >> This information is hidden in __init__, and sometimes elsewhere. > > Agreed. > Rather commonly elsewhere, I guess. > >> I think a >> mechanism like slots should be the norm, and dynamism the exception. > > I wish we could put in front place the set of intended data attributes, including ones w/o defaults and optional ones. Even better, instanciation with the same param names would automagically set those attributes. These days, a nice solution to that problem is to define a named tuple and inherit from it (see the 3.2 version of urllib.parse for a number of examples). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From steve at pearwood.info Mon Dec 20 11:08:03 2010 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 20 Dec 2010 21:08:03 +1100 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <971414FA-AD93-4BCF-A37D-9F179F9822FE@gmail.com> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> <201012191952.30430.eike.welk@gmx.net> <4D0E7882.1090206@pearwood.info> <971414FA-AD93-4BCF-A37D-9F179F9822FE@gmail.com> Message-ID: <4D0F2B03.4050405@pearwood.info> Michael wrote: > > On 19 Dec 2010, at 21:26, Steven D'Aprano wrote: > >> Eike Welk wrote: >> >>> My positive attitude towards this syntax comes from the only weakness that Python IMHO has: You can't easily see which data attributes an instance has. >> What's wrong with dir(obj) and vars(obj)? [...] > I think the issue that Elke is pointing out is that dir(Spam) knows nothing about y (whereas it would if you used __slots__). Then he should have said -- he explicitly said: "You can't easily see which data attributes an INSTANCE has." [emphasis added] As a general rule, you can't expect the class to know what attributes an instance has. This is a deliberate design choice. If you want to find out what attributes an object has, you ask the object, not the object's parent class. Think of this as a variation on the Law of Demeter: if you want to know how many fleas a dog has, inspect the dog, not the dog's owner. > One answer is to have class member defaults for all instance members. Well, that's an answer, but I'm not sure what the question is. From time to time I create a class which includes default class attributes, which then get optionally overridden by instance attributes. But that's much rarer than the usual case, where the instance has the attributes and the class doesn't. I would find it very disturbing to see classes declare meaningless class attributes (probably set to None) just to make it easier to predict what attributes the instances will get. That's pretty close to this horror: def spam(): # Declare all local variables which will be used. a = None b = None c = None # Now use them. a = 1 b = function(a, 23, -2) c = another_func(b) return a+b+c Personally, I've never found this to be a problem in practice. If I want to find out what attributes an instance of a class will have, I instantiate the class and inspect the instance. Or I read the docs. Looking at the source code is also an option (if the source code is available). For most classes, any attributes will be set in the __init__ method. If you have to read the source to determine what attributes exist, there's not much difference between: class Spam: a = 1 b = 2 c = 3 and class Spam: def __init(self): self.a = 1 self.b = 2 self.c = 3 But normally I just care about methods, not data attributes, and for that dir(cls) is perfectly adequate. (There may be the odd object that includes instance methods, but they'll be rare.) -- Steven From fuzzyman at voidspace.org.uk Mon Dec 20 12:51:57 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 20 Dec 2010 11:51:57 +0000 Subject: [Python-ideas] ML Style Pattern Matching for Python In-Reply-To: <4D0F2B03.4050405@pearwood.info> References: <201012180021.37232.eike.welk@gmx.net> <201012181223.46790.eike.welk@gmx.net> <4D0D59EF.3070900@pearwood.info> <201012191952.30430.eike.welk@gmx.net> <4D0E7882.1090206@pearwood.info> <971414FA-AD93-4BCF-A37D-9F179F9822FE@gmail.com> <4D0F2B03.4050405@pearwood.info> Message-ID: On 20 December 2010 10:08, Steven D'Aprano wrote: > Michael wrote: > >> >> On 19 Dec 2010, at 21:26, Steven D'Aprano wrote: >> >> Eike Welk wrote: >>> >>> My positive attitude towards this syntax comes from the only weakness >>>> that Python IMHO has: You can't easily see which data attributes an instance >>>> has. >>>> >>> What's wrong with dir(obj) and vars(obj)? >>> >> [...] > > I think the issue that Elke is pointing out is that dir(Spam) knows >> nothing about y (whereas it would if you used __slots__). >> > > Then he should have said -- he explicitly said: > > "You can't easily see which data attributes an INSTANCE has." > [emphasis added] > Well sure - just by looking at the class you can't tell what members an INSTANCE has... :-) > > As a general rule, you can't expect the class to know what attributes an > instance has. This is a deliberate design choice. If you want to find out > what attributes an object has, you ask the object, not the object's parent > class. > > Think of this as a variation on the Law of Demeter: if you want to know how > many fleas a dog has, inspect the dog, not the dog's owner. > > > > One answer is to have class member defaults for all instance members. >> > > Well, that's an answer, but I'm not sure what the question is. From time to > time I create a class which includes default class attributes, which then > get optionally overridden by instance attributes. But that's much rarer than > the usual case, where the instance has the attributes and the class doesn't. > > I would find it very disturbing to see classes declare meaningless class > attributes (probably set to None) just to make it easier to predict what > attributes the instances will get. That's pretty close to this horror: > [snip...] > > Personally, I've never found this to be a problem in practice. The only place I've found it relevant is when mocking out objects that you don't want instantiated in your test. mock.Mock (and mock.patch) can create mock objects that behave like the original (in terms of available methods and members) if you pass it a reference to the object it is mocking. If member creation is 'hidden' inside __init__ then Mock can't know what members are available. > If I want to find out what attributes an instance of a class will have, I > instantiate the class and inspect the instance. Or I read the docs. Looking > at the source code is also an option (if the source code is available). For > most classes, any attributes will be set in the __init__ method. If you have > to read the source to determine what attributes exist, there's not much > difference between: > Well, if there's anything you *can't* tell about the behaviour of objects by reading the source code then something odd is going on. ;-) Programmatic inspection is where it is relevant (to me at least). Michael > > class Spam: > a = 1 > b = 2 > c = 3 > > and > > class Spam: > def __init(self): > self.a = 1 > self.b = 2 > self.c = 3 > > > But normally I just care about methods, not data attributes, and for that > dir(cls) is perfectly adequate. (There may be the odd object that includes > instance methods, but they'll be rare.) > > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.spir at gmail.com Wed Dec 22 10:57:07 2010 From: denis.spir at gmail.com (spir) Date: Wed, 22 Dec 2010 10:57:07 +0100 Subject: [Python-ideas] for info: pattern matching in a C-like language Message-ID: <20101222105707.7325c492@o> Hello, Just found this http://dvanderboom.wordpress.com/2010/05/08/the-archetype-language-part-4/ (goto "Pattern Matching") which looks very much like Eike's recent proposal to the python-ideas mailing list. Denis -- -- -- -- -- -- -- vit esse estrany ? spir.wikidot.com From fuzzyman at voidspace.org.uk Wed Dec 29 14:36:53 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 13:36:53 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter Message-ID: Hello all, I often use python as a calculator or for simple operations using -c. It would be enormously useful if python -c "..." would output on stdout the result of the last evaluated expression in the same way that the interactive interpreter does. The following outputs nothing: python -c "12 / 4.1" So you always have to prefix expressions with print to see the result. Michael -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Dec 29 15:40:18 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 29 Dec 2010 14:40:18 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 13:36, Michael Foord wrote: > Hello all, > > I often use python as a calculator or for simple operations using -c. It > would be enormously useful if python -c "..." would output on stdout the > result of the last evaluated expression in the same way that the interactive > interpreter does. > > The following outputs nothing: > > ??? python -c "12 / 4.1" > > So you always have to prefix expressions with print to see the result. I like the idea, but that's a fairly big semantic change. What about adding an -e option that takes an expression, and prints its value? So you'd have python -e "12 / 4.1" (AFAICT, -e is unused at present). Paul. From fuzzyman at voidspace.org.uk Wed Dec 29 15:46:07 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 14:46:07 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 14:40, Paul Moore wrote: > On 29 December 2010 13:36, Michael Foord > wrote: > > Hello all, > > > > I often use python as a calculator or for simple operations using -c. It > > would be enormously useful if python -c "..." would output on stdout the > > result of the last evaluated expression in the same way that the > interactive > > interpreter does. > > > > The following outputs nothing: > > > > python -c "12 / 4.1" > > > > So you always have to prefix expressions with print to see the result. > > I like the idea, but that's a fairly big semantic change. What about > adding an -e option that takes an expression, and prints its value? So > you'd have > > python -e "12 / 4.1" > > (AFAICT, -e is unused at present). > That would be great. I did worry that changing the output would be backwards incompatible with code that shells out to Python using "-c", so a different command line option would be great. So long as it works with multiple statements (semi-colon separated) like the current "-c" behaviour. I use this trick for finding the source code of Python modules: $ python -c "import os;print os.__file__[:-1]" /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/os.py Often in the form: $ mate `python -c "import module;print module.__file__[:-1]"` All the best, Michael Foord > > Paul. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Dec 29 15:55:54 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 29 Dec 2010 15:55:54 +0100 Subject: [Python-ideas] python -c "..." should output result like the interpreter References: Message-ID: <20101229155554.041a00cd@pitrou.net> On Wed, 29 Dec 2010 13:36:53 +0000 Michael Foord wrote: > Hello all, > > I often use python as a calculator or for simple operations using -c. It > would be enormously useful if python -c "..." would output on stdout the > result of the last evaluated expression in the same way that the interactive > interpreter does. > > The following outputs nothing: > > python -c "12 / 4.1" > > So you always have to prefix expressions with print to see the result. Is sparing the need to type print() really worthwhile here? I often leave a Python interpreter prompt open in a terminal so that I can try out whatever I need there, so I don't have to type "python -c", I can reuse previous results and imports, etc. Unless you are severaly RAM-limited (or OS X has poor multiwindowing capabilities ;-)), this should probably suit you as well. Regards Antoine. From fuzzyman at voidspace.org.uk Wed Dec 29 16:00:56 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 15:00:56 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: <20101229155554.041a00cd@pitrou.net> References: <20101229155554.041a00cd@pitrou.net> Message-ID: On 29 December 2010 14:55, Antoine Pitrou wrote: > On Wed, 29 Dec 2010 13:36:53 +0000 > Michael Foord wrote: > > Hello all, > > > > I often use python as a calculator or for simple operations using -c. It > > would be enormously useful if python -c "..." would output on stdout the > > result of the last evaluated expression in the same way that the > interactive > > interpreter does. > > > > The following outputs nothing: > > > > python -c "12 / 4.1" > > > > So you always have to prefix expressions with print to see the result. > > Is sparing the need to type print() really worthwhile here? > Well, what usually happens is that I forget I have to print and do it without first. Then when there is no output I remember and redo with the print in place. Making the print unnecessary would therefore mean substantially less than half as much typing.... Michael > I often leave a Python interpreter prompt open in a terminal so that I > can try out whatever I need there, so I don't have to type "python -c", > I can reuse previous results and imports, etc. > Unless you are severaly RAM-limited (or OS X has poor multiwindowing > capabilities ;-)), this should probably suit you as well. > > Regards > > Antoine. > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Dec 29 16:09:12 2010 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 29 Dec 2010 15:09:12 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 14:46, Michael Foord wrote: > That would be great. I did worry that changing the output would be backwards > incompatible with code that shells out to Python using "-c", so a different > command line option would be great. So long as it works with multiple > statements (semi-colon separated) like the current "-c" behaviour. > > I use this trick for finding the source code of Python modules: > > $ python -c "import os;print os.__file__[:-1]" > /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/os.py I'd suggest allowing multiple -c and -e options, interspersed as needed, with each being treated as a separate line, so you could do $ python -c "import os" -e "os.__file__[:-1]" (Actually, I'd find "python -c stmt1 -c stmt2" mildly more readable than "python -c "stmt1; stmt2"" so that's another (equally small) benefit to me...) Paul. PS I take Antoine's point that it's a small benefit, but like you, I find myself needing it quite a lot in practice :-) From fuzzyman at voidspace.org.uk Wed Dec 29 16:13:24 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 15:13:24 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 15:09, Paul Moore wrote: > On 29 December 2010 14:46, Michael Foord > wrote: > > That would be great. I did worry that changing the output would be > backwards > > incompatible with code that shells out to Python using "-c", so a > different > > command line option would be great. So long as it works with multiple > > statements (semi-colon separated) like the current "-c" behaviour. > > > > I use this trick for finding the source code of Python modules: > > > > $ python -c "import os;print os.__file__[:-1]" > > /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/os.py > > I'd suggest allowing multiple -c and -e options, interspersed as > needed, with each being treated as a separate line, so you could do > > $ python -c "import os" -e "os.__file__[:-1]" > > (Actually, I'd find "python -c stmt1 -c stmt2" mildly more readable > than "python -c "stmt1; stmt2"" so that's another (equally small) > benefit to me...) > Well, I'd like both. :-) > > Paul. > > PS I take Antoine's point that it's a small benefit, but like you, I > find myself needing it quite a lot in practice :-) > Ditto. Michael Foord -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Wed Dec 29 16:13:15 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Dec 2010 16:13:15 +0100 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: Am 29.12.2010 14:36, schrieb Michael Foord: > Hello all, > > I often use python as a calculator or for simple operations using -c. It would > be enormously useful if python -c "..." would output on stdout the result of the > last evaluated expression in the same way that the interactive interpreter does. If it did, you couldn't do this anymore (with the obvious implementation): python -c 'print 1 print 2' which I guess a few people do rely upon. A new -e option would be fine, but -- YAGNI? Georg From g.brandl at gmx.net Wed Dec 29 16:18:52 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Dec 2010 16:18:52 +0100 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: Am 29.12.2010 15:46, schrieb Michael Foord: > I like the idea, but that's a fairly big semantic change. What about > adding an -e option that takes an expression, and prints its value? So > you'd have > > python -e "12 / 4.1" > > (AFAICT, -e is unused at present). > > That would be great. I did worry that changing the output would be backwards > incompatible with code that shells out to Python using "-c", so a different > command line option would be great. So long as it works with multiple statements > (semi-colon separated) like the current "-c" behaviour. Hey, what about this little module: import sys for x in sys.argv[1:]: exec compile(x, '', 'single') Then: $ python -me '1+1; 2+2' 2 4 Georg From solipsis at pitrou.net Wed Dec 29 16:20:49 2010 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 29 Dec 2010 16:20:49 +0100 Subject: [Python-ideas] python -c "..." should output result like the interpreter References: Message-ID: <20101229162049.3278ee48@pitrou.net> On Wed, 29 Dec 2010 16:13:15 +0100 Georg Brandl wrote: > Am 29.12.2010 14:36, schrieb Michael Foord: > > Hello all, > > > > I often use python as a calculator or for simple operations using -c. It would > > be enormously useful if python -c "..." would output on stdout the result of the > > last evaluated expression in the same way that the interactive interpreter does. > > If it did, you couldn't do this anymore (with the obvious implementation): > > python -c 'print 1 > print 2' > > which I guess a few people do rely upon. > > A new -e option would be fine, but -- YAGNI? Apparently, MIGNI. From fuzzyman at voidspace.org.uk Wed Dec 29 16:22:06 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 15:22:06 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: <20101229162049.3278ee48@pitrou.net> References: <20101229162049.3278ee48@pitrou.net> Message-ID: On 29 December 2010 15:20, Antoine Pitrou wrote: > On Wed, 29 Dec 2010 16:13:15 +0100 > Georg Brandl wrote: > > Am 29.12.2010 14:36, schrieb Michael Foord: > > > Hello all, > > > > > > I often use python as a calculator or for simple operations using -c. > It would > > > be enormously useful if python -c "..." would output on stdout the > result of the > > > last evaluated expression in the same way that the interactive > interpreter does. > > > > If it did, you couldn't do this anymore (with the obvious > implementation): > > > > python -c 'print 1 > > print 2' > > > > which I guess a few people do rely upon. > > > > A new -e option would be fine, but -- YAGNI? > > Apparently, MIGNI. > > Hah. :-) And PIGNI. Michael > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Wed Dec 29 16:22:57 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 15:22:57 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 15:18, Georg Brandl wrote: > Am 29.12.2010 15:46, schrieb Michael Foord: > > > I like the idea, but that's a fairly big semantic change. What about > > adding an -e option that takes an expression, and prints its value? > So > > you'd have > > > > python -e "12 / 4.1" > > > > (AFAICT, -e is unused at present). > > > > That would be great. I did worry that changing the output would be > backwards > > incompatible with code that shells out to Python using "-c", so a > different > > command line option would be great. So long as it works with multiple > statements > > (semi-colon separated) like the current "-c" behaviour. > > Hey, what about this little module: > > import sys > for x in sys.argv[1:]: > exec compile(x, '', 'single') > > Then: > > $ python -me '1+1; 2+2' > 2 > 4 > I *really* like that. :-) Added it to my standard Python installs. Thanks Michael > > Georg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Wed Dec 29 16:20:05 2010 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 29 Dec 2010 16:20:05 +0100 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: Am 29.12.2010 16:13, schrieb Michael Foord: > I'd suggest allowing multiple -c and -e options, interspersed as > needed, with each being treated as a separate line, so you could do > > $ python -c "import os" -e "os.__file__[:-1]" > > (Actually, I'd find "python -c stmt1 -c stmt2" mildly more readable > than "python -c "stmt1; stmt2"" so that's another (equally small) > benefit to me...) > > > Well, I'd like both. :-) That's another incompatible change though: currently, sys.argv is filled with everything after the argument of -c. Georg From fdrake at acm.org Wed Dec 29 16:27:59 2010 From: fdrake at acm.org (Fred Drake) Date: Wed, 29 Dec 2010 10:27:59 -0500 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On Wed, Dec 29, 2010 at 9:40 AM, Paul Moore wrote: > I like the idea, but that's a fairly big semantic change. What about > adding an -e option that takes an expression, and prints its value? +1 Changing -c would be unacceptable. ? -Fred -- Fred L. Drake, Jr.? ? "A storm broke loose in my mind."? --Albert Einstein From fuzzyman at voidspace.org.uk Wed Dec 29 16:45:59 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 15:45:59 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 15:18, Georg Brandl wrote: > Am 29.12.2010 15:46, schrieb Michael Foord: > > > I like the idea, but that's a fairly big semantic change. What about > > adding an -e option that takes an expression, and prints its value? > So > > you'd have > > > > python -e "12 / 4.1" > > > > (AFAICT, -e is unused at present). > > > > That would be great. I did worry that changing the output would be > backwards > > incompatible with code that shells out to Python using "-c", so a > different > > command line option would be great. So long as it works with multiple > statements > > (semi-colon separated) like the current "-c" behaviour. > > Hey, what about this little module: > > import sys > for x in sys.argv[1:]: > exec compile(x, '', 'single') > > Then: > > $ python -me '1+1; 2+2' > 2 > 4 > So now you can `pip install e` and then `python -me`... Michael > > Georg > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Wed Dec 29 17:35:27 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 16:35:27 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 15:45, Michael Foord wrote: > > > On 29 December 2010 15:18, Georg Brandl wrote: > >> Am 29.12.2010 15:46, schrieb Michael Foord: >> >> > I like the idea, but that's a fairly big semantic change. What about >> > adding an -e option that takes an expression, and prints its value? >> So >> > you'd have >> > >> > python -e "12 / 4.1" >> > >> > (AFAICT, -e is unused at present). >> > >> > That would be great. I did worry that changing the output would be >> backwards >> > incompatible with code that shells out to Python using "-c", so a >> different >> > command line option would be great. So long as it works with multiple >> statements >> > (semi-colon separated) like the current "-c" behaviour. >> >> Hey, what about this little module: >> >> import sys >> for x in sys.argv[1:]: >> exec compile(x, '', 'single') >> >> Then: >> >> $ python -me '1+1; 2+2' >> 2 >> 4 >> > > So now you can `pip install e` and then `python -me`... > http://pypi.python.org/pypi/e As an added bonus if the first argument is a module name it will print out the location of the module on the filesystem. Michael > > Michael > > >> >> Georg >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> http://mail.python.org/mailman/listinfo/python-ideas >> > > > > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > May you find forgiveness for yourself and forgive others > > May you share freely, never taking more than you give. > -- the sqlite blessing http://www.sqlite.org/different.html > > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.belopolsky at gmail.com Wed Dec 29 17:35:46 2010 From: alexander.belopolsky at gmail.com (Alexander Belopolsky) Date: Wed, 29 Dec 2010 11:35:46 -0500 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On Wed, Dec 29, 2010 at 9:40 AM, Paul Moore wrote: .. > I like the idea, but that's a fairly big semantic change. What about > adding an -e option that takes an expression, and prints its value? So > you'd have > > ? ?python -e "12 / 4.1" > > (AFAICT, -e is unused at present). > +1 For some reason I always type -e (for "eval" or "exec"?) before realizing that it should have been -c. What do you think ?python -e "for i in range(5): i" should print? Note that on >>> prompt: >>> for i in range(5): i ... 0 1 2 3 4 From fuzzyman at voidspace.org.uk Wed Dec 29 17:37:20 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 29 Dec 2010 16:37:20 +0000 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: On 29 December 2010 16:35, Alexander Belopolsky < alexander.belopolsky at gmail.com> wrote: > On Wed, Dec 29, 2010 at 9:40 AM, Paul Moore wrote: > .. > > I like the idea, but that's a fairly big semantic change. What about > > adding an -e option that takes an expression, and prints its value? So > > you'd have > > > > python -e "12 / 4.1" > > > > (AFAICT, -e is unused at present). > > > > +1 > > For some reason I always type -e (for "eval" or "exec"?) before > realizing that it should have been -c. > > What do you think python -e "for i in range(5): i" should print? > Note that on >>> prompt: > > >>> for i in range(5): i > ... > 0 > 1 > 2 > 3 > 4 > I'd be happy with the same behaviour as the interactive interpreter. Michael > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Dec 29 22:52:48 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 30 Dec 2010 07:52:48 +1000 Subject: [Python-ideas] Module aliases and/or "real names" Message-ID: Disclaimer: this is a currently half-baked idea that needs some discussion here if it is going to turn into something a bit more coherent :) On and off, I've been pondering the problem of the way implementation details (like the real file structures of the multiprocessing and unittest packages, or whether or not an interpreter use the pure Python or the C accelerated version of various modules) leak out into the world via the __module__ attribute on various components. This mostly comes up when discussing pickle compatibility between 2.x and 3.x, but in can show up in various guises whenever you start relying on dynamic introspection. As, I see it, there are 3 basic ways of dealing with the problem: 1. Allow objects to lie about their source module This is likely a terrible idea, since a function's global namespace reference would disagree with its module reference. I suspect much weirdness would result. 2. A pickle-specific module alias registry, since that is where the problem comes up most often A possible approach, but not necessarily a good one (since it isn't really a pickle-specific problem). 3. An inspect-based module alias registry That is, an additional query API (get_canonical_module_name?) in the inspect module that translates from the implementation detail module name to the "preferred" module name. The implementation could be as simple as a "__canonical__" attribute in the module namespace. I actually quite like option 3, with various things (such as pydoc) updated to show *both* names when they're different. That way people will know where to find official documentation for objects from pseudo-packages and acceleration modules (i.e. under the canonical name), without hiding where the actual implementation came from. Pickle *generation* could then be updated to only send canonical module names during normal operation, reducing the exposure of implementation details like pseudo-packages and acceleration modules. Whether or not runpy should set __canonical__ on the main module would be an open question (probably not, *unless* runpy was also updated to add the main module to sys.modules under its real name as well __main__). Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From rrr at ronadam.com Thu Dec 30 02:48:54 2010 From: rrr at ronadam.com (Ron Adam) Date: Wed, 29 Dec 2010 19:48:54 -0600 Subject: [Python-ideas] Module aliases and/or "real names" In-Reply-To: References: Message-ID: <4D1BE506.2070806@ronadam.com> On 12/29/2010 03:52 PM, Nick Coghlan wrote: > Disclaimer: this is a currently half-baked idea that needs some > discussion here if it is going to turn into something a bit more > coherent :) Sometimes half baked is good as it gets at the concept, rather than being bogged down with the details. ;-) > On and off, I've been pondering the problem of the way implementation > details (like the real file structures of the multiprocessing and > unittest packages, or whether or not an interpreter use the pure > Python or the C accelerated version of various modules) leak out into > the world via the __module__ attribute on various components. This > mostly comes up when discussing pickle compatibility between 2.x and > 3.x, but in can show up in various guises whenever you start relying > on dynamic introspection. This sounds like two different separate issues to me. One is the leaking-out of lower level details. The other is abstracting a framework with the minimal amount of details needed. Ron From cs at zip.com.au Thu Dec 30 06:53:32 2010 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 30 Dec 2010 16:53:32 +1100 Subject: [Python-ideas] python -c "..." should output result like the interpreter In-Reply-To: References: Message-ID: <20101230055332.GA31585@cskk.homeip.net> On 29Dec2010 10:27, Fred Drake wrote: | On Wed, Dec 29, 2010 at 9:40 AM, Paul Moore wrote: | > I like the idea, but that's a fairly big semantic change. What about | > adding an -e option that takes an expression, and prints its value? | | +1 | | Changing -c would be unacceptable. Likewise. My -1 on having -c output the final value is as for the shell: does "sh -c sh-cmd" have an implied echo? No, and lucky we are that it does not. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. - Brian W. Kernighan From ncoghlan at gmail.com Fri Dec 31 03:52:47 2010 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 31 Dec 2010 12:52:47 +1000 Subject: [Python-ideas] Module aliases and/or "real names" In-Reply-To: <4D1BE506.2070806@ronadam.com> References: <4D1BE506.2070806@ronadam.com> Message-ID: On Thu, Dec 30, 2010 at 11:48 AM, Ron Adam wrote: > This sounds like two different separate issues to me. > > One is the leaking-out of lower level details. > > The other is abstracting a framework with the minimal amount of details > needed. Yeah, sort of. Really, the core issue is that some objects live in two places: - where they came from right now, in the current interpreter - where they should be retrieved from "officially" (e.g. since another interpreter may not provide an accelerated version, or because the appropriate submodule may be selected at runtime based on the current platform) There's currently no systematic way of flagging objects or modules where the latter location differs from the former, so the components that leak the low level details (such as pickling and pydoc) have no way to avoid it. Once a system is in place to identify such objects (or perhaps just the affected modules), then the places that leak that information can be updated to deal with the situation appropriately (e.g. pickling would likely just use the official names, while pydoc would display both, indicating which one was the 'official' location, and which one reflected the current interpreter behaviour). So it's really one core problem (non-portable module details), which then leads to an assortment of smaller problems when other parts of the standard library are forced to rely on those non-portable details because that's the only information available. Cheers, Nick. -- Nick Coghlan?? |?? ncoghlan at gmail.com?? |?? Brisbane, Australia From guido at python.org Fri Dec 31 19:49:57 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 31 Dec 2010 10:49:57 -0800 Subject: [Python-ideas] Optimizing builtins Message-ID: [Changed subject *and* list] > 2010/12/31 Maciej Fijalkowski >> How do you know that range is a builtin you're thinking >> about and not some other object? On Fri, Dec 31, 2010 at 7:02 AM, Cesare Di Mauro wrote: > By a special opcode which could do this work. ]:-) That can't be the answer, because then the question would become "how does the compiler know it can use the special opcode". This particular issue (generating special opcodes for certain builtins) has actually been discussed many times before. Alas, given Python's extremely dynamic promises it is very hard to do it in a way that is *guaranteed* not to change the semantics. For example, I could have replaced builtins['range'] with something else; or I could have inserted a variable named 'range' into the module's __dict__. (Note that I am not talking about just creating a global variable named 'range' in the module; those the compiler could recognize. I am talking about interceptions that a compiler cannot see, assuming it compiles each module independently, i.e. without whole-program optimizations.) Now, *in practice* such manipulations are rare (with the possible exception of people replacing open() with something providing hooks for e.g. a virtual filesystem) and there is probably some benefit to be had. (I expect that the biggest benefit might well be from replacing len() with an opcode.) I have in the past proposed to change the official semantics of the language subtly to allow such optimizations (i.e. recognizing builtins and replacing them with dedicated opcodes). There should also be a simple way to disable them, e.g. by setting "len = len" at the top of a module, one would be signalling that len() is not to be replaced by an opcode. But it remains messy and nobody has really gotten very far with implementing this. It is certainly not "low-hanging fruit" to do it properly. I should also refer people interested in this subject to at least three PEPs that were written about this topic: PEP 266, PEP 267 and PEP 280. All three have been deferred, since nobody was bold enough to implement at least one of them well enough to be able to tell if it was even worth the trouble. I haven't read either of those in a long time, and they may well be outdated by current JIT technology. I just want to warn folks that it's not such a simple matter to replace "for i in range(....):" with a special opcode. (FWIW, optimizing "x[i] = i" would be much simpler -- I don't really care about the argument that a debugger might interfere. But again, apart from the simplest cases, it requires a sophisticated parser to determine that it is really safe to do so.) -- --Guido van Rossum (python.org/~guido) From fuzzyman at voidspace.org.uk Fri Dec 31 20:59:02 2010 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 31 Dec 2010 19:59:02 +0000 Subject: [Python-ideas] Optimizing builtins In-Reply-To: References: Message-ID: On 31 December 2010 18:49, Guido van Rossum wrote: > [Changed subject *and* list] > > > 2010/12/31 Maciej Fijalkowski > >> How do you know that range is a builtin you're thinking > >> about and not some other object? > > On Fri, Dec 31, 2010 at 7:02 AM, Cesare Di Mauro > wrote: > > By a special opcode which could do this work. ]:-) > > That can't be the answer, because then the question would become "how > does the compiler know it can use the special opcode". This particular > issue (generating special opcodes for certain builtins) has actually > been discussed many times before. Alas, given Python's extremely > dynamic promises it is very hard to do it in a way that is > *guaranteed* not to change the semantics. For example, I could have > replaced builtins['range'] with something else; or I could have > inserted a variable named 'range' into the module's __dict__. (Note > that I am not talking about just creating a global variable named > 'range' in the module; those the compiler could recognize. I am > talking about interceptions that a compiler cannot see, assuming it > compiles each module independently, i.e. without whole-program > optimizations.) > > Now, *in practice* such manipulations are rare Actually range is the one I've seen *most* overridden, not in order to replace functionality but because range is such a useful (or relevant) variable name in all sorts of circumstances... Michael > (with the possible > exception of people replacing open() with something providing hooks > for e.g. a virtual filesystem) and there is probably some benefit to > be had. (I expect that the biggest benefit might well be from > replacing len() with an opcode.) I have in the past proposed to change > the official semantics of the language subtly to allow such > optimizations (i.e. recognizing builtins and replacing them with > dedicated opcodes). There should also be a simple way to disable them, > e.g. by setting "len = len" at the top of a module, one would be > signalling that len() is not to be replaced by an opcode. But it > remains messy and nobody has really gotten very far with implementing > this. It is certainly not "low-hanging fruit" to do it properly. > > I should also refer people interested in this subject to at least > three PEPs that were written about this topic: PEP 266, PEP 267 and > PEP 280. All three have been deferred, since nobody was bold enough to > implement at least one of them well enough to be able to tell if it > was even worth the trouble. I haven't read either of those in a long > time, and they may well be outdated by current JIT technology. I just > want to warn folks that it's not such a simple matter to replace "for > i in range(....):" with a special opcode. > > (FWIW, optimizing "x[i] = i" would be much simpler -- I don't really > care about the argument that a debugger might interfere. But again, > apart from the simplest cases, it requires a sophisticated parser to > determine that it is really safe to do so.) > > -- > --Guido van Rossum (python.org/~guido ) > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Fri Dec 31 22:51:41 2010 From: guido at python.org (Guido van Rossum) Date: Fri, 31 Dec 2010 13:51:41 -0800 Subject: [Python-ideas] Optimizing builtins In-Reply-To: References: Message-ID: On Fri, Dec 31, 2010 at 11:59 AM, Michael Foord wrote: > > > On 31 December 2010 18:49, Guido van Rossum wrote: >> >> [Changed subject *and* list] >> >> > 2010/12/31 Maciej Fijalkowski >> >> How do you know that range is a builtin you're thinking >> >> about and not some other object? >> >> On Fri, Dec 31, 2010 at 7:02 AM, Cesare Di Mauro >> wrote: >> > By a special opcode which could do this work. ]:-) >> >> That can't be the answer, because then the question would become "how >> does the compiler know it can use the special opcode". This particular >> issue (generating special opcodes for certain builtins) has actually >> been discussed many times before. Alas, given Python's extremely >> dynamic promises it is very hard to do it in a way that is >> *guaranteed* not to change the semantics. For example, I could have >> replaced builtins['range'] with something else; or I could have >> inserted a variable named 'range' into the module's __dict__. (Note >> that I am not talking about just creating a global variable named >> 'range' in the module; those the compiler could recognize. I am >> talking about interceptions that a compiler cannot see, assuming it >> compiles each module independently, i.e. without whole-program >> optimizations.) >> >> Now, *in practice* such manipulations are rare > > Actually range is the one I've seen *most* overridden, not in order to > replace functionality but because range is such a useful (or relevant) > variable name in all sorts of circumstances... No, you're misunderstanding. I was not referring to the overriding a name using Python's regular syntax for defining names. If you set a (global or local) variable named 'range', the compiler is perfectly capable of noticing. E.g.: range = 42 def foo(): for i in range(10): print(i) While this will of course fail with a TypeError if you try to execute it, a (hypothetical) optimizing compiler would have no trouble noticing that the 'range' in the for-loop must refer to the global variable of that name, not to the builtin of the same name. I was referring to an innocent module containing a use of the builtin range function, e.g. # a.py def f(): for i in range(10): print(i) which is imported by another module which manipulates a's globals, for example: # b.py import a a.range = 42 a.f() The compiler has no way to notice this when a.py is being compiled. Variants of "hiding" a mutation like this include: a.__dict__['range'] = 42 or import builtins builtins.range = 42 and of course for more fun you can make it more dynamic (think obfuscated code contests). -- --Guido van Rossum (python.org/~guido)