From rosuav at gmail.com Tue Jul 1 00:05:19 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Jul 2014 08:05:19 +1000 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> Message-ID: On Tue, Jul 1, 2014 at 3:18 AM, wrote: > On Sat, Jun 28, 2014, at 01:28, Chris Angelico wrote: >> empty_set_literal = >> type(lambda:0)(type((lambda:0).__code__)(0,0,0,3,67,b't\x00\x00d\x01\x00h\x00\x00\x83\x02\x00\x01d\x00\x00S',(None,"I'm > > If you're embedding the entire compiler (in fact, a modified one) in > your tool, why not just output a .pyc? I'm not, I'm calling on the normal compiler. Also, I'm not familiar with the pyc format, nor with any of the potential pit-falls of that approach. But if someone wants to make an "alternative front end that makes a .pyc file" kind of thing, they're most welcome to. ChrisA From abarnert at yahoo.com Tue Jul 1 01:48:14 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 30 Jun 2014 16:48:14 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> Message-ID: <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> First, two quick side notes: It might be nice if the compiler were as easy to hook as the importer.?Alternatively, it might be nice if there were a way to do "inline bytecode assembly" in CPython, similar to the way you do inline assembly in many C compilers, so the answer to random's question is just "asm [('BUILD_SET', 0)]" or something similar. Either of those would make this problem trivial. I doubt either of those would be useful often enough that anyone wants to put in the work.?But then I doubt the empty-set literal would be either, so anyone who seriously wants to work on this might want to work on the inline assembly and/or hookable compiler first. Anyway: On Monday, June 30, 2014 3:12 PM, Chris Angelico wrote: >On Tue, Jul 1, 2014 at 3:18 AM,? wrote: >> On Sat, Jun 28, 2014, at 01:28, Chris Angelico wrote: >>> empty_set_literal = >>> type(lambda:0)(type((lambda:0).__code__)(0,0,0,3,67,b't\x00\x00d\x01\x00h\x00\x00\x83\x02\x00\x01d\x00\x00S',(None,"I'm I think it makes more sense to use types.FunctionType and types.CodeType here than to generate two extra functions for each function, even if that means you have to put an import types at the top of every munged source file. >> If you're embedding the entire compiler (in fact, a modified one) in >> your tool, why not just output a .pyc? > >I'm not, I'm calling on the normal compiler. Also, I'm not familiar >with the pyc format, nor with any of the potential pit-falls of that >approach. But if someone wants to make an "alternative front end that >makes a .pyc file" kind of thing, they're most welcome to. The tricky bit with making a .pyc file is generating the header information?last I checked (quite a while ago, and not that deeply?) that wasn't documented, and there were no helpers exported to Python. But I think what he was suggesting is something like this: Let?py_compile.compile generate the .pyc file as normal, then munge the bytecode in that file, instead of compiling each function, munging its bytecode, and emitting source that creates the munged functions. Besides being a lot less work, his version works for ? at top level, in class definitions, in lambda expressions, etc., not just for def statements. And it doesn't require finding and identifying all of the things to munge in a source file (which I assume you'd do bottom-up based on the ast.parse tree or something). But either way, this still doesn't solve the big problem. Compiling a function by hand and then tweaking the bytecode is easy; doing it programmatically is more painful.?You obviously need the function to compile, so you have to replace the ? with something else whose bytecode you can search-and-replace. But what? That something else has to be valid in an expression context (so it compiles), has to compile to a 3-byte opcode (otherwise, replacing it will screw up any jump targets that point after it), can't add any globals/constants/etc. to the list (otherwise, removing it will screw up any LOAD_FOO statements that refer to a higher-numbered foo), and can't appear anywhere in the code being compiled. The only thing I can think of off the top of my head is to replace it with whichever of [], (), or {} doesn't appear anywhere in the code being compiled, then you can search-replace BUILD_LIST/TUPLE/MAP 0 with BUILD_SET 0. But what if all three appear in the code? Raise a SyntaxError('Cannot use all 4 kinds of empty literals in the same scope')? One more thing that I'm sure you thought of, but may not have thought through all the way: To make this generally useful, you can't just hardcode creating a zero-arg top-level function; you need to copy all of the code and function constructor arguments from the compiled function.? So, if the function is a closure, how do you do that? You need to pass a list of closure cell objects that bind to the appropriate co_cellvars from the?current frame, and I don't think there's a way to do that from Python.?So, you need to do that by bytecode-hacking the outer function in the same way, just so it can build the inner function.?And, even if you could build closure cells, once you've replaced the inner function definition with a function constructor from bytecode, when the resulting code gets compiled, it won't have any cellvars anymore. And going back to the top, all of these problems are why I think random's solution would be a lot easier than yours, but why my solution (first build compiler hooks or inline assembly, then use that to implement the empty set trivially) would be no harder than either (and a lot more generally useful), and also why I think this really isn't worth doing. From ron3200 at gmail.com Tue Jul 1 02:16:38 2014 From: ron3200 at gmail.com (Ron Adam) Date: Mon, 30 Jun 2014 19:16:38 -0500 Subject: [Python-ideas] Special keyword denoting an infinite loop In-Reply-To: <20140630182030.GR13014@ando> References: <20140628091112.GI13014@ando> <1404149077.20890.136187005.57F7B11E@webmail.messagingengine.com> <20140630182030.GR13014@ando> Message-ID: On 06/30/2014 01:20 PM, Steven D'Aprano wrote: > On Mon, Jun 30, 2014 at 01:24:37PM -0400, random832 at fastmail.us wrote: >> On Sat, Jun 28, 2014, at 06:05, Stefan Behnel wrote: >>> Adding a new keyword needs very serious reasoning, and that's a good >>> thing. > [...] >> What about _just_ "while:" or "for:"? > > Why bother? Is there anything you can do with a bare "while:" that you > can't do with "while True:"? If not, what's the point? It looks like (in python3) "while 1:", "while True:", and while with a string, generates the same byte code. Just a bare SETUP_LOOP. Which would be the exact same as "while:" would. So no, it wouldn't make a bit of difference other than saving a few key strokes in the source code. >>> def L(): ... while True: ... break ... >>> L() >>> dis(L) 2 0 SETUP_LOOP 4 (to 7) 3 >> 3 BREAK_LOOP 4 JUMP_ABSOLUTE 3 >> 7 LOAD_CONST 0 (None) 10 RETURN_VALUE >>> def LL(): ... while 1: ... break ... >>> dis(LL) 2 0 SETUP_LOOP 4 (to 7) 3 >> 3 BREAK_LOOP 4 JUMP_ABSOLUTE 3 >> 7 LOAD_CONST 0 (None) 10 RETURN_VALUE >>> def LLL(): ... while "looping": ... break ... >>> dis(LLL) 2 0 SETUP_LOOP 4 (to 7) 3 >> 3 BREAK_LOOP 4 JUMP_ABSOLUTE 3 >> 7 LOAD_CONST 0 (None) 10 RETURN_VALUE Cheers, Ron From abarnert at yahoo.com Tue Jul 1 02:27:55 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Mon, 30 Jun 2014 17:27:55 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <20140701001814.GA27480@leliel.pault.ag> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <20140701001814.GA27480@leliel.pault.ag> Message-ID: <1404174475.669.YahooMailNeo@web181001.mail.ne1.yahoo.com> On Monday, June 30, 2014 5:18 PM, Paul Tagliamonte wrote: [snip] >Right, so, this was brought up before, but with Hylang >(https://github.com/hylang/hy), we abuse the PEP 302 new import hooks to >search sys.path for .hy files rather then .py files. > >You could do the same for your .pyu files (again, *without* the blessing >of the core team, as this is insane), and do the mangling before passing >it to the normal internals to turn it into bytecode / AST. > >Doing it this way means you won't have to futz with the compiler, >and you can remain happy. The reason for needing to futz with the compiler is to generate source code that actually compiles to the bytecode to build an empty set directly, instead of the bytecode to load and call the "set" global. I agree with both you and Guido that the whole thing is silly, and set() is fine. I also agree with your implied assumption that, even if you needed an empty set literal, having it compile to the same thing as set() would be fine. But those who disagree with both, and really want an empty set literal that compiles to "BUILD_SET 0", cannot have it without futzing with the compiler. So, I'd encourage them to work on making the compiler more futzable (which surely more people would have a use for than the number of people for whom set() is intolerably slow, or unusable because they want to redefine the global). From rosuav at gmail.com Tue Jul 1 02:39:14 2014 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 1 Jul 2014 10:39:14 +1000 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: On Tue, Jul 1, 2014 at 9:48 AM, Andrew Barnert wrote: > First, two quick side notes: > > It might be nice if the compiler were as easy to hook as the importer. Alternatively, it might be nice if there were a way to do "inline bytecode assembly" in CPython, similar to the way you do inline assembly in many C compilers, so the answer to random's question is just "asm [('BUILD_SET', 0)]" or something similar. Either of those would make this problem trivial. > That would be interesting, but it raises the possibility of mucking up the stack. (Imagine if you put BUILD_SET 1 in there instead. What's it going to make a set of? What's going to happen to the rest of the stack? Do you REALLY want to debug that?) Back when I did a lot of C and C++ programming, I used to make good use of a "drop to assembly" feature. There were two broad areas where I'd use it: either to access a CPU feature that the compiler and library didn't offer me (like CPUID, in its early days), or to hand-optimize some code. Then compilers got better and better, and the first set of cases got replaced with library functions... and the second lot ended up being no better than the compiler's output, and potentially a lot worse - particularly because they're non-portable. Allowing a "drop to bytecode" in CPython would have the exact same effects, I think. Some people would use it to create an empty set, others would use it to replace variable swapping with a marginally faster and *almost* identical stack-based swap: x,y = y,x LOAD_GLOBAL y LOAD_GLOBAL x ROT_TWO STORE_GLOBAL x STORE_GLOBAL y becomes LOAD_GLOBAL x LOAD_GLOBAL y STORE_GLOBAL x STORE_GLOBAL y Seems fine, right? But it's a subtle change to semantics (evaluation order), and not much benefit anyway. Plus, if it's decided that this semantic change is safe (if it's provably not going to have any significance), a future version of CPython would be able to make the exact same optimization, while leaving the code readable, and portable to other Python implementations. So while an inline bytecode assembler might have some uses, I suspect it'd be an attractive nuisance more than anything else. > On Monday, June 30, 2014 3:12 PM, Chris Angelico wrote: >>On Tue, Jul 1, 2014 at 3:18 AM, wrote: > >>> On Sat, Jun 28, 2014, at 01:28, Chris Angelico wrote: >>>> empty_set_literal = >>>> type(lambda:0)(type((lambda:0).__code__)(0,0,0,3,67,b't\x00\x00d\x01\x00h\x00\x00\x83\x02\x00\x01d\x00\x00S',(None,"I'm > > I think it makes more sense to use types.FunctionType and types.CodeType here than to generate two extra functions for each function, even if that means you have to put an import types at the top of every munged source file. Sure. This is just a proof-of-concept anyway, and it's not meant to be good code. Either way works, I just tried to minimize name usage (and potential name collisions). > But I think what he was suggesting is something like this: Let py_compile.compile generate the .pyc file as normal, then munge the bytecode in that file, instead of compiling each function, munging its bytecode, and emitting source that creates the munged functions. > > > Besides being a lot less work, his version works for ? at top level, in class definitions, in lambda expressions, etc., not just for def statements. And it doesn't require finding and identifying all of the things to munge in a source file (which I assume you'd do bottom-up based on the ast.parse tree or something). > Sure. But all I was doing was responding to the implied statement that it's not possible to write a .py file that makes a function with BUILD_SET 0 in it. Translating a .pyu directly into a .pyc is still possible, but was not the proposal. > But either way, this still doesn't solve the big problem. Compiling a function by hand and then tweaking the bytecode is easy; doing it programmatically is more painful. You obviously need the function to compile, so you have to replace the ? with something else whose bytecode you can search-and-replace. But what? That something else has to be valid in an expression context (so it compiles), has to compile to a 3-byte opcode (otherwise, replacing it will screw up any jump targets that point after it), can't add any globals/constants/etc. to the list (otherwise, removing it will screw up any LOAD_FOO statements that refer to a higher-numbered foo), and can't appear anywhere in the code being compiled. > What I did was put in a literal string. https://github.com/Rosuav/shed/blob/master/empty_set.py It uses "? is set()" as a marker, depending on that string not existing in the source. (I could compile the function twice, once with that string, and then a second time with another string; the first compilation would show what consts it uses, and the program could then generate an arbitrary constant which doesn't exist.) The opcode is the right length (assuming it doesn't go for EXTENDED_ARG, which I've never heard of; it seems to be necessary if you have more than 64K consts/globals/locals in a function???), and the resulting function has an unnecessary const in it. It wouldn't be hard to drop it (the code already parses through everything; it could just go "if it's LOAD_CONST, three options - if it's the marker, switch in a BUILD_SET, if it's less than the marker, no change, if it's more than the marker, decrement"), but it doesn't seem to be a problem to have an extra const in there. > One more thing that I'm sure you thought of, but may not have thought through all the way: To make this generally useful, you can't just hardcode creating a zero-arg top-level function; you need to copy all of the code and function constructor arguments from the compiled function. > It handles arguments and stuff. All the attributes of the original function object get passed through unchanged to the resulting function, with the exception of the bytecode, obviously. > So, if the function is a closure, how do you do that? You need to pass a list of closure cell objects that bind to the appropriate co_cellvars from the current frame, and I don't think there's a way to do that from Python. So, you need to do that by bytecode-hacking the outer function in the same way, just so it can build the inner function. And, even if you could build closure cells, once you've replaced the inner function definition with a function constructor from bytecode, when the resulting code gets compiled, it won't have any cellvars anymore. > Ah, that part I've no idea about. But it wouldn't be impossible for someone to develop that a bit further. > And going back to the top, all of these problems are why I think random's solution would be a lot easier than yours, but why my solution (first build compiler hooks or inline assembly, then use that to implement the empty set trivially) would be no harder than either (and a lot more generally useful), and also why I think this really isn't worth doing. > Right. I absolutely agree with your conclusion (not worth doing), and always have had that view. This is proof that it's kinda possible, but still a bad idea. Now, if someone comes up with a really compelling use-case for an empty set literal, then maybe it'd be more important; but if that happens, CPython will probably grow an empty set literal in ASCII somehow, and then the .pyu translation can just turn ? into that. ChrisA From paultag at gmail.com Tue Jul 1 02:18:14 2014 From: paultag at gmail.com (Paul Tagliamonte) Date: Mon, 30 Jun 2014 20:18:14 -0400 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <20140701001814.GA27480@leliel.pault.ag> On Mon, Jun 30, 2014 at 04:48:14PM -0700, Andrew Barnert wrote: > First, two quick side notes: > > It might be nice if the compiler were as easy to hook as the importer.?Alternatively, it might be nice if there were a way to do "inline bytecode assembly" in CPython, similar to the way you do inline assembly in many C compilers, so the answer to random's question is just "asm [('BUILD_SET', 0)]" or something similar. Either of those would make this problem trivial. > > I doubt either of those would be useful often enough that anyone wants to put in the work.?But then I doubt the empty-set literal would be either, so anyone who seriously wants to work on this might want to work on the inline assembly and/or hookable compiler first. Again, to be absolutely clear here, I hate this idea. `set()` is perfectly clear. Sorry. Had to be said before any of this. Right, so, this was brought up before, but with Hylang (https://github.com/hylang/hy), we abuse the PEP 302 new import hooks to search sys.path for .hy files rather then .py files. You could do the same for your .pyu files (again, *without* the blessing of the core team, as this is insane), and do the mangling before passing it to the normal internals to turn it into bytecode / AST. Doing it this way means you won't have to futz with the compiler, and you can remain happy. And we like being happy. More info: https://github.com/hylang/hy/blob/master/hy/importer.py http://slides.pault.ag/hy.html#/15 https://www.youtube.com/watch?v=AmMaN1AokTI https://www.youtube.com/watch?v=ulekCWvDFVI http://legacy.python.org/dev/peps/pep-0302/ Again, this approach can be a bit flaky, and this particular issue might very well cause problems for us as a community - seeing as how the syntax is almost exactly identical. Hylang (for what it's worth) is just a nice way for us Lisp nerds to stop complaining as much. Godspeed, Paul -- #define sizeof(x) rand() :wq -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: From ncoghlan at gmail.com Tue Jul 1 04:15:20 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 1 Jul 2014 12:15:20 +1000 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: On 30 Jun 2014 16:51, "Andrew Barnert" wrote: > > First, two quick side notes: > > It might be nice if the compiler were as easy to hook as the importer. Alternatively, it might be nice if there were a way to do "inline bytecode assembly" in CPython, similar to the way you do inline assembly in many C compilers, so the answer to random's question is just "asm [('BUILD_SET', 0)]" or something similar. Either of those would make this problem trivial. Eugene Toder & Dave Malcolm have some interesting patches on the tracker to help enhance the compiler (in particular, Dave's allowed compiler plugins to be written in Python). Neither set of patches made it to being merge ready, though, and they'll be rather stale at this point. Cheers, Nick. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Tue Jul 1 04:18:44 2014 From: guido at python.org (Guido van Rossum) Date: Mon, 30 Jun 2014 19:18:44 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: Like bytecode, the compiler's workings are not part of the language spec, and are likely to change incompatibly between versions and not work for anything besides CPython. I don't really want to go there (cool though it sounds for wannabe compiler hackers). On Mon, Jun 30, 2014 at 7:15 PM, Nick Coghlan wrote: > > On 30 Jun 2014 16:51, "Andrew Barnert" > wrote: > > > > First, two quick side notes: > > > > It might be nice if the compiler were as easy to hook as the > importer. Alternatively, it might be nice if there were a way to do "inline > bytecode assembly" in CPython, similar to the way you do inline assembly in > many C compilers, so the answer to random's question is just "asm > [('BUILD_SET', 0)]" or something similar. Either of those would make this > problem trivial. > > Eugene Toder & Dave Malcolm have some interesting patches on the tracker > to help enhance the compiler (in particular, Dave's allowed compiler > plugins to be written in Python). Neither set of patches made it to being > merge ready, though, and they'll be rather stale at this point. > > Cheers, > Nick. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From abarnert at yahoo.com Tue Jul 1 10:04:37 2014 From: abarnert at yahoo.com (Andrew Barnert) Date: Tue, 1 Jul 2014 01:04:37 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404201877.75807.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <1404211878.45311.YahooMailNeo@web181005.mail.ne1.yahoo.com> > On Tuesday, July 1, 2014 1:39 AM, Chris Angelico wrote: > > On Tue, Jul 1, 2014 at 6:04 PM, Andrew Barnert wrote: >>> On Monday, June 30, 2014 5:39 PM, Chris Angelico > wrote: >> >>> That would be interesting, but it raises the possibility of mucking up >>> the stack. (Imagine if you put BUILD_SET 1 in there instead. What's > it >>> going to make a set of? What's going to happen to the rest of the >>> stack? Do you REALLY want to debug that?) >> >> The same thing that happens if you use bad inline assembly in C, or a bad C > extension module in Python?bad things that you can't debug at source level. > And yet, inline assembly in C and C extension modules in Python are still quite > useful. > > Right, useful but it adds another set of problems. (Just out of > curiosity, what protection _is_ there for a smashed stack? I just > tried fiddling with it and didn't manage to crash stuff.) I believe there are cases where the interpreter can detect that you've gone below 0 and raise an exception, but in general there's no protection, or at least nothing you can count on. For example, assemble this code as a complete function: ? ? CALL_FUNCTION 1 ? ? RETURN_VALUE In 3.4.1, on my Mac, I get a bus error. But, even when you don't manage to crash the interpreter, when you just confuse it at the bytecode level, there's still no way to debug that except by dropping to gdb/lldb/etc. >> I'll ignore the second case for the moment, because I think it's > rarely if ever appropriate to Python, and just focus on the first. Those cases > did not go away because CPUID got replaced with library functions. Those library > functions?which are compiled with the same compiler you use for your code?have > inline assembly in them. (Or, if you're on linux, those library functions > read from a device file, but the device driver, which is compiled with the same > compiler you use, has inline assembly in it.) So, the compiler still needs to be > able to compile it. >> > > Or those library functions are written in assembly language directly. > It's entirely possible to write something that uses CPUID and doesn't > use inline assembly in a C source file. The equivalent here, I > suppose, would be hand-rolling a .pyc file. Yeah, that's entirely possible, but that's not how the linux device driver or the FreeBSD libc wrapper do it; they use inline assembly. Why? Well, for one thing, you get the function prolog and epilog code appropriate for your compiler automatically, instead of having to write it yourself. Also, you can do nice things like cast the result to a struct that you defined in C (which could be done with, e.g., a C macro wrapping the assembly source, but that's just making things more complicated for no benefit). And you don't need to know how to configure and run an assembler alongside the C compiler to build the device. And so on. Basically, the C versions of the exact same reasons you wouldn't want to hand-roll a .pyc file in Python? > 2) Do I think anyone would, if given the ability to tweak the > bytecode, go "Ah ha!" and proudly improve on what the compiler has > done, and then brag about the performance improvement? Definitely. > Someone will. It'll make some marginal difference to a microbenchmark, > and if you don't believe that would cause people to warp their code > into utter unreadability, you clearly don't hang out on python-list > enough :) Using ctypes to load Python.so to swap the pointers under the covers is already significantly faster, and would still be significantly faster than your optimized bytecode, and yes, people have suggested it on at least two StackOverflow questions. For that matter, you can already do exactly your optimization with a relatively simple bytecode hack, which would look a lot worse than the inline asm and have the same effect. Also, that bytecode hack could be factored out into a function, without any performance cost except a constant cost at .pyc time, while the inline asm obviously can't, another reason the inline asm (which would have to be written inline, and edited to fit the variables in question, each time) would be less of an attractive nuisance than what's already there. Sure, there may be a few people who are looking for horrible micro-optimizations like this, would know enough to figure out how to do this with inline asm, would not know how to do it with bytecode hacks, would not know any of the better (as in much worse, to anyone but them) alternatives, etc., but I think that number is vanishingly small. >>> What I did was put in a literal string? >>> It uses "? is set()" as a marker ? and the resulting function >>> has an unnecessary const in it. >> >> I assumed that leaving the unnecessary const behind was unacceptable. After > all, we're talking about (hypothetical?) people who find the cost of > LOAD_GLOBAL set; CALL_FUNCTION 0 to be unacceptable? But you're right that > fixing up all the other LOAD_CONST bytecodes' args is a feasible way to > solve that. > > I'm not sure whether the problem is the cost of LOAD_GLOBAL followed > by CALL_FUNCTION (and, by the way, one unnecessary constant in the > function won't have anything like that cost - a bit of wasted RAM, but > not a function call), or the fact that such a style is vulnerable to > shadowing of the name 'set', which admittedly is a very useful name. > But in any case, it's quite solvable. I realize the cost of an extra LOAD_GLOBAL is much smaller than an extra CALL_FUNCTION, it's just that I think in 99.9999% of real cases neither will make a difference, and anyone who's objecting to the latter on principle will probably also object to the former on principle? >>>> ? So, if the function is a closure, how do you do that? >>> Ah, that part I've no idea about. But it wouldn't be impossible > for >>> someone to develop that a bit further. >> >> Not impossible, but very hard, much harder than what you've done so > far. >> >> Ultimately, I think that just backs up your larger point: This is doable, > but it's going to be a lot of work, and the benefit isn't even nearly > worth the cost. My point is that there are other ways to do it that would be > less work and/or that would have more side benefits? but the benefit still > isn't even nearly worth the cost, so who cares? :) > > Yep. Maybe someone (great, that probably means me) should write this > up into a PEP for immediate rejection or withdrawal, just to be a > document to point to - if you want an empty set literal, answer these > objections. I think Terry Reedy actually had a better answer: just tell people to implement it, polish it up, put it on PyPI, and come back to us when they're ready to show off their tons of users who can't live without it. Random objected that wasn't possible, in which case Terry's idea is more of a dismissal than a helpful suggestion, but I think https://github.com/abarnert/emptyset proves that it is possible, and even pretty easy. From kn0m0n3 at gmail.com Tue Jul 1 16:23:16 2014 From: kn0m0n3 at gmail.com (www.leap.cc) Date: Tue, 01 Jul 2014 09:23:16 -0500 Subject: [Python-ideas] Enigmail on cell phone? Message-ID: Anyone done this successfully? How can I use python for hidden Markov mods in machine learning from closed circuit came for public transportation to make better schedualing ask local universities computer science competition... cheers, d.j.? Andrew Barnert wrote: >> On Tuesday, July 1, 2014 1:39 AM, Chris Angelico wrote: > >> > On Tue, Jul 1, 2014 at 6:04 PM, Andrew Barnert wrote: >>>> On Monday, June 30, 2014 5:39 PM, Chris Angelico >> wrote: >>> >>>> That would be interesting, but it raises the possibility of mucking up >>>> the stack. (Imagine if you put BUILD_SET 1 in there instead. What's >> it >>>> going to make a set of? What's going to happen to the rest of the >>>> stack? Do you REALLY want to debug that?) >>> >>> The same thing that happens if you use bad inline assembly in C, or a bad C >> extension module in Python?bad things that you can't debug at source level. >> And yet, inline assembly in C and C extension modules in Python are still quite >> useful. >> >> Right, useful but it adds another set of problems. (Just out of >> curiosity, what protection _is_ there for a smashed stack? I just >> tried fiddling with it and didn't manage to crash stuff.) > >I believe there are cases where the interpreter can detect that you've gone below 0 and raise an exception, but in general there's no protection, or at least nothing you can count on. > >For example, assemble this code as a complete function: > >? ? CALL_FUNCTION 1 >? ? RETURN_VALUE > >In 3.4.1, on my Mac, I get a bus error. > >But, even when you don't manage to crash the interpreter, when you just confuse it at the bytecode level, there's still no way to debug that except by dropping to gdb/lldb/etc. > >>> I'll ignore the second case for the moment, because I think it's >> rarely if ever appropriate to Python, and just focus on the first. Those cases >> did not go away because CPUID got replaced with library functions. Those library >> functions?which are compiled with the same compiler you use for your code?have >> inline assembly in them. (Or, if you're on linux, those library functions >> read from a device file, but the device driver, which is compiled with the same >> compiler you use, has inline assembly in it.) So, the compiler still needs to be >> able to compile it. >>> >> >> Or those library functions are written in assembly language directly. >> It's entirely possible to write something that uses CPUID and doesn't >> use inline assembly in a C source file. The equivalent here, I >> suppose, would be hand-rolling a .pyc file. > >Yeah, that's entirely possible, but that's not how the linux device driver or the FreeBSD libc wrapper do it; they use inline assembly. Why? Well, for one thing, you get the function prolog and epilog code appropriate for your compiler automatically, instead of having to write it yourself. Also, you can do nice things like cast the result to a struct that you defined in C (which could be done with, e.g., a C macro wrapping the assembly source, but that's just making things more complicated for no benefit). And you don't need to know how to configure and run an assembler alongside the C compiler to build the device. And so on. Basically, the C versions of the exact same reasons you wouldn't want to hand-roll a .pyc file in Python? > >> 2) Do I think anyone would, if given the ability to tweak the >> bytecode, go "Ah ha!" and proudly improve on what the compiler has >> done, and then brag about the performance improvement? Definitely. >> Someone will. It'll make some marginal difference to a microbenchmark, >> and if you don't believe that would cause people to warp their code >> into utter unreadability, you clearly don't hang out on python-list >> enough :) > > >Using ctypes to load Python.so to swap the pointers under the covers is already significantly faster, and would still be significantly faster than your optimized bytecode, and yes, people have suggested it on at least two StackOverflow questions. For that matter, you can already do exactly your optimization with a relatively simple bytecode hack, which would look a lot worse than the inline asm and have the same effect. Also, that bytecode hack could be factored out into a function, without any performance cost except a constant cost at .pyc time, while the inline asm obviously can't, another reason the inline asm (which would have to be written inline, and edited to fit the variables in question, each time) would be less of an attractive nuisance than what's already there. Sure, there may be a few people who are looking for horrible micro-optimizations like this, would know enough to figure out how to do this with inline asm, would not know how to do it > with bytecode hacks, would not know any of the better (as in much worse, to anyone but them) alternatives, etc., but I think that number is vanishingly small. > >>>> What I did was put in a literal string? > >>>> It uses "? is set()" as a marker ? and the resulting function >>>> has an unnecessary const in it. >>> >>> I assumed that leaving the unnecessary const behind was unacceptable. After >> all, we're talking about (hypothetical?) people who find the cost of >> LOAD_GLOBAL set; CALL_FUNCTION 0 to be unacceptable? But you're right that >> fixing up all the other LOAD_CONST bytecodes' args is a feasible way to >> solve that. >> >> I'm not sure whether the problem is the cost of LOAD_GLOBAL followed >> by CALL_FUNCTION (and, by the way, one unnecessary constant in the >> function won't have anything like that cost - a bit of wasted RAM, but >> not a function call), or the fact that such a style is vulnerable to >> shadowing of the name 'set', which admittedly is a very useful name. >> But in any case, it's quite solvable. > >I realize the cost of an extra LOAD_GLOBAL is much smaller than an extra CALL_FUNCTION, it's just that I think in 99.9999% of real cases neither will make a difference, and anyone who's objecting to the latter on principle will probably also object to the former on principle? > >>>>> ? So, if the function is a closure, how do you do that? >>>> Ah, that part I've no idea about. But it wouldn't be impossible >> for >>>> someone to develop that a bit further. >>> >>> Not impossible, but very hard, much harder than what you've done so >> far. >>> >>> Ultimately, I think that just backs up your larger point: This is doable, >> but it's going to be a lot of work, and the benefit isn't even nearly >> worth the cost. My point is that there are other ways to do it that would be >> less work and/or that would have more side benefits? but the benefit still >> isn't even nearly worth the cost, so who cares? :) >> >> Yep. Maybe someone (great, that probably means me) should write this >> up into a PEP for immediate rejection or withdrawal, just to be a >> document to point to - if you want an empty set literal, answer these >> objections. > > >I think Terry Reedy actually had a better answer: just tell people to implement it, polish it up, put it on PyPI, and come back to us when they're ready to show off their tons of users who can't live without it. Random objected that wasn't possible, in which case Terry's idea is more of a dismissal than a helpful suggestion, but I think https://github.com/abarnert/emptyset proves that it is possible, and even pretty easy. >_______________________________________________ >Python-ideas mailing list >Python-ideas at python.org >https://mail.python.org/mailman/listinfo/python-ideas >Code of Conduct: http://python.org/psf/codeofconduct/ From liam.marsh.home at gmail.com Tue Jul 1 18:31:22 2014 From: liam.marsh.home at gmail.com (Liam Marsh) Date: Tue, 1 Jul 2014 18:31:22 +0200 Subject: [Python-ideas] error in os.popen result Message-ID: hello, for a small server program, I wanted to know which ports were occuped. with the dos command 'netstat' so I tried this: *>>>a=os.popen('netstat')* *>>>bytes(a.read())* but this occured in the second step: *Traceback (most recent call last):* * File "", line 1, in * * bytes(a.read())* * File "C:\Apps\Programmation\Python3.2\lib\encodings\cp1252.py", line 23, in decode* * return codecs.charmap_decode(input,self.errors,decoding_table)[0]* *UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 79: character maps to * *how can I avoid it and why does the windows cmd does return an undecodable character?* *thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Tue Jul 1 18:58:37 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 1 Jul 2014 09:58:37 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <1404203220.53463.YahooMailNeo@web181002.mail.ne1.yahoo.com> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404203220.53463.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: On 1 July 2014 01:27, Andrew Barnert wrote: > (Replies to both Guido's top-post and Nick's reply-post below.) > > On Monday, June 30, 2014 7:19 PM, Guido van Rossum wrote: > >>Like bytecode, the compiler's workings are not part of the language spec, and are likely to change incompatibly between versions and not work for anything besides CPython. I don't really want to go there (cool though it sounds for wannabe compiler hackers). > > > But CPython does expose bytecode via the dis module, parts of inspect, etc. For that matter, it exposes some of the compiler's workings (especially if you consider everything up to AST generation part of the compiler, since every step up to there is exposed, including doing the whole thing in one whack with PyCF_ONLY_AST). So, I don't see how exposing the AST-to-bytecode transformation part (or, while we're at it, the .pyc generation part) is any more unportable than what's already there. > Note that the dis module has a "CPython implementation detail" disclaimer, and the AST structure is deliberately exempted from the usual backwards compatibility guarantees. As far as hooking compilation goes, https://docs.python.org/3/library/importlib.html#importlib.abc.InspectLoader.source_to_code was added in 3.4 specifically to make it easier to define custom loaders that make use of most of the existing import machinery (including bytecode cache files), but do something different for the source -> bytecode transformation step. Cheers, Nick. From steve at pearwood.info Tue Jul 1 19:04:22 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 2 Jul 2014 03:04:22 +1000 Subject: [Python-ideas] error in os.popen result In-Reply-To: References: Message-ID: <20140701170422.GS13014@ando> Hello Liam, This is a mailing list for discussing possible future ideas for the next version of Python, not for general support. I recommend that you use the python-list at python.org mailing list, also available via Usenet on comp.lang.python. Good luck. On Tue, Jul 01, 2014 at 06:31:22PM +0200, Liam Marsh wrote: > hello, for a small server program, I wanted to know which ports were > occuped. > with the dos command 'netstat' > so I tried this: > > *>>>a=os.popen('netstat')* > *>>>bytes(a.read())* > > but this occured in the second step: > > *Traceback (most recent call last):* > * File "", line 1, in * > * bytes(a.read())* > * File "C:\Apps\Programmation\Python3.2\lib\encodings\cp1252.py", line 23, > in decode* > * return codecs.charmap_decode(input,self.errors,decoding_table)[0]* > *UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 79: > character maps to * > > *how can I avoid it and why does the windows cmd does return an > undecodable character?* > > *thank you.* > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From steve at pearwood.info Tue Jul 1 19:15:29 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 2 Jul 2014 03:15:29 +1000 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: References: <1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404203220.53463.YahooMailNeo@web181002.mail.ne1.yahoo.com> Message-ID: <20140701171529.GT13014@ando> On Tue, Jul 01, 2014 at 09:58:37AM -0700, Nick Coghlan wrote: > On 1 July 2014 01:27, Andrew Barnert wrote: > > But CPython does expose bytecode via the dis module, parts of > > inspect, etc. [...] > > Note that the dis module has a "CPython implementation detail" > disclaimer, and the AST structure is deliberately exempted from the > usual backwards compatibility guarantees. Further to what Nick says, the *output* of dis is not expected to remain backwards compatible from version to version, only the dis API itself. There's a big difference between saying "we guarantee that the dis module will correctly and accurately disassemble valid bytecode", and saying "we guarantee that this specific chunk of bytecode will do these things". In order to use a hypothetical asm function, you need to know what pseudo-assembly to write, say `asm [SPAM, EGGS]`. That means that SPAM and EGGS must be stable and part of the language definition. (Or at least part of the CPython API.) That's a big step from the current situation. -- Steven From steve at pearwood.info Tue Jul 1 19:33:07 2014 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 2 Jul 2014 03:33:07 +1000 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: References: <1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404201877.75807.YahooMailNeo@web181004.mail.ne1.yahoo.com> Message-ID: <20140701173307.GU13014@ando> On Tue, Jul 01, 2014 at 06:38:37PM +1000, Chris Angelico wrote: [...] > 1) Do I really think anyone *should* do this? Your subsequent comments > support this question, and the answer is resoundingly NO. CPython is "This" being trying to micro-optimize code by bytecode-hacking. > not the sort of platform on which that kind of thing is ever worth > doing. You'll get far more performance by using Cython for parts, or > in some other way improving your code, than you will by hand-tweaking > the Python bytecode. I think that micro-optimization is probably the wrong reason to hack bytecodes. What I'm more interested in is exploring potential new features, or to add functionality, for example: Adding the ability to trace individual expressions, not just lines: http://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html Exploring dynamic scoping: http://www.voidspace.org.uk/python/articles/code_blocks.shtml A proposal from Python 2.3 days for a brand-new decorator syntax: http://code.activestate.com/recipes/286147 A (serious!) defence of GOTO in Python: http://www.dr-josiah.com/2012/04/python-bytecode-hacks-gotos-revisited.html (although even Josiah doesn't suggest using COMEFROM :-) I don't know that such bytecode manipulations should be provided in the standard library, and certainly not as a built-in "asm" command. But, I think that we ought to acknowledge that bytecode hacking has a role to play in the wider Python ecosystem. I'm lead to understand that in the Java community, bytecode hacking is, perhaps not common, but accepted as something that powerusers do when all else fails: https://weblogs.java.net/blog/simonis/archive/2009/02/we_need_a_dirty.html [Aside: does Python do any sort of verification of the bytecode before executing it, as Java does?] -- Steven From ncoghlan at gmail.com Tue Jul 1 20:07:35 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 1 Jul 2014 11:07:35 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <20140701173307.GU13014@ando> References: <1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404201877.75807.YahooMailNeo@web181004.mail.ne1.yahoo.com> <20140701173307.GU13014@ando> Message-ID: On 1 July 2014 10:33, Steven D'Aprano wrote: > On Tue, Jul 01, 2014 at 06:38:37PM +1000, Chris Angelico wrote: > [...] >> 1) Do I really think anyone *should* do this? Your subsequent comments >> support this question, and the answer is resoundingly NO. CPython is > > "This" being trying to micro-optimize code by bytecode-hacking. > >> not the sort of platform on which that kind of thing is ever worth >> doing. You'll get far more performance by using Cython for parts, or >> in some other way improving your code, than you will by hand-tweaking >> the Python bytecode. > > I think that micro-optimization is probably the wrong reason to hack > bytecodes. What I'm more interested in is exploring potential new > features, or to add functionality https://pypi.python.org/pypi/withhacks and https://pypi.python.org/pypi/byteplay may also be of interest to anyone wishing to seriously tinker with what the CPython VM (as opposed to Python-the-language) already supports. I also highly advise working Python 3.4, since we made some substantial improvements to the dis module API (adding the yield from tests for 3.3 highlighted how limited the previous API was for testing purposes, so we fixed it in a way that made bytecode easier to work with in general). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Tue Jul 1 20:16:29 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 1 Jul 2014 11:16:29 -0700 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <20140701173307.GU13014@ando> References: <1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404201877.75807.YahooMailNeo@web181004.mail.ne1.yahoo.com> <20140701173307.GU13014@ando> Message-ID: On 1 July 2014 10:33, Steven D'Aprano wrote: > [Aside: does Python do any sort of verification of the bytecode before > executing it, as Java does?] Nope, it will happily attempt to execute invalid bytecode. That's actually one of the reasons executing untrusted bytecode is even less safe than executing untrusted source code - it's likely to be possible to trigger segfaults that way. There's an initial attempt at a bytecode verifier on PyPI (https://pypi.python.org/pypi/Python-Bytecode-Verifier/), and I have a vague recollection that Google have a bytecode verifier kicking around somewhere, but there's nothing built in to the CPython runtime. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From python at mrabarnett.plus.com Tue Jul 1 20:59:21 2014 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 01 Jul 2014 19:59:21 +0100 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: References: <1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404201877.75807.YahooMailNeo@web181004.mail.ne1.yahoo.com> <20140701173307.GU13014@ando> Message-ID: <53B30509.4090808@mrabarnett.plus.com> On 2014-07-01 19:16, Nick Coghlan wrote: > On 1 July 2014 10:33, Steven D'Aprano wrote: > >> [Aside: does Python do any sort of verification of the bytecode >> before executing it, as Java does?] > > Nope, it will happily attempt to execute invalid bytecode. That's > actually one of the reasons executing untrusted bytecode is even less > safe than executing untrusted source code - it's likely to be > possible to trigger segfaults that way. > > There's an initial attempt at a bytecode verifier on PyPI > (https://pypi.python.org/pypi/Python-Bytecode-Verifier/), and I have > a vague recollection that Google have a bytecode verifier kicking > around somewhere, but there's nothing built in to the CPython > runtime. > The re module also uses a kind of bytecode that's generated by the Python front end and verified by the C back end. The bytecode contains things like offsets; for example, the bytecode that starts a repeated sequence has an offset to the corresponding bytecode that ends it, and vice versa. The problem with that is that the structure (i.e. the nesting) is no longer explicit, so it's more difficult to spot misnested structures. For the regex module, I decided that it would be easier to verify if I kept the structure explicit by using bytecodes to indicate the start and end of the structures. For example, a repeated sequence could be indicated by having a structure like GREEDY_REPEAT min_count max_count ... END. The C back end could then build the internal representation that's actually interpreted. From tjreedy at udel.edu Tue Jul 1 22:04:20 2014 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 01 Jul 2014 16:04:20 -0400 Subject: [Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict) In-Reply-To: <1404211878.45311.YahooMailNeo@web181005.mail.ne1.yahoo.com> References:

<1403931602.14407.135458493.44CF193B@webmail.messagingengine.com> <1404148690.18766.136186337.62A26E9E@webmail.messagingengine.com> <1404172094.55099.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404201877.75807.YahooMailNeo@web181004.mail.ne1.yahoo.com> <1404211878.45311.YahooMailNeo@web181005.mail.ne1.yahoo.com> Message-ID: <1404252197.45327.YahooMailNeo@web181001.mail.ne1.yahoo.com> > On Tuesday, July 1, 2014 1:05 PM, Terry Reedy wrote: > > On 7/1/2014 6:51 AM, Andrew Barnert wrote: > >> I think Terry Reedy actually had a better answer: just tell people to >> implement it,? polish it up, put it on PyPI, and come back to us when >> they're ready to show off their tons of users who can't live > without >> it. Random objected that wasn't possible, > > 'Random' said something quite different. He only noted that if > '?' were > translated to 'set()', then the resulting CPython-specific bytecode > would continue to be "LOAD_GLOBAL (set), CALL_FUNCTION 0" rather than > the 'optimized' "BUILD_SET 0". He also noted (objected) that > there is no > python code that CPython currently compiles as "BUILD_SET 0"? You're reading a lot into a 2-line message, but your take is that he interpreted the problem as needing to compile "BUILD_SET 0", and pointed out that there is no way to do that with a source preprocessor. You can insist that they're two separate problems to be solved (or, maybe, not solved), and I think you're right. You just have to make that point?as you, I, and half a dozen others have done since his original post. But meanwhile, Chris Angelico offered a solution to the problem that answers his complaint, and I offered another solution that doesn't even require bytecode hacking. That shows that even if you accept the objection, it still doesn't block anyone. > Well, its > unfortunate that {} is not available. If it were, there would be no > issue, to me anyway, of using '?'.? However, optimizing CPython > bytecode, and compiler hooks, are completely different issues from > translating unisym python to standard python that could run on any > implementation of Python. First, as others have pointed out, it's not just, or even primarily, an optimization, it's also a semantic difference. > If we thought the bytecode difference was? > important (which most do not), we could have a peephole optimizer to > 'fix' it, completely independently of the existence of '?' or > any idea > of using it in python code. But you can't make semantic changes in a peephole optimizer. You'd have to first change the language to document that set() may (or may not!) return an empty set even if the name set resolves to something different. While this isn't entirely unique in Python history (e.g., back when you could redefine False through various kinds of trickery, the compiler was still allowed to optimize out if False: code), but it's very unusual. And nobody's going to do that for a minor optimization (if False:, besides being a potentially huge optimization, also _fixes_ a semantic problem, rather than causing one, since False was supposed to be un-redefinable, but wasn't because of various holes). >> in which case Terry's idea is more of a dismissal than a helpful > suggestion, > > My post was a dismissal of the idea of changing python itself *and* a > suggestion of how to proceed without involving pydev. My point is that _if_ you take Random's objection as being critical, _then_ your post dismisses the idea, even though it wasn't intended to. You can follow up in two ways: challenge his objection, or answer his objection; there were replies doing both, and if either of the two succeeds, the idea is still alive for people to take further if they want. >> https://github.com/abarnert/emptyset proves that it is possible, and >> even pretty easy. > > I consider producing (or at least being able to produce) a standard .py > file that can be published outside the specialized group run on and > debugged on standard interpreters to be essential to any sensible idea? My approach is made up of nothing but standard .py files. Those files can be published outside a specialized group, and run and debugged on CPython 3.4+. They can also be edited by people outside that specialized group, without needing a specialized build process involving a preprocessor, just a standard Python module that they already have. Sure, it only works on CPython, but Python 3.4, scipy, etc. also currently only work on CPython, and that doesn't prevent a large community of users from making using of them, publishing code outside a specialized group, and?most importantly for the topic at hand?coming up with suggestions that are germane to Python as a whole and taken seriously. For example, nobody suggested that PEP 465 wasn't a sensible idea because all of the sample code presented only runs on CPython; the idea itself is clearly portable, the community using such code is gigantic and mature, and that's all that matters. Finally, I don't think anyone actually needs this feature, but I was able to whip up a proof of concept in an hour that provides it. Anyone who seriously wants to pursue it doesn't have to use my approach, much less my code; it still serves as an existence proof that what they want to do can be done, meaning they should go do it. From stefano.borini at ferrara.linux.it Wed Jul 2 00:36:48 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Wed, 02 Jul 2014 00:36:48 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments Message-ID: <53B33800.1030300@ferrara.linux.it> Dear all, after the first mailing list feedback, and further private discussion with Joseph Martinot-Lagarde, I drafted a first iteration of a PEP for keyword arguments in indexing. The document is available here. https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt The document is not in final form when it comes to specifications. In fact, it requires additional discussion about the best strategy to achieve the desired result. Particular attention has been devoted to present alternative implementation strategies, their pros and cons. I will examine all feedback tomorrow morning European time (in approx 10 hrs), and apply any pull requests or comments you may have. When the specification is finalized, or this community suggests that the PEP is in a form suitable for official submission despite potential open issues, I will submit it to the editor panel for further discussion, and deploy an actual implementation according to the agreed specification for a working test run. I apologize for potential mistakes in the PEP drafting and submission process, as this is my first PEP. Kind Regards, Stefano Borini From rosuav at gmail.com Wed Jul 2 03:06:24 2014 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 2 Jul 2014 11:06:24 +1000 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B33800.1030300@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> Message-ID: On Wed, Jul 2, 2014 at 8:36 AM, Stefano Borini wrote: > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt A good start! """ C0: a[1] -> idx = 1 # integer a[1,2] -> idx = (1,2) # tuple C1: a[Z=3] -> idx = {"Z": 3} # dictionary with single key C2. a[Z=3, R=4] -> idx = {"Z": 3, "R": 4} # dictionary/ordereddict [*] or idx = ({"Z": 3}, {"R": 4}) # tuple of two single-key dict [**] ... C5. a[1, 2, Z=3] -> idx = (1, 2, {"Z": 3}) """ Another possibility for the keyword arguments is a two-item tuple, which would mean that C1 comes up as ("Z", 3) (or maybe (("Z", 3),) - keyword arguments forcing a tuple of all args for consistency/clarity), C2 as (("Z", 3), ("R", 4)), and C5 as (1, 2, ("Z", 3)). This would be lighter and easier to use than the tuple of dicts, and still preserves order (unlike the regular dict); however, it doesn't let you easily fetch up the one keyword you're interested in, which is normally something you'd want to support for a **kwargs-like feature: def __getitem__(self, item, **kwargs): # either that, or kwargs is part of item in some way ret = self.base[item] if "precis" in kwargs: ret.round(kwargs["precis"]) return ret To implement that with a tuple of tuples, or a tuple of dicts, you'd have to iterate over it and check each one - much less clean code. I would be inclined to simply state, in the PEP, that keyword arguments in indexing are equivalent to kwargs in function calls, and equally unordered (that is to say: if a proposal to make function call kwargs ordered is accepted, the same consideration can be applied to this, but otherwise they have no order). This does mean that it doesn't fit the original use-case, but it seems very odd to start out by saying "here, let's give indexing the option to carry keyword args, just like with function calls", and then come back and say "oh, but unlike function calls, they're inherently ordered and carried very differently". For the OP's use-case, though, it would actually be possible to abuse slice notation. I don't remember this being mentioned, but it does preserve order; the cost is that all the "keywords" have to be defined as objects. class kw: pass # because object() doesn't have attributes def make_kw(names): for i in names.split(): globals()[i] = obj = kw() obj.keyword_arg = i make_kw("Z R X") # Now you can use them in indexing some_obj[5, Z:3] some_obj[7, Z:3, R:4] The parameters will arrive in the item tuple as slice objects, where the start is a signature object and the stop is its value. >>> some_obj[5, Z:3] getitem: (5, slice(<__main__.kw object at 0x016C5E10>, 3, None)) Yes, it uses a colon rather than an equals sign, but on the flip side, it already works :) ChrisA From rob.cliffe at btinternet.com Wed Jul 2 04:36:23 2014 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 02 Jul 2014 03:36:23 +0100 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B33800.1030300@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> Message-ID: <53B37027.9030808@btinternet.com> On 01/07/2014 23:36, Stefano Borini wrote: > Dear all, > > after the first mailing list feedback, and further private discussion > with Joseph Martinot-Lagarde, I drafted a first iteration of a PEP for > keyword arguments in indexing. The document is available here. > > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt A small bit of uninformed feedback (no charge :-) ): 1) Ahem, doesn't a[3] (usually) return the *fourth* element of a ? 2) """ Compare e.g. a[1:3, Z=2] with a.get(slice(1,3,None), Z=2). """ I think this is slightly unfair as the second form can be abbreviated to a.get(slice(1,3), Z=2), just as the first is an abbreviation for a[1:3:None, Z=2]. 3) You may not consider this relevant. But as an (I believe) intelligent reader, but one unfamiliar with the material, I cannot understand what your first example """ low_accuracy_energy = computeEnergy(molecule, BasisSet[Z=3]) """ is about, and whether it is really (conceptually) related to indexing, or just a slick hack. I guess it could be anything, depending on the implementation of __getitem__. Best wishes, Rob Cliffe From anthony at xtfx.me Wed Jul 2 04:58:44 2014 From: anthony at xtfx.me (C Anthony Risinger) Date: Tue, 1 Jul 2014 21:58:44 -0500 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> Message-ID: On Jul 1, 2014 8:06 PM, "Chris Angelico" wrote: > > [...] > > For the OP's use-case, though, it would actually be possible to abuse > slice notation. I don't remember this being mentioned, but it does > preserve order; the cost is that all the "keywords" have to be defined > as objects. > > class kw: pass # because object() doesn't have attributes > def make_kw(names): > for i in names.split(): > globals()[i] = obj = kw() > obj.keyword_arg = i > make_kw("Z R X") > > # Now you can use them in indexing > some_obj[5, Z:3] > some_obj[7, Z:3, R:4] > > The parameters will arrive in the item tuple as slice objects, where > the start is a signature object and the stop is its value. > > >>> some_obj[5, Z:3] > getitem: (5, slice(<__main__.kw object at 0x016C5E10>, 3, None)) > > Yes, it uses a colon rather than an equals sign, but on the flip side, > it already works :) This works great, IIRC you can pretty much pass *anything*: dict[{}:] dict[AType:lambda x: x] dict[::] dict[:] ...don't forget extended slice possibilities :) I've dabbled with this in custom dict implementations and it usefully excludes all normal dicts, which quickly reject slice objects. -- C Anthony [mobile] -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Wed Jul 2 09:06:53 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 2 Jul 2014 00:06:53 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B33800.1030300@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> Message-ID: On 1 July 2014 15:36, Stefano Borini wrote: > Dear all, > > after the first mailing list feedback, and further private discussion with > Joseph Martinot-Lagarde, I drafted a first iteration of a PEP for keyword > arguments in indexing. The document is available here. > > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt > > The document is not in final form when it comes to specifications. In fact, > it requires additional discussion about the best strategy to achieve the > desired result. Particular attention has been devoted to present alternative > implementation strategies, their pros and cons. I will examine all feedback > tomorrow morning European time (in approx 10 hrs), and apply any pull > requests or comments you may have. It's a well written PEP, but the "just use call notation instead" argument is going to be a challenging one to overcome. Given that part of the rationale given is that "slice(start, stop, step)" is uglier than the "start:stop:step" permitted in an indexing operation, the option of allowing "[start:]", "[:stop]","[start:stop:step]", etc as dedicated slice syntax should also be explicitly considered. Compare: a.get(slice(1,3), Z=2) # today a.get([1:3], Z=2) # slice sytax a[1:3, Z=2] # PEP Introducing a more general slice notation would make indexing *less* special (reducing the current "allows slice notation" special case to "allows slice notation with the surrounding square brackets implied". The reduction of special casing could be taken further, by allowing the surrounding square brackets to be omitted in tuple and list displays, just as they are in indexing operations. I'm not saying such a proposal would necessarily be accepted - I just see a proposal that takes an existing special case and proposes to make it *less* special as more appealing than one that proposes to make it even *more* special. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From nicholas.cole at gmail.com Wed Jul 2 09:45:47 2014 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Wed, 2 Jul 2014 08:45:47 +0100 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> Message-ID: > It's a well written PEP, but the "just use call notation instead" > argument is going to be a challenging one to overcome. > +1 The advantages the PEP suggests are very subjective ones to do with readability. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.borini at ferrara.linux.it Wed Jul 2 09:59:54 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Wed, 2 Jul 2014 09:59:54 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it>

Message-ID: <20140702075954.GA26500@ferrara.linux.it> On Wed, Jul 02, 2014 at 08:45:47AM +0100, Nicholas Cole wrote: > > It's a well written PEP, but the "just use call notation instead" > > argument is going to be a challenging one to overcome. > > > > +1 > > The advantages the PEP suggests are very subjective ones to do with > readability. I want to be honest, I agree with this point of view myself. it's not _needed_. it would be a nice additional feature but maybe only rarely used and in very specialized cases, and again, there are always workarounds. Even if rejected on the long run, it rationalizes and analyzes motivations and alternatives, and enshrines them formally on why it's a "not worth it" scenario. Thank you for all the feedback. I am including all the raised points in the PEP and I'll follow up with a revised version ASAP. Stefano Borini From stefano.borini at ferrara.linux.it Wed Jul 2 12:08:25 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Wed, 2 Jul 2014 12:08:25 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B37027.9030808@btinternet.com> References: <53B33800.1030300@ferrara.linux.it> <53B37027.9030808@btinternet.com> Message-ID: <20140702100825.GA29589@ferrara.linux.it> On Wed, Jul 02, 2014 at 03:36:23AM +0100, Rob Cliffe wrote: > 1) Ahem, doesn't a[3] (usually) return the *fourth* element of a ? Yes. I changed the indexes many times for consistency and that slipped through. It used to be a[2] > > low_accuracy_energy = computeEnergy(molecule, BasisSet[Z=3]) """ > > is about, and whether it is really (conceptually) related to indexing, > or just a slick hack. I guess it could be anything, depending on the > implementation of __getitem__. The reason behind an indexing is that the BasisSet object could be internally represented as a numeric table, where rows are associated to individual elements (e.g. row 0:5 to element 1, row 5:8 to element 2) and each column is associated to a given degree of accuracy (e.g. first column is low accuracy, second column is medium accuracy etc). You could say that users are not concerned with the internal representation, but if they are eventually allowed to create these basis sets in this tabular form, it makes a nice conceptual model to keep the association column <-> accuracy and keep it explicit in the interface. From xavier.combelle at gmail.com Wed Jul 2 13:47:03 2014 From: xavier.combelle at gmail.com (Xavier Combelle) Date: Wed, 2 Jul 2014 13:47:03 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <20140702100825.GA29589@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> <53B37027.9030808@btinternet.com> <20140702100825.GA29589@ferrara.linux.it> Message-ID: in this case: C1: a[Z=3] -> idx = {"Z": 3} # P1/P2 dictionary with single key as we can index with any object, I wonder how one could differency between the calls, a[z=3] and the actual a[{"Z":3}]. Do they should be return the same? -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.borini at ferrara.linux.it Wed Jul 2 14:20:03 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Wed, 2 Jul 2014 14:20:03 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> <53B37027.9030808@btinternet.com> <20140702100825.GA29589@ferrara.linux.it> Message-ID: <20140702122003.GA2183@ferrara.linux.it> On Wed, Jul 02, 2014 at 01:47:03PM +0200, Xavier Combelle wrote: > in this case: > > C1: a[Z=3] -> idx = {"Z": 3} # P1/P2 > dictionary with single key > > > as we can index with any object, I wonder how one could differency between > the calls, a[z=3] > and the actual a[{"Z":3}]. Do they should be return the same? indeed you can't, and if I recall correctly I wrote it somewhere. The point is eventually if such distinction is worth considering or if, instead, the two cases should be handled as degenerate (equivalent) notations. IMHO, they should be kept distinct, and this disqualifies that implementation strategy. Too much magic would happen otherwise. -- ------------------------------------------------------------ -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS d- s+:--- a? C++++ UL++++ P+ L++++ E--- W- N+ o K- w--- O+ M- V- PS+ PE+ Y PGP++ t+++ 5 X- R* tv+ b DI-- D+ G e h++ r+ y* ------------------------------------------------------------ From 4kir4.1i at gmail.com Wed Jul 2 17:14:47 2014 From: 4kir4.1i at gmail.com (Akira Li) Date: Wed, 02 Jul 2014 19:14:47 +0400 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments References: <53B33800.1030300@ferrara.linux.it> Message-ID: <878uobn66w.fsf@gmail.com> Stefano Borini writes: > Dear all, > > after the first mailing list feedback, and further private discussion > with Joseph Martinot-Lagarde, I drafted a first iteration of a PEP for > keyword arguments in indexing. The document is available here. > > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt > > The document is not in final form when it comes to specifications. In > fact, it requires additional discussion about the best strategy to > achieve the desired result. Particular attention has been devoted to > present alternative implementation strategies, their pros and cons. I > will examine all feedback tomorrow morning European time (in approx 10 > hrs), and apply any pull requests or comments you may have. > > When the specification is finalized, or this community suggests that > the PEP is in a form suitable for official submission despite > potential open issues, I will submit it to the editor panel for > further discussion, and deploy an actual implementation according to > the agreed specification for a working test run. > > I apologize for potential mistakes in the PEP drafting and submission > process, as this is my first PEP. > Strategy 3b: builtin named tuple C0. a[2] -> idx = 2; # scalar a[2,3] -> idx = (2, 3) # tuple idx[0] == 2 idx[1] == 3 C1. a[Z=3] -> idx = (Z=3) # builtin named tuple (pickable, etc) idx[0] == idx.Z == 3 C2. a[Z=3, R=2] -> idx = (Z=3, R=2) idx[0] == idx.Z == 3 idx[1] == idx.R == 2 C3. a[1, Z=3] -> idx = (1, Z=3) idx[0] == 1 idx[1] == idx.Z == 3 C4. a[1, Z=3, R=2] -> idx = (1, Z=3, R=2) idx[0] == 1 idx[1] == idx.Z == 3 idx[2] == idx.R == 2 C5. a[1, 2, Z=3] -> idx = (1, 2, Z=3) C6. a[1, 2, Z=3, R=4] -> (1, 2, Z=3, R=4) C7. a[1, Z=3, 2, R=4] -> SyntaxError: non-keyword arg after keyword arg Pros: - looks nice - easy to explain: a[1,b=2] is equivalent to a[(1,b=2)] like a[1,2] is equivalent to a[(1,2)] - it makes `__getitem__` *less special* if Python supports a builtin named tuple and/or ordered keyword args (the call syntax) Cons: - Python currently has no builtin named tuple (an ordered collection of named (optionally) values) - Python currently doesn't support ordered keyword args (it might have made the implementation trivial) Note: `idx = (Z=3)` is a SyntaxError so it is safe to produce a named tuple instead of a scalar. -- Akira From drekin at gmail.com Wed Jul 2 18:40:43 2014 From: drekin at gmail.com (drekin at gmail.com) Date: Wed, 02 Jul 2014 09:40:43 -0700 (PDT) Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B33800.1030300@ferrara.linux.it> Message-ID: <53b4360b.ebd3b40a.33ca.5890@mx.google.com> Hello, just some remarks: Ad degeneracy of notation: The case of a[Z=3] and a[{"Z": 3}] is similar to current a[1, 2] and a[(1, 2)]. Even though one may argue that the parentheses are actually not part of tuple notation but are just needed because of syntax, it may look as degeneracy of notation when compared to function call: f(1, 2) is not the same thing as f((1, 2)). Ad making dict.get() obsolete: There is still often used a_dict.get(key) which has to be spelled a_dict[key, default=None] with index notation. The _n keys used in strategy 3 may be indexed from zero like list indices. Regards, Drekin From joseph.martinot-lagarde at m4x.org Wed Jul 2 21:17:15 2014 From: joseph.martinot-lagarde at m4x.org (Joseph Martinot-Lagarde) Date: Wed, 02 Jul 2014 21:17:15 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it>

Message-ID: <53B45ABB.30802@m4x.org> Le 02/07/2014 09:45, Nicholas Cole a ?crit : > > It's a well written PEP, but the "just use call notation instead" > argument is going to be a challenging one to overcome. > > > +1 > > The advantages the PEP suggests are very subjective ones to do with > readability. Well, "Readability counts" is in the zen of python ! Having recently translated a Matlab program to python, I can assure you that the notation difference between call and indexing is really useful. .get() does not looks like indexing. From timothy.c.delaney at gmail.com Wed Jul 2 22:12:30 2014 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Thu, 3 Jul 2014 06:12:30 +1000 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B33800.1030300@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> Message-ID: On 2 July 2014 08:36, Stefano Borini wrote: > Dear all, > > after the first mailing list feedback, and further private discussion with > Joseph Martinot-Lagarde, I drafted a first iteration of a PEP for keyword > arguments in indexing. The document is available here. > > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt > > The document is not in final form when it comes to specifications. In > fact, it requires additional discussion about the best strategy to achieve > the desired result. Particular attention has been devoted to present > alternative implementation strategies, their pros and cons. I will examine > all feedback tomorrow morning European time (in approx 10 hrs), and apply > any pull requests or comments you may have. > > When the specification is finalized, or this community suggests that the > PEP is in a form suitable for official submission despite potential open > issues, I will submit it to the editor panel for further discussion, and > deploy an actual implementation according to the agreed specification for a > working test run. > > I apologize for potential mistakes in the PEP drafting and submission > process, as this is my first PEP. > One option I don't see is to have a[b=1, c=2] be translated to a.__getitem__((slice('b', 1, None), slice['c', 2, None)) automatically. That completely takes care of backwards compatibility in __getitem__ (no change at all), and also deals with your issue with abusing slice objects: a[K=1:10:2] -> a.__getitem__(slice('K', slice(1, 10, 2))) And using that we can have an ordered dict "literal" class OrderedDictLiteral(object): def __getitem__(self, t): try: i = iter(t) except TypeError: i = (t,) return collections.OrderedDict((s.start, s.stop) for s in i) odict = OrderedDictLiteral() o = odict[a=1, b='c'] print(o) # prints OrderedDict([('a', 1), ('b', 'c')]) On a related note, if we combined this with the idea that kwargs should be constructed using the type of the passed dict (i.e. if you pass an OrderedDict as **kwargs you get a new OrderedDict in the function) we could do: kw = OrderedDictLiteral() def f(**kw): print(kw) f('a', 'b', **kw[c='d', e=2]) always resulting in: {'c': 'd', 'e': 2} Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From timothy.c.delaney at gmail.com Wed Jul 2 22:14:00 2014 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Thu, 3 Jul 2014 06:14:00 +1000 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> Message-ID: On 3 July 2014 06:12, Tim Delaney wrote: > > a[K=1:10:2] -> a.__getitem__(slice('K', slice(1, 10, 2))) > Of course, that should have been: a[K=1:10:2] -> a.__getitem__(slice('K', slice(1, 10, 2), None)) Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.borini at ferrara.linux.it Wed Jul 2 23:29:53 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Wed, 2 Jul 2014 23:29:53 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> Message-ID: <20140702212953.GA16637@ferrara.linux.it> On Thu, Jul 03, 2014 at 06:12:30AM +1000, Tim Delaney wrote: > One option I don't see is to have a[b=1, c=2] be translated to > a.__getitem__((slice('b', 1, None), slice['c', 2, None)) automatically. it would be weird, since it's not technically a slice, but it would work. I personally think that piggybacking on the slice would appear hackish. One could eventually think to have a keyword() object similar to slice(), but then it's basically a single item dictionary (Strategy 1) with a fancy name. -- ------------------------------------------------------------ -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS d- s+:--- a? C++++ UL++++ P+ L++++ E--- W- N+ o K- w--- O+ M- V- PS+ PE+ Y PGP++ t+++ 5 X- R* tv+ b DI-- D+ G e h++ r+ y* ------------------------------------------------------------ From timothy.c.delaney at gmail.com Thu Jul 3 01:10:18 2014 From: timothy.c.delaney at gmail.com (Tim Delaney) Date: Thu, 3 Jul 2014 09:10:18 +1000 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <20140702212953.GA16637@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> Message-ID: On 3 July 2014 07:29, Stefano Borini wrote: > On Thu, Jul 03, 2014 at 06:12:30AM +1000, Tim Delaney wrote: > > One option I don't see is to have a[b=1, c=2] be translated to > > a.__getitem__((slice('b', 1, None), slice['c', 2, None)) automatically. > > it would be weird, since it's not technically a slice, but it would work. > I personally think that piggybacking on the slice would appear hackish. > One could eventually think to have a keyword() object similar to slice(), > but then it's basically a single item dictionary (Strategy 1) with a fancy > name. I really do think that a[b=c, d=e] should just be syntax sugar for a['b':c, 'd':e]. It's simple to explain, and gives the greatest backwards compatibility. In particular, libraries that already abused slices in this way will just continue to work with the new syntax. I'd maybe thought a subclass of slice, with .key (= .start) and and .value (= .stop) variables would work, but slice isn't subclassable so it would be a bit more difficult. That would also be backwards-compatible with existing __getitem__ that used slice, but would preclude people calling that __getitem__ with slice syntax, which I personally don't think is desireable. Instead, maybe recommend something like: ordereddict = OrderedDictLiteral() # using the definition from previous email class GetItemByName(object): def __getitem__(self, t): # convert the parameters to a dictionary d = ordereddict[t] return d['name'] Hmm - here's an anonymous named tuple "literal" as another example: class AnonymousNamedTuple(object): def __getitem__(self, t): d = ordereddict[t] t = collections.namedtuple('_', d) return t(*d.values()) namedtuple = AnonymousNamedTuple() print(namedtuple[a='b', c=1]) # _(a='b', c=1) As you can see, I'm in favour of keeping the order of the keyword arguments to the index - losing it would prevent things like the above. Tim Delaney -------------- next part -------------- An HTML attachment was scrubbed... URL: From ethan at stoneleaf.us Thu Jul 3 01:40:39 2014 From: ethan at stoneleaf.us (Ethan Furman) Date: Wed, 02 Jul 2014 16:40:39 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> Message-ID: <53B49877.6090304@stoneleaf.us> On 07/02/2014 04:10 PM, Tim Delaney wrote: > > I really do think that a[b=c, d=e] should just be syntax sugar for a['b':c, 'd':e]. It's simple to explain, and gives > the greatest backwards compatibility. In particular, libraries that already abused slices in this way will just continue > to work with the new syntax. +0.5 for keywords in __getitem__ +1 for this version of it ~Ethan~ From bruce at leapyear.org Thu Jul 3 09:37:45 2014 From: bruce at leapyear.org (Bruce Leban) Date: Thu, 3 Jul 2014 00:37:45 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B49877.6090304@stoneleaf.us> References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> Message-ID: On Wed, Jul 2, 2014 at 4:40 PM, Ethan Furman wrote: > On 07/02/2014 04:10 PM, Tim Delaney wrote: > >> >> I really do think that a[b=c, d=e] should just be syntax sugar for >> a['b':c, 'd':e]. It's simple to explain, and gives >> the greatest backwards compatibility. In particular, libraries that >> already abused slices in this way will just continue >> to work with the new syntax. >> > > +0.5 for keywords in __getitem__ > > +1 for this version of it If there weren't already abuse of slices for this purpose, would this be the first choice? I think not. This kind of abuse makes it more likely that there will be mysterious failures when someone tries to use keyword indexing for objects that don't support it. In contrast, using kwargs means you'll get an immediate meaningful exception. Tangentially, I think the PEP can reasonably reserve the keyword argument name 'default' for default values specifying that while __getitem__ methods do not need to support default, they should not use that keyword for any other purpose. Also, the draft does not explain why you would not allow defining __getitem__(self, idx, x=1, y=2) rather than only supporting the kwargs form. I don't know if I think it should or shouldn't at this point, but it definitely think it need to be discussed and justified one way or the other. --- Bruce Learn how hackers think: http://j.mp/gruyere-security https://www.linkedin.com/in/bruceleban -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.borini at ferrara.linux.it Thu Jul 3 19:00:36 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Thu, 3 Jul 2014 19:00:36 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> Message-ID: <20140703170036.GA13843@ferrara.linux.it> On Thu, Jul 03, 2014 at 09:10:18AM +1000, Tim Delaney wrote: > I really do think that a[b=c, d=e] should just be syntax sugar for a['b':c, > 'd':e]. It's simple to explain, and gives the greatest backwards > compatibility This is indeed a point, as the initialization for a dictionary looks very, very similar, however, it would definitely collide with the slice object. At the very least, it would be confusing. > In particular, libraries that already abused slices in this > way will just continue to work with the new syntax. Are there any actual examples in the wild of this behavior? From stefano.borini at ferrara.linux.it Thu Jul 3 19:15:09 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Thu, 3 Jul 2014 19:15:09 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <53B33800.1030300@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> Message-ID: <20140703171509.GB13843@ferrara.linux.it> On Wed, Jul 02, 2014 at 12:36:48AM +0200, Stefano Borini wrote: > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt I committed and pushed the most recent changes and they are now available. Some points have been clarified and expanded. Also, there's a new section about C interface compatibility. Please check the diffs for tracking the changes. Tonight I will comb the document and the thread again, further distilling the current hot spots. From shoyer at gmail.com Thu Jul 3 19:57:48 2014 From: shoyer at gmail.com (Stephan Hoyer) Date: Thu, 3 Jul 2014 10:57:48 -0700 (PDT) Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> Message-ID: <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> don't have strong opinions about the implementation, but I am strongly supportive of this PEP for the second case it lists -- the ability to index an a multi-dimensional array by axis name or label instead of position. Why? Suppose you're working with high dimensional data, where arrays may have any number of axes such as time, x, y and z. I work with this sort of data every day, as do many scientists. It is awkward and error prone to use the existing __getitem__ and __setitem__ syntax, because it's difficult to reliably keep track of axis order with this many indices: a[:, :, 0:10] vs. a[y=0:10] Keyword getitem syntax should be encouraged for the same reasons that keyword arguments are often preferable to positional arguments: it is both explicit (no implicit reliance on axis order), and more flexible (the same code will work on arrays with transposed or altered axes). This is particularly important because it is typical to be working with arrays that use some but not all the same axes. A method does allow for an explicit (if verbose) alternative to __getitem__ syntax: a.getitem(y=slice(0, 10)) But it's worse for __setitem__: a.setitem(dict(y=slice(0, 10)), 0) vs. a[y=0:10] = 0 ------------ Another issue: The PEP should address whether expressions with slice abbreviations like the following should be valid syntax: a[x=:, y=:5, z=::-1] These look pretty strange (=: looks like a form of assign), but the functionality would certainly be nice to support in some way. Surrounding the indices with [] might help: a[x=[:], y=[:5], z=[::-1]] ------------- On Thursday, July 3, 2014 12:39:09 AM UTC-7, Bruce Leban wrote: > > Tangentially, I think the PEP can reasonably reserve the keyword argument > name 'default' for default values specifying that while __getitem__ methods > do not need to support default, they should not use that keyword for any > other purpose. > -1 from me. The existing get method handles this case pretty well, with fewer keystrokes than the keyword only "default" index (as I think has already been pointed out). In my opinion, case 1 (labeled indices for a physics DSL) and case 2 (labeled indices to removed ambiguity) are basically the same, and the only use-cases that should be encouraged. Labeling tensor indices with names in mathematical notation is standard for precisely the same reasons that it's a good idea for Python. Best, Stephan (note: apologies for any redundant messages, I tried sending this message from the google groups mirror before I signed up, which didn't go out to the main listing list) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.borini at ferrara.linux.it Thu Jul 3 20:30:59 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Thu, 3 Jul 2014 20:30:59 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <20140703171509.GB13843@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> <20140703171509.GB13843@ferrara.linux.it> Message-ID: <20140703183059.GC13843@ferrara.linux.it> On Thu, Jul 03, 2014 at 07:15:09PM +0200, Stefano Borini wrote: > On Wed, Jul 02, 2014 at 12:36:48AM +0200, Stefano Borini wrote: > > https://github.com/stefanoborini/pep-keyword/blob/master/PEP-XXX.txt > > I committed and pushed the most recent changes and they are now available. > Some points have been clarified and expanded. Also, there's a new section about > C interface compatibility. Please check the diffs for tracking the changes. Forgot: I also added a possibility P4 for the first strategy: keyword (alternative name "keyindex") which was proposed in the thread. This solution would look rather neat >>> a[3] 3 >>> a[3:1] slice(3, 1, None) >>> a[slice(3,1,None)] # <- Note how this notation is a long and equivalent form of the slice(3, 1, None) # syntactic sugar above >>> a[z=4] # <- Again, note how this notation would be a syntactic sugar keyindex("z", 4) # for a[keyindex("z", 4)] >>> a[z=1:5:2] # <- Supports slices too. keyindex("z", slice(1,5,2)) # No ambiguity with dictionaries, and C compatibility is # straightforward >>> keyindex("z", 4).key "z" Another thing I observed is that the point of indexing operation is indexing, and a keyed _index_ is not the same thing as a keyed _option_ during an indexing operation. This has been stated during the thread but it's worth to point out explicitly in the PEPi (it isn't). Using it for options such as default would technically be a misuse, but an acceptable one for... broad definitions of indexing. The keyindex object could be made to implement the same interface as its value through forwarding, so it can behave just as its value if your logic cares only about position, and not key >>> keyindex("z", 4) + 1 5 Another rationalization: current indexing has only one degree of freedom, that is: positioning. Add keywords and now there are two degrees of freedom: position and key. How are these two degrees of freedom supposed to interact? From stefano.borini at ferrara.linux.it Thu Jul 3 21:33:56 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Thu, 3 Jul 2014 21:33:56 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> Message-ID: <20140703193356.GD13843@ferrara.linux.it> On Thu, Jul 03, 2014 at 10:57:48AM -0700, Stephan Hoyer wrote: > don't have strong opinions about the implementation, but I am strongly > supportive of this PEP for the second case it lists -- the ability to index > an a multi-dimensional array by axis name or label instead of position. thinking aloud. The biggest problem is that there's no way of specifying which labels the object supports, and therefore no way of binding a specified keyword, unless the __getitem__ signature is deeply altered. > It is awkward and error prone to use the existing __getitem__ and > __setitem__ syntax, because it's difficult to reliably keep track of axis > order with this many indices: > > a[:, :, 0:10] > > vs. > > a[y=0:10] This is indeed an important use case. I should probably stress it more in the PEP. > Another issue: The PEP should address whether expressions with slice > abbreviations like the following should be valid syntax: > > a[x=:, y=:5, z=::-1] looks ugly indeed > Surrounding the indices with [] might help: > > a[x=[:], y=[:5], z=[::-1]] better, but unusual > -1 from me. The existing get method handles this case pretty well, with > fewer keystrokes than the keyword only "default" index (as I think has > already been pointed out). > > In my opinion, case 1 (labeled indices for a physics DSL) and case 2 > (labeled indices to removed ambiguity) are basically the same, and the only > use-cases that should be encouraged. Labeling tensor indices with names in > mathematical notation is standard for precisely the same reasons that it's > a good idea for Python. Meaning dropping the use of keyword indexing for "options" use cases. From shoyer at gmail.com Thu Jul 3 21:43:59 2014 From: shoyer at gmail.com (Stephan Hoyer) Date: Thu, 3 Jul 2014 12:43:59 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <20140703193356.GD13843@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> Message-ID: On Thu, Jul 3, 2014 at 12:33 PM, Stefano Borini < stefano.borini at ferrara.linux.it> wrote: > > thinking aloud. > The biggest problem is that there's no way of specifying which labels the > object supports, and therefore no way of binding a specified keyword, > unless > the __getitem__ signature is deeply altered. I don't I follow you here. The object itself handles the __getitem__ logic in whatever way it sees fit, and it would be up to it to raise KeyError when an invalid label is supplied, much like the current situation with invalid keys. Stephan -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefano.borini at ferrara.linux.it Thu Jul 3 21:59:11 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Thu, 3 Jul 2014 21:59:11 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> Message-ID: <20140703195911.GE13843@ferrara.linux.it> On Thu, Jul 03, 2014 at 12:43:59PM -0700, Stephan Hoyer wrote: > On Thu, Jul 3, 2014 at 12:33 PM, Stefano Borini < > stefano.borini at ferrara.linux.it> wrote: > > > > thinking aloud. > > The biggest problem is that there's no way of specifying which labels the > > object supports, and therefore no way of binding a specified keyword, > > unless > > the __getitem__ signature is deeply altered. > > > I don't I follow you here. The object itself handles the __getitem__ logic > in whatever way it sees fit, and it would be up to it to raise KeyError > when an invalid label is supplied, much like the current situation with > invalid keys. NB: Still thinking aloud here... True, but the problem is that in a function def foo(x,y,z): pass calling the following will give the exact same result foo(1,2,3) foo(x=1, y=2, z=3) foo(z=3, x=1, y=2) this happens because at function definition you can specify the argument names. with __getitem__ you can't explain this binding. its current form precludes it __getitem__(self, idx) if you use a[1,2,3], you have no way of saying that "the first index is called x", so you have no way for these two to be equivalent in a similar way a function does a[1,2,3] a[z=3, x=1, y=2] unless you allow getitem in the form __getitem__(self, x, y, z) which I feel it would be a wasps' nest in terms of backward compatibility, both at the python and C level. I doubt this would fly. So if you want to keep __getitem__ signature unchanged, you will have to map labels to positions manuallyi inside __getitem__, a potentially complex task. Not even strategy 3 (namedtuple) would solve this issue. From shoyer at gmail.com Thu Jul 3 22:20:45 2014 From: shoyer at gmail.com (Stephan Hoyer) Date: Thu, 3 Jul 2014 13:20:45 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <20140703195911.GE13843@ferrara.linux.it> References: <53B33800.1030300@ferrara.linux.it> <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> <20140703195911.GE13843@ferrara.linux.it> Message-ID: On Thu, Jul 3, 2014 at 12:59 PM, Stefano Borini < stefano.borini at ferrara.linux.it> wrote: > So if you want to keep __getitem__ signature unchanged, you will have to > map labels > to positions manuallyi inside __getitem__, a potentially complex task. Not > even > strategy 3 (namedtuple) would solve this issue. > Yes, this is true. However, in practice many implementations of labeled arrays would have generic labeled axes, so they would need to use their own logic to do the mapping in __getitem__ anyways. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sturla.molden at gmail.com Thu Jul 3 22:48:06 2014 From: sturla.molden at gmail.com (Sturla Molden) Date: Thu, 3 Jul 2014 20:48:06 +0000 (UTC) Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments References: <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> <20140703195911.GE13843@ferrara.linux.it> Message-ID: <110367949426113052.835097sturla.molden-gmail.com@news.gmane.org> Stephan Hoyer wrote: > Yes, this is true. However, in practice many implementations of labeled > arrays would have generic labeled axes, so they would need to use their own > logic to do the mapping in __getitem__ anyways. If you are thiniking about Pandas, then each keyword should be allowed to take a slice as well. dataframe[apples=1:3, oranges=2:6] Sturla From shoyer at gmail.com Thu Jul 3 23:00:20 2014 From: shoyer at gmail.com (Stephan Hoyer) Date: Thu, 3 Jul 2014 14:00:20 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: <110367949426113052.835097sturla.molden-gmail.com@news.gmane.org> References: <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> <20140703195911.GE13843@ferrara.linux.it> <110367949426113052.835097sturla.molden-gmail.com@news.gmane.org> Message-ID: On Thu, Jul 3, 2014 at 1:48 PM, Sturla Molden wrote: > Stephan Hoyer wrote: > > > Yes, this is true. However, in practice many implementations of labeled > > arrays would have generic labeled axes, so they would need to use their > own > > logic to do the mapping in __getitem__ anyways. > > If you are thiniking about Pandas, then each keyword should be allowed to > take a slice as well. > > dataframe[apples=1:3, oranges=2:6] > Yes, I am indeed thinking about pandas and other similar libraries. Supporting slices with keywords would be essential. Stephan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Thu Jul 3 23:48:10 2014 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 3 Jul 2014 14:48:10 -0700 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> <20140703195911.GE13843@ferrara.linux.it> <110367949426113052.835097sturla.molden-gmail.com@news.gmane.org> Message-ID: On 3 July 2014 14:00, Stephan Hoyer wrote: > On Thu, Jul 3, 2014 at 1:48 PM, Sturla Molden > wrote: >> >> Stephan Hoyer wrote: >> >> > Yes, this is true. However, in practice many implementations of labeled >> > arrays would have generic labeled axes, so they would need to use their >> > own >> > logic to do the mapping in __getitem__ anyways. >> >> If you are thiniking about Pandas, then each keyword should be allowed to >> take a slice as well. >> >> dataframe[apples=1:3, oranges=2:6] > > > Yes, I am indeed thinking about pandas and other similar libraries. > Supporting slices with keywords would be essential. Some more concrete pandas-based examples could definitely help make a more compelling case. I genuinely think the hard part here is to make the case for offering the feature *at all*, so adding a "here is current real world pandas based code" and "here is how this PEP could make that code more readable" example could be worthwhile. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From stefano.borini at ferrara.linux.it Fri Jul 4 08:25:13 2014 From: stefano.borini at ferrara.linux.it (Stefano Borini) Date: Fri, 04 Jul 2014 08:25:13 +0200 Subject: [Python-ideas] PEP pre-draft: Support for indexing with keyword arguments In-Reply-To: References: <20140702212953.GA16637@ferrara.linux.it> <53B49877.6090304@stoneleaf.us> <44dd6995-1335-41aa-8ab0-8e1ae79f8818@googlegroups.com> <20140703193356.GD13843@ferrara.linux.it> <20140703195911.GE13843@ferrara.linux.it> <110367949426113052.835097sturla.molden-gmail.com@news.gmane.org>