Mailman 3 - Python-ideas

new operators via backquoting
by Chris Rebert Jan. 3, 2007

Jan. 3, 2007

In Haskell, foo `baz` bar means (baz foo bar), which translates to baz(foo, bar) in Python. This allows Haskell programmers to use functions as infix operators. If I recall correctly, in Py3k, enclosing something in backticks will no longer cause it to be repr()-ed, leaving the backtick without a meaning in Python. Thus, I propose one of the following as the new use for the backtick (`): [Note: In both, the characters between the backticks must be a valid Python identifier.] (A) `baz` is treated as an operator, named "baz", just as / is "div". foo `baz` bar thus causes python to try to call foo.__baz__(bar), and failing that, bar.__rbaz__(foo), and if both those fail, raise TypeError. This is, if I understand correctly, how the builtin operators work. (B) `baz` is a special way to call a callable. foo `baz` bar is translated to baz(foo, bar) with the standard lookup rules for resolving "baz" Example use cases, stolen from Haskell: The Craft of Functional Programming: 2 `max` 5 => 5 7 `cons` tail => ConsCell(val=7, next=tail) matrix1 `crossproduct` matrix2 => cross-product of the matrices [1, 2, 3] `zip` ['a', 'b', 'c'] => [[1, 'a'], [2, 'c'], [3, 'c']] I believe that this would improve the readability of code, such as Numeric, without going off the deep end and offering programmable syntax. - Chris Rebert

5 4

Attribute Docstrings and Annotations
by Tony Lownds Jan. 3, 2007

Jan. 3, 2007

I'd like to propose annotations and docstrings on attributes for Python 3000 with the following Grammar and properties: expr_stmt: test (':' test ['=' (yield_expr|testlist)] | augassign (yield_expr|testlist) | [',' testlist] ('=' (yield_expr|testlist))* ) * annotations can appear for attributes that are not defined yet * code to generate and populate __annotations__ and __attrdoc__ would appear for all modules and class bodies, not for functions. * attribute annotations allow a name as target only * attribute annotations without assignment are illegal for functions * attribute annotations with assignment should probably be illegal for functions * docstring annotations only apply to the first target, and only if it is a name * docstring annotations do not apply to augmented assignments * docstring and annotations on functions do not get populated in __annotations__ * the class docstring is not reused as a function docstring The basic rationale for annotations on attributes is completeness with PEP3107. I'm proposing attribute docstrings as well because I think it's preferable to have a spot for documentation that isn't tied to annotations, like functions' __doc__. Also attribute docstrings as specified look similar to function docstrings; both have statements consisting of only a string that is taken as documentation. Here is an interactive session and the code that might be generated. What do you think? Thanks -Tony >>> class X: ... "class docstring" ... foo: 1 = 1 ... bar: 2 ... "attribute docstring" ... attr = None ... "another attribute docstring" ... fields = __slots__ = ['fields', 'attr'] ... "docstring ignored" ... x, y = 1, 2 ... >>> X.__attrdoc__ {'fields': 'another attribute docstring', 'attr': 'attribute docstring'} >>> X.__annotations__ {'foo': 1, 'bar': 2} >>> X.foo 1 >>> X.bar Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: type object 'X' has no attribute 'bar' >>> def f(): ... x: 1 ... File "<stdin>", line 2 SyntaxError: annotation without assignment not allowed in function 2 0 LOAD_NAME 0 (__name__) 3 STORE_NAME 1 (__module__) 6 BUILD_MAP 0 9 STORE_NAME 2 (__annotations__) 12 BUILD_MAP 0 15 STORE_NAME 3 (__attrdoc__) 3 18 LOAD_CONST 0 ('class docstring') 21 STORE_NAME 4 (__doc__) 5 24 LOAD_CONST 1 ('attribute docstring') 27 LOAD_NAME 3 (__attrdoc__) 30 LOAD_CONST 2 ('attr') 33 STORE_SUBSCR 34 LOAD_NAME 5 (None) 37 STORE_NAME 6 (attr) 6 40 LOAD_CONST 3 (1) 48 STORE_NAME 7 (foo) 44 LOAD_CONST 3 (1) 51 LOAD_NAME 2 (__annotations__) 54 LOAD_CONST 4 ('foo') 57 STORE_SUBSCR 7 58 LOAD_CONST 5 (2) 61 LOAD_NAME 2 (__annotations__) 64 LOAD_CONST 6 ('bar') 67 STORE_SUBSCR 9 68 LOAD_CONST 7 ('another attribute docstring') 71 LOAD_NAME 3 (__attrdoc__) 74 LOAD_CONST 8 ('fields') 77 STORE_SUBSCR 78 LOAD_CONST 8 ('fields') 81 LOAD_CONST 2 ('attr') 84 BUILD_LIST 2 87 DUP_TOP 88 STORE_NAME 8 (fields) 91 STORE_NAME 9 (__slots__) 94 LOAD_LOCALS 95 RETURN_VALUE

6 7

Re: [Python-ideas] [Python-3000] PEP 3107 Function Annotations: overloadable ->
by Josiah Carlson Jan. 2, 2007

Jan. 2, 2007

To Tony and Kay, my short answer is: use __returns__ . Moving to python-ideas as per Guido's request: "Guido van Rossum" <guido(a)python.org> wrote: > This is sufficiently controversial that I believe it ought to go to > python-ideas first. If it comes to a PEP it should be a separate one > from PEP 3107. > On 1/1/07, Talin <talin(a)acm.org> wrote: > > Tony Lownds wrote: > > >> From: Tony Lownds <tony(a)pagedna.com> > > > What do people here think? > > 1) Normally, we don't name operators based on their shape - we don't > > call '/' the __slash__ operator, for example, nor do we call '|' the > > "__vbar__" operator. Certainly, but those two operators, and basically every other operator used in Python have long-standing semantics in basically every language that Python is even remotely related to. > > 2) I think that all operators should have a "suggested semantic". When > > someone overloads the '+' operator, its a good bet that the meaning of > > the overload has something to do with addition or accumulation in a > > general sense. This won't *always* be true, but it will be true often > > enough. I don't buy your "suggested semantic" argument. And even if I did, operator overloading allows people to choose the semantics of operations for themselves; suggesting a semantic for an operation would be a set of documentation that would never be read, and if it was read, ignored. > > But an arbitrary operator with no guidelines as to what it means is > > anyone's guess; It means that when we see a '->' operator embedded in > > the code, we have no idea what is being said. Ahh, but in this case there is precisely one place where the '->' operator is planned on being found for Py3k: def <name>(<arglist with or without annotations>) -> <annotation>: <body> In that sense, we don't need a fcn_obj.__becomes__(<annotation>) method, and that wasn't what the discussion was about, it was "what should the attribute be called for this already agreed upon *function annotation*?". Now, because this *particular* annotation was created to allow for the annotation of "returns", I agree with Kay's last suggestion, the attribute should be called __returns__, as that is *exactly* what the annotation was meant to convey. > > From an HCI perspective, punctuation symbols improve code readability > > only if their meanings are familiar to the reader; An operator whose > > meaning is constantly changing is a hindrance to readability rather than > > a help. Claiming "we want a particular operation to always refer to the same method/attribute" is only applicable if a particular operation has a chance of meaning more than one thing. Currently it means *exactly* one thing in the context of Python, 'this function returns X', so in my opinion, your argument isn't applicable. If you can manage to convince more people in python-ideas of arbitrary operations (as per your previous message(s) on the subject), and/or you can convince Guido to say "I would like more operations in Python", then your argument is applicable. However, I don't believe that you will be able to convince Guido that the large set of operations that you have previously posted about would be a good idea, and I certainly don't believe it would happen with sufficient time to make it into the first Py3k release. - Josiah

6 10

copyable iterators, named loops, continue a expression per indentation and a few functional things
by David Blaschke Dec. 27, 2006

Dec. 27, 2006

Using break for parent, grandparent, etc. only works until someone modifies the code incorrectly. It is a problem waiting to happen. Mathias' suggestion breaks the outer loop, but a slight modification allows you to break any one of the loops "Well this is a good way to break the outer most loop without any new syntax:" cond1 = 1 cond2 = 1 cond3 = 1 while cond1: while cond2: while cond3: if cond4: cond2 = 0 ## break second loop, etc

2 1

Python as meta-language
by Talin Dec. 25, 2006

Dec. 25, 2006

One of my long-standing interests is in mini-languages, particularly declarative languages. I'm always looking out for examples where a declarative language is used to represent some idea or concept that is not easily written in an imperative language. Examples are the behaviors of particle systems, kinematic constraints, formalized grammars, logical inferencing systems, query languages, and so on. In other words, you are using a language to describe a complex set of relationships, but you aren't giving specific commands to execute in a specific order. Often such mini-languages are implemented by writing a custom parser. However, often this is not necessary if the underlying language (such as Python) is flexible enough. Python's ability to declare complex structures as literals, combined with its ability to overload operators, means that one can often embed the mini-language within the Python syntax itself, and use the Python compiler as the parser for your mini language. It also allows you a convenient means to "escape" back into the procedural world when needed. Examples of the kind of things I am talking about include the SConstruct file format from SCONS and the SQLBuilder syntax from SQLObject. And although it's not directly related to Python, JSON has a lot of the same ideas - that is, using a scripting language source code as an efficient representation of complex data structures. And these are just a few of the many examples out there. What I'd be interested in doing, in this python-ideas list, is brainstorming some ideas for how we can improve Python's ability to 'host' other kinds of mini-languages within the Python syntax. We can start perhaps by examining some of the use cases I listed in the first paragraph - particle systems, etc - and see how one would want to represent those kinds of semantic structures within Python. Of course, there are some languages (Lisp and Dylan come to mind), which are even more flexible in this regard - the languages can be 'morphed' out of all recognition to the original syntax. (For example, in Dylan, a 'macro' was not a simple textual substitution as in C, but in fact added new production rules to the parser.) In I'm in no way advocating such a course. (Well, at least not at this moment :) ) I certainly recognize that there is a danger in making a language too 'plastic', in that it can easily be obfuscated with too much cleverness and lose it's identity. So I'm more interested in ideas that are subtle yet powerful. I'm sure that there are lots of approaches to this general concept. I'm going to throw out a couple of ideas, but I am going to post them as separate replies to this email, and not right away - the reason is, I don't want this thread to be taken over by the discussion / criticism of those specific ideas, when I'm more interested in brainstorming the general concept. -- Talin

3 6

copyable iterators, named loops, continue a expression per indentation and a few functional things
by Mathias Panzenböck Dec. 25, 2006

Dec. 25, 2006

Copyable iterators ------------------ There are a few ways how you could implement this. However, it only makes sense for iterators, not for generators! Way 1: Copyable iterators have a copy-method: >>> L = [1,2,3,4] >>> it1 = iter(L) >>> it1.next() 1 >>> it2 = it1.copy() >>> it2.next() 2 >>> it1.next() 2 Way 2: Copyable iterators will be copied when you call it's __iter__-method: >>> L = [1,2,3,4] >>> it1 = iter(L) >>> it1.next() 1 >>> it2 = iter(it1) >>> it2.next() 2 >>> it1.next() 2 In way 1 you could check for copy ability per hasattr(it,"copy"). Named loops ----------- With named loops you could break or continue other than only the inner most loop. I'm not sure about a syntax, though. for x in X as loop_a: for y in Y as loop_b: if cond1(): break loop_a elif cond2(): continue loop_a elif cond3(): break # breaks loop_b elif cond4(): continue # continues loop_b Auto-curryfication for operators -------------------------------- I like haskells feature for curryfication of operators. I think this would be nice to have in python. You could write instead of 'operator.add' the much shorter '(+)'. Or even '(2+)' works. >>> map((2+),[1,2,3,4]) [3,4,5,6] >>> filter((>=2),[1,2,3,4]) [2,3,4] __add__-method for functions ---------------------------- In haskell you can write 'f . g . h', which would be the same like '(\a -> f(g(h(a))))' (in python: 'lambda a: f(g(a))'). In python it would also be nice to have such a functionality. It could be implemented with a __add__-method. Here is this functionality simulated with functors: >>> class FunctionBase(object): def __add__(self,other): return lambda *args,**kwargs: self(other(*args,**kwargs)) >>> class F(FunctionBase): def __call__(self,s): return s+" foo" >>> class G(FunctionBase): def __call__(self,s): return s+" bar" >>> f = F() >>> g = G() >>> (f + g)("x") 'x bar foo' >>> (g + f)("x") 'x foo bar' What do you think? panzi

8 10

Improving Pydoc
by Ron Adam Dec. 23, 2006

Dec. 23, 2006

Improving pydoc has been suggested before by me and others. I've been working on a version that is probably 80% done and would like to get feed back at this point to determine if I'm approaching this in the best way. Basically pydoc.py consists of over 2,000 lines of python code and is not well organized inside which means (my thoughts) it will pretty much stay the way it is and not be as useful as it could be. Currently pydoc.py has the following uses. * It is imported and used as pythons console help function. * It can be used to generate help text files. * It can open a tkinter search index and from that launch a web server and a browser to veiw a html help page. * It can be used to generate static html help files. It looks (to me) like splitting it into two modules would be good. One module for just the text help and introspection functions, and the other for the html server and html output stuff. 1. pyhelp.py - Pythons help function from inside the console, and running it directly would open an interactive text help session. 2. _pydoc.py - Python html help browser. This starts an html server and opens a web page with a modules/package index. The html page headers would contain the current Python version info and the following inputs fields. * get field - directly bring up help on an object/module/or package. * Search field - returns a search results index. * Modules button - modules index page * Keywords button - keywords index page * Help button - pydoc Help instructions, version, and credits info. Note: The leading underscore "_pydoc.py" is to keep it from clashing with the current pydoc version. It probably should be something else. An additional feature is clicking on a filename opens up a numbered source listing. Which is nice for source code browsing and for referencing specific lines of code in python list discussions. ;-) The colors, fonts and general appearance can be changed by editing the style sheet. The output is readable as plain (outline form) text if the style sheet is ignored by the browser. _pydoc.py imports pyhelp and uses it to do the introspection work and extends classes in pyhelp to produce html output. I've tried to make pyhelp.py useful in a general way so that it can more easily be used as a base that other output formats can be built from. That's something that can't be done presently. These improvements to pydoc mean you can browse pythons library dynamically without ever leaving the web browser. Currently you switch back and forth between the browser and a tkinter index window. Something I found to be annoying enough to discourage me from using pydoc. The version I'm working on is split up into eight python files, each addressing a particular function of pydoc. That was to help me organize things better. These will be recombined into fewer files. Some parts of it could be moved to other modules if they seem more generally useful. For example, the console text pager could be used in many other console applications. Things that still need to be done are adding the object resolution order output back in. And adding inter-page html links back in. And a few other things I just haven't gotten to yet. I got a bit burned out on this project a while back, and then moved to a new house.. etc.. etc.. But I'm starting to have more time, and with the current discussion s turning on to refining pythons library this seems like it would be a useful tool in that effort. Any comments on this general approach? Any suggestions, questions, or comments? I'll post a link to the zipped files in the next day or two and announce it here. I need to look into a glitch on my computer first that's probably a windows path/name problem. I don't think it's anything that needs to be fixed in the files but I want to be sure. Cheers, Ron Adam

3 5

bitwise operations on bytes
by Terry Carroll Dec. 22, 2006

Dec. 22, 2006

I'll start. I would like to be able to have the capability of doing bitwise operations on a string-like field. For this example, I'll use strings, but as I understand it, the planned byte type would probably be more appropriate. An example. Suppose I have a string (say, read in from a binary file) that has a 4-byte field in it. Bit 11 indicates whether the accompanying data is in glorped form (something I'm just making up for this example). For example, if the field has 0x000480c4, the data is not glorped. If the data is glorped, bit 11 would be on, i.e., 0x001480c4. Let's say I want to turn on the glorp bit; what I have to do now: GLORPED = 0x10 newchar = chr(ord(flags[1]) | GLORPED) flags = flags[0] + newchar + flags[2:] (messy) What I'd like to be able to do is something like: GLORPED = "x\00x\10x\00\x00" flags = flags | GLORPED I've actually run into this, on a read-only side, in reading and interpreting MP3 files. As I said, this is probably better for a bytes object than string.

2 1

The point of this list
by Brett Cannon Dec. 20, 2006

Dec. 20, 2006

I have gotten some emails from people saying how they expect this list to be the Wild West of Python propoals. I personally don't think it will be nor do I want it to be. I view this list as a stop-gap between python-dev/python-3000 and comp.lang.python. Totally wild ideas that have no hope of being accepted in the language (e.g., add curly braces) should not be brought up here. c.l.pyis a more proper place to start flame wars. Plus c.l.py is usually a good place to vent initial ideas anyway. I view this list as a staging area for ideas that actually have some form of a chance to either be turned into a solid proposal to take to the Python committers or to reject them and have the archives act as a place to point people to why. Basically I expect people to get redirected here from python-dev and python-3000 to gestate an idea before going to those lists to get approval to add it to the language. In other words, I have no problem telling people their idea is ridiculous and to stop talking about it. I am hoping we don't have to go that far and we can give people a fair chance to bring up ideas that they come up with. But if this list gets too ridiculous I have no problem policing it or even shutting the list down. I doubt it will come to that, though. =) -Brett

1 0