Mailman 3 July 2009 - Python-ideas

Heap data type
by Facundo Batista 07 Dec '09

07 Dec '09

Using the module heapq, I found that it's not easy to use. Or, at least, that it could be straightforward to use. My main issue is that for correct usage: - user should provide an external list, that shouldn't use for anything else to don't break the invariant - to alter the heap queue, a bunch of functions must be used, passing always the external list I think that changing "external list" for "internal attribute", and "bunch of functions " for "methods", it will leave the module easier to use, safer, and more object oriented. So, I basically coded it. What do you think about including this class in the heap module? """ class Heap(object): '''Maintains a heapified list, always conserving the invariant. Heaps are arrays for which heap[k] <= heap[2*k+1] and heap[k] <= heap[2*k+2] for all k, counting elements from zero. ''' def __init__(self, iterable=None): '''Initializes the heap from any iterable. >>> Heap([1, 2]) Heap([1, 2]) >>> Heap([]) Heap([]) >>> Heap() Heap([]) >>> Heap((1, 2, 3)) Heap([1, 2, 3]) >>> Heap(x**2 for x in range(3)) Heap([0, 1, 4]) ''' if iterable is None: self._queue = [] else: self._queue = list(iterable) heapq.heapify(self._queue) def __repr__(self): return "Heap(%s)" % self._queue def push(self, item): '''Push the item to the heap queue. >>> h = Heap() >>> h.push(5) >>> h.push(4) >>> h.push(3) >>> h.push(2) >>> h.push(1) >>> h.push(0) >>> h Heap([0, 2, 1, 5, 3, 4]) ''' heapq.heappush(self._queue, item) def pop(self): '''Pop one item from the heap queue. >>> h = Heap([0, 2, 1, 5, 3, 4]) >>> h.pop() 0 >>> h.pop() 1 >>> h.pop() 2 >>> h.pop() 3 >>> h.pop() 4 >>> h.pop() 5 >>> h Heap([]) >>> h.pop() Traceback (most recent call last): ... IndexError: index out of range ''' return heapq.heappop(self._queue) def pushpop(self, item): '''Push the item and pop one from the heap queue. Note that this method is more efficient than calling both push() and pop() separately. >>> h = Heap() >>> h.pushpop(7) 7 >>> h.push(5) >>> h.push(4) >>> h.push(3) >>> h.pushpop(7) 3 >>> h.pushpop(7) 4 >>> h Heap([5, 7, 7]) ''' return heapq.heappushpop(self._queue, item) def replace(self, item): '''Pop one item and push the received one into the heap queue Note that this method is more efficient than calling both pop() and push() separately. >>> h = Heap() >>> h.replace(3) Traceback (most recent call last): ... IndexError: index out of range >>> h.push(7) >>> h.replace(2) 7 >>> h Heap([2]) >>> h.push(3) >>> h Heap([2, 3]) >>> h.replace(1) 2 >>> h.replace(9) 1 >>> h Heap([3, 9]) ''' return heapq.heapreplace(self._queue, item) """ Regards, -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

11 14

Python Bytecode Verifier
by Kornél Pál 20 Sep '09

20 Sep '09

Hi, I've created a Python Bytecode Verifier in Python. I'm not a Python guru so I borrowed coding patterns from C/C++. I also created this with C portability in mind. The only reason I used Python was to experiment with Python and was easier to morph code during development. If this program were ported to C it would only need 8 bytes per opcode (some additional storage to track blocks) and a single pass. I haven't found backward jumps to previously unused code in any compiled Python code but it can easily be supported. In that case some more partial passes are required. I also was able to successfully verify all Python files in the Python-2.5.2 source package. The verification algorythm should be quite complete but I may have missed some limitations of the interpreter that could be checked by the verifier as well. The ability to create this verfier proved that although Python bytecode is designed for a dynamically typed interpreter, is still easily verifiable. I am willing port this code C but only in the case if there is any chance to be included in Python. I believe that Python in general would benefit having the ability to safely load .pyc files and create code objects on the fly. Both Java and .NET have the ability to safely load compiled byte code. .NET Framework, just like Python also has the ability to create and execute new code at run-time. You may feel that enabling closed source applications and/or creating a multi-language runtime would hurt Python but both of these have contributed to the success of Java (both the language and the runtime). Kornél

2 2

os.listdir with current working directory as default
by Tarek Ziadé 01 Aug '09

01 Aug '09

Hello, A very small change: what about making the path argument optional for os.listdir ? and use the current working directory if not provided. listdir(...) listdir(path=None) -> list_of_strings Return a list containing the names of the entries in the directory. path: path of directory to list. If path is None, os.getcwd() is used. The list is in arbitrary order. It does not include the special entries '.' and '..' even if they are present in the directory. Cheers Tarek -- Tarek Ziadé | http://ziade.org

11 18

Changing the default install location, script versioning (Packages and PEP 370)
by Jesse Noller 25 Jul '09

25 Jul '09

So, I've finally been able to kick my python 2.5 boat anchor, and I'm now relishing in the delightful future of 2.6. Two things struck me though, digging into PEP 370's (per user site-packages) behavior. The first is this - when you install a package (say, pip) into the .local directory via the --user command, the package ends up in the correct lib directory; but the binary get dropped into ./local/bin - this seems wrong. The reason being, is that what if I also have python 2.7 (which i do) installed, as well as python 3.1 and the release-which-will-not-be-named (3.0) - if I install that same package into one of the other versions, a new binary would be written with the same name - a script-pythonversion might also be installed, but the fact that the original script was overwritten seems broken to me. I think the "best" fix for this is to make the bin/ directory mirror the lib layout - each version would get it's own bin directory: .local/ bin/ python2.6/ python3.1/ lib/ python2.6/ python3.1/ Of course, doing this begs the question - why have /bin and /lib be top-level directories, instead favoring a layout which looks like: .local/ python2.6/ bin/ lib/ The data and doc directories which some packages might include should follow the same convention, to avoid conflicts in those directories as well. The second behavior is more of a "man I wish for a unicorn" type of thing - I think that --user should be removed, and by removed, I mean made the default action when "python setup.py install" is invoked. Users should need to instead pass in a --global flag to alter the behavior to install into the system's site-packages and bin directories. The reason is simple - installation to the global site is typically a super-user task, and can side-effect all users, it's growing into more of a no-no as more OS vendors grow to rely on the packaged/included versions of python. .002 cents jesse

9 21

universal set object for use in set manipulation
by Andy Kish 25 Jul '09

25 Jul '09

Hi all, I think the addition of a universal set object would be a nice touch for python sets. Manipulation of collections of sets becomes much easier with it. Right now, certain operations are easy because empty sets are easy to create while the dual operation requires special cases: set_union = set() for s in sets: set_union |= s # this doesn't work set_intersection = set(???) for s in sets: set_intersection &= s Instead you have to do something contorted like: if len(sets) == 0: set_intersection = set() else: sets_i = iter(sets) set_intersection = sets_i.next() for s in sets: set_intersection &= s Implementation of a universal set would be pretty trivial. Trails of EasyExtend [1] has an implementation (albeit used a bit differently) that's basically the entirety of what's needed. The above intersection case would end up looking something like: set_intersection = set.universal() for s in sets: set_intersection &= s Thoughts? I'm happy to write the patch myself if people like this idea. Thanks, Andy Kish. [1] http://fiber-space.de/wordpress/?p=322

14 20

Re: [Python-ideas] universal set object for use in set manipulation
by Terry Jones 25 Jul '09

25 Jul '09

>>>>> "Nick" == Nick Coghlan <ncoghlan(a)gmail.com> writes: Nick> Greg Ewing wrote: >> Andy Kish wrote: >> >>> It's really annoying to having appropriate identity element for >>> union built in while missing the *correct* identity element for >>> intersection. >> >> What would >> >> for x in set.universal(): >> ... >> >> do? Nick> Raise an exception, just like a signalling NaN in decimal. Agreed. The code I stuck on PyPI today raises InfiniteSetError on __len__ and __iter__. Nick> An expanded set concept with support for complementary set Nick> definitions is definitely something that should cook on PyPI for a Nick> while though. Just to repeat the URL: http://pypi.python.org/pypi/compset/0.1 It's had exactly zero downloads :-) Terry

2 1

Re: [Python-ideas] universal set object for use in set manipulation
by Terry Jones 24 Jul '09

24 Jul '09

>>>>> "Terry" == Terry Reedy <tjreedy(a)udel.edu> writes: Terry> Did you register a module with PyPI? OK, I put my original set code into http://pypi.python.org/pypi/compset/0.1 It's just a quick hack, but it works. I didn't implement all the ideas I suggested in http://mail.python.org/pipermail/python-dev/2006-May/064977.html (e.g., universeExclusionFunc) but they'd be easy to add. Hopefully someone can take that code as a starting point for a more fully fleshed out package. The test suite could do with some love too. Terry

1 0

Re: [Python-ideas] universal set object for use in set manipulation
by Terry Jones 24 Jul '09

24 Jul '09

>>>>> "Terry" == Terry Reedy <tjreedy(a)udel.edu> writes: Terry> Terry Jones wrote: >> See also this thread: >> >> http://mail.python.org/pipermail/python-dev/2006-May/064977.html >> >> I implemented some of what I was proposing there (in Python though, not C), >> and am happy to send people the code if there's any interest. [snip] Terry> Did you register a module with PyPI? No. Terry> In any case, it seems to me that adding a 'universal set' to the set Terry> class does not work. On the otherhand, a separate set-complement Terry> class (which I am guessing is what you did) should work fine as far Terry> as it goes. Each instance would have normal set as its data member, Terry> and operations on complements and mixed operations would be defined Terry> in terms of operations on sets. A universal set would then be an Terry> instance with an empty set for a complement. Yes, that's all exactly what I did. I'll clean it up a little and post it somewhere, PyPI I guess. BTW, in my original posting I was wondering about this being added to Python as a generalization of the set type. After the discussion in the above thread and thinking through the various oddities it would introduce, I also concluded that would be a mistake - there's just too much that's a bit weird. Raymond is right that a strong virtue of the current set module is its simplicity. It's not worth blowing that so as to add a more general behavior that very few people will ever use. But it would be a nice 3rd party module. Terry

1 0

Proposed convenience functions for re module
by Steven D'Aprano 24 Jul '09

24 Jul '09

Following the thread "Experiment: Adding "re" to string objects.", I would like to propose the addition of two convenience functions to the re module: def multimatch(s, *patterns): """Do a re.match on s using each pattern in patterns, returning the first one to succeed, or None if they all fail.""" for pattern in patterns: m = re.match(pattern, s) if m: return m def multisearch(s, *patterns): """Do a re.search on s using each pattern in patterns, returning the first one to succeed, or None if they all fail.""" for pattern in patterns: m = re.search(pattern, s) if m: return m The rationale is to make the following idiom easier: m = re.match(s, pattern1) if not m: m = re.match(s, pattern2) if not m: m = re.match(s, pattern3) if not m: m = re.match(s, pattern4) if m: m.group() which will become: m = re.multimatch(s, pattern1, pattern2, pattern3, pattern4) if m: m.group() Is there any support or objections to this proposal? Any comments? -- Steven D'Aprano

14 28

Re: [Python-ideas] universal set object for use in set manipulation
by Raymond Hettinger 24 Jul '09

24 Jul '09

[Andy Kish] > set_intersection = set.universal() > for s in sets: > set_intersection &= s -1 This complicates the API and the implementation for very little benefit (saving you from writing a tiny, clear helper function for an uncommon use case). Currently, sets enjoy a near zero learning curve. That would be lost by adding set.universal() whose semantics are not immediately obvious -- for example, with s=set.universal() what is the meaning of list(s) or frozenset(s) or s.pop(); what is its repr value; and what is s^set('abc') or s-set('abc') or other operations? You may be able to come-up with definitions that work, but those won't be intuitive to most users. Raymond

2 1

Re: [Python-ideas] universal set object for use in set manipulation
by Terry Jones 23 Jul '09

23 Jul '09

See also this thread: http://mail.python.org/pipermail/python-dev/2006-May/064977.html I implemented some of what I was proposing there (in Python though, not C), and am happy to send people the code if there's any interest. These things can't be 100% clean (e.g., what happens when you call len on an infinite set), but you can still get a lot of useful functionality if what's provided matches your use case. Having efficient support for very large sets via complements, having a Universal set, etc., would all be nice. Anyway, see the above thread for more discussion / opinions. Terry

2 1

universal set object for use in set manipulation
by Andy Kish 23 Jul '09

23 Jul '09

Hi all, I think the addition of a universal set object would be a nice touch for python sets. Manipulation of collections of sets becomes much easier with it. Right now, certain operations are easy because empty sets are easy to create while the dual operation requires special cases: set_union = set() for s in sets: set_union |= s # this doesn't work set_intersection = set(???) for s in sets: set_intersection &= s Instead you have to do something contorted like: if len(sets) == 0: set_intersection = set() else: sets_i = iter(sets) set_intersection = sets_i.next() for s in sets: set_intersection &= s Implementation of a universal set would be pretty trivial. Trails of EasyExtend [1] has an implementation (albeit used a bit differently) that's basically the entirety of what's needed. The above intersection case would end up looking something like: set_intersection = set.universal() for s in sets: set_intersection &= s Thoughts? I'm happy to write the patch myself if people like this idea. Thanks, Andy Kish. [1] http://fiber-space.de/wordpress/?p=322

1 0

Re: [Python-ideas] Experiment: Adding "re" to string objects.
by Nick Coghlan 22 Jul '09

22 Jul '09

<switching lists from python-dev to python-ideas> Michael Foord wrote: > MRAB wrote: >> [snip] >> Why not drop the ".re" part? You would, however, then need a new name >> for the re split, eg "re_split". >> >> Or you could make the string the pattern, eg r'whatever(.*)'.match(s). > > +1 for re support built-in to strings. The idea of adding a mutable attribute to strings (even if it plays no part in string equality) sends shivers down my spine. It also seems like it would be difficult to do this in a way that didn't increase the size of all strings by at least one pointer slot. However, the idea of adding more convenience classes to the re module may still have some merit. In particular, when checking against multiple regexes, being able to do something like the following might be helpful: m = re.Matcher(s) if m.match(r'whatever(.*)'): print m.group(1) elif m.match(r'something (.*) else(.*)'): print m.group(2) Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia ---------------------------------------------------------------

10 18

2c about package directories
by Tennessee Leeuwenburg 22 Jul '09

22 Jul '09

Hi all, Apologies if any of this has been said before. I've been loosely following the ongoing discussions about packaging utilities, plus a few people's experiences with attempting to manage their own Python environments. I was thinking about a potential directory layout approach which might be attractive to some people and which I think could be useful for automatic packaging. /usr/local/bin (or whatever the system prefix is) python --> 'standard python version' python2.6 --> 'standard python 2.6' python2.7 python3.1 etc /usr/local/lib/python (or whatever the system prefix is) /python2.6 /core /site-packages /applications /application-name-packages /python2.7... etc /home/username/.python/ /python2.6 /user-packages /applications /application-name-packages With that layout, it should be possible for: -- Anyone to always be able to run a system python installation with 'core' libraries only -- Run a system python installation with 'sitewide' libraries -- Deploy application-specific libraries to a system python installation

1 0

Proposal for extending the collections module - bags / multisets, ordered sets / unique lists
by Michael Lenzen 21 Jul '09

21 Jul '09

In a nutshell, I want to add 2 new classes (and respective frozen varients) to the collections module, namely a bag (multiset) class and a setlist (ordered set/unique list) class. I know this has been floated before, but I think it merits more consideration. Everything I'm writing here is available with better formatting here: http://code.google.com/p/python-data-structures/wiki/collections The latest version of my implementation is here: http://python-data-structures.googlecode.com/svn/trunk/collections_.py I look at the basic data structures of Python like this (ignoring mutability which I will mention later) : | Unordered | Ordered | ----------------------------------- Non-Unique | | list | ----------------------------------- Unique | set | | ----------------------------------- Python is missing 2 basic collections. First, an unordered non-unique collection, a bag aka multiset, the simplest collection there is. You can't even really call them a data structure because there's no structure to them (which is their beauty). Secondly it's missing an ordered unique collection, an ordered set or unique list, what I'm calling a setlist. That is the scope of when I started out, of course it grew and grew. The last addition but the first I will list, because I think it is the coolest and provides the most functionality, is the new function I propose adding to collections. The function is simply called 'collection' and has the following signature: collection(iterable=None, mutable=False, ordered=False, unique=False) What it does is return a collection (instantiated from iterable if passed) with the passed properties. This will only be possible with then new classes I propose, because they don't exist yet. New ABCs (abstract base classes) -------------------------------- After starting to study the ABCs built in to Python I came up with 2 more ABCs that I think should be included: Collection - implements Sized, Iterable and Container. Sequence, Set and Mapping would then all inherit Collection instead of the three aforementioned classes. Built-in classes that would be subclasses of Collection are set, frozenset, list and tuple. Collection would provide 2 methods: * _from_iterable(cls, it) - This is a classmethod that I took from Set, for convenience of returning new instantiations * __getitem__(key, value) - This is an abstract method. Every Collection would have to make its elements accessible. Of course set and frozenset do not currently implement this method so they will have to be changed, more later. Mutable - I think there should be one metaclass for all mutable types just like there is currently Iterable, Sized and Container. We could then check instanceof(obj, Mutable). MutableSequence, MutableSet and MutableMapping would inherit Mutable, or we might be able to just do away with them altogether. There are 3 abstractmethods defined: * pop(self) - This is the simplest to implement, all mutable classes already do. * __setitem__(self, key, value) - This is already implemented for list and dict but would have to be defined for set, more later * __delitem__(self, key) Same as above. Extending set to fit the ABCs ----------------------------- I mentioned before that sets don't currently fit into my model. Here are my propositions for the three methods that would need to be implemented. * set.__setitem__(self, elem, value) - Set whether or not elem is in the set based on the boolean evaluation of value. s[elem] = 1 would call s.add(elem) and s[elem] = False would call s.remove(elem), appropriately raising a KeyError if elem not in s. The reason I chose this way is that the only thing you can set about an element of a set is whether or not it is present. * __getitem__(self, item) - Simply returns item in self. Just like a bag returns the multiplicity of an item when you get it, this returns the number of times item appears in self. This also corresponds to the definition of setitem. * __delitem__(self, item) - This is the simplest case, just make it an alias for set.remove(item) I think this is the most controversial part of my proposition, but the more I think about it, the more I like it. []s aren't reserved for one single purpose in Python, lists restrict the index to an integer while dicts allow any Hashable object. list allows slicing inside the []s while it is meaningless for a dict. Finally, the New Classes ------------------------ These are the 2 new Collections (4 if you count the frozen variants) to fill in the holes I mentioned in the beginning. bags - These would be true multisets, AKA unordered tuples. See the wikipedia page on Multisets for more info. The benefits of a bag vs a list are that order doesn't matter for equality testing and that searching for an item is constant time. The latter is the real convincer for me. I think this should replace the Counter class in the collections module which doesn't provide anything more than defaultdict(int) does. setlist - AKA unique list, ordered set, bijection int->hashable, and ordered collection of unique elements. I find myself often creating a list of unique elements, then an inverse dict to look up the location quickly or test inclusion. I chose the name setlist because it's not a set xor a list, it's both. Then it occurred to me that bands' setlists are a perfect example of this data structure, an ordered collection of unique elements, real bands that is, http://en.wikipedia.org/wiki/Vertigo_Tour#The_encores -Michael Lenzen

14 42

Proposal for function expressions
by Chris Perkins 20 Jul '09

20 Jul '09

I have a proposal for a language feature - a limited form of function- definition expression, similar (superficially) to Ruby's blocks. The big problem with making "def" an expression is the indentation of the function body. You can't just embed an indented block of code into the middle of an expression. The solution I propose (inspired by Ruby) is to allow a block of code to be appended to the end of an expression in certain, limited circumstances. The block is simply syntactic sugar for a local anonymous function definition which is passed as an argument to the function call that it follows. First, a simple example to give the broad strokes: foo() do: BODY is equivalent to: def ANON(): BODY foo(ANON) where the name ANON is a placeholder for illustration - it is never actually bound in the local namespace. Specifically, here is what I propose: * A new syntactic construct, the "block", which consists of: * the keyword "do" (alternative: "def", repurposed) * an optional argument list * a colon * an indented block * A block is only allowed immediately following a call (or indexing expression? see below), on the same line as the closing right brace * The block defines an anonymous local function which is passed as an argument to the call that it follows. * The block is passed as the last argument to the function by default, but this can be changed by putting a placeholder, the character "&", into the parameter list of function. This is best illustrated by example: # 1) In the following, foo recieves TWO arguments: 23 and a callable, in that order. foo(23) do: pass # 2) Here, foo also recieves TWO arguments: a callable and 23, in that order. foo(&, 23) do: pass Why "&"? No particular reason - just because Ruby uses "&" for something similar (but not the same). Why do we need the "&" feature? Consider this example: map(&, my_list) do(item): return do_something_to(item) This also works for keyword arguments: foo(a=3, b=&, c=7) do(): whatever() To make this syntax work, we need several restrictions on where a block is allowed. Intuitively, the rules are: * If the line following "do" would normally be indented, then a block is not allowed. * If the line following the "do" would be one on which leading indentation is insignificant, a block is not allowed. To clarify, the first rule means that this is not allowed: if foo() do: # are we in the body of the block, or of the if? The second rule means that this is not allowed: bar(23, foo() do: body_of_block() ) # closing brace of call to bar Here are some properties of blocks that may not be immediately obvious: * Blocks are a feature of the call site, and do not affect function definitions. In other words, there is no such thing as a "function expecting a block", as there is in Ruby. From the callee's point of view, he has simply been passed a callable as an argument. * Blocks can be used with any existing callable that expects to be passed a function as one of its arguments. OK, let's move on to the fun part: Motivating Examples. ################### # Networking dfr = twisted.whatever(...) dfr.addCallback() do(result): handle(result) dfr.addErrback() do(err): handle_err(err) ################### # GUI (for some hypothetical library) b = my_widget.add_button('Go') b.on('click') do(evt): if evt.ctrl_key: do_something() else: do_other_stuff() ################### # Routing HTTP requests map.match('/') do(req): return render_home(req) map.match('/about') do(req): return render_about(req) map.match(/(?P<controller>\w+)\/$/) do(req, **kw): return handlers[kw['controller']](req) ################### # Threads thread.start_new() do: do_some_work() ################### # Reduce the temptation to cram too much into a lambda. # Sort a list of filenames that look like this: # Xyz-1.1.3.tgz, acbd-4.7.tgz, Xyz-1.2.5.tgz, ... my_list.sort(key=&, reverse=True) do(item): name, _ = os.path.splitext(item) root, _, version = name.partition('-') parts = version.split('.') return (root.lower(),) + tuple(map(int, parts)) stuff = filter(&, my_list) do(item): # some code here... return pred(item) stuff = re.sub(rx, &, s) do(m): t = m.group(0) if cond(t): return t.upper() else if othercond(t): return t.lower() else: return '' ################### # Faking switch switch(expr) do(case): case(int) do: stuff() case(str, bytes) do: other_stuff() switch(value) do(case): case[0:10] do(val): handle_small(val) case[10:100] do(val): handle_medium(val) case.default() do(val): handle_big(val) ################### # Overloaded functions (adapted from PEP 3124) @overload def flatten(when): when(basestring) do(ob): yield ob when(Iterable) do(ob): for o in ob: for ob in flatten(o): yield ob when.otherwise() do(ob): yield ob # later: flatten.when(file) do(f): for line in f: yield line ################### # Sort-of a "with(lock)" block, but running the body in a thread. pool.run_with(my_lock) do: # lock is held here work() other_work() # in parallel ################### Well, that should give you the general idea. Some final random points: * As proposed, blocks are a new kind of beast in Python - an indented block that is part of an expression, rather than being a statement. I think that by constraining a block to be the last thing in an expression, this works out OK, but I may be missing something * Since writing this, I have started to lean towards "def" as the keyword, rather than "do". I used "do" in the examples mostly to make them look like Ruby, but I suspect that making Python "look like Ruby" is not a design goal for most people :) * Terminology: "block"? "def expression"? "anonymous def"? "lambda++"? * Idea: Could we relax the rules for where a block is allowed, removing the constraint that it follows a call or indexing expression? Then we could do, for example, this: def f(): return def(args): pass * Would a proliferation of nameless functions be a nightmare for debugging? Implementation is here: http://bitbucket.org/grammati/python-trunk/ Patch is here: http://bugs.python.org/issue6469 To be honest, I'm not entirely sure if the patch is in the right format - it comes from hg, not svn. Chris Perkins

24 80

Re: [Python-ideas] Make return inside a finally a SyntaxError
by Terry Jones 19 Jul '09

19 Jul '09

I agree that having finally subsume returns and exceptions in the try block will at first be surprising to people, and that it can lead to easy-to-create but difficult-to-find bugs. But I think the current behavior is definitely for the best: - It's dead simple. The finally clause is always done last, and is always responsible for the return of the function *no matter what* may have happened in the try clause. - If finally were to automatically propagate any exception from the try clause, how would you turn that off when you needed/wanted to? You'd soon wind up wanting another keyword, perhaps seriouslyfinally, that was guaranteed to be run at the very end. If finally propagated exceptions, what should it do if there's an exception raised in the finally block? It begins to get complicated. The primitive blanket solution is better, IMO. - Besides, if you do want a finally clause to propagate an exception from its try clause, that's simple as you can just store the exception somewhere in an except clause and then let the finally detect it and re-raise it, if appropriate. By keeping finally's semantics simple and resisting the temptation to make it more sophisticated (e.g., propagating exceptions or return values) you wind up with a better tool for the programmer who does know what they're doing. The more sophisticated behavior can easily be added as you need it on a case-by-case basis. The added functionality shouldn't be provided unconditionally by the language because you then can't disable it. I.e., I prefer the control I get from the current semantics of finally. Terry

6 5

Make return inside a finally a SyntaxError
by Michael 18 Jul '09

18 Jul '09

Here are two examples of why allowing return inside a finally block is a bad idea: def f(): try: return 3 finally: return 4 def f(): try: raise Exception() finally: return 4 Michael Foord -- http://www.ironpythoninaction.com

17 32

Augment dis.dis to autocompile strings
by Terry Reedy 18 Jul '09

18 Jul '09

Currently "dis.dis(x=None) Disassemble the x object. x can denote either a module, a class, a method, a function, or a code object. " The behavior depends on the class of x, so dis already has an internal switch. Currently, to see the assembly equivalent of CPython bytecode for a small snippet, one must either wrap it in a function or *remember* the exact idiom: compile("while True: break", "", "exec") Proposal: if the input is a string, do the above (do what I mean) instead of raising TypeError: don't know how to disassemble str objects Any negatives before I submit a feature request? Terry Jan Reedy

4 6

Re: [Python-ideas] Make return inside a finally a SyntaxError
by Terry Jones 18 Jul '09

18 Jul '09

>>>>> "Michael" == Michael Foord <fuzzyman(a)gmail.com> writes: Michael> The bit that has to be *added* in Python (and usually isn't added Michael> because it isn't at all obvious) is: Michael> Exceptions are not reraised if the finally returns or breaks. OK. >> If finally propagated exceptions, Michael> It does. Sorry - I meant propagates the try exception even in the case that the finally does a return, break, or raises itself. >> It begins to get complicated. Michael> No, it gets simpler - no special cases for the exception handling Michael> of finally in the presence of a return. I think it's more complex. Either, or both, of the try block and the finally block may return/raise etc. What to do when those things conflict? Which block's opinion has priority? It seems clear (to me) that the finally block should have the final word, no matter what - including (perhaps even especially) if its ultimate conclusion conflicts with what the try block did. Terry

1 0

Re: [Python-ideas] Where-statement (Proposal for function expressions)
by Carl Johnson 18 Jul '09

18 Jul '09

Daniel Stutzbach wrote: > Unlikely, but this is: > > total =3D sum(f(item) for item in lst) where: > def f(item): > return item.size if item.size > 0 else 1 Yes, that seems much more sensible than trying to repeat the where's inside of list comp or gen exp. Daniel Stutzbach wrote: > How about the following as additional syntactic sugar for the common one-= function case? > > x =3D blah(f) where def f(item): > body_of_f I like it. Gerald Britton wrote: > Why not just use the Haskell approach? > > foo(x,y) =3D myfunc(bar) where: > =A0 =A0 =A0 =A0 =A0 myfunc, bar =3D f(x), g(y) where: > =A0 =A0 =A0 =A0 =A0 =A0 =A0f,g =3D func1, func2 Because I've been looking at that for a couple minutes and still have no idea what it is supposed to mean. :-( List comprehensions and pattern matching are the only Haskell features that make any sense to me. Jan Kaliszewski wrote: > I don't like the direction of limiting content of the block to set of sin= gular simple statements. I suppose where-block should simply create a separ= ate nested scope. Yes. It strikes me that one of the problems with Ruby is that there are different scoping rules for blocks, lambda, methods, procs, and whatever else=85 It would be much easier for users if "where" blocks had the exact same scoping rules as functions. Daniel Stutzbach's suggested equivalence strikes me as exactly correct: [return|yield|x=3D] expression_list where: suite is roughly equivalent to: def where_expression(): suite return expression_list [return|yield|x=3D] where_expression() -- Carl Johnson

20 56

Immemorial desire for "do-while"-like construction
by Jan Kaliszewski 16 Jul '09

16 Jul '09

Hello, The issue has been coming back repeatedly: How we could elegantly and pythonic'ly avoid repeating ourselves if we need the control flow structure that e.g. in PASCAL has the form: repeat ... until CONDITION In Python we must use while-loop in not very DRY (Don't Repeat Yourself) manner: SOME ACTIONS HERE ... while CONDITION: THE SAME ACTIONS AGAIN ... (See also: http://www.python.org/dev/peps/pep-0315/ http://mail.python.org/pipermail/python-dev/2006-February/060718.html http://preview.tinyurl.com/nodgwt ) Maybe here the simplest approach can lead us to the solution? I.e.: repeat while CONDITION: SOME ACTIONS HERE ... In human language it'd mean: do ACTIONS once unconditionally, then *repeat* them while the CONDITION is true. Additional advantage is that probably for many people associate 'repeat' with such construction (from experience with other languages). What do you think about the idea? -- Jan Kaliszewski <zuo(a)chopin.edu.pl>

8 10

Where-statement (Proposal for function expressions)
by Weeble 15 Jul '09

15 Jul '09

> Date: Wed, 15 Jul 2009 19:55:15 +1200 > From: Greg Ewing <greg.ewing(a)canterbury.ac.nz> > To: python-ideas(a)python.org > Subject: [Python-ideas] Where-statement (Proposal for function > expressions) > Message-ID: <4A5D8B63.4040509(a)canterbury.ac.nz> > Content-Type: text/plain; charset=UTF-8; format=flowed > > Carl Johnson wrote: > >> My suggestion for future would be language extenders is that they try >> to solve a broader problem than just the "how do I make callbacks more >> convenient" problem. > > One such extension might be a "where" block. Applied > to the current problem: > > foo(f) where: > def f(x): > ... I was just thinking of something similar: in: foo(f,a) let: def f(x): ... a = ... (Keywords open to debate.) It's a little verbose, but I dislike it much less than all the proposals that introduce special sigils or do weird things to the syntax of the code. Things I like about it: there's a clear warning (the "in:") right at the start that the statement (or block?) is special and relies on code that comes after; the syntax isn't complex; both lets you provide a name and prevents it leaking into places you don't want it; and the contents of both the "in:" block and the "let:" block are normal Python. I'd expect the behaviour to be something like this: execute the "let:" block in its own nested scope, then execute the "in:" block in the regular local scope but with the symbols defined by the "let:" block available. Does that make some sort of sense? Most of the things I like are achieved by Greg's "where:" block, the only thing I don't like about it is tacking a keyword onto the end of a statement, which feels a bit awkward to me. That may just be aesthetics. I certainly like the "where:" idea a lot better than the other proposals. It seems to achieve the most utility with the least confusing syntax.

1 0

Decorations
by Andrey Fedorov 13 Jul '09

13 Jul '09

Hi there, It's always bothered me that decorators require a lot of "cruft" to work. Consider an example, from PEP-318 <http://www.python.org/dev/peps/pep-0318/>(Decorators for Functions and Methods): def returns(rtype): def check_returns(f): def new_f(*args, **kwds): result = f(*args, **kwds) assert isinstance(result, rtype), \ "return value %r does not match %s" % (result,rtype) return result new_f.func_name = f.func_name return new_f return check_returns It seems to me everything but line 5 "assert..." is cruft. In my mind, it makes sense to call that line the "decoration" (part of the decorator). Since it's a decoration that's executed after the decorated function, it makes sense to call it a post-decoration, one that may be more nicely defined as: @postdecoration def returns(result, rtype): assert isinstance(result, rtype), "return value %r does not match %s" % (result, rtype) Then, `returns' can be used as in PEP-318 (but without the cruft). I've defined similar `decoration', and `predecoration' decorators. I use the latter for my Django views to accept POST parameters as arguments: def filter_dict(d, fltr): return dict( (str(x.lower()), d[x]) for x in d if x.lower() in fltr ) @predecoration def takesPOST(f, args): if len(args) == 1: fargs = getargspec(f).args get = args[0].POST return filter_dict(get, fargs[1:]) This way, I can define a view login(reqest, user, password) which can be called simply from another view, without having to construct a special `request' object. An initial shot at the definitions is here: http://gist.github.com/146235 The design decisions I'm least confident about is the argument specs. In the `predecoration' example, the argument `args' is a special name that specifies the arguments passed into the decoratee, `kwargs' would be the keyword arcgs passed into the decorateee, dec_args and dec_params being the arguments passed into the *decorator*. This doesn't seem like an optimal solution. The problem I'm trying to solve is "moving from a decoration which takes no parameters to a decoration which takes parameters". For example, it was trivial to refactor takesPOST to group POST parameters by regex passed into the decorator. Still, I'm open to questions, ideas, opinions, etc. What do you guys this? Cheers, Andrey

3 4

Signed eggs?
by Tim Lesher 10 Jul '09

10 Jul '09

In all the current discussion on python-dev about improving eggs and setuptools in general, I don't think I've seen anything regarding digitally signed eggs or verifiable egg distribution. Google doesn't seem to turn anything up, either. Has anyone put any thought into this? -- Tim Lesher <tlesher(a)gmail.com>

3 4