Mailman 3 December 2014 - Python-ideas

Map and filter should also convert StopIteration to RuntimeError
by Terry Reedy Dec. 15, 2014

Dec. 15, 2014

" exception StopIteration Raised by built-in function next() and an iterator‘s __next__() method to signal that there are no further items produced by the iterator." I have always taken this to mean that these are the only functions that should raise StopIteration, but other have not, and indeed, StopIteration in generator functions, which are not __next__ methods, has been accepted and passed on by generator.__next__. PEP 479 reverses this acceptance by having generator.__next turn … [View More]

13 26

PEP 479 and take()
by Oscar Benjamin Dec. 14, 2014

Dec. 14, 2014

I've been away from Python-related mailing lists for almost a year now and only a few days ago started following them again just in time to see the "hopefully final" text of PEP 479 posted on python-dev. I had been intending to just lurk around for a while but felt compelled to post something after seeing that. It's taken a while for me to find the time to read through what's already been written but I see that PEP 479 is apparently a done deal so I won't try to argue with it (except to … [View More]register a quick -1 here). Somehow the discussion of PEP 479 became obsessed with the idea that leaked StopIteration is a problem for generators when it is a problem for all iterators. I haven't seen this pointed out yet so I'll demonstrate that map() is susceptible to the same problem: $ cat tmp.py people = [ ['John Cleese', 1, 0, 1], ['Michael Palin', 123, 123], [], # Whoops! ['Terry Gilliam', 12, False, ''], ] def first_name(person): return next(iter(person)).split()[0] for name in map(first_name, people): print(name) $ python3.4 tmp.py John Michael (i.e. Terry was not printed and no error was raised.) There's nothing hacky about the use of map above: the mistake is just the bare next call. The same thing happens with filter, takewhile, etc. Essentially any of the itertools style functions that takes a user-defined function allows StopIteration to pass from the user function to the parent iterator-consumer. I believe this is by design since apparently the author Raymond Hettinger (like me) considered StopIteration fall-through a deliberate design feature of the iterator protocol. Fixing this problem so that a leaked StopIteration turns into a loud error message has been deemed important enough that a partial fix (applying only to generators) warrants breaking the backward compatibility of the core language in a minor release. So what should happen with all the other places that are susceptible? Is StopIteration fall-through to be considered an anti-pattern that anyone implementing the iterator protocol should avoid? With or without PEP 479 the root of the problem is simply in the careless use of next(). The PEP proposes to make this easier to track down but once located the problem will be an unguarded next call that needs to be fixed. (It will also force people to "fix" other things like "raise StopIteration" but these were not actually problematic before). Clearly people want a function like next() that isn't susceptible to this problem or they wouldn't use next in this way and the problem wouldn't exist. So I propose a new function called take() with the following semantics: class TakeError(Exception): pass def take(iterator, n=None): if n is None: try: return next(iterator) except StopIteration: raise TakeError else: return tuple(take(iterator) for _ in range(n)) The idea is that take(iterator) is the generic way to get the next item from the iterator and assert that the item should exist. When you use take(iterator) your intention that the item should exist is self documenting whereas a bare next() is ambiguous without a comment: x = next(iterator) # Never raises StopIteration x = next(iterator) # Propagate StopIteration x = next(iterator) # Haven't considered StopIteration This gives users a clear and un-ugly fix to any code that uses next inappropriately: s/next/take so that there is no excuse for not fixing that code to: x = take(iterator) # Either I get an item or a proper Error is raised. Similarly take(iterator, n) is like islice except that it immediately advances the iterator and raises if the required number of items was not found. Essentially this is a safer version of: firstn = [next(iterator) for _ in range(n)] # Leaks StopIteration firstn = tuple(next(iterator) for _ in range(n)) # Terminates silently firstn = list(islice(iterator, n)) # Terminates silently (Actually the second example would raise RuntimeError with PEP 479) With take it becomes: firstn = take(iterator, n) # n items returned or an Error Oscar [View Less]

8 12

Re: [Python-ideas] [Python-Dev] Python 2.x and 3.x use survey, 2014 edition
by Hernan Grecco Dec. 12, 2014

Dec. 12, 2014

On Thu Dec 11 2014 at 11:48:22 AM Giampaolo Rodola' <g.rodola(a)gmail.com> wrote: > On Wed, Dec 10, 2014 at 5:59 PM, Bruno Cauet <brunocauet(a)gmail.com> wrote: > >> Hi all, >> Last year a survey was conducted on python 2 and 3 usage. >> Here is the 2014 edition, slightly updated (from 9 to 11 questions). >> It should not take you more than 1 minute to fill. I would be pleased if >> you took that time. >> > It would be really nice to … [View More]

7 12

functools.lru_cache manual cache modification
by Constantin Berhard Dec. 11, 2014

Dec. 11, 2014

Hi, I've written a tail call optimization lib for python3. <https://titania.fs.uni-saarland.de/projects/libtco> It works fine. I use a custom return, which throws the next function arguments as an exception and a decorator, which handles the exception. One strength of the functools.lru_cache lies in caching results of calls initiated by the function itself (i.e. recursive call results). However because of the exception, the intermediate results from the tail recursion don't end up in … [View More]

9 22

Restitute imp.load_source or equivalent in Python 3?
by Mart Sõmermaa Dec. 9, 2014

Dec. 9, 2014

http://bugs.python.org/issue14551 discusses the deprecation and removal of imp.load_source. In the discussion, there seems to be a general consensus that (1) a convenience function for loading modules from an arbitrary file is a good thing and (2) that many people use it. I can only concur, the need for loading a module from an arbitrary file without having to worry about sys.path or __init__.py is useful for supporting plugin-like extensibility/configurability and I personally use imp.… [View More]

3 2

Multi-index Containers Library
by SF Markus Elfring Dec. 8, 2014

Dec. 8, 2014

Hello, I find a data structure like it is provided by the "Boost Multi-index Containers Library" interesting for efficient data processing. http://www.boost.org/doc/libs/1_57_0/libs/multi_index/doc/index.html How are the chances to integrate a class library with similar functionality into the Python software infrastructure? https://bugs.python.org/issue22957 Regards, Markus

9 12

Asynchronous IO ideas for Python
by Paul Colomiets Dec. 6, 2014

Dec. 6, 2014

Hi, I've written an article about how I perceive the future of asynchronous I/O in Python. It's not something that should directly be incorporated into python now, but I believe it's useful for python-ideas list. https://medium.com/@paulcolomiets/the-future-of-asynchronous-io-in-python-c… And a place for comments at Hacker News: https://news.ycombinator.com/item?id=8662782 I hope being helpful with this writeup :) -- Paul

7 7

Improved support for prepared SQL statements
by SF Markus Elfring Dec. 1, 2014

Dec. 1, 2014

Hello, An interface for parameterised SQL statements (working with placeholders) is provided by the execute() method from the Cursor class at the moment. https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.execute I assume that the "SQL Statement Object" from the SQLite C interface is reused there already. http://sqlite.org/c3ref/stmt.html I imagine that it will be more efficient occasionally to offer also a base class like "prepared_statement" so that the parameter specification … [View More]

2 1

Fast context creation
by Stefano Borini Dec. 1, 2014

Dec. 1, 2014

I have a situation where, no matter how the routine ends, I need to return the focus to a widget. It would be rather clever to do this with a context, but I would expect something like this to be a possible strategy with contextlib.context(enter=None, exit=lambda *args: my_widget.setFocus()): do what I need to do as far as I know, at the moment it's not possible. Am I right? I think it would be an easy and practical addition to the contextlib module to quickly register two … [View More]

8 10