On 1/24/2016 7:54 AM, Nick Coghlan wrote:
On 24 January 2016 at 15:16, Guido van Rossum <guido@python.org> wrote:
I wonder if kids today aren't too much in love with local function definitions. :-) There's a reason why executor.submit() takes a function *and arguments*. If you move the function out of the for loop
What I've concluded from this thread is that function definitions (with direct use 'def' or 'lambda') do not fit well within loops, though I used them there myself. When delayed function calls are are needed, what belongs within loops is packaging of a pre-defined function with one or more arguments within a callable. Instance.method is an elegant syntax for doing so. functools.partial(func, args, ...) is a much clumsier generalized expression, which requires an import. Note that 'partial' returns a function for delayed execution even when a complete, not partial, set of arguments is passed. A major attempted (and tempting) use for definitions within a loop is multiple callbacks for multiple gui widgets, where delayed execution is needed. The three answers to multiple 'why doesn't this work' on both python-list and Stackoverflow are multiple definitions with variant 'default args', a custom make_function function outside the loop called multiple times within the loop, and a direct function outside the loop called with partial within the loop. I am going to start using partial more. Making partial a builtin would make it easier to use and more attractive. Even more attractive would be syntax that abbreviates delayed calls with pre-bound arguments in the way that inst.meth abbreviates a much more complicated expression roughly equivalent to "bind(inst.__getattr__('meth'), inst)". A possibility would be to make {} a delayed and possibly partial call operator, in parallel to the current use of () as a immediate and total call operator. expr{arguments} would evaluate to a function, whether of type <function> or a special class similar to bound methods. The 'arguments' would be anything allowed within partial, which I believe is anything allowed in any function call. I chose {} because expr{...} is currently illegal, just as expr(arguments) is for anything other than a function call. On the other hand, expr[...] is currently legal, at least up to '[', as is expr<...> at least up to '<'.
and pass the url as a parameter to submit(), problem solved, and you waste fewer resources on function objects and cells to hold nonlocals.
executor.submit appears to me to be a specialized version of partial, with all arguments required. With the proposal above, I think submit(func{all args}) would work.
Aye, that's how the current example code in the docs handles it - there's an up front definition of the page loading function, and then the submission to the executor is with a dict comprehension.
I presume you both are referring to ThreadPoolExecutor Example. The load_url function, which I think should be 'get_page' has a comment that is wrong (it does not 'report the url') and no docstring. My suggestion: # Define an example function for the executor.submit call below. def get_page(url, timeout): "Return the page, as a string, retrieved from the url." with ...
The only thing "wrong" with it is that when reading the code, the potentially single-use function is introduced first without any context, and it's only later that you get to see what it's for.
A proper comment would fix this I think. That aside, if the main code were packaged within def main, as in the following ProcessPoolExecutor Example, so as to delay the lookup of 'load_url' or 'get_page', then the two functions definitions could be in *either* order. The general convention in Pythonland seems to be to put main last (bottom up, define everything before use), but in a recent python-list thread, at least one person, and I think two, said they like to start with def main (top down style, which you seem to like). I just checked and PEP8 seems to be silent on the placement of 'def main'. So unless Guido says otherwise, I would not mind if you revised one of the examples to start with def main, just to show that that is a legitimate alternative. It is a feature of Python that one can do this without having to add, before the first appearance of a function name within a function, a dummy 'forward declaration' giving the function signature.
A generation ago most people would have naturally used such a solution (since most languages didn't support the alternative :-).
In programming we would have, but I don't think the same is true when writing work instructions for other people to follow - for those, we're more likely to use nested bullets to describe subtasks, and only pull them out to a separate document or section if we need to reference the same subtask from multiple places.
People can and do jump around while reading code for understanding. They can do this without markers as explicit as needed for machines. Current compilers and interpreters initially read code linearly, with only one character or token lookahead. For Python, a def header is needed for forward reference, to delay name resolution to call time, after the whole file has been read.
While my view is admittedly only based on intuition rather than hard data, it seems to me that when folks are reaching for nested functions, it's that "subtask as a nested bulleted list" idiom they're aiming to express, and Python is otherwise so accommodating of English structural idioms that it's jarring when it doesn't work properly. (I also suspect that's why it's a question we keep returning to - as a *programming language*, making closures play more nicely with iteration variables doesn't add any real power to Python, but as *executable pseudo-code*, it makes it a little bit easier to express certain ideas in the same way we'd describe them to another person).
I thought about some explicit examples and it is not necessarily clear how to translate bullet points to code. But in general, I do not believe that instructions to another person are meant to induce in the mind of a listener multiple functions that only differ in a default argumnet object. In other words, I do not see for i in it: def f(i=i): pass as corresponding to natural language. Hence my initial statement above. -- Terry Jan Reedy