Hi guys, A couple more thoughtsAs to why I didn't use an "or" in the original example. I am aware that the example doesn't actually need an inline function, but it demonstrates the concept in a simple way (so that it would fit in an email). Most use-cases are more complicated, often the spots where you want to execute the same code are buried inside long, branching logic statements where other things are going on, making restructuring the code much less trivial, if not impossible.I had never really considered using closures that way (I haven't really used them much at all), that is extremely cool! It does exactly what I was envisioning as long as you only need it in one spot. If inline function got implemented I'm sure they would steal a lot of code from closures.The remaining use cases for inline functions are times when you would need the same capabilities in multiple functions (calling in multiple methods of a class for example), or when you wish to define what is essentially a closure at the first level of execution, since closures are not possible when you are not in a function.I always knew you could do it the way skip showed, but I feel like that is a bit clunky. This would allow you to do that in a more straight-forward way.

On Thu, Feb 6, 2014 at 2:15 AM, Alex Rodrigues <lemiant@hotmail.com> wrote:

I had never really considered using closures that way (I haven't really used them much at all), that is extremely cool! It does exactly what I was envisioning as long as you only need it in one spot. If inline function got implemented I'm sure they would steal a lot of code from closures.

Or, maintain compatibility with older versions of Python by simply using closures :) Remember, something discussed here has to then be coded, be accepted into trunk, be tested, and eventually be distributed in a new version. You can't possibly get something like this added to Python 3.3 or earlier; 3.4 is already too close to add something like this to; so you'd be looking at 3.5 at the absolute soonest. Maybe that's not a problem, if your code's for you and nobody else (you can run a Python alpha or even a modified Python with additional code in it), but if you want to distribute your code and have people install Python and run your program, you'd be waiting an absolute minimum of 2-3 years for this to happen. And the consequence of that is that a proposed new feature really has to be all that awesome; it has to be worth waiting several years for (and probably putting in some coding work to create, unless you can excite someone else enough to have him/her write it for you). That's a pretty steep hump to get over.

The remaining use cases for inline functions are times when you would need the same capabilities in multiple functions (calling in multiple methods of a class for example), or when you wish to define what is essentially a closure at the first level of execution, since closures are not possible when you are not in a function.

The nearest concept I can think of to your proposed inline functions is a preprocessor macro. Effectively, you create a simple token that expands to a block of source code, and it's handled prior to the parsing and compilation of the code. There are times when that sort of thing is really useful, but it's oh so easy to make horribly unreadable code. (Plus, it'd be hard to get indentation right, if the different contexts are indented to different levels.) There's usually some other way to do what you want. Python's pretty amazingly dynamic (often at the cost of performance); offer a specific use-case (maybe on python-list rather than here) and there's a reasonable chance someone can suggest a way to refactor it. Post actual working code (in the verbose form), highlighting the duplicated bits, and see what people offer. You'd be amazed what the language already has!

I always knew you could do it the way skip showed, but I feel like that is a bit clunky. This would allow you to do that in a more straight-forward way.

Yes, that proposal is VERY clunky. Passing locals() around is almost always a baaaad idea :) ChrisA

On Thu, Feb 6, 2014 at 11:26 AM, Andrew Barnert <abarnert@yahoo.com> wrote:

On Feb 5, 2014, at 7:28, Chris Angelico <rosuav@gmail.com> wrote:

...

On Thu, Feb 6, 2014 at 2:15 AM, Alex Rodrigues <lemiant@hotmail.com> wrote:

...

...

The remaining use cases for inline functions are times when you would need the same capabilities in multiple functions (calling in multiple methods of a class for example), or when you wish to define what is essentially a closure at the first level of execution, since closures are not possible when you are not in a function.

The nearest concept I can think of to your proposed inline functions is a preprocessor macro. Effectively, you create a simple token that expands to a block of source code, and it's handled prior to the parsing and compilation of the code. There are times when that sort of thing is really useful, but it's oh so easy to make horribly unreadable code. (Plus, it'd be hard to get indentation right, if the different contexts are indented to different levels.)

Higher-level macros are a much better match. In lisp terms, what you want is a non-hygienic (or optionally-hygienic) syntactic macro. In Python terms, you can do that by working on ASTs rather than source code. Which has already been implemented very nicely by Li Haoyi--see MacroPy on PyPI.

Ah, possibly. I've never dug into that side of things, but that sounds reasonable. It'd solve the indentation issues, at least. In any case, it wouldn't be a function any more; it'd have to be a compile-time alteration. You can't, at run time, choose between several such macros - imagine if they differed in what names they made local. ChrisA

On Feb 5, 2014 8:18 PM, "David Mertz" <mertz@gnosis.cx> wrote:

It's hard for me to see the use case for this "inline function" idea.

The example given is just some code that is repeated within branches of an 'if'; but the obvious thing is to move that repetition outside the if/elif clauses.

I understand that the example is simplified to present it, but it's hard

for me easily to imagine a case where either using a closure or passing in 'locals()' directly fails to cover everything you are asking for. I mean, you did say that passing locals() is "clunky", but it's hard to see how any function that dealt with the availability of different variables on an ad hoc basis could really be non-clunky (and that what every inline function would have to do) How about: Def somefun(): a = 5 b = 7 return fmt('{a} is not {b}') inline def fmt(fmtstr): return fmtstr.format_map(locals())

On Wed, Feb 05, 2014 at 12:03:46PM -0800, Ethan Furman wrote:

On 02/05/2014 07:15 AM, Alex Rodrigues wrote:

...

3. I always knew you could do it the way skip showed, but I feel like that is a bit clunky. This would allow you to do that in a more straight-forward way.

I disagree.

saveline(**locals())

is very clear about what it's doing, while

saveline()

is not.

This is true, and there's no need to write this: def saveline(**kwargs): # Trivial one-liner implementation used for brevity. return kwargs['a'] + kwargs['b'] This ought to work and be much nicer: def saveline(a, b, **kwargs): # kwargs accumulates all the other, unused, locals from the caller. return a + b I think what Alex is describing is a type of function which operates using *dynamic scoping*, rather than static/lexical scoping like Python functions normally do. This means that the function can see the variables from where it is called, not where it is defined. https://en.wikipedia.org/wiki/Scope_%28computer_science%29 http://c2.com/cgi/wiki?DynamicScoping My understanding is that most languages prefer static scoping because it is safer and less error-prone for the programmer, and more easily understood when reading code. But a few languages, such as Perl, offer dynamic scoping as a option. -- Steven

On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman <ethan@stoneleaf.us> wrote:

Possibly, but it's easy enough to simulate (at least in a read-only fashion) with **locals().

Eew. Use of locals() for any purpose other than debugging should be discouraged. -- --Guido van Rossum (python.org/~guido)

On Wed, Feb 5, 2014 at 4:37 PM, MRAB <python@mrabarnett.plus.com> wrote:

On 2014-02-06 00:15, Guido van Rossum wrote:

...

On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman <ethan@stoneleaf.us <mailto:ethan@stoneleaf.us>> wrote:

Possibly, but it's easy enough to simulate (at least in a read-only fashion) with **locals().

Eew. Use of locals() for any purpose other than debugging should be discouraged.

Even when you're using "format_map"?

Yes. Explicit is better than implicit. locals() is hard for the compiler to get right -- I wouldn't be surprised if some compilers would have to generate less optimal code if locals() is used. (Similar to sys._getframe().) -- --Guido van Rossum (python.org/~guido)

On 02/05/2014 04:15 PM, Guido van Rossum wrote:

On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman wrote:

...

Possibly, but it's easy enough to simulate (at least in a read-only fashion) with **locals().

Eew. Use of locals() for any purpose other than debugging should be discouraged.

Well, as I recall (although I can't find it now) there was discussion about auto-filling a functions parameters with local values when the names matched, which would be much nicer is this situation (as well as extending situations). For example: class MyString(str): def find(sub, start, end): ... # manipulate sub and/or start and/or end ... return super().find(#blah#) where #blah# means "pick out sub, start, and end from locals and use them". -- ~Ethan~

On Wed, Feb 5, 2014 at 4:53 PM, Ethan Furman <ethan@stoneleaf.us> wrote:

On 02/05/2014 04:15 PM, Guido van Rossum wrote:

On Wed, Feb 5, 2014 at 2:26 PM, Ethan Furman wrote:

...

...

Possibly, but it's easy enough to simulate (at least in a read-only fashion) with **locals().

Eew. Use of locals() for any purpose other than debugging should be discouraged.

Well, as I recall (although I can't find it now) there was discussion about auto-filling a functions parameters with local values when the names matched, which would be much nicer is this situation (as well as extending situations). For example:

class MyString(str): def find(sub, start, end): ... # manipulate sub and/or start and/or end ... return super().find(#blah#)

where #blah# means "pick out sub, start, and end from locals and use them".

And how on earth would the compiler know the argument names? -- --Guido van Rossum (python.org/~guido)

On 02/05/2014 08:49 AM, Skip Montanaro wrote:

I'm not convinced this particular example needs an inline function. Just write saveLine() as a normal function and pass in your locals:

def f(): a = 1 b = 7 compute(**locals())

def g(): a = 3 b = 0.9 compute(**locals())

def compute(**args): print args["a"] + args["b"]

f() g()

That's not to say inline functions might not be handy. It's possible that you can implement them without modifying the language either. A peephole optimizer could inline small functions which are used a lot.

Skip

The other day I was trying to figure out how to do lamba calculus without using closures (or other tricks).

...

...

def I (x): return x def K (x): ... return lambda y: x def S (x): ... return lambda y: lambda z: x(z)(y(z)) I("A") 'A' K("A")("B") 'A' S(K)(K)("A") 'A'

To do it without closures isn't easy. But it did give me an interesting idea that could simplify a lot of problems. Basically... # S := λx.λy.λz.x z (y z) def continue S: :from(x) :from(y) :from(z) return x(z)(y(z)) And it would be used exactly like the lambda example above. S(K)(K)("A") --> 'A' That's equation is supposed to give the same result back out.. (* That is if I have it right?) Each call returns the updated frame but we only have one code object which makes it easier to write and think about. In this example.. g1 = S(K) g2 = g1(K) result = g2('A') --> also S(K)(K)('A') It depends on how it's implemented weather g1 and g2 are the same object, or new objects with different states. Each ":from()" can use a full function signature, So these could be used as decorators. def continue foo: :from(func) :from(*args, **kwds) # do something with the function arguments return func(*args, **kwds) @foo def bar(...): ... Or probably also be used as with statement context managers. There is very little duplication. The :from(...) arguments get pulled directly into locals, (the same as a function call updates them).. so there is no assignment needed and no need to play with locals(). You could also think of this as an explicit closure turned inside out. The closure become locals, and return that with each function signature.... but there is only one code object shared between calls, it makes writing a lot of things easier. About the funny grammar... I think it needs something different to say these are functions signatures. Normally, the body is on the right of the colon, I used the reverse in this case because it is presenting the function signature from the inside, rather than the outside. (Well, it makes sense to me anyway.) I'm sure there are lots of other colours for this bike. Cheers, Ron

On Feb 6, 2014, at 3:14, Nick Coghlan <ncoghlan@gmail.com> wrote:

On 6 February 2014 20:34, Ron Adam <ron3200@gmail.com> wrote:

...

On 02/05/2014 08:49 AM, Skip Montanaro wrote:

...

I'm not convinced this particular example needs an inline function. Just write saveLine() as a normal function and pass in your locals:

def f(): a = 1 b = 7 compute(**locals())

def g(): a = 3 b = 0.9 compute(**locals())

def compute(**args): print args["a"] + args["b"]

f() g()

That's not to say inline functions might not be handy. It's possible that you can implement them without modifying the language either. A peephole optimizer could inline small functions which are used a lot.

Skip

The other day I was trying to figure out how to do lamba calculus without using closures (or other tricks).

...

...

...

def I (x): return x def K (x): ... return lambda y: x

This returns a closure.

...

...

...

...

def S (x): ... return lambda y: lambda z: x(z)(y(z))

This returns a closure that returns a closure.

...

To do it without closures isn't easy.

Lambda expressions aren't special from a scoping point of view, they're just a different way to write:

def <lambda>(*args, **kwds): return expr

They're only special from a structural point of view (as expressions, you can put them in places where full functions aren't allowed).

As far as what you're proposing goes, is it essentially a way to declare a function like (spelling out the lambdas fully):

def S(x): def _second(y): def _third(z): return x(z)(y(z)) return _third return _second

As something much shorter like this:

def S (x)(y)(z): return x(z)(y(z))

I think his point is that he wants it to be implemented _without_ closures--or at least as a single closure, with the same scope shared by all of the functions instead of three scopes and closure cells linking them together. Personally, I'm not sure what that adds; I'm just raising what I think is someone else's point. There could conceivably be performance benefits, but then again, there could also be a cost to keeping alive an entire scope when you don't need everything. Other than that, the visible effect would be exactly the same as with your explicit closure version.

The main potential benefit

I think the arguable brevity and readability benefit is the main potential benefit here, as your decorator example. If the abbreviated examples are clearer, the idea is a win even without the pickling; if it's more of an attractive nuisance that makes it easier to write impenetrable code more than it makes it easier to write simpler code, the idea isn't worth doing even with the pickling.

I could see to a construct like that is that it may allow the more consistent creation of closures that support pickling, since the outer functions are guaranteed not to have any side effects and to have argument capture as their *only* significant state. This means that you could take the inner function, pickle it along with its closure variables and reconstruct that at the far end, only relying on the name of the outer function.

Out of curiosity, how would you implement that? Right now, functions don't reduce, and pickle just stores their global name. Would you make pickle detect side-effect-free closures (by way of a flag exposed via inspect or something?), dig out the arguments to the outer function, and pickle them as calls with a new opcode similar to new-style class instances? Or change functions so they can reduce themselves in appropriate circumstances?

Such a construct could also make decorator factories easier to write.

def decorator(f): # Do something with f

def decorator_factory(some, args, here)(f): # Do something with f, but have access to the bound args.

There's an argument to be made that the extra parens in the function header are too easy to miss, but I still see "make it easier to write side-effect free closures" as an idea worth discussing further.

I like this idea, and your syntax. With or without collapsing frames or doing anything special (like pickling) with the results.

I'm not under any illusions about how hard it is to actually get something added to standard python . But I thought this would be a fun and informative undertaking that possibly even helps people. Thank you for your kind words. After some consideration I have come up with another use case that demonstrates a little more of how this might be useful:Let's say that I am a weather forecaster and I maintain a large library of different simulation techniques. All the techniques need to get their initial data from some database. Let's take a look at some way of writing this code and their advantages/disadvantages:Hard code the startup code into each simulation - This works, but it will lead to lots of code re-use and all the problems that come with that (i.e. if I later need to change how the startup values are downloaded, now I have to change every single function)Use a closure - This isn't really an option, both because there are multiple functions and because closure cannot create variables in their containing scopeReturn the values from a function - This will add lots of unnecessary typing at the top of each function to list out all of the variables and if I wish to add a new input to one simulation (maybe UV just became important) I must now modify all of the functions to have that value passed in.Classes - This is probably the best option currently available. Make a simulation class that has a initialize function and have each simulation inherit from it. This also has some drawbacks though, since I must now instantiate a copy of the object to find the output of the simulation, which makes calling the function harder (especially on a one time basis).Use an inline def - This is very modular and maintainable with an absolute minimum of boilerplate. The main downside being that it is not explicit what is available in the namespace once you initialize the simulation.Here's how it might look: inline def start_simulation(): date = datetime.utcnow() date.replace(tzinfo=utc) start_data = urllib2.urlopen('http://www.source.com/seed_sims').read().split() num_clouds = int(start_data[0]) temp = float(start_data[1]) grid = [] for a in range(int(start_data[2])): grid.append([]) for b in range(int(start_data[3])): grid[a].append(0) def sim_basic(): start_simulation() return temp def sim_iterative(): start_simulation() for row in grid: for cell in row: cell = temp+random.random() for i in range(10): for row in grid for cell in row: cell += random.random()-0.5 return average([cell for row in grid for cell in row]) def sim_clouds(): start_simulation() return temp - numclouds/10.

Date: Wed, 5 Feb 2014 08:57:35 -0700 Subject: Re: [Python-ideas] Inline Functions - idea From: ericsnowcurrently@gmail.com To: lemiant@hotmail.com CC: python-ideas@python.org

On Feb 5, 2014 7:34 AM, "Alex Rodrigues" <lemiant@hotmail.com> wrote:

...

Hi everyone,

This is my first time on the Python mailing lists.

Welcome!

...

I've been learning a lot about how python is run recently and today I thought I ran across an idea which might create in interesting discussion. Today I was writing a piece of software where I had a block of code that would take a bunch of local variables, apply some transformations to them and then output them as a a string to a log. Then I realized that I actually wanted to reuse this code in multiple locations - there were multiple cases where I might need to do this. My natural inclination was to write a function in order to maintain DRY programming. This was prohibitively challenging, however, since the code itself interacted with lots of variables in the namespace. The number of arguments to the function would have to be very large and possibly would change on a fairly regular basis. This seems like a fairly common problem in programming, having a piece of code which is both reused and heavily integrated with the namespace making it necessary to use copy-paste. As a solution to this I propose the idea of an inline function. An inline function would run in it's parent's namespace instead of creating a new one.

This sounds like one aspect of Ruby's blocks. Nick Coghlan (one of the more active Python committers) wrote up a nice block post that describes them:

http://www.curiousefficiency.org/posts/2011/10/correcting-ignorance-learning...

...

What do you think?

You've described an interesting idea. I think Paul is right that it's a bit of a code smell that the function is so complex as to make "inlining" desirable. Yet, it seems like there is a practical idea hiding in there. Keep at the core idea and see if anything more focused, including a better use case, pops out.

To be honest the end result will likely be no changes to Python (see this list's archives and the list of deferred/rejected PEPs). However, you and others will probably learn a thing or two in the process. Unlike some traffic on this list, your idea and the discussion of it are well within the realm of not wasting people's time.

-eric

[1] Though not exactly applicable to your "inline" functions, aother Ruby blocks-like idea is statement local namespaces. See PEPs 403 and 3150 (at http://www.python.org/dev/peps/).

On Thu, Feb 6, 2014 at 6:48 AM, Alex Rodrigues <lemiant@hotmail.com> wrote:

I'm not under any illusions about how hard it is to actually get something added to standard python . But I thought this would be a fun and informative undertaking that possibly even helps people. Thank you for your kind words.

A good discussion is seldom a waste of time. :)

After some consideration I have come up with another use case that demonstrates a little more of how this might be useful: Let's say that I am a weather forecaster and I maintain a large library of different simulation techniques. All the techniques need to get their initial data from some database. Let's take a look at some way of writing this code and their advantages/disadvantages:

Classes - This is probably the best option currently available. Make a simulation class that has a initialize function and have each simulation inherit from it. This also has some drawbacks though, since I must now instantiate a copy of the object to find the output of the simulation, which makes calling the function harder (especially on a one time basis).

This was my immediate thought on reading your description. Either that, or your start_simulation function is really a single simple class. Try this: class simulation(object): # the explicit inherit is optional in Py3 def __init__(self): self.date = datetime.utcnow() self.date.replace(tzinfo=utc) self.start_data = urllib2.urlopen('http://www.source.com/seed_sims').read().split() self.num_clouds = int(start_data[0]) self.temp = float(self.start_data[1]) # List comprehension is simpler than the nested loop self.grid = [[0]*int(self.start_data[3]) for _ in range(int(start_data[2]))] def sim_basic(): sim = simulation() return sim.temp def sim_iterative(): sim = simulation() for row in sim.grid: # You had this, which won't work anyway: # for cell in row: # cell = random.random() for i in range(len(row)): row[i] = random.random() for i in range(10): for row in sim.grid # Again, modifying cell wouldn't work for i in range(len(row)): row[i] += random.random()-0.5 return average([cell for row in sim.grid for cell in row]) def sim_clouds(): sim = simulation() return sim.temp - sim.numclouds/10. As you see, it's a very straight-forward translation (apart from the bits that I had to change because Python's iteration isn't done with writable references). It's easy to add more to the namespace - and there's no way that it can accidentally collide with a variable name used elsewhere in the simulation. With your model, suppose you wanted to have start_simulation return a thing called 'row'... oops, it's been overwritten by sim_iterative, that's not good. With my model, the new thing would be called self.row or sim.row, so it won't collide. Yes, it's slightly more typing; and when you refactor something (say you find that 95% of your sim_*() functions are deriving the value temp_per_cloud from self.temp and self.num_clouds - yeah, I know, shows how much I know about weather), you have to stick 'sim.' in front of it. But it's guaranteed to be safe. It gives a pretty big benefit in readability, since you know where to go looking for something. Remember, assigning to a name inside a function makes it local; anything you _don't_ assign to is automatically global. That means that you could be happily referencing a global, but then start_simulation gets updated to assign to that name... now your code's broken. This will be very confusing. I'm sure there are situations where inline blocks are extremely useful. I'm just concerned that, in a language where scoping is defined by fairly free rules based on assignment, it'd be way too easy to have the inline block break stuff all over the place. ChrisA