[Baypiggies] Decorators

Reitmeyer_Richard at emc.com Reitmeyer_Richard at emc.com
Mon Jul 19 20:49:14 CEST 2010


Glen Jarvis wrote:

> I'm trying to put together a very *very* basic
> introduction to decorators.  This has been done before
> (several times), so I know there is no silver bullet.
> 
> What I'm focusing on is: examples of decorators -- from a
> user perspective -- not how to write one, but how to use
> one.
> 
> Could you send any examples (off-list) (either real or
> fictional) that you would use to help explain to a friend
> "This is a decorator for your use."  (again, currently
> avoiding the conversation on how to write one).
> 
> For example, Django has a "login_required" decorator that
> is handy so that one doesn't have to understand how the
> login process works. Having the decorator will either work
> if one is logged in; or will first prompt for a login if
> one is not.
> 
> I can think of the logging example also -- where the
> decorator will log when the function is being
> called. We've seen this presented as a defacto standard on
> how a decorator is helpful.
> 
> Any examples that you can send -- even if it's a line or
> two -- would be helpful. I think *explaining* decorators
> are hard. If you already get decorators, then the talks
> will bore you to tears. If you don't, the explanations
> never seem good enough.
> 
> The focus of my very quick presentation will be on the
> *why to use* (not how to write) -- which seems to be the
> biggest stumbling block for those just learning. I think
> this is mitigated by giving concrete small-but-not-toy
> examples.
> 
> Once a person gets the *why*, I think its easier to get to
> the *how* (although there can be some new concepts in
> there too)...
> 
> Let the brainstorming/ideas roll :) pretty please :)


Hello, Glen.

Here's my stab at it.  Complete with typos, no doubt.

Decorators do several things, but the best single sentence I can come up with is that they allow extension of existing code through (mathematical) composition without requiring change to callers.

If you think of the Template Method design pattern, the most common case is to have the TM call just a single concrete function with some before and after, even though the TM could call as many functions as it liked.  Mathematical composition is just that common TM case: it allows us to change the 'before' and 'after' although not the 'guts.'  And we don't have to restrict it to functions in a class hierarchy, we can do it with any function we wish.

There are many reasons we might want to augment existing code with before and after processing, while leaving the body of the code alone:
- Add logging at function entrance / exit
- Add security check (EG, Django permissions) at function entrance / exit
- Grab and release resources (EG, locks) at function entrance / exit

If you have to do this sort of thing, structurally there's no reason you couldn't take your existing code and edit all of it to do the changes.  You'd put in try ... finally blocks and be good:

	def foo(a, b, c):
		# new setup code here
		try:
			# existing code goes here
		finally:
			# new teardown code does here

(If you have ever written code like this, when this talk is done I hope you'll realize you had a candidate for a decorator.)

Of course, if you had to do that for foo() and bar() and baz() with pretty much the same setup/teardown code, you'd probably be better served by writing something closer to this:

	def my_wrapper(fn, *args, **kwds):
		# setup code here
		try:
			fn(*args, **kwds)
		finally:
			# teardown code here

As an aside, note how much easier this is in python, with its support of generic programming, than it would be in C or C++, where you'd pretty much need to do different wrappers for functions having different sets of arguments types/counts.

The 'my_wrapper' function is much better than fixing foo() and bar() and baz()... except now you need to edit all of the call sites that use to call foo() and replace them with my_wrapper(foo, ...).

That will work, technically.

BUT, if we're sure no one should ever call a naked foo(), which we are because we were originally going to change foo directly, there is danger with this approach: We could miss one call site in our editing pass.  Or someone else could check in their code which uses foo() directly, because they didn't know we were changing it.  Or someone using our module has old code using foo()...  This may all sound paranoid if you're just working in a small team on a project that does not ship a module for reuse.  But in broader circumstances, this could be a major problem, and defensive programming is always worth some consideration.  We would be safer if we renamed our foo() to something like _foo() and made a whole new foo() like this:

	def foo(a, b, c):
		my_wrapper(_foo, a, b, c)

Now we don't have to change call sites, but we do have to rename every function and put in a placeholder function that calls my_wrapper on the private one.  

That's still a bit tedious, and we have all of these extra _foo functions running around...  Maybe we should just buckle down and edit all the original functions, like we'd have done in C?  

No!  We're python programmers, and we don't put up with crap like that :)  If we have a object with an attribute that suddenly needs more processing than "here's your value", we don't rewrite out code to replace mypoint.x with mypoint.get_x()... we use properties.  Same sort of thing here.

Python allows assignment of functions on classes, just like assignment of variables, so it is tempting to think of it like this:

	def foo(a, b, c):
		... original code here ...

	foo = lambda a, b, c: my_wrapper(foo, a, b, c)    # does not work

But that gets into problems based on when evaluation of the 'foo' in my_wrapper is done --- it happens at execution of foo(), which makes it recursive.  So we need closures, to bind foo-as-was when we do the assignment of the foo function.  In python, here is what that would look like:

	def my_decorator(fn):
		def x(*args, **kwds):
			# could just do the my_wrapper body here....
			return my_wrapper(fn, *args, **kwds)  
		return x

As you can see, 'fn' is bound when x is defined, which is when my_decorator is called.

Now we have this:

	def foo(a, b, c):
		... original code here ...

	foo = my_decorator(foo)

Pretty slick:
- write one function that has all our logic
- don't change call sites, or risk a call site is left unchanged
- apply it everywhere, at "cost" of one line like 
	foo = my_decorator(foo)
  per function to apply the logic to.

Criticism of this is that the foo = foo line is a little ugly, and perhaps hard to spot.

As python is an evolving language that works to Make Things Better, the BDFL and his merry minions adopted some syntactic sugar to make this 1) even easier; 2) stand out even better.  The language now supports this:

	@my_decorator			# just add this one line!
	def foo(a, b, c):
		... original code here ...
	

And we're set.

Questions?


Richard






More information about the Baypiggies mailing list