[Python-ideas] Optional extra globals dict for function objects

[Brett Cannon]

On Nov 17, 2007 11:27 AM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> I set out trying to redo the 3.0 autosuper metaclass in 2.5 without
> bytecode hacking and ran into a problem: a function's func_globals isn't
> polymorphic. That is, the interpreter uses PyDict_* calls to access it,
> and in one case (LOAD_GLOBAL), actually inlines PyDict_GetItem manually.
> If it weren't for this, I could have easily done 3.0 super without
> bytecode hacking, by making a custom dict that allows another dict to
> shadow it, and putting the new super object in the shadowing dict.
>
> I know it's for performance, and that if func_globals were made
> polymorphic, it'd bring the pystone benchmark to its knees, begging for
> a quick and merciful death. That's not what I'm proposing.
>
> I propose adding a read-only attribute func_extra_globals to the
> function object, default NULL. In the interpreter loop, global lookups
> try func_extra_globals first if it's not NULL. It's accessed using
> PyObject_* functions.
>

My initial response is "eww".  I say this as I don't want to
complicate the scoping rules anymore than they are.  This adds yet
another place to check for things.  While it might not be a nasty
performance hit (although you neglect to say what happens if something
is not found in func_extra_globals; do you check func_globals as well?
 That will be a penalty hit), it does complicate semantics slightly.

> Here are the reasons I think this is a good idea:
>
> - It should have near zero impact on performance in the general case
> because NULL checks are quick. There would be another attribute in the
> frame object (f_extra_globals), almost always NULL.
>

That is only true if you skip a func_globals check if the
func_extra_globals check doesn't happen.

> - Language enhancement prototypes that currently use bytecode hacking
> could be accomplished with a method wrapper and a func_extra_globals
> dict. The prototypes could be pure Python, and thus more general, less
> brittle, and easier to get right. Hacking closures is nasty business.

Which are what?  the auto-super example is not exactly common.

>
> - I'm sure lots of other stuff that I can't think of, where it'd be nice
> to dynamically add information to a method or function that can be
> accessed as a variable. Pure-Python function preambles whose results can
> be seen by the original function would be pretty sweet.

Basing an idea on unknown potential is not a good reason to add
something to the language.  I don't think the Air Force needs to
protect against flying pigs just because there is the possibility
someone might genetically engineer some to carry nuclear bombs.  =)

>
> - Because func_extra_globals would be read-only and default NULL, it'd
> almost always be obvious when it's getting messed with. A
> wrapper/decorator or a metaclass, and a call to types.FunctionType()
> would signal that.

Read-only?  Then how are you supposed to set this?  Do you want to
introduce something like __build_class__ for functions and methods?
Requiring the use of Types.FunctionType() will be a pain and dilute
the usefulness.

>
> - func_globals would almost never have to be overridden: for most
> purposes (besides security), shadowing it is actually better, as it
> leaves the function's module fully accessible.
>

If that's the case why worry about func_extra_globals?  =)  It solves
%95 of the uses you might have (and I suspect 94% of the uses are "I
don't need to muck with func_globals").

> Anybody else think it's awesome? :) How about opinions of major suckage?
>

I'm -1 on the idea personally.

> If it helps acceptance, I'd be willing to make a patch for this. It
> looks pretty straightforward.

It always helps acceptance, it's just a question of whether it will
push it over the edge into actually being accepted.

-Brett