Slowness in compile()/eval()

[Alex Martelli]

"Pedro Rodriguez" <pedro_rodriguez at club-internet.fr> wrote in message
news:pan.2002.01.17.17.35.37.25082.4228 at club-internet.fr...
    ...
> > Right.  Local is faster than global, exactly because it can be loaded
> > without a lookup (LOAD_NAME).
>
> Sorry, but there is no global variable named 'i'. Does LOAD_NAME do a
> standard look up in namespaces, while LOAD_FAST is used when the compiler
> know that a local lookup will be sufficient ?

Yes.  If you use an exec statement inside a function, then the
compiler knows it doesn't know any more whether something is to
be looked up locally or globally, so it has to use LOAD_NAME
throughout the function, and that does make things a lot worse.

> If this is so, this means that 'compile()' takes the worst, or better
> said the more 'general', guess on how to compile it, by not assuming
> that variables in expression may be in local scope.

Sure, just as the compiler has to do if it sees an exec statement
anywhere inside a function's scope.


> > Change your function to:
> >     def f(): return i+i
> > and the time difference should just about go away.
>
> No, because I don't know what expression I have to compute. I am trying

I'm talking about the difference with the function you had coded
just like this but with i as an argument: it WILL be similarly slow.

I do understand you want to compile code dynamically.  What I'm saying
is, the non-fast-local lookup will be slower in any case.


> - if this is the case, could this an idiom :
>   instead of writing :
>       code = compile("some expr", '<string>', "eval")
>       val = eval(code)
>
>   one should consider writing :
>       code = compile("def __f(v1,v2,...):\n return some_expr\n"
>                      , '<string>', 'exec')
>       eval(code)

You must use exec code here, not eval(code).

>       f = locals()["__f"]
>       val = f(v1, v2, ...)

You can then choose to access __f(v1, v2) directly.  But given
the exec, which turns off fast-lookup, it will be slow anyway.


> - the proposed idiom above looks dirty to me (locals trick), there

Unnecessary, when using the exec statement.

>   may be a better way through the 'new' module, but I don't have found
>   it yet.

There's almost invariably a better way than using exec, yes, for
just about any task.  You may need "bytecode hacks" to let you
produce the codeobject you want, though.


> - where do the definition of 'f' goes in :
>
> # ---------------------------------------------------------------------
> m.py
> # ---------------------------------------------------------------------
> def a():
>     code = compile("def f():\n pass\n", '<string>', "exec")
>     eval(code)

eval should reject code, since it was compiled for exec, not for
eval (it's a statement, not an expression).  You should open a
bug report about this, I believe, since this is the bug (in
eval) which produces the following anomaly:

>     print locals().keys()
>     print locals()["f"]
>     print f

Since no exec has been used (just eval), the compiler "knows"
the f name used in the last statement cannot be local and
only looks for it globally (and doesn't find it).

> - why is eval("some_expr") so slow with Python 2.x (will profile it)

Profiling is a good idea, but I think you won't find it slower
(once the compile step is removed, of course -- recompiling each
and every time WILL slow things down) than the same expression
in a normal setting, when both (e.g.) access data globally.


Alex