Creating functions programmatically

[Anders J. Munch]

Hi all,

I'm working on a project where I'm nesting Python code within a
different file format.  I'm lexing the file using the tokenize module,
extracting segments of Python code as indented blocks.

Next step is to programmatically create functions objects from token
sequences representing argument lists and function bodies.  This
shouldn't be so hard, if nothing else I can invent a unique name,
create a full textual function definition and 'exec' it in an
appropriate dict.

But perhaps someone can point me to a better way?  I am slightly
overwhelmed by the many options python has to offer for dealing with
python code (compiler, compiler.{pycodegen,symbols,syntax,..}, new,
ast, code, py_compile, compile, exec, eval, ...).

The 'compile' builtin in combination with new.function looked
promising at first, but I can't see how to make that work for
functions that take arguments and return values.

At some point I'll be doing programmatic imports as well.  I could use
__import__, but to gain access to the the full range of syntactic
options I'd just as well execute lines beginning with 'import' or
'from'.  Again, exec..in seems like a workable option.

One thing that I don't see how to achieve with exec is control over
location in diagnostic messages.  Regular Python diagnostics are fine,
but file and line should not point to the exec statement in the .py
file, but to the appropriate place in the file I'm parsing.  I could
use something akin to C's #line here, is there any Python equivalent?

or-have-I-really-found-something-that-C-can-do-but-Python-can't?-ly 
y'rs, Anders