[Tutor] What are *appropriate* uses for exec() and eval() ?

[Steven D'Aprano]

On Mon, Feb 16, 2015 at 01:52:16PM -0600, boB Stepp wrote:
> I have heard periodically about the potential evils of using exec()
> and eval(), including today,  on this list. I gather that the first
> requirement for safely using these functions is that the passed
> argument MUST be from a trusted source. So what would be examples
> where the use of these functions IS appropriate?

The flippant reply would be "there aren't any".

But that is not true.

In the broader context of programming in general, not just Python, the 
use of eval/exec is incredibly powerful because it allows you to write 
dynamic code that uses information available at runtime to solve 
problems which cannot even be expressed at compile time.

Think about it like this:

A programming language is like a toolbox filled with tools for solving 
problems. You combine those tools like Lego blocks, combining them in 
different ways to make new tools. One of those Lego blocks is a robot, 
called eval or exec, which you can instruct to make new tools, instead 
of making them yourself.

There are various downsides: the extra complexity of telling the robot 
which Lego blocks to use, instead of just directly using the blocks 
yourself, means that using the robot is slower, more complex, harder to 
read, error messages are less useful, and if your instructions contain 
data coming from strangers, they may be able to subvert your intention, 
sneak instructions into your code, and take control of the robot. But it 
means you can put off the decision for which Lego block to use until 
runtime when more information is available.

exec is literally a Python interpreter embedded inside Python, so if you 
have a particularly hard problem to solve, one of the ways you can solve 
it is to write a program to *write a program to solve it*, then use exec 
to run that second program.

All this discussion has been very abstract. Here are some concrete 
examples of good use of eval/exec.

In the standard library, we have the timeit module which takes a code 
snippet from the user, executes it as Python code, and measures how long 
it takes. There's no way to take *code* from the user except as a 
string, if you type it directly Python will interpret it immediately. To 
delay execution, you have to put the code inside a string, and then 
later interpret the string as Python code. In other words, exec.

Likewise, we have the doctest module. Inside your function docstrings, 
you can write samples of interactive output:

def spam(n):
    """Return n lots of spam.

    >>> spam(3)
    'spam spam spam'

    """
    ...


The doctest module scans the docstring, extracts anything which looks 
like interactive output (starting with >>> prompt), execs that text as 
Python code, and checks that the output matches what you gave it. Your 
sample code is *executable documentation*, so long as you remember to 
run dotest over it, you can always be sure that the sample code is 
correct.

In the collections module, there is a factory function called namedtuple 
for creating record-like tuples with named fields. How it works is you 
provide the name of the fields, they get plugged into a class definition 
template, and the template executed as Python code, which creates a new 
class. That class is returned for you to use:

py> from collections import namedtuple
py> Record = namedtuple("Record", "x y z")
py> point = Record(23, 42, 19)
py> print(point)
Record(x=23, y=42, z=19)
py> point.x
23


Here is the original version which eventually became part of the 
collections module:

http://code.activestate.com/recipes/500261-named-tuples/

Here is a fork of that recipe. It uses an inner class for the new 
namedtuple class. The only thing which needs exec is the __new__ method.

http://code.activestate.com/recipes/578918-yet-another-namedtuple/

This demonstrates a powerful truth about Python: *most of the time* you 
don't need to use exec or eval because the standard language features 
are powerful enough to solve the problem for you. Using a dynamically 
created inner class is *almost* enough to solve this problem, only the 
__new__ method defeats it. If our requirements where just a little less 
demanding, we could avoid exec completely.

In some languages, if you want to define functions are runtime, the only 
way to do it is to write a function template, fill in the blanks at 
runtime, then exec it:


template = """
def add(x):
    return x + %s
"""
namespace = {}
exec(template % 10, namespace)
addTen = namespace['add']
print(addTen(23))


With Python, going to all that trouble is unnecessary:


def factory(n):
    """Return a new function which adds n to its argument."""
    def add(x):
        return x + n
    return add

addTen = factory(10)
print(addTen(23))


The result is easier to read, faster, and more secure.


-- 
Steve