[Tutor] Scientific Notation + 18 Digit Precision

[Danny Yoo]

On Sun, 27 Nov 2005, Hubert Fitch wrote:

> Thanks to all of you (Alan, Chris, Kent) for your responses!
>
> Before I try to follow up on these suggestions, it might be good to
> examine the display function module, and provide a few lines from a .py
> module that needs display.

[warning: long message.  It's hard to show how to refactor code without
actually doing it.  *grin*]

Hi Hubert,

The code that you've shown is a bit unusual since the heart of it is
'exec'.  I'll ignore that issue for the moment since it's so core to what
the program intends.  But instead, I'll concentrate on the overall
structure of the program, and try to give some constructive criticism.

(I'll try to condemn the use of exec at the end of this message.  *grin*)

The DisplayPyFile function feels too big: its inner workings obscures the
main idea of the function: to execute every functional definition in the
input file and display each result value.  Let's see if we can refactor
the program a bit to make that logic clearer.


There's a whole block of code here that does the parsing of each
definition:

>         line = line.strip()
>         if not line:
>             continue
>         leftAndRight = line.split('#',2)
>         comment = ''
>         if len(leftAndRight)>1:
>             comment = leftAndRight[1]
>         assignment = leftAndRight[0]
>         leftAndRight = assignment.split('=')
>         if len(leftAndRight) == 2:
>             name = leftAndRight[0].strip()

Conceptually, the "input" into this block is a line from the file, and the
"output" is either an equation or not.  Let's informally say that an
equation is a 3-tuple (name, assignment, comment).

We can take the block above and turn it into a function:

##########################################
def extractEquation(line):
    """Either returns a 3-tuple (name, assignment, comment) from the line,
    or None if we can't extract a definition from the line."""
    line = line.strip()
    if not line:
        return None
    leftAndRight = line.split('#',2)
    comment = ''
    if len(leftAndRight) > 1:
        comment = leftAndRight[1]
    assignment = leftAndRight[0]
    leftAndRight = assignment.split('=')
    if len(leftAndRight) == 2:
        name = leftAndRight[0].strip()
        return (name, assignment, comment)
    return None
##########################################


This function is a bit messy, but we can clean that up in a moment.  But
let's see how this might work:

######
>>> extractEquation("")
>>>
>>> extractEquation("x = 42")
('x', 'x = 42', '')
>>> extractEquation("x = 42  # this is a test comment")
('x', 'x = 42  ', ' this is a test comment')
######

So this appears to work, although it's a bit rough.


If we have this extractEquation, then the logic in the original
DisplayPyFile()  simplifies to:

##################################################################
def DisplayPyFile(fname, context=None):
    if context is None:
        context = globals()
    f = open(fname)
    lines = f.readlines()
    f.close()

    for line in lines:
        equation = extractEquation(line)
        if equation:
            name, assignment, comment = equation
            exec(assignment, context)
            if context.get(name,None) is not None:
                value = context.get(name,'???')
                print "%10s  =  %18.15e (%s)\t[%s]" % (name, value,
                                                       assignment,
                                                       comment)
            else:
                print "%s not defined. %s" % (name, assignment)
        else:
            print line
##################################################################


We can chip at DisplayPyFile() a bit more: there's a promising block here:

######################################
name, assignment, comment = equation
exec(assignment, context)
if context.get(name,None) is not None:
    value = context.get(name,'???')
######################################

This block tries to evaluate the equation and get its value.  Actually, on
further inspection, some of the logic is redundant: if we get into that
'if' block, context.get(name) has to work --- the default clause in
"context.get(name, '???') is useless.  We can simplify the block to:

######################################
name, assignment, comment = equation
exec(assignment, context)
if context.get(name,None) is not None:
    value = context.get(name)
######################################


Let's turn that equation-evaluating block into a real function:

##################################################################
def evaluateEquation(equation, context):
    """Evaluates the equation in a given context.  If we can't get a
    value, returns None."""
    assert equation != None
    name, assignment, comment = equation
    exec(assignment, context)
    return context.get(name, None)
##################################################################


Let's do a quick pass to see if this function works out:

######
>>> evaluateEquation(('x', 'x=42', ''), globals())
42
######

Ok, good; it's doing something.  *grin*



Once we have evaluateEquation(), we can use it to simplify DisplayPyFile
some more:

##################################################################
def DisplayPyFile(fname, context=None):
    if context is None:
        context = globals()
    f = open(fname)
    lines = f.readlines()
    f.close()

    for line in lines:
        equation = extractEquation(line)
        if equation:
            value = evaluateEquation(equation, context)
            if value:
                name, assignment, comment = equation
                print "%10s  =  %18.15e (%s)\t[%s]" % (name, value,
                                                       assignment,
                                                       comment)
            else:
                print "%s not defined. %s" % (name, assignment)
        else:
            print line
##################################################################



Finaly, there's one more block that seems ripe, the part that prints out
the value and the equation:

#######################################################
if value:
    name, assignment, comment = equation
    print "%10s  =  %18.15e (%s)\t[%s]" % (name, value,
                                           assignment,
                                           comment)
else:
    print "%s not defined. %s" % (name, assignment)
#######################################################


And, of course, I'll push this into a function.  *grin*

#######################################################
def printEquationAndValue(equation, value):
    """Prints out the equation and its value."""
    if value:
        name, assignment, comment = equation
        print "%10s  =  %18.15e (%s)\t[%s]" % (name, value,
                                               assignment,
                                               comment)
    else:
        print "%s not defined. %s" % (name, assignment)
#######################################################




The refactored version of DisplayPyFile() now looks like:

#################################################################
def DisplayPyFile(fname, context=None):
    if context is None:
        context = globals()
    f = open(fname)
    lines = f.readlines()
    f.close()
    for line in lines:
        equation = extractEquation(line)
        if equation:
            value = evaluateEquation(equation, context)
            printEquationAndValue(equation, value)
        else:
            print line
#################################################################




Whew.  *grin* I think I'll stop refactoring here for a moment and look
back at your original question.  Your original question was:

> Most of the formatted results are already provided in the correct form.
> What should be changed in this display function module?

With all the refactoring, we can now point definitively at either
evaluateEquation() or printEquationAndValue().  *grin*


Seriously though, can you give us one of the definitions that's giving an
incorrect result?  With the revised program, we can now test if it's
either evaluateEquation() that's messing up, or if it's the other function
printEquationAndValue().


Actually, I know that there's something wrong with evaluateEquation(),
because it's using Python's rules for doing arithmetic, and that includes
its behavior on integer division.  Here's an example:

######
>>> evaluateEquation(extractEquation('x = 1 / 2'), globals())
0
>>> evaluateEquation(extractEquation('x = 1.0 / 2'), globals())
0.5
######

There are a few ways of fixing this.  The easiest is to start Python up
with the '-Qnew' command line option, which turns on true division
globally:

######
mumak:~ dyoo$ python -Qnew
Python 2.3.5 (#1, Mar 20 2005, 20:38:20)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1809)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1 / 2
0.5
>>> eval('1 / 2')
0.5
######

and since it looks like you're a beginner programmer, I'd recommend this
approach: it means you won't have to change much.  Unfortunately, this
means that you'll have to remember to use '-Qnew' all the time, and that's
bug-prone.


An alternative way is to fix the bug is to modify extractEquation and
evaluateEquation so that it doesn't depend on Python's evaluation
functions.  To an experienced programmer, I'd recommend that approach, to
replace the call to 'exec' with a customized parser and evaluator: we have
more control over the arithmetic.

If you really need something like this, one of us can probably cook up
such an evaluator for you.


Best of wishes to you!