advice needed for lazy evaluation mechanism

[Steven D'Aprano]

On Sun, 08 Nov 2009 14:41:27 -0800, markolopa wrote:

> Hi,
> 
> Could you please give me some advice on the piece of code I am writing?
> 
> My system has several possible outputs, some of them are not always
> needed. I started to get confused with the code flow conditions needed
> to avoid doing unnecessary work. 

How many dozens of man-hours (yours, and the people who have to maintain 
the software after you have moved on) of confusion are you going to spend 
to avoid how many microseconds of execution time? So what if your system 
takes 35ms instead of 18ms to calculate the result?

As Tony Hoare famously said: "We should forget about the small 
efficiencies, say about 97% of the time: Premature optimization is the 
root of all evil."

Of course, all of this assumes that the routines you are trying to avoid 
calling don't require hours of running time each time you call them... 


> So I am trying to restructure it using lazy evaluation.

Oh great, avoiding confusion with something even more confusing.


> - Is there a more standard (pythonic) way to do what I am trying to do?

Yes. Avoid it. Do the simplest thing that works until you *know* -- 
because you have profiled it -- that it is too slow. Until then, all that 
complication you've built, all that over-engineered jungle of classes and 
abstract classes, is unnecessary. I find it beyond all credibility that 
your data is so complicated that you need a plug-in system just to manage 
the methods you need to calculate your data.

Just create some properties, like this one:

class Example(object):
    def __init__(self, birth_year):
        self.birth_year = birth_year
    @property
    def age(self):
        return 2009 - self.birth_year  # FIXME -- it won't be 2009 forever

And there you have a lazily-evaluated age property. Not complicated 
enough? The 3ms it takes to calculate the age is too long? Cache it!

class Example(object):
    def __init__(self, birth_year):
        self.birth_year = birth_year
        self._age = None
    @property
    def age(self):
        a = self._age
        if a is None:
            a = 2009 - self.birth_year
            self._age = a
        return a

Now all you need is to invalidate the cache if the birth_year changes. So 
you make birth_year a property too:

class Example(object):
    def __init__(self, birth_year):
        self.birth_year = birth_year
    @property
    def birth_year(self):
        return self._birth_year
    @property.setter
    # Requires Python 2.6. Also untested.
    def birth_year(self, year):
        self._birth_year = year
        self._age = None
    @property
    def age(self):
        a = self._age
        if a is None:
            a = 2009 - self.birth_year
            self._age = a
        return a



> Are there libraries, design patterns, functional programming structures
> to use to achieve what I am looking for (i.e. am I trying to reinvent
> the wheel)?

The most important buzzwords you want are YAGNI and Premature 
Generalisation, and perhaps a touch of Architecture Astronaut:

http://www.joelonsoftware.com/items/2008/05/01.html


> - Is the coding style good?
> - Can I avoid the eval command in Repository.add_routine? What I want
> there is to be able to have a generic code for the repository which does
> not depend on the files containing the routines I want it to hold.

You mean the exec?

cmd = "from %s import %s\nroutine = %s()" % (file_name, class_name, 
class_name)
exec(cmd)  # XXX: ugly


Yes. Untested:

module = __import__(filename)
class_object = getattr(module, class_name)
routine = class_object()

Of course you can turn that into a one-liner:

routine = getattr(__import__(filename), class_name)()




-- 
Steven