optimization question

[Bengt Richter]

On Tue, 13 Aug 2002 00:58:54 GMT, Andrew Koenig <ark at research.att.com> wrote:

>Bengt> What if you wrap the external data as soon as it comes through
>Bengt> your black box's interface?
>
>Does wrapping it involve copying it?  If so, I might be wasting
>potentially large amounts of memory.  If not, I don't see how
>I've solved the problem.
>
Not copying immutables, in any case. A wrapper object would contain
only references to the external object(s) it wraps. I thought (part of)
the problem was that some of the native methods of the wrapped object(s)
might be implemented in a suboptimal fashion for your application. So I
was suggesting wrapping as a way of interposing control over which available
external object methods might be used and when.

Even if your use is only a single expression that you might otherwise write
a function for, using an on-the-fly constructed wrapped object gives you more options
for designing the look of expressions. E.g., with a string s you can write s[x:y]==t,
and wonder what it's going to do. If you write W(s)[x:y]==t you can be using
very similar expression syntax and get control over what it does.

So instead of using your
    def eqsub(s, i, j, t):
        return (len(t) == j-i) and s[i:j] == t

and writing
    eqsub(s, i, j, t)

you can write
    W(s)[i:j]==t

using (untested!)
    class W:
        from sys import maxint
        def __init__(self, s, lo=0, hi=None):
            self.s = s; self.lo = lo
            if hi is None or hi == self.maxint: self.hi = len(s)
            else: self.hi = hi
        def __getitem__(self, i ):
            if isinstance(i, int): return self.s[i] # single char
            return W(self.s, i.start, i.stop) # simplified slice assumption for example!
        def __len__(self): return self.hi-self.lo
        def __eq__(self, other): # like eqsub
            return len(other) == (self.hi-self.lo) and self.s[self.lo:self.hi] == other
        def __str__(self): return self.s[self.lo:self.hi]

which I assume could be made reasonably fast in C.

And it still leaves the door open to writing W(s).other_method(t) or maybe something quite
different -- e.g. like caching the last slice i,j and eq t parameters
in a class variable and implementing __call__ so that writing W(s2)() gets the same effect
as W(s2)[i:j]==t using the cached parameters with the new s2 string.

For more complex computations you could maintain a cache dictionary of results keyed with
selected frequently used parameter combinations for selected methods in tuples. Might be
worthwhile especially if some of the "external objects" wrapped are referenced by URLs
to the other side of the globe and you can make freshness assumptions or query freshness
before re-accessing the remote data.

What you want to cache in the class vs instances could vary. To construct a wrapped string
representing a remote HTML thing you could pass optional URL=... DOM_ITEM=...
parameters to W's __init__. Etc., etc. Obviously you can also wrap other kinds of things.

IOW, wrapping might do something for you, depending on your problem, IWT. Think of it
as constructors for (possibly single-use) smart pointers of your own design, possibly
to implement in C later.

Or is this kind of thing not useful at all for your application?

Regards,
Bengt Richter