Speed up Python by up to 5% ?

[Skip Montanaro]

    Edward> In short, this is a very tight loop.  But can't it be improved?
    Edward> The byte code is littered with SET_LINENO opcodes.  Has anyone
    Edward> considered moving the test:

    Edward> if (things_to_do || --tstate->ticker < 0)
    Edward>     { /* handle periodic things */ } // not usually executed.

    Edward> out of the main loop an into the case for SET_LINENO? 

This wouldn't work if the byte code being executed was generated by a
"python -O" session.  In this situation, there are no SET_LINENO
instructions at all.

    Edward> Are typical opcodes much more expensive than the simplest
    Edward> SET_LINENO case?  I wrote a simple program based on dis to count
    Edward> the static occurrences of bytecodes.  The results of running
    Edward> this program on most of the modules of Leo are as follows:

    ...

You might try defining the DYNAMIC_EXECUTION_PROFILE and DXPAIRS macros then
recompiling the interpreter (I think only the sys module and ceval.c need to
be recompiled).  When you run the interpreter you can call sys.getdxp() to
get the dynamic opcode frequencies.  If you've defined both macros, you'll
get dynamic opcode pair info.  Without DXPAIRS you'll get just the straight
frequencies.

About a year ago I wrote an XML-RPC server to which people could post opcode
frequency info (and retrieve it by version), but nobody bit on the idea.
Here's the entire client-side code necessary to register this info:

    def rle(l):
        import types
        newl = []
        lastel = None
        count = 0
        for elt in l:
            if elt == lastel:
                count = count + 1
                continue
            elif lastel is not None:
                if isinstance(lastel, types.ListType):
                    lastel = rle(lastel)
                newl.append([lastel, count])
            lastel = elt
            count = 1
        if isinstance(lastel, types.ListType):
            lastel = rle(lastel)
        newl.append([lastel, count])
        return newl

    def send_instruction_counts(appname, email):
        if not hasattr(sys, "getdxp"):
            return
        dxpserver = xmlrpclib.ServerProxy("http://manatee.mojam.com:7304",
                                          encoding="utf-8")
        dxpserver.add_dx_info(appname, email, sys.version_info[:3],
                              rle(sys.getdxp()))

    atexit.register(send_instruction_counts, "musi-cal", "skip at pobox.com")

I would probably do the data compression a bit differently today (pickle the
rle() output, then gzip it for transport).

-- 
Skip Montanaro
skip at pobox.com
consulting: http://manatee.mojam.com/~skip/resume.html