[Python-Dev] Dynamic bytecode analysis

Skip Montanaro skip@pobox.com
Thu, 27 Feb 2003 08:32:59 -0600


    damien> Im not sure how I could properly excersise Zope, and Mailman
    damien> doesn't run under windows. PySol is do-able. Any other
    damien> suggestions as to apps worthy of profiling?

Zope can be exercised by simply creating a shell script which retrieves a
given set of URLs from the server.  Obviously, what you fetch will depend on
what applications have been built using the server.

Using sitecustomize, all apps can be instrumented without modification to
dump their instruction counts at program termination.  Here's my new
sitecustomize.py, which dumps to my dxp server on manatee.mojam.com (please
change EMAIL to suit if you decide to use this):

    import atexit
    import sys, os, xmlrpclib

    EMAIL = 'skip@pobox.com'

    if not hasattr(sys, 'getdxp'):
        print >> sys.stderr, "instruction count tracking not available"
    else:
        def rle(l):
            newl = []
            lastel = None
            count = 0
            for elt in l:
                if elt == lastel:
                    count = count + 1
                    continue
                elif lastel is not None:
                    if isinstance(lastel, list):
                        lastel = rle(lastel)
                    newl.append([lastel, count])
                lastel = elt
                count = 1
            if isinstance(lastel, list):
                lastel = rle(lastel)
            newl.append([lastel, count])
            return newl

        def rld(l):
            newl = []
            for elt, count in l:
                if isinstance(elt, list):
                    elt = rld(elt)
                newl.extend([elt]*count)
            return newl

        def send_instruction_counts(email):
            print >> sys.stderr, "sending instruction counts to manatee"
            appname = os.path.basename(sys.argv[0]) or "interactive"
            dxpserver = xmlrpclib.Server('http://manatee.mojam.com:7304')
            dxpserver.add_dx_info(appname, email, sys.version_info[:3],
                                  rle(sys.getdxp()))

        atexit.register(send_instruction_counts, EMAIL)

You can grab data from the server like so:

    from sitecustomize import rld
    x = dxpserver.get_dxp_info([2,3,0])
    counts = rld(x['counts'])
    pairs = rld(x['pairs'])

Skip