[IronPython] Parrotbench and Pystone - adding external tests to IronPython

Martin Maly martmaly at exchange.microsoft.com
Fri Aug 12 02:59:54 CEST 2005


Hello,

last week I promised to send out a description how to incorporate Pie-thon (a.k.a. parrotbench) and Pystone tests into the IronPython distribution to run the tests along with the IronPython test suite. Here is how to do it.

The environment in which we run tests looks like this:

IRONPYTHON_BASE
├───External
│   ├───Parrotbench
│   └───Regress
│       └───Lib
└───Src
    ├───bin
    ├───Examples
    ├───IronMath
    ├───IronPython
    │   ├───AST
    │   ├───Hosting
    │   ├───Modules
    │   └───Objects
    ├───IronPythonConsole
    ├───IronPythonTest
    └───Scripts


"External" directory contains all external tests:
 - CPython library + tests inside the "Regress" subdirectory
 - Parrotbench test in the "Parrotbench" subdirectory

"Src" is the IronPython code that we release.
 - For the purpose of release we rename the directory (for example "IronPython-0.9"),
   renaming it back to "Src" will plug it right in place.

"Parrotbench" directory contains the parrotbench benchmark files (b*.py, parrotrun.py, ...)
To get parrotbench sources, download IronPython 0.6 from http://www.ironpython.com <http://www.ironpython.com> .
The IronPython 0.6 distribution contains the Parrotbench in its "tests" directory.

The parrotrun.py is a file Jim wrote to make running the tests easier. It has not been part of our releases yet and unfortunatelly I modified it so the version you get from IronPython 0.6 won't work. At the bottom of this email is the right version of the parrotrun.py that will work with the tests. Starting with our next release, we will include the parrotrun.py with the release so that things work smoother.

Pystone + CPython regression test suite
Download the 2.3.5 release of CPython from http://www.python.org/2.3.5 <http://www.python.org/2.3.5> 
Library and tests are in the "Lib" directory of the Python installation.
Copy the whole Lib directory into "Regress".

Once you have combined IronPython release files with parrot-bench from IronPython 0.6 and standard CPython libraries, you can run the tests by exe

Running the tests:

To run the tests, execute:
Src\bin\IronPythonConsole.exe Src\Scripts\RunTests.py

The RunTests.py accepts command line parameters, one of them being the type of the test - short, medium, long (run "RunTests.py -?" for more details)

Known issue is running "long" version of regression tests (runtests.py regr long). We had to make some changes to the Python test suite to get the long version running so if you run it, it will fail. For now, I would recommend running:

"IronPythonConsole.exe RunTests.py iron pystone parrot long"

I hope this helps for now and if you have any questions, don't hesitate to contact me so that we can address them.

Thanks and keep in touch,
Martin

parrotrun.py file:

##################################################

""" This provides a more convenient harness for running this
    benchmark and collecting separate timings for each component.
"""

from time import clock
import sys

def main(type="short"):
    t0 = clock()
    import b0
    import b1
    import b2
    import b3
    import b4
    import b5
    import b6  
    print 'import time = %.2f' % (clock()-t0)

    if type == "short":
        N = 1
        tests = [b0,b1,b2,b3,b4,b6]
    elif type == "medium":
        N = 2
        tests = [b0,b1,b2,b3,b4,b5,b6]
    elif type == "long":
        N = 4
        tests = [b0,b1,b2,b3,b4,b5,b6]
    else:
        raise AssertionError("Unknown test type: %s" % type)

    results = {}

    t0 = clock()
    for i in range(N):
        for test in tests:
            ts0 = clock()
            test.main()
            tm = (clock()-ts0)
            results.setdefault(test, []).append(tm)
            print '%.2f sec running %s' % ( tm, test.__name__)

    for test in tests:
        print '%s = %f -- %r' % (test.__name__, sum(results[test])/N, results[test])

    print 'all done in %.2f sec' % (clock()-t0)

if __name__=="__main__":
    kind = "short"
    if len(sys.argv) > 1:
        kind = sys.argv[1]
    main(kind)

##################################################

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20050811/0b0f472c/attachment.html>


More information about the Ironpython-users mailing list