Re: [Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide build)

On Mon, Oct 1, 2012 at 7:54 AM, Antoine Pitrou wrote:

On Sun, 30 Sep 2012 21:49:20 -0400 Brett Cannon wrote:

...

...

Note that Mako can use the Markupsafe library for faster operation. This will skew the result if one of your Pythons has Markupsafe installed and the other does not.

Should probably have the benchmark print out a warning when markupsafe is used. Turns out I have it installed in my user directory for Python 2.7 so that probably came into play.

...

Perhaps the benchmark runner should launch its subtests in a controlled environment to avoid such issues?

If we had venv in Python 2.7 that might be easy to do, but otherwise is there an easy way without having to try to pull in virtualenv or something crazy like a chroot or something?

The mako benchmark could manually exclude markupsafe from sys.modules. That only addresses that specific benchmark, though)

Good point. Might be a good short term fix but it would be nice to have a solution to prevent similar issues in the future.