[Python-Dev] PEP 3146: Merge Unladen Swallow into CPython

[Collin Winter]

Hey Maciej,

On Thu, Feb 11, 2010 at 6:39 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Snippet from:
>
> http://codereview.appspot.com/186247/diff2/5014:8003/7002
>
> *PyPy*: PyPy [#pypy]_ has good performance on numerical code, but is
> slower than Unladen Swallow on non-numerical workloads. PyPy only
> supports 32-bit x86 code generation. It has poor support for CPython
> extension modules, making migration for large applications
> prohibitively expensive.
>
> That part at the very least has some sort of personal opinion
> "prohibitively",

Of course; difficulty is always in the eye of the person doing the
work. Simply put, PyPy is not a drop-in replacement for CPython: there
is no embedding API, much less the same one exported by CPython;
important libraries, such as MySQLdb and pycrypto, do not build
against PyPy; PyPy is 32-bit x86 only.

All of these problems can be overcome with enough time/effort/money,
but I think you'd agree that, if all I'm trying to do is speed up my
application, adding a new x86-64 backend or implementing support for
CPython extension modules is certainly north of "prohibitively
expensive". I stand by that wording. I'm willing to enumerate all of
PyPy's deficiencies in this regard in the PEP, rather than the current
vaguer wording, if you'd prefer.

> while the other part is not completely true "slower
> than US on non-numerical workloads". Fancy providing a proof for that?
> I'm well aware that there are benchmarks on which PyPy is slower than
> CPython or US, however, I would like a bit more weighted opinion in
> the PEP.

Based on the benchmarks you're running at
http://codespeak.net:8099/plotsummary.html, PyPy is slower than
CPython on many non-numerical workloads, which Unladen Swallow is
faster than CPython at. Looking at the benchmarks there at which PyPy
is faster than CPython, they are primarily numerical; this was the
basis for the wording in the PEP.

My own recent benchmarking of PyPy and Unladen Swallow (both trunk;
PyPy wouldn't run some benchmarks):

| Benchmark    | PyPy  | Unladen | Change          |
+==============+=======+=========+=================+
| ai           | 0.61  | 0.51    |  1.1921x faster |
| django       | 0.68  | 0.8     |  1.1898x slower |
| float        | 0.03  | 0.07    |  2.7108x slower |
| html5lib     | 20.04 | 16.42   |  1.2201x faster |
| pickle       | 17.7  | 1.09    | 16.2465x faster |
| rietveld     | 1.09  | 0.59    |  1.8597x faster |
| slowpickle   | 0.43  | 0.56    |  1.2956x slower |
| slowspitfire | 2.5   | 0.63    |  3.9853x faster |
| slowunpickle | 0.26  | 0.27    |  1.0585x slower |
| unpickle     | 28.45 | 0.78    | 36.6427x faster |

I'm happy to change the wording to "slower than US on some workloads".

Thanks,
Collin Winter