Firefox people point the blame squarely at jqplot: --- Comment #4 from Boris Zbarsky (:bz) <bzbarsky@mit.edu> 2010-12-22 13:43:23 PST --- So what I see this page do, in horizontal mode, is create 17 canvases each of which is width="816" height="3587". That means that each of them has a backing store of 816*3587*4 = 11,707,968 bytes. So that's about 200MB of memory usage right there. I have no idea why they feel a need for 17 huge canvases, but if they want them, that's how much memory they'll take... ( https://bugzilla.mozilla.org/show_bug.cgi?id=620883 ) On 22 December 2010 01:24, Miquel Torres <tobami@googlemail.com> wrote:
Mmm, yes, it appears that the memory is not properly freed. I can try experimenting by destroying the dom element and recreating it each time that the plot changes...
Btw., instead of continuing here "polluting" the pypy-dev mailing list, we can move to http://groups.google.com/group/codespeed if needed.
Thanks for reporting this.
Cheers, Miquel
2010/12/22 Dima Tisnek <dimaqq@gmail.com>:
I sort of figured it out, although I don't have a ready solution.
This affects opera 11, stable ff, ff 4.0b7; amazingly it does not affect ie8. Basically memory consumption of the plot appears to be proportional to plot area. Normal plot that you get by default at http://speed.pypy.org/comparison/ costs about 100M of browser memory consumption: opera 130M, stable ff 70M, beta ff 90M at window size 1680x1050; opera 80M, stable ff 55M, beta ff 70M at window size 1024x600; Switching to "horizontal" produces a tall plot of same width and costs about 300~700M of browser memory: opera 720M, stable ff 370M, beta ff 370M at window wize 1680x1050; opera 350M, stable ff 370M, beta ff 370M at window size 1024x600;
Suprisingly window size only matters while javascript produces the plot, and not when window is resized, even though plot it resized with the window correctly.
This alone is pretty heavy, but doesn't grind the browser. What really grinds is that every time you change a tickbox on the left, a plot is redrawn and another 200M of browser memory is wasted. This is not double buffering, as next change adds yet another 200M or so and so on, it appears that either js doesn't free something, or browser caches or saves the previous page state.
As memory consumption grows, at some point browser hits the wall, causes heavy swapping for some time, and I think garbage collection, because practical (but not virtual) memory usage first drops to something like 20~50M and then returns to "normal" 300M. opera ~30 seconds, stable ff about a minute, beta ff several minutes (total system mem 1G, cpu Atom @1.6GHz)
Perhaps OS also plays a role in the grind, as it is clearly swapping and swaps out too much? or triggers gc too late and gc has to pull the pages back from disk to perform collection?
ie8 doesn't use that much memory, as a matter of fact memory consumption starts little (40M) and changes very little (only +10M) if you go to horizonatal view; the price is very slow rendering, more than 10 seconds per column change.
I'll post this on firefox bugzilla too, let's see if someone has a solution.
Meanwhile perhaps pypy speed center could start with a smaller plot area (or fewer columns as that makes horizontal plot smaller) to accomodate varying hardware and system mem usage that users might have? The simplest would be a warning next to "horizontal" checkbox.
On 21 December 2010 01:06, Miquel Torres <tobami@googlemail.com> wrote:
Hi Dima,
another temp problem with speed pypy is that it's terrubly slow in ff beta. it also occasionally grinds in stable ff and opera, but I guess this can be forgiven for the sake of simplicity / developer effort.
Well, speed.pypy is actually fast in all modern browsers. The problem you are referring to is probably caused by a bug in the javascript plotting library (jqPplot) that is triggered in the comparison view when there are some results with 0 values. It only appears for some plot types, but it is very annoying because it grinds the browser to a halt like you say. Is that what you meant?
We are looking into it, and will fix that library if necessary.
Cheers, Miquel
2010/12/21 Dima Tisnek <dimaqq@gmail.com>:
On 20 December 2010 19:21, William ML Leslie <william.leslie.ttg@gmail.com> wrote:
On 21 December 2010 11:59, Dima Tisnek <dimaqq@gmail.com> wrote:
More visibility for performance achievements would do good too.
Where are pypy's performance achievements *not* visible, but should be?
for example http://shootout.alioth.debian.org/ doesn't say which pypy version is used, what options, doesn't have performance figures for multithreaded/multicore
also benchmarks are kinda small, most of them are not docuemented, and I couldn't find any info if the same python code was used for cpython and pypy (both shootout and speed pypy) or interpreter-specific verions were used, that is each tweaked for best performance given the known tradeoffs for each implementation.further the most standard benchmarks, pystone, etc. completely ignore the fact that real world programs are large and only a few small paths are execured often.
another temp problem with speed pypy is that it's terrubly slow in ff beta. it also occasionally grinds in stable ff and opera, but I guess this can be forgiven for the sake of simplicity / developer effort.
if you google for 'python performance' you don't get a single link to pypy on the first page, as a matter of fact, codespeak is poorly indexed, it took me quite some time to get some of my questions answered with a search. also if you look up 'pypy gc' you get a page on codespeak, but to interpret what the data actually means is so far beyond me.
a good overview is found in the mainling list http://codespeak.net/pipermail/pypy-dev/2010q1/005757.html then again slowspitfire and spambayes bits are outdated by now.
the definitive good thing about pypy is that it's much easier to find out about its inner workings than that of cpython!
hopefully a bit more of end-user stuff get known. let's call it pypy public outreach (feature request)
Sidetracking... one day when pypy jit/gc/etc are all worked out, how hard would it be to use same techniques and most of backends for some unrelated language that doesn't have jit yet, e.g. php?
You know that pypy already has implementations of other languages, right - Scheme, Javascript, Prolog, IO and smallTalk? They aren't integrated with the translated pypy-c, but they show that it is not too difficult to write a runtime for any dynamic language you choose.
Oh I didn't know there were so many, and I mistakenly though that js was a target, not implmented langauge. In any case I read somewhere that js support was retired...
And how hard would it be to marry two dynamic languages, so that modules from one could be used in the other? Or that modules written in rpython could be used in several langs?
It's in the "interesting problems" bucket, and the effort required depends on the level of integration between languages you want. There are several projects already attempting to do decent integration between several languages, besides the approach used on the JVM, there are also GNU Guile, Racket, and Parrot, among others. It might be worth waiting to see how these different projects pan out, before spending a bunch of effort just to be an also-ran in the multi-language runtime market.
However, implementing more languages in rpython has the side-effect of propagating the l * o * p problem: it introduces more and more implementations that then have to be maintained, so good cross-language integration probably belongs /outside/ pypy itself, so existing runtimes can hook into it.
Makes perfect sense, after all any given other language hardly has the same data model as python.
But it would be an interesting experiment (to marry the various interpreters pypy ships with), if you wanted to try it.
My two cents.
-- William Leslie
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev