Mailman 3 March 2010 - pypy-dev

Nightly builds are unavailable
by Gasper Zejn 25 Mar '10

25 Mar '10

The links at the URL http://buildbot.pypy.org/nightly/ are failing with "resource unavailable"; misconfiguration? Just to make sure it's a known problem. KR, Gasper

3 4

dvcs conversation of pypy and maybe all the other projects in the repo, too
by Ronny Pfannschmidt 24 Mar '10

24 Mar '10

Hi, since pypy's history itself is kinda unsuitable for practically all converters/importers out there, i started to experiment with analysis of it in order to eventually convert to a dag based dvcs (most likely hg). Since that repository carries various other projects as well i wonder if anyone else is interested in converting to a dvcs. The needs of pypy's history alone create the need of a very powerfull tool anyway, so converting the other projects would be a nice sideeffect. -- Ronny

1 0

autonosiness in roundup
by Maciej Fijalkowski 23 Mar '10

23 Mar '10

Anyone knows how to set up autonosy on roundup? I only get information about new tickets and then I don't get followups.

7 9

Two bugs in PyPy 1.2
by Fredrik Johansson 16 Mar '10

16 Mar '10

Dear PyPy developers, I have found two bugs while trying to run mpmath on pypy-1.2. This is with the Windows binary: Python 2.5.2 (72130, Mar 12 2010, 12:25:40) [PyPy 1.2.0] on win32 My system is an HP 530, Intel Celeron M 520 @ 1.60 GHz, 1024 MB RAM running Windows XP. The first bug is an incompatibility with CPython. In CPython, builtin float functions accept arbitrary objects as long as they have a __float__ method. Consider the following code: class x(object): def __float__(self): return 1.0 class y(object): def __complex__(self): return 1.0j round(x()) import math; math.log(x()) import cmath; cmath.log(x()) import cmath; cmath.log(y()) CPython 2.6 gives: 1.0 0.0 0j 1.5707963267948966j CPython 2.5 gives: 1.0 0.0 0j TypeError: a float is required PyPy gives: TypeError: expected float, got x object TypeError: expected float, got x object 0j 1.5707963267948966j Presumably PyPy should do the same thing as 2.6 here. Secondly, there is an error somewhere in the long multiplication code: C:\pypy12>pypy Python 2.5.2 (72130, Mar 12 2010, 12:25:40) [PyPy 1.2.0] on win32 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``no random bugs in random, after all'' >>>> import mpmath >>>> mpmath.runtests() mpmath imported from C:\pypy12\lib-python\2.5.2\mpmath mpmath backend: python mpmath mp class: <mpmath.ctx_mp.MPContext object at 0x0000001a> mpmath version: 0.15-svn Python version: 2.5.2 (72130, Mar 12 2010, 12:25:40) [PyPy 1.2.0] [...] test_functions2 agm ok 0.0446171 s airy ok 0.1957078 s appellf1 ok 1.1814778 s bessel ok 0.6132425 s coulomb ok 0.4926749 s e1 ok 0.0109151 s ei ok 0.0346402 s elliptic_integrals ok 0.2575911 s erf ok 0.3858091 s exp_integrals ok 0.0715859 s RPython traceback: File "implement_4.c", line 35008, in opcode_impl_for_mul__AccessDirect_star_2 File "implement_9.c", line 47885, in __mm_mul_W_LongObject_W_LongObject File "implement_13.c", line 3727, in _k_lopsided_mul File "implement_9.c", line 38027, in _k_mul Fatal RPython error: AssertionError The test that triggers the failure is "from mpmath import expint; expint('1.01', '1e-1000')". For your convenience, I have tracked down the precise numbers. Since they have several hundred digits, I put them in an attachment. Thank you for your hard work! Fredrik

3 5

someone is wrong on the internet
by Maciej Fijalkowski 16 Mar '10

16 Mar '10

This is particularly good hate mail I found: http://www.linuxtoday.com/news_story.php3?ltsn=2010-03-15-013-35-NW-RL-0000 Enjoy :)

2 1

bzr import of pypy from svn
by Michael Hudson-Doyle 15 Mar '10

15 Mar '10

As I've mentioned a couple of times on #pypy, during the codespeak outage I imported pypy trunk from Subversion into bzr (from a local tarball of the repo I had). Since codespeak came back up again I've updated it, and Alexander asked for an import of all the branches rather than just trunk. This is now done and I've just finishing pushing them up to Launchpad: https://code.edge.launchpad.net/~mwhudson/pypy/ I can keep this up to date by hand, but automating it is a bit tricky given that uploading the end result requires my ssh key. I'll try to think of something. Launchpad can import subversion branches automatically of course, but this is currently broken by a subversion bug (http://subversion.tigris.org/issues/show_bug.cgi?id=3480). A workaround would be setting up read-only svnserve access or maaaybe upgrading svn. Cheers, mwh

1 0

speed.pypy.org quick update
by Miquel Torres 14 Mar '10

14 Mar '10

Hi! I wanted to explain a couple of things about the speed website: - New feature: the Timeline view now defaults to a plot grid, showing all benchmarks at the same time. It was a feature request made more than once, so depending on personal tastes, you can bookmark either /overview/ or /timeline/. Thanks go to nsf for helping with the implementation. - The code has now moved to github as Codespeed, a benchmark visualization framework (http://github.com/tobami/codespeed) - I have updated speed.pypy.org with version 0.3. Much of the work has been under the hood to make it feasible for other projects to use codespeed as a framework. For those interested in further development you can go to the releases wiki (still a work in progress): http://wiki.github.com/tobami/codespeed/releases Next in the line are some DB changes to be able to save standard deviation data and the like. Long term goals besides world domination are integration with buildbot and similarly unrealistic things. Feedback is always welcome. Cheers, Miquel

7 11

issue tracker
by Maciej Fijalkowski 14 Mar '10

14 Mar '10

Hello. I'm considering changing issue tracker as apparently a lot of people gave feedback after the release. For me the current incarnation is unbearable - I can't easily track things like "submitted" or "triaged" from "windows build requires superuser" which always pop at the top or "polish objspace initialization" which is probably important but I already decided not to deal with it. It might be the inherent problem with roundup or just our implementation. pylib for example moved to bitbucket, probably also for similar reasons. Do you have any thoughts or obvious choices? Cheers, fijal

4 3

memory recording... Re: speed.pypy.org quick update
by René Dudfield 14 Mar '10

14 Mar '10

Hi again, trying to do some research on ways to record memory usage in an X-platform way... keeping my notes here: http://renesd.blogspot.com/2010/03/memory-usage-of-processes-from-python.ht… So far people have come up with these two useful projects so far: http://code.google.com/p/psutil/ http://code.google.com/p/pympler/ I think psutil will have most info needed to construct a decent memory recording module for benchmarks. However, it includes C code, so will probably have to rip some of the memory parts out, and maybe reimplement with ctypes. cu, On Fri, Mar 12, 2010 at 10:49 AM, René Dudfield <renesd(a)gmail.com> wrote: > btw, for python memory usage on linux > /proc/PID/status > > Here is some code for linux... > > wget http://rene.f0o.com/~rene/stuff/memory_usage.py<http://rene.f0o.com/%7Erene/stuff/memory_usage.py> > > >>> import memory_usage > >>> bytes_of_resident_memory = memory_usage.resident() > > > Should be easy enough to add that to benchmarks at the start and end? > Maybe calling it in the middle would be a little harder... but not too hard. > > > TODO: Would need to be updated for other platforms, and support measuring > child processes, tests, and code cleanup :) > > cu, > > > > > On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall(a)gmail.com>wrote: > >> Hey. >> >> I'll answer questions that are relevant to benchmarks themselves and >> not running. >> >> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr(a)oz.net> wrote: >> > On 03/10/2010 12:14 PM Miquel Torres wrote: >> >> Hi! >> >> >> >> I wanted to explain a couple of things about the speed website: >> >> >> >> - New feature: the Timeline view now defaults to a plot grid, showing >> >> all benchmarks at the same time. It was a feature request made more >> >> than once, so depending on personal tastes, you can bookmark either >> >> /overview/ or /timeline/. Thanks go to nsf for helping with the >> >> implementation. >> >> - The code has now moved to github as Codespeed, a benchmark >> >> visualization framework (http://github.com/tobami/codespeed) >> >> - I have updated speed.pypy.org with version 0.3. Much of the work has >> >> been under the hood to make it feasible for other projects to use >> >> codespeed as a framework. >> >> >> >> For those interested in further development you can go to the releases >> >> wiki (still a work in progress): >> >> http://wiki.github.com/tobami/codespeed/releases >> >> >> >> Next in the line are some DB changes to be able to save standard >> >> deviation data and the like. Long term goals besides world domination >> >> are integration with buildbot and similarly unrealistic things. >> >> Feedback is always welcome. >> > >> > Nice looking stuff. But a couple comments: >> > >> > 1. IMO standard deviation is too often worse than useless, since it >> hides >> > the true nature of the distribution. I think the assumption of >> normalcy >> > is highly suspect for benchmark timings, and pruning may hide >> interesting clusters. >> > >> > I prefer to look at scattergrams, where things like clustering and >> correlations >> > are easily apparent to the eye, as well as the amount of data >> (assuming a good >> > mapping of density to visuals). >> >> That's true. In general a benchmark run over time is a period of >> warmup, when JIT compiles assembler followed by thing that can be >> described by average and std devation. Personally I would like to have >> those 3 measures separated, but didn't implement that yet (it has also >> some interesting statistical questions involved). Std deviation is >> useful to get whether a difference was meaningful of certain checkin >> or just noise. >> >> > >> > 2. IMO benchmark timings are like travel times, comparing different >> vehicles. >> > (pypy with jit being a vehicle capable of dynamic self-modification >> ;-) >> > E.g., which part of travel from Stockholm to Paris would you >> concentrate >> > on improving to improve the overall result? How about travel from >> Brussels to Paris? >> > Or Paris to Sydney? ;-P Different things come into play in different >> benchmarks/trips. >> > A Porsche Turbo and a 2CV will both have to wait for a ferry, if >> that's part of the trip. >> > >> > IOW, it would be nice to see total time broken down somehow, to see >> what's really >> > happening. >> >> I can't agree more with that. We already do split time when we perform >> benchmarks by hand, but they're not yet integrated into the whole >> nightly run. Total time is what users see though, that's why our >> public site is focused on that. I want more information available, but >> we have only limited amount of manpower and miquel already did quite >> amazing job in my opinion :-) We'll probably go into more details. >> >> The part we want to focus on past-release is speeding up certain parts >> of tracing as well as limiting it's GC pressure. As you can see, the >> split would be very useful for our development. >> >> > >> > Don't get me wrong, the total times are certainly useful indicators >> of progress >> > (which has been amazing). >> > >> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the >> trip distance to get time. >> > A 25% improvement in total time is not a 25% improvement in speed. >> I.e., (if you define >> > improvement as a percentage change in a desired direction), for e.g. >> 25%: >> > distance/(0.75*time) != 1.25*(distance/time). >> > >> > IMO 'speed' (the implication to me in the name speed.pypy.org) would >> be benchmarks/time >> > more appropriately than time/benchmark. >> > >> > Both measures are useful, but time percentages are easy to >> mis{use,construe} ;-) >> >> That's correct. >> >> Benchmarks are in general very easy to lie about and they're by >> definition flawed. That's why I always include raw data when I publish >> stuff on the blog, so people can work on it themselves. >> >> > >> > 4. Is there any memory footprint data? >> > >> >> No. Memory measurment is hard and it's even less useful without >> breaking down. Those particular benchmarks are not very good basis for >> memory measurment - in case of pypy you would mostly observe the >> default allocated memory (which is roughly 10M for the interpreter + >> 16M for semispace GC + cache for nursery). >> >> Also our GC is of a kind that it can run faster if you give it more >> memory (not that we use this feature, but it's possible). >> >> Cheers, >> fijal >> _______________________________________________ >> pypy-dev(a)codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev >> > >

2 1

slow benchmark?
by Waldemar Kornewald 13 Mar '10

13 Mar '10

Hi, maybe I'm doing something totally wrong, but the attached trivial benchmark runs several times slower than on CPython. I've tested it with the Windows binary from your website. Your other benchmarks run fast, BTW. Is there something wrong with my code? Bye, Waldemar Kornewald -- http://www.allbuttonspressed.com

2 1