Mailman 3 September 2011 - pypy-dev

Great results for MyHDL simulations
by Jan Decaluwe 01 Sep '11

01 Sep '11

I am seeing great improvements for MyHDL simulations by using PyPy, and I have written a page about it: http://www.myhdl.org/doku.php/performance I have asked for feedback from the MyHDL community to confirm these results, before creating some more buzz in the hardware design world :-) Thanks! Jan Decaluwe -- Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com Python as a HDL: http://www.myhdl.org VHDL development, the modern way: http://www.sigasi.com Analog design automation: http://www.mephisto-da.com World-class digital design: http://www.easics.com

3 4

Suggestions for small projects for getting started hacking on pypy?
by Christian Hudon 01 Sep '11

01 Sep '11

Hello, I'm interested in getting my feet wet hacking on pypy. I've successfully bootstrapped pypy from source, using the documentation on doc.pypy.org. What small projects are open right now that I could use to get started learning about the code? Thanks, Christian

4 4

Question about "Completing 'os' module" (issue 833)
by Mitchell Hashimoto 01 Sep '11

01 Sep '11

Hi all, I noticed the 'os' module is incomplete, and I'd like to help complete this. CPython does this by simply having these methods available on "posix" "nt" "os2" etc. and the "os" module imports those. It appears that PyPy does the same thing. I was able to successfully add 'getlogin' as practice, but I wanted to post here before going further. Some questions below: 1.) Should I mimic CPython and add the functionality to the OS-specific modules? 2.) I don't have a Windows computer on hand. What is the standard practice for implementing some stdlib for one OS but not the other? Would PyPy accept this temporarily? 3.) There are many missing methods, to simplify implementation time and the patch, would it be okay to submit a patch for each stdlib method, so that this was built up over time? I'd love to do this "right" the first time so I can contribute it back, so this feedback would be greatly appreciated. Best, Mitchell

3 5

djangobench performance
by Fenn Bailey 01 Sep '11

01 Sep '11

Hi all, As an experiment, I thought I'd test JKM's djangobench ( https://github.com/jacobian/djangobench) under pypy as a way of determining a (hopefully) more useful benchmark than the template-only "django" benchmark that's standard on speed.pypy.org and also to get an idea as to whether switching to pypy for production django apps could (currently) be a good idea. djangobench is designed to fairly comprehensively compare the performance of different aspects of differing versions of django in an effort to detect performance degradation/regression/etc. It's based on perf.py from the unladen swallow project, so it was fairly easy to crudely hack up to instead compare a single django version running under cpython 2.6 vs pypy 1.6. --- $ python -V Python 2.6.5 $ pypy -V Python 2.7.1 (d8ac7d23d3ec, Aug 17 2011, 11:51:19) [PyPy 1.6.0 with GCC 4.4.3] --- The results were a little surprising (and not in a good way): http://pastie.org/2463906 Based on the highly degraded performance (>2 orders of magnitude in some cases) I'm guessing there's some sort of issue in the way I'm benchmarking things. Code can be found here: https://github.com/fennb/djangobench Environment is ubuntu 10.04 64bit running in a VM on a macbook pro. cpython was the current ubuntu binary package, pypy was 1.6 precompiled binary from pypy.org. It's quite possible memory size issues may have impacted some of the benchmarks (but not all). Any ideas as to why the performance drop-off would be so significant? Cheers, Fenn.

3 3

Re: [pypy-dev] [Speed] Co-ordinating benchmarking
by Tennessee Leeuwenburg 01 Sep '11

01 Sep '11

Okay, so all of a sudden there seem to be a *lot* of people looking at this. This was a long thread quickly, and I only just got up to speed with it. There are a lot of new names, and I don't know what existing skills, interest and territories exist. Apologies for any faux pas. I would like to organise the list of tasks a bit more clearly if that is okay. I may be less familiar with the parts of this process than others, so I just want to get it down clearly. What I've done: -- Clone the Python repo into /home/speedracer/cpython/cpython (updated to 2.7) -- Installed os packages to support a reasonable build. -- Built and installed python2.7 into /home/speedraver/cpython/27_bin Presumably, people would like to be monitoring both PyPy and Cpython as they progress over time. This means some kind of auto runner which updates from the repo, re-runs the timings, and submits them to codespeed. I am unclear on whether there is a "clear winner" for this piece of technology. List of Required Infrastructure on speed.python.org: -- A home for the cpython repo (check) -- An installed and running codespeed server (pending) -- A buildbot / automation for keeping up to date with the PyPy and cpython repos (???) -- Automation to execute a process which must (???) 1) Run the benchmarks 2) Submit results to codespeed I would suggest that the codespeed server be installed into speedracer's home directory. This must all be installable and configurable from Chef (which appears to me like Fabric, i.e. an automation tool for deployment and management of systems). This is not yet accomplished. We also clearly need some kind of wiki documentation on what is going on, so that contributors (or just newbies like me) can figure out where things are at and what is going on. The bitbucket project is great, but the task titles are currently rather brief if someone isn't already totally up-to-speed on what is going on. There appear to me to be two unresolved questions: 1) What piece of technology should we use for a buildbot / build automation 2) What piece of technology should we use for the benchmark runner? I have no suggestions on (1), it's not a strong point for me. As regards (2), I am ignorant to what others might already be using, except to say the landscape seems unclear and fractured to me. My work, benchmarker.py, is likely to be adaptable to our needs and I am more than happy to support the package so it can be applied here. As I understand it, the GSOC project was about the actual benchmarking functions, not so much about automation and support for managing the results of benchmarking. If an already-working alternative to benchmarker.py exists and makes more sense to use, then that is fine by me. I would still like to help out to learn more about benchmarking. The main issue with me as a contributor will be time. I have a full plate as a result of Real Life, so I will sometimes go dark for a week or so. However, I'm motivated and interested, and can put in a few hours a week most weeks. Do I have this right? Is that a reasonable description of the work breakdown? Do we have clear names against tasks so that co-ordination can be done through those people (rather than via the whole list)? Regards, -Tennessee On Thu, Sep 1, 2011 at 6:28 AM, Noah Kantrowitz <noah(a)coderanger.net> wrote: > Its all branches all the way down, so we can start work anywhere and push it to an "official" PSF bin later I think. I'm sure we will want to host a mirror of it on the python.org hg server too, just for discoverability. > > --Noah > > On Aug 31, 2011, at 1:12 PM, Miquel Torres wrote: > >> Oh, cool, so there will be an Opscode hosted account for the PSF, >> right? Then the Chef repo should be for the PSF. Maybe in a current >> account somewhere? What do you propose? >> >> Miquel >> >> >> 2011/8/31 Noah Kantrowitz <noah(a)coderanger.net>: >>> Opscode has already agreed to donate a Hosted account as long we keep it under ~20 clients :-) I can hand out the info for it to anyone that wants. As for setting up the Chef repo, just remember we are trying to not manage this system in isolation and that it will be part of a bigger PSF infrastructure management effort. >>> >>> --Noah >>> >>> On Aug 31, 2011, at 11:34 AM, Miquel Torres wrote: >>> >>>> Hi all, >>>> >>>> though I took up on the task of installing a Codespeed instance >>>> myself, I didn't have time until now. This weekend I will definitely >>>> have a *lot* of time to work on this, so count on that task being >>>> done by then. >>>> >>>> The bitbucket issue tracker is a start (though a organization account >>>> would be better) and the splash page is great. So let's get started >>>> organizing things. >>>> >>>> Regarding the deployment strategy, it turns out I use Chef at work, so >>>> I am in full agreement with Noah here (yey!). Actually, I am the >>>> author of LittleChef (which we can use as a tool to execute Chef on >>>> the node). >>>> >>>> So, Configuration Management. I would propose that Noah starts the >>>> repo with the Chef cookbooks (preferably a complete LittleChef >>>> kitchen, but that is not a must :), and gets the main recipes (apache, >>>> django) going, while I create a cookbook for Codespeed. What do you >>>> think? >>>> >>>> The benchmark runner question is still open. We need to clarify that. >>>> Use the pypy runner? Tennessee's work? >>>> >>>> Regarding repositories and issues, we could maybe have a "speed" >>>> organization account (not sure on Bitbucket, you can do that in >>>> Github), where we have a wiki, issues, and runner + config management >>>> repo + other stuff. >>>> >>>> Cheers, >>>> Miquel >>>> >>>> 2011/8/31 Jesse Noller <jnoller(a)gmail.com>: >>>>> I've put up a splash page for the project this AM: >>>>> >>>>> http://speed.python.org/ >>>>> >>>>> jesse >>>>> _______________________________________________ >>>>> pypy-dev mailing list >>>>> pypy-dev(a)python.org >>>>> http://mail.python.org/mailman/listinfo/pypy-dev >>>>> >>>> _______________________________________________ >>>> Speed mailing list >>>> Speed(a)python.org >>>> http://mail.python.org/mailman/listinfo/speed >>> >>> > > > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed > > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

1 0