Mailman 3 September 2009 - pypy-dev

Advice for a talk...
by Michael Hudson-Doyle 05 Nov '09

05 Nov '09

I'm talking about PyPy at KiwiPyCon in a few weeks: http://nz.pycon.org/talks/talk/3/ My basic plan is to give a similar talk to the one I gave at OSDC last year (http://codespeak.net/svn/pypy/extradoc/talk/osdc2008/osdc08.pdf, which in turn is pretty similar to talks I gave back in 2006...). I've not been following recent progress super closely, so if someone who's been more involved could read the OSDC talk and point out things that are no longer true or not really relevant any more, that would be great :) I guess I should say a bit about the current approach to the JIT... Cheers, mwh

4 4

Mac osx patches
by Leonardo Santagada 15 Oct '09

15 Oct '09

To make compilation and asmgcroot happy on osx I made some changes, I would like to know if someone has anything against applying this stuff to trunk? Index: pypy/translator/c/gcc/trackgcroot.py =================================================================== --- pypy/translator/c/gcc/trackgcroot.py (revision 68036) +++ pypy/translator/c/gcc/trackgcroot.py (working copy) @@ -22,7 +22,11 @@ r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS) +").*$") r_functionstart_darwin = re.compile(r"_(\w+):\s*$") -OFFSET_LABELS = 2**30 +# darwin's ld complain about this hack +if sys.platform == 'darwin': + OFFSET_LABELS = 0 +else: + OFFSET_LABELS = 2**30 # inside functions LABEL = r'([.]?[\w$@]+)' Index: pypy/translator/platform/test/test_darwin.py =================================================================== --- pypy/translator/platform/test/test_darwin.py (revision 68036) +++ pypy/translator/platform/test/test_darwin.py (working copy) @@ -2,8 +2,8 @@ """ File containing darwin platform tests """ -import py, os -if os.name != 'darwin': +import py, sys +if sys.platform != 'darwin': py.test.skip("Darwin only") from pypy.tool.udir import udir Index: pypy/translator/platform/darwin.py =================================================================== --- pypy/translator/platform/darwin.py (revision 68036) +++ pypy/translator/platform/darwin.py (working copy) @@ -18,7 +18,7 @@ self.cc = cc def _args_for_shared(self, args): - return (self.shared_only + ['-bundle', '-undefined', 'dynamic_lookup'] + return (self.shared_only + ['-dynamiclib', '-undefined', 'dynamic_lookup'] + args) def include_dirs_for_libffi(self): -- Leonardo Santagada santagada at gmail.com

4 7

benchmarking input
by Anders Hammarquist 02 Oct '09

02 Oct '09

Hi all, I need some input for the benchmarking infrastructure. I'm nearly at the point where I need to have some place to run it before continuing (i.e. I need to try and use it, not just speculate). Anyway, what I was thinking about, and need input on, is how to get at the interpreters to run the benchmark. When we were talking just benchmarks, and not profiling, my thought was to just use whatever python the machine has, and fetch the pypy from the last buildbot run, but for profiling that will not work (and anyway, running the profiling on the standard python is quite pointless). So benchmarks will obviously have to specify what interpreter(s) they should be run by somehow. The bigger question is how to get those interpreters. Should running the benchmarks also trigger building one (or more) pypy interpreters according to specs in the benchmarking framework? (but then if you only want it to run one benchmark, you may have to wait for all the interpreters to build) Perhaps each benchmark should build its own interpreter (though this seems slow, given that most benchmarks can probably run on an identically built interpreter). Or maybe the installed infrastructure should only care about history, and if you want to run a single benchmark, you do that on your own. Thoughts please! /Anders

8 7

support for 64-bit processors and eliminating global state
by Jeff Anderson-Lee 01 Oct '09

01 Oct '09

I'm new to pypy but would encourage the development folks to apply some focus towards two things: support for both 32 and 64-bit processors and eliminating global state including the GIL. The near future of mainstream processors is multi-core x86_64. For the short-term both 32-bit and 64-bit platforms will be around. Code that makes "naked" assumptions about word size will break and needs to be re-factored to hide the word-size dependencies. Similarly code that assumes a single thread of execution or uses a GIL to protect global state will make efficient use of modern processors. Any language or system that cannot make the transition to 64-bit multi-core will start to loose ground to those that do. At the Parallel Computing Laboratory (UC Berkeley) one of the projects we are working on is called SEJITS which stands for Selective Embedded Just in Time Specialization. The idea is that one can extend a self-introspecting modern scripting language for calling native coded modules (e.g. C) at selected points for handling specialized operations (e.g. vector/matrix operations using tuned SIMD or CUDA code). You can see the abstract of a recent SEJITS paper at http://pmea.ac.upc.edu/program.html (session 1a) but unfortunately the paper is not online yet. Both Python and Ruby are being looked at as potential target languages for SEJITS work. Both have sufficient introspection facilities to support selective JIT operations. Python has an advantage in having been used by the scientific community for longer than Ruby with more established users. I'd love to see this work integrate with pypy. At the moment the folks involved are targeting CPython. In any case, I think the transition to multi-core/multi-threaded 64-bit machines is a potential watershed of major importance which it would behoove pypy-dev folks to keep in mind. Respectfully. Jeff Anderson-Lee

5 9

Re: [pypy-dev] Stackless Python and PyPy Stackless.py
by Gabriel Lavoie 30 Sep '09

30 Sep '09

2009/9/28 Leonardo Santagada <santagada(a)gmail.com>: > I am very interested, I thought to do something like it. Where could I see > your code? > > On Sep 28, 2009, at 10:04 AM, Gabriel Lavoie wrote: > >> Hello Andrew, >> I'm currently experimenting with PyPy's implementation of >> Stackless to add new features for a university master degree project. >> I chose PyPy's implementation because it's easier to play with Python >> code than with C code. Also, since PyPy is "still experimental", it >> was the best implementation to choose to hack with and I don't regret >> my choice. What I'm trying to achieve is to add distributed features >> to Stackless: >> >> - Local and networked channels with automatic switch between both >> - Easy tasklet migration to a remote host, keeping the channel >> connections between tasklets. >> - Transparent/automatic dependencies migration when a tasklet is sent >> to a remote host. >> >> Most of the features are done and I'm currently working on the >> dependencies migration. The only bad part is that I'm doing this >> project part time since I have a full time job but I have to complete >> the programming part in the next two months (I've been working for too >> long on this). >> >> If you're interested to see what I've done, just ask! :) >> >> See ya, >> >> Gabriel >> >> 2009/9/25 Andrew Francis <andrewfr_ice(a)yahoo.com>: >>> >>> Hi Folks: >>> >>> Again as a part of my Stackless Python talk, I wanted to include a >>> section on the "Future." I assume a part of Stackless Python's future is >>> PyPy? Or am I being presumptuous? >>> >>> Regardless I would like to end the talk with a brief section on PyPy. I >>> noticed the Stackless.py module in lib that contains the Stackless >>> implementation in Python. >>> >>> What I plan to do in my talk is show how a rough approximation of Limbo's >>> alt (selecting the first ready channel from a list) could be implemented. >>> >>> I am a newbie in regards to PyPy. However I have been reading the >>> Stackless documentation. I thought it would be neat if I ended the talk with >>> redoing this, but in PyPy as a part of how one could quickly prototype new >>> Stackless Python features. Any thoughts? Is there anything gotchas? >>> >>> Cheers, >>> Andrew >>> >>> >>> >>> _______________________________________________ >>> pypy-dev(a)codespeak.net >>> http://codespeak.net/mailman/listinfo/pypy-dev >>> >> >> >> >> -- >> Gabriel Lavoie >> glavoie(a)gmail.com >> _______________________________________________ >> pypy-dev(a)codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev > > -- > Leonardo Santagada > santagada at gmail.com > > > > I still haven't shown publicly my work. I'll try to quickly prepare something this week with my current test code samples and a quick description of the API. I think the code quality is pretty bad as this is my first real Python project. My priority is to have something that works before doing a big cleanup. Gabriel -- Gabriel Lavoie glavoie(a)gmail.com

2 3

Re: [pypy-dev] benchmarking input
by Anders Hammarquist 30 Sep '09

30 Sep '09

Since everyone but me seemed to want to take the discussion on IRC instead, I propose we meet up on #pypy-sync on monday at 14:00 /Anders

7 16

Question on Hard Switching and Soft Switching
by Andrew Francis 30 Sep '09

30 Sep '09

Hi Folks: As a part of an up-coming talk, I have been reading more about that happens under the hood. For instance "The Essentials of Stackless Python." The terms 'hard switching' and 'soft switching' constantly appear. 'Soft' switching is defined as cooperating switching. I will assume this is associated with a 'stackless.schedule()' or a blocked on channel. Hard switching is defined as brute-force. However what is brute-force? Is this when Stackless is in pre-emptive mode and one is relying on the ticks? Cheers, Andrew

2 1

Nightly Benchmarks
by Miquel Torres 29 Sep '09

29 Sep '09

Hi iko, we just had a #pypy discussion about setting up a benchmarks web page and I was told that you already have started some work in that direction, so it would be great to coordinate each other because I intend to implement the visualization part. (to all others that attended the chat, feel free to correct me on any point I do not correctly adress) (and to all others that did NOT attend, feel free to make suggestions :) We agreed on a Django DB backend, and plot rendering by the browser, so we need to talk about the way to store the nightly benchmark data into the DB. The end goal is to build a speed.pypy.org where you can compare all kind of pypy versions to cpython and other python interpreters and other languages in a similar vein to http://shootout.alioth.debian.org/u32q/benchmark.php?test=all&lang=java&lan… not a copy of). As a fist step we should adress the main interest right now: pypy performance evolution through trunk revisions. Can you please explain to me what you are doing in that area now? Best regards, Miquel

1 0

Stackless Python and PyPy Stackless.py
by Andrew Francis 28 Sep '09

28 Sep '09

Hi Folks: Again as a part of my Stackless Python talk, I wanted to include a section on the "Future." I assume a part of Stackless Python's future is PyPy? Or am I being presumptuous? Regardless I would like to end the talk with a brief section on PyPy. I noticed the Stackless.py module in lib that contains the Stackless implementation in Python. What I plan to do in my talk is show how a rough approximation of Limbo's alt (selecting the first ready channel from a list) could be implemented. I am a newbie in regards to PyPy. However I have been reading the Stackless documentation. I thought it would be neat if I ended the talk with redoing this, but in PyPy as a part of how one could quickly prototype new Stackless Python features. Any thoughts? Is there anything gotchas? Cheers, Andrew

3 3

has anybody submitted an abstract to PyCON USA?
by Laura Creighton 28 Sep '09

28 Sep '09

Time is running out.... Laura

2 1