Fwd: Work plan for PyPy
waaaah - forget to cc everyone :-( Begin forwarded message:
From: Richard Emslie <rxe@ukshells.co.uk> Date: June 27, 2007 10:56:39 AM PDT To: Armin Rigo <arigo@tunes.org> Subject: Re: [pypy-dev] Work plan for PyPy
Hi Armin!
On Jun 27, 2007, at 4:40 AM, Armin Rigo wrote:
Hi Richard,
On Mon, Jun 25, 2007 at 01:01:35PM -0700, Richard Emslie wrote:
Just a sidenote - rffi supports (and uses) macros. Not sure how this will look like in a ctypes-based solution.
if you are talking c macros - this could be very problematic for the llvm backend.
Anton (which I add to the CC's of this mail) proposed a solution for calling C APIs from llvm bytecode in a portable way, which would work for functions whose argument types are not exactly specified by the standards as well as for some kind of macros. The idea is to generate stub "helper" functions in C and compile them with llvm-gcc. The llvm bytecode that genllvm produces would then call the helper functions instead of calling directly the external C API. Shouldn't be a performance problem as llvm will inline the helpers agressively.
ok, that is somewhat what we have now - except the stub functions are high level and shared *somewhat* with genc.
Generating c code in llvm backend would sort of defeat the point of it IMHO. We already have a backend that generates c. On the other hand, if we hand write the c code - it may be an neverending task of adding hacks to the backend. For instance just getting at errno is particularly problematic right now - and would involve a function call from a constant value. which means injecting a few lines assembly and a temporary variable. Easy to do, but it is nice right now that we pretty much have a direct mapping from our our lli form to llvm assembly.
Of course, it points again to the question of whether it's really worthwhile to have an llvm backend in PyPy or if just producing C and using llvm-gcc on everything wouldn't work just as well. On the other hand, it's interesting to note that there are other reasons for which exactly the same kind of helper functions would be useful; for example, to wrap a C library full of macros and unspecified types and constants, ctypes alone is not enough.
Like you say, llvm-gcc (well at least v4) would probably produce equivalent results with genc. The way we are using llvm right now (as a static compiler) i'm not sure of its worthiness either of keeping the backend alive. Other than the potential of accurate gc, potential of jit (which is sort of orthogonal to this), potential of producing vectorized code from rpython...
Cheers, Richard
Hi Richard, On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote:
bytecode that genllvm produces would then call the helper functions instead of calling directly the external C API. Shouldn't be a performance problem as llvm will inline the helpers agressively.
ok, that is somewhat what we have now - except the stub functions are high level and shared *somewhat* with genc.
Precisely, by contrast I'm thinking about having lower level stub functions that are generated automatically from the lltype declarations. For example, an external ll function ptr would generate a simple stub, always the same. GenC and GenLLVM would share the code to produce the stubs. Similarly, getting at an external variable (or variable-like macro) like errno should be expressable in an lltype way (it's not right now) that gets turned into """int _rpyget_errno(void) { return errno; }""".
Generating c code in llvm backend would sort of defeat the point of it IMHO.
The difference is that the stubs would always be generated from simple templates, not from genc's complex logic that turns flow graphs into C code. A bientot, Armin.
Hi Armin, On Jun 28, 2007, at 4:15 AM, Armin Rigo wrote:
Hi Richard,
On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote:
bytecode that genllvm produces would then call the helper functions instead of calling directly the external C API. Shouldn't be a performance problem as llvm will inline the helpers agressively.
ok, that is somewhat what we have now - except the stub functions are high level and shared *somewhat* with genc.
Precisely, by contrast I'm thinking about having lower level stub functions that are generated automatically from the lltype declarations. For example, an external ll function ptr would generate a simple stub, always the same. GenC and GenLLVM would share the code to produce the stubs. Similarly, getting at an external variable (or variable-like macro) like errno should be expressable in an lltype way (it's not right now) that gets turned into """int _rpyget_errno(void) { return errno; }""".
Ok - cool. I was hoping to propose something along these lines - but was unsure how to go about it. This would be ideal and work great for backends. I am guessing all memory management with this would be explicitly controlled from outside the stubs?
Generating c code in llvm backend would sort of defeat the point of it IMHO.
The difference is that the stubs would always be generated from simple templates, not from genc's complex logic that turns flow graphs into C code.
Yeah - this sounds fantastic. No clue how to do it - but great! :-) Well - I'm in the process of migrating back home... again. Once i get back though I can devote lots of time to this. I doubt i can make the sprint, but will try. Cheers, Richard
Armin Rigo wrote:
Hi Richard,
On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote:
bytecode that genllvm produces would then call the helper functions instead of calling directly the external C API. Shouldn't be a performance problem as llvm will inline the helpers agressively.
ok, that is somewhat what we have now - except the stub functions are high level and shared *somewhat* with genc.
Precisely, by contrast I'm thinking about having lower level stub functions that are generated automatically from the lltype declarations. For example, an external ll function ptr would generate a simple stub, always the same. GenC and GenLLVM would share the code to produce the stubs. Similarly, getting at an external variable (or variable-like macro) like errno should be expressable in an lltype way (it's not right now) that gets turned into """int _rpyget_errno(void) { return errno; }""".
Just a little correction - you can express c-level constant in lltype. There is CConstant class for that. Cheers, fijal
Hi Maciek, On Sat, Jun 30, 2007 at 01:12:57PM +0200, Maciek Fijalkowski wrote:
Just a little correction - you can express c-level constant in lltype. There is CConstant class for that.
True, although I'm not sure it works for non-integer constants (e.g. static data). Also, I suppose that it could be abused to represent "errno" but it looks fragile. A bientot, Armin.
Armin Rigo wrote:
Hi Maciek,
On Sat, Jun 30, 2007 at 01:12:57PM +0200, Maciek Fijalkowski wrote:
Just a little correction - you can express c-level constant in lltype. There is CConstant class for that.
True, although I'm not sure it works for non-integer constants (e.g. static data). Also, I suppose that it could be abused to represent "errno" but it looks fragile.
? The primary reason for creating this class was to use it for errno, so I don't think it's an abuse. Anyway, I think it should be extended to present some non-primitive types (it works for any pre-defined primitive type right now). Cheers, fijal
participants (3)
-
Armin Rigo -
Maciek Fijalkowski -
Richard Emslie