[pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless support)

Leonardo Santagada santagada at gmail.com
Sat Nov 13 19:38:30 CET 2010


On Sat, Nov 13, 2010 at 3:40 PM, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> --- On Sat, 11/13/10, Leonardo Santagada <santagada at gmail.com> wrote:
>> I still can't see how the greenlet for cpython could
>> possibly work on pypy. If you are talking about
>> http://codespeak.net/pypy/dist/pypy/doc/stackless.html#greenlets
>> then I think the "only" problem is that it doesn't work with the
>> jit right?
>
> As I have stated in earlier posts, I am a newbie. However I want to help with Maciej Fijalkowski's suggestions about integrating Stackless with the JIT. I figured a way to start is to see what it would take to get greenlet style hard switching done. Since greenlet is a C extension, let's be naive, compile it and pypy-c, run it in a debugger and see exactly how it fails. Besides reading the documentation inside the greenlet.c source, I can take a similar approach with Standard C - that is run it in a debugger and see what happens. This way I get a feel for the issues and get familiar with the tools.
>
> Maybe trying to work off the greenlet source will prove to be too difficult.

I will repeat what armin said, but I think you should reread what he
said in the previous thread, the gist is it will probably not work and
give you strange crashes and there is probably no way to fix them.

I think what fijal meant is to use the ideas in greenlets.c and map
them to pypy, but not directly trying to run the extension on top of
pypy.

Here is a list of what is crazy about the idea:
- greenlets is a c module, but the stuff it does with the stack is
completely outside the C API which is what pypy supports from cpython.
- It will (probably) break the gc. It will not find the roots (address
on the stack to memory on the heap) from functions that are taken out
of the stack
- The layout of the stack is probably not the same also and that will
also break.

> Right now, I have a computer that will compile pypy-c. However it takes a long time. I just want to make sure that everything is in place to carry out my plan. Also it would be nice if folks can spot problems in the approach. I know I need to learn more about stuff like shadow stacks. Also I have started to read documentation on the bytecode interpreter.

My idea to make things faster for you:

- Forget about running greenlets inside pypy-c after running and
making sure that armin prediction is true (that it will just blowup),
don't try to fix it. I also think that greenlets are more or less
independent from the bytecode interpreter, then don't waste your time
with that either.
- To translate faster and using less memory turn your python to 32bit
and use --no-allworkingmodules.
- Look at how greenlets.c work and talk to fijal and the other on irc
after/during you understand how it unwinds the stack. Write tests for
it (or reuse the ones that probably exist for stackless and
greenlets).
- Make a solid plan on how to do it.
- Have fun

:)

ps: I hope I was clear (and also correct)
-- 
Leonardo Santagada



More information about the Pypy-dev mailing list