The other night I got this burning desire to recreate the Azul GPGC (the java pauseless collector) inside pypy, with a view that it could be used alongside the STM work to make pypy a low (no?) pause concurrent VM. When I started tackling the code I realised I might have bitten off a little more than I can chew. My question (probably one of many to irritate and annoy all the fine folks here) would be, is there a sensible way to compile into pypy a small amount of C code that can be used to bootstrap and bridge some esoteric c libraries into pypy, the code that I want to run, on startup of pypy would be the following https://bitbucket.org/GregBowyer/pypy-c4gc/changeset/0de575b3a8d1#chg-azm_me... I have noticed that there are a few bridges into direct c code in the pypy source, but I cannot fathom how these are linked into RPython -- Greg
2012/2/21 Greg Bowyer <gbowyer@fastmail.co.uk>
My question (probably one of many to irritate and annoy all the fine folks here) would be, is there a sensible way to compile into pypy a small amount of C code that can be used to bootstrap and bridge some esoteric c libraries into pypy, the code that I want to run, on startup of pypy would be the following https://bitbucket.org/**GregBowyer/pypy-c4gc/** changeset/0de575b3a8d1#chg-**azm_mem_test/test.c<https://bitbucket.org/GregBowyer/pypy-c4gc/changeset/0de575b3a8d1#chg-azm_me...>
It's not annoying at all, we use it in strategic places. For example, see how pypy/rlib/_rffi_stacklet.py implements a stacklet C library that can be used in RPython. It uses an "ExternalCompilationInfo" (eci) object: - separate_module_files lists the .c files you want to compile and link - separate_module_sources is an easy way to embed C snippets (each source will create a .c file) Then you can use rffi.llexternal with "compilation_info=eci" to declare a function defined in this library. -- Amaury Forgeot d'Arc
Thats pretty awesome. So if anyone else is willing to join in a challange, I have an example first steps piece of C that uses the azul interfaces to attempt to grab a blob of memory. https://bitbucket.org/GregBowyer/pypy-c4gc/raw/1889f31b43e5/azm_mem_test/tes... I was expecting my code to not work for the printf, however it does not actually seem to do the mreserve Anyone want to join my insanity ? -- Greg On 20/02/12 16:24, Amaury Forgeot d'Arc wrote:
2012/2/21 Greg Bowyer <gbowyer@fastmail.co.uk <mailto:gbowyer@fastmail.co.uk>>
My question (probably one of many to irritate and annoy all the fine folks here) would be, is there a sensible way to compile into pypy a small amount of C code that can be used to bootstrap and bridge some esoteric c libraries into pypy, the code that I want to run, on startup of pypy would be the following https://bitbucket.org/GregBowyer/pypy-c4gc/changeset/0de575b3a8d1#chg-azm_me...
It's not annoying at all, we use it in strategic places. For example, see how pypy/rlib/_rffi_stacklet.py implements a stacklet C library that can be used in RPython.
It uses an "ExternalCompilationInfo" (eci) object: - separate_module_files lists the .c files you want to compile and link - separate_module_sources is an easy way to embed C snippets (each source will create a .c file)
Then you can use rffi.llexternal with "compilation_info=eci" to declare a function defined in this library.
-- Amaury Forgeot d'Arc
Hi Greg, On Fri, Feb 24, 2012 at 22:23, Greg Bowyer <gbowyer@fastmail.co.uk> wrote:
So if anyone else is willing to join in a challange, I have an example first steps piece of C that uses the azul interfaces to attempt to grab a blob of memory.
Unsure how well using C code for the GC would work. Also, if you're thinking about STM specifically, note that it needs to be integrated with a special GC --- or at least, not strictly *needs* to, but if it is, we can design a much more efficient overall way to have STM. (This is documented in https://bitbucket.org/pypy/extradoc/raw/extradoc/planning/stm.txt , together with a plan for a simple STM-specific GC, which is partially implemented.) A bientôt, Armin.
Re-hi, Ah, another matter. I don't know how interesting the goal of a no-pause GC is together with STM. It would seem to me that STM is, by itself, not really well-suited if you want to absolutely avoid pauses: if a transaction is aborted a few times before eventually succeeding to commit, then you have an unexpected pause between the time were the transaction was first started and the time were it committed. I think it's a built-in property of all transactional memory systems to not be really "real-time safe". Maybe they can be made real-time safe by falling back to nontransactional single-threaded execution at some point, but that looks more advanced that I care for right now. A bientôt, Armin.
It has taken me a long time to get back to this, the algorithm I am playing with is soft real-time at best not hard realtime My thinking is far more around using (and abusing) the transaction queue to manage a barrier for the GC implementing much of a classical read/write barrier there On 25/02/12 09:42, Armin Rigo wrote:
Re-hi,
Ah, another matter. I don't know how interesting the goal of a no-pause GC is together with STM. It would seem to me that STM is, by itself, not really well-suited if you want to absolutely avoid pauses: if a transaction is aborted a few times before eventually succeeding to commit, then you have an unexpected pause between the time were the transaction was first started and the time were it committed. I think it's a built-in property of all transactional memory systems to not be really "real-time safe". Maybe they can be made real-time safe by falling back to nontransactional single-threaded execution at some point, but that looks more advanced that I care for right now.
A bientôt,
Armin.
participants (3)
-
Amaury Forgeot d'Arc -
Armin Rigo -
Greg Bowyer