stmgc: rethinking the GC?
Hi Remi, Just thinking about stmgc: currently it requires large transactions in order to work efficiently. I think it is mostly caused by the fact that transaction commits always force a minor collection. Maybe we should rethink that at some point, by changing the GC completely... There are moving GCs that are really concurrent but they tend to be an advanced mess. Maybe a better alternative would be to experiment again with non-moving GCs. Non-moving generational GCs are not very common as far as I know, but they do exist; for example see http://wiki.luajit.org/New-Garbage-Collector . A bientôt, Armin.
Hi Armin, I think we need to quantify how bad the minor GCs really are. They are thread-local and they should also take less time for shorter transactions. The last benchmark I looked at had a different problem that I would summarize as (1) time spent creating backup copies, (2) huge commit log entry size, (3) time spent pulling the changes in, (4) major collections. All of these need to be amortized with longer transactions, too. I mention this because these problems come from writing to *old* objects and at least (3) and (4) are non-thread-local issues that prevent scaling. If we go for a different GC type, we need to make sure that we still have ways to reduce the cost of these problems (currently with overflow objs and ignoring young objs). I don't know if minor GCs are really that high on the list of things that cause overhead. It may just be that short transactions produce more old objs relative to their runtime, and these old objs cause more work in following transactions. So I currently see two important directions to look into: (1) quantify the cost of minor GCs and estimate the advantage of a non-moving GC, and (2) think about ways to make old objs cheaper (e.g. by doing less work for thread-local objs). But I'll first have to read up on that LuaJIT GC over the holidays :) Cheers, Remi On 22 December 2015 at 08:30, Armin Rigo <arigo@tunes.org> wrote:
Hi Remi,
Just thinking about stmgc: currently it requires large transactions in order to work efficiently. I think it is mostly caused by the fact that transaction commits always force a minor collection. Maybe we should rethink that at some point, by changing the GC completely... There are moving GCs that are really concurrent but they tend to be an advanced mess. Maybe a better alternative would be to experiment again with non-moving GCs. Non-moving generational GCs are not very common as far as I know, but they do exist; for example see http://wiki.luajit.org/New-Garbage-Collector .
A bientôt,
Armin.
Armin: the luajit GC does not exist either, it's just a plan as far as I know On Tue, Dec 22, 2015 at 9:30 AM, Armin Rigo <arigo@tunes.org> wrote:
Hi Remi,
Just thinking about stmgc: currently it requires large transactions in order to work efficiently. I think it is mostly caused by the fact that transaction commits always force a minor collection. Maybe we should rethink that at some point, by changing the GC completely... There are moving GCs that are really concurrent but they tend to be an advanced mess. Maybe a better alternative would be to experiment again with non-moving GCs. Non-moving generational GCs are not very common as far as I know, but they do exist; for example see http://wiki.luajit.org/New-Garbage-Collector .
A bientôt,
Armin. _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
participants (3)
-
Armin Rigo
-
Maciej Fijalkowski
-
Remi Meier