On 17/03/14 20:04, Martin Koch wrote:
Well, it would appear that we have the problem because we're generating a lot of garbage in the young generation, just like we're doing in the example we've been studying here.
No, I think it's because your generating a lot of garbage in the *old* generation. Meaning objects which survive one minor collection but then die.
I'm unsure how we can avoid that in our real implementation. Can we force gc of the young generation? Either by gc.collect() or implcitly somehow (does the gc e.g. kick in across function calls?).
That would make matters worse, because increasing the frequency of minor collects means *more* objects get moved to the old generation (where they cause problems). So indeed, maybe in your case making the new generation bigger might help. This can be done using PYPY_GC_NURSERY, I think (nursery is the space reserved for young objects). The risk is that minor collections become unreasonably slow. Anyway, if the example code you gave us also shows the problem I think we should eventually look into it. It's not really fair to say "but you're allocating too much!" to explain why the GC takes a lot of time. Cheers, Carl Friedrich