ESR "Waning of Python" post

[dieter]

Marko Rauhamaa <marko at pacujo.net> writes:
> dieter <dieter at handshake.de>:
>> Marko Rauhamaa <marko at pacujo.net> writes:
>>> Keeping the number of long-term objects low is key.
>>
>> Right, if I need near realtime behaviour and must live
>> with [C]Python's garbage collector.
>
> Or any other GC ever invented.

There are "realtime garbage collection" algorithms. For them,
the creation of new objects must cooperate with the garbage
collection - likely with the need to acquire a lock for a short
period. But there is no (principle) need to block all
"normal" activity during the complete garbage collection (as [C]Python's
garbage collector has done this at the time of my problem).

>> But, a web application does usually not need near realtime behaviour.
>> An occasional (maybe once in a few days) garbage collection and
>> associated reduced response time is acceptable.
>> A problem only arises if a badly designed component produces
>> quite frequently hundreds of thousands of temporary objects
>> likely triggering (frequent) garbage collections.
>
> But I think you are barking up the wrong tree. You could rightly blame
> GC itself as an unworkable paradigm and switch to, say, C++ or Rust.

I am happy that [C]Python uses mainly reference counting for
its memory management and that GC is used quite sparingly.

> Or you could blame the parts of the software that create too many
> long-term objects.

I do not do that because I understand why in my application
there are many long living objects.

> You shouldn't blame the parts of the software that churn out zillions of
> short-term objects.

I do precisely that: the blamed component produced a very large
number of short living objects -- without need and while it should
have been aware that it operates on mass data - among others from the
fact that its complete environment took special care to work with
this mass data efficiently.

I solved my problem by replacing this single component by
one knowing what it does. No need to rewrite the complete
application, get rid of Python object caches or even
switch to a different language.