ESR "Waning of Python" post

[Marko Rauhamaa]

dieter <dieter at handshake.de>:

> Every system you use has its advantages and its drawbacks.
> Depending on the specific context (problem, resources, knowledge, ...),
> you must choose an appropriate one.

Yep. I use Python for numerous tasks professionally and at home. Just
this past week I used it to plan a junior soccer winter tournament.
Python is used to verify various team and match constraints and
Sudoku-solver-type match order generation.

> Python uses the GIL mainly because it uses reference counting (with
> almost constant changes to potentially concurrently used objects) for
> memory management. Dropping the GIL would mean dropping reference
> counting likely in favour of garbage collection.

Reference counting was likely a bad idea to begin with.

> I work in the domain of web applications. And I made there a nasty
> experience with garbage collection: occasionally, the web application
> stopped to respond for about a minute. A (quite difficult) analysis
> revealed that some (stupid) component created in some situations (a
> search) hundreds of thousands of temporary objects and thereby
> triggered a complete garbage collection. The garbage collector started
> its mark and sweep phase to detect unreachable objects - traversing a
> graph of millions of objects.
>
> As garbage collection becomes drastically more complex if the object
> graph can change during this phase (and this was Python), a global
> look prevented any other activity -- leading to the observed
> latencies.

Yes. The occasional global freeze is unavoidable in any
garbage-collected runtime environment regardless of the programming
language.

However, I challenge the notion that creating hundreds of thousands of
temporary objects is stupid. I suspect that the root cause of the
lengthy pauses is that the program maintains millions of *nongarbage*
objects in RAM (a cache, maybe?).

> When I remember right, there are garbage collection schemes that
> can operate safely without stopping other concurrent work.

There are heuristics, but I believe the worst case is the same.

> Nevertheless, even those garbage collectors have a significant impact
> on performance when they become active (at apparently
> non-deterministic times) and that may be inacceptable for some
> applications.

If performance is key, Python is probably not the answer. Python's
dynamism make it necessarily much slower than, say, Java or Go.


Marko