[Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability
ethan at stoneleaf.us
Tue Jul 18 12:08:08 EDT 2017
> One minor grumble: I think we need to give careful cost/benefit considerations to
> optimizations that complicate the implementation. Over the last several years, the
> source for Python has grown increasingly complicated. Fewer people understand it
> now. It is much harder to newcomers to on-ramp. The old-timers (myself included)
> find that their knowledge is out of date. And complexity leads to bugs (the C
> optimization of random number seeding caused a major bug in the 3.6.0 release; the
> C optimization of the lru_cache resulted in multiple releases having a hard to find
> threading bugs, etc.). It is becoming increasingly difficult to look at code and
> tell whether it is correct (I still don't fully understand the implications of the
> recursive constant folding in the peephole optimizer for example). In the case
> of this named tuple proposal, the complexity is manageable, but the overall trend
> isn't good and I get the feeling the aggressive optimization is causing us to
> forget key parts of the zen-of-python.
> As another example of this: while trading the global import lock for
> per-module locks eliminated most of the old import deadlocks, it turns
> out that it *also* left us with some fairly messy race conditions and
> more fragile code (I still count that particular case as a win
> overall, but it definitely raises the barrier to entry for maintaining
> that code).
> Unfortunately, these are frequently cases where the benefits are
> immediately visible (e.g. faster benchmark results, removing
> longstanding limitations on user code), but the downsides can
> literally take years to make themselves felt (e.g. higher defect rates
> in the interpreter, subtle bugs in previously correct user code that
> are eventually traced back to interpreter changes).
> Regardless of whether [namedtuple] optimization is a good idea or not, start up
> time *is* a serious challenge in many environments for CPython in particular and
> the perception of Python’s applicability to many problems. I think we’re better
> off trying to identify and address such problems than ignoring or minimizing them.
Speed is not the only factor, and certainly shouldn't be the first concern, but once
we have correct code we need to follow our own advice: find the bottlenecks and optimize
them. Optimized code will never be as pretty or maintainable as simple, unoptimized
code but real-world applications often require as much performance as can be obtained.
[My apologies if I missed any points from the namedtuple thread.]
More information about the Python-Dev