[Python-Dev] Impact of Namedtuple on startup time

[Raymond Hettinger]

> On Jul 17, 2017, at 8:49 AM, Guido van Rossum <guido at python.org> wrote:
> 
> One of the reasons to be wary of exec()/eval() other than the usual security concerns is that in some Python implementations they have a high overhead to initialize the parser and compiler. (Even in CPython it's not that fast.)

BTW, if getting rid of the template/exec pair is a goal, Joe Jevnik proposed a patch a couple of years ago the completely reimplemented namedtuple() in C.   The patch was somewhat complex and hard to semantic equivalence, but we could resurrect it and clean it up.   That way, we could like the existing namedtuple() code in-place and do a subsequent import from the C-version.

This path won't be fun (whenever we have both a C version and Python version, we get years of trying to sync-up tiny differences); however, it will give you take fastest startup times, the fastest lookups at runtime, and eliminate use of exec.

> On Jul 17, 2017, at 8:13 AM, Barry Warsaw <barry at python.org> wrote:


> Regardless of whether this particular optimization is a good idea or not, start up time *is* a serious challenge in many environments for CPython in particular and the perception of Python’s applicability to many problems.  I think we’re better off trying to identify and address such problems than ignoring or minimizing them.

I agree with that sentiment but think we ought to look at places where the payoffs would actually matter such a minimizing the number of disk accesses (Python performs a lot of I/O on startup).  Whenever I've addressed start-up time for my clients, named tuples we never the issue.  Also, it would have been trivially easy to replace the factory function call with the generated code, but that never proved necessary or beneficial.   IMO, we're about to turn the named tuple code into a mess but will find that most users, most of the time will get nearly zero benefit. 


Raymond