[Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

[Carl Shapiro]

On Tue, Sep 18, 2018 at 3:00 PM, Neil Schemenauer <nas-python at arctrix.com>
wrote:

> The users of Python are pretty diverse so it depends on who you ask.
> Some would like a giant executable that includes everything they
> need (so of like the Go approach).  Other people want an executable
> that has just importlib inside it and then mix-and-match different
> shared libs for their different purposes.  Some will not want work
> "old school" and load from separate .py or .pyc files.
>
> I see no reason why we can't support all these options.
>

Supporting those options is possible if a some of our simplifying
assumptions are revisited.  Here are a few

We know about all the objects being stored in the data segment.  That makes
it is easy to ensure that immutable objects are unique.  Knowing anything
less, that work would have to be done at load-time.

We do not have to worry about the relocation cost of the pointers we add to
the data segment.  We are compiled into an executable that typically gets
loaded at a fixed address.  This could become a performance concern if we
wrote our data into a shared library.

Because we are compiled into the runtime, we do not have versioning
issues.  There is no possibility of PyObject_HEAD or any PyObject subclass
being changed out from under us.  The existing marshal format abstracts
away from these details but our format is very sensitive to the host
environment.

All of these problems have technical solutions.  They should be evaluated
carefully to ensure that the added overhead does not wipe-out the
performance wins or add lots of complexity to the runtime.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180921/981c0445/attachment.html>