[Python-Dev] Python startup time

[Chris Barker - NOAA Federal]

Inspired by chg:

Could one make a little startup utility that, when invoked the first
time, starts up a raw python interpreter, keeps it running somewhere,
and then forks it to run the actual python code.

Then every invocation after that would make a new fork. I presume
forking is a LOT faster than re-invoking the entire startup.

I suspect that many of the cases where startup time really matters is
when a command line utility is likely to be invoked many times — often
in the same shell instance.

So having a “pre-built” warm interpreter ready to go could really help.

This is way past my technical expertise to know if it’s possible, or
to try to prototype it, but I’m sure many of you would know.

-CHB

Sent from my iPhone

> On May 7, 2018, at 12:28 PM, Neil Schemenauer <nas-python at arctrix.com> wrote:
>
> On 2018-05-03, Lukasz Langa wrote:
>>> On May 2, 2018, at 8:57 PM, INADA Naoki <songofacandy at gmail.com> wrote:
>>> * Add lazy compiling API or flag in `re` module.  The pattern is compiled
>>> when first used.
>>
>> How about go the other way and allow compiling at Python
>> *compile*-time? That would actually make things faster instead of
>> just moving the time spent around.
>
> Lisp has a special form 'eval-when'.  It can be used to cause
> evaluation of the body expression at compile time.
>
> In Carl's "A fast startup patch" post, he talks about getting rid of
> the unmarshal step and storing objects in the heap segment of the
> executable.  Those would be the objects necessary to evaluate code.
> The marshal module has a limited number of types that it handle.
> I believe they are: bool, bytes, code objects, complex, Ellipsis
> float, frozenset, int, None, tuple and str.
>
> If the same mechanism could handle more types, rather than storing
> the code to be evaluated, we could store the objects created after
> evaluation of the top-level module body.  Or, have a mechanism to
> mark which code should be evaluated at compile time (much like the
> eval-when form).
>
> For the re.compile example, the compiled regex could be what is
> stored after compiling the Python module (i.e. the re.compile gets
> run at compile time).  The objects created by re.compile (e.g.
> SRE_Pattern) would have to be something that the heap dumper could
> handle.
>
> Traditionally, Python has had the model "there is only runtime".
> So, starting to do things at compile time complicates that model.
>
> Regards,
>
>  Neil
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov