[Python-Dev] Python startup time

[Ryan Gonzalez]

<plug> https://refi64.com/uprocd/ </plug>


On May 11, 2018 9:39:28 AM Chris Barker - NOAA Federal via Python-Dev 
<python-dev at python.org> wrote:

> Inspired by chg:
>
> Could one make a little startup utility that, when invoked the first
> time, starts up a raw python interpreter, keeps it running somewhere,
> and then forks it to run the actual python code.
>
> Then every invocation after that would make a new fork. I presume
> forking is a LOT faster than re-invoking the entire startup.
>
> I suspect that many of the cases where startup time really matters is
> when a command line utility is likely to be invoked many times — often
> in the same shell instance.
>
> So having a “pre-built” warm interpreter ready to go could really help.
>
> This is way past my technical expertise to know if it’s possible, or
> to try to prototype it, but I’m sure many of you would know.
>
> -CHB
>
> Sent from my iPhone
>
>> On May 7, 2018, at 12:28 PM, Neil Schemenauer <nas-python at arctrix.com> wrote:
>>
>> On 2018-05-03, Lukasz Langa wrote:
>>>> On May 2, 2018, at 8:57 PM, INADA Naoki <songofacandy at gmail.com> wrote:
>>>> * Add lazy compiling API or flag in `re` module.  The pattern is compiled
>>>> when first used.
>>>
>>> How about go the other way and allow compiling at Python
>>> *compile*-time? That would actually make things faster instead of
>>> just moving the time spent around.
>>
>> Lisp has a special form 'eval-when'.  It can be used to cause
>> evaluation of the body expression at compile time.
>>
>> In Carl's "A fast startup patch" post, he talks about getting rid of
>> the unmarshal step and storing objects in the heap segment of the
>> executable.  Those would be the objects necessary to evaluate code.
>> The marshal module has a limited number of types that it handle.
>> I believe they are: bool, bytes, code objects, complex, Ellipsis
>> float, frozenset, int, None, tuple and str.
>>
>> If the same mechanism could handle more types, rather than storing
>> the code to be evaluated, we could store the objects created after
>> evaluation of the top-level module body.  Or, have a mechanism to
>> mark which code should be evaluated at compile time (much like the
>> eval-when form).
>>
>> For the re.compile example, the compiled regex could be what is
>> stored after compiling the Python module (i.e. the re.compile gets
>> run at compile time).  The objects created by re.compile (e.g.
>> SRE_Pattern) would have to be something that the heap dumper could
>> handle.
>>
>> Traditionally, Python has had the model "there is only runtime".
>> So, starting to do things at compile time complicates that model.
>>
>> Regards,
>>
>>  Neil
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com