[Python-ideas] PEP: Distributing a Subset of the Standard Library

[Nick Coghlan]

On 29 November 2016 at 03:28, Guido van Rossum <guido at python.org> wrote:
> On Mon, Nov 28, 2016 at 9:14 AM, Nathaniel Smith <njs at pobox.com> wrote:
>>
>> Also note that in Guido's option 2, we only incur the extra fstat calls if
>> the import would otherwise fail. In option 1, there are extra fstat calls
>> (and thus disk seeks etc.) adding some overhead to every import.
>
> Oh, that's an important consideration! Yes, adding .missing.py to the list
> of extensions would cause extra stat() calls and potentially slow down every
> import.

This is the second time I've seen "but stat calls!" concern in
relation to import today, so I'll echo what Brett pointed out in his
reply: the import system in recent 3.x releases, along with the
importlib2 backport to Python 2.7, builds a cache of the directory
contents for path entries rather than making multiple stat calls.

The current 3.x source code for that is at
https://hg.python.org/cpython/file/tip/Lib/importlib/_bootstrap_external.py#l1280

The significant reduction in the number of stat calls through better
caching is the main way the 3.3 import reimplementation in Python
managed to be competitive performance-wise with the previous C
implementation, and faster when importing from a network filesystem :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia