[Python-Dev] __file__

[Robert Collins]

On Sun, 2010-02-28 at 18:11 -0800, Brett Cannon wrote:
...

>         So then looking for a .pyc alongside a .py or vice versa
>         should be almost free, and we shouldn't be worrying about
>         it.

> But that is making the assumption that all filesystems operate this
> way (.e.g does NFS have the same performance characteristics?).

NFS doesn't cache pages, rather it caches individual entries. I do not
know if adjacent data is pre-populated into an NFS client cache. I
rather suspect not, but the general point is true - many filesystems
cache by pages, not all. And some have unsorted lists or might have hash
tables rather than b-trees or sorted lists, where locality of reference
doesn't help at all. (VFAT is unsorted list, IIRC).

>                 hot cache:
>                 % time     seconds  usecs/call     calls    errors
>                 syscall
>                 ------ ----------- ----------- --------- ---------
>                 ----------------
>                  45.10    0.000368          92         4
>                 getdents
>                 
>                  0.00    0.000000           0       734       625 stat
>                 
>         
>         Further supporting the idea that stat calls are negligible
>         once the cache is warmed up.
> 
> 
> But that's the point: once it's warmed up. This is not the case when
> executing a script once every once in a while compared to something
> bzr where you are most likely going to execute the command multiple
> times within a small timeframe.

bzr would /love/ cold cache times to come down. They are one of the most
glaring performance differences between a large C program, and a large
python program. Even though the second run is fast, it hurts the first
time you run 'bzr st'. (**)

-Rob

(**) To the extent that I've seriously considered an import hook to
disable normal imports under bzrlib, and special case how the search
works so that we only load pyc's and assume all imports are absolute.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100301/140c661b/attachment.pgp>