[Python-Dev] PEP 471 Final: os.scandir() merged into Python 3.5

[Ben Hoyt]

Thanks for committing this, Victor! And fixing the d_type issue on funky
platforms.

Others: if you want to benchmark this, the simplest way is to use my
os.walk() benchmark.py test program here: https://github.com/benhoyt/scandir
-- it compares the built-in os.walk() implemented with os.listdir() with a
version of walk() implemented with os.scandir(). I see huge gains on
Windows (12-50x) and modest gains on my Linux VM (3-5x).

Note that the actual CPython version of os.walk() doesn't yet use
os.scandir(). I intend to open a separate issue for that shortly (or Victor
can). But that part should be fairly straight-forward, as I already have a
version available in my GitHub project.

-Ben


On Sat, Mar 7, 2015 at 9:13 PM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> Hi,
>
> FYI I commited the implementation of os.scandir() written by Ben Hoyt.
> I hope that it will be part of Python 3.5 alpha 2 (Ben just sent the
> final patch today).
>
> Please test this new feature. You may benchmark here.
> http://bugs.python.org/issue22524 contains some benchmark tools and
> benchmark results of older versions of the patch.
>
> The implementation was tested on Windows and Linux. I'm now watching
> for buildbots to see how other platforms like os.scandir().
>
> Bad news: OpenIndiana doesn't support d_type: the dirent structure has
> no d_type field. I already fixed the implementation to support this
> case. os.scandir() is still useful on OpenIndiana, because the stat
> result is cached in a DirEntry, so only one syscall is required,
> instead of multiple, when multiple DirEntry methods are called (ex:
> entry.is_dir() and not entry.is_symlink()).
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/benhoyt%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150307/a9a4387d/attachment-0001.html>