[Python-ideas] Speed up os.walk() 5x to 9x by using file attributes from FindFirst/NextFile() and readdir()

[Nick Coghlan]

On Mon, Nov 12, 2012 at 7:17 PM, Ben Hoyt <benhoyt at gmail.com> wrote:

> Thoughts? What's the next step? If I come up with a patch against
> posixmodule.c, tests, etc, is this likely to be accepted? I could
> also flesh out my pure-Python proof of concept [1] to do what I'm
> suggesting above and go from there...
>

The issue with patching the stdlib directly rather than releasing something
on PyPI is that you likely won't get any design or usability feedback until
the first 3.4 alpha, unless it happens to catch the interest of someone
willing to tinker with a patched version earlier than that.

It only takes one or two savvy users to get solid feedback by publishing
something on PyPI (that's all I got for contextlb2, and the design of
contextlib.ExitStack in 3.3 benefited greatly from the process). Just the
discipline of writing docs, tests and giving people a rationale for
downloading your module can help a great deal with making the case for the
subsequent stdlib change.

The reason I suggested walkdir as a possible venue is that I think your
idea here may help with some of walkdir's *other* API design problems (of
which there are quite a few, which is why I stopped pushing for it as a
stdlib addition in its current state - it has too many drawbacks to be
consistently superior to rolling your own custom solution)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121112/ac78d3ee/attachment.html>