[Python-ideas] BetterWalk, a better and faster os.walk() for Python
Mike Meyer
mwm at mired.org
Thu Nov 22 17:33:52 CET 2012
On Thu, Nov 22, 2012 at 5:39 AM, Ben Hoyt <benhoyt at gmail.com> wrote:
> In the recent thread I started called "Speed up os.walk()..." [1] I
> was encouraged to create a module to flesh out the idea, so I present
> you with BetterWalk:
>
> https://github.com/benhoyt/betterwalk#readme
>
> It's basically all there, and works on Windows, Linux, and Mac OS X.
> It probably works on FreeBSD too, but I haven't tested that.
It doesn't work on FreeBSD 9.1. Here's a quick failure in the source
directory that might help
you diagnose the problem (I probably won't be able to look into it
deeper until after the holiday weekend):
>>>from betterwalk import walk
>>>for x in walk('.'):
... print x
...
('.', [u'', u'', u'erwalk.py', u'erwalk.pyc', u'p.py'], [u'ignore',
u'hmark.py', u's', u'erwalk.py', u'htree', u'attributes', u'GES.txt',
u'ME.md', u'NSE.txt']) (u'./', [u'', u'', u'erwalk.py', u'erwalk.pyc',
u'p.py'], [u'ignore', u'hmark.py', u's', u'erwalk.py', u'htree',
u'attributes', u'GES.txt', u'ME.md', u'NSE.txt'])
<break>
I also got an error trying to run the benchmark program on 3.2 (this
is python-ideas, which means things discussed here are bound for
Python 3):
bhuda% python3.2 benchmark.py
Creating tree at benchtree: depth=4, num_dirs=5, num_files=50
Traceback (most recent call last):
File "benchmark.py", line 121, in <module>
main()
File "benchmark.py", line 116, in main
create_tree(tree_dir)
File "benchmark.py", line 26, in create_tree
f.write(line * 20000)
TypeError: 'str' does not support the buffer interface
You need to use a bytestring instead of a string here. The best way
will depend on how old a version of python you want to support.
<mike
More information about the Python-ideas
mailing list