Is text processing with dicts a good use case for Python cross-compilers like Cython/Pyrex or ShedSkin?

python at bdurham.com python at bdurham.com
Thu Dec 16 15:03:13 EST 2010


Is text processing with dicts a good use case for Python
cross-compilers like Cython/Pyrex or ShedSkin? (I've read the
cross compiler claims about massive increases in pure numeric
performance).

I have 3 use cases I'm considering for Python-to-C++
cross-compilers for generating 32-bit Python extension modules
for Python 2.7 for Windows.

1. Parsing UTF-8 files (basic Python with lots of string
processing and dict lookups)

2. Generating UTF-8 files from nested list/dict structures

3. Parsing large ASCII "CSV-like" files and using dict's to
calculate simple statistics like running totals, min, max, etc.

Are any of these text processing scenarios good use cases for
tools like Cython, Pyrex, or ShedSkin? Are any of these
specifically bad use cases for these tools?

We've tried Psyco and it has sped up some of our parsing
utilities by 200%. But Psyco doesn't support Python 2.7 yet and
we're committed to using Python 2.7 moving forward.

Malcolm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20101216/02dde20b/attachment.html>


More information about the Python-list mailing list