Python 3.0 slow file IO

rdmurray at bitdance.com rdmurray at bitdance.com
Thu Feb 5 07:39:09 EST 2009


Quoth Christian Heimes <lists at cheimes.de>:
> thomasvangurp at gmail.com schrieb:
> > I just recently learned python, I'm using it mainly to process huge
> > <5GB txt files of ASCII information about DNA. I've decided to learn
> > 3.0, but maybe I need to step back to 2.6?
> > 
> > I'm getting exceedingly frustrated by the slow file IO behaviour of
> > python 3.0. I know that a bug-report was submitted here:
> > http://bugs.python.org/issue4533. And a solution was posted.
[snip stuff about patching]
> 
> I suggest you stick with Python 2.5 or 2.6. Python 3.0 isn't as mature
> as the 2.x series. It's brand new, lot's of things have changed, too.

3.1 will come out much sooner than a normal "next dot" release python
would, and will contain very significant improvements in the IO speed.
In the meantime, in your case at least, to get real work done you'll
be better off using 2.6.  You can use the 'from __future__ import' to
bring various 3.0 features into 2.6 and make 2.6 look more like 3.0,
and you can use the -3 flag to check your code for 3.0 compatibility, so
that you will be ready to move back to the 3 series when 3.1 comes out.
(Assuming you don't end up needing any 3rd party libraries that haven't
been ported yet by then, that is.)

Unless, of course, you are comfortable with compiling from source
and would like to help _test_ the 3.1 IO library.... :)

--RDM




More information about the Python-list mailing list