cleaning up an ASCII file?

Nick Matzke matzke at
Wed Jun 10 16:09:59 EDT 2009

Hi all,

So I'm parsing an XML file returned from a database.  However, the 
database entries have occasional non-ASCII characters, and this is 
crashing my parsers.

Is there some handy function out there that will schlep through a file 
like this, and do something like fix the characters that it can 
recognize, and delete those that it can't?  Basically, like the BBEdit 
"convert to ASCII" menu option under "Text".

I googled some on this, but nothing obvious came up that wasn't specific 
to fixing one or a few characters.


Nicholas J. Matzke
Ph.D. Candidate, Graduate Student Researcher
Huelsenbeck Lab
Center for Theoretical Evolutionary Genomics
4151 VLSB (Valley Life Sciences Building)
Department of Integrative Biology
University of California, Berkeley

Lab websites:
Dept. personal page:
Lab personal page:
Lab phone: 510-643-6299
Dept. fax: 510-643-6264
Cell phone: 510-301-0179
Email: matzke at

Mailing address:
Department of Integrative Biology
3060 VLSB #3140
Berkeley, CA 94720-3140

"[W]hen people thought the earth was flat, they were wrong. When people 
thought the earth was spherical, they were wrong. But if you think that 
thinking the earth is spherical is just as wrong as thinking the earth 
is flat, then your view is wronger than both of them put together."

Isaac Asimov (1989). "The Relativity of Wrong." The Skeptical Inquirer, 
14(1), 35-44. Fall 1989.

More information about the Python-list mailing list