[Tutor] parse text file

[bob gailer]

Please always reply-all so a copy goes to the list.

On 6/1/2010 6:49 PM, Colin Talbert wrote:
>
> Bob thanks for your response,
>         The file is about 9.3 gig and no I don't want read the whole 
> thing at once.  I want to read it in line by line.  Still it will read 
> in to the same point (900000 characters) and then act as if it came to 
> the end of the file.  Below is the code I using for this:
>
>
> import bz2
>
> input_file = bz2.BZ2File(r"C:\temp\planet-latest.osm.bz2","rb")
> for uline in input_file:
>                     print linecount
>                     linecount+=1
>
>
>
>
>
>
>
>
> Colin Talbert
> GIS Specialist
> US Geological Survey - Fort Collins Science Center
> 2150 Centre Ave. Bldg. C
> Fort Collins, CO 80526
>
> (970) 226-9425
> talbertc at usgs.gov
>
>
>
> From: 	bob gailer <bgailer at gmail.com>
> To: 	Colin Talbert <talbertc at usgs.gov>
> Cc: 	tutor at python.org
> Date: 	06/01/2010 04:43 PM
> Subject: 	Re: [Tutor] parse text file
>
>
> ------------------------------------------------------------------------
>
>
>
> On 6/1/2010 5:40 PM, Colin Talbert wrote:
>
>        I am also experiencing this same problem.  (Also on a OSM bz2 
> file).  It appears to be working but then partway through reading a 
> file it simple ends.  I did track down that file length is always 
> 900000 so it appears to be related to some sort of buffer constraint.
>
>
> Any other ideas?
>
> How big is the file?
>
> Is it necessary to read the entire thing at once?
>
> Try opening with mode rb
>
>
> import bz2
>
> input_file = bz2.BZ2File(r"C:\temp\planet-latest.osm.bz2","r")
> try:
>    all_data = input_file.read()
>    print str(len(all_data))
> finally:
>    input_file.close()

-- 
Bob Gailer
919-636-4239
Chapel Hill NC

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20100602/18f6bd3f/attachment.html>