[Tutor] parse text file

[Vincent Davis]

On Thu, Jun 3, 2010 at 1:02 PM, Colin Talbert <talbertc at usgs.gov> wrote:

>
> Dave,
>         I think you are probably right about using decompressor.  I
> couldn't find any example of it in use and wasn't having any luck getting it
> to work based on the documentation.  Maybe I should try harder on this
> front.
>

Is it possible write a python script to transfer this to a hdf5 file?  Would
this help?
Thanks
Vincent


> Colin Talbert
> GIS Specialist
> US Geological Survey - Fort Collins Science Center
> 2150 Centre Ave. Bldg. C
> Fort Collins, CO 80526
>
> (970) 226-9425
> talbertc at usgs.gov
>
>
>
>  From: Dave Angel <davea at ieee.org> To:
> Colin Talbert <talbertc at usgs.gov>
> Cc: Steven D'Aprano <steve at pearwood.info>, tutor at python.org Date: 06/03/2010
> 12:36 PM Subject: Re: [Tutor] parse text file
> ------------------------------
>
>
>
> Colin Talbert wrote:
> > <snip>
> > You are so correct.  I'd been trying numerous things to read in this file
>
> > and had deleted the code that I meant to put here and so wrote this from
> > memory incorrectly.  The code that I wrote should have been:
> >
> > import bz2
> > input_file = bz2.BZ2File(r'C:\temp\planet-latest.osm.bz2','rb')
> > str=input_file.read()
> > len(str)
> >
> > Which indeed does return only 900000.
> >
> > Which is also the number returned when you sum the length of all the
> lines
> > returned in a for line in file with:
> >
> >
> > import bz2
> > input_file = bz2.BZ2File(r'C:\temp\planet-latest.osm.bz2','rb')
> > lengthz = 0
> > for uline in input_file:
> >     lengthz = lengthz + len(uline)
> >
> > print lengthz
> >
> > <snip>
> >
> >
> Seems to me for such a large file you'd have to use
> bz2.BZ2Decompressor.  I have no experience with it, but its purpose is
> for sequential decompression -- decompression where not all the data is
> simultaneously available in memory.
>
> DaveA
>
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>
  *Vincent Davis
720-301-3003 *
vincent at vincentdavis.net
 my blog <http://vincentdavis.net> |
LinkedIn<http://www.linkedin.com/in/vincentdavis>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20100603/c402f133/attachment-0001.html>