reading large file

[Sophie Alléon]

Thanks to your comments, it is now possible to read my large file in a
couple of minutes
on my machine.

Guillaume
"Bengt Richter" <bokr at oz.net> a écrit dans le message de news:
bj5e61$pjr$0 at 216.39.172.122...
> On 3 Sep 2003 05:00:39 -0700, g_alleon at yahoo.fr (guillaume) wrote:
>
> >I have to read and process a large ASCII file containing a mesh : a
> >list of points and triangles.
> >The file is 100 MBytes.
> >
> >I first tried to do it in memory but I think I am running out of
> >memory therefore I decide to use the shelve
> >module to store my points and elements on disks.
> >Despite the fact it is slow ... Any hint ? I think I have the same
> >memory problem but I don't understand why
> >since  my aPoint should be removed by the gc.
> >
> >Have you any idea ?
> >
> Since your data is very homogeneous, why don't you store it in a couple of
> homogeneous arrays? You could easily create a class to give you convenient
> access via indices or iterators etc. Also you could write load and store
> methods that could write both arrays in binary to a file. You could
> consider doing this as a separate conversion from your source file, and
> then run your app using the binary files and wrapper class.
>
> Arrays are described in the array module docs ;-)
> I imagine you'd want to use the 'd' type for ponts and 'l' for faces.
>
> Regards,
> Bengt Richter