reading large file

[Bengt Richter]

On 3 Sep 2003 05:00:39 -0700, g_alleon at yahoo.fr (guillaume) wrote:

>I have to read and process a large ASCII file containing a mesh : a
>list of points and triangles.
>The file is 100 MBytes.
> 
>I first tried to do it in memory but I think I am running out of
>memory therefore I decide to use the shelve
>module to store my points and elements on disks.
>Despite the fact it is slow ... Any hint ? I think I have the same
>memory problem but I don't understand why
>since  my aPoint should be removed by the gc.
>
>Have you any idea ?
>
Since your data is very homogeneous, why don't you store it in a couple of
homogeneous arrays? You could easily create a class to give you convenient
access via indices or iterators etc. Also you could write load and store
methods that could write both arrays in binary to a file. You could
consider doing this as a separate conversion from your source file, and
then run your app using the binary files and wrapper class.

Arrays are described in the array module docs ;-)
I imagine you'd want to use the 'd' type for ponts and 'l' for faces.

Regards,
Bengt Richter