[AstroPy] Astropy and large VOTable files

[Michael Droettboom]

Thanks for the question.

Unfortunately, it will read the entire file into memory each time. It 
does read it in as a Numpy array, so the memory used should generally be 
less than the space on disk, however, depending on the content.

XML doesn't really support the kind of slicing that FITS (or another 
binary format) can, because you can't know how big something is (or even 
what it is!) without parsing the whole file.  That said, given the 
constraint of the file format, minimal memory usage is one of the main 
design features of astropy.io.votable, so I'd recommend trying it on 
large files and seeing how it goes.  It shouldn't ever take 
significantly more memory than a binary array of data, i.e. the same as 
the equivalent FITS file loaded entirely into memory.

Cheers,
Mike

On 05/17/2015 10:11 AM, Jennifer Baldwin wrote:
> Hi all,
>
> I was trying to find an answer to this but could not. I am wondering 
> if parse_single_table will attempt to read an entire VOTable file? Or 
> if it will operate the same way as for FITS files so that when you 
> slice the returned data array, it only loads the part it needs into 
> memory? I'm concerned with how it will perform with extremely large 
> xml files, but could not find a direct answer anywhere in the 
> documentation.
>
> Thanks!
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20150518/bc5faff0/attachment.html>