[SciPy-user] Newbie, deferring large array generation

[Joseph Anderson]

Hello All,

I have a problem in that on occasion I need to generate very large arrays that won't fit into memory. For what I'm doing, calling linspace is convenient, so I'd like to be able to use it. A very simple example of my problem is the case:

a = linspace(start, stop, very_big_num)
 
As a python/scipy newbie, I'm just becoming familiar with appropriate coding patterns. One thought was whether it is possible to use generators to defer the array creation. I'd like to be able to use linspace (pleasantly simple) but defer generation of the complete array, returning individual values or a range of values on demand. My question is, whether the following generator expression is appropriate or not? Or whether linspace will fill its array (and therefore overflow memory)?

g = (val for val in linspace(start, stop, very_big_num))

So will this work?

I realize that it is possible for me to write my own version of linspace as a generator--but hoping I don't have to.

If the above generator expression doesn't work, are there any other straight forward coding patterns that will do what I'm looking for?


Thanks for the help.


My best,
Jo


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Joseph Anderson
Lecturer in Music

School of Arts and New Media
University of Hull, Scarborough Campus,
Scarborough, North Yorkshire, YO11 3AZ, UK

T: +44.(0)1723.357341 T: +44.(0)1723.357370 F: +44.(0)1723.350815
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 3490 bytes
Desc: not available
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20080319/78e6a22e/attachment.bin>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20080319/78e6a22e/attachment.ksh>