In looking for simple ways to read and write data (in a text readable format) to and from a file and later restoring the actual data when reading back in, I've found that numpy arrays don't seem to play well with repr and eval.

E.g. to write some data (mixed types) to a file I can do this (fp is an open file),

thedata=[3.0,-4.9+2.0j,'another string'] repvars= repr(thedata)+"\n" fp.write(repvars)

Then to read it back and restore the data each to its original type,

strvars= fp.readline() sonofdata= eval(strvars) which gives back the original data list.

BUT when I try this with numpy arrays in the data list I find that repr of an array adds extra end-of-lines and that messes up the simple restoration of the data using eval.

Am I missing something simple? I know I've seen people recommend ways to save arrays to files, but I'm wondering what is the most straight-forward? I really like the simple, pythonic approach of the repr - eval pairing.

Thanks for any advice. (yes, I am googling, too)

-- Lou Pecora, my views are my own.

--- On Mon, 12/8/08, Matthieu Brucher matthieu.brucher@gmail.com wrote:

From: Matthieu Brucher matthieu.brucher@gmail.com Subject: Re: [Numpy-discussion] What to use to read and write numpy arrays to a file? To: "Discussion of Numerical Python" numpy-discussion@scipy.org Date: Monday, December 8, 2008, 3:56 PM Hi,

The repr - eval pair does not work with numpy. You can simply do a tofile() from file().

Matthieu

Yes, I found the tofile/fromfile pair, but they don't preserve the shape. Sorry, I should have been clearer on that in my request. I will be saving arrays whose shape I may not know later when I read them in. I'd like that information to be preserved.

Thanks.

-- Lou Pecora, my views are my own.

--- On Mon, 12/8/08, Robert Kern robert.kern@gmail.com wrote:

From: Robert Kern robert.kern@gmail.com Subject: Re: [Numpy-discussion] What to use to read and write numpy arrays to a file?

The most bulletproof way would be to use numpy.save() and numpy.load(), but this is a binary format, not a text one.

-- Robert Kern

Thanks, Robert. I may have to go that route, assuming that the save and load pair preserve shape, i.e. I don't have to know the shape when I read back in.

-- Lou Pecora, my views are my own.

Hi James,

2008/12/8 James james@fmnmedia.co.uk:

I have a very simple plot, and the lines join point to point, however i would like to add a line of best fit now onto the chart, i am really new to python etc, and didnt really understand those links!

Can anyone help me :)

It sounds like the second link, about linear regression, is a good place to start, and I've made a very simple example based on that:

----------------------------------------------- import numpy as np import matplotlib.pyplot as plt

x = np.linspace(0, 10, 11) #1 data_y = np.random.normal(size=x.shape, loc=x, scale=2.5) #2 plt.plot(x, data_y, 'bo') #3

coefs = np.lib.polyfit(x, data_y, 1) #4 fit_y = np.lib.polyval(coefs, x) #5 plt.plot(x, fit_y, 'b--') #6 ------------------------------------------------

Line 1 creates an array with the x values I have. Line 2 creates some random "data" I want to fit, which, in this case happens to be normally distributed around the unity line y=x. The raw data is plotted (assuming you have matplotlib installed as well - I suggest you do) by line 3, with blue circles.

Line 4 calculates the coefficients giving the least-squares best fit to a first degree polynomial (i.e. a straight line y = c0 * x + c1). So the values of coefs are c0 and c1 in the previous equation.

Line 5 calculates the y values on the fitted polynomial, at given x values, from the coefficients calculated in line 4, and line 6 simply plots these fitted y values, using a dotted blue line.

I hope that helps get you started. Keep posting questions on specific issues as they arise, and we'll see what we can do to help.

Angus.

Hi,

Thanks for all your help so far!

Right i think it would be easier to just show you the chart i have so far;

-------------------------- import numpy as np import matplotlib.pyplot as plt

plt.plot([4,8,12,16,20,24], [0.008,0.016,0.021,0.038,0.062,0.116], 'bo')

plt.xlabel("F (Number of washers)") plt.ylabel("v^2/r ms-2") plt.title("Circular Motion") plt.axis([2,26,0,0.120])

plt.show()

------------------------

Very basic i know, all i wish to do is add a line of best fit based on that data, in the examples there seems to be far more variables, do i need to split my data up etc?

Thanks

Scott Sinclair wrote:

...

2008/12/9 Angus McMorland amcmorl@gmail.com: Hi James,

2008/12/8 James james@fmnmedia.co.uk:

...

I have a very simple plot, and the lines join point to point, however i would like to add a line of best fit now onto the chart, i am really new to python etc, and didnt really understand those links!

Can anyone help me :)

It sounds like the second link, about linear regression, is a good place to start, and I've made a very simple example based on that:

---

import numpy as np import matplotlib.pyplot as plt

x = np.linspace(0, 10, 11) #1 data_y = np.random.normal(size=x.shape, loc=x, scale=2.5) #2 plt.plot(x, data_y, 'bo') #3

coefs = np.lib.polyfit(x, data_y, 1) #4 fit_y = np.lib.polyval(coefs, x) #5 plt.plot(x, fit_y, 'b--') #6

---

James, you'll want to add an extra line to the above code snippet so that Matplotlib displays the plot:

plt.show()

Cheers, Scott _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

On 12/8/2008 3:32 PM James apparently wrote:

I have a very simple plot, and the lines join point to point, however i would like to add a line of best fit now onto the chart, i am really new to python etc, and didnt really understand those links!

See the `slope_intercept` method of the OLS class at http://code.google.com/p/econpy/source/browse/trunk/pytrix/ls.py

Cheers, Alan Isaac