The ubiquitous "Numerical Recipes" by Press, et al, is a reasonable place to start.  Although 'real' numerical analysts are generally scornful of the book, I've found that it gives me a pretty good overview of whatever topic I'm trying to learn.  I also learned a lot from Himmelblau's "Process analysis by statistical methods" ... long out of print, but you might find it in a library.

The most important question to answer is that of what sort of information you wish to extract from the data.  Least squares is the optimum (in a certain sense) fit of a certain sort of model to a set of data with particular properties.  In my field (engineering), most people aren't even aware that there are alternatives, and blindly assume that a least squares fit is "best."  Fortunately, it usually works 'well enough' (like a lot of the things we do in engineering).

There isn't enough detail in this thread for me to make any useful suggestions, even in the unlikely eventuality that I might know something useful, but it does seem to have a faint scent of overkill.

john

David Huard wrote:
Trevis,

2007/6/21, Trevis Crane <t_crane@mrl.uiuc.edu>:

As an aside, will those of you who are *more* in the know on this topic than the rest of us suggest a good text that has a worthwhile treatment of this subject (as well as other related data analysis/statistical issues)?

My bible is Probability Theory : The Logic of Science by E. T. Jaynes.
http://omega.albany.edu:8008/JaynesBook.html
It's not so much a book about optimization and fitting than on the general principles of probability. It was worth the reading time though.

There is a paper in the hydrological literature (Sorooshian and Dracup, water resources research, vol.16, no.2, 1980) that discusses the calibration of hydrologic models in correlated and heteroscedastic error cases. I guess every discipline has a paper similar to this one but this is the one I know.

There is also a Book by A. Zellner, An Introduction to Bayesian Inference in Econometrics, 1971 that I found helpful.

As you can see, I'm not aware of a comprehensive treatise on the subject. I just picked up bits from different articles.
HTH,
David

 

I'd love to learn more about it, but just jumping on Amazon and picking a book at almost random seems like a good way to waste a lot of money I don't have on books that I don't need, so if you have a favorite reference or text, I'm interested in knowing about it.

 

thanks,

trevis