Hi Ryan, Note that 1e-16 is the approximate _fractional_ precision of double precision floats. You can get absolute numbers much smaller than that. There are a bunch of good references to floating point accuracy etc. Some of them are linked here: http://babbage.cs.qc.edu/courses/cs341/IEEE-754references.html Note that the FFTW website lists FFT accuracies in both single and double precision here: http://www.fftw.org/accuracy/ Scott On Wednesday 21 December 2005 02:53 pm, Ryan Krauss wrote:
I have run into this as well. 1e-16 seems large to me for double precision. Robert can you explain this a bit more please. Is there an IEEE spec or something that specifies how much of the 64bits is for exponent and how much is for the mantissa (I think that is the right word)? I was playing with some FORTRAN code and it seems like there was a big difference with complex vs. double complex. It seems like 1e-16 was the magnitude floor of complex and double complex was better.
Ryan
On 12/21/05, Robert Kern <robert.kern@gmail.com> wrote:
Darren Dale wrote:
I am running into problems related, I think, to numerical precision of fast Fourier transforms. If I Fourier transform a gaussian distribution:
absolute(fft(stats.norm.pdf(linspace(-10,10,512), loc=0, scale=1)))
I find a floor of about 1e-16. Does anyone know of a way to improve the precision?
1e-16 is the best you can do with double precision.
-- Robert Kern robert.kern@gmail.com
"In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.net http://www.scipy.net/mailman/listinfo/scipy-user
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.net http://www.scipy.net/mailman/listinfo/scipy-user
-- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: sransom@nrao.edu Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989