Re: [Numpy-discussion] printoption to allow hexified floats?
Thanks for the replies.
Robert is right; many numerical operations, particularly complex ones, generate different values across platforms, and we deal with these by storing the values from some platform as a reference and using allclose(), which requires extra work. But many basic operations generate the same underlying values on IEEE 754-compliant platforms but don't always format floats consistently (see http://bugs.python.org/issue1580 for a lengthy discussion on this). My impression is that Python 2.7 does a better job here, but at this point a lot of differences also crop up between 2.6 (or less) and 2.7 due to the changed formatting built into 2.7, and these are the result of formatting differences; the numbers themselves are identical (in our experience so far, at any rate). This is a current pain-point which an exact representation would alleviate.
In response to David, we haven't implemented a separate print; we rely on the Numpy repr/str for ndarrays and the printoptions that allow some control over float formatting. I'm basically proposing to add a bit more control there. And thanks for the info on supported versions of Python.
Ken
On 12/2/10 8:14 AM, Robert Kern wrote:
On Wed, Dec 1, 2010 at 13:18, Ken Basyekbasye1@jhu.edu wrote:
Hi Numpy folks, ? ? When working with floats, I prefer to have exact string representations in doctests and other reference-based testing; I find it helps a lot to avoid chasing cross-platform differences that are really about the string conversion rather than about numerical differences.
Unfortunately, there are still cross-platform numerical differences that are real (but are irrelevant to the validity of the code under test). Hex-printing for floats only helps a little to make doctests useful for numerical code.
On Thu, Dec 2, 2010 at 11:17 AM, Ken Basye kbasye1@jhu.edu wrote:
Thanks for the replies.
Robert is right; many numerical operations, particularly complex ones, generate different values across platforms, and we deal with these by storing the values from some platform as a reference and using allclose(), which requires extra work. But many basic operations generate the same underlying values on IEEE 754-compliant platforms but don't always format floats consistently (see http://bugs.python.org/issue1580 for a lengthy discussion on this). My impression is that Python 2.7 does a better job here, but at this point a lot of differences also crop up between 2.6 (or less) and 2.7 due to the changed formatting built into 2.7, and these are the result of formatting differences; the numbers themselves are identical (in our experience so far, at any rate). This is a current pain-point which an exact representation would alleviate.
In response to David, we haven't implemented a separate print; we rely on the Numpy repr/str for ndarrays and the printoptions that allow some control over float formatting. I'm basically proposing to add a bit more control there. And thanks for the info on supported versions of Python.
Ken
Another approach to consider is to save the numerical data in a platform-independent standard file format (maybe like netcdf?). While this isn't a fool-proof approach because the calculations themselves may introduce differences that are platform dependent, this at least puts strong controls on one aspect of the overall problem.
One caveat that does come across my mind is if the save/load process for the file might have some platform-dependent differences based on the compression/decompression schemes. For example, the GRIB file format does a compression where the mean value and the differences from those means are stored. Calculations like these might introduce some slight differences on various platforms.
Just food for thought, Ben Root
participants (2)
-
Benjamin Root -
Ken Basye