Typically when NaNs/infs occur in my arrays something is going wrong, so ideally I'd like an exception to be raised when that happens. Failing that I'd like to have some robust (across python and Numeric/numarray versions) way of testing wether an array contains NaNs or inifinities. I'd be happy as long as this works on my particular platform (linux/intel). At the moment I don't have much of a clue how to achieve this, which can be pretty annoying (IIRC neither python nor Numeric is commited to any support of NaN values; python comes with the optional fpectl module, which apparently is however unmaintained, and generally deemed unsuitable for almost anything -- not that I even manged to build it. Also NaN behavior doesn't seem consistent across python scalars and Numeric arrays and different python version (again IIRC)). Any suggestions? How are other people dealing with these issues? 'as
On Mon, 2003-11-10 at 04:53, Alexander Schmolck wrote:
Typically when NaNs/infs occur in my arrays something is going wrong, so ideally I'd like an exception to be raised when that happens. Failing that I'd like to have some robust (across python and Numeric/numarray versions) way of testing wether an array contains NaNs or inifinities. I'd be happy as long as this works on my particular platform (linux/intel).
At the moment I don't have much of a clue how to achieve this, which can be pretty annoying (IIRC neither python nor Numeric is commited to any support of NaN values; python comes with the optional fpectl module, which apparently is however unmaintained, and generally deemed unsuitable for almost anything -- not that I even manged to build it. Also NaN behavior doesn't seem consistent across python scalars and Numeric arrays and different python version (again IIRC)).
Any suggestions? How are other people dealing with these issues?
Increasingly we are turning to R (http://www.r-project.org) because it provides fully integrated support for Inf, NaN and NA (missing) in all its data structures, including matrices and higher-rank arrays, viz:
2/0 [1] Inf 0/0 [1] NaN NA/1 [1] NA
R can be embedded in Python and passed NumPy arrays or other data structures via Walter Moriera's excellent RPy modules (see http;//rpy.sf.net). However, the fundamental problem is that support for NA (missing), Inf and NaN seems to be afterthoughts in both NumPy and numarray, whereas such support should have been integrated into the design right from the outset. Is it too late to incorporate them into the fabric of numarray? -- Tim C PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere or at http://members.optushome.com.au/tchur/pubkey.asc Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0
Tim Churches wrote:
Increasingly we are turning to R (http://www.r-project.org) because it provides fully integrated support for Inf, NaN and NA (missing) in all its data structures, including matrices and higher-rank arrays, viz:
2/0 [1] Inf 0/0 [1] NaN NA/1 [1] NA
R can be embedded in Python and passed NumPy arrays or other data structures via Walter Moriera's excellent RPy modules (see http;//rpy.sf.net).
However, the fundamental problem is that support for NA (missing), Inf and NaN seems to be afterthoughts in both NumPy and numarray, whereas such support should have been integrated into the design right from the outset.
Is it too late to incorporate them into the fabric of numarray?
I'm a little surprised at the comment that these issues are viewed as afterthoughts in numarray. We gave ieee 754 special value handling fairly careful thought. We may have missed something however, so I'm interested to hear what is considered missing in that regard. We allow the user to set whether ieee errors are ignored, print a warning, or raise an exception, and do so individually for all four kinds of errors. We also allow testing of and setting ieee special values. What is missing? (By the way, if one expects to be able to invoke an error handler for each individual exception that occurs in an array computation, we determined that that was too difficult to handle in any, portable and efficient way.) As to "NA" or "missing values", what is expected? These are not part of the ieee standard (unless I've missed something). How does this differ from using NaN, for example? Perry
On Tue, 2003-11-11 at 03:26, Perry Greenfield wrote:
Tim Churches wrote:
Increasingly we are turning to R (http://www.r-project.org) because it provides fully integrated support for Inf, NaN and NA (missing) in all its data structures, including matrices and higher-rank arrays, viz:
2/0 [1] Inf 0/0 [1] NaN NA/1 [1] NA
R can be embedded in Python and passed NumPy arrays or other data structures via Walter Moriera's excellent RPy modules (see http;//rpy.sf.net).
However, the fundamental problem is that support for NA (missing), Inf and NaN seems to be afterthoughts in both NumPy and numarray, whereas such support should have been integrated into the design right from the outset.
Is it too late to incorporate them into the fabric of numarray?
I'm a little surprised at the comment that these issues are viewed as afterthoughts in numarray.
I was being more than a little unfair to both NumPy and numarray. Unfair to NumPy because support for these things was considered, but deliberately excluded for sound performance reasons. And unfair to numarray because I haven't really studied what it does differently in these respects to NumPy. I'll now roll up my sleeves and take a good look at what numarray does before commenting further. Tim C
We gave ieee 754 special value handling fairly careful thought. We may have missed something however, so I'm interested to hear what is considered missing in that regard. We allow the user to set whether ieee errors are ignored, print a warning, or raise an exception, and do so individually for all four kinds of errors. We also allow testing of and setting ieee special values. What is missing? (By the way, if one expects to be able to invoke an error handler for each individual exception that occurs in an array computation, we determined that that was too difficult to handle in any, portable and efficient way.)
As to "NA" or "missing values", what is expected? These are not part of the ieee standard (unless I've missed something). How does this differ from using NaN, for example?
Perry
------------------------------------------------------- This SF.Net email sponsored by: ApacheCon 2003, 16-19 November in Las Vegas. Learn firsthand the latest developments in Apache, PHP, Perl, XML, Java, MySQL, WebDAV, and more! http://www.apachecon.com/ _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion --
Tim C PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere or at http://members.optushome.com.au/tchur/pubkey.asc Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0
participants (3)
-
Alexander Schmolck -
Perry Greenfield -
Tim Churches