I've just checked in a fix for fastumath's logical functions. They used to give wrong answers on OSX and caused some tests to fail. I believe this to be an endianness issue, but I don't have another big-endian machine to test on. All tests still pass on Athlon/Linux. The changes made largely track what Numeric did, so I'm not sure if these were conscious deviations to speed up fastumath or just a failure to keep up with the bugfixes in Numeric. I have only implemented fastumath_unsigned.inc as yet. I will make similar changes in fastumath_nounsigned.inc as I accumulate tuits. Travis, could you check my changes to make sure I didn't revert differences you deliberately made? If they were deliberate, then we need to figure a way to revert to the correct, if slower, Numeric implementations on the Mac. Also, if anyone is using SciPy on a Sun or other big-endian machine, I would appreciate the output of scipy.test() with and without the patch. Thank you, everyone. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
Robert Kern wrote:
I've just checked in a fix for fastumath's logical functions. They used to give wrong answers on OSX and caused some tests to fail. I believe this to be an endianness issue, but I don't have another big-endian machine to test on. All tests still pass on Athlon/Linux.
The changes made largely track what Numeric did, so I'm not sure if these were conscious deviations to speed up fastumath or just a failure to keep up with the bugfixes in Numeric.
I have only implemented fastumath_unsigned.inc as yet. I will make similar changes in fastumath_nounsigned.inc as I accumulate tuits.
Travis, could you check my changes to make sure I didn't revert differences you deliberately made? If they were deliberate, then we need to figure a way to revert to the correct, if slower, Numeric implementations on the Mac.
I'm not sure what the problem is here. But I think you undid deliberate changes that have implications for alter_numeric() and need to be discussed. I had intentially made the results of all logical operations be unsigned bytes. What was broken about this on the Mac? Could you explain your changes here, better? -Travis
Travis Oliphant wrote:
Robert Kern wrote:
I've just checked in a fix for fastumath's logical functions. They used to give wrong answers on OSX and caused some tests to fail. I believe this to be an endianness issue, but I don't have another big-endian machine to test on. All tests still pass on Athlon/Linux.
The changes made largely track what Numeric did, so I'm not sure if these were conscious deviations to speed up fastumath or just a failure to keep up with the bugfixes in Numeric.
I have only implemented fastumath_unsigned.inc as yet. I will make similar changes in fastumath_nounsigned.inc as I accumulate tuits.
Travis, could you check my changes to make sure I didn't revert differences you deliberately made? If they were deliberate, then we need to figure a way to revert to the correct, if slower, Numeric implementations on the Mac.
I'm not sure what the problem is here. But I think you undid deliberate changes that have implications for alter_numeric() and need to be discussed. I had intentially made the results of all logical operations be unsigned bytes. What was broken about this on the Mac?
Here are the tests that fail. The first two are what led me to the "fix" in the first place. The CDF failures are newer (I haven't seen them until today because I always used my version of fastumath_unsigned.inc for some months now), but all tests pass with my version. A quick summary:
scipy.logical_and.reduce([1,1]) 16777217
====================================================================== FAIL: check_nd (scipy_base.function_base.test_function_base.test_all) ---------------------------------------------------------------------- Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/scipy_base/tests/test_function_base.py", line 39, in check_nd assert_array_equal(alltrue(y1),[0,0,1]) File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/scipy_test/testing.py", line 659, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 33.3333333333%): Array 1: [ 0 0 16777217] Array 2: [0 0 1] ====================================================================== FAIL: check_nd (scipy_base.function_base.test_function_base.test_any) ---------------------------------------------------------------------- Traceback (most recent call last): File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/scipy_base/tests/test_function_base.py", line 23, in check_nd assert_array_equal(sometrue(y1),[1,1,0]) File "/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/scipy_test/testing.py", line 659, in assert_array_equal assert cond,\ AssertionError: Arrays are not equal (mismatch 66.6666666667%): Array 1: [16777216 16777216 0] Array 2: [1 1 0] ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_alpha) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.8901307480205483,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_anglit) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.134833494822; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_arcsine) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.107380700111; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_beta) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.3058217448839688, 1.638923226210558) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_betaprime) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.7946828675209945, 1.3659679931964808) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_bradford) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.995535943496068,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_burr) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.7999727389493319, 1.0291342027239256) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_cauchy) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.143052525322; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_chi) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.6750617919359232,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_chi2) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.5046670059082969,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_dgamma) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.8843762379065128,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_dweibull) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.1843576971636054,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_erlang) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (4, 0.91640849579812245, 0.79817883530002565) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_expon) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.107035405269; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_exponpow) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.1084215205292667,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_exponweib) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.9765503019823849, 1.8216168224929503) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_f) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.8477848988062577, 1.7773122402806831) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_fatiguelife) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.354274749801879,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_fisk) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.2761579637690608,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_foldcauchy) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.6521602833511251,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_foldnorm) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.7554136027893463,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_frechet_l) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.3212613080139093,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_frechet_r) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.831564794039293,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_gamma) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.6083153459617701,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_genextreme) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.0909121314685; pval = nan; alpha = 0.01 args = (1.7210441119052327,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_gengamma) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.15651460886; pval = nan; alpha = 0.01 args = (1.1265602049356369, 1.789773633933041) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_genhalflogistic) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.0612444996834; pval = nan; alpha = 0.01 args = (1.8490619343641432,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_genlogistic) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.419426229431517,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_genpareto) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.135486149788; pval = nan; alpha = 0.01 args = (1.4346656353328937,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_gilbrat) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.131796498598; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_gompertz) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.2018899800407579,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_gumbel_l) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.168374500672; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_gumbel_r) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.150828138987; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_halfcauchy) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.101199208697; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_halflogistic) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.113176395496; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_halfnorm) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.150618899083; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_hypsecant) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.128143958251; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_laplace) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.213923497995; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_loggamma) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.1019995326307017,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_logistic) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.0543847779433; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_lognorm) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.7085755404653427,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_lomax) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.3353241977770063,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_maxwell) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.115218121035; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_nakagami) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.886918043114695,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_ncf) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.6070744080464818, 1.4502566675774289, 1.4943408411354844) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_nct) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.4704379855857841, 1.7611435834086047) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_ncx2) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.2394290524371847, 1.3550394000897465) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_norm) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.159503246745; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_pareto) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.7004776605275855,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_powerlaw) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.5178586665615643,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_rayleigh) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.0521100146552; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_reciprocal) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.0984953045845; pval = nan; alpha = 0.01 args = (0.52198748250163574, 1.5219874825016357) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_t) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.1703651942244617,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_triang) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.120754329364; pval = nan; alpha = 0.01 args = (0.31036819444097996,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_tukeylambda) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.4154560974523536,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_uniform) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = 0.207966844241; pval = nan; alpha = 0.01 args = () ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_weibull_max) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.8048478136464641,) ====================================================================== FAIL: check_cdf (scipy.stats.distributions.test_distributions.test_weibull_min) ---------------------------------------------------------------------- Traceback (most recent call last): File "<string>", line 10, in check_cdf AssertionError: D = nan; pval = nan; alpha = 0.01 args = (1.4914168094327804,) ---------------------------------------------------------------------- Ran 936 tests in 14.693s FAILED (failures=60)
Could you explain your changes here, better?
Mostly, I just patched in what was in Numeric and things began to work. There's no more rationale than that. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
Robert Kern wrote:
Travis Oliphant wrote:
Robert Kern wrote:
I've just checked in a fix for fastumath's logical functions. They used to give wrong answers on OSX and caused some tests to fail. I believe this to be an endianness issue, but I don't have another big-endian machine to test on. All tests still pass on Athlon/Linux.
The changes made largely track what Numeric did, so I'm not sure if these were conscious deviations to speed up fastumath or just a failure to keep up with the bugfixes in Numeric.
I have only implemented fastumath_unsigned.inc as yet. I will make similar changes in fastumath_nounsigned.inc as I accumulate tuits.
Travis, could you check my changes to make sure I didn't revert differences you deliberately made? If they were deliberate, then we need to figure a way to revert to the correct, if slower, Numeric implementations on the Mac.
I'm not sure what the problem is here. But I think you undid deliberate changes that have implications for alter_numeric() and need to be discussed. I had intentially made the results of all logical operations be unsigned bytes. What was broken about this on the Mac?
Here are the tests that fail. The first two are what led me to the "fix" in the first place. The CDF failures are newer (I haven't seen them until today because I always used my version of fastumath_unsigned.inc for some months now), but all tests pass with my version.
A quick summary:
I appreciate you finding this bug. I had no idea it was there and it has probably been there for some time. What I don't get is that your fix just changes the type of the pointer to be assigned to, but (presumably) doesn't change the type of the array being populated. So, why doesn't this create problems? How can you have an output array that is UInt8 but assign to it as if it were Int32? This seems like an endian-ness issue and your fix is fooling the MAC platform. But, how would this work on other little-endian platforms? -Travis
Robert Kern wrote:
Travis Oliphant wrote:
Robert Kern wrote:
I've just checked in a fix for fastumath's logical functions. They used to give wrong answers on OSX and caused some tests to fail. I believe this to be an endianness issue, but I don't have another big-endian machine to test on. All tests still pass on Athlon/Linux.
The changes made largely track what Numeric did, so I'm not sure if these were conscious deviations to speed up fastumath or just a failure to keep up with the bugfixes in Numeric.
I have only implemented fastumath_unsigned.inc as yet. I will make similar changes in fastumath_nounsigned.inc as I accumulate tuits.
Travis, could you check my changes to make sure I didn't revert differences you deliberately made? If they were deliberate, then we need to figure a way to revert to the correct, if slower, Numeric implementations on the Mac.
I'm not sure what the problem is here. But I think you undid deliberate changes that have implications for alter_numeric() and need to be discussed. I had intentially made the results of all logical operations be unsigned bytes. What was broken about this on the Mac?
Here are the tests that fail. The first two are what led me to the "fix" in the first place. The CDF failures are newer (I haven't seen them until today because I always used my version of fastumath_unsigned.inc for some months now), but all tests pass with my version.
I think I know what is wrong. Apparently, the reduce method expects the output type to be the same as the input type. So, when the underlying code is called in that context, what actually gets passed in is not an unsigned byte array. Thus, when the code casts the pointer to an unsigned byte pointer, it is filling in only a portion of the memory. On little-endian machines this is working fine, but on big-endian machines it's the wrong portion to be filling in. Robert's fix will correct the problem for reduce on MacOS but then I'm not sure that logical_and will work correctly on two arrays still as in that context the output array should be a UInt8 and so you will be filling a UInt8 array assuming it is IntXX. Bottom line is I don't think the fixes will work on all platforms but will fix one at the expense of another? Can you comment on this Robert? Darn, this seems like a pretty serious problem. -Travis
Travis Oliphant wrote: [snip]
I think I know what is wrong. Apparently, the reduce method expects the output type to be the same as the input type. So, when the underlying code is called in that context, what actually gets passed in is not an unsigned byte array. Thus, when the code casts the pointer to an unsigned byte pointer, it is filling in only a portion of the memory. On little-endian machines this is working fine, but on big-endian machines it's the wrong portion to be filling in. Robert's fix will correct the problem for reduce on MacOS but then I'm not sure that logical_and will work correctly on two arrays still as in that context the output array should be a UInt8 and so you will be filling a UInt8 array assuming it is IntXX.
Check the signatures again. The logical_* functions use divide_safe_signatures which does not output to UInt8. CVS on Athlon/Linux:
from scipy import * a = array([0,1]) b = array([1,1]) logical_and(a,b) array([0, 1]) logical_and(a.astype(Float),b.astype(Float)) array([ 0.00000000e+000, 4.94065646e-324]) logical_and(a.astype(Float32),b.astype(Float32)) array([ 0.00000000e+000 1.40129846e-45],'f') logical_and.reduce(array([[0,1],[1,1]]).astype(Float)) array([ 0., 1.]) import struct struct.unpack('f', '\x01\x00\x00\x00') (1.4012984643248171e-45,) struct.unpack('d', '\x01\x00\x00\x00\x00\x00\x00\x00') (4.9406564584124654e-324,)
Bottom line is I don't think the fixes will work on all platforms but will fix one at the expense of another? Can you comment on this Robert?
Yes, just changing the signatures to correctly output to UInt8 solves the logical_and(a, b) problem just fine with either endianness. The logical_and.reduce(b) problem needs more work. Even changing the signatures still makes logical_and.reduce(b) return an array the same type as b.
Darn, this seems like a pretty serious problem.
-Travis
-- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
Robert Kern wrote:
Travis Oliphant wrote: [snip]
I think I know what is wrong. Apparently, the reduce method expects the output type to be the same as the input type. So, when the underlying code is called in that context, what actually gets passed in is not an unsigned byte array. Thus, when the code casts the pointer to an unsigned byte pointer, it is filling in only a portion of the memory. On little-endian machines this is working fine, but on big-endian machines it's the wrong portion to be filling in. Robert's fix will correct the problem for reduce on MacOS but then I'm not sure that logical_and will work correctly on two arrays still as in that context the output array should be a UInt8 and so you will be filling a UInt8 array assuming it is IntXX.
Check the signatures again. The logical_* functions use divide_safe_signatures which does not output to UInt8.
O.K.. Apparently, the logical_and and logical_or functions cannot be converted to have UInt8 output and still have reduce work. Reduce will only work reliably when the output type is the same as the input type. As logical_and and logical_or are used with reduce quite a bit, I think our best option is to do what Robert was doing and keep logical_and, logical_or, and logical_xor as returning the type of their inputs. So, if there are no complaints, go ahead Robert and submit the changes you have made to CVS (I won't get in your way again...) so that divide_safe_signatures are O.K. and the logical_XXX functions are changed so the output type is the same as the input type. -Travis
participants (2)
-
Robert Kern -
Travis Oliphant