Hi Travis et al, I started looking at your seterr changes. I stared at yours for a while then I stared at mine for a while. Then I decided that mine wouldn't work right in the presence of threads. Then I decided that yours wouldn't work right in the presence of threads either. Specifically, it looks like ufunc_update_use_defaults isn't going to work. I think I know how to fix that, but I'm not sure that it's worth the trouble since I also did some benchmarking and it appears that the benefit of special casing is minimal. I looked at six cases: small (len-1), medium (len-1e4) and large (len-1e6) arrays with error checking on and error checking off. For medium and large arrays, I could discern no difference at all. For small arrays, there may be some difference, but it appears to be less than 5%. I'm not sure it's worth working through a bunch of finicky thread stuff to get just 5% back. If these benchmark numbers hold up I'd be inclined to rip out the use_default support since it's complicated enough that I know we'll end up chasing a few evil thread related bugs down through it. I'll include the benchmarking code below. If people could (a) look it over and confirm that I'm not doing something bogus and (b) try it on some different platforms and see if they see a more signifigant difference, I'd appreciate it. I'm also curious about the seterr interface. It returns ufunc_values_obj. I'm wasn't sure how one is supposed to pass that back in to seterr, so I modified seterr to instead return a dictionary. I also modified it so that the seterr function itself has no defaults (or rather they're all None). Instead, any unspecified values are taken from the current error state. Thus seterr(divide="warn") changes only the divide state, leaving the other entries alone. Regards, -tim if True: from timeit import Timer setup = """ import numpy numpy.seterr(divide="%s") a = numpy.zeros([%s], dtype=float) """ for size in [1, 10000, 1000000]: for i in range(3): for state in ['ignore', 'warn']: reps = min(100000000 / size, 100000) timer = Timer("a * a", setup % (state, size)) print "%s|%s =>" % (state, size), timer.timeit(reps) print print
Tim Hochberg wrote:
Hi Travis et al,
I started looking at your seterr changes.
Thank you very much for the help on this. I'm not an expert on threaded code by any means. In fact, as you clearly point out, I don't eat and drink what will work under threaded environments and what wont. Clearly global variables are problematic. That is the problem with the update_use_defaults bit, right? This is the way it was being managed before and I just changed names a bit to use PyThreadState_GetDict for the dictionary (it seems possible to use only from C until Python 2.4). I say if it only buys 5% on small arrays then it's not worth it as there are other fish to fry to make up for that 5% and I agree that tracking down threading problems due to a fanagled global variable is sticky. I did not think about the threading issues deeply enough.
I'm also curious about the seterr interface. It returns ufunc_values_obj. I'm wasn't sure how one is supposed to pass that back in to seterr, so I modified seterr to instead return a dictionary. I also modified it so that the seterr function itself has no defaults (or rather they're all None). Instead, any unspecified values are taken from the current error state. Thus seterr(divide="warn") changes only the divide state, leaving the other entries alone. Returning an object is a late-in-the-game idea and should be critiqued. It can be passed to seterr (an attribute check grabs the actual list --- did you want to change it to a dictionary?). Doesn't a small list have faster access than a small dictionary?
I'll look over your commits and comment later if I think of anything... I'm thrilled with your work. Best, -Travis
Hello all I was just wondering if someone could provide some example code that would cause an error if invalid is set to 'raise'? I also noticed that seterr returns the old values. Is this really useful? Consider its use in an IPython session: In [184]: N.geterr() Out[184]: {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'} In [185]: N.seterr(over='raise') Out[185]: {'over': 'ignore', 'divide': 'ignore', 'invalid': 'ignore', 'under': 'ignore'} I think the following pattern would make sense, but it seems it doesn't work at present: old=N.geterr() N.seterr(over='raise') # so some calculations that might overflow N.seterr(old) This currently causes the following error: Traceback (most recent call last): File "<ipython console>", line 1, in ? File "C:\Python24\Lib\site-packages\numpy\core\numeric.py", line 426, in seterr maskvalue = ((_errdict[divide] << SHIFT_DIVIDEBYZERO) + TypeError: dict objects are unhashable Is this intended? I think it would be useful to be able to restore all the error states in one go. Regards, Albert
-----Original Message----- From: numpy-discussion-admin@lists.sourceforge.net [mailto:numpy- discussion-admin@lists.sourceforge.net] On Behalf Of Travis Oliphant Sent: 22 April 2006 04:50 To: tim.hochberg@ieee.org; numpy-discussion Subject: [Numpy-discussion] Re: seterr changes
Tim Hochberg wrote:
Hi Travis et al,
I started looking at your seterr changes.
Thank you very much for the help on this. I'm not an expert on threaded code by any means. In fact, as you clearly point out, I don't eat and drink what will work under threaded environments and what wont. Clearly global variables are problematic. That is the problem with the update_use_defaults bit, right? This is the way it was being managed before and I just changed names a bit to use PyThreadState_GetDict for the dictionary (it seems possible to use only from C until Python 2.4).
I say if it only buys 5% on small arrays then it's not worth it as there are other fish to fry to make up for that 5% and I agree that tracking down threading problems due to a fanagled global variable is sticky. I did not think about the threading issues deeply enough.
I'm also curious about the seterr interface. It returns ufunc_values_obj. I'm wasn't sure how one is supposed to pass that back in to seterr, so I modified seterr to instead return a dictionary. I also modified it so that the seterr function itself has no defaults (or rather they're all None). Instead, any unspecified values are taken from the current error state. Thus seterr(divide="warn") changes only the divide state, leaving the other entries alone. Returning an object is a late-in-the-game idea and should be critiqued. It can be passed to seterr (an attribute check grabs the actual list --- did you want to change it to a dictionary?). Doesn't a small list have faster access than a small dictionary?
I'll look over your commits and comment later if I think of anything...
I'm thrilled with your work.
Best,
-Travis
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Albert Strasheim wrote: | old=N.geterr() | N.seterr(over='raise') | # so some calculations that might overflow | N.seterr(old) You should try (but I didn't): N.seterr(**old) Rob - -- Rob W.W. Hooft || rob@hooft.net || http://www.hooft.net/people/rob/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFESno1H7J/Cv8rb3QRAppZAKCGBRSvL++wg3wFer6odmG8sxyrFwCfQ1nq p0aVr4r+Z1ZfRBGQgir+KX0= =eZMa -----END PGP SIGNATURE-----
participants (4)
-
Albert Strasheim -
Rob Hooft -
Tim Hochberg -
Travis Oliphant