Hi all, The following code succeeds, while I thought it should fail: a = np.zeros(6) # real b= np.arange(6)*(2+3j) # complex a[1] = b[1] # shouldn't this break? What is the rationale behind this behaviour? Cheers Stéfan
On Fri, Mar 6, 2009 at 6:18 PM, Stéfan van der Walt <stefan@sun.ac.za>wrote:
Hi all,
The following code succeeds, while I thought it should fail:
a = np.zeros(6) # real b= np.arange(6)*(2+3j) # complex a[1] = b[1] # shouldn't this break?
What is the rationale behind this behaviour?
The same as this: In [1]: a = zeros(2) In [2]: a[0] = '1' In [3]: a Out[3]: array([ 1., 0.]) The question is whether such usage is likely in error, calling for an exception, or a useful convenience to avoid a cast. I tend to think that casts should be made explicit in such cases but it's a fine line. What about this? In [5]: a = zeros(2) In [6]: a[0] = 1 In [7]: a Out[7]: array([ 1., 0.]) Should a cast be required? What if the lhs is a float32 and the rhs is a python float? Chuck
2009/3/7 Charles R Harris <charlesr.harris@gmail.com>:
a = np.zeros(6) # real b= np.arange(6)*(2+3j) # complex a[1] = b[1] # shouldn't this break?
What is the rationale behind this behaviour?
The same as this:
In [1]: a = zeros(2)
In [2]: a[0] = '1'
In [3]: a Out[3]: array([ 1., 0.])
This difference is that, in your example, no information is lost. When assigning a complex value to a real array, you are probably doing something wrong. Cheers Stéfan
On Sat, Mar 7, 2009 at 03:30, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Charles R Harris <charlesr.harris@gmail.com>:
a = np.zeros(6) # real b= np.arange(6)*(2+3j) # complex a[1] = b[1] # shouldn't this break?
What is the rationale behind this behaviour?
The same as this:
In [1]: a = zeros(2)
In [2]: a[0] = '1'
In [3]: a Out[3]: array([ 1., 0.])
This difference is that, in your example, no information is lost. When assigning a complex value to a real array, you are probably doing something wrong.
In [5]: z = zeros(3, int) In [6]: z[1] = 1.5 In [7]: z Out[7]: array([0, 1, 0]) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
2009/3/7 Robert Kern <robert.kern@gmail.com>:
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
Would it be possible to, optionally, throw an exception? S.
Stéfan van der Walt wrote:
Would it be possible to, optionally, throw an exception?
S. _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
I'm certain that it is possible. And, I believe that if this option is selected via a Python flag, the run-time performance implications should be nil. I wonder if there is some way of taking a vote to see how many people would like such an option. -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
On Sat, Mar 7, 2009 at 5:18 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
In that case, do you know why this raises an exception: np.int64(10+20j) Darren
Darren Dale wrote:
On Sat, Mar 7, 2009 at 5:18 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
In that case, do you know why this raises an exception:
np.int64(10+20j)
Darren
I think that you have a good point, Darren, and that Robert is oversimplifying the situation. NumPy and Python are somewhat out of step. The NumPy approach is stricter and more likely to catch errors than Python. Python tends to be somewhat laissez-faire about numerical errors and the correctness of results. Unfortunately, NumPy seems to be a sort of step-child of Python, tolerated, but not fully accepted. There are a number of people who continue to use Matlab, despite all of its deficiencies, because it can at least be counted on to produce correct answers most of the time. Dr. Phillip M. Feldman -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
ti, 2009-12-08 kello 22:26 -0800, Dr. Phillip M. Feldman kirjoitti:
Darren Dale wrote:
On Sat, Mar 7, 2009 at 5:18 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
I'd think that downcasting is different from dropping the imaginary part. Also, I doubt a bit that there is a large body of correct code relying on the implicit behavior. This kind of assertions should of course be checked experimentally -- make the complex downcast an error, and check a few prominent software packages. An alternative to an exception would be to make complex numbers with nonzero imaginary parts to cast to *nan*. This would, however, likely lead to errors difficult to track. Another alternative would be to raise an error only if the imaginary part is non-zero. This requires some additional checking in some places where no checking is usually made. At least I tend to use .real or real() to explicitly take the real part. In interactive use, it occasionally is convenient to have the real part taken "automatically", but sometimes this leads to problems inside Matplotlib. Nevertheless, I can't really regard dropping the imaginary part a significant issue. I've sometimes bumped into problems because of it, and it would have been nice to catch them earlier, though. (As an example, scipy.interpolate.interp1d some time ago silently dropped the imaginary part -- not nice.) -- Pauli Virtanen
Pauli Virtanen wrote:
ti, 2009-12-08 kello 22:26 -0800, Dr. Phillip M. Feldman kirjoitti:
Darren Dale wrote:
On Sat, Mar 7, 2009 at 5:18 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
I'd think that downcasting is different from dropping the imaginary part. Also, I doubt a bit that there is a large body of correct code relying on the implicit behavior. This kind of assertions should of course be checked experimentally -- make the complex downcast an error, and check a few prominent software packages.
An alternative to an exception would be to make complex numbers with nonzero imaginary parts to cast to *nan*. This would, however, likely lead to errors difficult to track.
Another alternative would be to raise an error only if the imaginary part is non-zero. This requires some additional checking in some places where no checking is usually made.
At least I tend to use .real or real() to explicitly take the real part. In interactive use, it occasionally is convenient to have the real part taken "automatically", but sometimes this leads to problems inside Matplotlib.
Nevertheless, I can't really regard dropping the imaginary part a significant issue. I've sometimes bumped into problems because of it, and it would have been nice to catch them earlier, though. (As an example, scipy.interpolate.interp1d some time ago silently dropped the imaginary part -- not nice.)
FWIW, +1. And if nothing is done, there should at least be big fat red warnings prominently in the documentation. (Knowing that imaginary parts can in some situations be dropped without warning makes me rather uneasy...at least now I know to look out for it and double check the dtypes on the lhs.) Dag Sverre
On Wed, Dec 9, 2009 at 2:04 AM, Pauli Virtanen <pav@iki.fi> wrote:
ti, 2009-12-08 kello 22:26 -0800, Dr. Phillip M. Feldman kirjoitti:
Darren Dale wrote:
On Sat, Mar 7, 2009 at 5:18 AM, Robert Kern <robert.kern@gmail.com> wrote:
On Sat, Mar 7, 2009 at 04:10, Stéfan van der Walt <stefan@sun.ac.za> wrote:
2009/3/7 Robert Kern <robert.kern@gmail.com>:
In [5]: z = zeros(3, int)
In [6]: z[1] = 1.5
In [7]: z Out[7]: array([0, 1, 0])
Blind moment, sorry. So, what is your take -- should this kind of thing pass silently?
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
I'd think that downcasting is different from dropping the imaginary part. Also, I doubt a bit that there is a large body of correct code relying on the implicit behavior. This kind of assertions should of course be checked experimentally -- make the complex downcast an error, and check a few prominent software packages.
An alternative to an exception would be to make complex numbers with nonzero imaginary parts to cast to *nan*. This would, however, likely lead to errors difficult to track.
Another alternative would be to raise an error only if the imaginary part is non-zero. This requires some additional checking in some places where no checking is usually made.
At least I tend to use .real or real() to explicitly take the real part. In interactive use, it occasionally is convenient to have the real part taken "automatically", but sometimes this leads to problems inside Matplotlib.
Nevertheless, I can't really regard dropping the imaginary part a significant issue. I've sometimes bumped into problems because of it, and it would have been nice to catch them earlier, though. (As an example, scipy.interpolate.interp1d some time ago silently dropped the imaginary part -- not nice.)
It looks like a lot of folks have written or met buggy code at one point or another because this behaviour. And finding the problem is a hassle because it doesn't stand out. I think that makes it a candidate for a warning simply because we want to help people write correct code and that is a significant issue. So +1 for raising a warning in this case. I feel the same about silently casting floats to integers, although that doesn't feel quite as strange because one at least expects the result to be close to the original. I think the boundaries are unsigned discrete <- signed discrete <- real line <- complex plane The different kinds all have different domains and crossing the boundaries between kinds should only happen if it is the clear intent of the programmer. Because python types don't match up 1-1 with numpy types this can be tricky to enforce but I think is worthwhile to keep it in mind. Chuck
Pauli Virtanen-3 wrote:
I'd think that downcasting is different from dropping the imaginary part.
There are many ways (in fact, an unlimited number) to downcast from complex to real. Here are three possibilities: - Take the real part. - Take the magnitude (root-mean-square of the real and imaginary parts). - Assign a NaN. -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
Pauli Virtanen-3 wrote:
Nevertheless, I can't really regard dropping the imaginary part a significant issue.
I am amazed that anyone could say this. For anyone who works with Fourier transforms, or with electrical circuits, or with electromagnetic waves, dropping the imaginary part is a huge issue because we get answers that are totally wrong. When I recently tried to validate a code, the answers were wrong, and it took two full days to track down the cause. I am now forced to reconsider carefully whether Python/NumPy is a suitable platform for serious scientific computing. -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
On Wed, Dec 09, 2009 at 05:54:07PM -0800, Dr. Phillip M. Feldman wrote:
Pauli Virtanen-3 wrote:
Nevertheless, I can't really regard dropping the imaginary part a significant issue.
I am amazed that anyone could say this. For anyone who works with Fourier transforms, or with electrical circuits, or with electromagnetic waves, dropping the imaginary part is a huge issue because we get answers that are totally wrong.
When I recently tried to validate a code, the answers were wrong, and it took two full days to track down the cause. I am now forced to reconsider carefully whether Python/NumPy is a suitable platform for serious scientific computing.
You've now said this a couple times. And it is fine if that is your opinion. However, I think it is incorrect. I've been using numeric/numarray/numpy for about 12 years as my main scientific computing platform. And I do extensive work with Fourier Transforms and other complex numbers. I have not once run into this issue and in fact, my use of numpy has improved my scientific productivity dramatically. Most of the casting rules were set a very long time ago and are there for very good reasons. While it is certainly possibly that there could be bugs in corner cases of some of them, or that those rules surprise some people due to their familiarity with other behaviours, but that does not change the fact that most of them are in place in numpy for good reasons. Scott -- Scott M. Ransom Address: NRAO Phone: (434) 296-0320 520 Edgemont Rd. email: sransom@nrao.edu Charlottesville, VA 22903 USA GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989
2009/12/9 Dr. Phillip M. Feldman <pfeldman@verizon.net>:
Pauli Virtanen-3 wrote:
Nevertheless, I can't really regard dropping the imaginary part a significant issue.
I am amazed that anyone could say this. For anyone who works with Fourier transforms, or with electrical circuits, or with electromagnetic waves, dropping the imaginary part is a huge issue because we get answers that are totally wrong.
I agree that dropping the imaginary part is a wart. But it is one that is not very hard to learn to live with. I say this as someone who has been burned by it while using Fourier analysis to work with astronomical data.
When I recently tried to validate a code, the answers were wrong, and it took two full days to track down the cause. I am now forced to reconsider carefully whether Python/NumPy is a suitable platform for serious scientific computing.
While I find the current numpy complex->real conversion annoying, I have to say, this kind of rhetoric does not benefit your cause. It sounds childish and manipulative, and makes even people who agree in principle want to tell you to go ahead and use MATLAB and stop pestering us. We are not here to sell you on numpy; if you hate it, don't use it. We are here because *we* use it, warts and all, and we want to discuss interesting topics related to numpy. That you would have implemented it differently is not very interesting if you are not even willing to understand why it is the way it is and what a change would cost, let alone propose a workable way to improve. Anne
Anne Archibald wrote:
2009/12/9 Dr. Phillip M. Feldman <pfeldman@verizon.net>:
When I recently tried to validate a code, the answers were wrong, and it took two full days to track down the cause. I am now forced to reconsider carefully whether Python/NumPy is a suitable platform for serious scientific computing.
While I find the current numpy complex->real conversion annoying, I have to say, this kind of rhetoric does not benefit your cause. It sounds childish and manipulative, and makes even people who agree in principle want to tell you to go ahead and use MATLAB and stop pestering us. We are not here to sell you on numpy; if you hate it, don't use it. We are here because *we* use it, warts and all, and we want to discuss interesting topics related to numpy. That you would have implemented it differently is not very interesting if you are not even willing to understand why it is the way it is and what a change would cost, let alone propose a workable way to improve.
At this point I want to remind us about Charles Harris' very workable proposal: Raise a warning. That should both keep backward compatability and prevent people from wasting days. (Hopefully, we can avoid wasting days discussing this issue too :-) ). Dag Sverre
Thu, 10 Dec 2009 09:54:32 +0100, Dag Sverre Seljebotn wrote: [clip]
At this point I want to remind us about Charles Harris' very workable proposal: Raise a warning. That should both keep backward compatability and prevent people from wasting days. (Hopefully, we can avoid wasting days discussing this issue too :-) ).
Yes. We can even make it a PendingDeprecationWarning, when we become convinced that this feature should go away in entirety. -- Pauli Virtanen
On Thu, Dec 10, 2009 at 04:24, Pauli Virtanen <pav+sp@iki.fi> wrote:
Thu, 10 Dec 2009 09:54:32 +0100, Dag Sverre Seljebotn wrote: [clip]
At this point I want to remind us about Charles Harris' very workable proposal: Raise a warning. That should both keep backward compatability and prevent people from wasting days. (Hopefully, we can avoid wasting days discussing this issue too :-) ).
Yes. We can even make it a PendingDeprecationWarning, when we become convinced that this feature should go away in entirety.
PendingDeprecationWarnings are off by default, I think. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
to, 2009-12-10 kello 10:18 -0600, Robert Kern kirjoitti:
On Thu, Dec 10, 2009 at 04:24, Pauli Virtanen <pav+sp@iki.fi> wrote:
Thu, 10 Dec 2009 09:54:32 +0100, Dag Sverre Seljebotn wrote: [clip]
At this point I want to remind us about Charles Harris' very workable proposal: Raise a warning. That should both keep backward compatability and prevent people from wasting days. (Hopefully, we can avoid wasting days discussing this issue too :-) ).
Yes. We can even make it a PendingDeprecationWarning, when we become convinced that this feature should go away in entirety.
PendingDeprecationWarnings are off by default, I think.
So it seems. r7993 makes Numpy to raise a custom ComplexWarning when these things occur. It's easy for the user to change that into an error or silence it. -- Pauli Virtanen
Hi Pauli, On Thu, Dec 10, 2009 at 12:34 PM, Pauli Virtanen <pav@iki.fi> wrote:
to, 2009-12-10 kello 10:18 -0600, Robert Kern kirjoitti:
On Thu, Dec 10, 2009 at 04:24, Pauli Virtanen <pav+sp@iki.fi<pav%2Bsp@iki.fi>> wrote:
Thu, 10 Dec 2009 09:54:32 +0100, Dag Sverre Seljebotn wrote: [clip]
At this point I want to remind us about Charles Harris' very workable proposal: Raise a warning. That should both keep backward compatability and prevent people from wasting days. (Hopefully, we can avoid wasting days discussing this issue too :-) ).
Yes. We can even make it a PendingDeprecationWarning, when we become convinced that this feature should go away in entirety.
PendingDeprecationWarnings are off by default, I think.
So it seems.
r7993 makes Numpy to raise a custom ComplexWarning when these things occur. It's easy for the user to change that into an error or silence it.
You need a small fixup to support python-2.4. For instance, the deprecation warning is implemented as: #if PY_VERSION_HEX >= 0x02050000 #define DEPRECATE(msg) PyErr_WarnEx(PyExc_DeprecationWarning,msg,1) #else #define DEPRECATE(msg) PyErr_Warn(PyExc_DeprecationWarning,msg) #endif Thanks for settling this discussion ;) Chuck
On Thu, Dec 10, 2009 at 2:54 AM, Dag Sverre Seljebotn <dagss@student.matnat.uio.no> wrote:
Anne Archibald wrote:
2009/12/9 Dr. Phillip M. Feldman <pfeldman@verizon.net>:
When I recently tried to validate a code, the answers were wrong, and it took two full days to track down the cause. I am now forced to reconsider carefully whether Python/NumPy is a suitable platform for serious scientific computing.
While I find the current numpy complex->real conversion annoying, I have to say, this kind of rhetoric does not benefit your cause. It sounds childish and manipulative, and makes even people who agree in principle want to tell you to go ahead and use MATLAB and stop pestering us. We are not here to sell you on numpy; if you hate it, don't use it. We are here because *we* use it, warts and all, and we want to discuss interesting topics related to numpy. That you would have implemented it differently is not very interesting if you are not even willing to understand why it is the way it is and what a change would cost, let alone propose a workable way to improve.
At this point I want to remind us about Charles Harris' very workable proposal: Raise a warning. That should both keep backward compatability and prevent people from wasting days. (Hopefully, we can avoid wasting days discussing this issue too :-) ).
+1 Completely agree. And to be clear, I realize the need not to break anything relying on this behavior. I just don't want people passing this off as a non-issue/'not a big deal'. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma
On 9-Dec-09, at 1:26 AM, Dr. Phillip M. Feldman wrote:
NumPy and Python are somewhat out of step. The NumPy approach is stricter and more likely to catch errors than Python. Python tends to be somewhat laissez-faire about numerical errors and the correctness of results.
The bug/quirk you've been complaining about seems to suggest just the opposite. Darren's error is being raised because it is a Python complex being coerced. np.int64(np.complex64(10+20j)) raises no such error.
Unfortunately, NumPy seems to be a sort of step-child of Python, tolerated, but not fully accepted. There are a number of people who continue to use Matlab, despite all of its deficiencies, because it can at least be counted on to produce correct answers most of the time.
Except that you could never fully verify that it produces correct results, even if that was your desire. There are legitimate reasons for wanting to use Matlab (e.g. familiarity, because collaborators do, and for certain things it's still faster than the alternatives) but correctness of results isn't one of them. That said, people routinely let price tags influence their perceptions of worth. David
On Wed, Dec 9, 2009 at 3:51 AM, David Warde-Farley <dwf@cs.toronto.edu> wrote:
On 9-Dec-09, at 1:26 AM, Dr. Phillip M. Feldman wrote:
Unfortunately, NumPy seems to be a sort of step-child of Python, tolerated, but not fully accepted. There are a number of people who continue to use Matlab, despite all of its deficiencies, because it can at least be counted on to produce correct answers most of the time.
Except that you could never fully verify that it produces correct results, even if that was your desire.
There are legitimate reasons for wanting to use Matlab (e.g. familiarity, because collaborators do, and for certain things it's still faster than the alternatives) but correctness of results isn't one of them. That said, people routinely let price tags influence their perceptions of worth.
While I'm not going to argue in favor of Matlab, and think it's benefits are being over-stated, let's call a spade a spade. Silent downcasting of complex types to float is a *wart*. It's not sensible behavior, it's an implementation detail that smacks new users in the face. It's completely insensible to consider converting from complex to float in the same vein as a simple loss of precision from 64-bit to 32-bit. The following doesn't work: a = np.array(['bob', 'sarah']) b = np.arange(2.) b[:] = a --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /home/rmay/<ipython console> in <module>() ValueError: invalid literal for float(): bob Why doesn't that silently downcast the strings to 0.0 or something silly? Because that would be *stupid*. So why doesn't trying to stuff 3+4j into the array get the same error, because 3+4j is definitely not a float value either. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma
On Wed, Dec 9, 2009 at 9:46 AM, Ryan May <rmay31@gmail.com> wrote:
On Wed, Dec 9, 2009 at 3:51 AM, David Warde-Farley <dwf@cs.toronto.edu> wrote:
On 9-Dec-09, at 1:26 AM, Dr. Phillip M. Feldman wrote:
Unfortunately, NumPy seems to be a sort of step-child of Python, tolerated, but not fully accepted. There are a number of people who continue to use Matlab, despite all of its deficiencies, because it can at least be counted on to produce correct answers most of the time.
Except that you could never fully verify that it produces correct results, even if that was your desire.
There are legitimate reasons for wanting to use Matlab (e.g. familiarity, because collaborators do, and for certain things it's still faster than the alternatives) but correctness of results isn't one of them. That said, people routinely let price tags influence their perceptions of worth.
While I'm not going to argue in favor of Matlab, and think it's benefits are being over-stated, let's call a spade a spade. Silent downcasting of complex types to float is a *wart*. It's not sensible behavior, it's an implementation detail that smacks new users in the face. It's completely insensible to consider converting from complex to float in the same vein as a simple loss of precision from 64-bit to 32-bit. The following doesn't work:
a = np.array(['bob', 'sarah']) b = np.arange(2.) b[:] = a --------------------------------------------------------------------------- ValueError Traceback (most recent call last)
/home/rmay/<ipython console> in <module>()
ValueError: invalid literal for float(): bob
Why doesn't that silently downcast the strings to 0.0 or something silly? Because that would be *stupid*. So why doesn't trying to stuff 3+4j into the array get the same error, because 3+4j is definitely not a float value either.
Real numbers are a special case of complex, so I think the integer/float analogy is better. Numpy requires quite a bit more learning than programs like matlab and gauss with a more rigid type structure. And numpy has quite a few issues with "Is this a bug or a feature". numpy downcasting looks pretty consistent (for most parts) and it just one more thing to keep in mind like integer division and integer powers. Instead of requiring numpy to emit hundreds of warnings, I think it's better to properly unit test the code. For example, inspection and a test case showed pretty quickly that the way I tried to use scipy.integrate.quad with complex numbers didn't return the correct complex answer but only the correct real part. Compared to some questionable behavior with views and rearranging the axes with fancy indexing, I think the casting problem is easy to keep track of. Maybe we should start to collect these warts for a numpy 3000. Josef
Ryan
-- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Robert Kern-2 wrote:
<snip>
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
Robert Kern
This might be the time to recognize that that was a bad choice and reverse it. It is not clear to me why downcasting from complex to real sometimes raises an exception, and sometimes doesn't. Here are two examples where the behavior of NumPy is inexplicably different: Example #1: IPython 0.10 [on Py 2.5.4] [~]|1> z= zeros(3) [~]|2> z[0]= 1+1J TypeError: can't convert complex to float; use abs(z) Example #2: ### START OF CODE ### from numpy import * q = ones(2,dtype=complex)*(1 + 1J) r = zeros(2,dtype=float) r[:] = q print 'q = ',q print 'r = ',r ### END OF CODE ### [~]|9> run err q = [ 1.+1.j 1.+1.j] r = [ 1. 1.] [~]|10> In example #1, when we attempt to assign a complex value to a single element of a float array, an exception is triggered. In example #2, when we assign the entire array (actually, a slice), the imaginary part is silently discarded and we get a wrong result. At a minimum, this inconsistency needs to be cleared up. My preference would be that the programmer should have to explicitly downcast from complex to float, and that if he/she fails to do this, that an exception be triggered. -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
Robert Kern-2 wrote:
<snip>
Downcasting data is a necessary operation sometimes. We explicitly made a choice a long time ago to allow this.
Robert Kern
This might be the time to recognize that that was a bad choice and reverse it. It is not clear to me why downcasting from complex to real sometimes raises an exception, and sometimes doesn't. Here are two examples where the behavior of NumPy is inexplicably different: Example #1: IPython 0.10 [on Py 2.5.4] [~]|1> z= zeros(3) [~]|2> z[0]= 1+1J TypeError: can't convert complex to float; use abs(z) Example #2: ### START OF CODE ### from numpy import * q = ones(2,dtype=complex)*(1 + 1J) r = zeros(2,dtype=float) r[:] = q print 'q = ',q print 'r = ',r ### END OF CODE ### [~]|9> run err q = [ 1.+1.j 1.+1.j] r = [ 1. 1.] [~]|10> In example #1, when we attempt to assign a complex value to a single element of a float array, an exception is triggered. In example #2, when we assign the entire array (actually, a slice), the imaginary part is silently discarded and we get a wrong result. At a minimum, this inconsistency needs to be cleared up. My preference would be that the programmer should have to explicitly downcast from complex to float, and that if he/she fails to do this, that an exception be triggered. Dr. Phillip M. Feldman -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
On 7-Dec-09, at 11:13 PM, Dr. Phillip M. Feldman wrote:
Example #1: IPython 0.10 [on Py 2.5.4] [~]|1> z= zeros(3) [~]|2> z[0]= 1+1J
TypeError: can't convert complex to float; use abs(z)
The problem is that you're using Python's built-in complex type, and it responds to type coercion differently than NumPy types do. Calling float() on a Python complex will raise the exception. Calling float() on (for example) a numpy.complex64 will not. Notice what happens here: In [14]: z = zeros(3) In [15]: z[0] = complex64(1+1j) In [16]: z[0] Out[16]: 1.0
Example #2:
### START OF CODE ### from numpy import * q = ones(2,dtype=complex)*(1 + 1J) r = zeros(2,dtype=float) r[:] = q print 'q = ',q print 'r = ',r ### END OF CODE ###
Here, both operands are NumPy arrays. NumPy is in complete control of the situation, and it's well documented what it will do. I do agree that the behaviour in example #1 is mildly inconsistent, but such is the way with NumPy vs. Python scalars. They are mostly transparently intermingled, except when they're not.
At a minimum, this inconsistency needs to be cleared up. My preference would be that the programmer should have to explicitly downcast from complex to float, and that if he/she fails to do this, that an exception be triggered.
That would most likely break a *lot* of deployed code that depends on the implicit downcast behaviour. A less harmful solution (if a solution is warranted, which is for the Council of the Elders to decide) would be to treat the Python complex type as a special case, so that the .real attribute is accessed instead of trying to cast to float. David
At a minimum, this inconsistency needs to be cleared up. My preference would be that the programmer should have to explicitly downcast from complex to float, and that if he/she fails to do this, that an exception be triggered.
That would most likely break a *lot* of deployed code that depends on the implicit downcast behaviour. A less harmful solution (if a solution is warranted, which is for the Council of the Elders to decide) would be to treat the Python complex type as a special case, so that the .real attribute is accessed instead of trying to cast to float.
Except that the exception raised on downcast is the behavior we really want. We don't need python complex types introducing subtle bugs as well. I understand why we have the silent downcast from complex to float, but I consider it a wart, not a feature. I've lost hours tracking down bugs where I'm putting complex data from some routine into a new array (without specifying a dtype) ends up with the complex downcast silently to float64. The only reason you even notice it is because at the end you have incorrect answers. I know to look for it now, but for inexperienced users, it's a pain. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma
David Warde-Farley-2 wrote:
A less harmful solution (if a solution is warranted, which is for the Council of the Elders to decide) would be to treat the Python complex type as a special case, so that the .real attribute is accessed instead of trying to cast to float.
There are two even less harmful solutions: (1) Raise an exception. (2) Provide the user with a top-level flag to control whether the attempt to downcast a NumPy complex to a float should be handled by raising an exception, by throwing away the imaginary part, or by taking the magnitude. P.S. As things stand now, I do not regard NumPy as a reliable platform for scientific computing. -- View this message in context: http://old.nabble.com/Assigning-complex-values-to-a-real-array-tp22383353p26... Sent from the Numpy-discussion mailing list archive at Nabble.com.
On Tue, Dec 08, 2009 at 10:17:20AM -0800, Dr. Phillip M. Feldman wrote:
David Warde-Farley-2 wrote:
A less harmful solution (if a solution is warranted, which is for the Council of the Elders to decide) would be to treat the Python complex type as a special case, so that the .real attribute is accessed instead of trying to cast to float.
There are two even less harmful solutions: (1) Raise an exception.
This is not less harmful, since as I mentioned there is likely a lot of deployed code that is not expecting such exceptions. If such a change were to take place it would have to take place over several versions, where warnings are issued for a while (probably at least one stable release) before the feature being removed. Esoteric handling of ambiguous assignments may not speed adoption of NumPy, but monumental shifts in basic behaviour without any warning will make us even less friends. The best thing to do is probably to file an enhancement ticket on the bugtracker so that the issue doesn't get lost/forgotten.
(2) Provide the user with a top-level flag to control whether the attempt to downcast a NumPy complex to a float should be handled by raising an exception, by throwing away the imaginary part, or by taking the magnitude.
I'm not so sure that introducing more global state is looked fondly upon, but it'd be worth including this proposal in the ticket.
P.S. As things stand now, I do not regard NumPy as a reliable platform for scientific computing.
One man's bug is another's feature, I guess. I rarely use complex numbers and when I do I simply avoid this situation. David
On 12/08/09 02:32, David Warde-Farley wrote:
On 7-Dec-09, at 11:13 PM, Dr. Phillip M. Feldman wrote:
Example #1: IPython 0.10 [on Py 2.5.4] [~]|1> z= zeros(3) [~]|2> z[0]= 1+1J
TypeError: can't convert complex to float; use abs(z)
The problem is that you're using Python's built-in complex type, and it responds to type coercion differently than NumPy types do. Calling float() on a Python complex will raise the exception. Calling float() on (for example) a numpy.complex64 will not. Notice what happens here:
In [14]: z = zeros(3)
In [15]: z[0] = complex64(1+1j)
In [16]: z[0] Out[16]: 1.0
Example #2:
### START OF CODE ### from numpy import * q = ones(2,dtype=complex)*(1 + 1J) r = zeros(2,dtype=float) r[:] = q print 'q = ',q print 'r = ',r ### END OF CODE ###
Here, both operands are NumPy arrays. NumPy is in complete control of the situation, and it's well documented what it will do.
I do agree that the behaviour in example #1 is mildly inconsistent, but such is the way with NumPy vs. Python scalars. They are mostly transparently intermingled, except when they're not.
At a minimum, this inconsistency needs to be cleared up. My preference would be that the programmer should have to explicitly downcast from complex to float, and that if he/she fails to do this, that an exception be triggered.
That would most likely break a *lot* of deployed code that depends on the implicit downcast behaviour. A less harmful solution (if a solution is warranted, which is for the Council of the Elders to decide) would be to treat the Python complex type as a special case, so that the .real attribute is accessed instead of trying to cast to float.
I'm not sure how much code is actually relying on the implicit downcast, but I'd argue that it's bad programming anyways. It is really difficult to spot if you reviewing someone else's code. As others mentioned it's also a bitch to track down a bug that has been accidentally introduced by this behaviour. Jochen
participants (14)
-
Anne Archibald -
Charles R Harris -
Dag Sverre Seljebotn -
Darren Dale -
David Warde-Farley -
Dr. Phillip M. Feldman -
Jochen Schroeder -
josef.pktd＠gmail.com -
Pauli Virtanen -
Pauli Virtanen -
Robert Kern -
Ryan May -
Scott Ransom -
Stéfan van der Walt