[Python-bugs-list] [ python-Bugs-665761 ] reduce() masks exception

[SourceForge.net]

Bugs item #665761, was opened at 2003-01-10 15:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665761&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 3
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Nobody/Anonymous (nobody)
Summary: reduce() masks exception

Initial Comment:
In the following test script
-----
class Test:
   def __iter__(self):
      raise IOError

reduce(lambda x,y: x+y, Test())
-----
the real IOError exception is masked, i.e. the traceback is
-----
Traceback (most recent call last):
  File "test.py", line 5, in ?
    reduce(lambda x,y: x+y, Test())
TypeError: reduce() arg 2 must support iteration
-----
but IMHO should be
-----
Traceback (most recent call last):
  File "test.py", line 3, in ?
          raise IOError
IOError
-----
This can be fixed by removing the
PyErr_SetString(PyExc_TypeError, "reduce() arg 2 must
support iteration") call in
bltinmodule.c/buildtin_reduce().


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-30 12:29

Message:
Logged In: YES 
user_id=89016

Trapping only TypeError won't help:

class LazyFile:
   def __init__(self, name, mode="r"):
      self.name = name
      self.mode = mode
   def __iter__(self):
      return open(self.name, self.mode)
  
import operator
  
f = LazyFile(42)
  
s = reduce(operator.add, f)

Here the open call will raise a TypeError, that is totally
unrelated to the iterator protocol.

The cleanest solution would really be exception chaining.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-29 00:59

Message:
Logged In: YES 
user_id=80475

One way to mitigate the information loss is to mimic the 
style in zip() which only traps TypeErrors but passes 
through things like the IOError in your original example.  

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-28 21:38

Message:
Logged In: YES 
user_id=89016

Attached is a patch that fixes reduce(), map() and zip().
Unfortunately this loses the information about which
argument triggered the exception (in map() and zip())

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 05:00

Message:
Logged In: YES 
user_id=80475

There's a lot of this going around.  map() and zip() have 
the same issue.  I recommend fixing them all.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-13 14:18

Message:
Logged In: YES 
user_id=89016

The point is that Python/bltinmodule.c/builtin_reduce()
masks the error returned from PyObject_GetIter(). Errors
from PyIter_Next() are not masked.

What about the following example:

class LazyFile:
   def __init__(self, name, mode="r"):
      self.name = name
      self.mode = mode
   def __iter__(self):
      return open(self.name, self.mode)
  
import operator
  
f = LazyFile("does not exist")
  
s = reduce(operator.add, f)

LazyFile *does* support iteration, but the underlying
problem of the non existing file is masked. Removing the
call PyErr_SetString(PyExc_TypeError, "reduce() arg 2 must
support iteration"); in builtin_reduce(), will produce the
original exception "IOError: [Errno 2] No such file or
directory: 'does not exist'" and when the second argument is
not iteratable, the original exception is just as good:

>>> reduce(lambda x,y: x+y, 42)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: iteration over non-sequence


----------------------------------------------------------------------

Comment By: Jp Calderone (kuran)
Date: 2003-01-11 18:08

Message:
Logged In: YES 
user_id=366566

the __iter__ method is supposed to return an object that
defines a 'next' method.  The returned object is the one
used for iteration, not the original.  So I believe the
error message is correct - Test does not support iteration.
 If you change the code to:

>>> class test:
...   def __iter__(self):
...     return self
...   def next(self):
...     raise IOError
... 
>>> reduce(operator.add, test())

You get the expected result...

Traceback (most recent call last):
&nbsp;&nbsp;File "<stdin>", line 1, in ?
&nbsp;&nbsp;File "<stdin>", line 5, in next
IOError


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665761&group_id=5470