[Python-Dev] patch for review: unittest ImportError handling

[Chris Jerdonek]

On Wed, Apr 14, 2010 at 2:07 AM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> I'm still not convinced that this isn't a backwards incompatible change - up
> until now, however horrible it may be, TestLoader.loadTestsFromName only
> raised an AttributeError when it failed to load a test. Changing it to allow
> it propagate ImportError means that code catching errors by handling
> AttributeError will be potentially broken by the fix. I'm certainly happy to
> discuss this here though.

Thanks for your response. Michael.  To understand your position
better, would you view changing the AttributeError in *any* way an
incompatible change (e.g. changing just the error message), or is it
only changing the error type that you view as backwards incompatible?

The reason I ask is that I think it's the change in what the error
contains (e.g. the error message and stack trace) that is the more
important part of this change -- rather than whether that information
be wrapped in an ImportError versus an AttributeError.  It is the
information rather than the particular error type that will assist an
end-developer in diagnosing future unit-test failures.

> An alternative fix would be for a new API and deprecation of
> loadTestFromName. A new API could return a placeholder test that raises the
> original error when run - that way individual errors don't break the test
> collection phase but are still reported. This *could* be added to
> loadTestsFromName with an optional argument.

I'm not opposed to adding a new API, but I think it would be valuable
if we could find a way for people to enjoy the benefit of not having
the stack trace swallowed -- without having to change their code.  I'm
currently part of a project where the current behavior makes it harder
to track down unit test failures.  I imagine many developers are in a
similar situation.  This seems to be a bug, and such developers may be
in a position where they can't change how their unit tests are run,
but they're nevertheless stuck diagnosing the failures.

I'm writing on the assumption that there is a way to preserve the
stack trace and error message of an ImportError while re-raising it as
an AttributeError.

> By the way, in general please don't assign unittest bugs *away* from me on
> the tracker.

Will do.  I hadn't come across any guidance re: assigning in the
development workflow documentation.  Maybe we should add a cautionary
note about this in the documentation somewhere.  In general, do all
reports stay assigned to the maintainer (if a maintainer exists), or
is it a per-maintainer preference on how that is handled?

Thanks,
--Chris