[Python-Dev] OS X buildbots: why am I skipping these tests?

[Brett Cannon]

On Wed, Jun 30, 2010 at 15:20, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Jul 1, 2010 at 7:55 AM, Brett Cannon <brett at python.org> wrote:
>> So it isn't that it's "unexpected", it's that a dependency is missing.
>> So it seems the terminology needs to get tweaked.
>
> More that the phrase "expected skip" isn't clearly defined and people
> sometimes guess wrong as to what it means. As Martin pointed out,
> there are two possible meanings: "will never work on this OS" and
> "won't work with just the base OS install". Currently, the "expected
> skip" list is based purely on the former, but developers occasionally
> interpret it as the latter (as Bill did in this case).
>
> I will note that the first list is much easier to keep up to date,
> since the latter may vary significantly based on vendor decisions as
> to what they install by default (a fairly significant factor in the
> Linux and *BSD worlds).
>
> Adding "(Were all optional modules built successfully?)" to the end of
> the "skips were unexpected" line in the regrtest output may be enough
> to eliminate the confusion.

Probably. So I would still want to shift the test-specific info into
the tests instead of regrtest and raise a subclass od SkippedTest (or
whatever the name of the exception is) to signify that there is a
difference.

This would also do away with the possibility of having a test get
silently skipped by an ImportError even though the module should
definitely be available (didn't that bite you once, or was that
someone else?).