[Python-3000] Separating out CPython and core Python tests

[Brett Cannon]

On 4/11/06, John J Lee <jjl at pobox.com> wrote:
> On Tue, 11 Apr 2006, Michael Chermside wrote:
> [...]
> > There's a related change that I would love to see. Currently, we
> > have a large number of unit tests, but we have made no attempt to
> > distinguish between python language tests, and CPython implementation
> > tests. If we made such a distinction, then PyPy, Jython, Iron Python,
> > and perhaps others would benefit. Python 3000 might be a good time
> > to introduce the distinction.
> >
> > In fact, I don't think "we" even need to do the hard work (where "we"
> > is defined as the CPython developers). I am confident that if the
> > CPython team offered to make such a split, then either the PyPy or
> > Jython developers (maybe both) would happily provide us with a list of
> > which tests they thought were core language tests and which were
> > CPython-specific. Guido could review and pronounce on any controversial
> [...]
>
> +1, but why wait for Python 3000?
>
> Maybe add a "resource name" to regrtest.py, so e.g. regrtest.py-u -cpython
> ran everything except the CPython-specific tests?
>
>   RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
> -                  'decimal', 'compiler', 'subprocess', 'urlfetch')
> +                  'decimal', 'compiler', 'subprocess', 'urlfetch',
> +                  'cpython')
>
>
> (tests then use test_support.is_resouce_enabled('cpython') when building
> test suites)

But many module tests have mixed spec/implementation tests.  Wouldn't
a decorator be a better solution?  We keep talking about adding
decorators that signal test known to fail and we could probably stand
to flag tests known to leak (all of this could lead to a list of tests
with such decorators at the end of regrtest so we are constantly
reminded where things are broken).

-Brett