On 4 Mar 2013, at 06:26, Robert Collins <robertc@robertcollins.net> wrote:
On 4 March 2013 18:54, Guido van Rossum <guido@python.org> wrote:
On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins <robertc@robertcollins.net> wrote:
I'd like to talk about overhauling - not tweaking, overhauling - the standard library testing facilities.
That seems like too big a topic and too vague a description to discuss usefully. Perhaps you have a specific proposal? Or at least just a use case that's poorly covered?
I have both - I have a draft implementation for a new test result API (and forwards and backwards compat code etc), and use cases that drive it. I started a thread here - http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html , with blog posts https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-proto... https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/ https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-st... https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/
They are focused on subunit, but much of subunit's friction has been due to issues encountered from the stdlibrary TestResult API - in particular three things: - the single-active-test model that the current API (or at least implementation) has. - the expectation that all test outcomes will originate from the same interpreter (or something with a live traceback object) - the inability to supply details about errors other than the exception
All of which start to bite rather deep when working on massively parallel test environments.
It is of course possible for subunit and related tools to run their own implementation, but it seems ideal to me to have a common API which regular unittest, nose, py.test and others can all agree on and use : better reuse for pretty printers, GUI displays and the like depend on some common API.
TBH, your choice of words is ambiguous -- are you interested in overhauling the facilities for testing *of* the standard library (i.e. the 'test' package), or the testing facilities *provided by* the standard library (i.e. the unittest module)?
Sorry! Testing facilities provided by the standard library. They should naturally facilitate testing of the standard library too.
We can certainly talk about it - although as Guido says, something specific may be easier to have a useful discussion about. Reading through your blog articles it seemed like a whole lot of subunit context was required to understand the specific proposal you're making for the TestResult. It also *seems* like you're redesigning the TestResult for a single use case (distributed testing) with an api that looks quite "odd" for anything that isn't that use case. I'd rather see how we can make the TestResult play *better* with those requirements. That discussion probably belongs in another thread - or at the summit. Michael
-Rob
-- --Guido van Rossum (python.org/~guido)
-- Robert Collins <rbtcollins@hp.com> Distinguished Technologist HP Cloud Services _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html