[Chicago] The current state of Testing Stuff

[Brian Ray]

This is an intentionally vague topic regarding testing from
acceptability to unit tests. I am suggesting things have changed and
perhaps it is time to review testing methodologies.  I am sure many of
you have dealt with this change already. At least this could be an
interesting topic for some I hope.

I recall in 2005 ChiPy had a plethora of talks on unit testing, Mock
Objects and fitness testing. Later we talked a lot about nose tests
and runner tools. More recently we had talks on great tools like tox
but it still included things like hudson to run. Somewhere in between
we talked about Selenium.  There was a link (oh here it is) that lists
tools http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy . I am
not sure this test is exhaustive.

So here comes my question.... What did we learn from these different
efforts in testing; what has changed in testing; what is the best
modern method to implement large scale testing that covers the whole
stack? How much testing in development should be made available for
testing in QA and acceptability?

What is the point of writing test on something that will never fail?
Have testing tools changed to become more restful? Has anyone ever had
a test failure from an automated suite that actually pointed to
something useful? It seems testing the smaller lower level items can
be covered well with Unit Tests. Higher and middle level items perhaps
are covered well by fitness and maybe browser level? How do we
automate useful tests? How does one approach testing more complicated
things like events and threads?  Now things are becoming so mashed...
what are people doing to test interpolation with thinks like web
services that someone else maintains?

I guess I am also looking for life cycle configuration ideas.  I could
see someone saying something like this:  "We write UnitTests we run
from Nose for TDD, for acceptability in development we use urllib2 to
test RESTFul stuff, and then on top of that we use Selenium for
browser testing..."

-- 

Brian Ray