Phillip J. Eby wrote:
At 10:32 AM 8/8/2008 -0500, Chris Galvan wrote:
Phillip J. Eby wrote:
Hello all. I had some spare time the other day and went through the tracker, reclassifying a few things and marking some with a "needpatch" keyword. The ones marked "needpatch" vary from things I have no idea what to do with, to ones where I've practically spelled out the needed patch in the tracker.
Issues with patches that passed my initial review have been marked "in-progress"; these could use some testing before check-in. At this point, I haven't had an opportunity to review the results of the test sprint that was done; if somebody could throw that up as a patch on the tracker, or at least repost a link to where I can find that stuff, that would help. The work done on the test sprint is hosted in this bzr branch. We wanted to get your feedback on what had been done so far to make sure we were heading in the right direction.
My initial reaction is that it's off to a good start, but the tests themselves seem rather shallow; more like "smoke tests" (i.e., turn it on and see if smoke comes out) than functional tests.
I'm thinking it might help to use the setuptools.sandbox facility to log files created, deleted, modified, etc. by the process. That would probably be a better test of what has/hasn't been done than using ellipses on the logs, which is order-dependent as well as having the ability to skip lines where the wrong thing is being done, etc. The way things are being done now, they probably won't be able to test some of the things that are most likely to break (i.e., the complexities of easy_install).
(Probably in order to do that I'll need to add a new sandboxing class that creates a "mock" filesystem and allows before/after expectations to be set.)
You could also use ScriptTest: http://pythonpaste.org/scripttest/ -- it doesn't make any attempt to mock anything out, but it does keep track of what a command does. For testing PoachEggs (https://svn.openplans.org/svn/PoachEggs/trunk/poacheggs-tests) I'm creating a scratch virtualenv for the test, then running things inside there. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org