[Distutils] setuptools tracker "needpatch" keyword, tests, etc.

[Ian Bicking]

Phillip J. Eby wrote:
> At 10:32 AM 8/8/2008 -0500, Chris Galvan wrote:
>> Phillip J. Eby wrote:
>>> Hello all.  I had some spare time the other day and went through the 
>>> tracker, reclassifying a few things and marking some with a 
>>> "needpatch" keyword.  The ones marked "needpatch" vary from things I 
>>> have no idea what to do with, to ones where I've practically spelled 
>>> out the needed patch in the tracker.
>>>
>>> Issues with patches that passed my initial review have been marked 
>>> "in-progress"; these could use some testing before check-in.  At this 
>>> point, I haven't had an opportunity to review the results of the test 
>>> sprint that was done; if somebody could throw that up as a patch on 
>>> the tracker, or at least repost a link to where I can find that 
>>> stuff, that would help.
>> The work done on the test sprint is hosted in this bzr branch.  We 
>> wanted to get your feedback on what had been done so far to make sure 
>> we were heading in the right direction.
>>
>> https://code.launchpad.net/~setuptools/setuptools-test/main
> 
> My initial reaction is that it's off to a good start, but the tests 
> themselves seem rather shallow; more like "smoke tests" (i.e., turn it 
> on and see if smoke comes out) than functional tests.
> 
> I'm thinking it might help to use the setuptools.sandbox facility to log 
> files created, deleted, modified, etc. by the process.  That would 
> probably be a better test of what has/hasn't been done than using 
> ellipses on the logs, which is order-dependent as well as having the 
> ability to skip lines where the wrong thing is being done, etc.  The way 
> things are being done now, they probably won't be able to test some of 
> the things that are most likely to break (i.e., the complexities of 
> easy_install).
> 
> (Probably in order to do that I'll need to add a new sandboxing class 
> that creates a "mock" filesystem and allows before/after expectations to 
> be set.)

You could also use ScriptTest: http://pythonpaste.org/scripttest/ -- it 
doesn't make any attempt to mock anything out, but it does keep track of 
what a command does.  For testing PoachEggs 
(https://svn.openplans.org/svn/PoachEggs/trunk/poacheggs-tests) I'm 
creating a scratch virtualenv for the test, then running things inside 
there.

-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org