[Distutils] setuptools tracker "needpatch" keyword, tests, etc.

[Phillip J. Eby]

At 10:32 AM 8/8/2008 -0500, Chris Galvan wrote:
>Phillip J. Eby wrote:
>>Hello all.  I had some spare time the other day and went through 
>>the tracker, reclassifying a few things and marking some with a 
>>"needpatch" keyword.  The ones marked "needpatch" vary from things 
>>I have no idea what to do with, to ones where I've practically 
>>spelled out the needed patch in the tracker.
>>
>>Issues with patches that passed my initial review have been marked 
>>"in-progress"; these could use some testing before check-in.  At 
>>this point, I haven't had an opportunity to review the results of 
>>the test sprint that was done; if somebody could throw that up as a 
>>patch on the tracker, or at least repost a link to where I can find 
>>that stuff, that would help.
>The work done on the test sprint is hosted in this bzr branch.  We 
>wanted to get your feedback on what had been done so far to make 
>sure we were heading in the right direction.
>
>https://code.launchpad.net/~setuptools/setuptools-test/main

My initial reaction is that it's off to a good start, but the tests 
themselves seem rather shallow; more like "smoke tests" (i.e., turn 
it on and see if smoke comes out) than functional tests.

I'm thinking it might help to use the setuptools.sandbox facility to 
log files created, deleted, modified, etc. by the process.  That 
would probably be a better test of what has/hasn't been done than 
using ellipses on the logs, which is order-dependent as well as 
having the ability to skip lines where the wrong thing is being done, 
etc.  The way things are being done now, they probably won't be able 
to test some of the things that are most likely to break (i.e., the 
complexities of easy_install).

(Probably in order to do that I'll need to add a new sandboxing class 
that creates a "mock" filesystem and allows before/after expectations 
to be set.)