setuptools tracker "needpatch" keyword, tests, etc.
Hello all. I had some spare time the other day and went through the tracker, reclassifying a few things and marking some with a "needpatch" keyword. The ones marked "needpatch" vary from things I have no idea what to do with, to ones where I've practically spelled out the needed patch in the tracker. Issues with patches that passed my initial review have been marked "in-progress"; these could use some testing before check-in. At this point, I haven't had an opportunity to review the results of the test sprint that was done; if somebody could throw that up as a patch on the tracker, or at least repost a link to where I can find that stuff, that would help. All of this is with an eye to releasing an 0.6 final in the near future; sometime this month I'd guess/hope, depending on how my other projects are going.
Phillip J. Eby wrote:
Hello all. I had some spare time the other day and went through the tracker, reclassifying a few things and marking some with a "needpatch" keyword. The ones marked "needpatch" vary from things I have no idea what to do with, to ones where I've practically spelled out the needed patch in the tracker.
Issues with patches that passed my initial review have been marked "in-progress"; these could use some testing before check-in. At this point, I haven't had an opportunity to review the results of the test sprint that was done; if somebody could throw that up as a patch on the tracker, or at least repost a link to where I can find that stuff, that would help. The work done on the test sprint is hosted in this bzr branch. We wanted to get your feedback on what had been done so far to make sure we were heading in the right direction.
https://code.launchpad.net/~setuptools/setuptools-test/main
All of this is with an eye to releasing an 0.6 final in the near future; sometime this month I'd guess/hope, depending on how my other projects are going.
That would be awesome. To reach this goal, it would really help to get more tests written so that patches can be tested quicker and with more certainty that they haven't broken anything. -- Chris Galvan
At 10:32 AM 8/8/2008 -0500, Chris Galvan wrote:
Phillip J. Eby wrote:
Hello all. I had some spare time the other day and went through the tracker, reclassifying a few things and marking some with a "needpatch" keyword. The ones marked "needpatch" vary from things I have no idea what to do with, to ones where I've practically spelled out the needed patch in the tracker.
Issues with patches that passed my initial review have been marked "in-progress"; these could use some testing before check-in. At this point, I haven't had an opportunity to review the results of the test sprint that was done; if somebody could throw that up as a patch on the tracker, or at least repost a link to where I can find that stuff, that would help. The work done on the test sprint is hosted in this bzr branch. We wanted to get your feedback on what had been done so far to make sure we were heading in the right direction.
My initial reaction is that it's off to a good start, but the tests themselves seem rather shallow; more like "smoke tests" (i.e., turn it on and see if smoke comes out) than functional tests. I'm thinking it might help to use the setuptools.sandbox facility to log files created, deleted, modified, etc. by the process. That would probably be a better test of what has/hasn't been done than using ellipses on the logs, which is order-dependent as well as having the ability to skip lines where the wrong thing is being done, etc. The way things are being done now, they probably won't be able to test some of the things that are most likely to break (i.e., the complexities of easy_install). (Probably in order to do that I'll need to add a new sandboxing class that creates a "mock" filesystem and allows before/after expectations to be set.)
Phillip J. Eby wrote:
At 10:32 AM 8/8/2008 -0500, Chris Galvan wrote:
Phillip J. Eby wrote:
Hello all. I had some spare time the other day and went through the tracker, reclassifying a few things and marking some with a "needpatch" keyword. The ones marked "needpatch" vary from things I have no idea what to do with, to ones where I've practically spelled out the needed patch in the tracker.
Issues with patches that passed my initial review have been marked "in-progress"; these could use some testing before check-in. At this point, I haven't had an opportunity to review the results of the test sprint that was done; if somebody could throw that up as a patch on the tracker, or at least repost a link to where I can find that stuff, that would help. The work done on the test sprint is hosted in this bzr branch. We wanted to get your feedback on what had been done so far to make sure we were heading in the right direction.
My initial reaction is that it's off to a good start, but the tests themselves seem rather shallow; more like "smoke tests" (i.e., turn it on and see if smoke comes out) than functional tests.
I'm thinking it might help to use the setuptools.sandbox facility to log files created, deleted, modified, etc. by the process. That would probably be a better test of what has/hasn't been done than using ellipses on the logs, which is order-dependent as well as having the ability to skip lines where the wrong thing is being done, etc. The way things are being done now, they probably won't be able to test some of the things that are most likely to break (i.e., the complexities of easy_install).
(Probably in order to do that I'll need to add a new sandboxing class that creates a "mock" filesystem and allows before/after expectations to be set.)
You could also use ScriptTest: http://pythonpaste.org/scripttest/ -- it doesn't make any attempt to mock anything out, but it does keep track of what a command does. For testing PoachEggs (https://svn.openplans.org/svn/PoachEggs/trunk/poacheggs-tests) I'm creating a scratch virtualenv for the test, then running things inside there. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
On Fri, Aug 8, 2008 at 6:28 PM, Phillip J. Eby <pje@telecommunity.com>wrote:
...
My initial reaction is that it's off to a good start, but the tests themselves seem rather shallow; more like "smoke tests" (i.e., turn it on and see if smoke comes out) than functional tests.
Right, they are not detailed at all at this point
I'm thinking it might help to use the setuptools.sandbox facility to log files created, deleted, modified, etc. by the process. That would probably be a better test of what has/hasn't been done than using ellipses on the logs, which is order-dependent as well as having the ability to skip lines where the wrong thing is being done, etc. The way things are being done now, they probably won't be able to test some of the things that are most likely to break (i.e., the complexities of easy_install).
(Probably in order to do that I'll need to add a new sandboxing class that creates a "mock" filesystem and allows before/after expectations to be set.)
Here's a proposal that could be done in a new 'test' sprint I guess : - add something in the sandbox to record what is being done underneath (using for example what Ian has suggested - ScriptTest) - change our doctests so they actually use the recorded info : files created, removed, modified + return code maybe (but not the stdout which vary too much from one system to another) At least Chris Galvan and I are interested in helping in this. Tarek
participants (4)
-
Chris Galvan -
Ian Bicking -
Phillip J. Eby -
Tarek Ziadé