On Fri, May 19, 2006 at 10:41:35AM +1000, Justin Warren wrote:
break existing code, ie: regression testing. I'm personally a big fan of this feature of test suites.
I'm a big fun of test suites as well. I only disagree with wasting time delaying the integration of a valid bugfix just because the unit test doesn't exist yet. I absolutely never said unit tests are a waste of time. For the last months I've always said that "unit tests" are _welcome_ at the top of the cpushare-twisted webpage.
Incidentally, those with slightly more CS knowledge know that it is possible, though by no means easy, to build a system that is provably correct. An investigation of the Z specification language may prove enlightening. I think you mean that a test suite proves that the system passes all the tests, not that it is bug free.
What I mean is that a test suite cannot prove the code is bug free. Nor that any bugfix is correct. If nothing else because the test suite may be buggy too. This is obvious.
Clearly a test suite is welcome and can only help, but its mandatory requirement for any change to the code sounds way excessive.
I perfectly know about formal demonstrations being possible too (I spoke about those matters for a long time last year in a completely different context) but they're not unit-tests (certainly not the ones you see in the twisted reposistory), so I didn't mention this to avoid further confusion. I doubt it's feasible to demonstrate Twisted bug free formally (to back my guess I remind you Alan Cox quote saying twisted is a 6m unauditable weirdness, I guess he was partly joking though).
My current worries are the troubles with poll, I worry about the lack of epoll, I worry about scaling in SMP with one thread per cpu. Those are the things that should be discussed instead of receiving emails from people about lack of unit tests for fixes that can be trivially verified by reading the code.
make it so. If you have benchmarks that demonstrate your claim, and importantly, the method used to generate them, then by all means share them with others. If your claims are true, this information will help in fixing the problem. Fixing these bugs is what we all want, after all.
About the benchmarks to make an example I posted some benchmarks here:
I used the klive homepage for it. You can reproduce yourself downloading it (all GPL):
Older versions uses web+nevow, newer uses web2+Cheetah (I don't remember exactly the time of the switch but it's easy to find with some diff).
However here the kind of the answers I got:
So I didn't post more benchmarks, nor I tried to produce an official benchmark that is easier to run than to install klive locally. Also note, quite a lot of the time of klive is spent in the database. So it's one of the worst possible benchmarks for web1+nevow vs web2+cheetah since only little time is spent for the rendering. I measured much higher html delivery speedups in other pages that weren't asking the db such cpu intensive queries.
If there is interest I can produce an official benchmark (that sounds much more useful than unit-tests for every bugfix). However I guess we'll have to follow the above advice and do it on the cpushare-twisted list. Also note, if you've a better place than cpushare-twisted for things that may not be welcome here, that's fine with me. I made up this cpushare-twisted things to fit my needs, only because I didn't find any other alternate place.