[Python-Dev] [python-committers] (Windows) buildbots on 3.x

[Michael Foord]

On 04/08/2010 18:53, "Martin v. Löwis" wrote:
>>>> It happens when running test_smtplib before test_smtpb:
>>>>          
>>> Nice! How did you work that out? I'd like to learn how to diagnose
>>> this sort of thing, because it seems to come up a lot, and I'm not
>>> much use at the moment :-)
>>>        
>> I simply tried to run test_smtplib before test_smtpd.
>>      
> A more deterministic (and more tedious) way is this: if you
> suspect that some failure might be due to the order of test cases,
> take a build log from the build bot where it fails, and run the tests
> in the exact same order. See whether the problem reproduces.
>
> If it does, drop tests from the test sequence until you end up with
> the minimum number of tests that you need to run in sequence (and yes,
> I had interworkings of three test modules at some point).
>
>    

A colleague in a previous job wrote a tool that would repeat a test 
sequence using a binary chop to find test interactions... It could take 
a while to run, but was usually faster than manually trying to find them 
(assuming educated guessing had already failed).

Michael

> Of course, educated guessing can accelerate the process.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.