Re: [pytest-dev] [pytest-rerunfailures] pytest-rerunfailures not using fixtures on reruns (#10)
Hi Holger and other py.test mavens, Bob has reported a problem with my py.test plugin pytest-rerunfailures [1] not re-running the setup before rerunning the test. Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup, the test, and the teardown. [1] https://github.com/klrmn/pytest-rerunfailures [2] https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug... [3] https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a... I haven't taken the hours needed to get my head fully into py.test plugin development mode, but I'm not sure I can implement a fix at my layer. I'm hoping someone here will have some insight. Thanks, -Leah On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg <notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn <https://github.com/klrmn>?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub<https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...> .
Hi Leah, Bob, On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote:
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin pytest-rerunfailures [1] not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup, the test, and the teardown.
[1] https://github.com/klrmn/pytest-rerunfailures [2] https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug... [3] https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a...
I haven't taken the hours needed to get my head fully into py.test plugin development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with pip install -i http://pypi.testrun.org -U pytest which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have problems please try to send a minimal test file which shows undesired behaviour. A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item. best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg <notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn <https://github.com/klrmn>?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub<https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...> .
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
Hey Holger, Thank you for the timely patch. I haven't seen Bob online today, but hopefully he'll be able to test it very soon. A word of warning: your calling of runtestprotocol() is not quite right
and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
I agree that that would be ideal, but it only re-runs the test if it fails, so I would have to know the future in order to know what value to send nextitem=. Since hopefully most tests pass the first try, sending nextitem instead of item seems to be the better answer. Thanks! -Leah On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <holger@merlinux.eu> wrote:
Hi Leah, Bob,
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin pytest-rerunfailures [1] not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: the
test, and the teardown.
https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug...
[3]
https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a...
I haven't taken the hours needed to get my head fully into py.test plugin development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with
pip install -i http://pypi.testrun.org -U pytest
which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have problems please try to send a minimal test file which shows undesired behaviour.
A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn <https://github.com/klrmn ?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub< https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...
.
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
Hey Leah, On Mon, Apr 22, 2013 at 15:08 -0700, Leah Klearman wrote:
Hey Holger,
Thank you for the timely patch. I haven't seen Bob online today, but hopefully he'll be able to test it very soon.
A word of warning: your calling of runtestprotocol() is not quite right
and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
I agree that that would be ideal, but it only re-runs the test if it fails, so I would have to know the future in order to know what value to send nextitem=. Since hopefully most tests pass the first try, sending nextitem instead of item seems to be the better answer.
FYI pytest perform teardown/finalization according to nextitem: if your next item uses different fixtures, the "item" ones may be torn down (it also depends on the caching scope of the fixtures). Do you generally want to re-run the setup or could you also settle on just re-running the "call" phase of a test? The latter one would be easier and probably not disrupt fixture mechanics. best, holger
Thanks! -Leah
On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <holger@merlinux.eu> wrote:
Hi Leah, Bob,
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin pytest-rerunfailures [1] not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: the
test, and the teardown.
https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug...
[3]
https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a...
I haven't taken the hours needed to get my head fully into py.test plugin development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with
pip install -i http://pypi.testrun.org -U pytest
which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have problems please try to send a minimal test file which shows undesired behaviour.
A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn <https://github.com/klrmn ?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub< https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...
.
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
Holger, The only reason I can think of why py.test would perform teardown / finalization differently depending on the nextitem is if class, module, and package level fixtures are being used. If only method level fixtures are used, then they should always get created and always get torn down for each test case. Is that not how it works? We can't re-run just the 'call' phase because the failure of the previous 'call' phase may have left behind a partial situation (see automated testing before we had setup and teardown and fixtures). -Leah On Tue, Apr 23, 2013 at 12:16 AM, holger krekel <holger@merlinux.eu> wrote:
Hey Leah,
On Mon, Apr 22, 2013 at 15:08 -0700, Leah Klearman wrote:
Hey Holger,
Thank you for the timely patch. I haven't seen Bob online today, but hopefully he'll be able to test it very soon.
A word of warning: your calling of runtestprotocol() is not quite right
and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
I agree that that would be ideal, but it only re-runs the test if it fails, so I would have to know the future in order to know what value to send nextitem=. Since hopefully most tests pass the first try, sending nextitem instead of item seems to be the better answer.
FYI pytest perform teardown/finalization according to nextitem: if your next item uses different fixtures, the "item" ones may be torn down (it also depends on the caching scope of the fixtures).
Do you generally want to re-run the setup or could you also settle on just re-running the "call" phase of a test? The latter one would be easier and probably not disrupt fixture mechanics.
best, holger
Thanks! -Leah
On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <holger@merlinux.eu> wrote:
Hi Leah, Bob,
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin
not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: pytest-rerunfailures [1] the
test, and the teardown.
https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug...
[3]
I haven't taken the hours needed to get my head fully into py.test
https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a... plugin
development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with
pip install -i http://pypi.testrun.org -U pytest
which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have problems please try to send a minimal test file which shows undesired behaviour.
A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn < https://github.com/klrmn ?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub<
https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...
.
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
On Tue, Apr 23, 2013 at 09:56 -0700, Leah Klearman wrote:
Holger,
The only reason I can think of why py.test would perform teardown / finalization differently depending on the nextitem is if class, module, and package level fixtures are being used.
Yes, i tried to say something equivalent in my previous mail.
If only method level fixtures are used, then they should always get created and always get torn down for each test case. Is that not how it works?
Effectively it should be like this and probably independently of the value of nextitem. If you find issues let us know.
We can't re-run just the 'call' phase because the failure of the previous 'call' phase may have left behind a partial situation (see automated testing before we had setup and teardown and fixtures).
Sure, good point. cheers, holger
-Leah
On Tue, Apr 23, 2013 at 12:16 AM, holger krekel <holger@merlinux.eu> wrote:
Hey Leah,
On Mon, Apr 22, 2013 at 15:08 -0700, Leah Klearman wrote:
Hey Holger,
Thank you for the timely patch. I haven't seen Bob online today, but hopefully he'll be able to test it very soon.
A word of warning: your calling of runtestprotocol() is not quite right
and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
I agree that that would be ideal, but it only re-runs the test if it fails, so I would have to know the future in order to know what value to send nextitem=. Since hopefully most tests pass the first try, sending nextitem instead of item seems to be the better answer.
FYI pytest perform teardown/finalization according to nextitem: if your next item uses different fixtures, the "item" ones may be torn down (it also depends on the caching scope of the fixtures).
Do you generally want to re-run the setup or could you also settle on just re-running the "call" phase of a test? The latter one would be easier and probably not disrupt fixture mechanics.
best, holger
Thanks! -Leah
On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <holger@merlinux.eu> wrote:
Hi Leah, Bob,
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin
not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: pytest-rerunfailures [1] the
test, and the teardown.
https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug...
[3]
I haven't taken the hours needed to get my head fully into py.test
https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a... plugin
development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with
pip install -i http://pypi.testrun.org -U pytest
which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have problems please try to send a minimal test file which shows undesired behaviour.
A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn < https://github.com/klrmn ?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub<
https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...
.
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
Hi Holger, Bob appears to be on vacation, so I asked Stephen to run the job in it's natural environment. The run with the dev version of pytest and my plugin enabled had the same two failures as the project had with the previous release of pytest and no plugin. An analysis of whether the failures actually act differently will have to await Bob returning from vacation. For our reference, the jenkins config: http://pastie.org/7705026 Wish I could have given you better feedback on the patch. Please use your discretion as to whether or not to include it in the upcoming release. Thanks for your help. -Leah On Tue, Apr 23, 2013 at 1:30 PM, holger krekel <holger@merlinux.eu> wrote:
On Tue, Apr 23, 2013 at 09:56 -0700, Leah Klearman wrote:
Holger,
The only reason I can think of why py.test would perform teardown / finalization differently depending on the nextitem is if class, module, and package level fixtures are being used.
Yes, i tried to say something equivalent in my previous mail.
If only method level fixtures are used, then they should always get created and always get torn down for each test case. Is that not how it works?
Effectively it should be like this and probably independently of the value of nextitem. If you find issues let us know.
We can't re-run just the 'call' phase because the failure of the previous 'call' phase may have left behind a partial situation (see automated testing before we had setup and teardown and fixtures).
Sure, good point.
cheers, holger
-Leah
On Tue, Apr 23, 2013 at 12:16 AM, holger krekel <holger@merlinux.eu> wrote:
Hey Leah,
Hey Holger,
Thank you for the timely patch. I haven't seen Bob online today, but hopefully he'll be able to test it very soon.
A word of warning: your calling of runtestprotocol() is not quite right
and might lead to problems. "nextitem" should really be the item
On Mon, Apr 22, 2013 at 15:08 -0700, Leah Klearman wrote: that
is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
I agree that that would be ideal, but it only re-runs the test if it fails, so I would have to know the future in order to know what value to send nextitem=. Since hopefully most tests pass the first try, sending nextitem instead of item seems to be the better answer.
FYI pytest perform teardown/finalization according to nextitem: if your next item uses different fixtures, the "item" ones may be torn down (it also depends on the caching scope of the fixtures).
Do you generally want to re-run the setup or could you also settle on just re-running the "call" phase of a test? The latter one would be easier and probably not disrupt fixture mechanics.
best, holger
Thanks! -Leah
On Mon, Apr 22, 2013 at 1:27 AM, holger krekel <holger@merlinux.eu> wrote:
Hi Leah, Bob,
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin
not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: pytest-rerunfailures [1] the
test, and the teardown.
https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug...
[3]
I haven't taken the hours needed to get my head fully into
plugin
development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with
pip install -i http://pypi.testrun.org -U pytest
which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have
please try to send a minimal test file which shows undesired behaviour.
A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item
https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a... py.test problems that
is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < notifications@github.com>wrote:
> I just verified this behaviour myself with a simple test [1]. I see it > with both funcargs and fixtures, but I'm not sure if it has to do with the > plugin, or the way py.test works. It does inject the value into the test > method, but it doesn't rerun the fixture, so it seems like it is caching > the first run of the fixture and using that on subsequent runs. > > I'm not sure if this is something that the plugin can have any effect on, > or if it's just the way fixtures work. It is specified for this fixture > that it is scope='function', and perhaps py.test makes that happen by > checking the function name, which is, of course, the same for each run. I > did try removing the scope argument from the fixture but that had no > effect. > > Do you have any thoughts about this, @klrmn < https://github.com/klrmn ? > > [1] https://gist.github.com/bobsilverberg/5385035 > > — > Reply to this email directly or view it on GitHub<
https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...
> . >
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
Hi Holger, Sorry to take so ridiculously long to respond to this. I have tried my reduced test case with pytest 2.3.5 and the problem does seem to be fixed. Thanks for your help, Bob On Mon, Apr 22, 2013 at 4:27 AM, holger krekel <holger@merlinux.eu> wrote:
Hi Leah, Bob,
Hi Holger and other py.test mavens,
Bob has reported a problem with my py.test plugin pytest-rerunfailures [1] not re-running the setup before rerunning the test.
Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup,
On Fri, Apr 19, 2013 at 20:55 -0700, Leah Klearman wrote: the
test, and the teardown.
https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plug...
[3]
https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39a...
I haven't taken the hours needed to get my head fully into py.test plugin development mode, but I'm not sure I can implement a fix at my layer.
I'm hoping someone here will have some insight.
It's a bit intricate. A function item keeps around some fixture state and was so far not intended to be run multiple times. I went ahead and tried to improve the behaviour to better allow re-running. Please try with
pip install -i http://pypi.testrun.org -U pytest
which should give you pytest-2.3.5.dev16 at least. This is bound to be released soon so quick feedback is welcome. If you still have problems please try to send a minimal test file which shows undesired behaviour.
A word of warning: your calling of runtestprotocol() is not quite right and might lead to problems. "nextitem" should really be the item that is going to be run next. So if you re-run three times the first two invocations should have nextitem=item.
best, holger
Thanks, -Leah
On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg < notifications@github.com>wrote:
I just verified this behaviour myself with a simple test [1]. I see it with both funcargs and fixtures, but I'm not sure if it has to do with the plugin, or the way py.test works. It does inject the value into the test method, but it doesn't rerun the fixture, so it seems like it is caching the first run of the fixture and using that on subsequent runs.
I'm not sure if this is something that the plugin can have any effect on, or if it's just the way fixtures work. It is specified for this fixture that it is scope='function', and perhaps py.test makes that happen by checking the function name, which is, of course, the same for each run. I did try removing the scope argument from the fixture but that had no effect.
Do you have any thoughts about this, @klrmn <https://github.com/klrmn ?
[1] https://gist.github.com/bobsilverberg/5385035
— Reply to this email directly or view it on GitHub< https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-1636364...
.
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
-- Bob Silverberg www.silverwareconsulting.com
participants (4)
-
Bob Silverberg -
holger krekel -
Leah Klearman -
Ronny Pfannschmidt