Hi,
My PR tests are failing seemingly due to some bug in enum.rst Which might be caused by https://github.com/python/cpython/pull/22392 (that's the superficial cause, it might not be the root cause).
Example failure: https://github.com/python/cpython/pull/25069 https://github.com/python/cpython/pull/25069/checks?check_run_id=2235991560
Why are we merging PRs that fail the CI tests? Why is that even allowed?
Cheers, Mark.
One of the reasons I might be tempted to merge a pull request before the tests is completed is because there’s no mechanism to merge a pull request after the tests have completed (when there’s no reviewer).
I imagine it’s allowed because there still is the case where tests can fail for some reason unrelated to the pull request at hand.
In short, the system isn’t reliable or fast or flexible enough to depend solely on automated systems, so it still relies on human judgment.
At least, that’s my suspicion.
On 31 Mar, 2021, at 06:52, Mark Shannon <mark@hotpy.org> wrote:
Hi,
My PR tests are failing seemingly due to some bug in enum.rst Which might be caused by https://github.com/python/cpython/pull/22392 (that's the superficial cause, it might not be the root cause).
Example failure: https://github.com/python/cpython/pull/25069 https://github.com/python/cpython/pull/25069/checks?check_run_id=2235991560
Why are we merging PRs that fail the CI tests? Why is that even allowed?
Cheers, Mark.
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/3... Code of Conduct: https://www.python.org/psf/codeofconduct/
Doesn't the automerge label block on failing CI? I think that should solve the issue you are describing.
On March 31, 2021 1:06:15 PM UTC, "Jason R. Coombs" <jaraco@jaraco.com> wrote:
One of the reasons I might be tempted to merge a pull request before the tests is completed is because there’s no mechanism to merge a pull request after the tests have completed (when there’s no reviewer).
I imagine it’s allowed because there still is the case where tests can fail for some reason unrelated to the pull request at hand.
In short, the system isn’t reliable or fast or flexible enough to depend solely on automated systems, so it still relies on human judgment.
At least, that’s my suspicion.
On 31 Mar, 2021, at 06:52, Mark Shannon <mark@hotpy.org> wrote:
Hi,
My PR tests are failing seemingly due to some bug in enum.rst Which might be caused by https://github.com/python/cpython/pull/22392 (that's the superficial cause, it might not be the root cause).
Example failure: https://github.com/python/cpython/pull/25069 https://github.com/python/cpython/pull/25069/checks?check_run_id=2235991560
Why are we merging PRs that fail the CI tests? Why is that even allowed?
Cheers, Mark.
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/3... Code of Conduct: https://www.python.org/psf/codeofconduct/
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/L... Code of Conduct: https://www.python.org/psf/codeofconduct/
On 3/31/21 3:52 AM, Mark Shannon wrote:
My PR tests are failing seemingly due to some bug in enum.rst Which might be caused by https://github.com/python/cpython/pull/22392 (that's the superficial cause, it might not be the root cause).
As far as I can tell, it is not the root cause. I (and others) have also had issues with ssl/socket randomly failing tests.
Why are we merging PRs that fail the CI tests?
Because I could not find any error in the documentation that would cause the problem (the first three cases succeeded, using the same construct).
Why is that even allowed?
Because the tests are not perfect.
I did post a message to python-dev to see if anybody had any idea on why that particular test was failing, but there were no replies.
-- ~Ethan~
On 31.03.2021 15:54, Ethan Furman wrote:
Because I could not find any error in the documentation that would cause the problem (the first three cases succeeded, using the same construct).
Why is that even allowed?
Because the tests are not perfect.
I did post a message to python-dev to see if anybody had any idea on why that particular test was failing, but there were no replies.
It seems that some of the doc tests are missing imports of e.g. Flag from enum.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Experts (#1, Mar 31 2021)
Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/
On 3/31/21 6:59 AM, M.-A. Lemburg wrote:
It seems that some of the doc tests are missing imports of e.g. Flag from enum.
My understanding of doctest is that the global execution environment is cumulative. For example. the three previous tests, which all pass, are also not reimporting Flag.
Also note that the tests pass fine locally, suggesting that this is a CI problem. [1]
-- ~Ethan~
On 31.03.2021 16:29, Ethan Furman wrote:
On 3/31/21 6:59 AM, M.-A. Lemburg wrote:
It seems that some of the doc tests are missing imports of e.g. Flag from enum.
My understanding of doctest is that the global execution environment is cumulative. For example. the three previous tests, which all pass, are also not reimporting Flag.
Looking at the doctest.py source code, there appear to be plenty of ways managing the globals:
https://github.com/python/cpython/blob/master/Lib/doctest.py#L872
The globals for each DocTest is formed by combining `globs`
and `extraglobs` (bindings in `extraglobs` override bindings
in `globs`). A new copy of the globals dictionary is created
for each DocTest. If `globs` is not specified, then it
defaults to the module's `__dict__`, if specified, or {}
otherwise. If `extraglobs` is not specified, then it defaults
to {}.
Also note that the tests pass fine locally, suggesting that this is a CI problem. [1]
Perhaps CI is running the tests in a different way than the local Makefile. E.g. CI could be using the "per DocText glob copy", while the Makefile uses the module namespace.
I'm only guessing here... never used doctest.
[1] https://bugs.python.org/issue43681
Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Experts (#1, Mar 31 2021)
Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com/
participants (5)
-
Ethan Furman
-
Jason R. Coombs
-
M.-A. Lemburg
-
Mark Shannon
-
Paul G