Mailman 3 June 2018 - python-committers

Core Dev Sprints
by Steve Dower 07 Jun '18

07 Jun '18

Hi all Just a quick update to let everyone know that I've started sending out invites for the sprints. We have a limited number of *funded* places available, but room to accommodate more people actually being there. So the way I'm approaching this is by working down a somewhat arbitrarily-ordered list (with a bias towards previous attendees, recent contributions, and newer core developers) and sending out only as many invitations as we have funded places. As I hear that people either can't make it or don't need funding, I'll invite the next person. So if you didn't receive an invite today, it doesn't mean you aren't getting one. And if you _did_ receive an invite today, the sooner you reply the more it'll help out everyone else. Also, if you know already that you don't need PSF funding to attend but you haven't got an invite yet, please reply to me off-list and let me know. Our space is really only limited by "how many people in our large room is too many" (the fire marshal says about 120 people...), so you won't miss out. But you are able to skip the queue, as it were. Finally, I am not the person you want designing the t-shirt for this event. If someone wants to step up and design/produce a t-shirt, now is the time to claim that responsibility. Otherwise we'll probably just try and off-load old Microsoft swag ;) Cheers, Steve

2 2

Re: [python-committers] VSTS: how to rebuild a failing build?
by Victor Stinner 06 Jun '18

06 Jun '18

> Right now, close/reopen the PR is the easiest way. :( test_asyncio sometimes still fail randomly on Travis CI or AppVeyor: https://bugs.python.org/issue33694 If I close/reopen the PR, I take the risk of getting a failure on a differrent CI :-) Victor 2018-06-06 16:13 GMT+02:00 Steve Dower <steve.dower(a)python.org>: > Right now, close/reopen the PR is the easiest way. :( Even with login, it’s > not so easy to restart a GitHub PR build because the integration isn’t fully > there yet (I think you can only do it through the API and not the UI). I’m > assured it’s coming – this was top of my list of “things we need” that I > sent to the VSTS team. > > > > Top-posted from my Windows 10 phone > > > > From: Victor Stinner > Sent: Wednesday, June 6, 2018 6:45 > To: python-committers > Subject: [python-committers] VSTS: how to rebuild a failing build? > > > > Hi, > > > > While we are discussing to make VSTS mandatory, I have a question: how > > can I schedule a rebuild when VSTS failed (and the failure is > > unrelated to my change)? > > > > I'm logged in, but I don't see any action button on: > > https://python.visualstudio.com/cpython/_build?buildId=6469 > > > > I guess that I lack some permissions. I clicked on at top right (on > > "(VS)") -> "My profile" but I get an error :-) I cannot see my own > > profile: > > > > https://python.visualstudio.com/_details/profile/redirect > > "Sorry, but Victor Stinner <victor.stinner(a)gmail.com> (Microsoft > > account) is not authorized to access this page" > > > > Note: the Windows-PR build failed because of an "internal error". > > Steve Dower already passed the bug to the right people. > > https://bugs.python.org/issue33782 > > > > Victor > > _______________________________________________ > > python-committers mailing list > > python-committers(a)python.org > > https://mail.python.org/mailman/listinfo/python-committers > > Code of Conduct: https://www.python.org/psf/codeofconduct/ > >

1 0

VSTS: how to rebuild a failing build?
by Victor Stinner 06 Jun '18

06 Jun '18

Hi, While we are discussing to make VSTS mandatory, I have a question: how can I schedule a rebuild when VSTS failed (and the failure is unrelated to my change)? I'm logged in, but I don't see any action button on: https://python.visualstudio.com/cpython/_build?buildId=6469 I guess that I lack some permissions. I clicked on at top right (on "(VS)") -> "My profile" but I get an error :-) I cannot see my own profile: https://python.visualstudio.com/_details/profile/redirect "Sorry, but Victor Stinner <victor.stinner(a)gmail.com> (Microsoft account) is not authorized to access this page" Note: the Windows-PR build failed because of an "internal error". Steve Dower already passed the bug to the right people. https://bugs.python.org/issue33782 Victor

2 1

Re: [python-committers] Comments on moving issues to GitHub
by Mariatta Wijaya 05 Jun '18

05 Jun '18

> > "Old and languishing issues should just be closed / ignored" > I disagree with doing this blindly, and I would be mightily annoyed if > someone did so with IDLE issues and hide valuable ideas and code. Since you are IDLE's maintainer, I'll also be annoyed if other people except yourself (or other IDLE maintainers) blindly close IDLE issues without consulting you. Many modules don't have maintainers anymore. If such issues have been ignored for all these years, we'll probably never get to it. Might as well close it. The proposed idea is to provide a button that can copy over conversations from a b.p.o issue to GitHub, and to continue discussions in GitHub. If core devs have a list of issues they still care about, then they use this not yet existing magic button. The closed issue will still be in bpo, and anyone motivated enough can migrate it to GitHub. To deal with issues better, we need > 1. More core developers, so more modules can have maintainers. We need more core developers anyway, regardless of what tracker we're using. That is a separate problem. And perhaps this is to be discussed in a separate thread: even though in the b.p.o we appear to have 170 committers, really there are 90 core devs (people who has commit right to CPython on GitHub). and out of those 90, I think only about half are currently active (since the migration to GitHub). So yes, we need more (active) core devs. 2. Better support for core developers in the tracker. Not sure what you mean by "support"? There are only two maintainers of the bug tracker, they both are also Python core developers: Brett and Ezio. My personal opinion is: they're more valuable elsewhere instead of supporting the bug tracker. At its current state, the bug tracker is not ready to take up new contributors, and it will not be easy effort to onboard new bpo maintainers. 2b. Associated (linked) manager or dashboard for issues pertaining to a > module or group of modules. We can try the project boards as Ivan mentioned? https://help.github.com/ articles/about-project-boards/ * Labels can be grouped using name prefix and color, for example (we have > similar structure in mypy): > - priority-low > - priority-normal > - priority-etc... > - kind-bug > - kind-docs > - kind-feature > - topic-asincio > - topic-etc.. I kinda like those! I wonder if other hosted services, such as Gitlab, offer a more > sophisticated issue tracker. One thing I'm trying to avoid is having separate issue tracker and repo, and needing different accounts. Possibly useful for planning, if we had someone who was responsible for that > Perhaps for a project the size of Python we should have a dedicated Project manager. Mariatta ᐧ

13 18

Wrongly stopping merges discourages merging.
by Terry Reedy 05 Jun '18

05 Jun '18

When we used hg, core dev committers could actually commit to the repository when they judged it appropriate. When we moved to github, Brett, with whoever's approval, decided that we should no longer be trusted to make commits without approval of a couple of mindless robots. However, the premise that the robots should be trusted is false. So I again request that there be a manual override for when the robots are obviously giving false failures. Exhibit 1. For at least a couple of weeksin may, faults in the asyncio test (and another) caused the asyncio test to randomly fail about half the time. With one retest, each CI bot failed about 1/4 the time. At least one bot of the two bots failed about 1/2 the time. The AppVeyor queue ballooned. One could decrease the frustration and time to success (but only partly) by only re-starting the bot that failed. Doing so for Travis is fairly easy. Doing so with AppVeyor is obscure and error prone. I twice requested that the randomly failing tests be disabled. Victor said he wanted to keep monitoring what they did. I think he overly discounted the pain and frustration of having good merges blocked. I think either 1) bad tests should be disabled, or 2. the CI code should be able to ignore failures by bad tests, or 3. responsible core devs should be able to. Exhibit 2. AppVeyor is badly broken. This morning Cheryl Sabella submitted a nice patch fixing an annoying behavior of IDLE's editor/shell/output windows. The CI tests passed, the patch worked great, it only needed expansion of the placeholder blurb. I was really excited. With some trepidation, I made the edit. Unfortunately, both CI bots rerun the code tests even when the code is unchanged. Blurb edits should be treated as doc-only changes, with only the blurb rechecked. My trepidation turned out to be well-founded. My excitement is gone. After an error, AppVeyor just quit without reporting any failure cause. https://ci.appveyor.com/project/python/cpython/build/3.8build16869 Since then, there have been 2 successes and 7 similar unexplained failures. https://ci.appveyor.com/project/python/cpython/history https://ci.appveyor.com/project/python/cpython/build/3.6build16871/job/c2q8… https://ci.appveyor.com/project/python/cpython/build/3.8build16872 https://ci.appveyor.com/project/python/cpython/build/3.7build16873 https://ci.appveyor.com/project/python/cpython/build/3.8build16874 https://ci.appveyor.com/project/python/cpython/build/3.6build16876/job/t9ny… https://ci.appveyor.com/project/python/cpython/build/3.8build16877 https://ci.appveyor.com/project/python/cpython/build/3.8build16878 The last is my restart. The time wasted by this broken blockbot is time not spent doing something useful. I would really like to have this patch in the .rc in a week -- along with a few others that should follow. Guido once asked what is off-putting about being a core developer. This is one thing. Terry Jan Reedy

7 7

Re: [python-committers] Wrongly stopping merges discourages merging.
by Ned Deily 04 Jun '18

04 Jun '18

On Jun 3, 2018, at 22:30, Steve Dower <steve.dower(a)python.org> wrote: > We probably have enough data on the VSTS builds by now to see whether they are comparable/faster than AppVeyor. Obviously the idea of doing that work was to be able to migrate builds if it made sense, and if we decide not to then they get ripped out (non-binding PR checks are confusing IMHO, particularly when they duplicate required checks). > > I have no idea whether that discussion is still ongoing on core-workflow, but if it seems better then maybe it’s time? Anyone can view the VSTS build history starting from https://python.visualstudio.com/cpython/_build and browsing into the build definition of interest. My gut feel from observing the progress of PRs over the past couple of weeks is that the VSTS CI builds are faster and much less problematic than the AppVeyor builds have been. That said, one significant Windows test bottleneck was identified last week (largefile tests in test_mmap) on some buildbots and was disabled. We've now also disabled it on AppVeyor, once AppVeyor starts running our tests again, and I've suggested it be disabled on the VSTS Windows CI runs as well. That, along with a number of other test fixes made over the past week, may help eliminate with AppVeyor. But, at this point, I think we should seriously consider dropping mandatory AppVeyor CI runs in favor of the VSTS ones. Brett, do you concur? And, if so, can you make it happen? -- Ned Deily nad(a)python.org -- []

2 1

AppVeyor slowed down?
by Victor Stinner 04 Jun '18

04 Jun '18

Hi, Is it just me or AppVeyor became slower than it was a few months ago? First of all, for AppVeyor, *all* builds are in the same queue and we are only allowed to run one job in parallel. Currently, AppVeyor is the obvious bottleneck of our workflow. It directly limits the number of commits that we can merge per day. Hopefully, thanks to Mariatta, backports are now merged as soon as CIs pass if a core developer approved the PR. It's no longer needed to watch the CIs to know when the Merge button will be enabed! I looked more closely at AppVeyor last week since it took up to 2 hours to run tests on my CI, preventing me to merge it. (Q1) AppVeyor runs again tests once the PR is merge. Was it the point of running tests in each branch since... nobody look at test results of these builds? Would it be possible to disable these jobs? (Q2) AppVeyor runs tests twice on 3.6: VS 2015 and VS2017. It *seems* (I'm not sure) that there are tested... sequentially. Each job takes 12 min so the who build takes 24 min... It seems like the official binary on python.org is built on VS 2015: so maybe we could only keep VS 2015 for the blocking CI, and maybe add a new buildbot (if there is not already one) for VS 2017? (Q3) How can we be allowed to run more jobs in parallel? Would it be free? Zachary Ware wrote that he disabled the "largefile" resources of Python test suite, so tests should now run faster. Last week I found the cool build history: https://ci.appveyor.com/project/python/cpython/history It helped me to find some tests which fail randomly. I noticed that they fail much more frequently than what I expected. Most developers just schedule a new build. Victor

1 0

Comments on moving issues to GitHub
by Victor Stinner 03 Jun '18

03 Jun '18

Hi, I failed to get the microphone after Mariatta's secret talk about moving Python issues from bugs.python.org (Roundup) to GitHub. I just wanted to add that it's common (at least once per month) that someone comes to #python-dev to report that they cannot connect to bugs.python.org (the authentication is broken). Usually, I tried to point them to the meta-bug tracker, but I hate doing that... The concept of a meta-bug tracker blows my mind :-) Right now, I tied to add a comment on an issue and I got the error: """ Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request POST /issue32534. Reason: Error reading from remote server ________________________________ Apache/2.2.16 (Debian) Server at bugs.python.org Port 443 """ Sometimes, I get a SSL error. I reported the issue 7 months ago, and sometimes I still get the error: https://github.com/python/psf-infra-meta/issues/4 It's a random error, but using a loop, it's easy to reproduce it. I don't have a strong opinion about moving issues to GitHub. I just wanted to point out that if we keep bugs.python.org, we need more volunteers to maintain it. I would prefer to not have to redirect new comers, who want to report a simple bug, to a "meta bug tracker"... I don't plan to report the proxy error, since my previous bug report (SSL error) is still not fixed and it's the first time that I got this error. Victor

13 15