[Twisted-Python] Azure Pipelines
Hello, I submitted ticket:9655 <https://twistedmatrix.com/trac/ticket/9655> / PR#1155 <https://github.com/twisted/twisted/pull/1155> for review a bit over a week ago. It is suffering from a red mark on the CI report due to Azure Pipelines complaining there was some "communication" problem "with the server". I imagine this is a spurious failure due to transient network conditions on the CI system. There doesn't appear to be a retry button to find out, though. I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage <https://twistedmatrix.com/trac/ticket/9454>. I can't sign in to Azure or whatever the system driving this is called. What is to become of this PR? Thanks, Jean-Paul
On Jun 21, 2019, at 6:39 AM, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote:
Hello,
I submitted ticket:9655 <https://twistedmatrix.com/trac/ticket/9655> / PR#1155 <https://github.com/twisted/twisted/pull/1155> for review a bit over a week ago. It is suffering from a red mark on the CI report due to Azure Pipelines complaining there was some "communication" problem "with the server". I imagine this is a spurious failure due to transient network conditions on the CI system. There doesn't appear to be a retry button to find out, though.
You need to be signed in to see it, but there is a "retry" button under the vertical-ellipsis button in the top right, next to "all logs". I clicked it for you (annoyingly I had to do this twice, there was another transient CI-system-related error on a different pipeline, maybe they've been having an outage?) and the build is now green.
I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage <https://twistedmatrix.com/trac/ticket/9454>.
Once you're logged in, I think you can also see the build configuration, under "edit pipeline" in the same menu the "retry" item is in. Not sure if there are some permissions to manage here for either the retry or edit buttons. If there are, and you can't see either of them, let's get them set up for anyone who wants them; but I think Github org membership should be sufficient, since from what I can tell in their control panel that's how I had permission.
I can't sign in to Azure or whatever the system driving this is called.
The system is broadly called "Microsoft", I think :-), but you do have to log in to this specific corner of it (Azure DevOps?). Anyway - if you click on the Azure build status "Details" link on Github, you'll get to this page: https://twistedmatrix.visualstudio.com/twisted/_build/results?buildId=958 <https://twistedmatrix.visualstudio.com/twisted/_build/results?buildId=958> On that page, there's a "Sign in" link in the top right. When you click it, you'll see a login.microsoftonline.com <http://login.microsoftonline.com/> page with a "sign in with github" link at the bottom. Use that and you should be all set.
What is to become of this PR?
Hopefully it'll get landed now ;). -g
On Fri, Jun 21, 2019 at 12:05 PM Glyph <glyph@twistedmatrix.com> wrote:
On Jun 21, 2019, at 6:39 AM, Jean-Paul Calderone < exarkun@twistedmatrix.com> wrote:
Hello,
I submitted ticket:9655 <https://twistedmatrix.com/trac/ticket/9655> / PR#1155 <https://github.com/twisted/twisted/pull/1155> for review a bit over a week ago. It is suffering from a red mark on the CI report due to Azure Pipelines complaining there was some "communication" problem "with the server". I imagine this is a spurious failure due to transient network conditions on the CI system. There doesn't appear to be a retry button to find out, though.
You need to be signed in to see it, but there is a "retry" button under the vertical-ellipsis button in the top right, next to "all logs".
I clicked it for you (annoyingly I had to do this twice, there was another transient CI-system-related error on a different pipeline, maybe they've been having an outage?) and the build is now green.
Annoying, indeed. :/
I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage <https://twistedmatrix.com/trac/ticket/9454>.
Once you're logged in, I think you can also see the build configuration, under "edit pipeline" in the same menu the "retry" item is in. Not sure if there are some permissions to manage here for either the retry or edit buttons. If there are, and you can't see either of them, let's get them set up for anyone who wants them; but I think Github org membership should be sufficient, since from what I can tell in their control panel that's how I had permission.
I can't sign in to Azure or whatever the system driving this is called.
The system is broadly called "Microsoft", I think :-), but you do have to log in to this specific corner of it (Azure DevOps?).
Anyway - if you click on the Azure build status "Details" link on Github, you'll get to this page:
https://twistedmatrix.visualstudio.com/twisted/_build/results?buildId=958
On that page, there's a "Sign in" link in the top right. When you click it, you'll see a login.microsoftonline.com page with a "sign in with github" link at the bottom. Use that and you should be all set.
When I follow that link and grant permission on the GitHub side, I get taken back to a microsoft.com URL that eventually fails with an infinite redirect error. :/ So I am not able to look at the build page while signed in. Presumably there is something about my browser profile that triggers this but, of course, I have no idea what. Jean-Paul
What is to become of this PR?
Hopefully it'll get landed now ;).
Thanks!
On Jun 21, 2019, at 9:09 AM, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote:
When I follow that link and grant permission on the GitHub side, I get taken back to a microsoft.com <http://microsoft.com/> URL that eventually fails with an infinite redirect error. :/ So I am not able to look at the build page while signed in.
Presumably there is something about my browser profile that triggers this but, of course, I have no idea what.
Oh, no. I've been in this state once before, and it's definitely annoying to get out of. Can you log in in incognito mode? I think the way I eventually fixed it was manually clearing cookies for a whole slew of Microsoft domains and logging in afresh; off the top of my head, these were the ones I nuked: microsoftonline.com <http://microsoftonline.com/> microsoft.com <http://microsoft.com/> live.com <http://live.com/> skype.com <http://skype.com/> visualstudio.com <http://visualstudio.com/> and all subdomains thereof. I'm Cc:ing the generous fellow who set us up with Azure Pipelines to begin with - Steve Dower (and you directly because I don't think Steve is on this list) because I mentioned to him at PyCon that I'd had trouble with this login mess before, but by the time I was talking with him I was no longer in this state and had no useful information for a repro other than "everything about this sucks". Maybe you could save those cookies before deleting them, or otherwise help figure this out so others won't have the same issue? Good luck, -glyph
On 2019-06-21 12:05, Glyph wrote:
On Jun 21, 2019, at 6:39 AM, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote: I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage.
Once you're logged in, I think you can also see the build configuration, under "edit pipeline" in the same menu the "retry" item is in. Not sure if there are some permissions to manage here for either the retry or edit buttons. If there are, and you can't see either of them, let's get them set up for anyone who wants them; but I think Github org membership should be sufficient, since from what I can tell in their control panel that's how I had permission.
Do we want Azure Pipelines config on their website while Travis, AppVeyor, and Circle are all in our source repository? Cheers, -kyle
The matrix stuff we're using is not yet supported as a pipeline workflow, so, when it is, we will move it :) - Amber On 22/6/19 4:51 am, Kyle Altendorf wrote:
On 2019-06-21 12:05, Glyph wrote:
On Jun 21, 2019, at 6:39 AM, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote: I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage.
Once you're logged in, I think you can also see the build configuration, under "edit pipeline" in the same menu the "retry" item is in. Not sure if there are some permissions to manage here for either the retry or edit buttons. If there are, and you can't see either of them, let's get them set up for anyone who wants them; but I think Github org membership should be sufficient, since from what I can tell in their control panel that's how I had permission.
Do we want Azure Pipelines config on their website while Travis, AppVeyor, and Circle are all in our source repository?
Cheers, -kyle
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 2019-06-21 18:40, Amber Brown wrote:
The matrix stuff we're using is not yet supported as a pipeline workflow, so, when it is, we will move it :)
Hmm... I'm running Python inside a build to generate the matrix for that build (with a yaml file for a pipeline). Now I'm curious what matrixing is being used that isn't supported yet. But perhaps it isn't worth bothering to work through. Cheers, -kyle
On 22/6/19 4:51 am, Kyle Altendorf wrote:
On 2019-06-21 12:05, Glyph wrote:
On Jun 21, 2019, at 6:39 AM, Jean-Paul Calderone <exarkun@twistedmatrix.com> wrote: I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage.
Once you're logged in, I think you can also see the build configuration, under "edit pipeline" in the same menu the "retry" item is in. Not sure if there are some permissions to manage here for either the retry or edit buttons. If there are, and you can't see either of them, let's get them set up for anyone who wants them; but I think Github org membership should be sufficient, since from what I can tell in their control panel that's how I had permission.
Do we want Azure Pipelines config on their website while Travis, AppVeyor, and Circle are all in our source repository?
Cheers, -kyle
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 22/6/19 9:59 am, Kyle Altendorf wrote:
Hmm... I'm running Python inside a build to generate the matrix for that build (with a yaml file for a pipeline). Now I'm curious what matrixing is being used that isn't supported yet. But perhaps it isn't worth bothering to work through.
Cheers, -kyle
It now says: #Multi-configuration and multi-agent job options are not exported to YAML. Configure these options using documentation guidance: https://docs.microsoft.com/vsts/pipelines/process/phases So maybe it'd work? Anyone want to volunteer to see? :) - Amber
On Jun 22, 2019, at 11:05 AM, Amber Brown <hawkowl@atleastfornow.net> wrote:
On 22/6/19 9:59 am, Kyle Altendorf wrote:
Hmm... I'm running Python inside a build to generate the matrix for that build (with a yaml file for a pipeline). Now I'm curious what matrixing is being used that isn't supported yet. But perhaps it isn't worth bothering to work through.
Cheers, -kyle
It now says:
#Multi-configuration and multi-agent job options are not exported to YAML. Configure these options using documentation guidance: https://docs.microsoft.com/vsts/pipelines/process/phases
So maybe it'd work? Anyone want to volunteer to see? :)
I'm a little confused - I thought maybe you meant "now" instead of "not"; but then the only place Google could find this exact phrasing was https://github.com/Azure/azure-iot-sdk-python-preview/blob/master/vsts/build... <https://github.com/Azure/azure-iot-sdk-python-preview/blob/master/vsts/build...>, and it definitely says "not". Except then the "phases" link you sent clearly shows some YAML under the "Multi-configuration" section. Did something change? -glyph
On 23/6/19 8:04 am, Glyph wrote:
On Jun 22, 2019, at 11:05 AM, Amber Brown <hawkowl@atleastfornow.net <mailto:hawkowl@atleastfornow.net>> wrote:
On 22/6/19 9:59 am, Kyle Altendorf wrote:
Hmm... I'm running Python inside a build to generate the matrix for that build (with a yaml file for a pipeline). Now I'm curious what matrixing is being used that isn't supported yet. But perhaps it isn't worth bothering to work through.
Cheers, -kyle
It now says:
#Multi-configuration and multi-agent job options are not exported to YAML. Configure these options using documentation guidance: https://docs.microsoft.com/vsts/pipelines/process/phases
So maybe it'd work? Anyone want to volunteer to see? :)
I'm a little confused - I thought maybe you meant "now" instead of "not"; but then the only place Google could find this exact phrasing was https://github.com/Azure/azure-iot-sdk-python-preview/blob/master/vsts/build..., and it definitely says "not". Except then the "phases" link you sent clearly shows some YAML under the "Multi-configuration" section.
Did something change?
-glyph
I think it means it doesn't have the functionality to export the options, but you could write it yourself. - Amber
On 2019-06-22 14:05, Amber Brown wrote:
On 22/6/19 9:59 am, Kyle Altendorf wrote:
Hmm... I'm running Python inside a build to generate the matrix for that build (with a yaml file for a pipeline). Now I'm curious what matrixing is being used that isn't supported yet. But perhaps it isn't worth bothering to work through.
Cheers, -kyle
It now says:
#Multi-configuration and multi-agent job options are not exported to YAML. Configure these options using documentation guidance: https://docs.microsoft.com/vsts/pipelines/process/phases
So maybe it'd work? Anyone want to volunteer to see? :)
I don't know what our setup is now so I can't really compare. Do we have a matrix built from a few axes? Like Windows/macOS and py2.7/3.7 generating 4 builds for us? AFAIK that isn't yet supported. https://github.com/microsoft/azure-pipelines-yaml/issues/20 But, you can certainly write a job and list out the four (or whatever) parameter sets you want it to be run against. For romp I went ahead and implemented a little Python cross matrixer which runs as the first job and outputs JSON. It's output is used as the matrix def for another job which does whatever it's supposed to. So there are options anywhere from hand-coded YAML to build-parameter-dependent on-the-fly-generated JSON to configure a 'matrix', and whatever in between. I'm guessing at present we'd be satisfied with just hand-coded YAML parameter sets. Maybe if we moved all platforms etc to Azure we'd like a generated matrix (either a manual script when the matrix definition is changed or in-build generation though that of course adds a bit of build time). Honestly, I would expect to usually want exceptions to any basic matrix cross-producting system so just defining it with regular Python code from the get-go has some upsides. I need to tweak exttrs a bit and romp has a bit of a specialized purpose but they are the two examples I have. https://github.com/altendky/exttr/blob/60aa5d6e9f04631b8e86552620f2d9cd2a0a3... https://github.com/altendky/romp/blob/0b05a7830ba1b02e503dd0004ffdce5e0ae316... After all that... is this actually what we are talking about when it comes to matrixing? Cheers, -kyle
participants (4)
-
Amber Brown -
Glyph -
Jean-Paul Calderone -
Kyle Altendorf