From brett at python.org  Sat Nov  5 19:38:24 2016
From: brett at python.org (Brett Cannon)
Date: Sat, 05 Nov 2016 23:38:24 +0000
Subject: [core-workflow] Getting help w/ testing Travis and CircleCI
Message-ID: <CAP1=2W4_-EF93HUgs7Lq6JnVhBUsgqCtO_xKp43iMCR=mgSozg@mail.gmail.com>

I have set up https://github.com/brettcannon/cpython-ci-test to help test
both Travis and CircleCI as possible CI services so that we can start the
transition at an advantage instead of breaking even in terms of feature set
compared to the current workflow. (Thanks to Alexander Belopolsky for
initially looking into this.)

If you have experience with Travis and/or CircleCI, please look at the
issues and the configuration files to help test both services at their best
and submit PRs to that repo as appropriate. What I'm looking for is not
just the best build scenario coverage (i.e. as many compilers and OSs as
possible), but also fast solutions (e.g. don't bother building the docs on
every platform/compiler combo if possible). The goal is for PRs to have
good checks to make sure they don't break anything with those checks not
taking forever so that the turn-around time on PRs can be as fast as
possible.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161105/b67b7c53/attachment.html>

From ncoghlan at gmail.com  Sun Nov  6 00:39:15 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 6 Nov 2016 14:39:15 +1000
Subject: [core-workflow] Getting help w/ testing Travis and CircleCI
In-Reply-To: <CAP1=2W4_-EF93HUgs7Lq6JnVhBUsgqCtO_xKp43iMCR=mgSozg@mail.gmail.com>
References: <CAP1=2W4_-EF93HUgs7Lq6JnVhBUsgqCtO_xKp43iMCR=mgSozg@mail.gmail.com>
Message-ID: <CADiSq7cJBvX-S3fJv=Cd-RFkh9-VqK6m0TS8L-L8hCc66Y-a3Q@mail.gmail.com>

On 6 November 2016 at 09:38, Brett Cannon <brett at python.org> wrote:
> If you have experience with Travis and/or CircleCI, please look at the
> issues and the configuration files to help test both services at their best
> and submit PRs to that repo as appropriate. What I'm looking for is not just
> the best build scenario coverage (i.e. as many compilers and OSs as
> possible), but also fast solutions (e.g. don't bother building the docs on
> every platform/compiler combo if possible). The goal is for PRs to have good
> checks to make sure they don't break anything with those checks not taking
> forever so that the turn-around time on PRs can be as fast as possible.

A couple of things worth noting on that front: directory filtering
that looks at which files changed can make a decent difference. For
example, if only Doc and Misc contain changes, then the entire
compile-and-test aspect can be skipped, since it's just a docs change.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Fri Nov 18 20:15:59 2016
From: brett at python.org (Brett Cannon)
Date: Sat, 19 Nov 2016 01:15:59 +0000
Subject: [core-workflow] GitHub migration status as of 2016-11-18
Message-ID: <CAP1=2W410Jou0CLW5-m6GcKPSwdww_5OoQK1kbsNyqsib8V=yQ@mail.gmail.com>

Since Python 3.6.0 is due in just under a month it means we're hopefully in
the home stretch of getting things in line to do the GitHub migration by
the end of the year! This is a quick update on where things stand.

The devguide is still finished. :) That's the end of the "completed" list
for now.

In progress are reviews for the patches against bugs.python.org thanks to
Maciej and his GSoC student. Ezio is reviewing those patches and once they
go in we will then be able to test some things to make sure nothing is
going to break. I have also configured Travis and CircleCI at
https://github.com/brettcannon/cpython-ci-test to test them out (separate
email on that coming later).

I'm trying to track down the location of the code for hg.python.org/lookup
so that it can be updated appropriately. Emails for checkins and IRC
messages can be done the day of the migration (or shortly thereafter).
Updating PEP 101 or getting sys._git in can come after the migration. That
leaves updating how the buildbots get triggered and pull down code (email
out to python-buildbots on that topic).

And due to the lack of tags in the mirror we will have to recreate the repo
using Senthil's script. I don't know if it's best to simply delete the
project and recreate it or to do a force-push over the mirror.

And that's where everything stands. It all seems doable by the end of the
year!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161119/f1ef3324/attachment.html>

From brett at python.org  Fri Nov 18 20:26:00 2016
From: brett at python.org (Brett Cannon)
Date: Sat, 19 Nov 2016 01:26:00 +0000
Subject: [core-workflow] CircleCI or Travis?
Message-ID: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>

https://github.com/brettcannon/cpython-ci-test has configurations for both
CircleCI and Travis. If I'm missing something in either configuration then
please let me know (e.g. there is still a free container to use on
CircleCI). If I also missed a better CI service then also let me know (but
I don't think I have).

Otherwise please look at the output of both CI services and let it be known
which service you prefer (i.e. +1/+0/-0/-1 for both services). Ignore the
fact that the doc tests are failing as I have fixed that in Python 3.6 and
3.7 already. Consider ease of reading the output, how long it took to build
(e.g. I might turn off macOS support on Travis because they seem to have
under-provisioned those machines), etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161119/04cc1b9b/attachment.html>

From ncoghlan at gmail.com  Fri Nov 18 22:09:12 2016
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 19 Nov 2016 13:09:12 +1000
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
Message-ID: <CADiSq7eNe_QUG4-m1r3NdntytqiHSLGRe8QhE9Aka-XWCTht9g@mail.gmail.com>

On 19 November 2016 at 11:26, Brett Cannon <brett at python.org> wrote:
> https://github.com/brettcannon/cpython-ci-test has configurations for both
> CircleCI and Travis. If I'm missing something in either configuration then
> please let me know (e.g. there is still a free container to use on
> CircleCI). If I also missed a better CI service then also let me know (but I
> don't think I have).

My main question was whether we'd be able to eventually add the
manylinux1 build container to the pre-merge CI configuration, but it
looks like both services support that:

Travis: https://docs.travis-ci.com/user/docker/
CircleCI: https://circleci.com/docs/docker/

Of the two configuration files, I find the Travis one easier to read,
and the test matrix set up also makes it easier to see the results for
different configurations. It's also the preferred CI service for PyPA,
and I think consistency there will be helpful, especially if the PSF
ends up approaching our CI provider regarding either a paid account or
an in-kind sponsorship.

So call it +1 for Travis CI, -0 for Circle CI (I don't think the
latter would be a *bad* choice, I just think Travis is a better choice
for us specifically)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From donald at stufft.io  Fri Nov 18 22:32:31 2016
From: donald at stufft.io (Donald Stufft)
Date: Fri, 18 Nov 2016 22:32:31 -0500
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
Message-ID: <0CAA5EF4-1BC2-4076-B6AC-EFEC0B7D8B57@stufft.io>


> On Nov 18, 2016, at 8:26 PM, Brett Cannon <brett at python.org> wrote:
> 
> Otherwise please look at the output of both CI services and let it be known which service you prefer (i.e. +1/+0/-0/-1 for both services). Ignore the fact that the doc tests are failing as I have fixed that in Python 3.6 and 3.7 already. Consider ease of reading the output, how long it took to build (e.g. I might turn off macOS support on Travis because they seem to have under-provisioned those machines), etc.


+1 Travis
-0 CircleCI

FWIW I know the folks behind Travis so getting limits increased and stuff if we need it is something that *may* be possible.

?
Donald Stufft



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161118/04fa7e8b/attachment.html>

From berker.peksag at gmail.com  Fri Nov 18 22:42:31 2016
From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=)
Date: Sat, 19 Nov 2016 06:42:31 +0300
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
Message-ID: <CAF4280KfePu9bnS_S5FaVLeU=-SwUP4SokEeHrLMnq_974X-EQ@mail.gmail.com>

On Sat, Nov 19, 2016 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
> https://github.com/brettcannon/cpython-ci-test has configurations for both
> CircleCI and Travis. If I'm missing something in either configuration then
> please let me know (e.g. there is still a free container to use on
> CircleCI). If I also missed a better CI service then also let me know (but I
> don't think I have).
>
> Otherwise please look at the output of both CI services and let it be known
> which service you prefer (i.e. +1/+0/-0/-1 for both services). Ignore the
> fact that the doc tests are failing as I have fixed that in Python 3.6 and
> 3.7 already. Consider ease of reading the output, how long it took to build
> (e.g. I might turn off macOS support on Travis because they seem to have
> under-provisioned those machines), etc.

+1 for Travis CI because I'm more familiar with it. I only looked at
the latest builds and it looks like Travis CI is faster than Circle CI
(12 min vs. 30 min, excluding installing dependencies etc.)

Thanks for working on this, Brett! :)

--Berker

From berker.peksag at gmail.com  Fri Nov 18 23:54:03 2016
From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=)
Date: Sat, 19 Nov 2016 07:54:03 +0300
Subject: [core-workflow] GitHub migration status as of 2016-11-18
In-Reply-To: <CAP1=2W410Jou0CLW5-m6GcKPSwdww_5OoQK1kbsNyqsib8V=yQ@mail.gmail.com>
References: <CAP1=2W410Jou0CLW5-m6GcKPSwdww_5OoQK1kbsNyqsib8V=yQ@mail.gmail.com>
Message-ID: <CAF4280Jg2uo7Ptu+ok-+EwKQcK5MGNRRokmMum-SD=tej4dC=g@mail.gmail.com>

On Sat, Nov 19, 2016 at 4:15 AM, Brett Cannon <brett at python.org> wrote:
> I'm trying to track down the location of the code for hg.python.org/lookup
> so that it can be updated appropriately.

Did you find it? I spent an hour and all I could find was a path of a
custom WSGI application on hg.python.org:
https://github.com/python/psf-salt/blob/master/salt/hg/config/hg.apache.conf.jinja#L30
:)

From soltysh at gmail.com  Sat Nov 19 13:34:55 2016
From: soltysh at gmail.com (Maciej Szulik)
Date: Sat, 19 Nov 2016 19:34:55 +0100
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAF4280KfePu9bnS_S5FaVLeU=-SwUP4SokEeHrLMnq_974X-EQ@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
 <CAF4280KfePu9bnS_S5FaVLeU=-SwUP4SokEeHrLMnq_974X-EQ@mail.gmail.com>
Message-ID: <CALC2N4Yc9Zhbtuog=BnOzK+-371EkEadidiwEjhEONvpezwanA@mail.gmail.com>

+1 for Travis for its readability and knowledge with it
0 for CircleCI

On Sat, Nov 19, 2016 at 4:42 AM, Berker Peksa? <berker.peksag at gmail.com>
wrote:

> On Sat, Nov 19, 2016 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
> > https://github.com/brettcannon/cpython-ci-test has configurations for
> both
> > CircleCI and Travis. If I'm missing something in either configuration
> then
> > please let me know (e.g. there is still a free container to use on
> > CircleCI). If I also missed a better CI service then also let me know
> (but I
> > don't think I have).
> >
> > Otherwise please look at the output of both CI services and let it be
> known
> > which service you prefer (i.e. +1/+0/-0/-1 for both services). Ignore the
> > fact that the doc tests are failing as I have fixed that in Python 3.6
> and
> > 3.7 already. Consider ease of reading the output, how long it took to
> build
> > (e.g. I might turn off macOS support on Travis because they seem to have
> > under-provisioned those machines), etc.
>
> +1 for Travis CI because I'm more familiar with it. I only looked at
> the latest builds and it looks like Travis CI is faster than Circle CI
> (12 min vs. 30 min, excluding installing dependencies etc.)
>
> Thanks for working on this, Brett! :)
>

+10000....00000 :-)


>
> --Berker
> _______________________________________________
> core-workflow mailing list
> core-workflow at python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161119/58bb2b01/attachment.html>

From brett at python.org  Sat Nov 19 14:06:32 2016
From: brett at python.org (Brett Cannon)
Date: Sat, 19 Nov 2016 19:06:32 +0000
Subject: [core-workflow] GitHub migration status as of 2016-11-18
In-Reply-To: <CAF4280Jg2uo7Ptu+ok-+EwKQcK5MGNRRokmMum-SD=tej4dC=g@mail.gmail.com>
References: <CAP1=2W410Jou0CLW5-m6GcKPSwdww_5OoQK1kbsNyqsib8V=yQ@mail.gmail.com>
 <CAF4280Jg2uo7Ptu+ok-+EwKQcK5MGNRRokmMum-SD=tej4dC=g@mail.gmail.com>
Message-ID: <CAP1=2W4KBjDD46PTWSpz+rP-ivL1cmWf1+geMMbLfCXs4HgG1A@mail.gmail.com>

I was told where the code for hg.python.org/lookup is on the machine, but I
lack machine access. :) I've asked for the code to either be tossed up
under version control or to have it sent to me manually.

On Fri, 18 Nov 2016 at 20:54 Berker Peksa? <berker.peksag at gmail.com> wrote:

> On Sat, Nov 19, 2016 at 4:15 AM, Brett Cannon <brett at python.org> wrote:
> > I'm trying to track down the location of the code for
> hg.python.org/lookup
> > so that it can be updated appropriately.
>
> Did you find it? I spent an hour and all I could find was a path of a
> custom WSGI application on hg.python.org:
>
> https://github.com/python/psf-salt/blob/master/salt/hg/config/hg.apache.conf.jinja#L30
> :)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161119/0ea2c6b3/attachment.html>

From songofacandy at gmail.com  Mon Nov 21 06:46:55 2016
From: songofacandy at gmail.com (INADA Naoki)
Date: Mon, 21 Nov 2016 20:46:55 +0900
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
Message-ID: <CAEfz+TzNhm_qpM6LgtUVCFhEsbsVLiq5r3=3v8wB03CNb8--Cw@mail.gmail.com>

> Ignore the
> fact that the doc tests are failing as I have fixed that in Python 3.6 and
> 3.7 already.

test_datetime fails too on Travis-CI.
https://travis-ci.org/brettcannon/cpython-ci-test/jobs/177187425#L5962

+1 for Travis-CI, if we can fix it.

-- 
INADA Naoki  <songofacandy at gmail.com>

From elprans at gmail.com  Mon Nov 21 09:39:09 2016
From: elprans at gmail.com (Elvis Pranskevichus)
Date: Mon, 21 Nov 2016 09:39:09 -0500
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAEfz+TzNhm_qpM6LgtUVCFhEsbsVLiq5r3=3v8wB03CNb8--Cw@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
 <CAEfz+TzNhm_qpM6LgtUVCFhEsbsVLiq5r3=3v8wB03CNb8--Cw@mail.gmail.com>
Message-ID: <2261578.PuqdKIoB0P@hammer.magicstack.net>

On Monday, November 21, 2016 8:46:55 PM EST INADA Naoki wrote:
> > Ignore the
> > fact that the doc tests are failing as I have fixed that in Python 3.6 and
> > 3.7 already.
> 
> test_datetime fails too on Travis-CI.
> https://travis-ci.org/brettcannon/cpython-ci-test/jobs/177187425#L5962
> 
> +1 for Travis-CI, if we can fix it.

Tests seem to pass for me with:

dist: trusty
sudo: false
group: beta

https://travis-ci.org/elprans/cpython-ci-test/jobs/177665369#L1745
https://github.com/brettcannon/cpython-ci-test/pull/22


                               Elvis

From brett at python.org  Mon Nov 21 13:28:42 2016
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Nov 2016 18:28:42 +0000
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAF4280KfePu9bnS_S5FaVLeU=-SwUP4SokEeHrLMnq_974X-EQ@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
 <CAF4280KfePu9bnS_S5FaVLeU=-SwUP4SokEeHrLMnq_974X-EQ@mail.gmail.com>
Message-ID: <CAP1=2W6GWC_wOYXcqhE-2XO9RA=Qh+093q3Q+hzG0bYFK8svqg@mail.gmail.com>

On Fri, 18 Nov 2016 at 19:43 Berker Peksa? <berker.peksag at gmail.com> wrote:

> On Sat, Nov 19, 2016 at 4:26 AM, Brett Cannon <brett at python.org> wrote:
> > https://github.com/brettcannon/cpython-ci-test has configurations for
> both
> > CircleCI and Travis. If I'm missing something in either configuration
> then
> > please let me know (e.g. there is still a free container to use on
> > CircleCI). If I also missed a better CI service then also let me know
> (but I
> > don't think I have).
> >
> > Otherwise please look at the output of both CI services and let it be
> known
> > which service you prefer (i.e. +1/+0/-0/-1 for both services). Ignore the
> > fact that the doc tests are failing as I have fixed that in Python 3.6
> and
> > 3.7 already. Consider ease of reading the output, how long it took to
> build
> > (e.g. I might turn off macOS support on Travis because they seem to have
> > under-provisioned those machines), etc.
>
> +1 for Travis CI because I'm more familiar with it. I only looked at
> the latest builds and it looks like Travis CI is faster than Circle CI
> (12 min vs. 30 min, excluding installing dependencies etc.)
>

So total execution may be faster, but for Travis there's a bigger delay in
actually starting the build.


>
> Thanks for working on this, Brett! :)
>

Welcome!

-Brett


>
> --Berker
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161121/49238079/attachment.html>

From brett at python.org  Mon Nov 21 14:18:18 2016
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Nov 2016 19:18:18 +0000
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <2261578.PuqdKIoB0P@hammer.magicstack.net>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
 <CAEfz+TzNhm_qpM6LgtUVCFhEsbsVLiq5r3=3v8wB03CNb8--Cw@mail.gmail.com>
 <2261578.PuqdKIoB0P@hammer.magicstack.net>
Message-ID: <CAP1=2W5gZ7GghxTQr8mog17ZYX8TCEQmuDBn=RA_OP6=aXqCvA@mail.gmail.com>

Thanks for testing out the trusty environment! That did seem to fix the
test failures related to datetime (the doc failure is expected).

On Mon, 21 Nov 2016 at 06:39 Elvis Pranskevichus <elprans at gmail.com> wrote:

> On Monday, November 21, 2016 8:46:55 PM EST INADA Naoki wrote:
> > > Ignore the
> > > fact that the doc tests are failing as I have fixed that in Python 3.6
> and
> > > 3.7 already.
> >
> > test_datetime fails too on Travis-CI.
> > https://travis-ci.org/brettcannon/cpython-ci-test/jobs/177187425#L5962
> >
> > +1 for Travis-CI, if we can fix it.
>
> Tests seem to pass for me with:
>
> dist: trusty
> sudo: false
> group: beta
>
> https://travis-ci.org/elprans/cpython-ci-test/jobs/177665369#L1745
> https://github.com/brettcannon/cpython-ci-test/pull/22
>
>
>                                Elvis
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161121/2782d8e5/attachment.html>

From brett at python.org  Mon Nov 21 16:06:09 2016
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Nov 2016 21:06:09 +0000
Subject: [core-workflow] CircleCI or Travis?
In-Reply-To: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
References: <CAP1=2W41dpKqBEGQ7zW-z=dD+P7xwoX8N4XX=vMiTk_mCPKx=Q@mail.gmail.com>
Message-ID: <CAP1=2W5h3uoEN5AUw=V3_YN0qXJX1Wo7FAtQTVmJ=oaRUo0NTA@mail.gmail.com>

Everyone who expressed an opinion seems to explicitly prefer Travis over
CircleCI, so that solves that debate. :)

I did end up disabling macOS builds as Travis has under-provisioned those
machines to the point that it adds an extra hour to the CI run (goes from
1.5 hours to under 30 minutes by removing macOS). I left building against
both gcc and clang as well as the doc build.

Thanks everyone for the input!

On Fri, 18 Nov 2016 at 17:25 Brett Cannon <brett at python.org> wrote:

> https://github.com/brettcannon/cpython-ci-test has configurations for
> both CircleCI and Travis. If I'm missing something in either configuration
> then please let me know (e.g. there is still a free container to use on
> CircleCI). If I also missed a better CI service then also let me know (but
> I don't think I have).
>
> Otherwise please look at the output of both CI services and let it be
> known which service you prefer (i.e. +1/+0/-0/-1 for both services). Ignore
> the fact that the doc tests are failing as I have fixed that in Python 3.6
> and 3.7 already. Consider ease of reading the output, how long it took to
> build (e.g. I might turn off macOS support on Travis because they seem to
> have under-provisioned those machines), etc.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161121/53cbd99d/attachment.html>

From brett at python.org  Mon Nov 21 16:22:36 2016
From: brett at python.org (Brett Cannon)
Date: Mon, 21 Nov 2016 21:22:36 +0000
Subject: [core-workflow] Coverage reporting through Travis?
Message-ID: <CAP1=2W4nSFnJW7AEg32tNkVkEptZj6od56XMyzjdEymZaWxMMQ@mail.gmail.com>

Now that we have settled on Travis, the next question is whether we can
calculate coverage results fast enough to not have the process killed, as
well as who to use for the coverage report.

For the measurement run, we can add a separate run to the build matrix so
as to not accidentally mix any weird interaction with coverage.py and
Python itself. It also allows us to flag the coverage reporting as
acceptable to fail and thus not get a false-negative on the CI run. I don't
know if procedural or concurrent coverage measurement
<http://coverage.readthedocs.io/en/coverage-4.2/subprocess.html#configuring-python-for-sub-process-coverage>
will work out best, so both will need to be tested. I also don't know if
branch coverage will be too slow for Travis' time limit. If anyone thinks
this logic is flawed or missing something then please let me know.

As for coverage-reporting services, I know of https://codecov.io/ and
https://coveralls.io/. If people have any other recommendations I'm open to
hearing about them.

I will continue to use https://github.com/brettcannon/cpython-ci-test to
experiment with code coverage services.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161121/3be37e18/attachment-0001.html>

From phd at phdru.name  Mon Nov 21 17:31:12 2016
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 21 Nov 2016 23:31:12 +0100
Subject: [core-workflow] Coverage reporting through Travis?
In-Reply-To: <CAP1=2W4nSFnJW7AEg32tNkVkEptZj6od56XMyzjdEymZaWxMMQ@mail.gmail.com>
References: <CAP1=2W4nSFnJW7AEg32tNkVkEptZj6od56XMyzjdEymZaWxMMQ@mail.gmail.com>
Message-ID: <20161121223112.GA6594@phdru.name>

Hi!

On Mon, Nov 21, 2016 at 09:22:36PM +0000, Brett Cannon <brett at python.org> wrote:
> As for coverage-reporting services, I know of https://codecov.io/ and
> https://coveralls.io/. If people have any other recommendations I'm open to
> hearing about them.

   The only minor difference between them that I know of is: if one
updates a pull request using force-push codecov updates its comment and
coveralls adds a new comment every time. See for example this PR:
https://github.com/andialbrecht/sqlparse/pull/279 - I've updated it 5
times; there is one updated comment from codecov and 5 comments from
coveralls.
   If you prefer one way or the other - you know what to choose.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From brett at python.org  Tue Nov 22 15:08:48 2016
From: brett at python.org (Brett Cannon)
Date: Tue, 22 Nov 2016 20:08:48 +0000
Subject: [core-workflow] Choosing a code coverage reporter
Message-ID: <CAP1=2W6nHxSU9UU6y5oFBaBzZ6RvtSqqZBF+X1pMXV9QH_Tx3Q@mail.gmail.com>

I have both Codecov and Coveralls up and running. If you have an
opinion/preference, please vote at
https://github.com/brettcannon/cpython-ci-test/issues/27 using the
reactions on the appropriate comment (.e.g +1/-1 reaction for Codecov
and/or Coveralls). Links to the code coverage results are in the issue. If
people want to see how they interact with PRs, feel free to look at
https://github.com/brettcannon/cpython-ci-test/pull/32 (once the CI run is
finished).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161122/9a572950/attachment.html>

From solipsis at pitrou.net  Wed Nov 23 06:35:46 2016
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 23 Nov 2016 12:35:46 +0100
Subject: [core-workflow] Choosing a code coverage reporter
References: <CAP1=2W6nHxSU9UU6y5oFBaBzZ6RvtSqqZBF+X1pMXV9QH_Tx3Q@mail.gmail.com>
Message-ID: <20161123123546.5a943388@fsol>

On Tue, 22 Nov 2016 20:08:48 +0000
Brett Cannon <brett at python.org> wrote:
> I have both Codecov and Coveralls up and running. If you have an
> opinion/preference, please vote at
> https://github.com/brettcannon/cpython-ci-test/issues/27 using the
> reactions on the appropriate comment (.e.g +1/-1 reaction for Codecov
> and/or Coveralls). Links to the code coverage results are in the issue. If
> people want to see how they interact with PRs, feel free to look at
> https://github.com/brettcannon/cpython-ci-test/pull/32 (once the CI run is
> finished).

There are interesting oddities, such as coverage for the toplevel of
weakref.py:
https://codecov.io/gh/brettcannon/cpython-ci-test/src/065dc67d234b5550fb847b79a963d13b90002787/Lib/weakref.py

I presume weakref is being imported early in the interpreter
initialization process, when coverage hasn't been enabled yet?

Regards

Antoine.



From brett at python.org  Wed Nov 23 14:41:10 2016
From: brett at python.org (Brett Cannon)
Date: Wed, 23 Nov 2016 19:41:10 +0000
Subject: [core-workflow] Choosing a code coverage reporter
In-Reply-To: <20161123123546.5a943388@fsol>
References: <CAP1=2W6nHxSU9UU6y5oFBaBzZ6RvtSqqZBF+X1pMXV9QH_Tx3Q@mail.gmail.com>
 <20161123123546.5a943388@fsol>
Message-ID: <CAP1=2W7ga9baf+6U6qhM01=qMKif8Z=iKUqnT2RFKj49PNmz=g@mail.gmail.com>

Yep, and it's a known issue. If you look at the devinabox README you will
see there are complex instructions o  how to work around it, but since they
are theoretically brittle I didn't bother setting it up.

On Wed, Nov 23, 2016, 03:36 Antoine Pitrou, <solipsis at pitrou.net> wrote:

> On Tue, 22 Nov 2016 20:08:48 +0000
> Brett Cannon <brett at python.org> wrote:
> > I have both Codecov and Coveralls up and running. If you have an
> > opinion/preference, please vote at
> > https://github.com/brettcannon/cpython-ci-test/issues/27 using the
> > reactions on the appropriate comment (.e.g +1/-1 reaction for Codecov
> > and/or Coveralls). Links to the code coverage results are in the issue.
> If
> > people want to see how they interact with PRs, feel free to look at
> > https://github.com/brettcannon/cpython-ci-test/pull/32 (once the CI run
> is
> > finished).
>
> There are interesting oddities, such as coverage for the toplevel of
> weakref.py:
>
> https://codecov.io/gh/brettcannon/cpython-ci-test/src/065dc67d234b5550fb847b79a963d13b90002787/Lib/weakref.py
>
> I presume weakref is being imported early in the interpreter
> initialization process, when coverage hasn't been enabled yet?
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> core-workflow mailing list
> core-workflow at python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20161123/675da248/attachment.html>