Hi All, This is a heads up that we have already exceeded our allotment of free time on Cirrus CI. They are giving us a pass this month, but next month they will start enforcing the limits. That will impact both our testing and our releases. We have taken steps to reduce our use of Cirrus, but it could still be a problem, we should have a contingency plan in place for at least the next two months. Thoughts? Chuck.
On Tue, Aug 15, 2023 at 4:19 PM Charles R Harris <charlesr.harris@gmail.com> wrote:
Hi All,
This is a heads up that we have already exceeded our allotment of free time on Cirrus CI. They are giving us a pass this month, but next month they will start enforcing the limits. That will impact both our testing and our releases. We have taken steps to reduce our use of Cirrus, but it could still be a problem, we should have a contingency plan in place for at least the next two months.
Thoughts?
At the current rate, our bill would be around $100/month. Cirrus CI is very useful, and I don't think we should move away from it - having to run 64-bit ARM platforms under QEMU would be quite bad. So I think the contingency plan here should be: just pay the bill. We're not exactly wealthy as a project, but it's not 2015 anymore either - we have funds at https://opencollective.com/numpy, and a monthly income of a few thousand dollars a month (hat tip to Tidelift). So we can easily afford it, and it'll be money well spent. The most annoying thing with non-free things is not the money itself, but the logistics around it and that someone has to be responsible for it. Assuming there's no better idea to avoid paying the bill, and the steering council signs off on paying the bill, I think we can manage that though. Cheers, Ralf
There's a scipy issue on this that discusses how to reduce usage, https://github.com/scipy/scipy/issues/19006. Main points: - at the moment CI is run on PR and on Merge. Convert to only running on PR commits. I've just submitted a PR to do this for numpy. - add a manual trigger. Simple to achieve, but requires input from a maintainer. - reduce wheel build frequency. At the moment I believe they're made every week. However, that decision has to factor in the increased frequency that may be desired as numpy2.0 is worked on.
On Wed, 16 Aug 2023 at 10:51, Andrew Nelson <andyfaff@gmail.com> wrote:
There's a scipy issue on this that discusses how to reduce usage, https://github.com/scipy/scipy/issues/19006.
Main points:
- at the moment CI is run on PR and on Merge. Convert to only running on PR commits. I've just submitted a PR to do this for numpy. - add a manual trigger. Simple to achieve, but requires input from a maintainer. - reduce wheel build frequency. At the moment I believe they're made every week. However, that decision has to factor in the increased frequency that may be desired as numpy2.0 is worked on.
Also, it's significantly more expensive to test on macOS M1 compared to linux_aarch64. The latter isn't tested on cirrus. However, you could use linux_aarch64 as a proxy for general ARM testing, and only run macOS when necessary.
On Wed, Aug 16, 2023 at 5:01 AM Andrew Nelson <andyfaff@gmail.com> wrote:
On Wed, 16 Aug 2023 at 10:51, Andrew Nelson <andyfaff@gmail.com> wrote:
There's a scipy issue on this that discusses how to reduce usage, https://github.com/scipy/scipy/issues/19006.
Main points:
- at the moment CI is run on PR and on Merge. Convert to only running on PR commits. I've just submitted a PR to do this for numpy.
Thanks! We shouldn't be running any CI on merge commits, other than for building and deploying new versions of the docs to https://numpy.org/devdocs/.
- add a manual trigger. Simple to achieve, but requires input from a
maintainer.
I'd like to avoid that, things that require more actions from maintainers tend to be counterproductive.
- reduce wheel build frequency. At the moment I believe they're made every
week. However, that decision has to factor in the increased frequency that may be desired as numpy2.0 is worked on.
We're doing the x86-64 wheel builds in GHA daily now, which is necessary. The aarch64/arm64 ones can be done less frequently, but once a week seems good at least in the run-up to 2.0.
Also, it's significantly more expensive to test on macOS M1 compared to linux_aarch64. The latter isn't tested on cirrus. However, you could use linux_aarch64 as a proxy for general ARM testing, and only run macOS when necessary.
That sounds like a good idea to me. Or at least checking linux_aarch64 first. I like how you configured that for SciPy. Also, now that we're on the topic of CI: I opened https://github.com/numpy/numpy/issues/24410 for a larger overhaul of our GitHub Actions jobs. I put that on the agenda for today's community meeting, but folks interested in CI may also want to comment on that issue. Cheers, Ralf
With regards to scheduling wheel builds (and this may already be completely obvious), but numpy should consider a scheduled build time that comes reasonably before scipy's scheduled build time so that there is a quicker turn-around time with possible breaking changes. If you schedule the numpy wheel builds to be after scipy's, then it could be a whole week before you'd find out about any breakages. Cheers! Ben Root On Wed, Aug 16, 2023 at 7:12 AM Ralf Gommers <ralf.gommers@gmail.com> wrote:
On Wed, Aug 16, 2023 at 5:01 AM Andrew Nelson <andyfaff@gmail.com> wrote:
On Wed, 16 Aug 2023 at 10:51, Andrew Nelson <andyfaff@gmail.com> wrote:
There's a scipy issue on this that discusses how to reduce usage, https://github.com/scipy/scipy/issues/19006.
Main points:
- at the moment CI is run on PR and on Merge. Convert to only running on PR commits. I've just submitted a PR to do this for numpy.
Thanks! We shouldn't be running any CI on merge commits, other than for building and deploying new versions of the docs to https://numpy.org/devdocs/.
- add a manual trigger. Simple to achieve, but requires input from a
maintainer.
I'd like to avoid that, things that require more actions from maintainers tend to be counterproductive.
- reduce wheel build frequency. At the moment I believe they're made
every week. However, that decision has to factor in the increased frequency that may be desired as numpy2.0 is worked on.
We're doing the x86-64 wheel builds in GHA daily now, which is necessary. The aarch64/arm64 ones can be done less frequently, but once a week seems good at least in the run-up to 2.0.
Also, it's significantly more expensive to test on macOS M1 compared to linux_aarch64. The latter isn't tested on cirrus. However, you could use linux_aarch64 as a proxy for general ARM testing, and only run macOS when necessary.
That sounds like a good idea to me. Or at least checking linux_aarch64 first. I like how you configured that for SciPy.
Also, now that we're on the topic of CI: I opened https://github.com/numpy/numpy/issues/24410 for a larger overhaul of our GitHub Actions jobs. I put that on the agenda for today's community meeting, but folks interested in CI may also want to comment on that issue.
Cheers, Ralf
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: ben.v.root@gmail.com
On Wed, Aug 16, 2023, 23:48 Benjamin Root <ben.v.root@gmail.com> wrote:
With regards to scheduling wheel builds (and this may already be completely obvious), but numpy should consider a scheduled build time that comes reasonably before scipy's scheduled build time so that there is a quicker turn-around time with possible breaking changes. If you schedule the numpy wheel builds to be after scipy's, then it could be a whole week before you'd find out about any breakages.
Scipy upped its frequency to daily in the run up to numpy2
participants (4)
-
Andrew Nelson -
Benjamin Root -
Charles R Harris -
Ralf Gommers