On 2018-09-07 01:58, Glyph wrote:

...

On Sep 5, 2018, at 5:44 AM, Kyle Altendorf <sda@fstab.net> wrote:

On 2018-09-05 03:25, Glyph wrote:

...

...

On Aug 29, 2018, at 10:13 AM, Kyle Altendorf <sda@fstab.net> wrote: wheel links: https://github.com/twisted/twisted/pull/1051#issuecomment-416743261 (and next comment) Now that I've got the wheel builds happening I figured it'd be good to try them out on 'real' machines. Turns out we get a failure on twisted.cred.test.test_strcred.SSHCheckerTests.test_isChecker for at least the two checks I've done so far (Windows and OSX). I haven't done more than a cursory look at that yet, but it's on the list to understand and resolve. More testing would of course be welcome. Real world, just trial Twisted's own tests, whatever would be appreciated if you are interested. failures with wheels: https://github.com/twisted/twisted/pull/1051#issuecomment-416977723 It's not clear to me that we have any test environments that accurately test what happens after you install from PyPI. If we could build these wheels and then actually run tests on installing them that would be a big step forward in test fidelity! But given the current state of the art, I'd be happy if they just got built from the same code that passed our tests in existing configurations.

cibuildwheel does have an option to provide a test command. If we did that would we replace the existing tests or supplement? If supplementing that will presumably be a significant increase in build times. Especially if we did it for all wheels.

Definitely replace. What would you want to even supplement them with?

I was referring to supplementing the existing src tests with the wheel tests.

...

So, how much of a build time hit do we want to take?

I was thinking that for starters here the goal would just be build automation, independent of trying to get a more realistic test path; only doing these builds when pushing a relevant tag. If it's easy to just switch over everything to this then... great, I guess? :) But let's not gild the lily here.

Alrighty, another PR for swapping over tests from src to wheels. I would tend to always build the wheels. It's easier for people to test intermediate releases for bugs. It makes a release 'less special' so there is less likely to be a latent issue with the process or artifacts. Etc. But, let's see what the build time hit is. Travis: 4 minutes, about half the time of a single version test job AppVeyor: 7 minutes, about the same as a single version test job Circle OS X: 9 minutes, a bit over a single version test job Circle Linux: 5 minutes, a bit under a single (OS X) version test job Ok, so for about the cost of testing a single Python version we can get wheels. Especially if we want to switch over to testing wheels instead of source, I'd plan on building wheels for every commit.

...

but sure, I didn't mean to argue against automation.

Could we do a test run by uploading wheels to, say, https://test.pypi.org <https://test.pypi.org/> first?

Shall the process be build wheel, upload to test PyPI, install from test PyPI, test... :] But sure, I could see ending up with a test PyPI upload at least for a tag/release. Could technically do it for every build but that seems excessive, even given my comments above about exercising the process more often rather than less.

...

Here's one spot I did automatic deployment off of Travis to PyPI.

https://github.com/altendky/gitignoreio/blob/aeeb9546b6f27f411b191699d3625b4...

Pretty straightforward though I think it does the wheel build which isn't what we'd really want.

That's pretty cool. I assumed you'd have to write a 'twine' command - Travis just has a 'deploy' provider called "pypi"? What's actually happening behind the scenes there, I wonder!

(Also, you should probably upload an sdist as well as a bdist_wheel; there are cases where wheels aren't always what you want.)

Yeah, it's cool but then you have to know what is behind it... for each CI host... and how to make it use the files you want... So I'd tend towards a twine call. I made this silly little script the other day to handle this in another repo. https://github.com/altendky/pyqt5-tools/blob/master/twineontag.py

...

I'd have to check on how to deploy existing wheels as well as those from OS X and Windows. I'd suggest a separate PR? I mean maybe it'd be easier to get one PR reviewed than two but that doesn't seem like a great reason to mash things together.

Smaller PRs are always easier to review - they just take more latency to build :).

Ok, I'll make separate tickets for: * Testing wheels vs. src * Automatic deployment to PyPI Cheers, -kyle