As part of the importlib_resources skunkworks project Brett and I have been working on (just announced), we’ve also put together a nice Docker image that we’re using for our automated testing. This image is based on Ubuntu 16.04 and provides the latest stable releases of Python 2.7, and 3.4-3.6, along with a mostly up-to-date git checkout of master, currently Python 3.7 of course. Once 3.7 is released to beta, we intend to track its release tarballs too.
We also install a few other commonly needed tools, like pip, git, unzip, wget, mypy, and tox.
Here’s the project repo:
https://gitlab.com/python-devs/ci-images
Huge shout out to Abhilash Raj who helped us clean up and compactify the image, and also for setting up automated publishing to quay.io. In case you weren’t aware, Abhilash is who I passed GNU Mailman project leadership to, so he has a lot of Python experience, and a ton of expertise in the image space. He’s an amazing amount of work to improve the quality of this image!
Brett and I want to promote this more widely within the Python community as the “official Python Docker image” that projects can use in their own testing environments, or base their own images on it. We wanted to give you guys a heads up first to get your feedback, and maybe thoughts on the best places to promote this, e.g. on the python.org website or other places.
We welcome your participation too of course! I’m happy to give write access to the project to any Python committer who wants to help.
Cheers, -Barry, Brett, Abhilash
On Dec 7, 2017, at 13:00, Barry Warsaw barry@python.org wrote:
He’s an amazing amount of work to improve the quality of this image!
Um, let me rephrase :)
He’s done an amazing amount of work to improve the quality of this image!
-Barry
On Thu, 07 Dec 2017 13:00:31 -0500, Barry Warsaw barry@python.org wrote:
Brett and I want to promote this more widely within the Python community as the “official Python Docker image” that projects can use in their own testing environments, or base their own images on it. We wanted to give you guys a heads up first to get your feedback, and maybe thoughts on the best places to promote this, e.g. on the python.org website or other places.
Well, the place to get a python Docker image listed would be https://hub.docker.com/_/python/. That's the first google hit for python docker, and it already has a host of images available. How does yours differ from those? It sounds like it is by having multiple versions and tox so you can test your library/ap against multiple Python versions using a single container. That does sound useful :)
But, be careful with names. "official Python Docker image" sounds like what one would use when one wants to *run* a python application in a docker container, which is what the images that are currently listed on the page mentioned above seem directed at.
--David
[Continuing to CC Abhilash, who is not on this list. -B]
On Dec 7, 2017, at 14:36, R. David Murray rdmurray@bitdance.com wrote:
On Thu, 07 Dec 2017 13:00:31 -0500, Barry Warsaw barry@python.org wrote:
Brett and I want to promote this more widely within the Python community as the “official Python Docker image” that projects can use in their own testing environments, or base their own images on it. We wanted to give you guys a heads up first to get your feedback, and maybe thoughts on the best places to promote this, e.g. on the python.org website or other places.
Well, the place to get a python Docker image listed would be https://hub.docker.com/_/python/. That's the first google hit for python docker, and it already has a host of images available. How does yours differ from those? It sounds like it is by having multiple versions and tox so you can test your library/ap against multiple Python versions using a single container. That does sound useful :)
There are several differences. Probably the two biggest are that 1) the ones on docker hub are not maintained by *us* but instead by “the Docker Community”; 2) AFAICT, the ones on docker hub aren’t development focused images, so they only have one single version of Python installed. For any Python-based project that wants to e.g. run CI against multiple versions of Python, that isn’t helpful (and hence why we built this one). Our image is bigger, but more generally useful to library and application developers, rather than say, application deployment users.
But, be careful with names. "official Python Docker image" sounds like what one would use when one wants to *run* a python application in a docker container, which is what the images that are currently listed on the page mentioned above seem directed at.
That’s good feedback! I want to be clear about the purpose of this image, both in that it’s blessed and maintained by us, and that its focus is on the Python library and application developer (primarily for testing purposes).
So maybe: Python.Org Official Multiversion Development Docker Image Doubleplus Good!
:)
Seriously, audience and naming are important to get right.
-Barry
On 2017-12-07 22:28, Barry Warsaw wrote:
That’s good feedback! I want to be clear about the purpose of this image, both in that it’s blessed and maintained by us, and that its focus is on the Python library and application developer (primarily for testing purposes).
So maybe: Python.Org Official Multiversion Development Docker Image Doubleplus Good!
Why is there 'Docker' in the name? Is the container image restricted to Docker runtime or can I use it with lxc, systemd-nspawn, rkt, or any other container runtime, too?
There are likely legal issues with the name, too. After all 'Docker' is a registered trademark of Docker, Inc [1]. I'd be careful and call it 'Python.Org Official Multiversion Development Supercalifragilistic Container Image' instead.
Christian
On 2017-12-07 19:00, Barry Warsaw wrote:
As part of the importlib_resources skunkworks project Brett and I have been working on (just announced), we’ve also put together a nice Docker image that we’re using for our automated testing. This image is based on Ubuntu 16.04 and provides the latest stable releases of Python 2.7, and 3.4-3.6, along with a mostly up-to-date git checkout of master, currently Python 3.7 of course. Once 3.7 is released to beta, we intend to track its release tarballs too.
We also install a few other commonly needed tools, like pip, git, unzip, wget, mypy, and tox.
Here’s the project repo:
https://gitlab.com/python-devs/ci-images
Huge shout out to Abhilash Raj who helped us clean up and compactify the image, and also for setting up automated publishing to quay.io. In case you weren’t aware, Abhilash is who I passed GNU Mailman project leadership to, so he has a lot of Python experience, and a ton of expertise in the image space. He’s an amazing amount of work to improve the quality of this image!
Brett and I want to promote this more widely within the Python community as the “official Python Docker image” that projects can use in their own testing environments, or base their own images on it. We wanted to give you guys a heads up first to get your feedback, and maybe thoughts on the best places to promote this, e.g. on the python.org website or other places.
We welcome your participation too of course! I’m happy to give write access to the project to any Python committer who wants to help.
Shiny! You'll get extra bonus points for not running as root. :)
I'm curious, what is the reason of compiling CPython yourself? Ubuntu has the deadsnakes project. Fedora has packages for Python 3.3, 3.4, and 3.5.
Could I convince you to put some builds of OpenSSL and LibreSSL into the container, too? Ubuntu 16.04 has only OpenSSL 1.0.2. A while ago I added a script to CPython that downloads, compiles and installs multiple versions in a shared directory (../multissl relative to cpython checkout). The script can be used to run the SSL test suites (ssl, asyncio, urllib, smtp, ...) against all installed versions.
https://github.com/python/cpython/blob/master/Tools/ssl/multissltests.py
Regards, Christian
On Dec 7, 2017, at 19:34, Christian Heimes christian@python.org wrote:
Shiny! You'll get extra bonus points for not running as root. :)
Don’t forget, there’s a bug tracker you can submit requests to. <wink>
I'm curious, what is the reason of compiling CPython yourself? Ubuntu has the deadsnakes project. Fedora has packages for Python 3.3, 3.4, and 3.5.
I really want clean builds of upstream tarball releases. Debian/Ubuntu for sure, and I would bet Fedora too, has downstream deltas applied, so if something fails there, how do you know it’s because of your package or something the distro added? Note that we do install the build dependencies for the Pythons so they’ll link against platform libraries and such.
Could I convince you to put some builds of OpenSSL and LibreSSL into the container, too?
You can totally file an issue so we can discuss it. :)
https://gitlab.com/python-devs/ci-images/issues
Cheers, -Barry
On 8 December 2017 at 13:14, Barry Warsaw barry@python.org wrote:
On Dec 7, 2017, at 19:34, Christian Heimes christian@python.org wrote:
Shiny! You'll get extra bonus points for not running as root. :)
Don’t forget, there’s a bug tracker you can submit requests to. <wink>
I'm curious, what is the reason of compiling CPython yourself? Ubuntu has the deadsnakes project. Fedora has packages for Python 3.3, 3.4, and 3.5.
I really want clean builds of upstream tarball releases. Debian/Ubuntu for sure, and I would bet Fedora too, has downstream deltas applied, so if something fails there, how do you know it’s because of your package or something the distro added?
Yeah, I'd advise the same: the CPython compatibility testing images should provide unpatched builds. That way, checking against the upstream builds becomes a first step in triage for problems encountered with redistributor builds: if a problem only shows up in the distro version, then distro specific patches would be the first place to look.
Cheers, Nick.
P.S. This thread would probably be better located on python-dev :)
On Dec 7, 2017, at 19:00, Christian Heimes christian@python.org wrote:
Why is there 'Docker' in the name? Is the container image restricted to Docker runtime or can I use it with lxc, systemd-nspawn, rkt, or any other container runtime, too?
I honestly don’t know, and I’ve only used it with Docker, but there is the Open Container Initiative to promote standards, so maybe it can.
https://www.opencontainers.org/about
There are likely legal issues with the name, too. After all 'Docker' is a registered trademark of Docker, Inc [1]. I'd be careful and call it 'Python.Org Official Multiversion Development Supercalifragilistic Container Image' instead.
Yeah, that’s a good suggestion. Well, maybe without the Potential Poppins Infringement.
(And yes, I’m going to follow up on python-dev now.)
-Barry
-
Barry Warsaw -
Christian Heimes -
Nick Coghlan -
R. David Murray