Re: [Mailman-Developers] Rolling releases for Container Images and Funding Campaign for Mailman

On 11/08/2017 12:03 AM, Abhilash Raj wrote:
You either use released versions, or make sure that the master branches are always compatible. You can't have both with the current development model.

On Nov 8, 2017, at 04:38, Simon Hanna <simon@hannaweb.eu> wrote:
Is there a free (software) equivalent to Quay? Caveat: I don’t know anything about Quay, so I don’t know what services and advantages it provides.
The funds all go to the GNU Mailman project’s FSF donation fund. A portion of every donation goes to the FSF, and the Mailman Steering Committee ultimately decides what to do with what’s left in our account. In the past we’ve used it sponsor GSoC students coming to Pycon sprints.
There are folks working on Debian packages. I’m not aware of similar efforts for other distro ecosystems. But I don’t think it’s all or nothing, and it does make sense for us (the GNU Mailman project) to support containers to help with adoption, experimentation, and deployment, while working with distro volunteers to package the code up for those platforms.
Cheers, -Barry

On Wed, Nov 8, 2017, at 04:38 AM, Simon Hanna wrote:
For those who don't know about it, Quay is a container registry offering from CoreOS. The main reason why I chose to move to Quay, from DockerHub that we are currently using, is security.
Some parts of both Dockerhub and Quay are open source and some aren't. There isn't a very clear boundary/transparency in what technologies they use. Both are free to use for open source projects. However, Quay provides an improved security model with RBAC to push images from build server, without having to put my account password on the build server. It also includes a (open source) security scanner for images, which looks for vulnerabilities in packages included in the image so that I can re-build them if one is found.
There are open source registries available, most popular of which I know is provided by Gitlab. However, I feel Quay is better for my current workflow, which involves lots of bash scripts for building and testing images on public CI system for rolling releases.
Given more free cycles, I do intend to have a better build pipeline and possibly use Gitlab's own registry to distribute images.
Not exactly, I still believe all our git-heads should be compatible with released (i.e. stable) versions of dependencies, regardless of the fact that we maintain those dependencies.
Also, there is at least one test per-project which tests with git-heads of dependencies we maintain, so that we know if we make any incompatible changes.
I am not sure what in the current development model prevents us to do both?
The story for container images is different than testing with git-heads. We need better integration testing for the entire suite to be stable in conjunction, rather than independently. In fact, containers might be the perfect way to actually do the integration testing.
https://www.google-melange.com/archive/gsoc/2015/orgs/gnu_mailman/projects/b...
Like Barry mentioned, it doesn't have to be all-or-nothing. There is work being done on distro packaging for Debian and I am ready to help others trying to do the same for other distros!
Systemd-nspawn isn't really a production grade tool to use containers. I am not sure about how much it adheres to the OCI specs, but if it does, it should not matter which deployment tool you are using with images.
I wouldn't mind adding configurations for systemd-nspawn in the documentation, contributions are most welcome.
I didn't say they will be served from the same core ;-)
Yes, the Core's API can be further improved to be more efficient. But that doesn't prevent us from scaling. I understand there are issues around multiple instances of containers, but I have ideas to get around most of them.
-- Abhilash Raj maxking@asynchronous.in

On Nov 8, 2017, at 04:38, Simon Hanna <simon@hannaweb.eu> wrote:
Is there a free (software) equivalent to Quay? Caveat: I don’t know anything about Quay, so I don’t know what services and advantages it provides.
The funds all go to the GNU Mailman project’s FSF donation fund. A portion of every donation goes to the FSF, and the Mailman Steering Committee ultimately decides what to do with what’s left in our account. In the past we’ve used it sponsor GSoC students coming to Pycon sprints.
There are folks working on Debian packages. I’m not aware of similar efforts for other distro ecosystems. But I don’t think it’s all or nothing, and it does make sense for us (the GNU Mailman project) to support containers to help with adoption, experimentation, and deployment, while working with distro volunteers to package the code up for those platforms.
Cheers, -Barry

On Wed, Nov 8, 2017, at 04:38 AM, Simon Hanna wrote:
For those who don't know about it, Quay is a container registry offering from CoreOS. The main reason why I chose to move to Quay, from DockerHub that we are currently using, is security.
Some parts of both Dockerhub and Quay are open source and some aren't. There isn't a very clear boundary/transparency in what technologies they use. Both are free to use for open source projects. However, Quay provides an improved security model with RBAC to push images from build server, without having to put my account password on the build server. It also includes a (open source) security scanner for images, which looks for vulnerabilities in packages included in the image so that I can re-build them if one is found.
There are open source registries available, most popular of which I know is provided by Gitlab. However, I feel Quay is better for my current workflow, which involves lots of bash scripts for building and testing images on public CI system for rolling releases.
Given more free cycles, I do intend to have a better build pipeline and possibly use Gitlab's own registry to distribute images.
Not exactly, I still believe all our git-heads should be compatible with released (i.e. stable) versions of dependencies, regardless of the fact that we maintain those dependencies.
Also, there is at least one test per-project which tests with git-heads of dependencies we maintain, so that we know if we make any incompatible changes.
I am not sure what in the current development model prevents us to do both?
The story for container images is different than testing with git-heads. We need better integration testing for the entire suite to be stable in conjunction, rather than independently. In fact, containers might be the perfect way to actually do the integration testing.
https://www.google-melange.com/archive/gsoc/2015/orgs/gnu_mailman/projects/b...
Like Barry mentioned, it doesn't have to be all-or-nothing. There is work being done on distro packaging for Debian and I am ready to help others trying to do the same for other distros!
Systemd-nspawn isn't really a production grade tool to use containers. I am not sure about how much it adheres to the OCI specs, but if it does, it should not matter which deployment tool you are using with images.
I wouldn't mind adding configurations for systemd-nspawn in the documentation, contributions are most welcome.
I didn't say they will be served from the same core ;-)
Yes, the Core's API can be further improved to be more efficient. But that doesn't prevent us from scaling. I understand there are issues around multiple instances of containers, but I have ideas to get around most of them.
-- Abhilash Raj maxking@asynchronous.in
participants (3)
-
Abhilash Raj
-
Barry Warsaw
-
Simon Hanna