[scikit-image] A request for more frequent releases

[Mark Harfouche]

 Hi Egor, Stephan,

1. I definitely commend Matthew Brett on his work. My suggestion would be
move his infrastructure to run on every PR rather than getting rid of his
infrastructure.
2. (a) Too expensive in terms of workforce:

This is definitely true with the current infrastructure. Simplifying this
is non-trivial and would need to be a focused effort.
This discussion aims to find a path toward that goal.
I thank everybody for their input and suggestions, and most importantly,
for taking the time to respond to this email.

(b) discounts the release value
Fast releases are probably immensely valuable to the people that raised the
underlying issues.

Taking an example from a different library:
A one line fix in OpenCV I submitted this week makes it possible for me to
pass arrays directly to PIL
https://github.com/opencv/opencv/pull/12465
With the current release cycle of OpenCV, it may be at least 1 month (and
up to 6!) until I see that fix propagate across different computers.
Now I either have to recompile OpenCV myself (which is difficult) or just
not use the feature I need.
scikit-image is easier to compile (or get from the wheelhouse) but those
are all still additional barriers that you have to communicate with your
collaborators.

(c) reduces their noticeability
People tend to notice only the features they need. I my projects, I don't
always have a ">=" for all dependencies because I can often "just use any
version that comes from pypi/conda-forge/anaconda".
Sometimes even enabling the `conda-forge` channel is difficult for
beginners.

(d) very little number of PRs per release
Once again, even a single PR can be of immense value (eg. float32s passing
through unchanged in `img_as_float`-- I wouldn't know how to change that
functionality via monkey patching :/ )

Stephan,

Daily wheel builds are fine, unfortunately, I have a dependency in my
pipeline on opencv, and as such I am stuck on conda.
I think you were working on getting a shorter url for the wheel house, I
think that is great!

Regarding experimenting. I think this definitely needs to be included in
the discussion. Thanks for bringing it up.

I think I'll try to summarize your other points in focused issues on github
as Juan suggested.

Mark

On Tue, Sep 11, 2018 at 4:47 PM Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> On Tue, 11 Sep 2018 23:16:29 +0300, Egor Panfilov wrote:
> > - For the release cycle, I'd, however, agree with you only partially.
> From
> > my standpoint, having weekly/bi-weekly/montly releases (a) is too
> expensive
> > in terms of workforce, (b) discounts the release value and significantly
> > reduces their noticeability, and (c) doesn't really make a lot of sense.
> In
> > my experience, every 4 weeks only a very little number of significant PRs
> > are developed and merged. Having said that, I believe, a reasonable cycle
> > would be something like 1 release / 2-3 months. Once we see a progression
> > in the contribution activity, we can consider discussing further changes
> in
> > the schedule.
>
> In an ideal world, we would be ready to punch out a release at a day's
> notice, but would probably only release every three months or so.  A
> release is not merely a tagging of the current master, but also a
> statement about the readiness of the code it contains.  E.g., we are
> more lenient about merging experimental features early on in the release
> cycle, and sometimes modify or even remove APIs that we feel are not
> mature enough.
>
> Mark, I wonder if some of your issues can be addressed by daily wheel
> builds (which we already have)?  Then you can point pip to our Rackspace
> wheels container, and install from there.
>
> The 32-bit issues mentioned were long known, because Matthew builds for
> that platform daily, and reported the failures a long time ago.  It has
> not been fixed because no one has taken the time to investigate
> thoroughly (perhaps not that many developers care about 64-bit?).
>
> I fully support attempts to reduce the burden of turning `master` into a
> release.
>
> One such a proposed change was versioneer, but I recall from Mark's
> previous PR (https://github.com/scikit-image/scikit-image/issues/3144)
> that it introduced quite a bit of complexity for little gain.  I'm not
> sure why the current versioning is considered troublesome; it is stored
> in `skimage/__init__.py`, and is reused everywhere else.
>
> W.r.t. listing new features for a release, Nathaniel's suggestion of
> Towncrier looks sensible, and helps to avoid conflicts.  On the other
> hand, the more tools we use, the higher the overhead to contributing.
> So, is it really a big problem to update the relevant text files?  If
> that process is not documented clearly enough, that is something we can
> address right now.
>
> W.r.t. adding the wheel building machinery into the skimage source repo,
> I'm not sure I see how that would be useful, or why that would be a good
> fit.
>
> W.r.t. backports to 0.14.x—since we no longer support 2.7 on master,
> this really is the only way to give 2.7 users access to new features.
> When a backport is easy to do, what motivation do we have for witholding
> it?
>
> Thanks for your thoughts!
> Stéfan
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20180911/589b7c98/attachment-0001.html>