[Numpy-discussion] Issue Tracking

[Fernando Perez]

Hi folks,

sorry for not jumping in before, swamped with deadlines...

On Mon, Apr 30, 2012 at 8:14 PM, Jason Grout
<jason-sage at creativetrax.com> wrote:

> I've been pretty impressed with the lemonade that the IPython folks have
> made out of what I see as pretty limiting shortcomings of the github
> issue tracker.  I've been trying to use it for a much smaller project

I think your summary is all very valid, and mostly we've just learned
to live with some of its limitations and hacked around some of them.
Now, one thing that the hyper-simplistic github tracker has made me
think is about the value of over-sophisticated tools, which sometimes
can become an end in and of themselves... Because the tracker is so
absurdly simple, you just can't spend much time playing with it and
may spend more time coding on the project itself.

But having said that, I still think GHI (short for github issues) has
*real* issues that are true limitations, and I keep hoping they'll
improve (though it's starting to look more like unreasonable hope, as
time goes by).

> 1. No key:value pairs for labels (Fernando brought this up a long time
> ago, I think).  This is brilliant in Google code's tracker, and allows
> for custom fields that help in tracking workflow (like status, priority,
> etc.).  Sure, you can do what the IPython folks are doing and just
> create labels for every possible status, but that's unwieldy and takes a
> lot of discipline to maintain.  Which means it takes a lot of developer
> time or it becomes inconsistent and not very useful.

I don't think it takes too much time in practice, but it's indeed a
poor man's substitute for the google system.  And for certain things,
like priority, you *really* want a proper way of saying 'show me all
issues of priority X or higher', which you can't do with labels.

> 2. The disjointed relationship between pull requests and issues.  They

This is the one that pisses me off the most.  It's a real, constant
drag to not be able to label issues and see those labels.  I can't
fathom why on Earth GH hasn't added this, and what bizarre thought
process could possibly have led to PRs being implemented alongside
issues and yet, being hobbled by deliberately *hiding* their labels.
It feels almost as a misfeature written out of spite against the
users.  I fume every time I try to prioritize work on open PRs and
have to do it via post-it notes on the wall because GH decided to
*hide* the labels for PRs from me.  Argh...

> 3. No attachments for issues (screenshots, supporting documents, etc.).
>  Having API access to data won't help you here.

This one doesn't actually bother me in practice at all.  Gist works
perfectly for text snippets, and since they're version-controlled,
it's even better than static attachments. And for images, a simple
imgur upload (many screenshot programs even upload for you) along with
![tag](url) provides even better results: the images are inlined in
the discussion. See for example:

https://github.com/ipython/ipython/issues/1443

So here, I actually *prefer* gist+image urls to an attachment system.
Arguably the externally hosted images could be lost if that server
goes down, so it would perhaps be better to have them hosted at GH
itself.  They might consider that (or simply allowing gists to take
binary uploads).

> 4. No custom queries.  We love these in the Sage trac instance; since we
> have full access to the database, we can run any sort of query we want.
>  With API data access, you can build your own queries, so maybe this
> isn't insurmountable.

Yes, this is doable with a little elbow grease.

> 5. Stylistically, the webpage is not very dense on information.  I get
> frustrated when trying to see the issues because they only come 25 at a
> time, and never grouped into any sort of groupings, and there are only 3
> options for sorting issues.  Compare the very nice, dense layout of
> Google Code issues or bitbucket.  Google Code issues also lets you
> cross-tabulate the issues so you can quickly triage them.  Compare also
> the pretty comprehensive options for sorting and grouping things in trac.

Agreed.  Not a big deal breaker for us, but perhaps we're just living
in blissful ignorance of what we're missing :)

> 6. Side-by-side diffs are nice to have, and I believe bitbucket and
> google code both have them.  Of course, this isn't a deal-breaker
> because you can always pull the branch down, but it would be nice to
> have, and there's not really a way we can put it into the github tracker
> ourselves.

I guess they could add a markdown extension that displays a .diff url
as a real diff...

> That said, it is nice to have code and dev conversations happening in
> one place.  There are great things about github issues, of course.  But
> I'm not so sure, for me, that they outweigh some of the administrative
> issues listed above.

This is the real deal.  In the end, we've learned to live with some of
the GHI limitations and grumble under our breath about others, but
there's genuine benefit to having a smooth flow of information in one
environment.  People get more efficient with the toolchain (I've
learned markdown without even trying, simply from gradually doing
nicer and nicer reports as I use the system), and there's a 'network
effect' at play here.

On Mon, Apr 30, 2012 at 11:16 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:

> I find the trac email notifications pretty good in that regard, although it
> would be nice to have everything in one place. The main issue I have,
> actually, is that github won't send me everything. I want to see every
> posted comment and every commit show up in the mail, including my own
> comments. The RSS feeds seem better for those notifications, but I have two
> feeds from github and they don't show the same things. Maybe we need to put
> together a tracking workflow document to supplement the git workflow
> document.

This also bothers me quite a bit.  I'd like my email thread to be a
full record of the ticket, not just other people's responses.
Furthermore, if you use markdown in an email reply, for some reason
they don't render it on the site anymore.  So I end up using the site
directly most of the time so the issue reads nicely formatted.

>> Can we at least agree to have all the wiki pages and web-site managed by
>> github?     For issue tracking,  I'm very anxious for your and Ralf's

This one I think is a no-brainer, at least the website pages.  For
that, the system using dual repos and a web team on github, like we
do, is simply perfect.  I see no reason whatsoever not to do this.


In summary, I think we made the right decision for IPython with using
GHI, and I do think that the simplicity of the tool does also help in
focusing on writing more code instead of staring at reports of open
issues :)  But I do wish dearly they made improvements to some of the
system's limitations listed above, because several of them are truly
annoying to have to put up with.

As for numpy/scipy, it's hard to say.  The view of people like Ralf,
Pauli, Chuck and others is what matters most here.  If they are going
to get so aggravated by the system's limitations that they don't want
to work on the project, that's not a risk worth taking.

On the other hand, I'm famous for being persnickety about my tools
(Brian says IPython is the embodiment in source code of my untreated
OCD :), and yet I've managed to adapt.  So better souls than me can
probably do it too ;)

But if you do decide to go with GHI, it should be based on what the
system is like *today*, not on the hope that it will get better.
About a month ago they broke label filtering by turning multi-label
filters into an OR operation, which effectively rendered the labels
completely useless.  Despite reporting it multiple times via their
support tracker AND speaking in person at someone from GH, it still
took well over a month or two to fix.  For something so simple and so
essential, I consider that to be atrociously bad response.  So don't
go for GHI on the hope it will get a lot better soon, b/c their recent
record doesn't support a hopeful viewpoint.

tl,dr; GHI is a net positive for IPython, but it does have real
annoyances you can't just wish away.

Best,

f