[Python-Dev] API for bugs.python.org or some statistic about lib/module merge rate?

[Wes Turner]

On Monday, April 24, 2017, Louie Lu <me at louie.lu> wrote:

> Hi all,
>
> thanks for all your reply, since roundup didn't contain JSON REST API, and
> I'm not sure did roundup robot have deal with "patched but still open"
> situation, thus I do some change to bpo-mergerate and have some result.
>
> GitHub repo: https://github.com/lulouie/bpo-mergerate
> Demo site: https://bpo-mergerate.louie.lu
>
> Now bpo-mergerate using SQLite database and no more CSV file, and an
> update.py script was added, on my host, it will automatically update the
> bpo patch status hourly.
> For the site, it is now using ajax and can deal with fuzzy search.
>
> The merge rate calculate is now find-grained to per-file, and will
> automatically count the prefix merge rate, for example:
>
> Searching: Lib/test
> Result:
>     path                                     merged  open  total    rate
>     /Lib/test                                   3013  1359   4372  68.916
>     Lib/test/test_ssl.py                  60      15       75      80.000
>     Lib/test/test_inspect.py           56      12       68      82.353
>
>
> I think the site function is complete now, welcome for any suggestion.
> (maybe like speed.python.org make a timeline for the merge rate?)
>

Nice work.

Additional per-file factors worth visualizing:

- change in test coverage
- change in relevant docs.rst
- build job ids (which patch on which arch)
- benchmark timings
- git log --stat

Gource is a pretty cool way to visualize file changes by author over time:
http://gource.io

"Evolution of CPython (gource visualization)"
https://youtu.be/tZVG57xFwMk


Is there something like `hg churn` for git? I haven't had need; but just
found:

- https://www.mercurial-iscm.org/wiki/ChurnExtension
  -
https://gist.github.com/aras-p/8289222 (cheat sheet)
  Does ChurnExtension work with hg-git?

- https://github.com/arzzen/git-quick-stats
  https://github.com/arzzen/git-quick-stats/blob/master/git-quick-stats
(bash script)

- https://github.com/IonicaBizau/git-stats
  Json?

JSON-LD with the file path as the resource @id really would be great (and
mergeable from various build artifacts):
- https://www.google.com/search?q=elasticsearch+git+log
- westurner/pyrpo canould write streaming JSON from `git log` but not yet
`git log --stat`
- {SQLAlchemy(SQLite), } > JSON-LD reports with @id URIs for {source, docs}
file paths


> Best Regards,
> Louie.
>
> 2017-04-24 12:08 GMT+08:00 Nick Coghlan <ncoghlan at gmail.com
> <javascript:_e(%7B%7D,'cvml','ncoghlan at gmail.com');>>:
>
>> On 21 April 2017 at 06:04, Wes Turner <wes.turner at gmail.com
>> <javascript:_e(%7B%7D,'cvml','wes.turner at gmail.com');>> wrote:
>> > Roundup has an xmlrpc API:
>> >
>> > - https://sourceforge.net/p/roundup/code/ci/default/tree/round
>> up/xmlrpc.py
>> > - https://sourceforge.net/p/roundup/code/ci/default/tree/test/
>> test_xmlrpc.py
>>
>> There's also a pending patch to add a HTTPS+JSON REST API to our
>> Roundup instance:
>> http://psf.upfronthosting.co.za/roundup/meta/issue581
>>
>> Definite +1 for the general idea of collecting and publishing these
>> kinds of statistics, though - I can see them being quite useful to a
>> variety of audiences.
>>
>> Cheers,
>> Nick.
>>
>> P.S. The upstream Roundup project is also interested in offering that
>> capability by default: http://issues.roundup-tracker.org/issue2550734
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com
>> <javascript:_e(%7B%7D,'cvml','ncoghlan at gmail.com');>   |   Brisbane,
>> Australia
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170424/c40bc642/attachment.html>