![](https://secure.gravatar.com/avatar/8fd5badb01bbc9389476e9394ab6607e.jpg?s=120&d=mm&r=g)
Hi there, The twistedmatrix deployment is here BTW: http://speed.twistedmatrix.com/ This is an interesting idea (and Codespeed is ace!). I have a few things that Codespeed could help with, although I'm, not sure I could justify paying for them, but certainly this is something that would interest me. One of my undergrads (Sam Giles) used it in his final year thesis and it worked very well for him. A few random thoughts for you: - To be more useful for academics Codespeed would need to generate more statistical information and much better quality charts. Statistical tests on improvements between versions, confidence intervals, etc. would be handy. This piece of work has some very nice ideas: https://github.com/jamesbornholt/plotty This would move Codespeed from being able to provide a basic "smoke test" for performance improvement, to something much more rigorous. I guess not everyone would necessarily need that, but I think it would not do anyone harm to have it. - Table output in LaTeX would also be useful for academic types, as would some way to make the charts more "configurable", or maybe just being able to dump the Python code (+ data) that creates the charts would do, then people could mess around offline. Another option would be to use https://plot.ly/ which gives you the code to generate each chart you create on its API (e.g. click on the "code" tab on the left of this: https://plot.ly/~snim2/43/ ). You could go a step further and have something like an "open my results in LaTeX / Overleaf [ https://www.overleaf.com/devs ] or ShareLaTeX. The big thing here is workflow - how does the user go from pushing a changeset in their code to generating high quality analysis and output for a blog or paper publication or a business pitch? There are several ways to reduce friction there, you can probably think of more ideas than I have. - The style files could do with an update, which probably doesn't matter much to people on this list, but may matter more if you want to commercialise - For any user ease of use is really the key thing, but also flexibility. It is important to be able to get at the data in different formats, but people will probably find uses for Codespeed that you hadn't intended (and I guess not all future users would necessarily be full time software developers), and there are cases where the PyPy way of doing things isn't quite what you want. Here's a simple example use case: maybe instead of tracking the progress of some software project, I want to compare benchmarks for different pieces of hardware. When I push changes to the benchmark code and re-run the analysis I would not necessarily expect the results to change over time (although they might with updated firmware / SDKs, etc) but I might want to add more hardware platforms to the data and more benchmarks to the code. Codespeed doesn't quite match that use case at the moment, but I don't see why it shouldn't be possible to reconfigure it to be useful in that situation. - If you decide not to go down the commercial route, just having a "deploy to Heroku" button [ https://devcenter.heroku.com/articles/heroku-button ] might bring in many more users. Cheers, Sarah On Thu, Mar 12, 2015 at 7:37 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hi Christian
We as PyPy are not generally interested in freemium services (we are poor to start with), however I think there is some potential with codespeed being notoriously hard to deploy. I've seen http://speed.pyston.org/ and http://speed.twistedmatrix.org/ (which is under some URL, not clue where) deployed already and I do know about a few commercial deployments.
Cheers, fijal
On Thu, Mar 12, 2015 at 5:10 AM, Christian Peel <chris.peel@ieee.org> wrote:
Hi all,
I'm working with the author of Codespeed (Miquel Torres) to determine whether there would be interest in a hosted service which provided information similar to those from Codespeed as used at speed.pypy.org.
Our belief is that pypy.org would be interested in using a freemium hosted service with features similar to (or better than!) those in codespeed and which was as easy to set up as Travis CI; can you confirm this? What additional features would you want in such a service? Are any of you using Codespeed commercially?
My best
Chris
-- chris.peel@ieee.org
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
-- Dr. Sarah Mount, Senior Lecturer, University of Wolverhampton website: http://www.snim2.org/ twitter: @snim2