So to address the enzo testing side of things:
Sam, do you mean, make a gold standard prior to making changes, then after
changes test against the previously-made gold standard? This addresses the
issues of seeing if your changes break the code, but what if a bad
changeset somehow gets into the codebase? I like having a way of comparing
long-term (ie after several changesets) to see if the gold standard has
drifted substantially, say from changeset 1000 to 1100. Or perhaps there
is a mechanism in your new jenkins instance that precludes this? Is
jenkins already being run on every PR to make sure we're not merging in bad
code that breaks the test suite relative to the previous gold standard? I
think what kacper has done in yt to make this happen is one of the major
successes of our development process, but it seems like it's a lot of work
to make it happen.
Another concern I have is that the defaults right now in the enzo testing
suite give you a bunch of errors because they're looking for the external
gold-standard. So to a new user, even one who has read through the
documentation on how to use the test suite, it just appears like enzo is
broken and failing on a bunch of fronts. At the very least if we keep this
model, we should make sure that it doesn't default to looking for a
non-existent gold standard and that the docs are updated to reflect this.
Cameron
On Thu, Jul 17, 2014 at 7:39 AM, Sam Skillman
I'll just mention that the enzo tests are set up such that having a public gold standard is not really a priority. It's best to just create a local gold standard and test against that.
On Thu, Jul 17, 2014 at 7:23 AM, Matthew Turk
wrote: On Thu, Jul 17, 2014 at 9:19 AM, Cameron Hummels
wrote: Thanks for the update, Kacper. I guess I didn't realize it was just a man-hour thing.
It definitely is. We had to take it down and migrate it because of funds (and it was to the tune of a couple hundred a month, but that could come way down), but we haven't had the opportunity to bring it up because it will take time and effort to make it do what we want it to do.
These aspects of it were around before:
* Answer tests -> this has been switched over, as Kacper noted * Project indexing -> Britton's going to move this to a repository system * Image and notebook sharing -> This will probably use SEEDME, but it's a person-hour thing * Data widgets -> These were hardly used, but eventually we will try to bring them back up
If anybody wants to try to help out, that would be great. I'm not going to be able to work on this for some time. It'll require some web work, some infrastructure management, and probably some patience!
-Matt
Cameron
On Thu, Jul 17, 2014 at 2:08 AM, Kacper Kowalik <
wrote:
On 07/17/2014 07:28 AM, Cameron Hummels wrote:
Hey everyone,
Is there a timeline for the yt hub coming back up? I believe the
hold
up had been the fee of hosting the hub, which I think was being covered by Matt's personal funds. Is there ever any hope of getting some dedicated funds to support this in the future? If not, I'm happy to chip in some money to help this get back up. It seemed like a great resource both for hosting datasets and projects tangentially related to yt. Also, I
xarthisius.kk@gmail.com> think
it was the location of the enzo test gold standard, right?
Anyway, I just wanted to see if there was a timeline for it coming back or if was out for the foreseeable future, or if we could help.
Cameron
Hi Cameron, I think at this point all the required infrastructure is available, so it's not a matter of funds, but rather time. The original hub code needs to be rewritten in some parts to fit into new environment.
Some of this work has been already done, e.g. Matt ported answer tests' storage to rackspace[1], there's http://nbviewer.yt-project.org running.
Additionally, there's a very interesting project that we are planning to use wrt datasets sharing: seedme.org. I'm going to adapt existing commandline client in order to make 'yt upload_notebook' work again. However, my time is very limited at the moment and I don't expect it to change before September.
Cheers, Kacper
[1] https://bitbucket.org/yt_analysis/yt/pull-request/832
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-- Cameron Hummels Postdoctoral Researcher Steward Observatory University of Arizona http://chummels.org