(Fun!) Call for testing: Alpha version of the yt data hub
Hi all, This cycle I applied to Amazon Research, and last week I received notice that the grant was accepted (woo hoo!) Using the AWS credits, I've been able to deploy an alpha version of a "data hub" on Amazon EC2. It's nearly fully backed by S3, SimpleDB and EC2 (only user authentication is stored on the instance, which I'd like to change eventually.) The idea behind this is to make an easy way to share *data*, not just images, scripts, and so on. It's not designed to be robust for many years (like a proper archiving solution would be) and it's not designed to be fully generic, but rather it's designed ... as a pastebin for data. You construct a widget, a representation of a data object, and then you can shove it up there and display the data through the widget. Right now it contains widgets for: * 3D Vertices, displayed with WebGL * Variable mesh maps (i.e., the mapserver) * Image collections * Parameter files There are a number of places where it's not yet finished: the vertices and image collections haven't been wrapped into yt proper but as it stands, yt can upload both parameter files and variable mesh (slices, projections) really easily. The Data Hub has been deployed here: https://data.yt-project.org/ and some example scripts are here: https://bitbucket.org/MatthewTurk/yt.hub_examples/ You might get errors about the certificate, which is self-signed, or about loading non-secure content (the XTK source for the 3D models), and you'll *probably* be able to crash the view or get a "Server incorrectly configured" error. It's still pretty early! But I'd like to request that you try hammering on this, try uploading data, and I would really appreciate any help with design, coding, new widgets, etc. The source is at http://bitbucket.org/MatthewTurk/yt.hub/ . As you'll find, it's still a bit hacked together, but I am interested in continuing to refine it, make it look better and nicer, and to ensure that it's maintainable. (If you are interested in this, fork away!) Any feedback would be *greatly* appreciated. I'm pretty excited about using this for collaboration and data sharing! Down the road I could see adding on more functionality like synchronized views, annotations (halos, points of interest, etc), 3D volumes (for phase plots) and on and on and on. In fact, this could be a way to share results with collaborators from a running simulation. Better than feedback, though, would be if you tried it out -- and uploaded some data! -Matt PS The variable mesh maps should work on phones and tablets. :)
Congratulations, Matt! This is really a great step toward the future of collaborative scientific coding. Thank you for doing this! Cameron On 3/23/12 5:17 PM, Matthew Turk wrote:
Hi all,
This cycle I applied to Amazon Research, and last week I received notice that the grant was accepted (woo hoo!) Using the AWS credits, I've been able to deploy an alpha version of a "data hub" on Amazon EC2. It's nearly fully backed by S3, SimpleDB and EC2 (only user authentication is stored on the instance, which I'd like to change eventually.)
The idea behind this is to make an easy way to share *data*, not just images, scripts, and so on. It's not designed to be robust for many years (like a proper archiving solution would be) and it's not designed to be fully generic, but rather it's designed ... as a pastebin for data. You construct a widget, a representation of a data object, and then you can shove it up there and display the data through the widget.
Right now it contains widgets for:
* 3D Vertices, displayed with WebGL * Variable mesh maps (i.e., the mapserver) * Image collections * Parameter files
There are a number of places where it's not yet finished: the vertices and image collections haven't been wrapped into yt proper but as it stands, yt can upload both parameter files and variable mesh (slices, projections) really easily.
The Data Hub has been deployed here:
and some example scripts are here:
https://bitbucket.org/MatthewTurk/yt.hub_examples/
You might get errors about the certificate, which is self-signed, or about loading non-secure content (the XTK source for the 3D models), and you'll *probably* be able to crash the view or get a "Server incorrectly configured" error. It's still pretty early! But I'd like to request that you try hammering on this, try uploading data, and I would really appreciate any help with design, coding, new widgets, etc. The source is at http://bitbucket.org/MatthewTurk/yt.hub/ . As you'll find, it's still a bit hacked together, but I am interested in continuing to refine it, make it look better and nicer, and to ensure that it's maintainable. (If you are interested in this, fork away!)
Any feedback would be *greatly* appreciated. I'm pretty excited about using this for collaboration and data sharing! Down the road I could see adding on more functionality like synchronized views, annotations (halos, points of interest, etc), 3D volumes (for phase plots) and on and on and on. In fact, this could be a way to share results with collaborators from a running simulation.
Better than feedback, though, would be if you tried it out -- and uploaded some data!
-Matt
PS The variable mesh maps should work on phones and tablets. :) _______________________________________________ yt-dev mailing list yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
participants (2)
-
Cameron Hummels -
Matthew Turk