Thanks, Sam! As a quick note, I changed all of the PNG writing to not use tempfile but instead write directly to in-memory, which was a pretty glaring problem waiting to rear its head. I was idly thinking about this yesterday, and after some conversations I've had with Tom and others I think that one could imagine a very cool project coming out of this. The idea would be that the minimum amount of information to do an entire pan-n-scan is really just the 5xN array for a given projection or slice: (px, py, pdx, pdy, z). This is how it currently works -- it simply repixelizes that array for each tile segment, and returns a PNG of that newly created tile. The storage on the backend is really quite small; these 5xN arrays are much more efficient than creating all the possible tiles in advance. They're not free in terms of web-server memory, like the tiles would be, but they're quite inexpensive. I believe that even for the 1024^3 L7 dataset it was on the order of a hundred megs or so. So imagine having a server that lived somewhere that stored a bunch of these 5xN arrays in memory, along with appropriate metadata. Go to the frontend of the server, select one to view, and then having that dataset pop up. Although they aren't in the mapserver command, one can supply annotations and even drawings with the leaflet library. (I've basically just described a number of existing services. Except, ours would be based on simulation data exclusively and would probably be able to scale to having many, many datasets!) You'd have to implement a frontend, a backend that stored these datasets, and likely have some kind of garbage collection -- if the last request for a given dataset was more than N minutes ago, remove it from memory. I think this could be a viable, fun project, and would be really excellent to use as supplemental data for papers and presentations -- particularly for star formation and cosmology, where there are a number of points of interest within relatively high dynamical range. Anyway, if someone is interested in working on this, it could be VERY fun and possibly quite straightforward to implement. I will confess I don't know enough about web apps, server deployment and database backends to move forward on it, but I'd be more than happy to help out with the data transport and visualization aspects, which I think the mapserver already has a good start on. Coming up with money to run this on, say, EC2 is probably also possible; it may even be reasonable to think we could make this a project that people in the community could upload to and use. -Matt On Wed, May 18, 2011 at 1:37 PM, Sam Skillman <samskillman@gmail.com> wrote:
Matt, This is awesome, and I'm definitely a +1 on rolling this into reason in the future. I'm very tempted to throw this up on a big set of displays, and up the window size :). Well done! Sam On Wed, May 18, 2011 at 1:42 AM, Matthew Turk <matthewturk@gmail.com> wrote:
Thanks for the positive feedback, Stephen -- I'm pretty excited about this, too.
As a stopgap before it gets rolled into reason, which may end up being sort of tricky, I have added a "mapserver" command to the yt command line utility. If you update your installation, you can do:
yt mapserver DD0030/DD0030
and it'll spawn on http://127.0.0.1:8080/ , which you can then open in your browser. There are a couple options for this command, too -- axis, field, projection, and weight, which you can see with --help.
-Matt
On Tue, May 17, 2011 at 4:15 PM, Stephen Skory <s@skory.us> wrote:
Hi Matt,
I'd love it if people could download and try it out. If you have yt installed, then you don't need anything other than this repo:
I just gave it a go on a couple datasets and it worked pretty well! The periodicity is handled interestingly - just like Google Maps where you can pan left and right and get the same thing, but up and down cuts off the data. Really neat!
-- Stephen Skory s@skory.us http://stephenskory.com/ 510.621.3687 (google voice) _______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
_______________________________________________ Yt-dev mailing list Yt-dev@lists.spacepope.org http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org