eric jones wrote:

Right. The Kiva API is DisplayPDF and therefore vector based. That won't change. The PDF backend will definitely stay, and I would like to see PS/EPS and SVG backends built (thanks for the PIL backend Dave).

The SVG backend is most of the way there, and PS/EPS will be easy -- I may get to them over the weekend. PS/EPS and SVG all have the problem that Chaco wants to measure font widths to figure out "by itself" where to place left-aligned text. Font width computations aren't available to "pure" PS/EPS/SVG/PDF. The PDF backend works because reportlab ships with a "scary" database of glyph widths. I don't particularly like that approach, since it doesn't scale well (if you want to use any but the standard 14 fonts, you're SOL), puts a significant memory burden on the app, but most of all, I think it's unnecessary in most cases. It would be possible to avoid the need to ask for text width for the sake of drawing labels in many back-ends -- however, I realize that Chaco needs to figure out dimensions of things for its layout engine -- not just to draw frames around titles, but to figure out how big graphs are, etc. What I'll probably do is to rip out the fontdata and string-width-computation code out of reportlab and make it more standalone. With some database compression, it can probably be fairly lightweight. That way any backend which uses the standard fonts can get font widths without having to load the fonts into memory -- obviously not relevant for FreeType-based backends, since they need the fonts to produce their output, but useful for any "delayed rendering" formats. [...]

Building a new Kiva backend is then as simple as learning how to blit this thing into the screen for a new toolkit.

Yup.

The GUI interaction portion of Chaco might be significantly refactored when we go down this road also to make porting that part of Chaco much easier also. It may not even be any better to keep a OpenGL version around since we can blit anti-grain images into it just as easily as other APIs. I guess this decision will depend on speed.

Right -- OpenGL has hardware acceleration at least on some platforms and is the result of hundreds of man-years of optimization and testing. It'll be a while before anti-grain matches that, although anti-grain's 2-d model is a better fit for Kiva, and probably benefits from shortcuts which OpenGL can't afford.

The only potential downside that I see is if anti-grain is slower at drawing than the GUI toolkits. Anti-grain looked plenty fast to me in the demos I have looked at -- it looked just as fast as GDI+ (windows new path based rendering API), but we'll just have to wait and see on this one.

Right -- the demos that I tried, AFAICT, only computed once -- everything else was presumably cached. Regardless, having more backends is only for the good -- it helps define the API better, provides portability (if anti-grain isn't available when creating a png, use PIL instead, etc.). If I didn't have such an allergy to C++, I'd probably have volunteered to help w/ the agg backend =). --david

eric jones wrote:

Yeah. Getting font sizes is an absolute must to layout graphs that look any good. Otherwise axis labels and other text will be running over graph lines, etc.

Axis labels aren't the issue, though -- you can delegate the positioning of a text element to the back-end. But title boxes etc. are an issue.

So this same fontdata code will be usable across all the PS/EPS/SVG/PDF backends? Will SVG need different data than the others?

Correct. Although since I spoke, I cheated and simply imported the reportlab module and delegated the computation to it. Repackaging the code is still something we should do, but now it's just a packaging requirement, which motivates me much less than getting good-looking graphs =). It also has maintenance implications -- some of that code deals with font encoding issues which I know little about. [Aside: reportlab is way overkill for a PDF backend for Kiva -- if that's an issue, know that you could probably do a PDF backend in 200 lines + the font sizing code & data].

It amazes me that, with all the advanced capabilities Kiva will have (anti-aliased rendering, transparency, etc.), rendering text and dealing with fonts is the part that most worries me. FreeType is only half the battle on bitmap systems -- finding/handling fonts across platforms is the other. On vector systems, it sounds like the problems are worse -- not better. Hopefully the repackaged reportlab "standard font size calculator" will solve 80-90% of the problem.

Don't get me started about fonts. Both with PIL and with FreeType, chaco/kiva/scipy will need to define a font registry. Reportlab has something like that, which apparently deals with Type 1 fonts, and BDF fonts. That's probably overkill for this project at this point.

My sense is that HW accelerated OpenGL is amazingly fast for 3D with less attention paid to the 2D stuff (which Doom doesn't care about I guess).

Back when I looked at this, the major thing that OpenGL did amazingly well in the 2-d space was anti-aliasing, transparency and zooming. Those are very expensive to do in software, and cheap to do in hardware. I've used PyOpenGL to do interactive 2-d applications that used all three of those, and the results were impressive. I didn't have anti-grain to compare with, though.

If you left click and move the mouse around, the image is scaled/rotated. It looks plenty fast.

Cool.

To a point. It does create a maintenance headache. Most Python GUI APIs are, ahhemm, under documented (man is that the pot calling the kettle black coming from a SciPy developer...).

Having spent some days looking in chaco/kiva, I'd have to agree =).

Dave Morrill has the same allergy. What's with you guys?? Someone must have a shot for that. I'll ask Dave Abrahams... way to many Dave's around here.

In my case, it's got to do with the fact that I was exposed to C++ when it was too young for public consumption (IMO). I never got to know "grown-up" C++. Current svgcore2d.py attached -- it works well with simple plots (try "python svgcore2d.py output.svg" and then look at the resulting SVG file). I still have a problem with layout of composite plots, which probably has to do with the coordinate transforms that I have to apply. I'll send you and Dave an example output and you can tell me what line I need to change. =) Cheers, --david """ Chaco's SVG backend Copyright: ActiveState License: BSD Style Author: David Ascher (davida@activestate.com) Version: $Revision: #4 $ """ #### # # Known limitations # # * BUG: Weird behavior with compound plots # * Limitation: text widths are lousy if reportlab is not installed # * Missing feature: rotated text """ Miscellaneous notes: * the way to do links: <a xlink:href="http://www.w3.org"> <ellipse cx="2.5" cy="1.5" rx="2" ry="1" fill="red" /> </a> """ # wherever you want me to write debug info. import os import tempfile _logfile = os.path.join(tempfile.gettempdir(), "chacosvg.log") #_logfile = os.path.join(r"C:\tmp", "chacosvg.log") ##### Nothing below needs to be customized. import sys if __name__ == '__main__': # for use when testing from outside of kiva import kiva sys.path.insert(0, os.path.dirname(kiva.__file__)) logfile = "kiva-svg.log" from numcompat import * from constants import * import affine import basecore2d import copy import cStringIO import fonttools import os import sys try: import logging hdlr = logging.FileHandler(_logfile) BASIC_FORMAT = "%(levelname)s: %(name)s: %(message)s" fmt = logging.Formatter(BASIC_FORMAT) hdlr.setFormatter(fmt) logging.root.addHandler(hdlr) log = logging.getLogger('') log.setLevel(logging.INFO) except ImportError: class FakeLogger: def debug(self, message): print >> sys.stderr, "DEBUG:", message def info(self, message): print >> sys.stderr, "INFO:", message def warn(self, message): print >> sys.stderr, "WARN:", message def error(self, message): print >> sys.stderr, "ERROR:", message def critical(self, message): print >> sys.stderr, "CRITICAL:", message log = FakeLogger() def _strpoints(points): c = cStringIO.StringIO() for x,y in points: c.write('%3.2f,%3.2f ' % (x,y)) return c.getvalue() def _mkstyle(kw1): kw = {} for (k,v) in kw1.items(): if type(v) == type(()): if v[0] != v[1]: kw[k] = v[0] else: kw[k] = v return '"' + '; '.join([str(k) + ':' + str(v) for k,v in kw.items()]) +'"' line_cap_map = { CAP_ROUND: 'round', CAP_SQUARE: 'square', CAP_BUTT: 'butt' } line_join_map = { JOIN_ROUND: 'round', JOIN_BEVEL: 'bevel', JOIN_MITER: 'miter' } font_map = {'Arial': 'Helvetica', } try: # expensive way of computing string widths import reportlab.pdfbase.pdfmetrics import reportlab.pdfbase._fontdata _reportlab_loaded = 1 except ImportError: _reportlab_loaded = 0 xmltemplate = """<?xml version="1.0"?> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.0//EN" "http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd"> <svg xmlns="http://www.w3.org/2000/svg" xmlns:text="http://xmlns.graougraou.com/svg/text/" xmlns:a3="http://ns.adobe.com/AdobeSVGViewerExtensions/3.0/" a3:scriptImplementation="Adobe" width="100%%" height="100%%" viewBox="0 0 %(width)f %(height)f" > %(contents)s </svg> """ htmltemplate = """<html xmlns:svg="http://www.w3.org/2000/svg"> <object id="AdobeSVG" CLASSID="clsid:78156a80-c6a1-4bbf-8e6a-3cd390eeb4e2"> </object> <?import namespace="svg" implementation="#AdobeSVG"?> <body> <svg:svg width="100%%" height="100%%" viewBox="0 0 %(width)f %(height)f"> %(contents)s </svg:svg> </body> </html> """ class SVGGC(basecore2d.GraphicsContextBase): def __init__(self, size): basecore2d.GraphicsContextBase.__init__(self) self.size = size self._height = size[1] self.contents = cStringIO.StringIO() def save(self, filename): f = open(filename, 'w') ext = os.path.splitext(filename)[1] if ext == '.svg': height, width = self.size contents = self.contents.getvalue().replace("<svg:", "<").replace("</svg:", "</") f.write(xmltemplate % locals()) elif ext == '.html': height, width = self.size[0]*3, self.size[1]*3 contents = self.contents.getvalue() f.write(htmltemplate % locals()) else: raise ValueError, "don't know how to write a %s file" % ext # Text handling code def set_font(self, font): self.font = font self.font_name = font_map[self.font.face_name] #if _reportlab_loaded: # self.pdffont = reportlab.pdfbase.pdfmetrics.Font(self.font_name, self.font_name, "WinAnsiEncoding") def device_show_text(self, text): # need to support font, size, rotation x,y = self.get_text_position() x = '"%f"' % x y = '"%f"' % (self._height - y) ttm = self.get_text_matrix() ctm = self.get_ctm() # not device_ctm!! m = affine.concat(ctm,ttm) tx,ty,sx,sy,angle = affine.trs_factor(m) x,y = self.get_text_position() angle = int(angle / pi * 180) self._emit('text', contents=text, x=x, y=y, rotate=angle, kw={'font-family':repr(self.font.face_name), 'font-size': '"'+ str(self.font.size) + '"'}) def get_full_text_extent(self, text): if _reportlab_loaded: ascent,descent=reportlab.pdfbase._fontdata.ascent_descent[self.font_name] descent = (-descent) * self.font.size / 1000.0 ascent = ascent * self.font.size / 1000.0 height = ascent + descent width = reportlab.pdfbase.pdfmetrics._slowStringWidth(text, self.font_name, self.font.size) return width, height, descent, height*1.2 # assume leading of 1.2*height else: # use a (lousy) heuristic return .7*self.font.size*len(string),self.font.size*1.2,0,0 # actual implementation =) def device_fill_points(self, points, mode): points = self._fixpoints(points) if mode in (FILL, FILL_STROKE, EOF_FILL_STROKE): fill = self._color(self.state.fill_color) else: fill = 'none' if mode in (STROKE, FILL_STROKE, EOF_FILL_STROKE): stroke = self._color(self.state.line_color) else: stroke = 'none' if mode in (EOF_FILL_STROKE, EOF_FILL): rule = 'evenodd' else: rule = 'nonzero' linecap = line_cap_map[self.state.line_cap] linejoin = line_join_map[self.state.line_join] dasharray = self._dasharray() width = '%3.3f' % self.state.line_width if mode == STROKE: opacity = '%1.3f' % self.state.line_color[-1] self._emit('polyline', points='"'+_strpoints(points)+'"', style=_mkstyle({'opacity': (opacity, "1.000"), 'stroke': stroke, 'fill': 'none', 'stroke-width': (width, "1.000"), 'stroke-linejoin': (linejoin, 'miter'), 'stroke-linecap': (linecap, 'butt'), 'stroke-dasharray': (dasharray, 'none')})) else: opacity = '%1.3f' % self.state.fill_color[-1] self._emit('polygon', points='"'+_strpoints(points)+'"', style=_mkstyle({'opacity': (opacity, "1.000"), 'stroke-width': (width, "1.000"), 'fill': fill, 'fill-rule': rule, 'stroke': stroke, 'stroke-linejoin': (linejoin, 'miter'), 'stroke-linecap': (linecap, 'butt'), 'stroke-dasharray': (dasharray, 'none')})) def device_stroke_points(self, points, mode): # handled by device_fill_points pass def device_destroy_clipping_path(self): # Not Implemented Yet pass def device_set_clipping_path(self, x, y, width, height): # Not Implemented Yet pass # utility routines def _fixpoints(self, points): # convert lines from Kiva coordinate space to PIL coordinate space # XXX I suspect this is the location of the bug w.r.t. compound graphs and # "global" sizing. # XXX this should be made more efficient for NumPy arrays np = [] for (x,y) in points: np.append((x,self._height-y)) return np def _emit(self, name, contents=None, kw={}, **otherkw): self.contents.write('<svg:%(name)s ' % locals()) for k, v in kw.items(): self.contents.write("%(k)s=%(v)s " % locals()) for k, v in otherkw.items(): self.contents.write("%(k)s=%(v)s " % locals()) if contents is None: self.contents.write('/>\n') else: self.contents.write('>\n') self.contents.write(contents) self.contents.write('</svg:'+name+'>') def _color(self, color): r,g,b,a = color return '#%02x%02x%02x' % (r*255,g*255,b*255) def _dasharray(self): dasharray = '' for x in self.state.line_dash: if type(x) == type(arange(3)): # why is this so hard? x = x.flat[0] dasharray += ' ' + '%3.2f' % x if not dasharray or dasharray == " 0.00 0.00": dasharray = 'none' return dasharray # noops which seem to be needed def device_update_line_state(self): pass def device_update_fill_state(self): pass class Canvas: def __init__(self, filename='', id = -1, size = (200,200)): self.filename = filename self._size = size self.gc = SVGGC(size) def size(self): return self._size def save(self): self.gc.save(self.filename) if __name__ == "__main__": import sys if len(sys.argv) == 1: print "Usage: %s output_file (where output_file ends in .html or .svg" % sys.argv[0] raise SystemExit import chaco.plot_canvas, chaco.plot_frame, chaco.plot_base canvas = Canvas( filename=sys.argv[1], size=(400,400)) plot = chaco.plot_canvas.PlotValue( 2*cos(arange(500.)/23.)*sin(arange( 500.0 ) / 40.), axis = chaco.plot_frame.PlotAxis( help = 'Default Index Axis', rotate=90), line_color='red', line_weight=2) plot.gc = canvas.gc # Connect the kiva GraphicsContext to the chaco object hierarchy plot._min_size() # Do initial layout pass over all chaco objects plot._layout( 10, 10, 390, 390) # Do the actual layout of all chaco objects plot._draw() # Now render them into the kiva GraphicsContext canvas.save() # Save the contents of the GraphicsContext to the file

eric jones wrote:

??? I don't quite get this. We have to know the sizes of the axis labels (both titles and numbers) to trim off enough real estate around a plot so that the numbers don't overlap onto the graph (y labels) or onto each other (x labels). I'm not sure how you would do this without precalculating the actual screen size of the text.

You're right. I didn't realize the sophistication in your algorithms (I should have, I've admired the tick marks and labeling choices).

It would be better not to have the reportlab dependency if we don't loose any capabilities, so if the backend is trivial (nearly identical to the PS one I guess?), I'm for removing the dependency.

Ok, I'll try and do that later this week.

I'm betting OpenGL will win -- or if it doesn't now, it will in the future. Unless there is a compelling reason to get rid of it at some point, keeping an OpenGL backend seems like a good idea. There are only a few features that will be problematic with it (and probably won't be supported). These include "join mitering" and "line end caps."

I've never needed those in display code -- only in printouts. PIL will never support that either, and I don't care =).

The other thing I'm a little worried about is thick lines. I think some OpenGL implementations don't support lines thicker than 10 points. Beyond that, you have to draw a polygon. This will lead to silly if/then code. There are probably other things, but Dave Morrill, has gotten it all 90% there.

There are other weirdnesses, such as the fact that OpenGL "points" are in screen pixels, not in viewspace coordinates. But since Kiva doesn't support points yet, you haven't noticed it =). [..docs...]

I promise that this will be remedied, but I won't promise when. :-)

I'll produce a quick writeup of how to do a backend like the ones I did -- someone else can write the "interactive" part. What I'm most in need of isn't Kiva information -- that code is fairly straightforward. It's the Chaco class hierarchy and fancy 'delegate' handling which I'd love to learn more about. Ooh. I just figured out how to do compound plots -- there's a minor bug in plot_container.py:NestedContainers._add_item() -- it shouldn't fail if there's no "window" attribute. With a try/except around that access, I get compound plots w/o having to invoke any of the GUI stuff. Yeehah! [Aside: why do some lines in .co files have the format "foo = bar" while others have "foo: bar"?]

Wait a second... Something must be wrong with the email transmission here. I'm missing the file where you defined all the javascript code to support interaction with the plot.:-)

Until I get marker support code in Kiva, there's not much point to doing that. I'm don't see SVG as the base for a full-fledged interactive UI (although that would be cool in a sick kind of way) -- but I do want to support some "data mining operations" -- click on a marker hits another web page that gives you more details about that data point, etc. That will happen by adding metadata to individual data points (aka markers). --david