[Matplotlib-devel] [Matplotlib-users] paletted PNG backend

[Jesper Larsen]

Hi Chris

gd seems like a great candidate for 8-bit rendering (which is what we
want). And thanks for the insights into PIL. I will look more into the
links you provided.

Best regards,
Jesper

2016-02-17 19:23 GMT+01:00 Chris Barker <chris.barker at noaa.gov>:

>
> On Mon, Feb 15, 2016 at 3:19 AM Jesper Larsen <jesper.webmail at gmail.com>
>> wrote:
>>
>>> We are using Matplotlib for a web service which makes PNG images on the
>>> fly for presentation on a map (web site using the web service is here:
>>> https://ifm-beta.fcoo.dk)
>>>
>>> Performance and image size are two major concerns for us. We therefore
>>> save the resulting RGBA PNG to a buffer and afterwards use Pillow (PIL) to
>>> convert it to a P PNG (paletted PNG) to reduce the image size dramatically.
>>>
>>
> We have similar issues, though not with MPL -- in fact one reason we are
> not using MPL is because we want highly optimized rendering -- all the
> features MPL provides do add overhead...
>
> And doing experiments with various rendering engines (PIL, AGG, SKIA, GD)
> with found that they all have pretty darn similar performance, but that
> performance scales pretty much with how many bytes you are pushing -- i.e.
> larger images take longer to render, and 8 bits per pixel is faster than 32
> bits per pixel (by pretty much a factor of four, if I recall).
>
> So while you are now taking a lot of time converting, you might also get
> faster rendering in the first place if you use 8bpp rendering.
>
> But, as Thomas suggested, AGG is pretty much all about anti-aliasing and
> that requires more than 255 colors....
>
> So: the "right" way to solve this problem is to use a non-anti-aliasing,
> 8bpp rendering lib. You could make a new back-end that uses that instead of
> AGG. ONE of the core MPLS devs (I can't remember who) suggested that MPL
> could use a maybe-not-the-best-quality-but-fast back end last year at
> SciPy, so maybe you'll get some help if you want to do it.
>
> Also, a non-anti-aliasing back-end would be nice for things like contours
> where you can get anti-aliasing artifacts where adjacent polygons are
> supposed to line up exactly.
>
> I suggest the libgd rendering lib:
>
> https://github.com/libgd/libgd
>
> It's not seeing a massive amount of development, but that's because it's
> been around forever and is pretty darn robust and stable. It's also got an
> ancient, semi-ugly C API, but we can deal with that :-)
>
> If you want to make a MP back end with it, I suggest you start with my
> Cython-based python wrapper:
>
> https://github.com/NOAA-ORR-ERD/py_gd
>
> It's not terribly complete, but provides the core functionality for
> rendering 8 bit images, and numpy interactivity -- i.e. passing coordinates
> of large polygons as a numpy array, and passing the raw image data in and
> out as a numpy array.
>
> I've also got conda packages for libgd and py_gd in our channel here:
>
> https://conda.anaconda.org/noaa-orr-erd
>
> The recipes for those are here:
>
> https://github.com/NOAA-ORR-ERD/orr-conda-recipes
>
> Honestly, AGG has been pretty integral to MPL from the beginning (Or near
> beginning), so I'm not sure how hard it would be to drop in a new renderer,
> but I'd be glad to help.
>
> 2) Is it better to make a separate Pillow based backend for this (Pillow
>>> is probably not as fast as AGG)?
>>>
>>
> NOTE: we looked at using PIL for our rendering, and it was kind-of-sort of
> fast enough, but the rendering code, and python calls to it is all
> hand-written C, so it looked a lot harder to extend and optimize
> performance for.
>
> With py_gd, we can direct calls to the C lib form numpy arrays directly,
> and write time-sensitive loops in cython where there are speed bottlenecks.
>
> One issue that will take a bit of effort is colormaps -- IIUC, MPL pretty
> much assumes 32 bit  (or 24 bit anyway) color. With 8bpp, colormaps have to
> be limited to 255 colors, and, in practice, you probably want to save a few
> "pure" colors for rendering text and the axes, etc: transparent, white,
> black -- a few others?
>
> I think you could get a fine colormap with, say 245 colors, saving 10 for
> other uses, but then you could only have one colormap in use at a time. So
> there may be some working around MPL's colormap code for this....
>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20160217/edc24275/attachment.html>