Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem
On Mon, 2004-06-28 at 19:38, Sebastian Haase wrote:
Is 10x a measured number or a gut feel?
I put some time.clock() statements into the wxPyPlot code I got this: (the times are differences: T_after-T_before) one run with numarray: <__main__.PolyLine instance at 0x868d414> time= 1.06 <__main__.PolyMarker instance at 0x878e9c4> time= 1.37 a second run with numarray: <__main__.PolyLine instance at 0x875da1c> time= 0.85 <__main__.PolyMarker instance at 0x86da034> time= 1.04 first run with Numeric: <__main__.PolyLine instance at 0x858babc> time= 0.07 <__main__.PolyMarker instance at 0x858bc4c> time= 0.14 a second one: <__main__.PolyLine instance at 0x858cd7c> time= 0.08 <__main__.PolyMarker instance at 0x8585d8c> time= 0.17 This seems to be consistent with the profiling I did before: I get this w/ numarray: ncalls tottime percall cumtime percall filename:lineno(function) 1 1.140 1.140 1.320 1.320 gdi.py:554(DrawLines) 1 1.250 1.250 1.520 1.520 gdi.py:792(_DrawRectangleList) 50230 0.450 0.000 0.450 0.000 numarraycore.py:501(__del__) and this with Numeric: 1 0.080 0.080 0.080 0.080 gdi.py:554(DrawLines) 1 0.090 0.090 0.090 0.090 gdi.py:792(_DrawRectangleList)
so this looks to me like a factor of 10x.
Me too. Can you (or somebody) post the application code which does the drawlines? I can definitely instrument the bottleneck C-code, but I don't have time to ascend the wxPython learning curve. Todd
Todd Miller wrote:
On Mon, 2004-06-28 at 19:38, Sebastian Haase wrote:
Is 10x a measured number or a gut feel?
[SNIP]
so this looks to me like a factor of 10x.
Me too. Can you (or somebody) post the application code which does the drawlines? I can definitely instrument the bottleneck C-code, but I don't have time to ascend the wxPython learning curve.
Let me second that. With a little nudge from Chris Barker, I managed to get wxPython to compile here and I have some changes that may speed up drawlines, but it'd probably be best if we were all using the same benchmark. -tim
Tim Hochberg wrote:
Todd Miller wrote:
Me too. Can you (or somebody) post the application code which does the drawlines? I can definitely instrument the bottleneck C-code, but I don't have time to ascend the wxPython learning curve.
Let me second that. With a little nudge from Chris Barker, I managed to get wxPython to compile here and I have some changes that may speed up drawlines, but it'd probably be best if we were all using the same benchmark.
Well, if I can't code, at least I can nudge! Perhaps I can also help get Todd what he's asking for, except that I'm not sure what you mean by "application code". Do you mean a small wxPython application that uses DC.DrawLines (and friends)? If so, yes, I can do that. What version of wxPython should I target? CVS head? 2.5.1? (The latest "released" version) there are some slight incompatibilities between versions, particularly with DC calls, unfortunately. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Tue, 2004-06-29 at 17:05, Chris Barker wrote:
Tim Hochberg wrote:
Todd Miller wrote:
Me too. Can you (or somebody) post the application code which does the drawlines? I can definitely instrument the bottleneck C-code, but I don't have time to ascend the wxPython learning curve.
Let me second that. With a little nudge from Chris Barker, I managed to get wxPython to compile here and I have some changes that may speed up drawlines, but it'd probably be best if we were all using the same benchmark.
Well, if I can't code, at least I can nudge!
Perhaps I can also help get Todd what he's asking for, except that I'm not sure what you mean by "application code". Do you mean a small wxPython application that uses DC.DrawLines (and friends)?
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
If so, yes, I can do that. What version of wxPython should I target? CVS head? 2.5.1? (The latest "released" version)
I'd prefer 2.5.1 unless Tim says otherwise.
there are some slight incompatibilities between versions, particularly with DC calls, unfortunately.
I'm hoping this won't affect the profile. Regards, Todd
Todd Miller wrote:
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
Actually, it was someone else that did the profiling, but here is a sample, about as simple as I could make it. It draws an N point line and M points. At the moment, it is using Numeric for the points, and numarray for the lines. Numeric is MUCH faster (which is the whole point of this discussion). Otherwise, it takes about the same amount of time to draw the lines as the points. Another note: if use the tolist() method in the numarray first, it's much faster also: dc.DrawLines(LinesPoints.tolist()) Obviously, the tolist method is much faster than wxPython's sequence methods, as would be expected. I'm going to send a note to Robin Dunn about this as well, and see what he thinks. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov #!/usr/bin/env python2.3 import wx import numarray from numarray import random_array import RandomArray # the Numeric version import time NumLinePoints = 5000 NumPointPoints = 5000 ## Make some random data to draw things with. MaxX = 500 LinesPoints = random_array.randint(1, MaxX, (NumLinePoints,2) ) #PointsPoints = random_array.randint(1, MaxX, (NumPointPoints,2) ) PointsPoints = RandomArray.randint(1, MaxX, (NumPointPoints,2) ) # Numeric class TestFrame(wx.Frame): def __init__(self): wx.Frame.__init__(self, None, -1, "DrawLines Test", wx.DefaultPosition, size=(500,500), style=wx.DEFAULT_FRAME_STYLE | wx.NO_FULL_REPAINT_ON_RESIZE) ## Set up the MenuBar MenuBar = wx.MenuBar() file_menu = wx.Menu() ID_EXIT_MENU = wx.NewId() file_menu.Append(ID_EXIT_MENU, "E&xit","Terminate the program") wx.EVT_MENU(self, ID_EXIT_MENU, self.OnQuit) MenuBar.Append(file_menu, "&File") draw_menu = wx.Menu() ID_DRAW_MENU = wx.NewId() draw_menu.Append(ID_DRAW_MENU, "&ReDraw","DrawAgain") wx.EVT_MENU(self, ID_DRAW_MENU,self.ReDraw) MenuBar.Append(draw_menu, "&Draw") self.SetMenuBar(MenuBar) wx.EVT_PAINT(self, self.OnPaint) def OnPaint(self,event): dc = wx.PaintDC(self) dc.SetBackground( wx.Brush("White") ) dc.Clear() self.DrawLines(dc) self.DrawPoints(dc) def ReDraw(self, event = None): dc = wx.ClientDC(self) dc.SetBackground( wx.Brush("White") ) dc.Clear() self.DrawLines(dc) self.DrawPoints(dc) def DrawLines(self, dc): dc.BeginDrawing() dc.SetPen(wx.Pen('Black', 2)) start = time.clock() #dc.DrawLines(LinesPoints.tolist()) dc.DrawLines(LinesPoints) print "DrawLines Call took %f seconds"%(time.clock() - start) dc.EndDrawing() def DrawPoints(self, dc): dc.BeginDrawing() dc.SetPen(wx.Pen('Red', 2)) start = time.clock() dc.DrawPointList(PointsPoints) print "DrawPointList Call took %f seconds"%(time.clock() - start) dc.EndDrawing() def OnQuit(self,event): self.Close(True) class DemoApp(wx.App): def OnInit(self): frame = TestFrame() frame.Show(True) self.SetTopWindow(frame) return True if __name__ == "__main__": app = DemoApp(0) app.MainLoop()
On Jun 29, 2004, at 4:25 PM, Chris Barker wrote:
Todd Miller wrote:
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
Actually, it was someone else that did the profiling, but here is a sample, about as simple as I could make it.
It draws an N point line and M points. At the moment, it is using Numeric for the points, and numarray for the lines. Numeric is MUCH faster (which is the whole point of this discussion). Otherwise, it takes about the same amount of time to draw the lines as the points.
Another note: if use the tolist() method in the numarray first, it's much faster also:
dc.DrawLines(LinesPoints.tolist())
If you are going to move lots of lines and points, I would recommend pushing this stuff through PyOpenGL with array objects and vertex objects. Letting OpenGL handle the transformations, clipping, movement and iteration in hardware stomps all over even the best written C code. Most UI toolkits have some form of OpenGL widget. For lots of the code I have written, even Mesa (the open-souce software OpenGL renderer) was fast enough, and not having to write all of the display transformation code by hand was a huge win even when the speed was somewhat lagging. -a
I'd bet a case of beer (or cash equivalent) that one of the main bottlenecks is the path PySequence_GetItem->_ndarray_item->_universalIndexing->_simpleIndexing->_simpleIndexingCore. The path through _universalIndexing in particular, if I deciphered it correctly, looks very slow. I don't think it needs to be that way though, _universalIndexing could probably be sped up, but more promising I think _ndarray_item could be made to call _simpleIndexingCore without all that much work. It appears that this would save the creation of several intermediate objects and it also looks like a couple of calls back to python! I'm not familiar with this code though, so I could easily be missing something that makes calling _simpleIndexingCore harder than it looks. -tim
On Tue, 2004-06-29 at 16:19, Tim Hochberg wrote:
I'd bet a case of beer (or cash equivalent) that one of the main bottlenecks is the path PySequence_GetItem->_ndarray_item->_universalIndexing->_simpleIndexing->_simpleIndexingCore.
I won't take the bet but if this works out, you get the beer. If it doesn't, well, I don't drink anymore anyway.
The path through _universalIndexing in particular, if I deciphered it correctly, looks very slow. I don't think it needs to be that way though, _universalIndexing could probably be sped up, but more promising I think _ndarray_item could be made to call _simpleIndexingCore without all that much work. It appears that this would save the creation of several intermediate objects and it also looks like a couple of calls back to python! I'm not familiar with this code though, so I could easily be missing something that makes calling _simpleIndexingCore harder than it looks.
This looks very promising. I'll take a look tomorrow. Regards, Todd
participants (4)
-
Andrew P. Lentvorski, Jr. -
Chris Barker -
Tim Hochberg -
Todd Miller