Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem
![](https://secure.gravatar.com/avatar/faf9400121dca9940496a7473b1d8179.jpg?s=120&d=mm&r=g)
On Mon, 2004-06-28 at 19:38, Sebastian Haase wrote:
Me too. Can you (or somebody) post the application code which does the drawlines? I can definitely instrument the bottleneck C-code, but I don't have time to ascend the wxPython learning curve. Todd
![](https://secure.gravatar.com/avatar/55f7acf47233a7a98f5eb9dfd0b2d763.jpg?s=120&d=mm&r=g)
Todd Miller wrote:
Let me second that. With a little nudge from Chris Barker, I managed to get wxPython to compile here and I have some changes that may speed up drawlines, but it'd probably be best if we were all using the same benchmark. -tim
![](https://secure.gravatar.com/avatar/5dde29b54a3f1b76b2541d0a4a9b232c.jpg?s=120&d=mm&r=g)
Tim Hochberg wrote:
Well, if I can't code, at least I can nudge! Perhaps I can also help get Todd what he's asking for, except that I'm not sure what you mean by "application code". Do you mean a small wxPython application that uses DC.DrawLines (and friends)? If so, yes, I can do that. What version of wxPython should I target? CVS head? 2.5.1? (The latest "released" version) there are some slight incompatibilities between versions, particularly with DC calls, unfortunately. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
![](https://secure.gravatar.com/avatar/faf9400121dca9940496a7473b1d8179.jpg?s=120&d=mm&r=g)
On Tue, 2004-06-29 at 17:05, Chris Barker wrote:
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
I'd prefer 2.5.1 unless Tim says otherwise.
there are some slight incompatibilities between versions, particularly with DC calls, unfortunately.
I'm hoping this won't affect the profile. Regards, Todd
![](https://secure.gravatar.com/avatar/5dde29b54a3f1b76b2541d0a4a9b232c.jpg?s=120&d=mm&r=g)
Todd Miller wrote:
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
Actually, it was someone else that did the profiling, but here is a sample, about as simple as I could make it. It draws an N point line and M points. At the moment, it is using Numeric for the points, and numarray for the lines. Numeric is MUCH faster (which is the whole point of this discussion). Otherwise, it takes about the same amount of time to draw the lines as the points. Another note: if use the tolist() method in the numarray first, it's much faster also: dc.DrawLines(LinesPoints.tolist()) Obviously, the tolist method is much faster than wxPython's sequence methods, as would be expected. I'm going to send a note to Robin Dunn about this as well, and see what he thinks. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov #!/usr/bin/env python2.3 import wx import numarray from numarray import random_array import RandomArray # the Numeric version import time NumLinePoints = 5000 NumPointPoints = 5000 ## Make some random data to draw things with. MaxX = 500 LinesPoints = random_array.randint(1, MaxX, (NumLinePoints,2) ) #PointsPoints = random_array.randint(1, MaxX, (NumPointPoints,2) ) PointsPoints = RandomArray.randint(1, MaxX, (NumPointPoints,2) ) # Numeric class TestFrame(wx.Frame): def __init__(self): wx.Frame.__init__(self, None, -1, "DrawLines Test", wx.DefaultPosition, size=(500,500), style=wx.DEFAULT_FRAME_STYLE | wx.NO_FULL_REPAINT_ON_RESIZE) ## Set up the MenuBar MenuBar = wx.MenuBar() file_menu = wx.Menu() ID_EXIT_MENU = wx.NewId() file_menu.Append(ID_EXIT_MENU, "E&xit","Terminate the program") wx.EVT_MENU(self, ID_EXIT_MENU, self.OnQuit) MenuBar.Append(file_menu, "&File") draw_menu = wx.Menu() ID_DRAW_MENU = wx.NewId() draw_menu.Append(ID_DRAW_MENU, "&ReDraw","DrawAgain") wx.EVT_MENU(self, ID_DRAW_MENU,self.ReDraw) MenuBar.Append(draw_menu, "&Draw") self.SetMenuBar(MenuBar) wx.EVT_PAINT(self, self.OnPaint) def OnPaint(self,event): dc = wx.PaintDC(self) dc.SetBackground( wx.Brush("White") ) dc.Clear() self.DrawLines(dc) self.DrawPoints(dc) def ReDraw(self, event = None): dc = wx.ClientDC(self) dc.SetBackground( wx.Brush("White") ) dc.Clear() self.DrawLines(dc) self.DrawPoints(dc) def DrawLines(self, dc): dc.BeginDrawing() dc.SetPen(wx.Pen('Black', 2)) start = time.clock() #dc.DrawLines(LinesPoints.tolist()) dc.DrawLines(LinesPoints) print "DrawLines Call took %f seconds"%(time.clock() - start) dc.EndDrawing() def DrawPoints(self, dc): dc.BeginDrawing() dc.SetPen(wx.Pen('Red', 2)) start = time.clock() dc.DrawPointList(PointsPoints) print "DrawPointList Call took %f seconds"%(time.clock() - start) dc.EndDrawing() def OnQuit(self,event): self.Close(True) class DemoApp(wx.App): def OnInit(self): frame = TestFrame() frame.Show(True) self.SetTopWindow(frame) return True if __name__ == "__main__": app = DemoApp(0) app.MainLoop()
![](https://secure.gravatar.com/avatar/c7b89689e0799cf497f55e0f9c1260d7.jpg?s=120&d=mm&r=g)
On Jun 29, 2004, at 4:25 PM, Chris Barker wrote:
If you are going to move lots of lines and points, I would recommend pushing this stuff through PyOpenGL with array objects and vertex objects. Letting OpenGL handle the transformations, clipping, movement and iteration in hardware stomps all over even the best written C code. Most UI toolkits have some form of OpenGL widget. For lots of the code I have written, even Mesa (the open-souce software OpenGL renderer) was fast enough, and not having to write all of the display transformation code by hand was a huge win even when the speed was somewhat lagging. -a
![](https://secure.gravatar.com/avatar/55f7acf47233a7a98f5eb9dfd0b2d763.jpg?s=120&d=mm&r=g)
I'd bet a case of beer (or cash equivalent) that one of the main bottlenecks is the path PySequence_GetItem->_ndarray_item->_universalIndexing->_simpleIndexing->_simpleIndexingCore. The path through _universalIndexing in particular, if I deciphered it correctly, looks very slow. I don't think it needs to be that way though, _universalIndexing could probably be sped up, but more promising I think _ndarray_item could be made to call _simpleIndexingCore without all that much work. It appears that this would save the creation of several intermediate objects and it also looks like a couple of calls back to python! I'm not familiar with this code though, so I could easily be missing something that makes calling _simpleIndexingCore harder than it looks. -tim
![](https://secure.gravatar.com/avatar/faf9400121dca9940496a7473b1d8179.jpg?s=120&d=mm&r=g)
On Tue, 2004-06-29 at 16:19, Tim Hochberg wrote:
I won't take the bet but if this works out, you get the beer. If it doesn't, well, I don't drink anymore anyway.
This looks very promising. I'll take a look tomorrow. Regards, Todd
![](https://secure.gravatar.com/avatar/55f7acf47233a7a98f5eb9dfd0b2d763.jpg?s=120&d=mm&r=g)
Todd Miller wrote:
Let me second that. With a little nudge from Chris Barker, I managed to get wxPython to compile here and I have some changes that may speed up drawlines, but it'd probably be best if we were all using the same benchmark. -tim
![](https://secure.gravatar.com/avatar/5dde29b54a3f1b76b2541d0a4a9b232c.jpg?s=120&d=mm&r=g)
Tim Hochberg wrote:
Well, if I can't code, at least I can nudge! Perhaps I can also help get Todd what he's asking for, except that I'm not sure what you mean by "application code". Do you mean a small wxPython application that uses DC.DrawLines (and friends)? If so, yes, I can do that. What version of wxPython should I target? CVS head? 2.5.1? (The latest "released" version) there are some slight incompatibilities between versions, particularly with DC calls, unfortunately. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
![](https://secure.gravatar.com/avatar/faf9400121dca9940496a7473b1d8179.jpg?s=120&d=mm&r=g)
On Tue, 2004-06-29 at 17:05, Chris Barker wrote:
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
I'd prefer 2.5.1 unless Tim says otherwise.
there are some slight incompatibilities between versions, particularly with DC calls, unfortunately.
I'm hoping this won't affect the profile. Regards, Todd
![](https://secure.gravatar.com/avatar/5dde29b54a3f1b76b2541d0a4a9b232c.jpg?s=120&d=mm&r=g)
Todd Miller wrote:
Yes. What I most want to do is a 50000 point drawlines, similar to what you profiled. Friends are fine too.
Actually, it was someone else that did the profiling, but here is a sample, about as simple as I could make it. It draws an N point line and M points. At the moment, it is using Numeric for the points, and numarray for the lines. Numeric is MUCH faster (which is the whole point of this discussion). Otherwise, it takes about the same amount of time to draw the lines as the points. Another note: if use the tolist() method in the numarray first, it's much faster also: dc.DrawLines(LinesPoints.tolist()) Obviously, the tolist method is much faster than wxPython's sequence methods, as would be expected. I'm going to send a note to Robin Dunn about this as well, and see what he thinks. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov #!/usr/bin/env python2.3 import wx import numarray from numarray import random_array import RandomArray # the Numeric version import time NumLinePoints = 5000 NumPointPoints = 5000 ## Make some random data to draw things with. MaxX = 500 LinesPoints = random_array.randint(1, MaxX, (NumLinePoints,2) ) #PointsPoints = random_array.randint(1, MaxX, (NumPointPoints,2) ) PointsPoints = RandomArray.randint(1, MaxX, (NumPointPoints,2) ) # Numeric class TestFrame(wx.Frame): def __init__(self): wx.Frame.__init__(self, None, -1, "DrawLines Test", wx.DefaultPosition, size=(500,500), style=wx.DEFAULT_FRAME_STYLE | wx.NO_FULL_REPAINT_ON_RESIZE) ## Set up the MenuBar MenuBar = wx.MenuBar() file_menu = wx.Menu() ID_EXIT_MENU = wx.NewId() file_menu.Append(ID_EXIT_MENU, "E&xit","Terminate the program") wx.EVT_MENU(self, ID_EXIT_MENU, self.OnQuit) MenuBar.Append(file_menu, "&File") draw_menu = wx.Menu() ID_DRAW_MENU = wx.NewId() draw_menu.Append(ID_DRAW_MENU, "&ReDraw","DrawAgain") wx.EVT_MENU(self, ID_DRAW_MENU,self.ReDraw) MenuBar.Append(draw_menu, "&Draw") self.SetMenuBar(MenuBar) wx.EVT_PAINT(self, self.OnPaint) def OnPaint(self,event): dc = wx.PaintDC(self) dc.SetBackground( wx.Brush("White") ) dc.Clear() self.DrawLines(dc) self.DrawPoints(dc) def ReDraw(self, event = None): dc = wx.ClientDC(self) dc.SetBackground( wx.Brush("White") ) dc.Clear() self.DrawLines(dc) self.DrawPoints(dc) def DrawLines(self, dc): dc.BeginDrawing() dc.SetPen(wx.Pen('Black', 2)) start = time.clock() #dc.DrawLines(LinesPoints.tolist()) dc.DrawLines(LinesPoints) print "DrawLines Call took %f seconds"%(time.clock() - start) dc.EndDrawing() def DrawPoints(self, dc): dc.BeginDrawing() dc.SetPen(wx.Pen('Red', 2)) start = time.clock() dc.DrawPointList(PointsPoints) print "DrawPointList Call took %f seconds"%(time.clock() - start) dc.EndDrawing() def OnQuit(self,event): self.Close(True) class DemoApp(wx.App): def OnInit(self): frame = TestFrame() frame.Show(True) self.SetTopWindow(frame) return True if __name__ == "__main__": app = DemoApp(0) app.MainLoop()
![](https://secure.gravatar.com/avatar/c7b89689e0799cf497f55e0f9c1260d7.jpg?s=120&d=mm&r=g)
On Jun 29, 2004, at 4:25 PM, Chris Barker wrote:
If you are going to move lots of lines and points, I would recommend pushing this stuff through PyOpenGL with array objects and vertex objects. Letting OpenGL handle the transformations, clipping, movement and iteration in hardware stomps all over even the best written C code. Most UI toolkits have some form of OpenGL widget. For lots of the code I have written, even Mesa (the open-souce software OpenGL renderer) was fast enough, and not having to write all of the display transformation code by hand was a huge win even when the speed was somewhat lagging. -a
![](https://secure.gravatar.com/avatar/55f7acf47233a7a98f5eb9dfd0b2d763.jpg?s=120&d=mm&r=g)
I'd bet a case of beer (or cash equivalent) that one of the main bottlenecks is the path PySequence_GetItem->_ndarray_item->_universalIndexing->_simpleIndexing->_simpleIndexingCore. The path through _universalIndexing in particular, if I deciphered it correctly, looks very slow. I don't think it needs to be that way though, _universalIndexing could probably be sped up, but more promising I think _ndarray_item could be made to call _simpleIndexingCore without all that much work. It appears that this would save the creation of several intermediate objects and it also looks like a couple of calls back to python! I'm not familiar with this code though, so I could easily be missing something that makes calling _simpleIndexingCore harder than it looks. -tim
![](https://secure.gravatar.com/avatar/faf9400121dca9940496a7473b1d8179.jpg?s=120&d=mm&r=g)
On Tue, 2004-06-29 at 16:19, Tim Hochberg wrote:
I won't take the bet but if this works out, you get the beer. If it doesn't, well, I don't drink anymore anyway.
This looks very promising. I'll take a look tomorrow. Regards, Todd
participants (4)
-
Andrew P. Lentvorski, Jr.
-
Chris Barker
-
Tim Hochberg
-
Todd Miller