[Numpy-discussion] numpy ring buffer

Daniele Nicolodi daniele at grinta.net
Mon May 6 04:51:21 EDT 2013


Hello,

numpy arrays are great for interfacing python with libraries that expect
continuous memory buffers for data passing.  However, libraries
interfacing to data acquisition hardware often use those buffers as ring
buffers where, once the buffer has been filled with data, new data will
be written overwriting the data at the beginning of the buffer.  The
application is supposed to have read this data meanwhile.

Efficiently processing the data in the ring buffer requires addressing
efficiently the content of the ring buffer.  In C or Cython this is very
easy, just compute the wrap around when accessing the single elements of
the array:

  buffer = np.empty(size)
  adc.setup(buffer)
  start = 0
  for n in adc.read():
      # data processing
      for i in range(start, start + n):
          element = buffer[start % size]
	  ....
      start += n

My current approach to do the same thing in Python is to use the
np.roll() function to "linearize" the buffer access:

  buffer = np.empty(size)
  adc.setup(buffer)
  start = 0
  for n in adc.read():
      data = np.roll(buffer, -start)[:n]
      start += n
      # data processing
      process(data)

Since np.roll() returns a view on the array i suppose this is very
efficient.  Does anyone have a better idea on how to do this?

Thank you. Cheers,
Daniele



More information about the NumPy-Discussion mailing list