Parsing a serial stream too slowly

[Ulrich Eckhardt]

Am 23.01.2012 22:48, schrieb M.Pekala:
> I think that regex is too slow for this operation, but I'm uncertain
> of another method in python that could be faster. A little help would
> be appreciated.

Regardless of the outcome here, I would say that your code is still a 
bit wonky on the handling of partial data and gaps due to buffer 
overflows in the serial port buffers.

I'd start with something like this:

import unittest

class SensorReader(object):
     def __init__(self):
         pass
     def parse(self, buffer):
         """store and parse given input"""
         pass
     def get_sample(self):
         """retrieve a single sample parsed from input data

         This returns a tuple containing the sensor name and the
         value. In case some input data was not parseable, it
         returns a tuple with an empty string and the unrecognized
         data.

         """
         pass

class Test(unittest.TestCase):
     def setUp(self):
         self.handler = SensorReader()

     def test_empty_buffer(self):
         self.handler.parse(buffer='')
         r = self.handler.get_sample()
         self.assertEqual(r, None)

     def test_input_1(self):
         self.handler.parse(buffer='$A1234$')
         r = self.handler.get_sample()
         self.assertEqual(r, ('A', 1234))

     def test_input_2(self):
         self.handler.parse(buffer='$A1234$$B5678$')
         r = self.handler.get_sample()
         self.assertEqual(r, ('A', 1234))
         r = self.handler.get_sample()
         self.assertEqual(r, ('B', 5678))

     def test_invalid_input(self):
         self.handler.parse(buffer='234$$B5678$')
         r = self.handler.get_sample()
         self.assertEqual(r, ('', '234$'))
         r = self.handler.get_sample()
         self.assertEqual(r, ('B', 5678))

     def test_partial_input(self):
         self.handler.parse(buffer='$B56')
         r = self.handler.get_sample()
         self.assertEqual(r, None)
         self.handler.parse(buffer='78$$C90')
         r = self.handler.get_sample()
         self.assertEqual(r, ('B', 5678))

Concerning the parsing itself, there is a "find" function in the str 
class which you can use. Just search for dollar signs and take the data 
in between. If it's empty, the first one is the end and the second one 
the start of the next sample, so you can discard the content. Having 
tests as above will make it a breeze for you to figure out actual 
implementation. You could also start off with regexes and then switch to 
linear scanning for speed.


Uli