[Tutor] multipart socket streaming problem: reading the stream
richard kappler
richkappler at gmail.com
Tue Jan 5 10:38:51 EST 2016
This is a continuation of the thread 'reading an input stream' I had to
walk away from for a few days due to the holidays and then other work
considerations, and I figured it best to break my confusion into separate
chunks, I hope that's appropriate. In short, my script needs to read a
stream of xml data from a socket (port 2008), the data coming in from as
many as 30 different machines, but usually 4 or less, as many as 3 messages
per second from each machine at times, messages block format delimited by
stx(\x02) and etx (\x03), send the data in those blocks to a parser
(already built using lxml and an xslt file) and send it out to splunk using
a native 'event writer'.
I am a bit lost in the woods on 'reading' the stream. My first attempt
tried to read the stream directly (from the buffer 'I think'):
#!/usr/bin/env python
import socket
import lxml.etree as ET
# receive socket
sockrx = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sockrx_address = ('', 2008)
print 'opening sockrx on %s port %s' % sockrx_address
try:
sockrx.bind(sockrx_address)
except socket.error as msg:
print 'Bind failed. Error code: ' + str(msg)
sys.exit()
sockrx.listen(5)
print 'listening'
while True:
# wait for a connection
connection, client_address = sockrx.accept()
data = connection.recv(8192)
if data:
print 'receiving data'
f1 = open('parser.out', 'a')
xslt = ET.parse('stack13.xsl')
dom = ET.parse(data)
transform = ET.XSLT(xslt)
newdom = transform(dom)
f1.write(str(newdom))
# close sockrx
connection.close()
f1.close()
This did not work:
Traceback (most recent call last):
File "streamer-01.py", line 27, in <module>
dom = ET.parse(data)
File "lxml.etree.pyx", line 3239, in lxml.etree.parse
(src/lxml/lxml.etree.c:69955)
File "parser.pxi", line 1748, in lxml.etree._parseDocument
(src/lxml/lxml.etree.c:102066)
File "parser.pxi", line 1774, in lxml.etree._parseDocumentFromURL
(src/lxml/lxml.etree.c:102330)
File "parser.pxi", line 1678, in lxml.etree._parseDocFromFile
(src/lxml/lxml.etree.c:101365)
File "parser.pxi", line 1110, in lxml.etree._BaseParser._parseDocFromFile
(src/lxml/lxml.etree.c:96817)
File "parser.pxi", line 582, in
lxml.etree._ParserContext._handleParseResultDoc
(src/lxml/lxml.etree.c:91275)
File "parser.pxi", line 683, in lxml.etree._handleParseResult
(src/lxml/lxml.etree.c:92461)
File "parser.pxi", line 620, in lxml.etree._raiseParseError
(src/lxml/lxml.etree.c:91722)
IOError: Error reading file
and then the error message printed out the entire file to screen, which I
won't here for brevity.
I was, however, able to receive the the stream as above, write it to a
file, then read the file to the parser and it worked. I tired using
makefile() but that did not work either. I'm looking for some general
direction here, more specific questions to follow.
regards, Richard
More information about the Tutor
mailing list