[Twisted-Python] xmlstream problem
Hello, I am using xmlstream.XmlStream to process...XML streams :-) I'm having a bit of a problem, I have a client and a server, both protocols inherit from xmlstream.XmlStream. When the client sends 4 messages, one after another, I get a parse error. I debugged my program, and the buffer the expat parser is given contains in effect, 4 messages consecutively like so : str: <message>...</message><message>...</message><message>...</message><message>...</message> (I replaced the child elements with "..." for your reading enjoyment) and I get this Exception when debugging (otherwise it stays hidden) : ExpatError: junk after document element: line 1, column 196 now column 196 is the end of the 1st msg's root element, I think it doesn't like the next message being right after the first. I did override xmlstream.XmlStream.onDocumentEnd() because I didn't want it to close the connection in between messages, but in my method I told it to initialize the stream using : self._initializeStream() I did this since I noticed that closing the connection forces a new parser to be created when the previous statement is called. The reason I don't want the connection to be closed is that I'd like a persistent connection to be held. Does anyone know how to either have the messages given to the parser as separate messages so that it doesn't freak out? Thank you, Gabriel
On Tue, 2008-05-20 at 17:50 +0200, Gabriel Rossetti wrote:
Hello,
I am using xmlstream.XmlStream to process...XML streams :-) I'm having a bit of a problem, I have a client and a server, both protocols inherit from xmlstream.XmlStream. When the client sends 4 messages, one after another, I get a parse error. I debugged my program, and the buffer the expat parser is given contains in effect, 4 messages consecutively like so :
str: <message>...</message><message>...</message><message>...</message><message>...</message>
(I replaced the child elements with "..." for your reading enjoyment)
and I get this Exception when debugging (otherwise it stays hidden) :
ExpatError: junk after document element: line 1, column 196
now column 196 is the end of the 1st msg's root element, I think it doesn't like the next message being right after the first.
That's right. Streaming XML, as implemented by twisted.words.xish, works by exchanging two complete (virtual) XML documents, one in each direction. So, you need to have a root element around your messages, such as 'stream'. The unit of communication is first-level elements of that root element. So, you start out with sending the start tag of the root element, and then sending the messages. You would set up an observer for your messages like so: def onMessage(message): print "Got message!" factory = XmlStreamFactory() factory.addBootstrap("/message", onMessage)
I did override xmlstream.XmlStream.onDocumentEnd() because I didn't want it to close the connection in between messages, but in my method I told it to initialize the stream using :
self._initializeStream()
I did this since I noticed that closing the connection forces a new parser to be created when the previous statement is called. The reason I don't want the connection to be closed is that I'd like a persistent connection to be held.
There is no need to do this, what I showed above should meet your requirements. Also, this method is semi-private (leading _) for a reason. -- Groetjes, ralphm
Ralph Meijer wrote:
On Tue, 2008-05-20 at 17:50 +0200, Gabriel Rossetti wrote:
Hello,
I am using xmlstream.XmlStream to process...XML streams :-) I'm having a bit of a problem, I have a client and a server, both protocols inherit from xmlstream.XmlStream. When the client sends 4 messages, one after another, I get a parse error. I debugged my program, and the buffer the expat parser is given contains in effect, 4 messages consecutively like so :
str: <message>...</message><message>...</message><message>...</message><message>...</message>
(I replaced the child elements with "..." for your reading enjoyment)
and I get this Exception when debugging (otherwise it stays hidden) :
ExpatError: junk after document element: line 1, column 196
now column 196 is the end of the 1st msg's root element, I think it doesn't like the next message being right after the first.
That's right. Streaming XML, as implemented by twisted.words.xish, works by exchanging two complete (virtual) XML documents, one in each direction. So, you need to have a root element around your messages, such as 'stream'. The unit of communication is first-level elements of that root element. So, you start out with sending the start tag of the root element, and then sending the messages.
You would set up an observer for your messages like so:
def onMessage(message): print "Got message!"
factory = XmlStreamFactory() factory.addBootstrap("/message", onMessage)
I did override xmlstream.XmlStream.onDocumentEnd() because I didn't want it to close the connection in between messages, but in my method I told it to initialize the stream using :
self._initializeStream()
I did this since I noticed that closing the connection forces a new parser to be created when the previous statement is called. The reason I don't want the connection to be closed is that I'd like a persistent connection to be held.
There is no need to do this, what I showed above should meet your requirements. Also, this method is semi-private (leading _) for a reason.
Ralph, Thank you, I understand better now, so I could send a "session" opening tag (e.g. <session_start>) from each side (one from the client when it connects and one from the server when the client connects), then exchange as manny messages as I want in between the two and then when I am done I send the closing tag (e.g. </session_start>)? Is that correct? Thanks, Gabriel
On Wed, 2008-05-21 at 16:06 +0200, Gabriel Rossetti wrote:
Ralph Meijer wrote:
On Tue, 2008-05-20 at 17:50 +0200, Gabriel Rossetti wrote:
Hello,
I am using xmlstream.XmlStream to process...XML streams :-) I'm having a bit of a problem, I have a client and a server, both protocols inherit from xmlstream.XmlStream. When the client sends 4 messages, one after another, I get a parse error. I debugged my program, and the buffer the expat parser is given contains in effect, 4 messages consecutively like so :
str: <message>...</message><message>...</message><message>...</message><message>...</message>
(I replaced the child elements with "..." for your reading enjoyment)
and I get this Exception when debugging (otherwise it stays hidden) :
ExpatError: junk after document element: line 1, column 196
now column 196 is the end of the 1st msg's root element, I think it doesn't like the next message being right after the first.
That's right. Streaming XML, as implemented by twisted.words.xish, works by exchanging two complete (virtual) XML documents, one in each direction. So, you need to have a root element around your messages, such as 'stream'. The unit of communication is first-level elements of that root element. So, you start out with sending the start tag of the root element, and then sending the messages.
You would set up an observer for your messages like so:
def onMessage(message): print "Got message!"
factory = XmlStreamFactory() factory.addBootstrap("/message", onMessage)
I did override xmlstream.XmlStream.onDocumentEnd() because I didn't want it to close the connection in between messages, but in my method I told it to initialize the stream using :
self._initializeStream()
I did this since I noticed that closing the connection forces a new parser to be created when the previous statement is called. The reason I don't want the connection to be closed is that I'd like a persistent connection to be held.
There is no need to do this, what I showed above should meet your requirements. Also, this method is semi-private (leading _) for a reason.
Ralph,
Thank you, I understand better now, so I could send a "session" opening tag (e.g. <session_start>) from each side (one from the client when it connects and one from the server when the client connects), then exchange as manny messages as I want in between the two and then when I am done I send the closing tag (e.g. </session_start>)? Is that correct?
Yeah, that sounds right. I suggest using <stream/> as the root element. -- Groetjes, ralphm
Ralph Meijer wrote:
On Wed, 2008-05-21 at 16:06 +0200, Gabriel Rossetti wrote:
Ralph Meijer wrote:
On Tue, 2008-05-20 at 17:50 +0200, Gabriel Rossetti wrote:
Hello,
I am using xmlstream.XmlStream to process...XML streams :-) I'm having a bit of a problem, I have a client and a server, both protocols inherit from xmlstream.XmlStream. When the client sends 4 messages, one after another, I get a parse error. I debugged my program, and the buffer the expat parser is given contains in effect, 4 messages consecutively like so :
str: <message>...</message><message>...</message><message>...</message><message>...</message>
(I replaced the child elements with "..." for your reading enjoyment)
and I get this Exception when debugging (otherwise it stays hidden) :
ExpatError: junk after document element: line 1, column 196
now column 196 is the end of the 1st msg's root element, I think it doesn't like the next message being right after the first.
That's right. Streaming XML, as implemented by twisted.words.xish, works by exchanging two complete (virtual) XML documents, one in each direction. So, you need to have a root element around your messages, such as 'stream'. The unit of communication is first-level elements of that root element. So, you start out with sending the start tag of the root element, and then sending the messages.
You would set up an observer for your messages like so:
def onMessage(message): print "Got message!"
factory = XmlStreamFactory() factory.addBootstrap("/message", onMessage)
I did override xmlstream.XmlStream.onDocumentEnd() because I didn't want it to close the connection in between messages, but in my method I told it to initialize the stream using :
self._initializeStream()
I did this since I noticed that closing the connection forces a new parser to be created when the previous statement is called. The reason I don't want the connection to be closed is that I'd like a persistent connection to be held.
There is no need to do this, what I showed above should meet your requirements. Also, this method is semi-private (leading _) for a reason.
Ralph,
Thank you, I understand better now, so I could send a "session" opening tag (e.g. <session_start>) from each side (one from the client when it connects and one from the server when the client connects), then exchange as manny messages as I want in between the two and then when I am done I send the closing tag (e.g. </session_start>)? Is that correct?
Yeah, that sounds right. I suggest using <stream/> as the root element.
Ralph, I just wrote a quick minimal client/server to test it, and it works great!, thanks a lot! Gabriel
I don't think that the method onDocumentEnd will be triggered on that error. I assume that the received xml is invalid and the code that is disconnecting the client is on the overridden dataReceived method of xmlstream.py: def dataReceived(self, data): """ Called whenever data is received. Passes the data to the XML parser. This can result in calls to the DOM handlers. If a parse error occurs, the L{STREAM_ERROR_EVENT} event is called to allow for cleanup actions, followed by dropping the connection. """ try: if self.rawDataInFn: self.rawDataInFn(data) self.stream.parse(data) except domish.ParserError: self.dispatch(self, STREAM_ERROR_EVENT) *### Put some code here that will write the received invalid xml data to a file* self.transport.loseConnection() --> *Your clients get disconnected here when sending an invalid xml* Since it's a pyExpat/domish error your client gets disconnected by the transport.loseConnection() call. I suggest that you put something under except to check if your xmpp client is sending an invalid xml. Cheers, Alvin On Tue, May 20, 2008 at 11:50 PM, Gabriel Rossetti <mailing_lists@evotex.ch> wrote:
Hello,
I am using xmlstream.XmlStream to process...XML streams :-) I'm having a bit of a problem, I have a client and a server, both protocols inherit from xmlstream.XmlStream. When the client sends 4 messages, one after another, I get a parse error. I debugged my program, and the buffer the expat parser is given contains in effect, 4 messages consecutively like so :
str: <message>...</message><message>...</message><message>...</message><message>...</message>
(I replaced the child elements with "..." for your reading enjoyment)
and I get this Exception when debugging (otherwise it stays hidden) :
ExpatError: junk after document element: line 1, column 196
now column 196 is the end of the 1st msg's root element, I think it doesn't like the next message being right after the first. I did override xmlstream.XmlStream.onDocumentEnd() because I didn't want it to close the connection in between messages, but in my method I told it to initialize the stream using :
self._initializeStream()
I did this since I noticed that closing the connection forces a new parser to be created when the previous statement is called. The reason I don't want the connection to be closed is that I'd like a persistent connection to be held.
Does anyone know how to either have the messages given to the parser as separate messages so that it doesn't freak out?
Thank you, Gabriel
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Wed, May 21, 2008 at 01:50:59AM +0800, Alvin Delagon wrote:
I don't think that the method onDocumentEnd will be triggered on that error. I assume that the received xml is invalid and the code that is disconnecting the client is on the overridden dataReceived method of xmlstream.py:
I'm pretty sure this not the problem, but the outcome of misunderstanding of the concept of XML Streams, as I explained in the other reply. As I said, your receiving stream will, at the end of the session, be a complete XML document, conceptually, and the XML parser is used with that idea in mind. So if you try to feed it more than one XML document, it will raise an exception. -- Groetjes, ralphm
participants (3)
-
Alvin Delagon -
Gabriel Rossetti -
Ralph Meijer