[Twisted-Python] Knowing end of packet
Hello, I am sending a large amount of data to a twisted server. I want to know how the protocol class knows that no more data is to be received. I dont pass the length of the data in the packet, hence I'm looking for a termination field in the packet.Like </MESSAGE>. I did see a logging class but not sure if thats what I want. If I have to overload the dataReceived method, and buffer the data until I see the termination field thats ok, just need to know if this seems correct in twisted. Much appreciated for any help. Thanks, Garyc
On Mon, Sep 28, 2009 at 7:15 AM, gary clark <burslem2001@yahoo.com> wrote:
Hello,
I am sending a large amount of data to a twisted server. I want to know how the protocol class knows that no more data is to be received.
I dont pass the length of the data in the packet, hence I'm looking for a termination field in the packet.Like </MESSAGE>.
I think this FAQ might help you: http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#Whyisprotocol.da... Let us know if that answers your question.
Marvelous. Much appreciated. "from twisted.protocols.basic import NetstringReceiver" Thanks, Garyc --- On Mon, 9/28/09, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
From: Glyph Lefkowitz <glyph@twistedmatrix.com> Subject: Re: [Twisted-Python] Knowing end of packet To: "Twisted general discussion" <twisted-python@twistedmatrix.com> Date: Monday, September 28, 2009, 6:29 AM On Mon, Sep 28, 2009 at 7:15 AM, gary clark <burslem2001@yahoo.com> wrote:
Hello,
I am sending a large amount of data to a twisted server. I want to know how
the protocol class knows that no more data is to be received.
I dont pass the length of the data in the packet, hence I'm looking for a termination field in the packet.Like </MESSAGE>.
I think this FAQ might help you:
http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#Whyisprotocol.da...
Let us know if that answers your question.
-----Inline Attachment Follows-----
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Hello, I have a server that needs to track changes on a table database. When a table changes it needs the server accesses the table and send data out. Not sure what the best approach to do this in twisted. Do I create a thread in twisted and monitor for any changes in a table or can I create a callback method to be triggered when a change occurs to table. Thanks, Garyc
On 28 Sep, 01:07 pm, burslem2001@yahoo.com wrote:
Hello,
I have a server that needs to track changes on a table database.
When a table changes it needs the server accesses the table and send data out.
Not sure what the best approach to do this in twisted. Do I create a thread in twisted and monitor for any changes in a table or can I create a callback method to be triggered when a change occurs to table.
There's nothing special in Twisted for this. If your database provides callback functionality, then you should be able to hook this up to your Twisted application somehow. If you provide some more information about this, someone may be able to help you figure out how to do that. If your database doesn't, then you either need a separate out-of-band notification when updates happen or you need to poll the database. Jean-Paul
gary clark wrote:
Hello,
I have a server that needs to track changes on a table database.
When a table changes it needs the server accesses the table and send data out.
Not sure what the best approach to do this in twisted. Do I create a thread in twisted and monitor for any changes in a table or can I create a callback method to be triggered when a change occurs to table.
Check out twisted.internet.task.LoopingCall. It will run a callback periodically, during which you can poll for changes. -- Mark Visser, Software Director Lumière VFX Email: markv@lumierevfx.com Phone: +1-514-316-1080 x3030
Thanks for the information, thats what I need to use in this case "wisted.internet.task.LoopingCall". Great support. Much appreciated, Garyc
Hello, Probably a pretty standard question. However what are recommended mechanics of parsing XML on twisted? I have a humongous string that needs to be parsed and pushed into a database in the right columns. Much appreciated in advance, Garyc
On 1 Oct, 05:53 pm, burslem2001@yahoo.com wrote:
Hello,
Probably a pretty standard question. However what are recommended mechanics of parsing XML on twisted? I have a humongous string that needs to be parsed and pushed into a database in the right columns.
Depending on how big the strings are, you may just want to parse them in the obvious way and then deal with the results. If the strings are really epically big, then you have a few options. You can handle them in another thread in the usual way. twisted.internet.threads.deferToThread gives you easy access to a threadpool which you can use for tasks like this. You can hand them off to another process and deal with them there. Twisted has child process control built in, via reactor.spawnProcess. You may also find the Ampoule library (not part of Twisted) handy for this. You can also do the XML parsing incrementally. The Python standard library includes a SAX parser which you might want to use for this. I think the newer APIs (eg etree) also support some forms of incremental parsing. This should let you spread out the task of handling the XML over a longer period of time, thus avoiding blocking the reactor thread for unreasonable amounts of time. Jean-Paul
participants (4)
-
exarkun＠twistedmatrix.com -
gary clark -
Glyph Lefkowitz -
Mark Visser