[Twisted-Python] [ANN] txZMQ released
Hi! I would like to announce txZMQ, ZeroMQ bindings for Twisted. txZMQ is based on pyzmq and requires recent ØMQ version (>=2.1). txZMQ uses ØMQ APIs to get file descriptor that is used to signal pending actions from ØMQ library IO thread running in separate thread. txZMQ should be usable with any Twisted reactor. ØMQ is: * The socket library that acts as a concurrency framework. * Carries messages across inproc, IPC, TCP, and multicast. * Connect N-to-N via fanout, pubsub, pipeline, request-reply. * Fast enough for clustered products and supercomputing. * Asynch I/O for scalable multicore message-passing apps. Bridging ØMQ and Twisted makes a nice match: fast simple messaging between Twisted instances solving complex problems :) Links: * PyPi: http://pypi.python.org/pypi/txZMQ/ * Source code: https://github.com/smira/pyzmq * pyzmq: http://pypi.python.org/pypi/pyzmq * ØMQ: http://www.zeromq.org/ -- Andrey Smirnov, Qik Web Team Lead
This is quite exciting! Thanks a lot for writing this, Andrey. I noticed that you put it in the same package name (pyzmq, not tx0MQ or anything). Is the goal to get this merged back upstream? (That would be extra great :-)) thanks again for your awesome efforts (I will take this for a spin soon), lvh
Laurens, Oops, I've posted wrong link to github: https://github.com/smira/txZMQ (I've posted link to first draft which has been actually fork of pyzmq, but turned out it would be easier to keep it as a separate package) 2011/4/28 Laurens Van Houtven <_@lvh.cc>
This is quite exciting! Thanks a lot for writing this, Andrey.
I noticed that you put it in the same package name (pyzmq, not tx0MQ or anything). Is the goal to get this merged back upstream? (That would be extra great :-))
thanks again for your awesome efforts (I will take this for a spin soon), lvh _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Andrey Smirnov, Qik Web Team Lead
Yeah, I saw that one and got confused. Just to be clear, I'm supposed to ignore your pyzmq fork, right?
Yes, I'll drop it as soon as I won't need it anymore (probably in one week). 2011/4/28 Laurens Van Houtven <_@lvh.cc>
Yeah, I saw that one and got confused. Just to be clear, I'm supposed to ignore your pyzmq fork, right?
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Andrey Smirnov, Qik Web Team Lead
Hiya, I have created a client-server application which uses ssl to communicate. I am trying to determine if its the server or client. However its definetly sending a string in the connection.transport.write to the target client. The majority of the time is sends and receives, but occasionally it fails. Is there a way (other than the client sending an ACK with a sequence number in it) to know if the write was successful and if not resend. Much appreciated, Gary C --- On Thu, 4/28/11, Andrey Smirnov <smirnov.andrey@gmail.com> wrote: From: Andrey Smirnov <smirnov.andrey@gmail.com> Subject: Re: [Twisted-Python] [ANN] txZMQ released To: "Twisted general discussion" <twisted-python@twistedmatrix.com> Date: Thursday, April 28, 2011, 7:53 AM Yes, I'll drop it as soon as I won't need it anymore (probably in one week). 2011/4/28 Laurens Van Houtven <_@lvh.cc> Yeah, I saw that one and got confused. Just to be clear, I'm supposed to ignore your pyzmq fork, right? _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- Andrey Smirnov, Qik Web Team Lead -----Inline Attachment Follows----- _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Apr 28, 2011, at 4:18 PM, gary clark wrote:
Hiya,
I have created a client-server application which uses ssl to communicate.
I am trying to determine if its the server or client. However its definetly sending a string in the connection.transport.write to the target client.
Trying to determine what is the server or the client?
The majority of the time is sends and receives, but occasionally it fails.
Fails how?
Is there a way (other than the client sending an ACK with a sequence number in it) to know if the write was successful and if not resend.
If you're using SSL, then you're using TCP, which transparently _does_ send an ACK with a sequence number in it. You never have to re-send TCP segments at the application layer, that would result in a stream with duplicate data in it. Please be more specific about the problem you're having, so we can find an answer :). -g
Much appreciated on the quick response. It fails by writing a large string at the server side and not receiving any data at the client. The client uses open-ssl and is blocking to receive data. I dont know if its possible to determine if the write was successful or not on the server side i.e knowing how many bytes were written etc. I dont see any exceptions. The connection is still maintained and like I said its intermittent. So when a transport.write occurs is there an api I can use to see if it was successful? I'm trying to isolate if its a server problem or a client issue at the moment. Thanks, Gary C --- On Thu, 4/28/11, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
From: Glyph Lefkowitz <glyph@twistedmatrix.com> Subject: Re: [Twisted-Python] problems with transport.write To: "Twisted general discussion" <twisted-python@twistedmatrix.com> Date: Thursday, April 28, 2011, 6:43 PM
On Apr 28, 2011, at 4:18 PM, gary clark wrote:
Hiya, I have created a client-server application which uses ssl to communicate. I am trying to determine if its the server or client. However its definetly sending a string in the connection.transport.write to the target client.
Trying to determine what is the server or the client?
The majority of the time is sends and receives, but occasionally it fails.
Fails how?
Is there a way (other than the client sending an ACK with a sequence number in it) to know if the write was successful and if not resend.
If you're using SSL, then you're using TCP, which transparently _does_ send an ACK with a sequence number in it. You never have to re-send TCP segments at the application layer, that would result in a stream with duplicate data in it.
Please be more specific about the problem you're having, so we can find an answer :).
-g
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
gary clark wrote:
Much appreciated on the quick response.
It fails by writing a large string at the server side and not receiving any data at the client. The client uses open-ssl and is blocking to receive data. I dont know if its possible to determine if the write was successful or not on the server side i.e knowing how many bytes were written etc. I dont see any exceptions. The connection is still maintained and like I said its intermittent. So when a transport.write occurs is there an api I can use to see if it was successful?
I'm trying to isolate if its a server problem or a client issue at the moment.
Most platforms' socket APIs don't provide a convenient (or necessarily any) way to see how much data has been ACKed by the peer (which I assume is what you mean by “if the write was successful”). So Twisted doesn't have any API for reporting that to you either. The usual way to observe how much data was sent is to run tcpdump (or wireshark or simliar). Similarly to observe how much data is received run tcpdump on the receiver. If there's a discrepancy then something between those two points is losing your data. Intermittent failures involving large writes may be a PMTU issue on your network. Try reducing the MTU value on your router(s) perhaps. -Andrew.
On Thu, Apr 28, 2011 at 8:39 PM, gary clark <burslem2001@yahoo.com> wrote:
It fails by writing a large string at the server side and not receiving any data at the client. The client uses open-ssl and is blocking to receive data. I dont know if its possible to determine if the write was successful or not on the server side i.e knowing how many bytes were written etc. I dont see any exceptions. The connection is still maintained and like I said its intermittent. So when a transport.write occurs is there an api I can use to see if it was successful?
Are you using threads on the server side? If so, are you using callFromThread? http://pythonquirks.blogspot.com/2011/01/twisted-callwhenrunning-callfromthr... Can you create a SSCCE (http://sscce.org/) and show it to us? Jason -- Jason Rennie Research Scientist, ITA Software 617-714-2645 http://www.itasoftware.com/
Congrats bro! stick to your course. С уважением, Александр Бурцев Skype: eburus FB: www.facebook.com/eburus http://bramabrama.com On 28 Apr 2011, at 16:53, Andrey Smirnov <smirnov.andrey@gmail.com> wrote:
Yes, I'll drop it as soon as I won't need it anymore (probably in one week).
2011/4/28 Laurens Van Houtven <_@lvh.cc> Yeah, I saw that one and got confused. Just to be clear, I'm supposed to ignore your pyzmq fork, right?
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Andrey Smirnov, Qik Web Team Lead _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Thank you Andrey for making this release possible ;) On Sun, Jul 20, 2014 at 9:20 AM, Alexander Burtsev <eburus@gmail.com> wrote:
Congrats bro! stick to your course.
С уважением, Александр Бурцев
Skype: eburus FB: www.facebook.com/eburus http://bramabrama.com
On 28 Apr 2011, at 16:53, Andrey Smirnov <smirnov.andrey@gmail.com> wrote:
Yes, I'll drop it as soon as I won't need it anymore (probably in one week).
2011/4/28 Laurens Van Houtven <_@lvh.cc>
Yeah, I saw that one and got confused. Just to be clear, I'm supposed to ignore your pyzmq fork, right?
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Andrey Smirnov, Qik Web Team Lead _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (8)
-
Alexander Burtsev -
Andrew Bennetts -
Andrey Smirnov -
gary clark -
Glyph Lefkowitz -
Jason Rennie -
Jonas Brunsgaard -
Laurens Van Houtven