[Twisted-Python] Building a TCP server, having issues
Guys, I'll try to be as exact as I can be with my question, but it may end up being pretty broad. (I'm also relatively new to python, please go easy on my code although suggestions are of course welcome.) The issue I'm running into: When a user connects to the TCP server, they are displayed a welcome menu of things they can do. I'm simply catching the 'data' in an if statement to determine which menu item was picked. The problem is that some of the menu items require further input. I've had to use a series of bools to determine if a certain menu option was previously picked that will then directly send the data into the function / class. Is there an easier way to go about this? My current method seems a bit backwards and over complicated. I'm thinking I missed something in the documentation. Anyway, here's the code: https://infotomb.com/q4293 // Pass: twisted (it was part of a challenge, so to speak, don't take too much out of context) You can see I had trouble with next() and dh(). I'm now trying to move this TCP 'framework' over to another project (the above was already submitted), and I'm still running into the same issues. Thoughts? -A
On Feb 16, 2013, at 1:09 PM, Adam Heller <sk82712@gmail.com> wrote:
Guys, I'll try to be as exact as I can be with my question, but it may end up being pretty broad. (I'm also relatively new to python, please go easy on my code although suggestions are of course welcome.)
The main suggestion is "you can't parse TCP like this" :). You'll have this problem: <http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#Whyisprotocol.da...>
The issue I'm running into: When a user connects to the TCP server, they are displayed a welcome menu of things they can do. I'm simply catching the 'data' in an if statement to determine which menu item was picked. The problem is that some of the menu items require further input. I've had to use a series of bools to determine if a certain menu option was previously picked that will then directly send the data into the function / class. Is there an easier way to go about this? My current method seems a bit backwards and over complicated. I'm thinking I missed something in the documentation.
Anyway, here's the code: https://infotomb.com/q4293 // Pass: twisted
You should really attach the code itself to the email, so that future readers of the archive have some idea what you're talking about. Pastebins in general are not great, since the code will expire before your archived message does, and pastebins with passwords (!?) are even worse.
(it was part of a challenge, so to speak, don't take too much out of context)
You can see I had trouble with next() and dh(). I'm now trying to move this TCP 'framework' over to another project (the above was already submitted), and I'm still running into the same issues.
You should try separating out your concerns a bit. The protocol should just parse the bytes into something meaningful, then call methods on some other, higher-level object that isn't just manipulating strings. The "other, higher-level object" in your case might be a state machine of some kind, or you might just have an object representing the mode that you're currently in and switch that out for different modes. This isn't an issue with Twisted though; Twisted just calls some methods on your objects in order to tell you what happened; what happens next is up to you. So it's a general program architecture issue. This is very general advice, obviously, because I'm not sure what you are specifically asking, nor what your program is really intended to do. What kind of client is talking to this system? Did you invent the protocol yourself, or did it come from somewhere else? What is 'dh()' even trying to accomplish? Etc, etc. -glyph
Glyph, I will definately look into that. As far as the server spec, it's over TOR (This part of the 'competition' has since closed, but feel free to catch up: http://uncovering-cicada.wikia.com/wiki/Uncovering_Cicada_Wiki and http://mentalfloss.com/article/31932/chasing-cicada-exploring-darkest-corrid... last year's 3301). I'm simply trying to do the python thing and move classes around instead of trying to reinvent the wheel [dh() is a rudimentary Diffie Helman key exchange] : " In the programming language of your choice build a TCP server that implements the protocol below. The server code must be written by you and you alone, although you are free to use any modules or libraries publicly available for the selected programming language. Once you have done this, make it accessible as a Tor hidden service. Then provide us with the onion address and port via a GPG-encrypted email to this address. You have until 0:00 UTC on 3 Feb, 2013. Any emails received after that time will be ignored. Good luck. 3301 ==================================================================== 1. INTRODUCTION The TCP server MUST listen on an arbitrary port, and send and receive plain text with lines separated by <CRLF> (representing a carriage return followed by a line feed). The TCP server MUST disregard the case of input. In the examples below, lines sent by the server will be preceded with "S:" and lines sent by the client will be preceded by "C:" Each message sent by the server MUST conform to the format: [CODE] [RESPONSE NAME] [RESPONSE (optional)]<CRLF> Where [CODE] and [RESPONSE NAME] is one of: CODE RESPONSE NAME 00 Welcome 01 Ok 02 Error 03 Data 99 Goodbye 2. PROCEDURES a. Remote Connection Upon receiving a remote connection, the server MUST greet the client with a 00 WELCOME message. The RESPONSE of a welcome message MAY contain arbitrary text. The arbitrary text MUST at the very least contain the name of the programming language used to implement the server. Upon receiving a 00 WELCOME message, the client may begin initiating procedures. Example: S: 00 WELCOME [ARBITRARY RESPONSE TEXT]<CRLF> b. RAND [n] Upon receiving a "RAND" request by the client, the server will first send a 01 OK response, and will then provide the client with [n] cryptographically random numbers within the range of 0-255. Each number MUST be followed by <CRLF>. After the last number has been sent, the server MUST send a dot (.) on a line by itself. Example: C: RAND 3<CRLF> S: 01 OK<CRLF> S: [first random number]<CRLF> S: [second random number]<CRLF> S: [third random number]<CRLF> S: .<CRLF> c. QUINE Upon receiving a "QUINE" request by the client, the server will first send a 01 OK response, and will then provide the client with a quine in the programming language used to implement the server. This quine does not have to be original. After the last line of code has been sent, the server MUST send a dot (.) on a line by itself. Example: C: QUINE<CRLF> S: 01 OK<CRLF> S: [quine code]<CRLF> S: .<CRLF> d. BASE29 [n] Upon receiving a "BASE29" request by the client, the server will send a 01 OK response followed by the number [n] converted into its base 29 representation. Example: C: BASE29 3301<CRLF> S: 01 OK 3QO<CRLF> e. CODE Upon receiving a "CODE" request by the client, the server will send a 01 OK response followed by its own source code. After the last line of code has been sent, the server MUST send a dot(.) on a line by itself. Example: C: CODE<CRLF> S: 01 OK<CRLF> S: [Server Source Code]<CRLF> s: .<CRLF> f. KOAN Upon receiving a "KOAN" request by the client, the server will send a 01 OK response followed by a koan. After the last line of the koan, the server MUST send a dot (.) on a line by itself. Example: C: KOAN<CRLF> S: 01 OK<CRLF> S: A master who lived as a hermit on a mountain was asked by a<CRLF> S: monk, "What is the Way?<CRLF> S: "What a fine mountain this is," the master said in reply<CRLF> S: "I am not asking you about the mountain, but about the Way.<CRLF> S: "So long as you cannot go beyond the mountain, my son, you<CRLF> S: cannot reach the Way," replied the master<CRLF> S: . g. DH [p] Upon receiving a "DH" request by the client, the server will proceed to perform a Diffie-Hellman key exchange using [p] as the prime modulus. The server will then select a base [b] to use in the protocol, as well as its secret integer. The server will then compute its exponent result [e] as specified within the Diffie-Hellman key exchange protocol. The server MUST then respond with a 01 OK response followed by the selected base [b] and computed exponent [e] separated by white space. The client MUST respond with its exponent result [e2], and the client and server will follow the rest of the Diffie-Hellman key exchange protocol. The server MUST then compute the resulting secret key, and provide it using 03 DATA [k]. Example: C: DH 23<CRLF> S: 01 OK 5 8<CRLF> C: 19<CRLF> S: 03 DATA 2<CRLF> j. NEXT Upon receiving a "NEXT" request by the client, the server will respond with 01 OK and then listen for text data to be provided by the client. The client will send a dot (.) on a line by itself after the last line of text. The server MUST record this. This data will be the next set of instructions. Once the data is received the server will respond with 01 OK. Example: C: NEXT<CRLF> S: 01 OK<CRLF> C: -----BEGIN PGP SIGNED MESSAGE-----<CRLF> C: [MESSAGE CONTENTS]<CRLF> C: -----END PGP SIGNATURE-----<CRLF> C: .<CRLF> S: 01 OK<CRLF> i. GOODBYE Upon receiving a "DH" request by the client, the server MUST respond with 99 GOODBYE and then gracefully close the connection. Example: C: GOODBYE<CRLF> S: 99 GOODBYE<CRLF> ===================================================================="
From a signed GPG message.
On Sat, Feb 16, 2013 at 11:53 PM, Glyph <glyph@twistedmatrix.com> wrote:
On 02/17/2013 01:09 AM, Adam Heller wrote:
Glyph has already mentioned the buffering and state machine options, but since it's a line-based protocol, you chould also look at t.p.basic.LineReceiver: http://twistedmatrix.com/documents/current/api/twisted.protocols.basic.LineR... ...which does the buffering for you and calls a method with full "lines". You may want to look at the source for that class' dataReceived method, for an example of how it does the buffering Glyph talks about. There are other examples of doing state machine dispatch in the Twisted sources, but it's a pretty simple technique - set a "state" variable, and a big if/then clause (or, for performance, a dict of state->handler)
Hello, As wrote Phil Mayers LinReceiver is a nice protocol helpers for you. All the commands use a CRLF delimiter and for your logic it is easy to know when a command is fully received or if you need to wait for more data. As wrote Glyph dataReceived (or lineReceived) Should only use to received data, sanitize them like gather the command name, and the arguments, and pass it to another big function named handleCommand that will use the big if/else condition to know what to do. regards, Sofiane Akermoun 2013/2/18 Phil Mayers <p.mayers@imperial.ac.uk>:
-- Sofiane AKERMOUN akersof@gmail.com
On Feb 16, 2013, at 1:09 PM, Adam Heller <sk82712@gmail.com> wrote:
Guys, I'll try to be as exact as I can be with my question, but it may end up being pretty broad. (I'm also relatively new to python, please go easy on my code although suggestions are of course welcome.)
The main suggestion is "you can't parse TCP like this" :). You'll have this problem: <http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#Whyisprotocol.da...>
The issue I'm running into: When a user connects to the TCP server, they are displayed a welcome menu of things they can do. I'm simply catching the 'data' in an if statement to determine which menu item was picked. The problem is that some of the menu items require further input. I've had to use a series of bools to determine if a certain menu option was previously picked that will then directly send the data into the function / class. Is there an easier way to go about this? My current method seems a bit backwards and over complicated. I'm thinking I missed something in the documentation.
Anyway, here's the code: https://infotomb.com/q4293 // Pass: twisted
You should really attach the code itself to the email, so that future readers of the archive have some idea what you're talking about. Pastebins in general are not great, since the code will expire before your archived message does, and pastebins with passwords (!?) are even worse.
(it was part of a challenge, so to speak, don't take too much out of context)
You can see I had trouble with next() and dh(). I'm now trying to move this TCP 'framework' over to another project (the above was already submitted), and I'm still running into the same issues.
You should try separating out your concerns a bit. The protocol should just parse the bytes into something meaningful, then call methods on some other, higher-level object that isn't just manipulating strings. The "other, higher-level object" in your case might be a state machine of some kind, or you might just have an object representing the mode that you're currently in and switch that out for different modes. This isn't an issue with Twisted though; Twisted just calls some methods on your objects in order to tell you what happened; what happens next is up to you. So it's a general program architecture issue. This is very general advice, obviously, because I'm not sure what you are specifically asking, nor what your program is really intended to do. What kind of client is talking to this system? Did you invent the protocol yourself, or did it come from somewhere else? What is 'dh()' even trying to accomplish? Etc, etc. -glyph
Glyph, I will definately look into that. As far as the server spec, it's over TOR (This part of the 'competition' has since closed, but feel free to catch up: http://uncovering-cicada.wikia.com/wiki/Uncovering_Cicada_Wiki and http://mentalfloss.com/article/31932/chasing-cicada-exploring-darkest-corrid... last year's 3301). I'm simply trying to do the python thing and move classes around instead of trying to reinvent the wheel [dh() is a rudimentary Diffie Helman key exchange] : " In the programming language of your choice build a TCP server that implements the protocol below. The server code must be written by you and you alone, although you are free to use any modules or libraries publicly available for the selected programming language. Once you have done this, make it accessible as a Tor hidden service. Then provide us with the onion address and port via a GPG-encrypted email to this address. You have until 0:00 UTC on 3 Feb, 2013. Any emails received after that time will be ignored. Good luck. 3301 ==================================================================== 1. INTRODUCTION The TCP server MUST listen on an arbitrary port, and send and receive plain text with lines separated by <CRLF> (representing a carriage return followed by a line feed). The TCP server MUST disregard the case of input. In the examples below, lines sent by the server will be preceded with "S:" and lines sent by the client will be preceded by "C:" Each message sent by the server MUST conform to the format: [CODE] [RESPONSE NAME] [RESPONSE (optional)]<CRLF> Where [CODE] and [RESPONSE NAME] is one of: CODE RESPONSE NAME 00 Welcome 01 Ok 02 Error 03 Data 99 Goodbye 2. PROCEDURES a. Remote Connection Upon receiving a remote connection, the server MUST greet the client with a 00 WELCOME message. The RESPONSE of a welcome message MAY contain arbitrary text. The arbitrary text MUST at the very least contain the name of the programming language used to implement the server. Upon receiving a 00 WELCOME message, the client may begin initiating procedures. Example: S: 00 WELCOME [ARBITRARY RESPONSE TEXT]<CRLF> b. RAND [n] Upon receiving a "RAND" request by the client, the server will first send a 01 OK response, and will then provide the client with [n] cryptographically random numbers within the range of 0-255. Each number MUST be followed by <CRLF>. After the last number has been sent, the server MUST send a dot (.) on a line by itself. Example: C: RAND 3<CRLF> S: 01 OK<CRLF> S: [first random number]<CRLF> S: [second random number]<CRLF> S: [third random number]<CRLF> S: .<CRLF> c. QUINE Upon receiving a "QUINE" request by the client, the server will first send a 01 OK response, and will then provide the client with a quine in the programming language used to implement the server. This quine does not have to be original. After the last line of code has been sent, the server MUST send a dot (.) on a line by itself. Example: C: QUINE<CRLF> S: 01 OK<CRLF> S: [quine code]<CRLF> S: .<CRLF> d. BASE29 [n] Upon receiving a "BASE29" request by the client, the server will send a 01 OK response followed by the number [n] converted into its base 29 representation. Example: C: BASE29 3301<CRLF> S: 01 OK 3QO<CRLF> e. CODE Upon receiving a "CODE" request by the client, the server will send a 01 OK response followed by its own source code. After the last line of code has been sent, the server MUST send a dot(.) on a line by itself. Example: C: CODE<CRLF> S: 01 OK<CRLF> S: [Server Source Code]<CRLF> s: .<CRLF> f. KOAN Upon receiving a "KOAN" request by the client, the server will send a 01 OK response followed by a koan. After the last line of the koan, the server MUST send a dot (.) on a line by itself. Example: C: KOAN<CRLF> S: 01 OK<CRLF> S: A master who lived as a hermit on a mountain was asked by a<CRLF> S: monk, "What is the Way?<CRLF> S: "What a fine mountain this is," the master said in reply<CRLF> S: "I am not asking you about the mountain, but about the Way.<CRLF> S: "So long as you cannot go beyond the mountain, my son, you<CRLF> S: cannot reach the Way," replied the master<CRLF> S: . g. DH [p] Upon receiving a "DH" request by the client, the server will proceed to perform a Diffie-Hellman key exchange using [p] as the prime modulus. The server will then select a base [b] to use in the protocol, as well as its secret integer. The server will then compute its exponent result [e] as specified within the Diffie-Hellman key exchange protocol. The server MUST then respond with a 01 OK response followed by the selected base [b] and computed exponent [e] separated by white space. The client MUST respond with its exponent result [e2], and the client and server will follow the rest of the Diffie-Hellman key exchange protocol. The server MUST then compute the resulting secret key, and provide it using 03 DATA [k]. Example: C: DH 23<CRLF> S: 01 OK 5 8<CRLF> C: 19<CRLF> S: 03 DATA 2<CRLF> j. NEXT Upon receiving a "NEXT" request by the client, the server will respond with 01 OK and then listen for text data to be provided by the client. The client will send a dot (.) on a line by itself after the last line of text. The server MUST record this. This data will be the next set of instructions. Once the data is received the server will respond with 01 OK. Example: C: NEXT<CRLF> S: 01 OK<CRLF> C: -----BEGIN PGP SIGNED MESSAGE-----<CRLF> C: [MESSAGE CONTENTS]<CRLF> C: -----END PGP SIGNATURE-----<CRLF> C: .<CRLF> S: 01 OK<CRLF> i. GOODBYE Upon receiving a "DH" request by the client, the server MUST respond with 99 GOODBYE and then gracefully close the connection. Example: C: GOODBYE<CRLF> S: 99 GOODBYE<CRLF> ===================================================================="
From a signed GPG message.
On Sat, Feb 16, 2013 at 11:53 PM, Glyph <glyph@twistedmatrix.com> wrote:
On 02/17/2013 01:09 AM, Adam Heller wrote:
Glyph has already mentioned the buffering and state machine options, but since it's a line-based protocol, you chould also look at t.p.basic.LineReceiver: http://twistedmatrix.com/documents/current/api/twisted.protocols.basic.LineR... ...which does the buffering for you and calls a method with full "lines". You may want to look at the source for that class' dataReceived method, for an example of how it does the buffering Glyph talks about. There are other examples of doing state machine dispatch in the Twisted sources, but it's a pretty simple technique - set a "state" variable, and a big if/then clause (or, for performance, a dict of state->handler)
Hello, As wrote Phil Mayers LinReceiver is a nice protocol helpers for you. All the commands use a CRLF delimiter and for your logic it is easy to know when a command is fully received or if you need to wait for more data. As wrote Glyph dataReceived (or lineReceived) Should only use to received data, sanitize them like gather the command name, and the arguments, and pass it to another big function named handleCommand that will use the big if/else condition to know what to do. regards, Sofiane Akermoun 2013/2/18 Phil Mayers <p.mayers@imperial.ac.uk>:
-- Sofiane AKERMOUN akersof@gmail.com
participants (4)
-
Adam Heller -
Glyph -
Phil Mayers -
Sofiane Akermoun