asyncore and OOB data

[Steve Holden]

Douglas Wells wrote:
> In article <1184172867.420979.215300 at q75g2000hsh.googlegroups.com>,
>  billiejoex <gnewsg at gmail.com> writes:
>> In an asyncore based FTP server I wrote I should be able to receive
>> OOB data from clients.
>> A client sending such kind of data should act like this:
>>
>>>>> import socket
>>>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>>>> s.connect(('127.0.0.1', 21))
>>>>> s.sendall('hello there\r\n', socket.MSG_OOB)
>> According to asyncore documentation I should handle this kind of event
>> in "handle_expt" method of dispatcher class, that should be called
>> when OOB is received by the underlying socket. I've tried to override
>> handle_expt method in such way:
>>
>>     def handle_expt(self):
>>         print "OOB data arrived"
>>         data = self.socket.recv(1024, socket.MSG_OOB)
>>         print data
>>
>> ...but, even if it is called, "data" contains only a "\n" character
>> instead of the entire string ("hello there\r\n").
>> Why does this happen? Should I have to use a different approach?
> 
> You should use a different approach, but first you need to read
> the relevant documentation.
> 
> I've never used asyncore, but when I look at the documentation,
> the only reference that I find to OOB says:
> 	"Called when there is out of band (OOB) data for a socket
> 	connection.  This will almost never happen, as OOB is
> 	tenuously supported and rarely used."
> That should be fair warning.
> 
> Second, when I look at the FTP specification, I don't find the
> concept of OOB anywhere.  So, it's not clear what OOB data would
> mean in terms of the defined FTP commands in any case.
> 
> Third, the TCP protocol, which you have selected via the SOCK_STREAM
> option doesn't support OOB at all, so there's no way that you can
> even send OOB data in the manner that you are expecting.
> 
Wrong. The URGENT pointer indicates that the data begins with 
out-of-band data. You are correct, however, in stating that the FTP 
protocol doesn't support or implement out-of-band data transmissions.

> Let's switch to the positive, however.  What TCP does have is the
> concept of "urgent" data.  Urgent data is data that the sender
> believes is so important that the receiver should discard intermediate
> data, if necessary, in order to process the urgent data -- but the
> urgent data is sent *in-band*.
> 
No, technically the urgent data is out-of-band, isn't it? There is no 
requirement to ditch the other data in the packet, so urgent data can be 
interspersed without loss of regular in-band data if required.

> In the case of FTP, the concept of urgent data is used to allow
> the user side to interrupt the data transfer, which it does by
> sending an ABOR command.  The underlying problem is that the server
> might be blocked on a receive on the data connection and not
> listening for commands on the control connection.  So the user
> side is required to perform some "special action" that includes
> sending urgent data, which the server can then process specially.
> This problem arises primarily in systems without either threading
> or asynchronous I/O APIs.  Your use of asyncore should alleviate
> this problem.
> 
The data and control connections are typically dealt with in most 
implementations by separate processes, so an ABOR command can be sent 
without having to wait even if the data connection is blocked. Of course 
if a specific implementation uses a single-threaded single process to 
drive both the data and control connections then all bets are off.

> The thing that may be confusing you is that for whatever reason
> the designers of the Berkeley socket API (which is what you are
> using when running under POSIX/Linux/MS Windows), chose to use the
> OOB flags (e.g., MSG_OOB) to indicate the *transmission* of OOB
> data -- but it's not really OOB data.  In fact, when I attempt to
> receive using the MSG_OOB flag using the OS interfaces on my system,
> I get back an error (EINVAL - Invalid argument).  I would expect
> that asyncore would report this error somehow or other.
> 
You can, of course, expect what you like, but experience is the best 
guide to what will actually happen.

> In summary, you almost certainly can't use the concept of a separate
> OOB channel to transfer data in an FTP environment.  If you really,
> really need something like this, you will need to extend the FTP
> protocol for your special purposes.
> 
I'd certainly concur with that: anything that's processing FTP streams 
will almost certainly barf when urgent data arrives, since this is 
outside the scope of the FTP protocol specification.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd           http://www.holdenweb.com
Skype: holdenweb      http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------