TCP packet size?

[chris]

I'll try a more specific explanation.  My application is pretty much a
multi-user application similar to Net Meeting or a chat room.  One major
piece of data that the users are sharing is a xml document (in memory
not a file).  One application is the host who accepts TCP connections
from any number of clients.   If a user changes the xml document, those
changes are sent to the host and then resent to each client.  Again, all
connections are TCP.   A problem arises when I send a large message
(3000-4000 bytes). Despite the fact that I give a large buffer size to
socket.read() method, I often receive that message dissembled.  Also,
sometimes I'll find small messages combined in the same read() call.
The messages just aren't reassembled the way I thought TCP worked.  I'm
hoping to find a way to received one message at a time in its entirety,
or determine the start and end of my messages.    I'm not really using
xml as a network protocol, its just the data structure of the
application.  I'd be happy to show you my networking module if you like
(you probably just cringed :) )

One way I tried to solve my problem was to break my messages into
smaller (1000 byte) messages that included a header.  However, even
though I broke the message up, it was not received in the same format.
Some of the message were combined and broken at unpredictable positions.

FYI, this might be silly to add, but I'm not creating a new TCP
connection for every message. I'm using the same connection repeatedly.

Thanks for the help.  I'm kind of at my wits end.

Mike Fletcher wrote:

> Hmm, I'm very surprised you're needing to get down to this level when
> you're using something as "high level" as XML as your encoding.  XML
> is so inefficient as a network protocol that most people would just
> use standard TCP streams (i.e. open a stream socket and let TCP take
> care of the ordering/guaranteeing/decomposition/reassembly), instead
> of trying to send TCP packets as raw packets (heck of a lot of work
> for no real benefit I can see).  Or do you mean that you're doing a
> network sniff and are upset that the packets are getting broken up and
> reassembled?I do lots of work with UDP-based protocols where these
> issues come up, but since UDP is non-guaranteeing, you wind up
> explicitly dealing with the issues regardless (they're not easy,
> incidentally, lots of soft tradeoffs dependent on current network
> conditions, required QOS, priority among messages, etceteras).  I tend
> to favour breaking messages up more (I target 256 bytes, include 26
> byte UDP header), rather than less, as small UDP packets may get
> through a crowded network, while larger UDP messages will get dropped
> when load is high (or at least, that's what certain articles have me
> believe :) ).  To do it right, of course, you need a dynamically
> adapting system that increases message size when load is low (and does
> a thousand other optimisations, as well).If I've missed the point,
> bear with me and explain again.  Since I've not actually grokked what
> the "problem" being tackled is, it's quite likely I have missed the
> point one way or another.  Enjoy yourself,Mike
> -----Original Message-----
> From: chris [mailto:chris at rpgarchive.com]
> Sent: Thursday, June 15, 2000 2:53 AM
> To: Python List
> Subject: TCP packet size?
> Is there a way to ensure or predict packet size when using TCP?  I'm
> developing a Multi-user application that functions much like a
> muli-player game.  The applications are sharing xml data.  The xml
> data can get very big 3000-6000 bytes.  Unfortunately, when the size
> of the network message gets over about 2000, its to get broken up into
> smaller packets.  So, when I send a 3000 byte message it gets cut up
> into 2-3 packets.  Anther problem I get is when I send two small
> messages back to back, they are sometimes received together.  Is there
> something I'm missing here?
>
> I tried using headers and breaking up messages into smaller chunks,
> but the size of the receiving packets doesn't always match what I
> sent.  I'd appreciate any help or suggestions on how I can tackle this
> problem.  I'd be happy to send someone my network code.
>
> Thanks
>
> --
> chris davis
> chris at rpgarchive.com
>
> http://www.rpgarchive.com
> news and adventure database!
>
> http://www.openrpg.com
> open source role playing!
>
>

--
chris davis
chris at rpgarchive.com

http://www.rpgarchive.com
news and adventure database!

http://www.openrpg.com
open source role playing!


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20000615/46ecaa46/attachment.html>