taking python enterprise level?...

[simn_stv]

On Feb 25, 5:18 pm, "D'Arcy J.M. Cain" <da... at druid.net> wrote:
> On Thu, 25 Feb 2010 15:29:34 +0000
> "Martin P. Hellwig" <martin.hell... at dcuktec.org> wrote:
>
> > On 02/25/10 13:58, D'Arcy J.M. Cain wrote:
> > > On Thu, 25 Feb 2010 02:26:18 -0800 (PST)
> > <cut>
> > > Our biggest problem was in
> > > a network heavy element of the app and that was low level TCP/IP stuff
> > > that rather than being Python's problem was something we used Python to
> > > fix.
> > <cut>
> > Out off interest, could you elaborate on that?
>
> Somewhat - there is an NDA so I can't give exact details.  It was
> crucial to our app that we sync up databases in Canada and the US (later
> Britain, Europe and Japan) in real time with those transactions.  Our
> problem was that even though our two server systems were on the
> backbone, indeed with the same major carrier, we could not keep them in
> sync.  We were taking way to long to transact an update across the
> network.
>
> The problem had to do with the way TCP/IP works, especially closer to
> the core.  Our provider was collecting data and sending it only after
> filling a buffer or after a timeout.  The timeout was short so it
> wouldn't normally be noticed and in most cases (web pages e.g.) the
> connection is opened, data is pushed and the connection is closed so
> the buffer is flushed immediately.  Our patterns were different so we
> were hitting the timeout on every single transaction and there was no
> way we would have been able to keep up.
>
> Our first crack at fixing this was to simply add garbage to the packet
> we were sending.  Making the packets an order of magnitude bigger sped
> up the proccessing dramatically.  That wasn't a very clean solution
> though so we looked for a better way.
>
> That better way turned out to asynchronous update transactions.  All we
> did was keep feeding updates to the remote site and forget about ACKS.
> We then had a second process which handled ACKS and tracked which
> packets had been properly transferred.  The system had IDs on each
> update and retries happened if ACKS didn't happen soon enough.
> Naturally we ignored ACKS that we had already processed.
>
> All of the above (and much more complexity not even discussed here) was
> handled by Python code and database manipulation.  There were a few
> bumps along the way but overall it worked fine.  If we were using C or
> even assembler we would not have sped up anything and the solution we
> came up with would have been horrendous to code.  As it was I and my
> chief programmer locked ourselves in the boardroom and had a working
> solution before the day was out.

sure it wouldnt have sped it up a bit, even a bit?; probably the
development and maintenance time would be a nightmare but it should
speed the app up a bit...

>
> Python wins again.
>
> --
> D'Arcy J.M. Cain <da... at druid.net>         |  Democracy is three wolveshttp://www.druid.net/darcy/               |  and a sheep voting on
> +1 416 425 1212     (DoD#0082)    (eNTP)   |  what's for dinner.

seriously added to the reputation of python, from my own
perspective...kudos python!