Using Python for processing of large datasets (convincing man agment)

[Delaney, Timothy]

> From: Thomas Jensen [mailto:spam at ob_scure.dk]
> 
> Delaney, Timothy wrote:
> 
> > "select a from T_A;select b from T_B;select c from T_C;"
> > 
> > Doing this should reduce your network latency massively.
> 
> I've used this method before, but never to reduce latency, thanks for 
> the advice!

It's actually got dual benefits.

1. You have reduced your latency by approx a third (you have one send - wait
for network - calculate - wait for network - receive cycle instead of
three);

2. The calculate - wait for network - receive portion can actually overlap -
while you are retrieving the data from the first query, the second and third
can still be calculating.

Another method which works well is to spin each of the selects off into its
own thread, then wait for each to finish. However, doing multiple selects in
a single request will almost certainly perform better (and have less code
change).

Tim Delaney