Fastest way to get thousands of db records to client in 3 tier?

[Max M]

R. Alan Monroe wrote:


> I intend to write both client and server in Python, and my main goal 
> in the first cut is speed. Suggestions?

Why do you need thousand of rows at the client? Normally you would get 
the results in batches, so that you only transfer at most a few hundred 
at a time to the client.

I am not saying that you never need to do it. It's just rare. So perhaps 
all you need is a change in user-interface?

Anyhoo you can reduce the number of queries/connections to get better 
speed. It is easy to inadvertently make queries like:

for i in range(1000):
     "select * from table1 where some_number=i;"

Instead of:
     "select * from table1 where some_number=<1000;"

Normally it is not as simple as this, but the basic problem is the same.

But if your problem is that you transfer the data over the wire in xml 
format and that is too slow, you need to use another format. XML 
conversion can have a lot of overhead. Especially if it is many thousand 
rows.

Why not let the client ask directly if it is faster?

But if you can bring down the number of rows, using xml might not be a 
problem at all.



regards Max M