Fastest way to get thousands of db records to client in 3 tier?

Chris Reedy creedy at mitretek.org
Thu Sep 25 23:20:17 CEST 2003


R. Alan Monroe wrote:

> Subject says it all... what's the fastest way to get many thousands of 
> records to the client?
> 
> I'm trying to sketch out a traditional 3 tier client-server system. 
> Multiple clients would make requests to the server. The server should 
> execute sql select commands against a server-side database, then 
> return the results to the client.
> 
> I tried the standard xmlrpclib and SimpleXMLRPCServer tools that came 
> with Python. I liked the simplicity, but it was about 30-35 times 
> slower than executing the sql directly from the client...
> 
> I intend to write both client and server in Python, and my main goal 
> in the first cut is speed. Suggestions?
> 
> Alan

I suspect that it will be difficult for you to match the speed of using 
sql to do direct record retrieval from the client. People who build 
databases spend a lot of time and effort making this work as fast as 
they can because of its impact on overall database performance.

My suggestion: You need to think harder about your performance versus 
functionality trade-offs. E.g., if you're not moving too many records 
and the server isn't heavily used, you shouldn't care about the extra 
overhead.

On the other hand, if you're moving a lot of records _and_ the server is 
heavily used, then you're trying for bleeding edge performance and you 
should either expect to spend a lot of time making ugly compromises in 
order to meet your requirements or to spend money buying a bigger server 
and more communications bandwidth.

   Chris



-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----==  Over 100,000 Newsgroups - 19 Different Servers! =-----




More information about the Python-list mailing list