[Tutor] web application and database caching technique

[anil maran]

i m trying to build a webapplication, i need to cache db records as it has become very slow the sql queries and retrival
can you guys helpe me out in designing cache structures/ hash tables for these cases 


1. Scenario One: 
    * Huge number of reads 
    * Few writes 
    * One row accessed from db per request 
    * All data written to db is used. 
    * Medium number of records. 
 
Since the large number of read need to be fast I chose this design: 
 
A hashtable caches records. They are cleared based on expiration time 
 and limit on maximum number of cached records. When a record is updated 
 in the db, it is deleted from the cache and recached next time it is 
 queried. This part could be handled by implementation you linked to. 
 
A persistent background thread periodically flushes the changed/new 
 values in the cache to the database. Additional calculations could be 
 added here to chose the optimal cache entrees to delete or precache. 
 
2. Scenario Two: 
    * Few reads 
    * Huge number of writes 
    * Many rows accessed from db per request 
    * Most data written to db is discarded. 
    * Huge number of small records. 
 
This is the strangest so far. Because of the huge number of records, 
 most of which will just be discarded anyway, I try to keep away from 
 the database as much as possible. When it's time to flush them to the 
 db I aggregate the records into groups, cpickle, and lzo them. I am 
 still playing with this but I notice a good performance improvement 
 already. 
 
3. Scenario Three: 
    * Huge number of writes 
    * Huge number of reads 
    * One row accessed from db per request 
    * Small number of records. 
 
This is the same as the first except I have space to cache everything. 
 This means I can get rid of the time() check for each cache check. 

 		
---------------------------------
Stay in the know. Pulse on the new Yahoo.com.  Check it out. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/tutor/attachments/20060817/5e7d90f8/attachment.htm