persistent database results for web page

[Thomas A. Bryan]

James Vincent wrote:
 
>         As I learn to hook Python into my database I am faced with a
> simple yet vexing problem. I would like to query a database and give
> the results to a user, however, the query will run for a very long
> time. When it finishes it will return many more rows of data than
> will fit on a page (several thousand).  

What database are you using?  Can't you just use a cursor?

> Is there a way to run a query once and store the results 
> either with the client or on the server in a very simple 
> way that would then allow easy perusal of the result set?

I suppose the Python interface could use shelve or pickle or anydbm
to store the results.  Then it could read from that persistent 
storage before requerying the database.  Of course, then you 
would have to come up with a way for the Python code to determine 
whether the query is in the persistent store or whether it should 
query the database and create a new persistent store.  There 
also doesn't seem to be any good way to remove the persistent 
stores except for removing every one that is older than some
preset amount of time.

Use the capabilities that have been pushed into the DBMSs.  If 
yours permits you to use a cursor, then use it.  It should make
a much cleaner and probably faster solution.