Caching in memory for Apache

[grahamd at dscpl.com.au]

Simon  Johnson wrote:
> Dear All,
>
> I have decided to take the big plunge and drop the Microsoft platform
> and use Mod_Python and Apache in it's place.
>
> I've never used Linux before this project so it's a really big learning
> curve, but so far it's going well. I've managed to create some simple
> pages using the basic of Python.
>
> One of the questions I have is: How do you cache stuff in memory? Say
> I've got a bunch of stuff I need to create from a set of database rows.
> Obviously, I do not want to consult the database each time I have a
> request for my page. Is there anyway in Mod_python for me to store
> values inside memory so I can look it up on the next GET request?

It all depends a bit on what you want to cache and whether the data is
specific to one users interaction with a page, or whether the data is
able to be used across requests from multiple users. This will dictate
whether you can use a global data cached within modules used to
generate
the page, whether you can use sessions, shared memory or whether you
have no choice but to always go back to the database.

As others have pointed out, don't get trapped in doing premature
optimisation. More often than not what you may think will be a bottle
neck will not and any problems if they even exist will be elsewhere.

In respect of mod_python, it is also vitally important to understand
the
process/interpreter module under which Python runs within the context
of Apache. A starting point for getting information about this is:

  http://www.dscpl.com.au/articles/modpython-004.html

If you do decide that sessions may be useful to what you are doing, as
pointed out by others, there can be limitations as to what data can be
stored. For limitations in respect of sessions in mod_python have a
look
at:

  http://www.dscpl.com.au/articles/modpython-005.html

Finally, make sure you use the latest version of mod_python available.
Avoid using mod_python 2.7.X or 3.1.X as there are lots of known issues
and problems. Not all problems are solved in 3.2.X version, but good
progress has been made on getting rid of many issues. The main area
which can still cause problems, especially when trying to cache data in
the Apache process, is that of module loading and reloading. For a run
down on issues in that area see:

  http://www.dscpl.com.au/articles/modpython-003.html

A lot of work has already been done on solving these issues for 3.3.X
version of mod_python.

BTW, the best place to ask questions about mod_python is the mod_python
mailing list. Thus, get yourself subscribed to it. Details of the
mailing list
are on the mod_python web site.

Graham