[Catalog-sig] start on static generation, and caching - apache config.

[René Dudfield]

Hi,

I tried using memcached for caching the database queries - for logged
in users.  It did speed it up a little, but not that much.  It turns
out that the templates take most of the time - at least on my machine.
 I guess pagetemplates are not that quick?

Here's the modified files if you want to try it out yourself:
http://rene.f0o.com/~rene/stuff/store.py
http://rene.f0o.com/~rene/stuff/webui.py

I just tried out on the queries that /pypi /pypi/ use.

There's some timing in the webui that gets written to a file in /tmp/asdfsdaf

For concurrent access then memcached will make more of a difference though.

Memcache could help even for logged in people, but I think replacing
the template language with something faster will have the most effect.


Cheers,



On 7/7/07, René Dudfield <renesd at gmail.com> wrote:
> Yeah, that could be triggered then.
>
> For the case of multiple changes at a similar time, we could add some
> checks to make sure the updater process is only running once.
> Otherwise for the case when there are a few changes happening at a
> time, the machine would get unnecessarily overloaded.
>
>
> On 7/7/07, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> > > Now I just need to finish off the static file generation code.  It
> > > needs a tool which can run every minute or so, which will look for any
> > > changes.
> >
> > Would it be possible to trigger that explicitly by a write operation?
> > I'm doubtful about cron jobs for that kind of stuff - they run both
> > too often and too infrequent. It's too often because most of the time,
> > nothing changes, and too infrequent, because the user making the change
> > won't see it, and wonders where it got lost (they will see the change
> > as long they are logged in, then they log out, and the release is not
> > there).
> >
> > IIUC, every addition to the journals should trigger a change, and then
> > the updating of the download counters. There are also changes to the
> > templates, but it would be ok if one would have to trigger regeneration
> > manually in this case.
> >
> > Regards,
> > Martin
> >
>