[Mailman-Developers] re: Much cpu/memory load
Dan Mick
Dan.Mick@sun.com
Fri Nov 1 20:20:11 2002
Simone Piunno wrote:
> On Thu, Oct 31, 2002 at 07:03:50PM -0800, Dan Mick wrote:
>
>
>>The problem's affecting me badly right now too. So far it seems that the
>>holdup is in the 'date.html' index file processing; everything else is
>>finished, but the "bin/qrunner -r Arch -o" process has lost its little mind
>>trying to write the date index. It's in a loop, chewing up 409600 bytes of
>>memory (brk), and then opening, reading, and closing archidxentry.html,
>>hundreds of times in a row.
>
>
> Ok, I believe this is because of my patch for i18n in archives.
> Do you feel like trying this patch?
> It should speed up things a lot....
>
> Index: HyperArch.py
> ===================================================================
> RCS file: /cvsroot/mailman/mailman/Mailman/Archiver/HyperArch.py,v
> retrieving revision 2.22
> diff -u -r2.22 HyperArch.py
> --- HyperArch.py 19 Oct 2002 20:59:27 -0000 2.22
> +++ HyperArch.py 1 Nov 2002 10:13:21 -0000
> @@ -923,10 +923,14 @@
> 'sequence': article.sequence,
> 'author': author
> }
> - print Utils.maketext(
> - 'archidxentry.html', d, raw=1,
> - lang=self.maillist.preferred_language,
> - mlist=self.maillist)
> + print """<LI><A HREF="%(filename)s">%(subject)s
> + </A><A NAME="%(sequence)i"> </A>
> + <I>%(author)s
> + </I>""" % d
> + #print Utils.maketext(
> + # 'archidxentry.html', d, raw=1,
> + # lang=self.maillist.preferred_language,
> + # mlist=self.maillist
>
That's definitely the cause of the many opens of archidxentry.html, but it
doesn't explain the 400K-per-iteration growth.
Barry, wasn't there some Python instrumentation trick for using the GC to
find unreferenced objects and complain, or invoking GC manually in a loop
to try to alleviate such problems?....was it in the gc module?...