[Mailman-Users] Problem with archrunner using large %'s of cpu (read faq & archives)

[Jon Carnes]

On Fri, 2003-10-31 at 21:29, Scott Lambert wrote:
> On Sat, Nov 01, 2003 at 12:59:24AM +0100, Brad Knowles wrote:
> > At 6:21 PM -0500 2003/10/31, Scott Lambert wrote:
> > > I haven't looked at the code yet, and probably won't (ENOTIME), but
> > > it almost sounds to me like it's not pruning it's list of handled
> > > messages and has to walk all of them each time.  I would have
> > > expected queue handling to get faster as the queue got smaller due
> > > to fewer files in the directory that it needs to search through.
> > > Maybe it's just a function of the python datastructure being used.
> >
> >       If it's using files as the queue mechanism, then deleting a file
> > simply marks the entry in the directory as "available", and it still
> > takes just at long to scan the directory afterwards as it did before.
> 
> If we were talking about more than 10,000 files, I might buy it.  But we
> are talking about 1300 files.  Also the processing goes something like
> O(n), in reverse, slower as it processes the files in the directory.  I
> might buy it staying slow if it started slow but it doesn't.
>  
To me it sounds like a memory problem.

I wonder how fast we can fix it?