[Mailman-Users] Problem with archrunner using large %'s of cpu (read faq & archives)

[Scott Lambert]

On Sat, Nov 01, 2003 at 12:59:24AM +0100, Brad Knowles wrote:
> At 6:21 PM -0500 2003/10/31, Scott Lambert wrote:
> > I haven't looked at the code yet, and probably won't (ENOTIME), but
> > it almost sounds to me like it's not pruning it's list of handled
> > messages and has to walk all of them each time.  I would have
> > expected queue handling to get faster as the queue got smaller due
> > to fewer files in the directory that it needs to search through.
> > Maybe it's just a function of the python datastructure being used.
>
>       If it's using files as the queue mechanism, then deleting a file
> simply marks the entry in the directory as "available", and it still
> takes just at long to scan the directory afterwards as it did before.

If we were talking about more than 10,000 files, I might buy it.  But we
are talking about 1300 files.  Also the processing goes something like
O(n), in reverse, slower as it processes the files in the directory.  I
might buy it staying slow if it started slow but it doesn't.
 
-- 
Scott Lambert                    KC5MLE                       Unix SysAdmin
lambert at lambertfam.org