[Python-Dev] New regex module for 3.2?

[Gregory P. Smith]

On Tue, Jul 27, 2010 at 6:43 PM, R. David Murray <rdmurray at bitdance.com>wrote:

> On Tue, 27 Jul 2010 08:27:35 +0200, Stefan Behnel <stefan_ml at behnel.de>
> wrote:
> > Gregory P. Smith, 27.07.2010 07:40:
> > > A max cache size of 100 was too small.  I just increased it to 500 in
> the
> > > py3k branch along with implementing a random replacement cache overflow
> > > policy.  It now randomly drops 20% of the compiled regular expression
> cache
> > > instead of simply dropping the entire cache on overflow.
> > >
> > > With the regex_v8 benchmark, the better cache replacement policy sped
> it up
> > > ~7% while raising the cache size on top of that (likely meaning the
> cache
> > > was never overflowing) sped it up ~25%.
> > >
> > > Random replacement without dropping everything at least means apps
> thrashing
> > > the cache degrade much more gracefully.
> >
> > The same algorithm should be helpful in ElementTree's ElementPath module.
>
> We recently added the old re cache-clearing strategy to
> fnmatch, because previously its cache would grow indefinitely.
> It sounds like this should be applied there as well.
>
> That's three...time to figure out how to share the code?
>

No doubt.

Its already a standalone _shrink_cache function with unit tests that doesn't
care the dictionaries it is shrinking are composed of.  Easy enough to move
somewhere more useful.  Any proposed stdlib locations?  I'll be offline on
vacation soon so I may not get to it for a couple weeks but feel free to
move it without me if anyone is interested.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100727/2a5da3bb/attachment.html>