[Python-Dev] (no subject)

[Guido van Rossum]

On 11/24/05, Duncan Grisby <duncan-pythondev at grisby.org> wrote:
> Hi,
>
> I posted this to comp.lang.python, but got no response, so I thought I
> would consult the wise people here...
>
> I have encountered a problem with the re module. I have a
> multi-threaded program that does lots of regular expression searching,
> with some relatively complex regular expressions. Occasionally, events
> can conspire to mean that the re search takes minutes. That's bad
> enough in and of itself, but the real problem is that the re engine
> does not release the interpreter lock while it is running. All the
> other threads are therefore blocked for the entire time it takes to do
> the regular expression search.

Rather than trying to fight the GIL, I suggest that you let a regex
expert look at your regex(es) and the input that causes the long
running times. As Fredrik suggested, certain patterns are just
inefficient but can be rewritten more efficiently. There are plenty of
regex experts on c.l.py.

Unless you have a multi-CPU box, the performance of your app isn't
going to improve by releasing the GIL -- it only affects the
responsiveness of other threads.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)