[Python-ideas] re.compile_lazy - on first use compiled regexes

[Antoine Pitrou]

On Fri, 22 Mar 2013 15:42:49 -0700
"Gregory P. Smith" <greg at krypto.org> wrote:
> On Fri, Mar 22, 2013 at 3:31 PM, Ronny Pfannschmidt <
> Ronny.Pfannschmidt at gmx.de> wrote:
> 
> > Hi,
> >
> > while reviewing urllib.parse i noticed a pretty ugly pattern
> >
> > many functions had an attached global and in their own code they would
> > compile an regex on first use and assign it to that global
> >
> > its clear that compiling a regex is expensive, so having them be compiled
> > later at first use would be of some benefit
> >
> 
> It isn't expensive to do, it is expensive to do repeatedly for no reason.
>  Thus the use of compiled regexes.  Code like this would be better off
> refactored to reference a precompiled global rather than conditionally
> check if it needs compiling every time it is called.

Precompiled regexes were a major contributor in Python startup time:
http://bugs.python.org/issue13150
http://hg.python.org/cpython/rev/df950158dc33/

In the stdlib we should be rather careful about this. Third-party
libraries can ignore such concerns if they aren't meant to use in
interactive applications.

Regards

Antoine.