[issue32408] Performance regression in urllib.proxy_bypass_environment

[Xiang Zhang]

New submission from Xiang Zhang <angwerzx at 126.com>:

Recently we update our environment from Python2.7.5 to Python2.7.13. Then one process's CPU usage grow from 15% to 70%. The cause is urllib.proxy_bypass_environment, the commit I wrote in #26864. Our environments get a no_proxy environment variable which contains 4000+ items. See the performance difference:

cascading-controller:~ # time python2 -c 'import urllib; urllib.proxy_bypass_environment("1.1.1.1")'

real	0m1.134s
user	0m1.126s
sys	0m0.007s
cascading-controller:~ # time python2 -c 'import urllib; urllib.proxy_bypass_environment("1.1.1.1")'

real	0m0.037s
user	0m0.024s
sys	0m0.013s

Temporarily I increased regex cache size to 6000 and the CPU usage and time return to a reasonable range.

----------
components: Library (Lib)
messages: 308924
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: Performance regression in urllib.proxy_bypass_environment
type: performance
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32408>
_______________________________________