[issue21826] Performance issue (+fix) AIX ctypes.util with no /sbin/ldconfig present

[tw.bert]

New submission from tw.bert:

Preample: This is my first post to the python issue tracker, I included a fix.

This issue is probably related to http://bugs.python.org/issue11063 .

The problem:

After upgrading a package on AIX 7.1 x64 that started using the uuid module, we experienced serious performance issues.

The culprit (found after a day of debugging) is here:

File: ctypes/util.py
Note: The file /sbin/ldconfig does *not* exist, so no useful information is collected here.
The statement: 

`f = os.popen('/sbin/ldconfig -p 2>/dev/null')`

To be more specific about the performace at popen(), the performance degradation happens in it's close() method. It takes 300 ms, which is unacceptable. In a larger scope, statements that took 200ms now take 1400ms (because the above is called several times.

If I simply check for os.path.exists before the popen, the performance is fine again. See the included simple patch. It's a simple unix diff, we don't have git on that machine. Git can handle those diffs easily to my knowledge.

More information:

Small scope, culprit identified:

import os, time, traceback
print os.__file__
print time.clock(),'pre'
f = None
try:
  #if os.path.exists('/sbin/ldconfig'):
  f = os.popen('/sbin/ldconfig -p 2>/dev/null')
except:
  print traceback.format_exc()
finally:
  print time.clock(),'post close'
  if f: f.close()
  print time.clock(),'post finally'

This takes 300ms (post finally) without the check for exists.

Large scope, before patch:
time python -c "import hiredis;import redis;print 'redis-py version: %s , hiredis-py version: %s' %(redis.VERSION,hiredis.__ver
sion__,)"
redis-py version: (2, 10, 1) , hiredis-py version: 0.1.3

real    0m1.409s
user    0m0.416s
sys     0m0.129s

Large scope, after patch:
time python -c "import hiredis;import redis;print 'redis-py version: %s , hiredis-py version: %s' %(redis.VERSION,hiredis.__ver
sion__,)"
redis-py version: (2, 10, 1) , hiredis-py version: 0.1.3

real    0m0.192s
user    0m0.056s
sys     0m0.050s

----------
components: ctypes
files: patch_ctypes_util_py.diff
keywords: patch
messages: 221266
nosy: tw.bert
priority: normal
severity: normal
status: open
title: Performance issue (+fix) AIX ctypes.util with no /sbin/ldconfig present
versions: Python 2.7
Added file: http://bugs.python.org/file35726/patch_ctypes_util_py.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21826>
_______________________________________