[python-win32] Memory Consumption when running Python COM Script

[Johan Lindvall]

Hi,

We are having performance problems when running python COM Scripts. The app
starts it own internal scripting engine and if we use VBScript, everything
is fine. If we use PythonScript, the app consumes large amounts of memory
(up to 3 GB) and the crashes because it runs out of memory.

This is the (simplified) PythonScript:
=====================================
counter = 0
for AnObject in OurApp.Array():
    counter = counter + AnObject.Value()
=====================================

The corresponding VBScript:
=====================================
for each AnObject in OurApp.Array()
counter = counter + AnObject.Value()
next
=====================================

This is the (simplified) C++ COM Code:
=====================================
VARIANT COurApp::Array()
{
  HRESULT hr;

  SAFEARRAY * pSa;
  pSa = SafeArrayCreateVector(VT_VARIANT, 0, 100000);

  LPVARIANT psadata = NULL;
  hr = SafeArrayAccessData(pSa, (void**)&psadata);
  if (FAILED(hr)) {
    SafeArrayDestroy(pSa);
  }
  LPDISPATCH t = GetIDispatch(FALSE);
  for (unsigned i = 0; i < 100000; ++i) {
    psadata[i].pdispVal = t;
    psadata[i].vt = VT_DISPATCH;
    t->AddRef();
  }

  hr = SafeArrayUnaccessData(pSa);
  if (FAILED(hr)) {
    SafeArrayDestroy(pSa);
  }

  VARIANT rgvar;
  VariantInit(&rgvar);
  V_VT(&rgvar) = VT_ARRAY|VT_VARIANT;
  V_ARRAY(&rgvar) = pSa;
  return rgvar;
}

float COurObject::Value()
{
  return rand();
}
==========================================

The returned Array can in some cases be very large. For this example,
VBScript uses 1,5 Megs and Python uses about 500 Megs of RAM.

I have set breakpoint on malloc and free in the runtime library and have
observed that Python calls these functions frequently, which I suspect is
part of the problem.

What can we do to optimise our application / script? Any hints / pointers
are much appreciated.

Thanks.
-- 
/Johan.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-win32/attachments/20060825/536b08ba/attachment.html