Safe to call Py_Initialize() frequently?

[Graham Dumpleton]

On Mar 23, 10:00 pm, Mark Hammond <skippy.hamm... at gmail.com> wrote:
> On 23/03/2009 12:14 PM, Graham Dumpleton wrote:
>
> > On Mar 21, 10:27 am, Mark Hammond<skippy.hamm... at gmail.com>  wrote:
> >> Calling
> >> Py_Initialize and Py_Finalize multiple times does leak (Python 3 has
> >> mechanisms so this need to always be true in the future, but it is true
> >> now for non-trivial apps.
>
> > Mark, can you please clarify this statement you are making. The
> > grammar used makes it a bit unclear.
>
> Yes, sorry - s/this need to/this need not/
>
> > Are you saying, that effectively by design, Python 3.0 will always
> > leak memory upon Py_Finalize() being called, or that it shouldn't leak
> > memory and that problems with older versions of Python have been fixed
> > up?
>
> The latter - kindof - py3k provides an enhanced API that *allows*
> extensions to be 'safe' in this regard, but it doesn't enforce it.
> Modules 'trivially' ported from py2k will not magically get this ability
> - they must explicitly take advantage of it.  pywin32 is yet to do so
> (ie, it is a 'trivial' port...)
>
> I hope this clarifies...

Yes, but ...

There still may be problems. The issues is old, but suspect that
comments in the issue:

  http://bugs.python.org/issue1856

maybe still hold true.

That is, that there are some things that Python doesn't free up which
are related to Python simplified GIL state API. Normally this wouldn't
matter as another call to Py_Initialize() would see existing data and
reuse it. So, doesn't strictly leak memory in that sense.

In mod_wsgi however, Apache will completely unload the mod_wsgi module
on a restart. This would also mean that the Python library is also
unloaded from memory. When it reloads both, the global static
variables where information was left behind have been lost and nulled
out. Thus Python when initialised again, will recreate the data it
needs.

So, for case where Python library unloaded, looks like may well suffer
a memory leak regardless.

As to third party C extension modules, they aren't really an issue,
because all that is done in Apache parent process is Py_Initialize()
and Py_Finalize() and nothing else really. Just done to get
interpreter setup before forking child processes.

There is more detail on this analysis in that thread on mod_wsgi list
at:

Graham