Is it possible to register a function to be called at finalization
time? By finalization time I'm referring to the point at which Python
is still operational, but is shutting down, for example the time when
the modules are getting cleaned up. Functions registered with
Py_AtExit get called too late, when it's no longer allowed to invoke
any Python API function.
I currently simulate this using an object whose __del__ invokes my
function, and sticking that object in a private module. During module
cleanup Python unhooks the object from the module, firing its __del__
which calls my function. But I consider it a hack, and a fragile one
at that -- if anyone deletes the object from the module, the function
will be called too early. I'd prefer a cleaner way to deal with this
if possible.
I working on a language interoperability tool called Babel,
https://computation.llnl.gov/casc/components/. It provides
high-performance, bi-directional communication between any combination
of C, C++, F77, F90, Java, and Python. Right now, I am trying to make
sure I am handling the GIL locking and unlocking correctly, and I am
wondering if the Python community can confirm whether I am handling
things appropriately.
Babel handles the many-to-many language interoperability problem with a
hub and spoke architecture, so all inter-language calls go through C.
Hence, I can discuss how I am handling the GIL in terms of
communications between C and Python. I am going to describe what I am
doing using pseudo-code and leaving out irrelevant details.
CASE 1 (some other language calling Python via C)
========================================
foo_Factorial(struct foo_t *self, int32_t arg) {
/* C routine that calls Python */
PyGILState_STATE _gstate;
_gstate = PyGILState_Ensure();
/* convert incoming arguments from C to Python */
_result = PyObject_CallObject(_pfunc, _args);
/* process outgoing arguments or exceptions */
PyGILState_Release(_gstate);
/* return to caller in whatever language */
}
CASE 2 (Python calling some other language via C)
========================================
static PyObject *
pStub_foo_Factorial(PyObject *_self, PyObject *_args, PyObject *_kwdict) {
PyObject *result;
/* unpack incoming Python arguments into their C equivalents */
Py_BEGIN_ALLOW_THREADS
/* dispatch to method implementation */
foo_Factorial(i,k,l); /* this step is actually done through a function
pointer */
Py_END_ALLOW_THREADS
/* pack outgoing C arguments into the Python return value or process
any exceptions */
return result;
}
Discussion
========================================
Case 1 seems pretty straight forward. The current thread must own the
GIL before making any Python C API calls, and it should release it when
it's done. Case 2 also seems appropriate because function x_y_Method
could very well do blocking I/O operation. For example, Babel also
supports remote method invocation (RMI), so x_y_Method might dispatch
the method across the network to another machine.
In ceval.h where Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS are
defined, it says "WARNING: NEVER NEST CALLS TO Py_BEGIN_ALLOW_THREADS
AND Py_END_ALLOW_THREADS!!!". This makes me wonder if there is an issue
with Python calling itself through Babel. Let's assume that Python is
the main driver, and it calls a Babel wrapped Python method to calculate
factorial or something. This will result in nested calls to
PyGILState_Ensure/PyGILState_Release and
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS.
Python interactive shell (invokes foo.Factorial(3))
Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
PyGILState_Ensure (in foo_Factorial (case 1) arg=3)
Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
PyGILState_Ensure (in foo_Factorial (case 1) arg=2)
Py_BEGIN_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
PyGILState_Ensure (in foo_Factorial (case 1) arg=1)
PyGILState_Release (in foo_Factorial (case 1) arg=1)
Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
PyGILState_Release (in foo_Factorial (case 1) arg=2)
Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
PyGILState_Release (in foo_Factorial (case 1) arg=3)
Py_END_ALLOW_THREADS (in pStub_foo_Factorial (case 2))
Python interactive shell
Is this approach correct?
Regards,
Tom Epperly
LLNL/CASC
Palm Vacations, A Travel Guide For Cologne
Cologne is the four most populated cities in Germany. These days the city is
a beautiful cultured destination with great leisure facilities, pubs and
shops. However, during WWII, Cologne was heavily bombed and damaged. After
years of restoration work it has been rebuilt to portray the wonderful
historic Roman city it use to be.
Travelling to Cologne Flying is the recommended choice of transport as the
city has K?ln/Bonn Konrad Adenauer Airport located 10km from the centre.
Also, for people arriving from countries that are outside Europe, Frankfurt
Airport is the closest and provides transfers for the 200km journey. Trains
and large roads make it easy to get to the city via other methods.
Accommodation in Cologne Hotels are the most popular choice of accommodation
in Cologne. However, check for the festival dates as this can seriously
affect the availability of rooms. Hotels are great as they offer different
classes of rooms and are conveniently situated around the city.
A Brief History of Cologne Cologne has a vast history as it is the eldest
city in Germany. The city has great importance in Roman History as it is one
of four sanctified cities that formed the north corner spot of the Empire.
At the beginning of the thirteenth century Cologne was renowned for being
the largest fortified city in the world. However, it has been changed around
a lot since then as 90% of Cologne was destroyed during WWII.
City Attractions and Sights Since the heavy bombing in WWII, restoration
efforts have seen the city burgeon into the historic Roman centre that it
used to be. The Cologne Cathedral (K?lner Dom) that was originally
established in 1248 was rebuilt to its previous state. It is an important
feature of the city and one of the worlds most impressive Cathedrals.
If you are interested in museums then head for the Aldstadt city?s most
famous museums are such as the Museum Ludwig. For relaxation check out
Claudius Spa that is found in the Rheinpark area although, you will need a
car to make the journey.
Shopping in Cologne Cologne has a great range of shops that cater for
everybody?s needs. The most popular shopping areas are in the centre where
there are a number of large. Other smaller streets that head out of the
centre have more unique shops and indoor markets. The shopping areas are
sectioned off for pedestrians, attractive and well laid out.
Food and Drink Cologne has a great range of top quality restaurants that
offer all kinds of international food. There are also numerous attractive
cafes and bars as well as all takeaway options.
Hi -
I'm currently looking into a few deployment issues with our embedded
Python interpreter and I'm looking for any information about deploying
embedded Python that people may have. Specifically, I'm looking for the
following information:
1) How to define a useful subset of the stdlib that can serve as an
initial basis for the installation but later allows upgrade to the
"full" library if desirable. In other words, I'd like to deploy a small
subset of the stdlib to begin with (simply because of size constraints)
which may later be extended to a full stdlib if this is desirable. Has
someone done this before? I'd love to have a small "Python.zip"
cross-platform stdlib surrogate that just gets shipped with the product.
If not, what is the right starting point for analyzing the dependencies
inside the stdlib?
2) How to isolate the embedded interpreter from environmental effects. I
have found that on occasion, the interpreter would pick up "stray"
installations which can cause weird problems. Which environmental
settings affect the startup of an embedded Python interpreter? How does
one work around/remove those dependencies? Is there any information
available about how exactly the startup works? What is being read/loaded
in which order etc?
3) General advice about deploying embedded Python. Pointers to web
sites, general experience (good or bad) etc. are all very welcome.
Thanks,
- Andreas
