[XML-SIG] Advice on how to deal with locking problems using python-wrapped C libs in MT frameworks like Zope/Twisted?

[Craeg K Strong]

Thanks for responding to my query!

Martin v. L=F6wis wrote:

>Craeg K Strong <cstrong@arielpartners.com> writes:
> =20
>
>>I am noticing lockup/timeout behavior whenever the integration
>>framework calls back out to Zope.
>>
>>It is crucial to determine whether they are lockup (won't terminate on
>>its own) or timeout (will terminate after some time) problems.
>>   =20
>>
Pyana 0.8 locks up and never returns.=20
GNOME libxslt 1.0.27 times out after a minute or two.

>>My guess is that the Python wrappers might be using the Global
>>Interpreter Lock in such a way that Zope gets locked out trying to
>>access the file resource?
>>   =20
>>
>If it is a timeout, this likely is not the cause - you would get a
>deadlock with that, and a deadlock would stay indefinitely.
>
Understood.

>>My question: has anyone else had to deal with such issues? =20
>>   =20
>>
>Most certainly. Usually, a slowdown is caused by somebody trying to
>resolve URLs, contacting remote systems in the process.
>
Yes.  But these problems are occuring in my unit test suite.  All of my=20
unit tests refer to local URLs
either absolute "localhost:8080/foo/bar...."  or relative URLs=20
"../foo/bar"      I get the same results
whether I am using real Zope or unit tests via ZopeTestCase.

>>Do you think something like using asyncore or just creating a worker
>>thread for the Python wrapper might make a difference?
>>

>>Uncertain. You should try to understand the problem first.
>>
Yes.  Based on the above information, can you suggest a direction of=20
inquiry?
I thought about using the python threading primitives so I could invoke=20
the XSLT transformation
in another thread and wait for it to complete or timeout, grabbing the=20
(string) result from a known
place once it is complete.    Do you think this technique might be=20
helpful in either case?

Thanks very much,

--Craeg