[Python-ideas] Fwd: Concurrent safety?

[Mike Meyer]

On Wed, Nov 2, 2011 at 1:49 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Mike Meyer writes:
> Well, if you want help chasing down bugs in concurrent code, I would
> think that you would want to focus on concurrent code.  First, AFAICS
> ordinary function calls don't expose additional objects to concurrency
> (they may access exposed objects, of course, but they were passed in
> from above by a task, or are globals).  So basically every object
> exposed to concurrency is in either args or kwargs in a call to
> threading.Thread (or thread.start_new_thread), no?

No. You missed two things. First, all the objects that can be accessed
- however indirectly - through those objects are exposed to
concurrency. Also, any globals and anything that can be accessed
through them are exposed to concurrency.  No, make that three things:
a wrapped C library that has call backs to Python code and uses
threads internally can expose anything it's passed (and anything
accessible from those objects) to concurrency, without ever using the
Python threading code.

I mentioned see a bug yesterday. My clients response means I can't in
good faith try and fix it (it's in a testing framework, so doesn't
affect the product, so they don't care). So this is a guess, but
here's what I think is going on:

1) We're using a test framework that creates a mock time module for
some reason. At some point, the mock object has the value None.
2) The application being tested  uses a database module that uses
threads as part of managing a connection pool.

The concurrency unsafe codde in the test framework (which is clearly
and obviously single-threaded, right?) managed to get the None-valued
mock inserted in sys.modules due to a concurrency bug. So when I then
use the time module in testing, I get an exception trying to access
it's attributes.

This does show a bigger problem. Look for my next mail...

   <mike