[Python-ideas] Fwd: Concurrent safety?

[Stephen J. Turnbull]

Mike Meyer writes:

 > No. You missed two things.

Actually, I didn't.  Containers contain; I considered that
sufficiently obvious that I neglected to mention it.  And I did
mention globals as "exposed to concurrency".  Both just expand the
list of objects needing protection, but don't change the basic
principle AFAICS.  I point this out because I hope to convince you to
concentrate on arguing *for* your proposal, instead of wasting time
scoring points *against* those who question it.  Please understand:
*nobody* is against better support for developing concurrent programs,
including those using threads to implement concurrency.  But nobody is
willing to pay arbitrarily high cost for arbitrarily small
improvements.

As for "wrapped C libraries", I'm having trouble imagining what you're
talking about.  Wrapped libraries won't be accessing data via Python
protocols anyway, so any accesses that your framework could intercept
will happen in the marshaling code.  If that creates threads, I don't
see why it wouldn't use threading.Thread.  Ditto for Python modules
implemented in C.

 > So this is a guess, but here's what I think is going on:
 > 
 > 1) We're using a test framework that creates a mock time module for
 > some reason. At some point, the mock object has the value None.
 > 2) The application being tested  uses a database module that uses
 > threads as part of managing a connection pool.
 > 
 > The concurrency unsafe codde in the test framework (which is clearly
 > and obviously single-threaded, right?)

Again, I don't understand.  In the von Neumann model of computation,
all *code* is single-threaded pretty much by definition.  With
preemptively scheduled threads, the process executing that code is
single-threaded if and only if it blocks until all other threads exit.

The test framework is "obviously" multiple-threaded in the scenario
you describe.

 > managed to get the None-valued mock inserted in sys.modules due to
 > a concurrency bug.

I don't see your point.  You claim your proposal is supposed to help
identify which objects are shared.  Taking that literally, this
example is useless as an argument for your framework -- you already
know which object is the problem.  Of course, I suppose you actually
meant to get more precise information in cases like this, specifically
about in what execution contexts the object is mutated.  But in this
case, you probably already know precisely where in the code the
offending mutation happens.  The bug is that execution of the
framework code resumes "unexpectedly" before the "subordinate" thread
restores the problem object's state to status quo ante.  Well, these
are *threads*, so you can't say anything about when to expect them to
resume, you have to expect resumption at any point.

*Now* what does your framework buy you?  If the answer is "nothing,"
you need to find better use cases to motivate your proposal.