[Python-Dev] Sandboxed Threads in Python

[Josiah Carlson]

Bruce Eckel <BruceEckel-Python3234 at mailblocks.com> wrote:
> 
> > I can (but won't) point out examples for days of bad decisions made for
> > the sake of speed, or policy that has been ignored for the sake of speed
> > (some of these overlap and some don't).
> 
> As long as you've entered premature-optimization land, how about
> decisions made because it's *assumed* that (A) We must have speed here
> and (B) This will make it happen.

A. From what I understand about sandboxing threads, the point was to
remove the necessity for the GIL, so that every thread can go out on its
own and run on its own processor.

B. Shared memory vs. queues vs. pipes vs. ...  Concurrency without
communication is almost totally worthless.  Historically, shared memory
has tended to be one of the fastest (if not the fastest) communication
methods available.  Whether or not mutable shared memory would be faster
or slower than queues is unknown, but I'm going to stick with my
experience until I am proved wrong by this mythical free threaded system
with immutables.


> My hope would be that we could find a solution that would by default
> keep you out of trouble when writing concurrent programs, but provide
> a back door if you wanted to do something special. If you choose to go
> in the back door, you have to do it consciously and take
> responsibility for the outcome.
> 
> With Java, in contrast, as soon as you step into the world of
> concurrency (even if you step in by accident, which is not uncommon),
> lots of rules change. What was an ordinary method call before is now
> something risky that can cause great damage. Should I make this
> variable volatile? Is an operation atomic? You have to learn a lot of
> things all over again.
>
> I don't want that for Python. I'd like the move into concurrency to be
> a gentle slope, not a sudden reality-shift. If a novice decides they
> want to try game programming with concurrency, I want there to be
> training wheels on by default, so that their first experience will be
> a successful one, and they can then start learning more features and
> ideas incrementally, without trying a feature and suddenly having the
> whole thing get weird and crash down on their heads and cause them to
> run screaming away ...

I don't want to get into an argument here.  While I agree that
concurrent programming should be easier, my experience with MPI (and
other similar systems) and writing parallel algorithms leads me to
believe that even if you have a simple method for communication, even if
you can guarantee that thread/process A won't clobber thread/process B,
actually writing software which executes in some way which made the
effort of making the software concurrent worthwhile, is less than easy.
I'd love to be proved wrong (I'm hoping to do it myself with tuple
spaces).

I do, however, doubt that free threading approaches will be the future
for concurrent programming in CPython.

 - Josiah