Sigh. Sent this only to bruce by accident. Sorry about the duplicates Bruce.<br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Mike Meyer</b> <span dir="ltr"><<a href="mailto:mwm@mired.org">mwm@mired.org</a>></span><br>
Date: Mon, Oct 31, 2011 at 10:30 PM<br>Subject: Re: [Python-ideas] Concurrent safety?<br>To: Bruce Leban <<a href="mailto:bruce@leapyear.org">bruce@leapyear.org</a>><br><br><br>On Mon, 31 Oct 2011 18:07:19 -0700<br>
Bruce Leban <<a href="mailto:bruce@leapyear.org">bruce@leapyear.org</a>> wrote:<br>
> On Mon, Oct 31, 2011 at 4:19 PM, Mike Meyer <<a href="mailto:mwm@mired.org">mwm@mired.org</a>> wrote:<br>
> >> That would mean that all of my current code is broken.<br>
> > Pretty much, yes. It's like adding garbage collection and removing<br>
> > alloc*/free. It's going to break a *lot* of code. That's why I said "not in<br>
> > 3. and possibly never in cPython."<br>
> In order to make concurrent code slightly safer, you're going to break all<br>
> existing programs that don't use concurrency. That seems to me like a new<br>
> language, not Python. You've been on this list long enough to see the<br>
> attention that's paid to backward compatibility.<br>
<br>
The way adding automatic memory management just made pointer<br>
arithmetic slightly safer.<br>
<br>
But yeah, the first thing I said was "never in 3.x, possibly never in<br>
cPython." I've been on this list long enough to know that, while the<br>
community pays a lot of attention to backwards compatibility, it is<br>
willing to throw it out when there's enough benefit.<br>
<br>
As you point out, this is a hard problem. I know I haven't covered all<br>
the issues. That's why the second thing I said was that I'm hoping to<br>
get people smarter than me to look at things.<br>
<br>
The cpu manufacturers have switched to improving performance by adding<br>
more cores instead of cranking clock speeds. As time goes by, more and<br>
more programmers are going to want to leverage that in serious ways. I<br>
already do, and find that Python makes some design choices unavailable<br>
or very fragile(*). I'd like to make those choices available, and help<br>
Python get ready for the time when that desire is the norm instead of<br>
the exception.<br>
<br>
> > Furthermore, merely *reading* an object that isn't locked can cause<br>
> >> problems. This code is not thread-safe:<br>
> >> if element in dictionary: return dictionary[element]<br>
> >> so you have to decide how much safety you want and what cost we're<br>
> >> willing to pay for this.<br>
> > You're right - it's not thread safe. However, it also doesn't suffer from<br>
> > the problem I'm trying to deal with, where you mutate an object in a way<br>
> > that leaves things broken, but won't be detected at that point. If it<br>
> > breaks because someone mutates the object underneath it, it'll throw an<br>
> > exception at that point. I know you can construct cases where that isn't<br>
> > so.<br>
> I think the cases where non-thread-safe code won't throw an exception are<br>
> numerous, for example, the equally trivial:<br>
<br>
Again, I said such cases can be built. I *didn't* say they were<br>
exceptions, I proposed a change to deal with them.<br>
<br>
> If you're going to tackle thread safety, it should address more of the<br>
> problem. These bugs are in many ways worse than mutating "an object in a<br>
> way that leaves things broken, but won't be detected at that point." The<br>
> above bugs may *never* be detected. I've come across bugs like that that<br>
> were in code for many years before I found them (and I'm sure that's<br>
> happened to others on this list as well).<br>
<br>
Like me. That's part of why I want to get the interpreter to help find<br>
them.<br>
<br>
> The first thing to do is identify the problems you want to solve and make<br>
> sure that the problems are well understood. Then design some solutions.<br>
> Starting with a bad solution to a fraction of the problem isn't a good<br>
> start.<br>
<br>
I've identified the problem I want to solve: I want to make concurrent<br>
use of python objects "safe by default", so that doing unsafe things<br>
causes the programmer to have to do something explicit about making<br>
things safe. I believe this can be done at the mutation points<br>
(again, clojure shows that it can be done). I also want to preserve as<br>
much of Python's existing code as possible. It may be that Python's<br>
existing data structures mean my believe about mutation points is<br>
wrong. This may be the wrong solution. It may be that such a change is<br>
to large to be acceptable. But the only way to find out is to<br>
investigate it.<br>
<br>
This discussion has already generated significant changes in the<br>
original proposal, plus some implementation ideas.<br>
<br>
<mike<br>
<br>
*) I'll note that some of the alternatives make the choices available<br>
in Python unavailable in a similar way.<br>
<font color="#888888">--<br>
Mike Meyer <<a href="mailto:mwm@mired.org">mwm@mired.org</a>> <a href="http://www.mired.org/" target="_blank">http://www.mired.org/</a><br>
Independent Software developer/SCM consultant, email for more information.<br>
<br>
O< ascii ribbon campaign - stop html mail - <a href="http://www.asciiribbon.org" target="_blank">www.asciiribbon.org</a><br>
</font></div><br>