[Python-ideas] Fwd: Concurrent safety?

[Mike Meyer]

On Wed, 02 Nov 2011 10:45:42 +0900
"Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> Mike Meyer writes:
>  > On Tue, Nov 1, 2011 at 11:05 AM, Stephen J. Turnbull <stephen at xemacs.org>wrote:
>  > > I didn't say it did, it "merely" imposes substantial inconvenience in
>  > > hope that:
>  > >  > It just forces them to *think* about what they're doing beforehand.
>  > > which I believe to be un-Pythonic.
>  > Really? Thinking is unpythonic?
> No, "forcing" is.  Consenting adults and all that.

But you yourself admit that this isn't forcing you to think:

>  > If all you want to do is get the old semantics back in a single-threaded
>  > application, you could do something like turning:
>  > 
>  > if __name__ == '__main__':
>  >      main()
>  > 
>  > into:
>  > 
>  > if __name__ == '__main__':
>  >      locking:
>  >            main()
>  > 
>  > Actually, that achieves my goal - you hopefully thought about this long
>  > enough to realize that this was safe before doing it.
> 
> Anybody who does that is simply shutting off the warning/errors, and
> clearly is not thinking about their app at all.

So you admit this doesn't force you to think. It just makes you add a
statement to shut up the warnings. Pretty much the same thing as using
a bare except clause.

Me, I'd think about it long enough to convince myself that the app
really was single-threaded.

> But this is revealing: you say *your* goal is making *me* think.

Only if I may wind up maintaining the code you wrote. But that's a
driving factor in a *lot* of the design decisions when it comes to
extending python.

> That's what I consider un-Pythonic.

I feel just the opposite. Python doesn't allow errors to silently
pass, or guess what the programmer wanted to do, or make inferences
about things - it raises exceptions. That forces the programmer to
think about the exception and handle it properly. Or they can not
think about it, and just use a bare except clause. I think that's very
pythonic. In fact, getting tired of chasing down such bugs in Perl
code was why I switched from Perl to Python, and then cut my rates in
order to convince my clients to let me write in what was then a
strange new language.

This proposal builds on that base: it catches errors of a type that
are currently ignored and raises an exception. It also adds a new
statement for *dealing* with those errors, because handling them with
exceptions won't really work. There's even an analog for the bare
except if you want to use it. And it comes about for much the same
reason: I'm getting tired of chasing down bugs in concurrent
code. There are languages that offer that. Some even run in
environments I like, and are fun to write when they're applicable. But
I find myself wishing for Python's features when I write in them.

> A Pythonic approach would allow me to worry about it when *I* think it
> necessary.  Maybe we don't have that choice, maybe concurrency is too
> hard to solve without some annoying constraints.  But that's not at
> all clear to me, and I'd rather make gradual progress toward safety in
> a language that's fun and profitable to use, rather than have safety
> in a language that is a pain in the neck to use.

Based on my experience, your second sentence is true. If that were all
it were, the Queue module would be most of a solution, and there are
STM modules available if that's not good enough. But they only solve
half the problem - they make it easier to get things right once you
decide the data is shared. People are as likely to miss that data is
shared as they are to screw up the locking. In other words, if we do
it your way, it'll deal with less than half of whats bugging me.

It may be that Python's data structures will make this unworkable. It
may be that a workable solution will suck the performance out of
non-concurrent applications. It may be that anything that fixes both
of those will be unpalatable for other reasons.  There's no way to
find out except by trying. And I'd rather try that than start trying
to convince people to let me write in some strange new language again.

   <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org