[pypy-dev] Syntax for the 'transaction' module

[Armin Rigo]

Hi Laura,

On Tue, May 1, 2012 at 12:03, Laura Creighton <lac at openend.se> wrote:
> So how about 'schedule'?

Might work :-)  How about, if we decide to go for a class, the name of
the class?  Is it 'schedule.Runner()', 'schedule.Scheduler()', ...?  A
concrete instance of this class is no longer stateless, so
'schedule.Schedule()' is probably not so correct.

>>* Note that the added transactions are only run on __exit__(), not
>>when you call add() or map().  This might be another argument against
>>map().
>
> Or an argument to build a different sort of 'add me lots of things'
> function??  I am not sure this makes sense.  I am going kayaking
> overnight now, I will think of this more while paddling.

Yes, some kind of 'add lots of things' function.  But I'm rather
unsure it's worth creating a special function to do that.  I'm happy
with "for x in list: add(x)".

> transaction == 'set of things to schedule, atomically'? or something
> else?  Right now it is not clear to me why 'one' is important, probably
> because I do not understand something.

One transaction == one unit of code that runs atomically.  That's why
I complained that by doing one add() you would end up with sometimes
more than one transaction, if the function that you added breaks it in
several ones by calling select() or other external functions.  But
maybe it's still correct if we go for the  "breakable transaction"
model.

> This seems up-side down to me.  Do you want to decorate them as
> breakable?  Or assume that they are breakable and decorate them as
> unbreakable?

No, I want it precisely the way I said it: by default, transactions
must not be breakable.  If the default was the other way around, it
would mean that by default your program is not safe against random
breakage --- pun intended: with breakable transactions your program
risks to break.  With non-breakable ones it is safe, but just
potentially inefficient if it calls "print" or "select".  So the
@breakable decorator is something you would put only on, say, the
function that does the loop on select().

Alternatively we could have two kinds of add(): one that adds regular
and the other that adds breakable transactions.  That would mean the
transaction is then breakable *anywhere*, which is less safe.  On the
other hand, a @breakable decorator would be hard to implement in the
emulator module to run on CPython...  Argh.  At least a
"breakable-anywhere" transaction is closer to the notion of "spawn a
new thread to run this in the background"...

I have to think more, but it would seem that "breakable-anywhere"
transactions would really offer a replacement for (simple usages of)
threads.  Maybe in the end it's the best solution: two kinds of
transactions, and the "unsafe" one is the "breakable-anywhere" kind
that really works like a thread, releasing the GIL around calls to
blocking C syscalls --- except of course not with a real GIL in the
real pypy-stm, but giving you this illusion.

Bah, now I'm wondering if it could be used to reimplement the "thread"
or "threading" module interface instead, at the user Python level.
Maybe it can, and you have to use "thread.start_new_thread()" to start
such breakable-anywhere transactions.  It would give Yet Another bonus
to pypy-stm: "merely" the fact that existing programs using threads
are then able to use multiple cores too...


A bientôt,

Armin.