Antoine Pitrou wrote:
Having separate PEPs also makes it easier to discuss the issues piecewise rather than a whole big chunk of additions.
I am preparing 3 new PEPs now. One for the C level, one for micro-threads which provides some basic capabilities, and one for micro-pipes which I expect will be the one that will change the most and take the longest to put to bed. I'm also concentrating on APIs rather than implementations and will include examples of using these APIs.
They seem necessary to handle exception situations. For example, when a reader thread on a pipe dies with an exception, how is the write thread notified? What mechanism knows that this pipe was being read by the errant thread so that it will never be read from again?
A pipe could be divided into two half-objects: the receiving end and the sending end, each independently managed by the standard reference counting mechanism, and referencing each other through weakrefs. That way, when e.g. the last reference to the receiving end dies, the sending end will notice that and can raise an exception when a thread tries to write to it.
I don't think that this would be very portable to other flavors of python (jython/ironpython/pypy) that don't use reference counting. It makes the thread termination dependent on the implementation of the garbage collector.
In any case, I don't think creating standard "stdin" and "stdout" pipes for each thread makes things any easier. You still have to handle the case of whatever non-stdin/stdout pipes get created.
I imagine that the python programmer would not be allowed to create micro-pipes directly. I'm thinking that there would be a num_stdout (= 0) parameter on the micro_thread constructor that creates that many micro_pipes, and then the python programmer can connect these to the stdin of other micro-threads. The reason for this is so that the underlying implementation always knows how the dots are connected so that it can provide sensible exception/abort semantics.
I'm working on another project that uses generators a _lot_ and there are problems there because 'for' loops don't call 'close' on generators to clean things up. I've also hit problems where the code works fine on CPython, but fails on jython and ironpython because I'm relying on the reference counting to immediately collect abandoned generators and run their 'finally' clauses.
Good suggestion! I was hoping that some might show up here, but ... I guess I need to go looking for them!
A Twisted developer told me that he had tried to read your PEP (the first version) but found it difficult to understand.
I hope that the new versions will be easier to follow!