moving Connection/PipeConnection between processes
randall at tnr.cc
Sat Jun 13 20:02:24 CEST 2009
Mike Kazantsev wrote:
> On Sat, 13 Jun 2009 02:23:37 -0500
> Randall Smith <randall at tnr.cc> wrote:
>> I've got a situation in which I'd like to hand one end of a pipe to
>> another process. First, in case you ask why, a spawner process is
>> created early before many modules are imported. That spawner process is
>> responsible for creating new processes and giving a proxy to the parent
>> Looking at the pickle docs, I wonder if this could be resolved by adding
>> a __getnewargs__ method to _multiprocessing.Connection. But even if
>> that would work I couldn't do it now since it's an extension module.
>> I've thought about trying to recreate the Connection. Looks like it
>> should be possible with Connection.fileno(). The Unix portion looks
>> easy, but the win32 portion does not.
>> So if it's possible, what's the best way to pass a Connection to another
> Pickle has nothing to do with the problem since it lay much deeper: in
> the OS.
> From kernel point of view, every process has it's own "descriptor
> table" and the integer id of the descriptor is all the process gets, so
> when you say "os.pipe()" kernel actually gives you a number which is
> completely meaningless for any other process - it either doesn't exists
> in it's descriptor table or points to something else.
> So, what you actually need is to tell the kernel to duplicate
> underlying object in another process' table (with it's own numbering),
> which is usually done via special flag for sendmsg(2) in C, so you
> should probably look out for py implementation of this call, which I
> haven't stumbled upon, but, admittely, never looked for.
As I was referring to earlier, Unix is easy.
fd = mycon.fileno()
new_con = _multiprocessing.Connection(os.dup(fd))
The implementation of the pipe creation is on line 167 of connection.py.
I don't know where to start.
More information about the Python-list