[Python-ideas] An alternate approach to async IO
shibturn at gmail.com
Tue Nov 27 21:13:18 CET 2012
On 27/11/2012 7:59pm, Trent Nelson wrote:
>> >But you have to allocate the buffer*before* you initiate an overlapped
>> >read. And you may as well make that buffer a Python bytes object (which
>> >can be shrunk if it is too large). That leaves no "processing" that can
>> >usefully be done by a C level thread pool.
> I'm a little confused by that last sentence. The premise of my idea
> is being able to service AIO via simple GIL-independent threads that
> really just copy data from A to B. The simple fact that they don't
> have to acquire the GIL each time the completion port has an event
> seems like a big win, no?
> (So I'm not sure if you're saying that this wouldn't work because
> you may as well use Python bytes objects, and they can't be accessed
> willy-nilly from non-GIL threads... or if you're saying they can,
> but there's no benefit from a C-level thread copying data to/from
> buffers independent of the GIL.)
I am saying that there is no copying necessary if you use a bytes object
as your buffer. You can just use _PyBytes_Resize() afterwards to shrink
it if necessary.
More information about the Python-ideas