[Python-Dev] Slides from today's parallel/async Python talk

[Trent Nelson]

On Thu, Mar 14, 2013 at 12:59:57PM -0700, Stefan Ring wrote:
> >     Yup, in fact, if I hadn't come up with the __read[gf]sword() trick,
> >     my only other option would have been TLS (or the GetCurrentThreadId
> >     /pthread_self() approach in the presentation).  TLS is fantastic,
> >     and it's definitely an intrinsic part of the solution (the "Y" part
> >     of "if we're a parallel thread, do Y"), but it definitely more
> >     costly than a simple FS/GS register read.
> 
> I think you should be able to just take the address of a static
> __thread variable to achieve the same thing in a more portable way.

    Sure, but, uh, that's kinda' trivial in comparison to all the wildly
    unportable Windows-only functionality I'm using to achieve all of
    this at the moment :-)

    For the record, here are all the Windows calls I'm using that have
    no *direct* POSIX equivalent:

        Interlocked singly-linked lists:
            - InitializeSListHead()
            - InterlockedFlushSList()
            - QueryDepthSList()
            - InterlockedPushEntrySList()
            - InterlockedPushListSList()
            - InterlockedPopEntrySlist()

        Synchronisation and concurrency primitives:
            - Critical sections
                - InitializeCriticalSectionAndSpinCount()
                - EnterCriticalSection()
                - LeaveCriticalSection()
                - TryEnterCriticalSection()
            - Slim read/writer locks (some pthread implements have
              rwlocks)*:
                - InitializeSRWLock()
                - AcquireSRWLockShared()
                - AcquireSRWLockExclusive()
                - ReleaseSRWLockShared()
                - ReleaseSRWLockExclusive()
                - TryAcquireSRWLockExclusive()
                - TryAcquireSRWLockShared()
            - One-time initialization:
                - InitOnceBeginInitialize()
                - InitOnceComplete()
            - Generic event, signalling and wait facilities:
                - CreateEvent()
                - SetEvent()
                - WaitForSingleObject()
                - WaitForMultipleObjects()
                - SignalObjectAndWait()

        Native thread pool facilities:
            - TrySubmitThreadpoolCallback()
            - StartThreadpoolIo()
            - CloseThreadpoolIo()
            - CancelThreadpoolIo()
            - DisassociateCurrentThreadFromCallback()
            - CallbackMayRunLong()
            - CreateThreadpoolWait()
            - SetThreadpoolWait()

        Memory management:
            - HeapCreate()
            - HeapAlloc()
            - HeapDestroy()

        Structured Exception Handling (#ifdef Py_DEBUG):
            - __try/__except

        Sockets:
            - ConnectEx()
            - AcceptEx()
            - WSAEventSelect(FD_ACCEPT)
            - DisconnectEx(TF_REUSE_SOCKET)
            - Overlapped WSASend()
            - Overlapped WSARecv()


    Don't get me wrong, I grew up with UNIX and love it as much as the
    next guy, but you can't deny the usefulness of Windows' facilities
    for writing high-performance, multi-threaded IO code.  It's decades
    ahead of POSIX.  (Which is also why it bugs me when I see select()
    being used on Windows, or IOCP being used as if it were a poll-type
    "generic IO multiplexor" -- that's like having a Ferrari and speed
    limiting it to 5mph!)

    So, before any of this has a chance of working on Linux/BSD, a lot
    more scaffolding will need to be written to provide the things we
    get for free on Windows (threadpools being the biggest freebie).


        Trent.