[Python-Dev] Making python C-API thread safe (try 2)

[Tim Peters]

[Jeff Epler]
> I hear that on Windows, processes are very inefficient compared to
> threads.  However, I have no idea what this actually means.

Creating a process is a very heavy operation on Windows compared to creating
a thread (which is cheap), and especially under the older Windows flavors.
The various Windows automation APIs (like COM under 20 names) are also very
happy playing with threads.

> Does Harri? Or is he thinking of something else?  Is he thinking of
> something like the "higher context switching cost in terms of TLB
> misses"?  God help anybody who lets thoughts about TLB misses guide
> the way he writes Python code! (excepting maybe Tim Peters

Indeed, I think of nothing else!

> and anybody writing numpy/numarray)

Ya, but they're crazy <wink>.

> I suspect that on Windows, stuff like pipe/dup2/fork has been turned
> into a hopelessly complicated mess, rather than something I could
> "roll my own" Python object-passing system out of inside an hour
> (using pickle or the like).

fork() doesn't exist on Windows (unless you use Cygwin, and then you can
measure a fork's progress with an egg timer).

> Another cool thing about the pipe approach is that you can probably
> distribute the processing over a network with only a little more
> work... wowee, that's cool.

Pipes and sockets work fine on Windows.  For some really cool distributed
shenanigans, build on the Python Spread wrapper (which also works fine on
Windows):

    http://www.python.org/other/spread/

back-to-tlb-worries-ly y'rs  - tim