[Python-3000] Kill GIL?

[Ivan Krstić]

Calvin Spealman wrote:
> So reload the information you want available to worker tasks and pass
> that information along to them, or provide a mechanism for them to
> request it from its preloaded housing. 

With large sets, you can't afford duplicate copies in memory, so there's
nothing to reload. I specifically mentioned providing a mechanism for
retrieving individual pieces of information from the master process, but
if you're doing lots of reads, this introduces complexity and overhead
that's best avoided. Maybe it doesn't matter; with an appropriately nice
interface for it in the distribute-and-coordinate wrapper, we might be
able to hide the complexity from the programmer and use the best
available IPC mechanism in the background to ferry the requests. Sync
domain sockets are certainly fast enough, even though you're again
unnecessarily duplicating parts of memory for each of your workers.

> Alternatively, one could say you dont usually want any preloaded data
> because there is simply too much information to preload and reusable
> worker tasks can provide their own, more effectively targetted caches.

I'm talking about real-world problems, where this most often doesn't work.

> But in the end, you have to realize this is all at a higher level than
> we would really need to even consider for the discussion at hand.

I was answering a direct question.

-- 
Ivan Krstić <krstic at solarsail.hcs.harvard.edu> | GPG: 0x147C722D