[Python-ideas] Type Hinting - Performance booster ?

[Ron Adam]

On 12/27/2014 04:00 AM, Antoine Pitrou wrote:
> On Fri, 26 Dec 2014 19:40:25 -0600 Ron
> Adam<ron3200 at gmail.com>  wrote:

>>> It's a bit long but interesting.  At one point, (18:30 to 19:45
>>> minute marks), he mentions a tree design where you spawn threads for
>>> each child, and they spawn threads for each of their children.

> Yeah, so what? Spawning threads isn't hard, exploiting parallelism is.

I think you are using a stricter definition of parallelism here.

If many programs spawn many threads, and those threads are distributed 
across many cores, then you have a computer that gains from having more cores.

Yes, There are a lot of technical difficulties with doing that efficiently.

The point I was trying to make at the start of this is type-hints can be 
used to identify parts of programs that don't depend on other parts, and 
that some of those parts can be run in parallel.

Basically it involves converting a sequential program into a dependency 
tree, and allowing non-inter dependent child nodes to run in parallel. The 
type-hints may help with this.

As for the technical details of building a chip with 100 cores and how to 
manage the cache's for them.  I'm happy to let the chip engineers work on that.

I do believe we will see 100 core chips in a few years regardless of 
weather or not it makes complete sense to do so.  Keep in mind that other 
languages may be able to take advantage of them much easier than python 
can, including newly emerging languages built specifically to use 
parallelism more effectively.


>>> Yes, I think it got off to a slower start than many expected.

> It didn't "get off to a slower start". It's actually perfectly in line
> with predictions of people like me or Linus Torvalds:-)

     "... than many expected."

It wasn't surprising to me either.


>>> A web browser is about as mainstream as you can get.

> Then let's talk about it again when a web browser manages to fully
> exploit 100 threads without thrashing CPU caches.

How about 80% utilisation with only 20% thrashing?  ;-)

Cheers,
    Ron