Re: [Python-Dev] Fixing the GIL (with a BFS scheduler)
From: "Martin v. L?wis"
To: Dj Gilcrease Cc: python-dev@python.org Subject: Re: [Python-Dev] Fixing the GIL (with a BFS scheduler) Message-ID: <4BF385E3.9030903@v.loewis.de> Content-Type: text/plain; charset=ISO-8859-1 I think the new GIL should be given a year or so in the wild before you start trying to optimize theoretical issues you may run into. If in a year people come back and have some examples of where a proper scheduler would help improve speed on multi-core systems even more, then we can address the issue at that time.
Exactly my feelings.
Although I don't agree that the problem of I/O convoying is merely some "theoretical issue", I would agree with a go-slow approach---after all, the new GIL hasn't even appeared in any actual release yet. It might be a good idea to prominently document the fact that the new GIL has some known performance issues (e.g., possible I/O convoying), but that feedback concerning the performance of real-world applications is desired. Cheers, Dave
Does anybody think that by having problems with the new GIL that it might
further weaken the adoption rate for 3k? -peter
On 5/19/10 7:00 AM, "David Beazley"
From: "Martin v. L?wis"
To: Dj Gilcrease Cc: python-dev@python.org Subject: Re: [Python-Dev] Fixing the GIL (with a BFS scheduler) Message-ID: <4BF385E3.9030903@v.loewis.de> Content-Type: text/plain; charset=ISO-8859-1 I think the new GIL should be given a year or so in the wild before you start trying to optimize theoretical issues you may run into. If in a year people come back and have some examples of where a proper scheduler would help improve speed on multi-core systems even more, then we can address the issue at that time.
Exactly my feelings.
Although I don't agree that the problem of I/O convoying is merely some "theoretical issue", I would agree with a go-slow approach---after all, the new GIL hasn't even appeared in any actual release yet. It might be a good idea to prominently document the fact that the new GIL has some known performance issues (e.g., possible I/O convoying), but that feedback concerning the performance of real-world applications is desired.
Cheers, Dave
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/peter.a.portante%40gmail.c...
Peter Portante wrote:
Does anybody think that by having problems with the new GIL that it might further weaken the adoption rate for 3k? -peter
No, to the contrary. By having the new GIL being superior to the old implementation, the adoption rate for 3k will increase. Regards, Martin
Yes, but the million dollar question is whether or not it really is superior with the I/O convoying problem in the current implementation (an effect that is substantially worse with the new GIL than with the old one by the way). Personally, I think the convoying issue is something that will have to be fixed eventually. Although, I also agree that it merits more study--especially with real-world applications. Cheers, Dave On May 19, 2010, at 3:34 PM, Martin v. Löwis wrote:
Peter Portante wrote:
Does anybody think that by having problems with the new GIL that it might further weaken the adoption rate for 3k? -peter
No, to the contrary. By having the new GIL being superior to the old implementation, the adoption rate for 3k will increase.
Regards, Martin
Martin v. Löwis
Peter Portante wrote:
Does anybody think that by having problems with the new GIL that it might further weaken the adoption rate for 3k? -peter
No, to the contrary. By having the new GIL being superior to the old implementation, the adoption rate for 3k will increase.
Well, if that's a strategy, there's still time to make 2+2 equal 5 in the "old" implementation. Bill
On Wed, May 19, 2010 at 4:17 PM, Peter Portante
Does anybody think that by having problems with the new GIL that it might further weaken the adoption rate for 3k? -peter
Nope, because the remaining issues with the new GIL affect the old GIL as well, and have yet to be proven to affect real world apps. My server is the only real world example I have seen and the modest 5% slow down it experiences on the new GIL & multi-core vs new GIL & single core can potentially be explained by other issues. The old GIL has a 25 to 30% slowdown on multi vs single core so over all the new GIL gives my server a 20% speed boost. Incremental upgrades to core functionality like the GIL is the better way to go then to try an optimize every facet of it before hand, because some of those optimizations may have unintended side effects in real world apps and if you attempt all the optimizations at once it makes it harder to figure out which is the cause of the unintended side effects.
participants (5)
-
"Martin v. Löwis"
-
Bill Janssen
-
David Beazley
-
Dj Gilcrease
-
Peter Portante