python simply not scaleable enough for google?

John Nagle nagle at animats.com
Sun Nov 15 23:09:44 EST 2009


Paul Boddie wrote:
> On 15 Nov, 09:30, Terry Reedy <tjre... at udel.edu> wrote:
>> greg wrote:
>>
> 
> [Shed Skin]
> 
>>> These restrictions mean that it isn't really quite
>>> Python, though.
>> Python code that only uses a subset of features very much *is* Python
>> code. The author of ShedSkin makes no claim that is compiles all Python
>> code.
> 
> Of course, Shed Skin doesn't support all the usual CPython features,
> but the code you would write for Shed Skin's benefit should be Python
> code that runs under CPython. It's fair to say that Shed Skin isn't a
> "complete" implementation of what CPython defines as being "the full
> Python", but you're still writing Python. One can argue that the
> restrictions imposed by Shed Skin inhibit the code from being "proper"
> Python, but every software project has restrictions in the form of
> styles, patterns and conventions.
> 
> This is where the "Lesser Python" crowd usually step in and say that
> they won't look at anything which doesn't support "the full Python",
> but I think it's informative to evaluate which features of Python give
> the most value and which we could do without. The "Lesser Python"
> attitude is to say, "No! We want it all! It's all necessary for
> everything!" That doesn't really help the people implementing "proper"
> implementations or those trying to deliver better-performing
> implementations.
> 
> In fact, the mentality that claims that "it's perfect, or it will be
> if we keep adding features" could drive Python into a diminishing
> niche over time. In contrast, considering variations of Python as some
> kind of "Greater Python" ecosystem could help Python (the language)
> adapt to the changing demands on programming languages to which Go
> (the Google language, not Go! which existed already) is supposedly a
> response.

     Yes.  Niklaus Wirth, who designed Pascal, Modula, and Oberon, had
that happen to his languages.  He's old and bitter now; a friend of
mine knows him.

      The problem is that "Greater Python" is to some extent "the set of
features that are easy to implement if we look up everything at run time."
You can insert a variable into a running function of
another thread.  This feature of very marginal utility is free in a
naive lookup-based interpreter, and horribly expensive in anything that
really compiles.  Obsession with the CPython implementation as the language
definition tends to overemphasize such features.

      The big headache from a compiler perspective is "hidden dynamism" -
use of dynamic features that isn't obvious from examining the source code.
(Hidden dynamism is a big headache to maintenance programmers, too.)
For example, if you had the rule that you can't use "getattr" and "setattr"
on an object from the outside unless the class itself implements or uses getattr
and setattr, then you know at compile time if the machinery for dynamic
attributes needs to be provided for that class.  This allows the "slots"
optimization, and direct compilation into struct-type code.

      Python is a very clean language held back from widespread use by slow
implementations.  If Python ran faster, Go would be unnecessary.

      And yes, performance matters when you buy servers in bulk.

					John Nagle



More information about the Python-list mailing list