Python and the need for speed
steve at pearwood.info
Sun Apr 9 03:39:06 EDT 2017
On Sun, 09 Apr 2017 13:57:28 +1000, Chris Angelico wrote:
> On Sun, Apr 9, 2017 at 10:20 AM, <breamoreboy at gmail.com> wrote:
>> I've an idea that http://www.mos6581.org/python_need_for_speed is a
>> week late for April Fool's but just in case I'm sure that some of you
>> may wish to comment.
I'm not sure why Mark thinks that wanting Python to be faster is a joke.
As the page points out, there are many projects attempting (with greater
or lesser success) to speed up Python.
I'm happy for Mark if his need for high performance is low, but CPython
at least is definitely in the middle-tier of performance, and PyPy isn't
suitable for all use-cases. I don't know anyone who has ever said "this
interpreter is too fast, can you make it run slower?"
>>From that page:
>> Other candidates for banishment from TurboPython include eval and exec.
> Bye bye namedtuple.
All that would mean is that the implementation of namedtuple would have
to change. It would probably require some sort of support from the
compiler, but that's okay, the compiler can do (nearly) anything.
Something as important and useful as namedtuple would not be dropped from
this hypothetical TurboPython. It would just shift the implementation
from pure-Python to something else.
exec() is actually only needed for a *tiny* bit of namedtuple. The
original implementation by Raymond Hettinger takes the easy way out by
using exec on the entire class definition, but it is trivially easy to
restrict that to just the class __new__ method:
but even that could be avoided with a minimal amount of help from the
> And compile() is going to have to go,
> since you
> could implement eval/exec by creating a new function:
>>>> runme = r"""
> print("Hello, world!")
> import sys sys.stdout.write("I can import modules.\n")
>>>> type(lambda: 1)(compile("def f():" + runme, "exec",
>>>> "exec").co_consts, globals())()
> Hello, world!
> I can import modules.
> So if compile() goes, you also lose ast.parse,
> which means you lose introspection tools,
*Some* introspection tools. Python has many that don't rely on compile or
Hell, even Java has some introspection tools, based on reflection, which
lets you do Python-like things at runtime. Of course if you do this, you
> plus you lose literal_eval and friends.
I don't know what "friends" you are referring to, but again, if
literal_eval is important, the compiler can support it. If you can
support an entire Python interpreter and compiler in the form of
compile(), then you can support more restricted subset of the language.
Writing a parser to evaluate strings, integers, and a few other data
types is not exactly brain surgery.
> I'm also
> not sure whether the import machinery would have to be rewritten, but a
> quick 'git grep' suggests that it would. Removing eval and exec is not
> as simple as removing them from the standard library.
Well of course not, but removing eval and exec is only a necessary, not
sufficient, condition, for enabling a huge bunch of compiler optimizations
and speeding up Python.
This "TurboPython" would require a completely new implementation of the
Python interpreter to be fast. It's not as if eval and exec are great
heavy concrete weights chained to the leg of the compiler, and all you
need do is remove the chain and the compiler suddenly becomes thirty
> In fact, extreme dynamism is baked deep into the language. You'd have to
> make some fairly sweeping language changes to get any real benefits from
> restricting things.
Well, maybe. As is pointed out many, many times, 99% of Python code
avoids the sorts of extreme dynamism that keeps things slow. Lots of
people would be satisfied with a language *really close* to Python that
was ten or twenty times faster, even if it meant that you couldn't write
code like this:
answer = input("What's your name?")
exec("name = %r" % answer)
Even better would be if the compiler was smart enough to use the
optimized, fast runtime when the dynamic features aren't used, and fall
back on a slower implementation only when needed to support the more
I wonder how Victor Stinner's FAT Python is going?
> There are better ways to improve performance,
More information about the Python-list