[Python-ideas] A General Outline for Just-in-Time Acceleration of Python

[David Mertz]

Is there ever a case where removing all the type annotations from Cython
code does not produce code that can run in PyPy?  I don't know Cython well
enough to be certain the answer is 'no', but I think so.  So a function a
little like my 'silly()' function--but that did something actually
interesting in the loop--might run faster by removing the annotation and
running it in PyPy.  Or is might NOT, of course; the answer is not obvious
without looking at the exact code in question, and probably not without
actually timing it.

But the idea is that let's say I have some code with a loop and some
numeric operations inside that loop that I'm currently running using
CPython.  There are at least two ways I might speed up that code:

A) Edit the code to contain some type annotations, and compile it with
Cython.  However, if I do this, I *might* have to modify some other
constructs in the overall code block to get it to compile (i.e. if there's
any polymorphism about variable types).

B) Run the unchanged code using PyPy.

Well, in this description, PyPy sounds better... but it's not better if
option (A) makes faster code in *your* specific code, of course.  And
moreover, (B) is not true if your existing code relies on C extensions,
such as NumPy, which mostly aren't going to run on PyPy.

However, I do know about https://bitbucket.org/pypy/numpy.  At least some
substantial part of NumPy has been ported to PyPy.  This may or may not
support the code *you* need to run.


On Sat, Jun 14, 2014 at 12:38 AM, Joseph Martinot-Lagarde <
joseph.martinot-lagarde at m4x.org> wrote:

> Le 14/06/2014 09:30, David Mertz a écrit :
>
>> On Fri, Jun 13, 2014 at 11:54 PM, Joseph Martinot-Lagarde
>> <joseph.martinot-lagarde at m4x.org
>> <mailto:joseph.martinot-lagarde at m4x.org>> wrote:
>>
>>     Cython compiles all python, it is not restricted.
>>
>>
>> Well, kinda yes and no.  You are correct of course, that anything that
>> you can execute with 'python someprog' you can compile with 'cython
>> someprog'.  However, there is an obvious sense in which adding an
>> annotation (which is, of course, a syntax error for Python itself)
>> "restricts" the code in Cython.  E.g.:
>>
>>     def silly():
>>          cdef int n, i
>>          for i in range(10):
>>              if i < 5:
>>                  n = i + 1
>>              else:
>>                  n = str(i)
>>
>> This *silly* function isn't really Python code at all, of course.  But
>> if you ignore the annotation, it would be--pointless code, but valid. As
>> soon as you add the annotation, you *restrict* the type of code you can
>> write in the scope of the annotation.
>>
>>
> Yeah, the point is that *you* restrict, not cython. From your previous
> post I understood that you meant "pypy runs all python but cython doesn't,
> it is restricted".
>
> I use numpy regularely, and in this case it is the other way around: I can
> optimize my code using cython but I can't run it with pypy at all.
>
>
> ---
> Ce courrier électronique ne contient aucun virus ou logiciel malveillant
> parce que la protection avast! Antivirus est active.
> http://www.avast.com
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140614/43187714/attachment-0001.html>