[Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

[Nick Coghlan]

On 2 February 2018 at 06:15, Barry Scott <barry at barrys-emacs.org> wrote:
>
>
> On 30 Jan 2018, at 05:45, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> I'll also note that one of the things we (and others) *have* been
> putting quite a bit of time into is the question of "Why do people
> avoid using extension modules for code acceleration?".
>
>
> I think that is simple. Those that try give up because its a difficult API
> to
> call correctly.
>
> At PYCON UK on speaker explain how she, PhD level researcher,
> had failed to get the a C extension working.

Aye, indeed. That's a big part of why we've never had much motivation
to fill in the "How to do this by hand" instructions on
https://packaging.python.org/guides/packaging-binary-extensions/: it's
so hard to get the refcounting and GIL manager right by hand that it's
almost never the right answer vs either using a dedicated
extension-module-writing language like Cython, or writing a normal
shared external library and then using a wrapper generator like
cffi/SWIG/milksnake, or using a helper library like PySIP/PyCXX/Boost
to do the heavy lifting for you.

So while wheels and conda have helped considerably with the
cross-platform end user UX of extension modules, there's still a lot
of work to be done around the publisher UX, both for existing
publishers (to get more tools to work the way PySIP does and allow a
single wheel build to target multiple Python versions), and for new
publishers (to make the various extension module authoring tools
easier to discover, rather than having folks assuming that handcrafted
calls directly into the CPython C API is their only option).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia