[Cython] Big backwards compatibility problem with fused types

Robert Bradshaw robertwb at gmail.com
Tue Sep 19 16:55:23 EDT 2017

On Tue, Sep 19, 2017 at 10:48 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Hi devs,
> this is really bad:
> https://github.com/cython/cython/pull/1873
> The problem is that cdef functions with fused types are expanded
> arbitrarily on use, and their Entries removed and re-appended to the list
> of cdef function entries. But that list also defines the order of the
> entries in the vtable!
> Currently, when a method with fused types is defined in a .pxd file, it is
> the implementation order in the .pyx (!) file that determines the order in
> the vtable. Even worse, for modules that cimport from that .pxd file, it is
> the order in which these methods are *called* that determines the
> corresponding vtable on the other side, as the entries are expanded at
> first use, i.e. at their first occurrence in the code, and appended to the
> current list of cdef functions at this (arbitrary) point.
> What this means is that there is no way to safely infer the vtable order of
> an extension type with fused methods from a .pxd file.
> Now, the correct way to handle this would have been to *replace* the
> original fused entry with the expanded list of specialised entries. But
> it's too late for that now. All existing modules out there would need to
> get recompiled if we changed their vtable order, at least if they use fused
> any types methods anywhere in their hierarchy.
> However, I think we can assume code that uses a different method order in
> the .pxd and .pyx files to be broken already, so a way out of this misery
> would be to make the .pxd order the one and only source, with the twist
> that we must keep the "delete and re-append" algorithm to keep the order
> that existing translated modules are using.
> Thus the proposal:
> - we switch to a scheme now that is only defined by the first declaration
> order, i.e. the order in the .pxd file if there is one.
> - we make sure all fused methods are expanded in that same order, but
> continue to follow all non-fused methods in the vtable.
> - we break all code now that uses a different order of fused methods in
> their pyx and pxd files
> Thoughts?

+1. We should let the .pxd file, as it is declared, be the source of
truth. Code that doesn't already follow this is and does cross-module
fused-cdef function calls is already broken.

More information about the cython-devel mailing list