[Cython] Fedora failures with Cython 3.0a5

[Stefan Behnel]

Hi Marcel,

thanks for testing this, and for triaging the issues!

Marcel Plch schrieb am 27.05.20 um 10:27:
> In Fedora, we rebuilt 152 packages against Cython 3.0a5 we have found
> some breakages that might be relevant to you. We are not entirely sure
> whether these issues are caused by the individual projects' wrong usage
> of Cython, or if Cython broke them unintentionally. Do any of the
> failures look like Cython bugs to you? We can provide moredetail if one
> looks interesting.  Some expected problems, like python2
> syntax/semantics, were taken into account and noted below as issues of
> the individual projects.
> 
> The linked logs will be garbage-collected in about 2 weeks; let us know
> if this happens and we can rebuild the packages.
> 
> 
> # Missing .pxd files
> Four packages were broken by being unable to locate .pxd files to compile.
> Packages broken by this:
> - mlpack https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01409078-mlpack/builder-live.log.gz
> - MDAnalysis https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01409419-python-MDAnalysis/builder-live.log.gz
> - pplpy https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01411510-python-pplpy/builder-live.log.gz
> - bintrees https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01409724-python-bintrees/builder-live.log.gz
> 
> Can this be caused by different import semantics of Python 3?

Yes, that's most likely the reason. Could just be a "from . cimport …"
missing. You can also build them with the directive "py2_import=True" to
get back Py2 import semantics (i.e. trying a relative import first).


> # GIL issues
> In three packages, problems with the GIL occurred. Some operations
> requiring the gil were executed when gil wasn't being held and vice
> versa.
> Packages broken by this:
> - tables https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01408091-python-tables/builder-live.log.gz

The GIL errors here seem due to the missing .pxd imports. There's a failing
star-import somewhere which probably messes things up a lot.

> - scikit-image https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01412046-python-scikit-image/builder-live.log.gz

See Gregory Lee's email.

> - ssh2-python https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01412227-python-ssh2-python/builder-live.log.gz

Could also be due to failing cimports.


> # AttributeError in Cython
> One package might have revealed a bug inside of Cython. It triggers a
> scope inside of ExprNodes.py that seems to have a missing attribute that
> should be there. See the log for more info.
> Package broken by this:
> - giacpy https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01410600-python-giacpy/builder-live.log.gz

Seems a genuine bug in Cython. I created a ticket:

https://github.com/cython/cython/issues/3651

> # Some imports fail to be executed
> This might be related to some of the previous problems, or a bug in
> cimport. Two packages failed tests on importing a module from the the
> same package. In fisx, the import was done from  a Cython generated
> file. In cytoolz, a cythonized module was partially initialized,
> possibly due to cyclic import.
> Package broken by this:
> - fisx https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01410439-python-fisx/builder-live.log.gz

This could be trying to import a package-local module without a relative
import or qualified module name, i.e. a Py2-import.


> - cytoolz https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01410135-python-cytoolz/builder-live.log.gz

Yes, could be a cyclic import. This now shows due to PEP-489. Needs some
investigation to see what a good fix would be, but these things are usually
best resolved on user side.


> # Assignment to a read-only property
> One package broke on error stating attempts to assign to a read only properties.
> Package broken by this: Pandas https://download.copr.fedorainfracloud.org/results/dormouse/py38cy30a5/fedora-rawhide-x86_64/01408020-python-pandas/builder-live.log.gz

The pandas team made us aware of one such issue, and it's probably the
same. This is due to the old NumPy C-API, which was deprecated several
years ago (2013?) and is now being phased out. They'll probably resolve it
on their side (or might have already). Not directly Cython related, we just
follow NumPy here now.


> Some problems were expected/unrelated:
> - 9 packages use python 2 syntax and don't set the default

Might be easiest to set the directive "language_level=2" for these.


> - 2 packages were affected by the -fno-common default in GCC 10

Thanks for metioning these.

Stefan