[Cython] Shared Cython runtime (was: Upcoming cython/numpy breakage with stride checking)

[mark florisson]

On 9 April 2013 14:11, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Nathaniel Smith, 09.04.2013 15:00:
> > On 9 Apr 2013 13:50, "Stefan Behnel" wrote:
> >> Nathaniel Smith, 09.04.2013 14:25:
> >> There's also the problem of dependency hell and getting rid of old
> modules
> >> once they are no longer used on the user side. And also, how to get them
> >> there in the first place. Having one package overwrite the files of
> >> another during its installation is just asking for trouble.
> >
> > The system I described does not involve the addition of any new files to
> > any package.
>
> I take it then that you were envisaging a separate "cython-runtime" package
> on PyPI that Cython compiled modules would have to depend on?
>
> As long as people install their stuff using pip, that could work for them
> mostly ok, although with the regrettable Cython user impact of having to
> set that dependency for their packages in the first place.
>
> If people want to install stuff manually, dependency hell gets close.
>
> Or did you see any other ways of getting these things installed
> automatically, with a smaller user impact?
>
> Stefan
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>

For reference, here's a CEP about this written last year:
http://wiki.cython.org/enhancements/libcython

In numba we want to create a numba loader, which allows users to compile
their code to some intermediate form, making the dependency smaller but
most importantly allowing library authors to use a suitable version without
burdening users. This would also include a runtime which is substitutable
(which includes CPython API calls, exception control, etc). This would
always be needed to run numba code.

Cython doesn't necessarily need a runtime dependency, since each module
could create it, but I imagine it may reduce the memory footprint of
projects like sage? Maybe Cython could allow a runtime dependency as well
as allowing you to compile it into a module (e.g. cython
--include-runtime). Maybe this could be a single module that the user
imports from a package's __init__.py before any other cython module (and
this module will create and inject a 'cython.runtime' module into
sys.modules if not already there). That way there duplication is reduced to
different projects.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20130409/973adb10/attachment-0001.html>