[Cython] Store list of imported modules in .so file

Robert Bradshaw robertwb at gmail.com
Fri Nov 27 13:00:43 EST 2015


On Fri, Nov 27, 2015 at 9:13 AM, Kevin Norris <nykevin.norris at gmail.com> wrote:
> On Tue, Nov 24, 2015 at 12:50 PM, Hartmut Goebel
> <h.goebel at crazy-compilers.com> wrote:
>> Hi,
>>
>> I'm a core-developer for PyInstaller. We would like to enhance cython
>> support in PyInstaller and resolve dependencies of cython-compiled
>> modules automatically. This would ease freezing cython modules with
>> PyInstaller a lot and thus help users including cythonized-modules into
>> their shipped application.
>>
>> For us, the most elegant way to get the dependencies would be to have
>> the list of imported modules in the .so-file. E.g. some simple string
>> marked by some cookie which is easily grepped out of the binary.
>>
>> Is there any chance to get suche a list into the .so/.dll-file?
>
> Import statements are "just code."  They can appear inside functions
> and more generally can occur at runtime.  In the absolute worst case,
> you could have something like (Python 3.x):
>
>     importlib.import_module(input('What should I import today?'))
>
> This is not specific to Cython, but I imagine it's a lot harder to
> pull out via static analysis when you don't have Python-like source
> (see also: pkgutil.get_data(), any use of __file__, etc.).  OTOH, if
> you're dealing with importlib et al., you're already in a bad place
> for static analysis since people can do things like:
>
>     import importlib as foo; bar = foo.import_module; baz = bar('qux')
>
> (which is roughly equivalent to import qux as baz, but good luck
> spotting it automatically)
>
> Anyway, I think it might help to clarify what *exactly* you mean by
> "dependency."  If you mean "any module that might possibly get
> imported by module X," you're going to have a hard time with this.
> Setuptools takes a fully manual approach to this problem, requiring
> developers to explicitly list all dependencies.  That may not work for
> your needs, but it does seem to be more flexible since it lets you
> specify PEP 440 version constraints.

Yes, you can do all sorts of crazy stuff. On the other hand,
setuptools explicitness might be overkill for the 98% of projects that
whose dependencies could be easily inferred, and at least give you a
starting point.

- Robert


More information about the cython-devel mailing list