Fwd: Question about how best require compiler options for C sources
Hi all, I'd like to call attention to an issue I've been looking into for the past couple days: https://github.com/cython/cython/pull/360 To summarize the discussion, when building DLLs for Windows, functions that should be exported by that DLL must be marked __declspec(dllexport) in their declaration. However, when using the same header file in a project that links to that DLL the same function must be declared __declspec(dllimport). It's common practice to have a macro for this, whose value is controlled by whether or not a macro (often called something like "DLL_EXPORT"). When compiling the DLL we would define -DDLL_EXPORT to output __declspec(dllexport). Otherwise it outputs __declspec(dllimport). Cython currently handles this with such a macro called DL_IMPORT which comes from Python. However, this macro was deprecated some time ago, and is removed in current Python 3 versions. So Cython must replace it with its own. I'm working on a patch for this--to reduce confusion the macro is named specifically for the Cython module it's associated with. For example, for a Cython module named "foo.bar" there are two macros DLL_EXPORT__foo__bar and EXPORT__foo__bar. If the latter is defined then the former outputs dllexport, otherwise it outputs dllimport. I've attached the patch in progress. I'm open to comment on this, but where I'm stuck now is that in order for the "foo.bar" module to be compiled correctly it needs EXPORT__foo__bar to be defined at compile time. And it's not clear to me what the best way is for the Cython compiler to pass down options that are passed to the C/C++ compiler. In general the best way would be to attach this to the define_macros attribute of the associated distutils/setuptools Extension object and let the distutils compiler class generate the right compiler options. But it's not clear to me from Cython's internals where the best place to do that would be. Thanks, Erik
Can you give a tiny concrete example? My questions are basic enough that I feel like I'm missing something fundamental :-) My first question is why you even need this, since AFAIK there are no cases where it is correct to have a cython module dllexporting symbols that appear in header files. This is what cimport is for, right? My second question is why you would want to do this via the command line, when compiling the dll means that you are compiling some cython-generated .c, which means that you can put the #define directly in the source code, no? -n On Apr 8, 2016 5:35 AM, "Erik Bray" <erik.m.bray@gmail.com> wrote:
Hi all,
I'd like to call attention to an issue I've been looking into for the past couple days:
https://github.com/cython/cython/pull/360
To summarize the discussion, when building DLLs for Windows, functions that should be exported by that DLL must be marked __declspec(dllexport) in their declaration. However, when using the same header file in a project that links to that DLL the same function must be declared __declspec(dllimport).
It's common practice to have a macro for this, whose value is controlled by whether or not a macro (often called something like "DLL_EXPORT"). When compiling the DLL we would define -DDLL_EXPORT to output __declspec(dllexport). Otherwise it outputs __declspec(dllimport).
Cython currently handles this with such a macro called DL_IMPORT which comes from Python. However, this macro was deprecated some time ago, and is removed in current Python 3 versions. So Cython must replace it with its own.
I'm working on a patch for this--to reduce confusion the macro is named specifically for the Cython module it's associated with. For example, for a Cython module named "foo.bar" there are two macros DLL_EXPORT__foo__bar and EXPORT__foo__bar. If the latter is defined then the former outputs dllexport, otherwise it outputs dllimport. I've attached the patch in progress.
I'm open to comment on this, but where I'm stuck now is that in order for the "foo.bar" module to be compiled correctly it needs EXPORT__foo__bar to be defined at compile time. And it's not clear to me what the best way is for the Cython compiler to pass down options that are passed to the C/C++ compiler. In general the best way would be to attach this to the define_macros attribute of the associated distutils/setuptools Extension object and let the distutils compiler class generate the right compiler options. But it's not clear to me from Cython's internals where the best place to do that would be.
Thanks, Erik
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
On Fri, Apr 8, 2016 at 5:49 PM, Nathaniel Smith <njs@vorpus.org> wrote:
Can you give a tiny concrete example? My questions are basic enough that I feel like I'm missing something fundamental :-)
Yes, I think you might be missing something, but I'm not sure exactly where. In the issue I provided a tiny example which you can see here: https://gist.github.com/embray/12a67edb82b213217e31f408007898e6 The C code generated by this example currently does not compile on Windows, because of how Cython uses DL_IMPORT incorrectly. Regardless of what it *does* do, Cython should not be using the DL_IMPORT macro at all actually since that no longer even exists in Python.
My first question is why you even need this, since AFAIK there are no cases where it is correct to have a cython module dllexporting symbols that appear in header files. This is what cimport is for, right?
I don't think this has anything to do with cimport. Could you explain what you mean?
My second question is why you would want to do this via the command line, when compiling the dll means that you are compiling some cython-generated .c, which means that you can put the #define directly in the source code, no?
Not necessarily. As you can see in my example the file fooutil.c is hand-written C code that was not generated by Cython, but which uses a function in the Cython-generated code. It includes "foo.h". In principle you're right--the hand-written C code could set the proper #defines but it would have to do so *before* including "foo.h". It's not very obvious that this would be needed. Most other code I've seen that addresses this issue--including cpython itself--do so by passing an appropriate define to the compiler via the command-line, and that seems the clearest to me. Best, Erik
On Apr 8, 2016 5:35 AM, "Erik Bray" <erik.m.bray@gmail.com> wrote:
Hi all,
I'd like to call attention to an issue I've been looking into for the past couple days:
https://github.com/cython/cython/pull/360
To summarize the discussion, when building DLLs for Windows, functions that should be exported by that DLL must be marked __declspec(dllexport) in their declaration. However, when using the same header file in a project that links to that DLL the same function must be declared __declspec(dllimport).
It's common practice to have a macro for this, whose value is controlled by whether or not a macro (often called something like "DLL_EXPORT"). When compiling the DLL we would define -DDLL_EXPORT to output __declspec(dllexport). Otherwise it outputs __declspec(dllimport).
Cython currently handles this with such a macro called DL_IMPORT which comes from Python. However, this macro was deprecated some time ago, and is removed in current Python 3 versions. So Cython must replace it with its own.
I'm working on a patch for this--to reduce confusion the macro is named specifically for the Cython module it's associated with. For example, for a Cython module named "foo.bar" there are two macros DLL_EXPORT__foo__bar and EXPORT__foo__bar. If the latter is defined then the former outputs dllexport, otherwise it outputs dllimport. I've attached the patch in progress.
I'm open to comment on this, but where I'm stuck now is that in order for the "foo.bar" module to be compiled correctly it needs EXPORT__foo__bar to be defined at compile time. And it's not clear to me what the best way is for the Cython compiler to pass down options that are passed to the C/C++ compiler. In general the best way would be to attach this to the define_macros attribute of the associated distutils/setuptools Extension object and let the distutils compiler class generate the right compiler options. But it's not clear to me from Cython's internals where the best place to do that would be.
Thanks, Erik
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
participants (2)
-
Erik Bray -
Nathaniel Smith