[Python-bugs-list] [ python-Bugs-558488 ] DL_EXPORT on VC7 broken

noreply@sourceforge.net noreply@sourceforge.net
Mon, 22 Jul 2002 22:48:29 -0700


Bugs item #558488, was opened at 2002-05-21 10:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=558488&group_id=5470

Category: Build
Group: Platform-specific
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: David Aldridge (daldridge)
Assigned to: Mark Hammond (mhammond)
Summary: DL_EXPORT on VC7 broken

Initial Comment:
Extension modules (_sre, _socket, etc.) do not have 
their entry points exported correctly when building 
Python (2.1 onwards) with Microsoft Visual C++ 7 
(part of Visual Studio .NET).  (NOTE: Everything 
compiles and links without errors.  It's just the 
exports that are missing.)

Think through the prepocessor flow when compiling 
_sre.c, for example, the following happens (this is 
from memory so please excuse inaccuracies):

1) _sre.c #includes "python.h"

2) "python.h" pretty much immediatly 
#includes "config.h"

3) "config.h" has the following:

#ifdef USE_DL_IMPORT
#define DL_IMPORT(RTYPE) __declspec(dllimport) RTYPE
#endif
#ifdef USE_DL_EXPORT
#define DL_IMPORT(RTYPE) __declspec(dllexport) RTYPE
#define DL_EXPORT(RTYPE) __declspec(dllexport) RTYPE
#endif

4) USE_DL_EXPORT is NOT defined (and cannot be in the 
_sre project, or it will not import pythoncore's (?) 
exports), thus DL_EXPORT is not yet defined.

5) After preprocessing "config.h", we're back 
in "python.h", which has a block:

#ifndef DL_EXPORT
#define DL_EXPORT(RTYPE) RTYPE
#endif

6) Back in _sre.c, the extension module entry point 
is:

DL_EXPORT(void) init_sre()

This expands to

void init_sre()

And thus does not get exported.



If I change the "config.h" block above to be:

#ifdef USE_DL_IMPORT
#define DL_IMPORT(RTYPE) __declspec(dllimport) RTYPE
#define DL_EXPORT(RTYPE) __declspec(dllexport) RTYPE
#endif
#ifdef USE_DL_EXPORT
#define DL_IMPORT(RTYPE) __declspec(dllexport) RTYPE
#define DL_EXPORT(RTYPE) __declspec(dllexport) RTYPE
#endif

(ie. add the DL_EXPORT line if/when USE_DL_IMPORT is 
defined), everything works correctly.  Not 
guaranteeing this is the "correct" solution, but it 
seems to work.

My specific test case was to build everything, run 
python.exe from the command line, then 
attempt "import re".

This results in a Traceback with "ImportError: 
dynamic module does not define init function 
(init_sre)".


----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2002-07-23 15:48

Message:
Logged In: YES 
user_id=14198

Patch checked in an all windows modules use PyMODINIT_FUNC
which isn't broken :)

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-06-08 15:17

Message:
Logged In: YES 
user_id=14198

A patch can be found at http://www.python.org/sf/566100

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-07 10:24

Message:
Logged In: YES 
user_id=31435

How about:

1. Write the new macros first, and check them in.  The old 
macros are still defined at this point.

2. Convert one module to use them, and check it in.

3. Post a brief "conversion guide" to Python-Dev.  This 
will also end up in somebody's 2.3 migration guide.

4. Let people have at it.  A little bit of this kind of 
editing is fun, mindless work, and I expect lots of people 
will do just a little bit of it.  We can clean up what's 
left.

5. When all uses have been converted, we "deprecate" the 
current macro set, and argue on Python-Dev about when they 
can go away.

It's easy if it's possible to make everyone feel guilty 
<wink>.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-06-07 10:07

Message:
Logged In: YES 
user_id=14198

OK Tim - how do we go about doing this?  I like the idea of
PyAPI_FUNC, PyAPI_DATA and PyMODINIT_FUNC - but there are
literally *hundreds* of places that need to be changed.

While the changes are largely mechanical, it will still have
a large impact.  Should we coordinate with other people that
this change is about to happen, or shall I just put together
a patch, we can check it in, and then go on vacation while
the shit hits the fan? <wink>

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-06 11:33

Message:
Logged In: YES 
user_id=31435

I didn't say they were a mess here.  It's unkind to criticize 
in public <wink>.  I really like your idea of purpose-oriented 
names.  How about PyAPI_FUNC and PyMODINIT_FUNC?  
Would it also be helpful to distinguish public API functions 
from public API data?  We have plenty of the latter too, and 
it's always surprised me that no platform has popped up 
that requires treating cross-blob data in a different way 
than cross-blob entry points.  But maybe that's because 
only Windows Geeks enjoy pleasant support for cross-blob 
stuff of any kind <0.9 wink>.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-06-06 10:57

Message:
Logged In: YES 
user_id=14198

Actually, the problem is simply that the "export:init_sre"
option was dropped from the *linker*.

USE_DL_EXPORT was a noop when not building the core.  Hence,
_sre used an explicit "export:" linker option, while _winreg
(for example) just used an explicit dlexport() statement in
the code (which it could safely do as it is Windows specific)

So I am not sure that closing it is the right thing to do. 
IMO, the right thing to do is to rationalize these DL_EXPORT
macros once and for all (and I can remove yet another bit of
my shame from the core Python tree <wink>)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-06-06 10:50

Message:
Logged In: YES 
user_id=6380

I guess the converter didn't save the CPP flags.  The V6
project defines these in the "settings" dialog:
NDEBUG,WIN32,_WINDOWS,USE_DL_EXPORT

If you say that USE_DL_EXPORT is not defined (point 4 above)
that means the converter (or something else :-) lost that.

I'll close this now.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-06-06 10:49

Message:
Logged In: YES 
user_id=14198

This is a mess <wink>.

MSVC6 works as the linker command line specifies
"/export:init_sre" on the commandline.  It appears that the
MSVC7 migration process does *not* migrate this option. 
Hence src does not export that symbol in MSVC7.  ie, this
confirms exactly what Tim says.

As Tim says, this DL_* stuff is a mess.  I think we simply
need 2 macros - one that says "this is a public Python API
function" and another that says "this is a Python module
init function".  I see no other requirements in the core.

I am attaching a patch that fixes the VC7 build problems,
but not "once and for all".  See the comments in the patch.
 I am not necessarily suggesting it be applied.

Tim - do you agree we really only have these 2 requirements
(public Python API vs Python module init function).?  If so,
how about we just decide on names, and take the hit now?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-06 10:21

Message:
Logged In: YES 
user_id=31435

Note that the MSVC project files we ship contain

/export:init_sre

in the linker options for the _sre project, so it doesn't 
matter whether the macros do anything in this project.  
Presumably VC7 screws this up, or the .dsp conversion 
dropped this linker flag on the floor.  MarkH and I promised 
ach other to rework this macro maze someday, but I 
haven't had time for it, and Mark hasn't either.  That's why 
I'm leaving it assigned to him <ahem>.

----------------------------------------------------------------------

Comment By: David Aldridge (daldridge)
Date: 2002-06-06 09:48

Message:
Logged In: YES 
user_id=550130

That's what I figured, too  (the "wouldn't work with VC6 either" 
comment, not the "because he's lonely" part ;-)

We were compiling/linking/using 2.1 with VC6 just fine -- no 
clue why it would be broken in VC7 (BTW, always using the 
GUI, never the cl.exe or msdev.exe/devenv.exe from the 
command line).  Perhaps the VC6->7 project converter hosed 
the .dsw/.dsps in some way, though the conversion seemed 
to work fine for all of our own stuff... *shrug*  I didn't think to 
do a visual comparison of the original .dsp's with the 
created .vcproj's (or workspace equivalents).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-06 05:40

Message:
Logged In: YES 
user_id=31435

Assigned to Mark because he's lonely <wink>.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-06-06 05:16

Message:
Logged In: YES 
user_id=6380

If you were right, it wouldn't build with VC6 either. But it does.

So what's wrong? I dunno. Asking Tim.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=558488&group_id=5470