[Patches] [ python-Patches-614770 ] MSVC 7.0 compiler support

SourceForge.net noreply@sourceforge.net
Sun, 08 Jun 2003 14:33:37 -0700


Patches item #614770, was opened at 2002-09-25 20:09
Message generated for change (Comment added) made by djohnanderson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=614770&group_id=5470

Category: Distutils and setup.py
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: John Anderson (djohnanderson)
Assigned to: Jeremy Hylton (jhylton)
Summary: MSVC 7.0 compiler support

Initial Comment:
Distutils doesn't work with the current shipping
version of the Microsoft compiler (7.0). I've got a
patch that fixes it (context diffs of msvccompiler.py
against the latest code in CVS).

----------------------------------------------------------------------

>Comment By: John Anderson (djohnanderson)
Date: 2003-06-08 14:33

Message:
Logged In: YES 
user_id=618290

I just got the 7.1 version of the microsoft compiler and
read over the code jhylton submitted and it looks like there
is a problem with  get_msvc_paths, which hard codes the
registry path:

..\7.0\VC\VC_OBJECTS_PLATFORM_INFO\Win32\Directories

it needs to be:
..\7.1\VC\VC_OBJECTS_PLATFORM_INFO\Win32\Directories

for the 7.1 compiler.

Although, there may be other problems with 7.1, I suspect
they are minor, so it would probably be pretty easy to
include minor changes to support 7.1



----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2003-05-09 09:07

Message:
Logged In: YES 
user_id=31392

Applied variant as rev 1.54 of msvccompiler.py.
Tested with MSVC 7 for 2.3 and MSVC6 for 2.2 and 2.3.


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2003-05-01 15:44

Message:
Logged In: YES 
user_id=31392

I ran into this problem today, so I think I'll spend some
time to fix it.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-12-06 09:34

Message:
Logged In: YES 
user_id=31435

Nobody at PLabs has MSVC7, nor time to work on this even 
if we did.  So it would require people outside of PLabs to do 
all the work.  Incompatibility with MSCV6 needs to be 
addressed too, as I doubt Guido would be willing to say that 
anyone with an extension module on Windows *has* to use 
MSVC7 if they want their extension to run with Python 2.3.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-12-06 08:37

Message:
Logged In: YES 
user_id=11105

I want to support John's comment about MSVC 7. Not that I 
like it, but I'm forced to to use it. Seriously, what would be 
required to release the (official) 2.3 Python windows build with 
MSVC7?

----------------------------------------------------------------------

Comment By: John Anderson (djohnanderson)
Date: 2002-12-06 08:03

Message:
Logged In: YES 
user_id=618290

I think Tim's change which makes it possible to correctly
identify the compiler used to build Python makes a lot of
sense. So I'm happy to include this in the patch. I'll also
remove the Alpha code. I will test it with the version 6 and
7 compilers using python 2.3. I'm very busy these days so it
may take me awhile.

On a slightly different subject, I think you guys should
consider switching over to the version 7 compiler. Version 6
is over 6 years old. The version 7 compiler is nearly a year
old and a new version is on the horizon. Since I've been
building Python with the version 7 compiler I've identified
and f ixed the few problems necessary to get it to work and
would be happy to provide Version 7 project files. This
would make it easy for people to use either compiler, which
in turn would make it easier for you to eventually make the
transition to a newer compiler.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-12-06 01:56

Message:
Logged In: YES 
user_id=21627

John, given Tim's recent changes: Are you willing to revise
your patch?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-11 11:38

Message:
Logged In: YES 
user_id=31435

Martin, assuming the tests pass, I'm about to check in a 
change so that, under 2.3,

>>> import sys
>>> sys.version
'2.3a0 (#29, Nov 11 2002, 14:23:17) [MSC v.1200 32 bit 
(Intel)]
>>>

under MSVC 6.  Also gets rid of the _M_ALPHA code.  
Those are good changes regardless of the fate of the issue 
here, but at least poor John won't have to figure out how to 
trick the preprocessor into stringizing the value of 
_MSC_VER (that's not easy, alas).

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-11 09:07

Message:
Logged In: YES 
user_id=21627

The typical distutils user does use a Python binary compiled
by somebody else, and may have multiple MSVC versions
installed, or may have the wrong version installed.

I was hoping that you can find the MSVC version from
sys.version, but that only tells you that it is MSC (or that
it isn't). I would suggest to modify the compiler
identification in sys.version to include the MSC version.
Perhaps we could eliminate the identification of Win32-Alpha
at the same time.

IOW, the compiler string should read
[MSC v.12 32 bit]
or perhaps v.1200 if that makes it easier.

Then, distutils should look into sys.version for the
compiler revision, and assume v.1200 (i.e. MSVC6) if no
version indication is found.

----------------------------------------------------------------------

Comment By: John Anderson (djohnanderson)
Date: 2002-11-11 08:19

Message:
Logged In: YES 
user_id=618290

I'm a bit confused by the question. On my computer, I
installed the current Microsoft compiler (7.0), compiled
Python, installed distutils with my patch, and built several
extensions. So distutils did use the same compiler as was
used to build Python.

If you want to use the same compiler that someone else built
Python with, then you'll need to get the same compiler they
used, to make sure it's the only version that's installed,
and distutils works fine.

Perhaps you mean that there is some way to determine which
compiler was used by Python from examining something in
Python itself. If this is possible, please let me know how
to do it. In this case I'd propose modifying the patch to
write out of warning when a different compiler was used to
run distutils than was used to compile Python. In the case
of the Microsoft compiler not all versions have been
incompatible.

Or perhaps you mean distutils should have a generalized way
to specify which version of compiler to use when more than
one is installed on your computer.



----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-11 07:09

Message:
Logged In: YES 
user_id=21627

It seems clear that the patch is unacceptable in its current
form: distutils *must* use the compiler that was used to
compile Python.

Would you be willing to revise your patch in that direction?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-10-16 01:50

Message:
Logged In: YES 
user_id=11105

It seems Martin is correct.

I made the following experiment:
Compile pythoncore, python and pythonw with MSVC7, and 
the remaining extension modules with MSVC6 (all, except 
bsddb and _tkinter).

Now the test-suite crashes hard in test_parser.

When everything is built with MSVC7, the test-suite runs fine.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-16 01:20

Message:
Logged In: YES 
user_id=21627

The biggest problem is that VC.NET uses a new C library,
msvcr70[d].dll.

I have not fully studied all issues, but it appears that,
for purposes of Python, you cannot mix extensions, atleast
not in the general case. In particular, you cannot pass
FILE* between both C libraries. This is particularly
annoying, since MS has managed that struct _iobuf (aka FILE)
has identical layout in both compilers. Nevertheless, it
crashes in the following scenario:

VC7:fopen
VC: fputs

The problem is that fputs wants to lock the file. For that,
it tests whether the pointer comes from its own _iob
(_file.c:_lock_file). If the pointer comes from the _iob of
the other C library, it concludes that this must be a _FILEX
(which it isn't), and crashes :-(

So it appears that one *must* build extension modules with
the Visual Studio that also has built Python.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-10-15 14:01

Message:
Logged In: YES 
user_id=31435

Thomas, I'm just curious here:  if we were to create the 
Windows Python distribution with VC7, would the DLLs be 
compatible with extensions compiled by VC6?  In return for 
your answer to that <wink>, here's how to determine your 
MSVC6 service pack level:

HKLM\
SOFTWARE\
Microsoft\
VisualStudio\
6.0\
ServicePacks

and look at key "latest".

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-10-15 10:13

Message:
Logged In: YES 
user_id=11105

I've tried out the patch, and it works for me, at least with 
simple extensions - both with MSVC 6 and with MSVC 7, 
freshly installed.

I don't know the service pack level of my MSVC6, Help-
>About doesn't show it.

I'm not sure if using 7.0 (if available) should be the default. 
Python itself is still built with MSVC6 AFAIK.

----------------------------------------------------------------------

Comment By: John Anderson (djohnanderson)
Date: 2002-10-08 08:51

Message:
Logged In: YES 
user_id=618290

Thanks Martin for taking the time to review my proposed changes.

I think your comments about looking in
Software\Microsoft\VisualStudio instead of
Software\Microsoft\DevStudio is a good point. I decided to
two look only in VisualStudio because that works for both
the version 6 and 7 compiler I tested with. I don't know if
the version 5 compiler also stores the version in DevStudio.
Another alternative, which would of course complicate code,
would be two look in both places and choose the highest
version. Let me know if that's what you'd prefer and I'll
upload a new patch.

The problem I'm having with the version 6 compiler (latest
service Pack 5) is that
SOFTWARE\Microsoft\Devstudio\6.0\Build System\ doesn't
exist. Instead it looks like it's been moved to
SOFTWARE\Microsoft\Shared Tools\Build System\, but in that
new location, SOFTWARE\Microsoft\Shared Tools\Build
System\Components\Platforms\Win32 (x86)\Directories doesn't
exist. This has the effect of not getting the correct
include directories for builds.

This also points out a serious flaw in looking at
undocumented registry entries to find information for the
build -- there's no guarantee that the registry information
won't change even within the same version of the compiler. I
don't have a good solution for this problem, but I'd rather
distutils reported an error when it couldn't find the
registry entries it expected -- rather than silently
ignoring it as it does now. In a few places I added code to
report unexpected missing registry entries, but not all. I
could if you're interested add error code in all cases.

Fixing the problem I'm having with the version 6 compiler
seems difficult, since it seems to work for you and doesn't
work for me -- apparently are registries are different.
Personally I'm content with leaving the version 6 compiler
broken since it isn't obvious how to fix it and it
apparently works for some people and I only intend to use
the version 7 or newer compilers.

I added three new functions: convert_mbcs, read_key, and by
far the largest: expand_macros. The first two make the code
simpler, easier to read, avoid unnecessary duplications, and
minimize the risk that someone would forget to deal with
mbcs. It would be difficult to understand the bug fix
without these two functions. My hope was that these changes
would make it easier for the next person who needs to learn
the code. The last, expand_macros, is necessary because the
version 7 compiler introduces macros which didn't exist in
previous versions of the compiler. It would be awkward to
implement the macros without having adding a new function.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-08 02:21

Message:
Logged In: YES 
user_id=21627

I'm asking because you are not looking into
Software\Microsoft\Devstudio anymore to find the most
recent version. Not supporting MSVC5 anymore is probably
acceptable.

I never noticed that support for MSVC6 is broken - it works
fine for me... However, if you think you can improve that
somehow, please do - please elaborate what changes solve
what problems, though. It seems that a number of changes are
not strictly necessary (e.g. creation of new functions), to
really evaluate the patch, I have to know why you propose
these changes.

----------------------------------------------------------------------

Comment By: John Anderson (djohnanderson)
Date: 2002-10-07 08:27

Message:
Logged In: YES 
user_id=618290

It's been so long since I had a copy of MSVC 5 -- I think it
became obsolete about 6 or 7 years ago. None of my changes
should have any impact on the operation of MSVC 5, but of
course you never know unless you try it. Also, the MSVC 6
support in distutils is currently broken -- although it
finds the compiler, the code to find the include paths is
totally broken. I have MSVC 6 (latest service pack) and 7
and would be willing to make both those work. Anyone who's
using 5 is going to have lots of other problems to deal with
besides distutils. Wouldn't surprise me if the MSVC 6 code
for finding paths would differ in each service pack -- since
it depends upon unsupported registry entries.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-10-07 04:43

Message:
Logged In: YES 
user_id=21627

Can you report whether this patch works with MSVC 5?

----------------------------------------------------------------------

Comment By: John Anderson (djohnanderson)
Date: 2002-09-26 14:38

Message:
Logged In: YES 
user_id=618290

Opps, I guess I forgot to check that little box. Sorry about
that.

----------------------------------------------------------------------

Comment By: John Anderson (djohnanderson)
Date: 2002-09-26 14:38

Message:
Logged In: YES 
user_id=618290

There's no uploaded file!  You have to check the
checkbox labeled "Check to Upload & Attach File"
when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do
nothing about. :-( )

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-25 21:30

Message:
Logged In: YES 
user_id=21627

There's no uploaded file!  You have to check the
checkbox labeled "Check to Upload & Attach File"
when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do
nothing about. :-( )

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=614770&group_id=5470