[Distutils] [Python-Dev] Adventures with x64, VS7 and VS8 on Windows

[Mark Hammond]

Hi Martin,

> You are likely doing something wrong:
> a) I assume it's VS 7.1 (i.e. VS.NET 2003); VS 2002 is not supported
>    at all
> b) you probably didn't install vsextcomp, but you should.
>    In fact, you don't need all of it, but you do need the cl.exe and
>    link.exe wrappers it comes with - they dispatch to the proper
>    tools from the SDK
> c) in case it isn't clear: you also need an AMD64 compiler, e.g.
>    from the platform SDK.
>    Unfortunately, Microsoft keeps changing the registry settings for
>    the SDK, so vsextcomp only knows about some selected SDKs. If
>    that causes a problem, please let me know.

I'm using the full-blown VS.NET 2003, as given to a number of python-dev
people by Microsoft a number of years ago.  This appears to come with the
SDK and a 64bit compiler.  I'm guessing vsextcomp doesn't use the Visual
Studio 'ReleaseAMD64' configuration - would it be OK for me to check in
changes to the PCBuild projects for this configuration?

> This is an issue to be discussed for Python 2.6. I'm
> personally hesitant
> to have the "official" build infrastructure deviate from the
> layout that
> has been in-use for so many years, as a lot of things depend on it.

Yes, I agree - although I consider x64 new enough that an opportunity exists
to set a new 'standard'.  However, if most 'external' build processes will
not otherwise need to change for a 64bit environment, then I agree that
nothing should change in Python's layout.

> Right - PCbuild8 should not get formalized. It probably
> should continue to be maintained.

So is there something we can do to make distutils play better with binaries
built from PCBuild8, even though it is considered temporary?  It seems the
best thing might be to modify the PCBuild8 build process so the output
binaries are in the ../PCBuild' directory - this way distutils and others
continue to work fine.  Does that sound reasonable?

> For 2.6, the first question to answer is: what compiler should it use?
>
> I would personally like to see Python "skip" VS 2005 altogether,
> as it will be soon superceded by Orcas. Unfortunately, it's unclear
> how long Microsoft will need to release Orcas (and also, when Python
> 2.6 will be released), so I would like to defer that question by
> a few months.

I've no objection to that - but I'd like to help keep the pain to a minimum
for people who find themselves trying to build 64bit extensions in the
meantime.   Anecdotally, VS8 is the compiler most people start trying to use
for this (quite possibly because that is what they already have handy).

> See above; I disagree. First, "multiple architectures" only means x86,
> AMD64, and Itanium, and I would like to drop "official"
> Itanium binaries
> from 2.6 (even though they could continue to be supported in the build
> process). Then, even if the Python build itself support multiple
> simultaneous architectures, the extension modules don't all (correct
> me if I'm wrong).

Yes, I agree that it is unlikely to work in practice - at least for a number
of years as the external libs and extensions catch up.

> > * Wide characters in VS8: PC/pyconfig.h defines
> PY_UNICODE_TYPE as 'unsigned
> > short', which corresponds with both 'WCHAR' and 'wchar' in
> previous compiler
> > versions.  VS8 defines this as wchar_t, which I'm
> struggling to find a
> > formal definition for beyond being 2 bytes.
>
> In C or in C++? In C++, wchar_t is a builtin type, just like
> short, int,
> long. So there is no further formal definition.

This was in C++, but the problem was really WCHAR, as used by much of the
win32 API.

> I'd rather make it a platform-specific definition (for
> platform=Windows
> API). Correct me if I'm wrong, but isn't wchar_t also available in VS
> 2003 (and even in VC6?). And doesn't it have the "right" definition in
> all these compilers?

hrm - as above, I'm more concerned with the definition of WCHAR - which
means my problem is related more to the Platform SDK version rather than the
compiler.  This is unfortunate - on one hand we do consider
'platform=Windows API', and WCHAR is very much an API concept.  I'll need to
dig some more into this, but at least I know I'm not wasting my time :)

> > * Finally, as something from left-field which may well take
> > 12 months or
> > more to pull off - but would there be any interest to
> > moving the Windows
> > build process to a cygwin environment based on the existing autoconf
> > scripts?
>
> What compiler would you use there? I very much like using the VS
> debugger when developing on Windows, so that capability should not
> go away.

You would use whatever compiler the autoconf toolset found.  Recent versions
know enough about MSVC for simple projects.  Many people would need to take
care that their environment pointed at the correct compiler - especially the
person building releases.

But assuming MSVC was found and had the appropriate switches passed, there
would be no impact on the ability to use Visual Studio as a debugging
environment.

Thanks,

Mark