Mailman 3 Supporting Visual Studio 2010 - Python-Dev

Supporting Visual Studio 2010

older
clarification: subset vs equality...

Brian Curtin

5 Apr 2011 5 Apr '11

2:08 a.m.

Would it be reasonable to begin supporting Visual Studio 2010 for Windows builds of 3.3? I now have a personal interest in this happening for some stuff at work, and there's been a lot of questions in the last few months about when we'll support it coming from python-list, #python-dev, and in person at PyCon. I wasn't around for the transition from 2005 to 2008, but I see we have a few sub-folders in PC/ for previous versions, so apparently we may support multiple versions at one time. Does it make sense to start this process now for a change to 2010? If it's not feasible to release 3.3 from a 2010 build, when might we be able to make the change? Keep in mind the 3.3 final release is almost a year and a half away, and we already know that Microsoft is likely to pull the cord on VS2008 Express at some point now that 2010 has been out for a while. I'm willing to do the work on this, but I just want to make sure it's a worthwhile effort.

Attachments:

attachment.htm (text/html — 1.0 KB)

Show replies by date

"Martin v. Löwis"

5 Apr 5 Apr

3:10 a.m.

Am 04.04.2011 22:38, schrieb Brian Curtin:

...

Would it be reasonable to begin supporting Visual Studio 2010 for Windows builds of 3.3?

Interesting question. The following concerns have played a role in the past: - depending on the timing of the next VS release, Python may actually want to skip VS 2010, and move right to VS 2012 (say). - users have expressed concerns that they constantly need to upgrade VS releases when developing for Python. With VS Express, that concern may be reduced - but you still need to buy a full license if you want to support AMD64. - users have also expressed concerns that old VS versions become unavailable; of course, these are different users (since the first ones have already bought copies of VS 2008). The counter-argument is that you can still get cheap copies on Ebay, but that may be a red herring. - every time this comes up, people also suggest that we should stop building with VS, and use gcc in the first place.

...

Does it make sense to start this process now for a change to 2010?

I'll abstain from a vote here, and I think it essentially comes down to voting (or somebody putting the foot down saying "I want this now", which really was the way it worked the last time(s)). Somebody would need to take charge of this, and fix all the issues that come up: incompatibilities, generation of backwards-compatible project files, MSI packaging, getting licenses to buildbot operators. So if you want to lead this, and the votes are generally in favor, go ahead. Be prepared to do this *again* before the 3.3 release when switching to the next VS release (and yes, Microsoft's timing had been so unfortunate in the past that such a switch would have occurred just before the first beta release of Python).

...

If it's not feasible to release 3.3 from a 2010 build, when might we be able to make the change?

If we don't switch for 3.3, we'll definitely switch to VS 2012.

...

I'm willing to do the work on this, but I just want to make sure it's a worthwhile effort.

See above: it may or may not. Regards, Martin

Antoine Pitrou

3:51 a.m.

On Mon, 04 Apr 2011 23:40:33 +0200 "Martin v. Löwis" wrote:

...

- users have expressed concerns that they constantly need to upgrade VS releases when developing for Python.

Isn't that kind of a misguided argument? It's not Python who decides the lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the churn. If getting old (Microsoft-unsupported) MSVC releases is difficult, then I think switching to the newest MSVC as soon as possible is the best strategy, since it minimizes the annoyance for people wanting to build extensions several years after a release is made. Regards Antoine.

Michael Foord

4:13 a.m.

On 04/04/2011 23:21, Antoine Pitrou wrote:

...

...
- users have expressed concerns that they constantly need to upgrade VS releases when developing for Python. Isn't that kind of a misguided argument? It's not Python who decides the

On Mon, 04 Apr 2011 23:40:33 +0200 "Martin v. Löwis" wrote: lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the churn.

If getting old (Microsoft-unsupported) MSVC releases is difficult, then I think switching to the newest MSVC as soon as possible is the best strategy, since it minimizes the annoyance for people wanting to build extensions several years after a release is made.

Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python - and every time we switch version of Visual Studio it is usually accompanied by a switch in MSVCRT version. So for C extensions compiled with a specific version of Python will need to be recompiled for later versions of Python, even if they only use the stable ABI, if the newer version of Python is compiled against a different MSVCRT. (?) This would seem to circumvent one of the core use-cases of the stable ABI which was not needing to recompile extensions for new versions of Python. Of course I could be completely wrong about all this. All the best, Michael

...

Regards

Antoine.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...

-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

Scott Dial

4:42 a.m.

On 4/4/2011 6:43 PM, Michael Foord wrote:

...

Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python - and every time we switch version of Visual Studio it is usually accompanied by a switch in MSVCRT version.

My understanding (but I haven't looked closely) was that the stable ABI specifically excluded anything that would expose a problem due to a CRT mismatch -- making this a moot point. I'm sure Martin will correct me if I am wrong. -Scott -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu

Michael Foord

5:18 a.m.

On 05/04/2011 00:12, Scott Dial wrote:

...

On 4/4/2011 6:43 PM, Michael Foord wrote:

...
Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python - and every time we switch version of Visual Studio it is usually accompanied by a switch in MSVCRT version. My understanding (but I haven't looked closely) was that the stable ABI specifically excluded anything that would expose a problem due to a CRT mismatch -- making this a moot point. I'm sure Martin will correct me if I am wrong.

Ah, wouldn't surprise me at all to know he'd already thought of that. :-) Michael

...

-Scott

"Martin v. Löwis"

3:28 p.m.

...

Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python

Not if they use the stable ABI. There still might be issues if you mix CRTs, but none related to the Python ABI - in particular, none of those crashing conditions can arise from the stable ABI.

...

This would seem to circumvent one of the core use-cases of the stable ABI which was not needing to recompile extensions for new versions of Python. Of course I could be completely wrong about all this.

Not completely, but slightly (I hope). Regards, Martin

exarkun＠twistedmatrix.com

6:22 p.m.

On 09:58 am, martin@v.loewis.de wrote:

...

...
Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python

Not if they use the stable ABI. There still might be issues if you mix CRTs, but none related to the Python ABI - in particular, none of those crashing conditions can arise from the stable ABI.

Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously? That would be great. Jean-Paul

Glyph Lefkowitz

6 Apr 6 Apr

1:04 a.m.

On Apr 5, 2011, at 8:52 AM, exarkun@twistedmatrix.com wrote:

...

On 09:58 am, martin@v.loewis.de wrote:

...
...
Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python

Not if they use the stable ABI. There still might be issues if you mix CRTs, but none related to the Python ABI - in particular, none of those crashing conditions can arise from the stable ABI.

Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously? That would be great.

That *would* be great. But is it possible? http://www.python.org/dev/peps/pep-0384/ says "functions expecting FILE* are not part of the ABI, to avoid depending on a specific version of the Microsoft C runtime DLL on Windows". Can extension modules that need to read and write files practically avoid all of those functions? (If your extension module links a library with a different CRT, but doesn't pass functions back and forth to Python, is that OK?) The PEP also says that it will allow users to "check whether their modules conform to the ABI", but it doesn't say how that will be done. How can we build extension modules so that we're sure we're ABI-conformant?

"Martin v. Löwis"

1:32 a.m.

...

http://www.python.org/dev/peps/pep-0384/ says "functions expecting FILE* are not part of the ABI, to avoid depending on a specific version of the Microsoft C runtime DLL on Windows". Can extension modules that need to read and write files practically avoid all of those functions?

Certainly! fread/fwrite/fprintf is not part of the Python API at all, so clearly doesn't need to be part of the ABI.

...

(If your extension module links a library with a different CRT, but doesn't pass functions back and forth to Python, is that OK?)

It is (and always was). The difficult functions are PyRun_AnyFileFlags and friends.

...

The PEP also says that it will allow users to "check whether their modules conform to the ABI", but it doesn't say how that will be done. How can we build extension modules so that we're sure we're ABI-conformant?

If it compiles with Py_LIMITED_API defined, and links successfully on Windows, it should be ABI-conforming (unless you deliberately bypass the test, e.g. by replicating struct definitions in your own code). Regards, Martin

"Martin v. Löwis"

1:28 a.m.

...

Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously?

No, it doesn't. distutils was considered frozen, and changes to it to better support the ABI where rejected. Regards, Martin

exarkun＠twistedmatrix.com

7:09 a.m.

On 5 Apr, 07:58 pm, martin@v.loewis.de wrote:

...

...
Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously?

No, it doesn't. distutils was considered frozen, and changes to it to better support the ABI where rejected.

How about distutils2 then? Jean-Paul

Éric Araujo

10:03 p.m.

Le 06/04/2011 03:39, exarkun@twistedmatrix.com a écrit :

...

On 5 Apr, 07:58 pm, martin@v.loewis.de wrote:

...
...
Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously?

No, it doesn't. distutils was considered frozen, and changes to it to better support the ABI where rejected.

How about distutils2 then?

If there isn’t already an open bug about that, it would be welcome. Regards

"Martin v. Löwis"

7 Apr 7 Apr

12:06 a.m.

Am 06.04.2011 03:39, schrieb exarkun@twistedmatrix.com:

...

On 5 Apr, 07:58 pm, martin@v.loewis.de wrote:

...
...
Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously?

No, it doesn't. distutils was considered frozen, and changes to it to better support the ABI where rejected.

How about distutils2 then?

That certainly will be changed to support the ABI better. Regards, Martin

Greg Ewing

6 Apr 6 Apr

2:13 a.m.

Martin v. Löwis wrote:

...

Not if they use the stable ABI. There still might be issues if you mix CRTs, but none related to the Python ABI - in particular, none of those crashing conditions can arise from the stable ABI.

Won't there still be a problem of your extension module being linked with a CRT that may not be present on the target system? -- Greg

"Martin v. Löwis"

3:01 a.m.

Am 05.04.2011 22:43, schrieb Greg Ewing:

...

Martin v. Löwis wrote:

...
Not if they use the stable ABI. There still might be issues if you mix CRTs, but none related to the Python ABI - in particular, none of those crashing conditions can arise from the stable ABI.

Won't there still be a problem of your extension module being linked with a CRT that may not be present on the target system?

Certainly. Anybody packaging an extension module needs to make sure all libraries it uses are either already on the target system, or delivered along with the extension module. Developers could refer users to the redist package, or they could literally include the CRT with their package (which is easier with VS2010 than it was with VS2008). Regards, Martin

"Martin v. Löwis"

5 Apr 5 Apr

3:25 p.m.

Am 05.04.2011 00:21, schrieb Antoine Pitrou:

...

On Mon, 04 Apr 2011 23:40:33 +0200 "Martin v. Löwis" wrote:

...
- users have expressed concerns that they constantly need to upgrade VS releases when developing for Python.

Isn't that kind of a misguided argument? It's not Python who decides the lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the churn.

But we *can* be blamed for closely following the MS release cycle (if we actually did that). For Python 3.2, we resisted.

...

If getting old (Microsoft-unsupported) MSVC releases is difficult, then I think switching to the newest MSVC as soon as possible is the best strategy, since it minimizes the annoyance for people wanting to build extensions several years after a release is made.

OTOH, the very same people will have to buy licenses for all MSVC releases. If we manage to skip some of them, the zoo of products you need to install to support Python gets smaller. Of course, if you use the stable ABI, going forward, you can decouple from Python's product management. Regards, Martin

exarkun＠twistedmatrix.com

6:18 p.m.

On 09:55 am, martin@v.loewis.de wrote:

...

Am 05.04.2011 00:21, schrieb Antoine Pitrou:

...
On Mon, 04 Apr 2011 23:40:33 +0200 "Martin v. L�wis" wrote:

...
- users have expressed concerns that they constantly need to upgrade VS releases when developing for Python.

Isn't that kind of a misguided argument? It's not Python who decides the lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the churn.

But we *can* be blamed for closely following the MS release cycle (if we actually did that). For Python 3.2, we resisted.

...
If getting old (Microsoft-unsupported) MSVC releases is difficult, then I think switching to the newest MSVC as soon as possible is the best strategy, since it minimizes the annoyance for people wanting to build extensions several years after a release is made.

OTOH, the very same people will have to buy licenses for all MSVC releases. If we manage to skip some of them, the zoo of products you need to install to support Python gets smaller.

Recent Visual Studio Express editions are available as free downloads. Jean-Paul

Brian Curtin

6:39 p.m.

On Tue, Apr 5, 2011 at 07:48, wrote:

...

On 09:55 am, martin@v.loewis.de wrote:

...
Am 05.04.2011 00:21, schrieb Antoine Pitrou:

...
On Mon, 04 Apr 2011 23:40:33 +0200 "Martin v. Löwis" wrote:

...
- users have expressed concerns that they constantly need to upgrade VS releases when developing for Python.

Isn't that kind of a misguided argument? It's not Python who decides the lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the churn.

But we *can* be blamed for closely following the MS release cycle (if we actually did that). For Python 3.2, we resisted.

...
If getting old (Microsoft-unsupported) MSVC releases is difficult, then I think switching to the newest MSVC as soon as possible is the best strategy, since it minimizes the annoyance for people wanting to build extensions several years after a release is made.

OTOH, the very same people will have to buy licenses for all MSVC releases. If we manage to skip some of them, the zoo of products you need to install to support Python gets smaller.

Recent Visual Studio Express editions are available as free downloads.

Jean-Paul

On top of that, since you and others have asked on IRC: Visual Studio 2010 Express supports x64 compilation if you have the Windows SDK installed alongside VS2010. No more "support" via registry and config file hacking. http://msdn.microsoft.com/en-us/library/9yb4317s.aspx

4763

Age (days ago)

4765

Last active (days ago)

List overview

Download

18 comments

9 participants

participants (9)

"Martin v. Löwis"
Antoine Pitrou
Brian Curtin
exarkun＠twistedmatrix.com
Glyph Lefkowitz
Greg Ewing
Michael Foord
Scott Dial
Éric Araujo

Supporting Visual Studio 2010

Scott Dial

tags

participants (9)