-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On behalf of the Python development team, I'm happy to announce the availability of Python 3.0.1, the first bug fix release of Python 3.0. Version 3.0.1 fixes dozens of bugs reported since the release of Python 3.0 on December 3rd, 2008.
Python 3.0 represents a major milestone in Python's history. This new version of the language is incompatible with the 2.x line of releases, while remaining true to BDFL Guido van Rossum's vision.
For more information, links to documentation, and downloadable distributions, see the Python 3.0.1 release page:
http://www.python.org/download/releases/3.0.1/
To report bugs in Python 3.0.1, please submit them to the issue tracker at:
Enjoy! Barry
On Fri, Feb 13, 2009 at 9:15 PM, Barry Warsaw barry@python.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On behalf of the Python development team, I'm happy to announce the availability of Python 3.0.1, the first bug fix release of Python 3.0. Version 3.0.1 fixes dozens of bugs reported since the release of Python 3.0 on December 3rd, 2008.
Python 3.0 represents a major milestone in Python's history. This new version of the language is incompatible with the 2.x line of releases, while remaining true to BDFL Guido van Rossum's vision.
For more information, links to documentation, and downloadable distributions, see the Python 3.0.1 release page:
http://www.python.org/download/releases/3.0.1/
To report bugs in Python 3.0.1, please submit them to the issue tracker at:
Enjoy! Barry
Any chance of getting a Mac installer for this one?
Speaking of an OS X installer for 3.0.1, over the last few weeks I have been working on tidying up the OS X installer build process. While the basic OS X build/installer process is good, some cruft has accumulated over the past years and a number of mostly minor issues arose due to the 3.x split. IMO, the most important issues were with IDLE and, thanks to Ronald, we did get the most important fixes for OS X IDLE checked-in in time for 3.0.1; they are also in py3k and will be going into trunk and 26. I have a few other fixes that apply just to the OSX build/installer parts which did not get submitted in time for the 3.0.1 cutoff but which are ready to go for 3.x and 2.x. Basically they fix some version number updating and ensure that the installer image will be built reproducibly in a clean environment so there is no contamination of the installer images. Currently, that's easy to do as happened with the first round of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk). They also allow images to build for various OS X version. More on that below.
I would like to get to the point where the OS X images can be generated and tested automatically. I think that is reasonable and achievable. It's not quite there yet but I can now semi-automatically produce images for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional "fat" (32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit and 64-bit intel/ppc). They all have been passing the standard regrtest with no new obvious (to me) regressions but I certainly won't claim to have done complete and thorough testing. (In particular, I have no access to a G5 for 64-bit PPC testing.)
However the "official" OS X images are built, there is an important issue about them that should be addressed now. That issue is which OS X versions should be supported by installer images. (This may well have been discussed before so my apologies if I am covering old ground here.)
Some background: OS X 10.4 (Tiger) introduced 2-way ("fat") executables and libraries which include object code for both ppc and i386 in the same files. OS X 10.5 (Leopard) extended that support to 4-way ("universal"), adding 64-bit x86_64 and ppc64. In 10.5, much, but not all, of OS X and a growing number of applications are shipped as 4-way universal. The OS X developer tools include SDKs to allow cross-builds for various OS X levels and architectures. There is a build-time environ variable, MACOSX_DEPLOYMENT_TARGET, which specifies the minimum OSX releases that an executable will work with.
Starting in 2006, the 2.5.x series of Python OS X installers were released as "fat" but built with MACOSX_DEPLOYMENT_TARGET=10.3 so that the same installer image would work on 10.3 PPC and 10.4 PPC and intel. Unfortunately, along the way, there was a significant bug in distutils.util.get_platform not returning "fat" in the platform string; among other things this has had impact on packages on PyPI. Ronald fixed the problem prior to 2.5.2 (?) but, for compatibility, the fix is only effective if MACOSX_DEPLOYMENT_TARGET >= 10.4 and the 2.5.x installers continued to be built as 10.3. Also unfortunately, the deployment target for the first 2.6 OS X installer was not updated. In the meantime, Ronald added support for 64-bit and 4-way universal builds for 10.5 but, to date, no official installer images have been released with that enabled although clearly people are already building their own 64-bit OS X pythons.
The last Apple point release of 10.3 was in 4/2005. 10.4 was also released then. The last Apple maintenance release of 10.4 was in 11/2007. 10.5 was released in 10/2007 and, with subsequent update releases, remains the current system. While no release dates have been announced, 10.6 (Snow Leopard) is rumored to be nearing release. If Apple follows past practices, they will likely terminate all support for 10.4 (release n-2) when 10.6 releases and will continue to support 10.5 (n-1) for the lifetime of 10.6. Needless to say, Apple stopped supporting 10.3 a long time ago and, if 10.6 does release in the not too-distant future, 10.4's days are numbered.
What does this mean for Python? To date, there have been no official 3.0 OS X installers released. Because of the deficiencies of building for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit support, and probably other things I'm not aware of), I think it is very important that the 3.0.1 get off on the right foot by changing the minimum supported versions now.
I see three plausible options:
1. Release an installer built for 10.5 and higher. pros: delivers 32-support and 64-support; cons: prematurely disenfranchises 10.4 users
2. Release an installer built for 10.4 and higher. pros: one size fits all cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
3. Release two installers, one each for 10.4+ and 10.5+. pros: supports current and future systems; delivers 64-support to 10.5+ users; could choose to drop 10.4 installers anytime after 10.6 releases; cons: some extra work to build/release (but not much and not often); others??
Note, this is only talking about the python.org installer images, not Python itself. Users of older systems (10.3 and earlier) would still be able to build their own Pythons or even installers. However, there could (should?) be an explicit statement made as to what OS X versions are "officially" supported.
With the improvements to the build process and the experience I've had so far, I would strongly recommend option 3. I am willing to commit to supplying those images for 3.0.1 and beyond until we can get to a more automated process. Or I'd be happy to work with someone else to make it happen for 3.0.1. Now that the tarball is out, I will be able to get the images built and tested this weekend.
BTW, over on the pythonmac forum, there has been discussion of having some Mac activities at Pycon. Ronald is planning to be there and I'm hoping to, as well, so that could be a great opportunity to discuss and work on futures. And this same discussion and decision needs to be made going forward for 2.7 and 2.6.x (I think the change should be made for 2.6.2).
So, to summarize, I will build and test installer images for 3.0.1 built for 10.4+ and for 10.5+ and will make them available, if there is interest. If someone else is planning on making official installer(s), I'd be happy to pass along my experiences so far. In any case, I strongly urge that the minimum support level be 10.4.
--Ned
Ned Deily writes:
I see three plausible options:
- Release an installer built for 10.5 and higher. pros: delivers 32-support and 64-support; cons: prematurely disenfranchises 10.4 users
+0 This would bother me; I have a couple of older Macs that run 10.4. But it's acceptable (I can always use MacPorts or build from source). I would suppose most folks who are running 10.4 even today are "cranks like me, baby, we were born to fuss!"<wink> Ahem, anyway, I suspect people who care that much about stability are generally old-school types who are willing to roll their own to some extent.
- Release an installer built for 10.4 and higher. pros: one size fits all cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
-1 I think if you're going to have a break, make it a big one. By the time people who use installers are using Python 3.0 a lot, most of them will be on 10.6.
- Release two installers, one each for 10.4+ and 10.5+. pros: supports current and future systems; delivers 64-support to 10.5+ users; could choose to drop 10.4 installers anytime after 10.6 releases; cons: some extra work to build/release (but not much and not often);
+1
work on futures. And this same discussion and decision needs to be made going forward for 2.7 and 2.6.x (I think the change should be made for 2.6.2).
Generally my inclination is the same here, but the distinction between options 2 and 3 is a little less extreme.
On Sat, Feb 14, 2009 at 3:54 AM, Stephen J. Turnbull stephen@xemacs.org wrote:
I would suppose most folks who are running 10.4 even today are "cranks like me, baby, we were born to fuss!"<wink> Ahem, anyway, I suspect people who care that much about stability are generally old-school types who are willing to roll their own to some extent.
Actually I expect that to be fairly common among people who are not so much into technology, strapped for funds but appreciative of quality, bought an expensive Mac once expecting it would last a long time, and are hanging on to it until it dies (which could be a long time). Their hardware may not support newer OS X versions.
Guido van Rossum writes:
Actually I expect that to be fairly common among people who are not so much into technology, strapped for funds but appreciative of quality, bought an expensive Mac once expecting it would last a long time, and are hanging on to it until it dies (which could be a long time). Their hardware may not support newer OS X versions.
I'm quite familiar with that. Thing is, although I still have a Panther box running, I gave up on updating *anything* on it when Leopard came out, because I had to spend too much time fighting with bitrot in packages which stopped working after upgrades on my Tiger box, let alone on Panther. The Panther box still does the stuff that I want it to do, but it isn't going to learn any new tricks, I'm afraid.
If that is a more general phenomenon, I have to wonder whether people with old Macs with old OSes are going to be all that interested in shiny new Pythons.
I think that the extra effort of building an extra installer for 10.4 will be well-repaid in better support for 10.5 users, and the option of abandoning the 10.4 installer in a few months if it doesn't get very many downloads.
In article 878wo8swxe.fsf@xemacs.org, "Stephen J. Turnbull" stephen@xemacs.org wrote:
Guido van Rossum writes:
Actually I expect that to be fairly common among people who are not so much into technology, strapped for funds but appreciative of quality, bought an expensive Mac once expecting it would last a long time, and are hanging on to it until it dies (which could be a long time). Their hardware may not support newer OS X versions.
I'm quite familiar with that. Thing is, although I still have a Panther box running, I gave up on updating *anything* on it when Leopard came out, because I had to spend too much time fighting with bitrot in packages which stopped working after upgrades on my Tiger box, let alone on Panther. The Panther box still does the stuff that I want it to do, but it isn't going to learn any new tricks, I'm afraid.
If that is a more general phenomenon, I have to wonder whether people with old Macs with old OSes are going to be all that interested in shiny new Pythons.
I think that the extra effort of building an extra installer for 10.4 will be well-repaid in better support for 10.5 users, and the option of abandoning the 10.4 installer in a few months if it doesn't get very many downloads.
You mentioned MacPorts earlier. FWIW, here's their position on version support:
"We provide a single software tree that attempts to track the latest release of every software title (port) we distribute, without splitting them into "stable" Vs. "unstable" branches, targetting mainly the current Mac OS X release (10.5, A.K.A Leopard) and the immediately previous one (10.4, A.K.A. Tiger)."
And Fink, the other major open-source package provider for OS X, is even more explicit:
"The Fink project is no longer able to offer support for users of Fink on Mac OS X 10.3. In reality there has been very little support for some time, so this announcement simply formalizes that fact. This means that there will not be any further updates, not even security updates, for Mac OS X 10.3 users. Their existing installations will continue to work, but the software that is installed will not be updated. We believe that the majority of our users are using Mac OS X 10.4 or 10.5 and hope that this does not inconvenience too many people."
(This may well have been discussed before so my apologies if I am covering old ground here.)
There might have been discussions on pythonmac lists, but no recent ones on python-dev, AFAIR.
The last Apple point release of 10.3 was in 4/2005. 10.4 was also released then. [...] Needless to say, Apple stopped supporting 10.3 a long time ago and, if 10.6 does release in the not too-distant future, 10.4's days are numbered.
I think this is a matter of personal judgment. If there has been a point release as recent as three years ago, I wouldn't call that "long ago". People often continue to use operating systems 10 years after the vendor stopped supporting it.
So I would personally think that it is desirable to continue supporting 10.3 if possible. For comparison, the Python Windows binaries support systems back to Windows 2000, but not anymore the non-NT systems (ie. Windows 95, Windows 98, Windows ME).
- Release an installer built for 10.5 and higher. pros: delivers 32-support and 64-support; cons: prematurely disenfranchises 10.4 users
I don't think this should be done.
- Release an installer built for 10.4 and higher. pros: one size fits all cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
Why is it that such an installer couldn't include 64-bit support? Wouldn't 10.4 just ignore architectures it doesn't know about? Also, didn't 10.4 already support ppc64?
- Release two installers, one each for 10.4+ and 10.5+. pros: supports current and future systems; delivers 64-support to 10.5+ users; could choose to drop 10.4 installers anytime after 10.6 releases; cons: some extra work to build/release (but not much and not often); others??
I don't think this should be done, either. The whole point of "universal" binaries is that it simplifies choice for end users.
With the improvements to the build process and the experience I've had so far, I would strongly recommend option 3. I am willing to commit to supplying those images for 3.0.1 and beyond until we can get to a more automated process. Or I'd be happy to work with someone else to make it happen for 3.0.1. Now that the tarball is out, I will be able to get the images built and tested this weekend.
Ultimately, this is for the port maintainer to decide (IMO), which is still Ronald Oussouren. If he passes this role on to you, it would be all fine with me, and you could do with the port whatever you please (even though I might still dislike the additional clutter of two OSX installers)
So, to summarize, I will build and test installer images for 3.0.1 built for 10.4+ and for 10.5+ and will make them available, if there is interest. If someone else is planning on making official installer(s), I'd be happy to pass along my experiences so far. In any case, I strongly urge that the minimum support level be 10.4.
I still wish there were 10.3+ installers that also include 64-bit code. I don't get it why that can't be technically possible.
Regards, Martin
On 14 Feb, 2009, at 13:05, Martin v. Löwis wrote:
- Release an installer built for 10.4 and higher.
pros: one size fits all cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
Why is it that such an installer couldn't include 64-bit support? Wouldn't 10.4 just ignore architectures it doesn't know about? Also, didn't 10.4 already support ppc64?
10.4 only supported 64-bit code for a very limited number of system libraries, basicly just enough to support running 64-bit number crunching code. System-wide support for 64-bit code was added in 10.5.
A single installer could support both 32-bit on 10.4 and 64-bit on 10.5, but I don't think that's very useful because there are changes in the low-level unix API's that could result in different behaviour of a 32-bit and 64-bit script on the same system. In general 10.5 has much saner Unix API's than earlier releases.
So, to summarize, I will build and test installer images for 3.0.1 built for 10.4+ and for 10.5+ and will make them available, if there is interest. If someone else is planning on making official installer(s), I'd be happy to pass along my experiences so far. In any case, I strongly urge that the minimum support level be 10.4.
I still wish there were 10.3+ installers that also include 64-bit code. I don't get it why that can't be technically possible.
The problem with 10.3 support is that we need volunteers to actually investigate and fix issues that only occur on 10.3 systems. I cannot be that volunteer because I no longer have access to systems that are capable of running 10.3.
That said, the difference between a binary capable of running on 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for a number of symbols that are not present on 10.3.9 in the 2.5 timeframe and that could should continue to work in the future. I won't notice when someone introduces additional calls to functions not available on 10.3 though.
Ronald
A single installer could support both 32-bit on 10.4 and 64-bit on 10.5, but I don't think that's very useful because there are changes in the low-level unix API's that could result in different behaviour of a 32-bit and 64-bit script on the same system. In general 10.5 has much saner Unix API's than earlier releases.
I don't get that. Why would the scripts behave differently on 10.5 depending on whether the Python interpreter is 32-bit or 64-bit? Surely, the Unix API does the same thing, whether invoked from 32-bit code, or 64-bit code, no?
I still wish there were 10.3+ installers that also include 64-bit code. I don't get it why that can't be technically possible.
The problem with 10.3 support is that we need volunteers to actually investigate and fix issues that only occur on 10.3 systems. I cannot be that volunteer because I no longer have access to systems that are capable of running 10.3.
I don't think it is necessary to actually test whether the binaries work on 10.3; I don't test the Windows installers on Windows 2000, either. For me, it's good enough if we believe that the installer "should" work on 10.3.
Then, if somebody reports a problem, we can still consider what to do. If there are no reports, it either means there are no problems, or nobody uses it, or nobody bothers reporting the problems.
That said, the difference between a binary capable of running on 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for a number of symbols that are not present on 10.3.9 in the 2.5 timeframe and that could should continue to work in the future. I won't notice when someone introduces additional calls to functions not available on 10.3 though.
Sounds good to me!
Regards, Martin
In article 499707A0.7000200@v.loewis.de, "Martin v. Lowis" martin@v.loewis.de wrote:
That said, the difference between a binary capable of running on 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for a number of symbols that are not present on 10.3.9 in the 2.5 timeframe and that could should continue to work in the future. I won't notice when someone introduces additional calls to functions not available on 10.3 though.
Sounds good to me!
That's fine as long as the distutils issue is resolved. I believe the way things stand today is that a "fat" Python built with a deployment target of 10.3 will report a platform of "ppc" or "i386" even on 10.4 or 10.5 systems and even though the extensions are, in fact, "fat". This means developers who provide uploads to PyPI of packages with C extensions have to upload two versions even though the contents of both can be identical. (The appscript has run into this problem.) And it causes maintenance issues for users with multiple architectures.
With the target set to 10.4, the platform is reported correctly as "fat". And, for 10.5 4-way, "universal".
On 14 Feb, 2009, at 19:44, Ned Deily wrote:
In article 499707A0.7000200@v.loewis.de, "Martin v. Lowis" martin@v.loewis.de wrote:
That said, the difference between a binary capable of running on 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for a number of symbols that are not present on 10.3.9 in the 2.5 timeframe and that could should continue to work in the future. I won't notice when someone introduces additional calls to functions not available on 10.3 though.
Sounds good to me!
That's fine as long as the distutils issue is resolved. I believe the way things stand today is that a "fat" Python built with a deployment target of 10.3 will report a platform of "ppc" or "i386" even on 10.4 or 10.5 systems and even though the extensions are, in fact, "fat". This means developers who provide uploads to PyPI of packages with C extensions have to upload two versions even though the contents of both can be identical. (The appscript has run into this problem.) And it causes maintenance issues for users with multiple architectures.
With the target set to 10.4, the platform is reported correctly as "fat". And, for 10.5 4-way, "universal".
Please file an issue for this at the python tracker. The intended behaviour is that the reported platform is the platform that distutils can build for on the current machine (ppc only on 10.3, "fat" on 10.4 and "universal" on 10.5, given a 4-way build of python).
Getting the experience entire correct probably requires changes to other tools as well (such as setuptools, it should be taught that a "universal" egg can satisify a dependency when the current platform is "fat" or "ppc".
Issues like this is why I've asked on the pythonmac-sig if anyone is interested on sprinting on macpython at Pycon. Getting these things correct takes time and is much easier when several people can work on it at the same location.
Ronald
That's fine as long as the distutils issue is resolved.
I don't think this should be a prerequisite. As Ronald says: no fix without a bug report; if the system is capable of building the extension correctly, it should do so (so it's a bug and fixes can be backported to 2.6)
Regards, Martin
In article 499723CD.80404@v.loewis.de, "Martin v. Lowis" martin@v.loewis.de wrote:
That's fine as long as the distutils issue is resolved.
I don't think this should be a prerequisite. As Ronald says: no fix without a bug report; if the system is capable of building the extension correctly, it should do so (so it's a bug and fixes can be backported to 2.6)
And it turns out it should not be a problem for 3.x (and future releases of 2.x). Ronald fixed it several weeks ago here: http://bugs.python.org/issue4064
Python 3.0.1 (r301:69597, Feb 14 2009, 19:03:52) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information.
from distutils.util import get_platform; print(get_platform())
macosx-10.3-fat
My apologies to Ronald and the cohort for not re-examining this recently. The distutils "problem" isn't anymore.
On 14 Feb, 2009, at 19:04, Martin v. Löwis wrote:
A single installer could support both 32-bit on 10.4 and 64-bit on 10.5, but I don't think that's very useful because there are changes in the low-level unix API's that could result in different behaviour of a 32-bit and 64-bit script on the same system. In general 10.5 has much saner Unix API's than earlier releases.
I don't get that. Why would the scripts behave differently on 10.5 depending on whether the Python interpreter is 32-bit or 64-bit? Surely, the Unix API does the same thing, whether invoked from 32-bit code, or 64-bit code, no?
I should have been more clear: the unix API for code that runs on 10.4 is slightly different than that for code that runs on 10.5+, Apple basiclly fixed a number of UNIX API-compliance issues in 10.5.
Those differences might affect the behaviour of the Python interpreter, and at least would make the configure stage even more involved for such a build, because several configure-checks give different output for MACOSX_DEPLOYMENT_TARGET=10.4 and MACOSX_DEPLOYMENT_TARGET=10.5.
I haven't checked yet if it is even possible to have a 64-bit binary that runs on 10.4 and 10.5.
I still wish there were 10.3+ installers that also include 64-bit code. I don't get it why that can't be technically possible.
The problem with 10.3 support is that we need volunteers to actually investigate and fix issues that only occur on 10.3 systems. I cannot be that volunteer because I no longer have access to systems that are capable of running 10.3.
I don't think it is necessary to actually test whether the binaries work on 10.3; I don't test the Windows installers on Windows 2000, either. For me, it's good enough if we believe that the installer "should" work on 10.3.
Then, if somebody reports a problem, we can still consider what to do. If there are no reports, it either means there are no problems, or nobody uses it, or nobody bothers reporting the problems.
I'm not 100% happy with this, but I could live with this. As long as Apple ships a 10.3 SDK I might even be able to check for unsupported API calls once in while.
Having thought a little more about the issues, I do think that we should provide an installer that runs on as many systems as possible. As Guido noted many people don't bother to upgrade, and I'd expect many schools to be that category as well (as an example of a type of organisation that's often lacking in funding).
I need more time to think about the options, and I don't think there's a reason to rush this and would prefer to discuss this issue at Pycon.
Ronald
Ronald Oussoren wrote:
On 14 Feb, 2009, at 19:04, Martin v. Löwis wrote:
A single installer could support both 32-bit on 10.4 and 64-bit on 10.5, but I don't think that's very useful because there are changes in the low-level unix API's that could result in different behaviour of a 32-bit and 64-bit script on the same system. In general 10.5 has much saner Unix API's than earlier releases.
I don't get that. Why would the scripts behave differently on 10.5 depending on whether the Python interpreter is 32-bit or 64-bit? Surely, the Unix API does the same thing, whether invoked from 32-bit code, or 64-bit code, no?
I should have been more clear: the unix API for code that runs on 10.4 is slightly different than that for code that runs on 10.5+, Apple basiclly fixed a number of UNIX API-compliance issues in 10.5.
So how come? Are you really saying that 10.5 somehow knows whether the code might also run on 10.4, and if so, deliberately behaves differently?
So that a Python interpreter compiled for 10.4+ would behave differently on a specific 10.5 installation than a Python interpreter compiled for 10.5+, on the very same system?
If so, I'm -1 on providing two sets of binaries for the same Python version (one in 10.4+ mode and one in 10.5+ mode). Otherwise, this sounds like a good source of confusion: "are you using Python 3.1 as compiled for 10.4 or 3.1 as compiled for 10.5?" - "well, how do I tell?"
Is this Apple's DLL hell?
Regards, Martin
On 14 Feb, 2009, at 20:15, Martin v. Löwis wrote:
Ronald Oussoren wrote:
On 14 Feb, 2009, at 19:04, Martin v. Löwis wrote:
A single installer could support both 32-bit on 10.4 and 64-bit on 10.5, but I don't think that's very useful because there are changes in the low-level unix API's that could result in different behaviour of a 32-bit and 64-bit script on the same system. In general 10.5 has much saner Unix API's than earlier releases.
I don't get that. Why would the scripts behave differently on 10.5 depending on whether the Python interpreter is 32-bit or 64-bit? Surely, the Unix API does the same thing, whether invoked from 32- bit code, or 64-bit code, no?
I should have been more clear: the unix API for code that runs on 10.4 is slightly different than that for code that runs on 10.5+, Apple basiclly fixed a number of UNIX API-compliance issues in 10.5.
So how come? Are you really saying that 10.5 somehow knows whether the code might also run on 10.4, and if so, deliberately behaves differently?
Yes. OSX supports a variant of symbol versioning. The dynamic linker can see for which system a binary was compiler build and links the right version of a symbol at runtime.
There's the comment in the header files that defines the macro's used for symbol versioning:
/* * The __DARWIN_ALIAS macros are used to do symbol renaming; they allow * legacy code to use the old symbol, thus maintiang binary compatability * while new code can use a standards compliant version of the same function. * * __DARWIN_ALIAS is used by itself if the function signature has not * changed, it is used along with a #ifdef check for __DARWIN_UNIX03 * if the signature has changed. Because the __LP64__ enviroment * only supports UNIX03 sementics it causes __DARWIN_UNIX03 to be * defined, but causes __DARWIN_ALIAS to do no symbol mangling. * * As a special case, when XCode is used to target a specific version of the * OS, the manifest constant __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ * will be defined by the compiler, with the digits representing major version * time 100 + minor version times 10 (e.g. 10.5 := 1050). If we are targetting * pre-10.5, and it is the default compilation environment, revert the * compilation environment to pre-__DARWIN_UNIX03. */
Simply put: when a binary is compiled to run on 32-bit 10.4 or earlier it will be linked to pre-Unix03 symbols, otherwise it will be linked with the Unix'03 compliant symbols/APIs. Some API's just behave differently, others have a different signature as well.
So that a Python interpreter compiled for 10.4+ would behave differently on a specific 10.5 installation than a Python interpreter compiled for 10.5+, on the very same system?
If so, I'm -1 on providing two sets of binaries for the same Python version (one in 10.4+ mode and one in 10.5+ mode). Otherwise, this sounds like a good source of confusion: "are you using Python 3.1 as compiled for 10.4 or 3.1 as compiled for 10.5?" - "well, how do I tell?"
It should be easy to provide that information.
And anyway, we already have confusion about python builds anyway: the python.org, macports and fink builds of python are slightly different and some issues seem to crop up with some of the builds more often than with others. That is, I sometimes see questions on the pythonmac- sig from macports and fink users about issues that I've never run into with the python.org build.
Is this Apple's DLL hell?
Not quite a hell, but still inconvenient. I guess it is time to start digging to check which API's have a different definition in UNIX03 to check if those could affect the behaviour of the Python interpreter. I know the value of "GETPGRP_HAVE_ARG" is affected by this, but Python scripts are shielded from that because the interface of the posix module is the same regardless of the value of this macro.
We'd still end up with a build-hell when we'd want to provide a 4-way universal build that can run on 10.4 as well, because 64-bit code is always in UNIX03 mode, while 32-bit code that needs to run on 10.4 cannot be. I have some idea's about how to work around that issue, but need time to investigate the viability of those ideas.
Ronald
On 14 Feb, 2009, at 12:22, Ned Deily wrote:
Speaking of an OS X installer for 3.0.1, over the last few weeks I have been working on tidying up the OS X installer build process. While the basic OS X build/installer process is good, some cruft has accumulated over the past years and a number of mostly minor issues arose due to the 3.x split. IMO, the most important issues were with IDLE and, thanks to Ronald, we did get the most important fixes for OS X IDLE checked-in in time for 3.0.1; they are also in py3k and will be going into trunk and 26. I have a few other fixes that apply just to the OSX build/ installer parts which did not get submitted in time for the 3.0.1 cutoff but which are ready to go for 3.x and 2.x. Basically they fix some version number updating and ensure that the installer image will be built reproducibly in a clean environment so there is no contamination of the installer images. Currently, that's easy to do as happened with the first round of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk). They also allow images to build for various OS X version. More on that below.
There's one problem with building with an entirely clean environment though, the 2.5 installers were build with a locally instaled Tcl/Tk on purpose. Such a build uses a locally installed Tcl/Tk when available and uses the system version when it is not. Several Tkinter users on OSX pushed heavily for that because appearently the system Tk is much less usable than the 3th-party install.
I don't care much either way, IMHO the look and feel of Tk apps sucks with either version of Tcl/Tk and I don't use Tkinter anyway.
I would like to get to the point where the OS X images can be generated and tested automatically. I think that is reasonable and achievable. It's not quite there yet but I can now semi-automatically produce images for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional "fat" (32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit and 64-bit intel/ppc). They all have been passing the standard regrtest with no new obvious (to me) regressions but I certainly won't claim to have done complete and thorough testing. (In particular, I have no access to a G5 for 64-bit PPC testing.)
I'd love to get to the point where it's possible to completely automaticly build the installers. I noticed one issue with that that I'll fix in the near future: the current build script for the OSX installer relies on the availability of the HTML docs download on the Python ftp-site. That dependency should no longer be necessary now that the documentation is build using a pure python toolchain.
However the "official" OS X images are built, there is an important issue about them that should be addressed now. That issue is which OS X versions should be supported by installer images. (This may well have been discussed before so my apologies if I am covering old ground here.)
The last Apple point release of 10.3 was in 4/2005. 10.4 was also released then. The last Apple maintenance release of 10.4 was in 11/2007. 10.5 was released in 10/2007 and, with subsequent update releases, remains the current system. While no release dates have been announced, 10.6 (Snow Leopard) is rumored to be nearing release. If Apple follows past practices, they will likely terminate all support for 10.4 (release n-2) when 10.6 releases and will continue to support 10.5 (n-1) for the lifetime of 10.6. Needless to say, Apple stopped supporting 10.3 a long time ago and, if 10.6 does release in the not too-distant future, 10.4's days are numbered.
It's not just what Apple supports though, there's also the question of what people actually use.
That said, unless anyone wants to step in to support 10.3 systems I'm in favour of completely dropping 10.3 support in the binary installers. The reason for that is twofold: first of all 10.3 is ancient, and more importantly I no longer have access to hardware that will run 10.3 and can therefore no longer test if 10.3 support actually works.
What does this mean for Python? To date, there have been no official 3.0 OS X installers released. Because of the deficiencies of building for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit support, and probably other things I'm not aware of), I think it is very important that the 3.0.1 get off on the right foot by changing the minimum supported versions now.
I see three plausible options:
- Release an installer built for 10.5 and higher.
pros: delivers 32-support and 64-support; cons: prematurely disenfranchises 10.4 users
- Release an installer built for 10.4 and higher.
pros: one size fits all cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc
- Release two installers, one each for 10.4+ and 10.5+. pros: supports current and future systems; delivers 64-support to 10.5+ users; could choose to drop 10.4 installers anytime after 10.6 releases; cons: some extra work to build/release (but not much and not often); others??
I'm in favour of providing two installers: one installer for 10.4 and higher that support 32-bit only, and one installer for 10.5 and higher that supports ppc, x86 and x86_64. Ppc64 support could be added as well, but only if someone else is willing to support that, I don't have reliable access to a ppc64 system for development. I did notice that several libraries, such as libffi (used by ctypes and pyobjc) don't work reliably on osx/ppc64.
BTW, over on the pythonmac forum, there has been discussion of having some Mac activities at Pycon. Ronald is planning to be there and I'm hoping to, as well, so that could be a great opportunity to discuss and work on futures. And this same discussion and decision needs to be made going forward for 2.7 and 2.6.x (I think the change should be made for 2.6.2).
I'll be at Pycon and part of post-pycon sprint days (I'm flying back on april 1st). I had already planned to try to get a mac-related sprint going at pycon, I've added work on this on the list of possible topics.
Ronald
In article 6FC960B6-7723-4887-9342-FAC4D0C936C3@mac.com, Ronald Oussoren ronaldoussoren@mac.com wrote:
I'll be at Pycon and part of post-pycon sprint days (I'm flying back on april 1st). I had already planned to try to get a mac-related sprint going at pycon, I've added work on this on the list of possible topics.
Sounds like there will be no lack of things to talk about at Pycon!
Thanks to everyone for their input on this. And congratulations on getting 3.0.1 - with OS X support - out there.
On Sat, Feb 14, 2009 at 3:22 AM, Ned Deily nad@acm.org wrote: ...
have done complete and thorough testing. (In particular, I have no access to a G5 for 64-bit PPC testing.)
I have a PowerMac G5 at home and I'll be glad to run tests if it helps. (It runs 10.5: "family pack" licenses are cheap, so I've always routinely upgraded all my home Macs to recent OSX releases). (I'm not home during the weekend though).
Alex
In article e8a0972d0902141101x4ec55722ud7cf9f861937aba6@mail.gmail.com, Alex Martelli aleaxit@gmail.com wrote:
On Sat, Feb 14, 2009 at 3:22 AM, Ned Deily nad@acm.org wrote: ...
have done complete and thorough testing. (In particular, I have no access to a G5 for 64-bit PPC testing.)
I have a PowerMac G5 at home and I'll be glad to run tests if it helps.
Thanks, Alex. I'll contact you off-list about this.
In article nad-34F90E.03222214022009@news.gmane.org, Ned Deily nad@acm.org wrote:
Speaking of an OS X installer for 3.0.1, over the last few weeks I have been working on tidying up the OS X installer build process. While the basic OS X build/installer process is good, some cruft has accumulated over the past years and a number of mostly minor issues arose due to the 3.x split. IMO, the most important issues were with IDLE and, thanks to Ronald, we did get the most important fixes for OS X IDLE checked-in in time for 3.0.1; they are also in py3k and will be going into trunk and 26. I have a few other fixes that apply just to the OSX build/installer parts which did not get submitted in time for the 3.0.1 cutoff but which are ready to go for 3.x and 2.x. Basically they fix some version number updating and ensure that the installer image will be built reproducibly in a clean environment so there is no contamination of the installer images. Currently, that's easy to do as happened with the first round of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk).
I want to follow up on this a bit. In the past if the Mac Python installer was built on a machine that did NOT have a locally installed Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: Python would segfault when trying to use Tkinter.
The solution was to build the Mac python installer on a machine with a locally installed Tcl/Tk. The resulting installer package would work on all systems -- with or without locally installed Tcl/Tk.
So...has this problem been worked around, or is the Mac installer still built on a machine that has a locally installed Tcl/Tk?
I haven't run Python 2.6 yet because there is no release of numpy that is compatible. I did try upgrading from 2.5.2 to 2.5.4 and got a segfault when trying to display images using PIL and Tkinter; I have not had time to try to track that down, so I've just reverted for now.
Most people who makes serious use of Tkinter presumably have a locally installed Tcl/Tk because the version that Apple provides is ancient and is missing many important bug fixes and performance enhancements.
Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All development work is going on in Tcl/Tk 8.5. Presumably Apple will transition one of these days, and at that point we may need a separate Mac Python installer for the older operating systems vs. the newer.
-- Rusell
In article rowen-8731E0.13531325022009@news.gmane.org, "Russell E. Owen" rowen@u.washington.edu wrote:
I want to follow up on this a bit. In the past if the Mac Python installer was built on a machine that did NOT have a locally installed Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: Python would segfault when trying to use Tkinter.
The solution was to build the Mac python installer on a machine with a locally installed Tcl/Tk. The resulting installer package would work on all systems -- with or without locally installed Tcl/Tk.
So...has this problem been worked around, or is the Mac installer still built on a machine that has a locally installed Tcl/Tk?
Ronald will have to answer that for sure since he built the installer for 3.0.1.
However, it seems to be true that the most recent python.org OS X installers will always favor a /System/Library/Frameworks/{Tcl/Tk}:
$ cd /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynlo ad $ otool -L _tkinter.so _tkinter.so: /System/Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0) /System/Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.10)
I'm certainly not a Tcl expert nor overly familiar with the history of this issue but here's what I see. As a test, I installed the ActiveTcl 8.4.19, which installs in /Library/Frameworks, and confirmed that a simple "import tkinter; root = tkinter.Tk()" for 3.0.1 does indeed use the Apple-supplied 8.4.7 Tcl.
Running similar tests and otool -L on _tkinter.so from the 2.6.1 and 2.5.4 python.org installs gives similar results (both always use the Apple-supplied Tcl from /System/Library).
However, the _tkinter.so from the 2.4.4 python.org installer is different: $ cd /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynlo ad $ otool -L _tkinter.so _tkinter.so: /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.13) /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.13) /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 47.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.3)
and it does in fact use the /Library 8.4.19 ActiveTcl framework if installed and, as expected, reverts to the Apple-supplied 8.4.7 /System/Library framework version if the ActiveTcl framework is removed.
So it looks like you recall correctly that at least some earlier installers were built on a machine with a locally installed Tcl/Tk. It should be an easy matter to do that for future 10.4-10.5 installers so that newer 8.4 versions can be used.
Note, there was a Tcl/Tk problem with the initial 2.6 OS X installer (http://bugs.python.org/issue4017) but in that case the locally installed Tcl/Tk version was 8.5 and the fallback to a system-supplied 8.4 doesn't work across versions.
Most people who makes serious use of Tkinter presumably have a locally installed Tcl/Tk because the version that Apple provides is ancient and is missing many important bug fixes and performance enhancements.
Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All development work is going on in Tcl/Tk 8.5. Presumably Apple will transition one of these days, and at that point we may need a separate Mac Python installer for the older operating systems vs. the newer.
Yes, something will need to be done for that if Snow Leopard does have newer versions.
Thanks for bringing this up!
On 27 Feb, 2009, at 1:57, Ned Deily wrote:
In article rowen-8731E0.13531325022009@news.gmane.org, "Russell E. Owen" rowen@u.washington.edu wrote:
I want to follow up on this a bit. In the past if the Mac Python installer was built on a machine that did NOT have a locally installed Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: Python would segfault when trying to use Tkinter.
The solution was to build the Mac python installer on a machine with a locally installed Tcl/Tk. The resulting installer package would work on all systems -- with or without locally installed Tcl/Tk.
So...has this problem been worked around, or is the Mac installer still built on a machine that has a locally installed Tcl/Tk?
Ronald will have to answer that for sure since he built the installer for 3.0.1.
However, it seems to be true that the most recent python.org OS X installers will always favor a /System/Library/Frameworks/{Tcl/Tk}:
That's correct, I don't have a locally installed Tcl/Tk on my laptop at the moment and couldn't arrange for one in time for the 3.0.1 release.
BTW. The fact that this should result in crashes when a user does have a locally installed Tcl/Tk is new to me. The reason earlier builds of the OSX installer were build with a locally installed Tcl/Tk is that several Tkinter users indicated that the system version is significantly less useful than a local install.
Most people who makes serious use of Tkinter presumably have a locally installed Tcl/Tk because the version that Apple provides is ancient and is missing many important bug fixes and performance enhancements.
Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All development work is going on in Tcl/Tk 8.5. Presumably Apple will transition one of these days, and at that point we may need a separate Mac Python installer for the older operating systems vs. the newer.
Yes, something will need to be done for that if Snow Leopard does have newer versions.
Thanks for bringing this up!
Do Python 2.6 and 3.0 support building with Tcl/Tk 8.5? I vaguely recall messages about issues on python-dev but don't know if those issues were real and/or solved.
Ronald
In article AD5BF985-3C9B-4D49-BFE6-0CFF1B57A00A@mac.com, Ronald Oussoren ronaldoussoren@mac.com wrote:
On 27 Feb, 2009, at 1:57, Ned Deily wrote:
In article rowen-8731E0.13531325022009@news.gmane.org, "Russell E. Owen" rowen@u.washington.edu wrote:
I want to follow up on this a bit. In the past if the Mac Python installer was built on a machine that did NOT have a locally installed Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: Python would segfault when trying to use Tkinter.
The solution was to build the Mac python installer on a machine with a locally installed Tcl/Tk. The resulting installer package would work on all systems -- with or without locally installed Tcl/Tk.
So...has this problem been worked around, or is the Mac installer still built on a machine that has a locally installed Tcl/Tk?
Ronald will have to answer that for sure since he built the installer for 3.0.1.
However, it seems to be true that the most recent python.org OS X installers will always favor a /System/Library/Frameworks/{Tcl/Tk}:
That's correct, I don't have a locally installed Tcl/Tk on my laptop at the moment and couldn't arrange for one in time for the 3.0.1 release.
BTW. The fact that this should result in crashes when a user does have a locally installed Tcl/Tk is new to me. The reason earlier builds of the OSX installer were build with a locally installed Tcl/Tk is that several Tkinter users indicated that the system version is significantly less useful than a local install.
Any hope of an updated Mac installer for 2.5.4 and/or 2.6 that is built with a locally installed Tcl/Tk? If/when you do that, I strongly recommend ActiveState Tcl/Tk 8.4.19 -- the last 8.4 release -- it has few bugs and very good performance.
(For Python 2.6 you could install Tcl/Tk 8.5 instead, but it might not work for folks using Apple's built-in Tcl/Tk 8.4. Maybe somebody else knows; if not it would require some testing.)
Note that even Python 2.5 built against Tcl/Tk 8.5 and most of it worked, but there were some known bugs.
-- Russell
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Feb 13, 2009, at 11:46 PM, Benjamin Kaplan wrote:
Any chance of getting a Mac installer for this one?
I believe Ronald is planning to upload it soon. Barry
In article ec96e1390902132046k75fbaa12h7c54ab0f8f346bbe@mail.gmail.com, Benjamin Kaplan benjamin.kaplan@case.edu wrote:
Any chance of getting a Mac installer for this one?
BTW, I see the a link to the OS X installer has now been added to the 3.0.1 release page: http://www.python.org/download/releases/3.0.1/
It would be great if someone could add OS X links for the 3.0.1 and 2.6.1 to the main download page, too: http://www.python.org/download/
On Sun, Feb 15, 2009 at 9:15 PM, Ned Deily nad@acm.org wrote:
It would be great if someone could add OS X links for the 3.0.1 and 2.6.1 to the main download page, too: http://www.python.org/download/
I've now added them to the main download page.
-
"Martin v. Löwis" -
Alex Martelli -
Barry Warsaw -
Benjamin Kaplan -
Benjamin Peterson -
Guido van Rossum -
Ned Deily -
Ronald Oussoren -
Russell E. Owen -
Stephen J. Turnbull