Fwd: Broken link to download (Mac OS X)

Hey all, This seems to happen whenever we do a new release (we've had a couple of emails to webmaster@python.org about it since 2.6.5 was released). The main download page for Python has a broken link for the Mac installer (because it hasn't been built yet I assume): http://python.org/download/ The link 404s, with no explanation or alternate link - so for the casual user who wants to install Python 2.6 on Mac OS X they are sorely-out-of-luck. Not being able to provide a mac installer at the same time as other platforms is one thing (and I accept that is unavoidable), breaking the download links for Mac users for unspecified lengths of time is just bad practise. If we create a new stable release without a Mac installer can we at least provide a brief explanation and link to the *previous version* until the new version is ready? All the best, Michael -------- Original Message -------- Subject: Broken link to down Date: Sun, 21 Mar 2010 13:40:36 +0000 From: Ben Hodgson <ben@benhodgson.com> To: webmaster@python.org Hey there, In case you don't know, the link on http://www.python.org/download/ to the Python 2.6.5 Mac Installer Disk Image (http://www.python.org/ftp/python/2.6.5/python-2.6.5_macosx10.3.dmg) is broken. Cheers, Ben -- Ben Hodgson http://benhodgson.com/

Why is it unavoidable that the Mac build will languish behind others? Are we supporting MacOs or aren't we? If we are, why isn't the creation of the build a part of the release process? Clearly it's not a priority given that nobody has seen fit to (or had time to) reply to this mail in three weeks. regards Steve webmaster@python.org wrote:
-- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steve Holden wrote:
Maybe the PSF should make it a priority by funding acquisition of the appropriate proprietary hardware (Mac / Windows) for the release manager. Otherwise the avaialbility of binaries is going to lag source releases forever. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvFPQ0ACgkQ+gerLs4ltQ5EkQCfTCMQTefF8dnjn2sZ8fHgJNNc HD8Ani13Nq8q1zaG8ttlpoZ/gZ3tQ3Ln =DuRD -----END PGP SIGNATURE-----

Tres Seaver wrote:
Thanks for your note. One of the reasons I took the trouble to comment on the issue was to determine whether resources are needed. Generally I regard it as the PSF's job to respond to requests for resources from the dev team, not to tell them how to do the work they already excel at. We did, a while ago, accept a hardware donation from Apple. If we need more Mac hardware I will be happy to ask the Infrastructure Committee to look into it, but I'd suggest we need guidance from someone actually involved in the build work to ensure we get maximum utility out of any investment. I do think it makes us look bad to have one supported platform lag the others, but it wasn't obvious to me whether hardware alone was the reason. If it is, the fix should be relatively simple. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Steve> I do think it makes us look bad to have one supported platform Steve> lag the others, but it wasn't obvious to me whether hardware Steve> alone was the reason. If it is, the fix should be relatively Steve> simple. I can't believe it's a hardware issue. Probably half the people with laptops at the last PyCon I attended were Macs. I suspect anyone with a recent Mac running Leopard or Snow Leopard should be able to build the binary release. It's probably just a matter of knowing how. Skip

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. Löwis wrote:
I assumed that creation of installer binaries for a release depends on having the release manager or a lieutenant have access to the given platform (Windows, OS/X) and tools, For instance, the RM or lieutenant might only have access to such a platform part-time (e.g., only while at work, or only at home). In such a case, providing additional hardware could expedite creation of the binaries. I'm sorry if I misunderstood the issue. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvFmdkACgkQ+gerLs4ltQ5B/QCglQlANbOgn8BodKowku/aCBc0 QdgAoLQi2Ntlns/B4FdxNs0kCHC5VVqk =Xm+Q -----END PGP SIGNATURE-----

On Wed, Apr 14, 2010 at 06:33:03AM -0400, Tres Seaver wrote:
As with Windows, I personally find that building Python with all the associated libraries is blocked on getting the right libraries installed, not on getting the compilers (which are available to us for free) ... but I'm sure that easy access to hardware is an issue, too. I can provide command-line access to a Mac OS X machine but I'm not sure that's enough. Let me know if anyone wants that. Separately, I'd be happy to put forward a proposal to the PSF to fund RMs and their lieutenants with a Mac or a PC, whichever they needed to keep things moving. It's the least "we" can do, IMO, and hardware is just not that expensive compared to the dedication of the volunteers. If Georg, Benjamin, Martin, or Ronald are interested, please just tell me (or Steve, or the PSF board, or ...) what you want and I'll work on getting it funded. --titus -- C. Titus Brown, ctb@msu.edu

On Apr 14, 2010, at 06:39 AM, C. Titus Brown wrote:
While I appreciate the offer, I don't think this is as useful as it could be. Speaking as an RM, I would happily build all three releases (source tarballs, Windows installers and Mac disk images) if I had easy access to the necessary machines and environments over the 'net, and precise instructions on how to do the builds. Ideally, those instructions would be "run this script" or even rolled into the current release.py script that is used to build the tarballs. I have Macs and a Windows machine. The Windows machine is not easily accessible as it's a dual-boot with Ubuntu. As stated previously, the difficult part is getting all the necessary dependencies in place and the expertise to know the steps that are required to build. A network accessible machine for Mac and Windows, where other experts such as Martin and Roland can help maintain and configure, would be ideal. That way, and of the platform experts or RM could push a button and build the installers. -Barry

C. Titus Brown wrote:
For me, my company provides all the infrastructure I need (tools, bandwidth, hardware, etc). I agreed, in return, to mention that support on our group's home page: http://www.dcl.hpi.uni-potsdam.de/ Regards, Martin P.S. and yes, HPI is a company, but associated with the University of Potsdam.

I think you do misunderstand: the people that do (occasionally, or eventually) provide OSX binaries have hardware available to them all the time. They only have part-time to work on Python, though. Of course, if somebody would promise a more timely provision of binaries, and can't do that because of lack of hardware, we would have to look for a solution. Regards, Martin

In article <4BC54F4F.4090307@v.loewis.de>, "Martin v. Lowis" <martin@v.loewis.de> wrote:
That's true. But there wouldn't be a traditional OS X installer for 2.7b1 anyway since it turns out it is not possible to build a multi-arch installer without patching because of a bug that wasn't caught before the code cutoff since there are no OS X buildbots that test that configuration. But, at the moment, there aren't any OS X buildbots at all, are there? That *is* something that the PSF could help with. I would be happy to help with that myself, although my time to do so will be very limited for the next few weeks. -- Ned Deily, nad@acm.org

Ned> ... since there are no OS X buildbots that test that configuration. Ned> But, at the moment, there aren't any OS X buildbots at all, are Ned> there? That *is* something that the PSF could help with.... What happened to the big-ass computer farm for Python which was being put together by someone at (I think) Michigan State? Skip

What happened to the big-ass computer farm for Python which was being put together by someone at (I think) Michigan State?
That sounds a lot like Snakebite (www.snakebite.org), which is still... uhhh, a work in progress ;-) We've run into an issue recently that's thwarted progress, but that'll hopefully be resolved in the next couple of weeks. And then... full steam ahead! Trent.

Actually, for those that are interested, here's a copy of the presentation I gave at the Testing in Python session at PyCon a few months ago: http://www.snakebite.org/presentations/snakebite-pycon2010-tip.pptx (Office 2007-2010) http://www.snakebite.org/presentations/snakebite-pycon2010-tip.ppt (Office 97-2003) If anything, it'll shed some light on all the unforeseen issues we've been running into since the project's inception. The presentation is a little out of date -- I spent three months earlier this year on the network and it's definitely in the most respectable state it's been in yet. Coupla' photos for those that are interested: http://snakebite.org/images/IMG_4384.JPG http://snakebite.org/images/IMG_4392.JPG http://snakebite.org/images/IMG_4393.JPG http://snakebite.org/images/IMG_4394.JPG http://snakebite.org/images/IMG_4395.JPG http://snakebite.org/images/IMG_4396.JPG http://snakebite.org/images/IMG_4401.JPG http://snakebite.org/images/IMG_4402.JPG http://snakebite.org/images/IMG_4403.JPG http://snakebite.org/images/IMG_4405.JPG http://snakebite.org/images/IMG_4410.JPG http://snakebite.org/images/IMG_4418.JPG http://snakebite.org/images/IMG_4424.JPG http://snakebite.org/images/IMG_4425.JPG We've got three racks filled to the brim with all sorts of servers: - 4xItanium 2 @ 1.5GHz, 16GB RAM, HP-UX 11iv3 - 4xItanium 2 @ 1.5GHz, 30GB RAM, RHEL 5.3 - 2xUltraSPARC III 900MHz, 8GB, Solaris 10 (file/zfs/nfs server -- 16x146GB 2Gb FC) - 2xUltraSPARC III 1.2GHz, 4GB, Solaris 10 - 2xPA-RISC 875MHz, 8GB, HP-UX 11iv1 - 4 AIX boxes w/ 2x1.5GHz, 8GB, AIX 5.1, 5.2, 5.3 & 6.1 - 10 dedicated VMware x86/64 boxes, ranging from dual core 8GB to 8 core monsters with 64GB - 4x667MHz AlphaServer, 8GB, Tru64 - 4x600MHz SGI Octane 300, IRIX 6.22 - ....and lots of other stuff. Actually, the only platform we don't have is Mac OS X. Although I've got a contact at Apple that I'll start harassing again once I'm back in East Lansing. Trent.

Ned Deily wrote:
The PSF still has a machine that was donated by Apple that once used to be a build slave. Unfortunately, that machine had hardware problems (or atleast appeared to have hardware problems). So if anybody would be interested into maintaining it, please let us know. Regards, Martin

Ned> Any idea what type of machine it is and where it is currently Ned> located? I seem to recall it is/was a G4 XServe. My guess as to location would be at xs4all.nl. Skip

In article <19399.11323.946604.992452@montanaro.dyndns.org>, skip@pobox.com wrote:
If it were working that could be of use. It would not be able to run OS X 10.6 but having a 10.5 system PPC system as a buildbot would certainly be useful; it should be fine for the default installer configuration builds. (Alas, I don't expect to be anywhere in the vicinity in the foreseeable future.) -- Ned Deily, nad@acm.org

Ned Deily <nad@acm.org> wrote:
I've identified 4 G4/G5 machines at PARC I can set up as build slaves on either Tiger or Leopard, and the first one of those is up (G4/Tiger) and running. Can someone (Ned?) take a look at the output and see if the errors are currently expected, or if I need to do more config/install work? I'm using Xcode 2.5. If it's good, I'll go to work on more. The build slave name is "parc-tiger-1". Thanks! Bill

Le Thu, 29 Apr 2010 07:39:22 -0700, Bill Janssen a écrit :
No failures are currently expected, but the other Tiger buildbot is seeing the same ones as yours. In any case, somebody needs to investigate why these tests fail under Tiger, and I suppose the buildbot owners are in the best position to do so. Regards Antoine.

On 29 Apr, 2010, at 16:39, Bill Janssen wrote:
As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually? IMHO it would be better to do a framework build for at least some of the OSX buildbots because that's what is in the binary installer for OSX. Ronald

Ronald Oussoren <ronaldoussoren@mac.com> wrote:
As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually?
I'll have to fiddle with where it "is" on our network to do that. I'll take it down and move it to different subnet. What about this error? /bin/sh: line 1: pybuildbot.identify: command not found It looks like a configuration issue, or maybe a bug in pybuildbot. What about readline? Darwin doesn't have it (or rather, it has a different one). Why is that 'skip' unexpected?
Yes, that sounds good to me, too. But how do we make that a standard test so that appropriately enabled buildbot slaves will do it automatically? Something that gets configured in the build master? Bill

Bill Janssen <janssen@parc.com> writes:
Ronald Oussoren <ronaldoussoren@mac.com> wrote:
As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually?
What about readline? Darwin doesn't have it (or rather, it has a different one). Why is that 'skip' unexpected?
For what it's worth, for the libraries, I used the build-installer script to build the same versions of libraries that it uses for the binary installer, and installed them in /usr/local for my buildbot to use, until there's enough time to have the buildbot build such local libraries within its own tree. That at least matches up the external dependencies for buildbot builds with that used by the eventual binary installer. For my buildbot, generally the only unexpected skips are for test_bsddb185 (which feels right to me - I don't have that version of the library, nor do I think has the binary installer) and test_ioctl (which I have no idea on yet).
This came up in the earlier discussion, and while I do still think that's a better long term approach, I suspect that with respect to test coverage and code generation the non-framework build for the interpreter is a fair representation for testing. I suspect much of this is just a builder change on the master (to use the right Makefile targets for generating the framework build), but just given experience to date getting the binary installer building under buildbot, there may be some unexpected environmental stuff. Ideally we'd work up a way to do a universal framework build (though I'm not sure what if any extra support may be needed to have the system use the framework if not installed in a standard location), and then for full testing, maybe even use the makefile target that tests both the Intel and PPC path (the latter via Rosetta on an Intel system). What I'm not sure about at the moment is how much different in terms of testing such a setup might be (if, for example, any extra work is needed to be able to test a framework build while still local in the buildbot tree and not in a normal framework location), so how much energy is worth putting into it, especially if that might use resource among those able to resolve some of the open code issues. For my part, if there's free cycles I'd personally rather address the external library issue first, as opposed to the framework build stuff, since that feels more fragile to me (in terms of the buildbot environment for the Mac) at the moment. Over the past few weeks I'm also fairly sure that I'm finding stranded python test processes (not too dissimilar as on the Windows buildbots) that are bogging down the buildbot and thus more detrimental to ongoing buildbot operation than the lack of framework builds, so there may be other unknown issues more valuable to hit as we get more experience. -- David

On 29/04/2010 16:13, Ronald Oussoren wrote:
Well - I have nine failing tests on trunk for Mac OS X with Snow Leopard. 9 tests failed: test_cmd_line test_imp test_import test_posix test_pydoc test_runpy test_urllib2 test_urllib2_localnet test_warnings I believe that Victor has ensured that the buildbot failures have open issues against, I'm just going to check that the failures I see are the same. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Just built on Intel/Leopard with the trunk (almost vanilla, except for _ssl.c) and tested. Here's what I configured: ./configure --prefix=/local/python/trunk --disable-universalsdk --disable-framework --disable-toolbox-glue --with-pydebug And here's the results with "make test": 342 tests OK. 6 tests failed: test_argparse test_grp test_posix test_pwd test_socket_ssl test_urllib2_localnet 37 tests skipped: test_aepack test_al test_applesingle test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_epoll test_gdb test_gdbm test_gl test_imgfile test_largefile test_linuxaudiodev test_macos test_macostools test_ossaudiodev test_readline test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 8 skips unexpected on darwin: test_aepack test_applesingle test_gdb test_macos test_macostools test_readline test_scriptpackages test_ttk_guionly Why is the skip of "test_readline" unexpected on darwin? The readline on Darwin isn't what Python wants. Bill

R. David Murray <rdmurray <at> bitdance.com> writes:
The machine is likely lacking the readline development headers. In any case, please don't focus on the skipped tests. What's important is the failed tests. Bill, Michael, it would be nice if you could investigate a bit more on these failures. Regards Antoine.

Antoine Pitrou <solipsis@pitrou.net> wrote:
Well, test_grp test is failing because it assumes that group IDs are unique, which is not guaranteed by getgrent(). And apparently not true at PARC. test_posix fails on some group-related thing, too: the use of initgroups(), which is supposed to fail, succeeds. test_pwd fails because the fake uid chosen for one test is a real UID for a PARC user. So these are really problems with the test being based on an insufficiently general model of the real world. Bill

On Thu, 29 Apr 2010 17:27:44 -0700, Bill Janssen <janssen@parc.com> wrote:
Could you file bugs for these, please? The first and last others should be able to replicate easily; the middle one will probably require your help on the machine in question to debug. -- R. David Murray www.bitdance.com

On 29 Apr, 2010, at 20:28, Bill Janssen wrote:
...
The skip of test_readline is unexpected because readline should have been build with the options above and tests should have worked. Ronald

Martin v. Löwis wrote:
I seem to recall there was some issue (aside from the current lack of a reliable OS X buildbot) that prevented us from regularly grabbing the head of the tree and using it to automatically build the Windows and Mac installers (to check that the installers could actually be created, preventing a repeat of the Mac situation with 2.7b1). Am I misremembering the discussion from last time this topic came up? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Thu, Apr 15, 2010 at 12:47:50AM +1000, Nick Coghlan wrote:
Making the Windows binary build process automatic and robust is challenging if you hate Windows (like I do). Martin also made the point that it's been broken forever, so people don't seem to care :). ISTR Martin just makes them manually. It's on my TODO list to robustify this process. OTOH, my Mac automated builds/tests are working just fine. I could produce nightly binaries from those easily enough: http://lyorn.idyll.org/ctb/pb-dev/p/python/ Just need to figure out the magic doohickey commands... will add to list. cheers, --titus -- C. Titus Brown, ctb@msu.edu

That's true. In particular, people had been asking that the MSI installers are code-signed, which I now also do. I wouldn't like to keep the private key for the PSF code signing certificate available to some machine that is readily accessible from the internet - so at least that part is manual, in the sense that I have to provide my password. Regards, Martin

I only remember a similar discussion, which was about why the Windows nightly builds had been dropped. The reason was that the process was too flaky and required too much manual fixing, and that the demand for these binaries was too low. I don't recall OSX being mentioned in that discussion, though. Regards, Martin

Martin v. Löwis wrote:
Yep, that sounds like the discussion I was thinking of. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On 14/04/2010 06:13, Ned Deily wrote:
The problem is the process that creates a new release with a 404 link to the Mac installer with no explanation. The 2.6.5 release (as always) caused several requests to webmaster from Mac users unable to download Python - which is a further waste of volunteer time as well as a cause of frustration for users. Building the Mac installer requires volunteer time which I'm not sure that more hardware will fix - compiling a full build of Python for Mac OS X (with all the Python modules like Tkinter etc) requires expertise which only a few people have. A Mac OS X machine (and location to keep it) for the buildbots is a *big* need however. All the best, Michael -- http://www.ironpythoninaction.com/

On Apr 14, 2010, at 10:56 AM, Michael Foord wrote:
As the RM, that's my fault then. When I start creating the page for a new release I will leave the Windows and Mac links commented out. I do (now <wink>) wait for Martin to upload the Windows installers before announcing the release, but generally don't wait for Ronald to produce the Mac images, so I leave those commented out too. When Ronald builds the Mac image and provides it to me, I'll upload them and tweak the page. -Barry

On 14/04/2010 13:46, Barry Warsaw wrote:
Can we amend that to having some placeholder text saying that the Mac installer is not yet available and a link to the previous available version please. That can then be replaced with the normal link once the Mac installer is uploaded. Thanks Michael Foord
-Barry

On Apr 14, 2010, at 02:45 PM, Michael Foord wrote:
You mean, on the x.y.z release page, or a separate page? I guess you're saying that it's better to include some "not available yet" text on the x.y.x release page than to just comment out the platform. That makes sense, and I'd be happy to do that, since I already make such a caveat in the release announcement. -Barry

On 14/04/2010 13:58, Barry Warsaw wrote:
Yes, I mean on the release page. The issue is that the download links on the sidebar / front page go straight to the latest release page. If there isn't yet a Mac installer available, and no alternative link to get the previous version, it leaves Mac users with no obvious way to get a Mac installer *at all*. (Those who know the site well can find the previous versions themselves, but for users not intimately familiar with our site layout it is 'a challenge'.) Thanks Michael
-Barry

Michael Foord wrote:
I would actually use the "source only" release pages as a guide here. They provide a link to the download page for the last version that provided Mac and Windows binaries. This makes it clear to the downloader that the binaries are for an older version, while still making a binary version easy to find. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On 14/04/2010 17:36, Bill Janssen wrote:
Well - an XServe that we can run virtualisation on would be the *ideal* solution. I think the X serves are the only machines you are *allowed* to virtualise OS X on (?). All the best, Michael
Bill

On 14/04/2010 17:41, Michael Foord wrote:
An alternative that solves the problem of finding somewhere to put the machine would be to use a service like Mac Mini colocation hosting: http://www.macminicolo.net/ That could be used both for debugging OS X specific problems *and* as a buildbot. All the best, Michael

Much of it is, but it still takes expertise to run the script. It would take even more expertise to capture the remaining pieces in the script, too, and no living person has that much expertise to write the script (perhaps there are one or two people, and they don't have the time). Regards, Martin

Martin v. Löwis <martin@v.loewis.de> wrote:
Apparently not, according to Ronald's recent mail. I'd be happy to help with whatever remaining capture is necessary for OS X -- not automating this kind of thing is nuts. My buildbots build UpLib, and most of its optional packages (including ghostscript, xpdf, libtiff, openssl, t1lib, Lucene, PyLucene, etc.) from source every night, and whenever I do an UpLib check-in. On Linux and OS X, right now -- I'm working on Windows right now.
Well, God forbid they should ever be hit by a bus! This kind of thing needs to be written down. Bill

Bill Janssen wrote:
+1 regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Martin v. Löwis wrote:
I take it you don't mean to imply that there's a dead person somewhere with the necessary expertise? [ducks]. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Michael Foord wrote:
How about as a first step the release build process include a check for broken links before committing the web content for a new release? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

On 14/04/2010 19:25, Steve Holden wrote:
Yes - needed but orthogonal. :-) Michael
regards Steve

How about as a first step the release build process include a check for broken links before committing the web content for a new release?
You'd have to convince the release manager to add a step to the release process. Given that the release process has already too many steps, he is probably skeptical wrt. such a proposal. Regards, Martin

Why is it unavoidable that the Mac build will languish behind others?
Because of lack of volunteers, and expertise (i.e. the experts lack time).
Are we supporting MacOs or aren't we?
We aren't. Strictly speaking, "we" (python-dev) "support" nothing (in the sense that "we" can promise a support hotline, service contracts, or anything in that respect). That includes timely availability of stuff. If you want Python "support" on MacOS, contact Apple, ActiveState, or any other company that provides actual support. In a wider sense of "to support", MacOS is certainly supported by Python. There is everything in the source code that you need to make Python run on a Mac. Just download the sources and compile them yourself.
Clearly it's not a priority given that nobody has seen fit to (or had time to) reply to this mail in three weeks.
That is not surprising: none of the webmaster people would be able to answer the question. python-dev is indeed the right place to ask. Regards, Martin

Martin v. Löwis wrote:
OK. I should confirm that I mean "support" in your wider sense below.
And yet we don't regard the Windows release as complete until you have built the binaries (for which service you deserve many thanks, by the way). Is the Mac platform one on which users will be happy to compile from source? I know its users are savvier than Windows users, and have a better tool set available to them, but they still seem to expect downloadable installers.
I thought I'd picked this thread off python-dev. What point am I not understanding here? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

This phenomenon exists for a lot of other systems, as well. For example, we also support Solaris, but stopped providing Solaris binaries since Python 1.5 (when I last built binaries for Das Python-Buch). People still can get Solaris binaries from ActiveState or Sunfreeware; Sun also ships Python as part of the system.
The major difference in the "do it yourself" attitude is that Mac user get a compiler for free, as part of the operating system release, whereas for Windows, they have to pay for it (leaving alone VS Express for the moment). However, the real difference is motivation for contribution to open source projects. You normally contribute to scratch an itch. Unfortunately, these binaries don't come out such a motiviation. So the release manager roles are either altruistic, or rely on extrinsic motivations (money, reputation).
Maybe I misunderstood. I thought you copied it from the webmaster list (as the original To indicated). I don't recall having it seen on python-dev, but I kill many messages to python-dev without reading them. Regards, Martin

On 14 April 2010 07:37, Paul Rudin <paul@rudin.co.uk> wrote:
I believe that the express editions don't include some of the advanced optimisations (profile guided optimisation rings a bell) which are used to build the official binaries. So if the binaries were built using Express Edition, they would be somewhat slower. That is just my recollection, however - it may be out of date or wrong. Paul.

Paul Moore wrote:
I spent some considerable effort last year ensuring the developer community was well-supplied with MS developer licenses that give access to any necessary tools. Was I wasting my time? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

On Apr 14, 2010, at 01:36 PM, Steve Holden wrote:
At the time I didn't care because I had no access to anything Windows. Now I do, though it's a bit of a PITA (because I have to reboot my main dev box). How can I get a license and the tools? Installing the express version was pretty easy, though it has limitations of course. -Barry

On Wed, Apr 14, 2010 at 12:36, Steve Holden <steve@holdenweb.com> wrote:
I certainly hope not. I believe full MSDN licenses were involved there, which are a great resource to a Windows developer. I'll have a whack at building the Windows installer. It would be nice to see it integrated with the build process, including testing of an installed version. We already have one or more Linux buildbots which test an installed build, and that setup uncovered an additional bug in a recent fix of mine.

On 14 April 2010 18:36, Steve Holden <steve@holdenweb.com> wrote:
Definitely not - my offer is at least in part based on the fact that I have the full tools as a result and so can do (hopefully) useful work on assisting with any necessary build process improvements. My comment was in response to the question "why are we ignoring the express editions"? If having the full versions wasn't better than having the free ones, the developer licenses would be of less benefit (and that isn't the case, as I say). Paul.

Paul Moore <p.f.moore@gmail.com> writes:
I could be wrong too. However I think I had two things confused (it's been a while since I've done any windows development). There are two different suites of free compilers from MS. There's the windows sdk, which includes a command line toolchain and the express edition of VS which (essentially) is a cut down version of the commercial version of VS. Some info about the windows sdk versions is here <http://msdn.microsoft.com/en-us/windows/dd146047.aspx> and about versions - including Express - here (at least for VC++) <http://msdn.microsoft.com/en-us/library/hs24szh9%28VS.90%29.aspx> although that's a little old.

Paul Rudin wrote:
Primarily out of a historical perspective: I think "we" started providing Windows binaries primarily because some people bought the appropriate license (of Visual Studio, and Wise Installer). IMO, that's a stronger reason than mere convenience. As for available tools: I'm not sure how you would build an AMD64 release just with those tools. Regards, Martin

On Wed, Apr 14, 2010 at 07:36:25AM +0200, "Martin v. L?wis" wrote:
I personally think the Mac is pretty important, as one of the big three consumer operating systems...
Actually, I think the more pernicious factor is that a version of Python comes pre-installed on Mac OS X, which means the up-front demand is lower for a pre-compiled version. This is problematic, though, because that version of Python only gets upgraded with full releases of Mac OS X (which are not very well correlated with releases of Python, of course). So we have lots of Python installs out there that, in the absence of a precompiled binary version, can't be upgraded without installing the developer tools.
I don't know what to do about motivation but if there are barriers that we can lower, please let me know. cheers, --titus -- C. Titus Brown, ctb@msu.edu

For example, you could volunteer to produce OSX binaries in a timely manner for the next two years. Now would be a good point to do so, since your next release would still be a beta release. Expect to mess up the first time you do it, so it's good that it would be only a beta. Regards, Martin

On 14/04/2010 07:17, Steve Holden wrote:
Mac users definitely *do* expect installers. Building Python requires, I believe, the XCode development tools to be installed. Even then, building a full version of Python - with *all* the C extensions that are part of a Python release - is not a trivial task. All the best, Michael Foord

Michael> Mac users definitely *do* expect installers. Building Python Michael> requires, I believe, the XCode development tools to be Michael> installed. XCode is free, and I suspect many people have it (I do). Michael> Even then, building a full version of Python - with *all* the C Michael> extensions that are part of a Python release - is not a trivial Michael> task. Isn't that just a matter of having the recipe written down somewhere? Skip

On 14/04/2010 18:01, skip@pobox.com wrote:
Sure - but probably not your average Python-on-Mac user. Or at least a good proportion of them, particularly newbies who we are keen to keep the experience of obtaining Python simple. First download and then install 1gigabyte of developer tools (seriously) requiring registration, then compile Python from source yourself, is not the way to go. (And yes the XCode tools are included in the OS disks - but not the latest versions, especially if you are still running Leopard.)
Yes, that would be nice. :-) Preferably a recipe that doesn't involve Macports or Fink which some of us are allergic to. Michael
Skip

Bill Janssen wrote:
Please do, and let me know if resources of some kind would help. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

On 14/04/2010 18:49, Steve Holden wrote:
A Mac buildbot would *definitely* be useful and I know some of the Python-dev crew would like access to a Mac OS X machine for trying out fixes when none of the core Mac maintainers are around. A couple of co-located, or otherwise hosted, Mac boxes would be very useful. The obvious question is who would hold the keys (maintain the buildbot)? I don't know if Ronald has the spare capacity to do this (?). If someone will help me with the initial setup I can otherwise volunteer. All the best, Michael
regards Steve

On Wed, Apr 14, 2010 at 06:52:46PM +0200, Michael Foord wrote:
We have some space in our machine room, and some sysadmins that like open source. I will ask them if they are willing to do physical maintenance (profs wisely aren't allowed access to the machine room). That would really be ideal... I will report back to interested people. --titus -- C. Titus Brown, ctb@msu.edu

On Wed, Apr 14, 2010 at 09:37:34AM -0700, Bill Janssen wrote:
Please pass it along to me, when you get it working... I build python2.7 nightly on Mac OS X, but just at the command line. see: http://lyorn.idyll.org/ctb/pb-dev/p/python/ http://lyorn.idyll.org/ctb/pb-dev/p/python/show_all (the Windows build is flaky for me, so the 'show_all' shows mostly Windows builds). --titus -- C. Titus Brown, ctb@msu.edu

On 14/04/2010 20:21, "Martin v. Löwis" wrote:
Right - but we were discussing this in the context of barrier to entry, particularly to new users. We don't impose this requirement for Windows users though - we provide binary installers. I *know* we're a volunteer organisation (etc), but it is good for us to be aware of our process weaknesses without excusing ourselves. Unfortunately the Mac installer build script doesn't seem to run at all on Mac OS X 10.6 (at least not on my machine), but hopefully the situation is clarified so that one of us who does still have Mac OS X 10.5 will be able to build the installer in a timely manner for the next release should Ronald be too busy. All the best, Michael
Regards, Martin

I think nobody here is questioning that it is desirable to have OSX binaries. And I readily admit that this is a weakness.
I'm not sure whether 10.5 would be sufficient - it may be that you need to go back to 10.4 (*). Unfortunately, Apple manages to break compatibility and portability with every release, which makes this particular build task soooo tricky. You have to make all kinds of decisions and compromises where are really difficult to keep track of. Regards, Martin (*) I know that this is the version I had to go back to when I created OSX binaries for 2.5.x.

On 14/04/2010 21:37, "Martin v. Löwis" wrote:
In an earlier email Ronald said:
I can't verify that this is correct, I can verify it that the build-installer script doesn't appear to work under 10.6. Michael

On Apr 14, 2010, at 3:37 PM, Martin v. Löwis wrote:
I'm not sure whether 10.5 would be sufficient - it may be that you need to go back to 10.4 (*).
I think you just need to supply to configure MACOSX_DEPLOYMENT_TARGET=10.4 and have the appropriate SDK installed with Xcode. I believe it is installed by default with Xcode on Leopard (10.5), but with Xcode on Snow Leopard (10.6) the 10.4 SDK is an optional install.
Unfortunately, Apple manages to break compatibility and portability with every release, which makes this particular build task soooo tricky. You have to make all kinds of decisions and compromises where are really difficult to keep track of.
Hmm. Apple provided compatibility SDK and documented it. The only compromise I see in this build process right now is that we are building a Panther (10.3) compatible installer, while Mac OS X is a certified UNIX starting with 10.5. So, we are deliberately using obsolete model to provide backward compatibility. However, I'm fine with that for the installer. Power users can compile their own build optimized for their platform. Regards, Zvezdan

On Apr 14, 2010, at 4:40 PM, Martin v. Löwis wrote:
I was replying to your point about 10.4 build. Naturally, if you want a 10.3 build you'd pass 10.3 as the target and would have to have appropriate Xcode SDK installed.
Yes. x86_64, i386, and ppc are supported even in the Xcode supplied with the latest Mac OS X 10.6. Only ppc64 is not. So, ppc is not an issue. The problem is that enforcing backward compatibility with 10.3 and 10.4 makes 64-bit Intel architecture not feasible. You are right, it is a compromise. We are making more users happy by providing a 32-bit installer for a wider range of OS releases. However, if we want a more modern certified UNIX, 64-bit installer, then we'll have to draw a line and stop supporting older OS releases. Just as we stop supporting older releases of Python. Regards, Zvezdan

On 15 Apr, 2010, at 0:12, Zvezdan Petkovic wrote:
You don't have to install an SDK to be able to build binaries that run on older versions. The reason the binary installer gets build with the 10.4u SDK is that the default compiler on OSX 10.4 cannot build universal binaries without that SDK.
No. It is possible to build a binary that supports ppc, ppc64, x86 and x86_64, and that's even possible using a single additional configure switch. That binary will require OSX 10.5 to run though due to using symbols that aren't available on earlier versions of OSX (thanks to the better UNIX API compatibility in 10.5). PPC64 is not supported on OSX 10.6 though.
I want to provide 2 installers for Python 2.7 and 3.2: 1) The current 32-bit only installer that runs on OSX 10.3 or later 2) An installer that supports ppc, x86 and x86_64 and requires OSX 10.5 or later The latter would be the one that most users would want to use. Note that the second installer does not support ppc64 and three reasons: (1) PPC64 is a dead end on OSX, (2) libffi has issues on darwin/ppc64 that probably affect ctypes and (3) I do not have regular access to ppc64 machines and can therefore not provide any support for that platform. Ronald

Martin v. Löwis wrote:
Even on Linux, it takes a bit of fiddling. I finally remembered to write the steps down for Kubuntu when I set up my current machine (you need to apt-get half a dozen or so additional dev packages): http://boredomandlaziness.blogspot.com/2010/01/kubuntu-dev-packages-to-build... Take away the convenience of apt-get and add installing the compiler and installer toolchains and you've got a fair amount of setup time on your hands (and lots of things that can go wrong). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Michael Foord wrote:
What's non-trivial about it? I usually find that the normal "./configure; make; make install" sequence works fine without any further intervention. If you want a framework installation you have to read the README and use a couple of extra options, but it still works very smoothly. -- Greg

On 14/04/2010 23:32, Greg Ewing wrote:
A build on my machine produces output similar to: Python build finished, but the necessary bits to build these modules were not found: _bsddb dl gdbm imageop linuxaudiodev ossaudiodev readline spwd sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. Failed to build these modules: _tkinter Obviously many of those are not meant to be built and I usually build Python for running the test suite - so I don't care about not having Tkinter. A new user of Python would most certainly care about not having Tkinter. Michael -- http://www.ironpythoninaction.com/

In article <4BC63599.5020005@voidspace.org.uk>, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
third-party (Sleepycat) library needed (see the installer script)
dl
only available for 32-bit
gdbm
third-party library needed (not supplied in the installer build)
imageop
only available for 32-bit (and removed in 3.x)
linuxaudiodev ossaudiodev
neither supported on OS X
readline
requires either GNU readline lib (not included with OS X) or (with 2.6.5, trunk, or py3k) can now use OS X editline (libedit) replacement when targeting for 10.5 or 10.6 (add MACOSX_DEPLOYMENT_TARGET=10.5 or 10.6 to ./configure arguments).
spwd sunaudiodev
neither supported on OS X
Failed to build these modules: _tkinter
currently only supported for 32-bit archs as there was no 64-bit (non-X) Tk on OS X prior to 10.6 and there are unresolved problems with the 10.6 Apple-supplied Tk 8.5 and 2.6.x IDLE (Issue6864). -- Ned Deily, nad@acm.org

Whilst making Python easier to build on the Mac is certainly a worthy goal, the point of my post was to demonstrate (in reply to an email by Greg Ewing) *why* building a *full* Python from source was non-trivial. I personally only build Python from source to test changes to core-Python and am happy with the Python binary installers for the Mac - once they arrive. :-) All the best, Michael On 15/04/2010 14:18, Ned Deily wrote:

On 18/04/2010 15:13, Ronald Oussoren wrote:
10.6.3 and yes I have Tcl and Tk in /Library/Frameworks. How do I determine which versions they are? All the best, Michael
Ronald
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

Michael Foord <fuzzyman@voidspace.org.uk> writes:
10.6.3 and yes I have Tcl and Tk in /Library/Frameworks. How do I determine which versions they are?
You can use "info patchlevel" in tclsh - assuming you're running a tclsh linked to your /Library version (a normal Tcl install puts this in /usr/local/bin I think). Or, tcl.h (in the Headers folder beneath the framework install) has TCL_VERSION and TCL_PATCH_LEVEL defines near the top of the file. Given that your error is a failure to build and not a skip, it sounds like setup is finding Tcl/Tk. From a quick glance, it looks like tkinter may also require the X11 headers (you'd have to have installed X11 separately) - do you have output in your log from what exactly is failing when that module attempts to build? -- David

On 24/04/2010 21:34, David Bolen wrote:
$ tclsh % info patchlevel 8.5.7
Hmmm... looks like a 32 / 64 bit issue, which I believe may be the expected result when trying to build on Snow Leopard (?). i686-apple-darwin10-gcc-4.2.1: -framework: linker input file unused because linking not done i686-apple-darwin10-gcc-4.2.1: Tk: linker input file unused because linking not done ld: warning: in /Library/Frameworks//Tcl.framework/Tcl, missing required architecture x86_64 in file ld: warning: in /Library/Frameworks//Tk.framework/Tk, missing required architecture x86_64 in file *** WARNING: renaming "_tkinter" since importing it failed: dlopen(build/lib.macosx-10.4-x86_64-2.7-pydebug/_tkinter.so, 2): Symbol not found: _TclFreeObj Referenced from: /compile/python-trunk/build/lib.macosx-10.4-x86_64-2.7-pydebug/_tkinter.so Expected in: dynamic lookup I think my Tk/Tcl install came from an Activestate installer. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

Michael Foord <fuzzyman@voidspace.org.uk> writes:
Hmmm... looks like a 32 / 64 bit issue, which I believe may be the expected result when trying to build on Snow Leopard (?).
I think so - I haven't tried a 64-bit build myself, but there's a comment in setup.py indicating that none of the Tcl/Tk framework builds support 64-bit. So I suppose you'd have to build 32-bit if you wanted a working _tkinter. -- David

On 24/04/2010 21:50, David Bolen wrote:
Sorry for my ignorance - how do I force a 32 bit build? Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

On 24/04/2010 22:16, Michael Foord wrote:
Ok, so the following configure command line works and successfully builds Tkinter: ./configure --prefix=/dev/null --with-pydebug --enable-universalsdk All the best, Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

In article <4BC697D2.4020200@v.loewis.de>, "Martin v. Lowis" <martin@v.loewis.de> wrote:
As Ronald pointed out, the installer build script does all of the dirty work of building the install disk image (the .dmg file), including downloading and building necessary third-party libraries. What isn't automatically checked at the moment is that the third-party Tk framework is in place during the build and that there isn't contamination from the build user's environment. There is a patch in Issue5651 to do much of that. It doesn't currently try to handle the possibility of MacPorts (/opt/local) and Fink (/sw) contamination. I believe that issue should be addressed by the resolution of Issue7713. Keep in mind that Python on OS X supports the standard OS X multi-architecture (Intel vs PPC and/or 32-bit vs 64-bit executables in the same file) and multi-version features (the current installer builds are fully supported on 10.6, 10.5, and 10.4 and "best-effort" supported on 10.3 as well although building is not supported on 10.3) as well as "Unix" shared library vs OS X framework shared library configurations. That leads to a rather imposing matrix of potential build systems vs potential target systems. For the most part, Apple provides excellent upward compatibility; downward is somewhat trickier. OS X 10.6 (Snow Leopard) has complicated matters by the change to preferring 64-bit building and executing where possible. For python builds, some build configurations that worked by default under 10.5 and earlier no longer do so on 10.6, primarily due to standard library modules that depend on APIs, in many cases deprecated ones, that are only available in 32-bit versions. The old Macintosh modules comprise the biggest group of these and, while they have been removed in 3.x, they still remain in 2.x. But there are some others as well, which explain most of the build issues Michael reported. There are also newer versions of some open source libraries in 10.6 which cause problems for multi-version builds: openssl, for one, and Apple's 64-bit version of Tk 8.5. None of these problems are insolvable but with the very limited resources (i.e. people time) we've had for working on these issues, and when there have been so many other more critical problems, it has been easier up to now to stick with building on 10.5 (or even on 10.4 which I test build occasionally). I think the single most important thing that can be done to help right now is to get a robust system of OS X buildbots going so that many of the critical problems can be detected up front rather than when one of us gets around to building and install testing the multi-arch and multi-version installers. Since there are so many potential configurations, some thought needs to go into which would be of the most value. I would say that the most important build configurations are: (1) 32-bit Intel/PPC framework on 10.5 (and eventually 10.6) targeted for 10.3 through 10.6 (the current installer config setup); and (2) 32-/64- Intel-only framework for 10.6; followed by (3) 32-/64- Intel/PPC framework for 10.5 and 10.6. Building the whole installer and testing on as many targeted systems as possible would be ideal but that adds more complexity to the automation (again, not insurmountable). But even just doing framework multi-arch builds, installs, and tests (using the appropriate options) on only the build systems themselves without building an installer or third-party libs would go a long way towards catching many, if not most, problems early. I'd be happy to supply more detailed suggestions for specific configuration parameters for those interested in setting up buildbots. (There may be some delay, though, as I will have limited time and Internet access for the next three weeks.) -- Ned Deily, nad@acm.org

Hmm. When I tried it last, it didn't do anything - it just crashed. I then had to fix it, and also arrange to meet a certain number of assumptions that it assumed but that had not been setup on my system. This is some years ago, so I don't recall details.
There is one build slave now up, contributed by David Bolen. Regards, Martin

On Thu, Apr 15, 2010 at 04:41, Ned Deily <nad@acm.org> wrote:
I know for me the reason I have never tried to help with building the binaries is I simply lack the expertise. I always build straight UNIX versions of Python under OS X, so I have no experience with framework builds (which is what the binary distribution is). Heck, I didn't even know about the build script until just now since it's such a pain to build. Plus I don't know if you can use a framework build from within your svn checkout for testing, so I REALLY don't bother building a framework. And to top it off I use the latest libraries of things to look for possible breakage. I am sure I am not the only person who has all of these barriers preventing them from using a framework build consistently.
And this is another reason why Ronald has always been the only person to build the OS X binary; it's complicated. Knowing how to get the proper universal framework with the right things is annoying. Plus making sure something works on multiple versions of OS X is a pain as I am sure very few of us have 10.6, 10.5, and 10.4 lying around. I mean if the build script was dead-simple run and forget I am sure many of us would be willing to create dmgs. But it will take someone who knows what is needed (sounds like Ned does and obviously Ronald does) to get it to that place. -Brett

On 15 Apr, 2010, at 6:36, Martin v. Löwis wrote:
That *is* trivial: use Mac/BuildScript/build-installer.py on OSX 10.5. Making sure that unrelated changes don't accidently break the OSX build can be non-trivial though... That doesn't work with the py3k trunk at the moment, I just noticed that framework builds are broken there. I've filed issue #8441 for that and am working on a fix. Ronald

Ronald Oussoren <ronaldoussoren@mac.com> writes:
For what it's worth, the trunk currently takes ~25 minutes on my relatively modest Mini, when run in parallel with some buildbot stuff, and with third-party sources already downloaded. More or less evenly split among third-party packages, the interpreter itself, and then docs/framework/disk image creation. -- David

On 14/04/2010 07:11, "Martin v. Löwis" wrote:
Why is it unavoidable that the Mac build will languish behind others?
Because of lack of volunteers, and expertise (i.e. the experts lack time).
That doesn't explain why we leave a broken link in place when we do major releases - for days usually (if not weeks) with no explanation to users. That part of the release process is broken and should be fixed. Putting some placeholder text on the release page instead of the broken link is barely any more effort at all. For the last 2.6.5 release I changed the text and removed the dead link myself. When the Mac installer was uploaded someone else put the link back. [snip...]
Which is why I sent the email onto python-dev. However, no-one responded until Steve. All the best, Michael

Steve> Why is it unavoidable that the Mac build will languish behind Steve> others? Are we supporting MacOs or aren't we? If we are, why Steve> isn't the creation of the build a part of the release process? Steve> Clearly it's not a priority given that nobody has seen fit to (or Steve> had time to) reply to this mail in three weeks. I'm not sure who normally creates the Mac distribution, perhaps Ronald Ousorren? It would appear that either Ronald (or whoever) has been unavailable or there was no coordination between the Mac and non-Mac folks involved in the release. I'm willing to give it a whirl (I have both a MacBook Pro and a PowerMac G5 at home - both running Max OSX 10.5.x (Leopard)) though I will almost certainly need a cheat sheet for the process. I normally treat my Macs as Unix boxes from a Python perspective so don't make framework builds. Skip

On Apr 14, 2010, at 10:51 AM, skip@pobox.com wrote:
From the RM perspective, what I would really like to see is updates to the release.py script to check dependencies and automate as much as possible, as well as updates to PEP 101 for any process steps that can't be automated. This goes for both Windows and OS X. -Barry

Why is it unavoidable that the Mac build will languish behind others? Are we supporting MacOs or aren't we? If we are, why isn't the creation of the build a part of the release process? Clearly it's not a priority given that nobody has seen fit to (or had time to) reply to this mail in three weeks. regards Steve webmaster@python.org wrote:
-- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Steve Holden wrote:
Maybe the PSF should make it a priority by funding acquisition of the appropriate proprietary hardware (Mac / Windows) for the release manager. Otherwise the avaialbility of binaries is going to lag source releases forever. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvFPQ0ACgkQ+gerLs4ltQ5EkQCfTCMQTefF8dnjn2sZ8fHgJNNc HD8Ani13Nq8q1zaG8ttlpoZ/gZ3tQ3Ln =DuRD -----END PGP SIGNATURE-----

Tres Seaver wrote:
Thanks for your note. One of the reasons I took the trouble to comment on the issue was to determine whether resources are needed. Generally I regard it as the PSF's job to respond to requests for resources from the dev team, not to tell them how to do the work they already excel at. We did, a while ago, accept a hardware donation from Apple. If we need more Mac hardware I will be happy to ask the Infrastructure Committee to look into it, but I'd suggest we need guidance from someone actually involved in the build work to ensure we get maximum utility out of any investment. I do think it makes us look bad to have one supported platform lag the others, but it wasn't obvious to me whether hardware alone was the reason. If it is, the fix should be relatively simple. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Steve> I do think it makes us look bad to have one supported platform Steve> lag the others, but it wasn't obvious to me whether hardware Steve> alone was the reason. If it is, the fix should be relatively Steve> simple. I can't believe it's a hardware issue. Probably half the people with laptops at the last PyCon I attended were Macs. I suspect anyone with a recent Mac running Leopard or Snow Leopard should be able to build the binary release. It's probably just a matter of knowing how. Skip

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. Löwis wrote:
I assumed that creation of installer binaries for a release depends on having the release manager or a lieutenant have access to the given platform (Windows, OS/X) and tools, For instance, the RM or lieutenant might only have access to such a platform part-time (e.g., only while at work, or only at home). In such a case, providing additional hardware could expedite creation of the binaries. I'm sorry if I misunderstood the issue. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvFmdkACgkQ+gerLs4ltQ5B/QCglQlANbOgn8BodKowku/aCBc0 QdgAoLQi2Ntlns/B4FdxNs0kCHC5VVqk =Xm+Q -----END PGP SIGNATURE-----

On Wed, Apr 14, 2010 at 06:33:03AM -0400, Tres Seaver wrote:
As with Windows, I personally find that building Python with all the associated libraries is blocked on getting the right libraries installed, not on getting the compilers (which are available to us for free) ... but I'm sure that easy access to hardware is an issue, too. I can provide command-line access to a Mac OS X machine but I'm not sure that's enough. Let me know if anyone wants that. Separately, I'd be happy to put forward a proposal to the PSF to fund RMs and their lieutenants with a Mac or a PC, whichever they needed to keep things moving. It's the least "we" can do, IMO, and hardware is just not that expensive compared to the dedication of the volunteers. If Georg, Benjamin, Martin, or Ronald are interested, please just tell me (or Steve, or the PSF board, or ...) what you want and I'll work on getting it funded. --titus -- C. Titus Brown, ctb@msu.edu

On Apr 14, 2010, at 06:39 AM, C. Titus Brown wrote:
While I appreciate the offer, I don't think this is as useful as it could be. Speaking as an RM, I would happily build all three releases (source tarballs, Windows installers and Mac disk images) if I had easy access to the necessary machines and environments over the 'net, and precise instructions on how to do the builds. Ideally, those instructions would be "run this script" or even rolled into the current release.py script that is used to build the tarballs. I have Macs and a Windows machine. The Windows machine is not easily accessible as it's a dual-boot with Ubuntu. As stated previously, the difficult part is getting all the necessary dependencies in place and the expertise to know the steps that are required to build. A network accessible machine for Mac and Windows, where other experts such as Martin and Roland can help maintain and configure, would be ideal. That way, and of the platform experts or RM could push a button and build the installers. -Barry

C. Titus Brown wrote:
For me, my company provides all the infrastructure I need (tools, bandwidth, hardware, etc). I agreed, in return, to mention that support on our group's home page: http://www.dcl.hpi.uni-potsdam.de/ Regards, Martin P.S. and yes, HPI is a company, but associated with the University of Potsdam.

I think you do misunderstand: the people that do (occasionally, or eventually) provide OSX binaries have hardware available to them all the time. They only have part-time to work on Python, though. Of course, if somebody would promise a more timely provision of binaries, and can't do that because of lack of hardware, we would have to look for a solution. Regards, Martin

In article <4BC54F4F.4090307@v.loewis.de>, "Martin v. Lowis" <martin@v.loewis.de> wrote:
That's true. But there wouldn't be a traditional OS X installer for 2.7b1 anyway since it turns out it is not possible to build a multi-arch installer without patching because of a bug that wasn't caught before the code cutoff since there are no OS X buildbots that test that configuration. But, at the moment, there aren't any OS X buildbots at all, are there? That *is* something that the PSF could help with. I would be happy to help with that myself, although my time to do so will be very limited for the next few weeks. -- Ned Deily, nad@acm.org

Ned> ... since there are no OS X buildbots that test that configuration. Ned> But, at the moment, there aren't any OS X buildbots at all, are Ned> there? That *is* something that the PSF could help with.... What happened to the big-ass computer farm for Python which was being put together by someone at (I think) Michigan State? Skip

What happened to the big-ass computer farm for Python which was being put together by someone at (I think) Michigan State?
That sounds a lot like Snakebite (www.snakebite.org), which is still... uhhh, a work in progress ;-) We've run into an issue recently that's thwarted progress, but that'll hopefully be resolved in the next couple of weeks. And then... full steam ahead! Trent.

Actually, for those that are interested, here's a copy of the presentation I gave at the Testing in Python session at PyCon a few months ago: http://www.snakebite.org/presentations/snakebite-pycon2010-tip.pptx (Office 2007-2010) http://www.snakebite.org/presentations/snakebite-pycon2010-tip.ppt (Office 97-2003) If anything, it'll shed some light on all the unforeseen issues we've been running into since the project's inception. The presentation is a little out of date -- I spent three months earlier this year on the network and it's definitely in the most respectable state it's been in yet. Coupla' photos for those that are interested: http://snakebite.org/images/IMG_4384.JPG http://snakebite.org/images/IMG_4392.JPG http://snakebite.org/images/IMG_4393.JPG http://snakebite.org/images/IMG_4394.JPG http://snakebite.org/images/IMG_4395.JPG http://snakebite.org/images/IMG_4396.JPG http://snakebite.org/images/IMG_4401.JPG http://snakebite.org/images/IMG_4402.JPG http://snakebite.org/images/IMG_4403.JPG http://snakebite.org/images/IMG_4405.JPG http://snakebite.org/images/IMG_4410.JPG http://snakebite.org/images/IMG_4418.JPG http://snakebite.org/images/IMG_4424.JPG http://snakebite.org/images/IMG_4425.JPG We've got three racks filled to the brim with all sorts of servers: - 4xItanium 2 @ 1.5GHz, 16GB RAM, HP-UX 11iv3 - 4xItanium 2 @ 1.5GHz, 30GB RAM, RHEL 5.3 - 2xUltraSPARC III 900MHz, 8GB, Solaris 10 (file/zfs/nfs server -- 16x146GB 2Gb FC) - 2xUltraSPARC III 1.2GHz, 4GB, Solaris 10 - 2xPA-RISC 875MHz, 8GB, HP-UX 11iv1 - 4 AIX boxes w/ 2x1.5GHz, 8GB, AIX 5.1, 5.2, 5.3 & 6.1 - 10 dedicated VMware x86/64 boxes, ranging from dual core 8GB to 8 core monsters with 64GB - 4x667MHz AlphaServer, 8GB, Tru64 - 4x600MHz SGI Octane 300, IRIX 6.22 - ....and lots of other stuff. Actually, the only platform we don't have is Mac OS X. Although I've got a contact at Apple that I'll start harassing again once I'm back in East Lansing. Trent.

Ned Deily wrote:
The PSF still has a machine that was donated by Apple that once used to be a build slave. Unfortunately, that machine had hardware problems (or atleast appeared to have hardware problems). So if anybody would be interested into maintaining it, please let us know. Regards, Martin

Ned> Any idea what type of machine it is and where it is currently Ned> located? I seem to recall it is/was a G4 XServe. My guess as to location would be at xs4all.nl. Skip

In article <19399.11323.946604.992452@montanaro.dyndns.org>, skip@pobox.com wrote:
If it were working that could be of use. It would not be able to run OS X 10.6 but having a 10.5 system PPC system as a buildbot would certainly be useful; it should be fine for the default installer configuration builds. (Alas, I don't expect to be anywhere in the vicinity in the foreseeable future.) -- Ned Deily, nad@acm.org

Ned Deily <nad@acm.org> wrote:
I've identified 4 G4/G5 machines at PARC I can set up as build slaves on either Tiger or Leopard, and the first one of those is up (G4/Tiger) and running. Can someone (Ned?) take a look at the output and see if the errors are currently expected, or if I need to do more config/install work? I'm using Xcode 2.5. If it's good, I'll go to work on more. The build slave name is "parc-tiger-1". Thanks! Bill

Le Thu, 29 Apr 2010 07:39:22 -0700, Bill Janssen a écrit :
No failures are currently expected, but the other Tiger buildbot is seeing the same ones as yours. In any case, somebody needs to investigate why these tests fail under Tiger, and I suppose the buildbot owners are in the best position to do so. Regards Antoine.

On 29 Apr, 2010, at 16:39, Bill Janssen wrote:
As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually? IMHO it would be better to do a framework build for at least some of the OSX buildbots because that's what is in the binary installer for OSX. Ronald

Ronald Oussoren <ronaldoussoren@mac.com> wrote:
As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually?
I'll have to fiddle with where it "is" on our network to do that. I'll take it down and move it to different subnet. What about this error? /bin/sh: line 1: pybuildbot.identify: command not found It looks like a configuration issue, or maybe a bug in pybuildbot. What about readline? Darwin doesn't have it (or rather, it has a different one). Why is that 'skip' unexpected?
Yes, that sounds good to me, too. But how do we make that a standard test so that appropriately enabled buildbot slaves will do it automatically? Something that gets configured in the build master? Bill

Bill Janssen <janssen@parc.com> writes:
Ronald Oussoren <ronaldoussoren@mac.com> wrote:
As Antoine noted the test failures are unexpected, could you check if the tests pass if you do the build and testrun manually?
What about readline? Darwin doesn't have it (or rather, it has a different one). Why is that 'skip' unexpected?
For what it's worth, for the libraries, I used the build-installer script to build the same versions of libraries that it uses for the binary installer, and installed them in /usr/local for my buildbot to use, until there's enough time to have the buildbot build such local libraries within its own tree. That at least matches up the external dependencies for buildbot builds with that used by the eventual binary installer. For my buildbot, generally the only unexpected skips are for test_bsddb185 (which feels right to me - I don't have that version of the library, nor do I think has the binary installer) and test_ioctl (which I have no idea on yet).
This came up in the earlier discussion, and while I do still think that's a better long term approach, I suspect that with respect to test coverage and code generation the non-framework build for the interpreter is a fair representation for testing. I suspect much of this is just a builder change on the master (to use the right Makefile targets for generating the framework build), but just given experience to date getting the binary installer building under buildbot, there may be some unexpected environmental stuff. Ideally we'd work up a way to do a universal framework build (though I'm not sure what if any extra support may be needed to have the system use the framework if not installed in a standard location), and then for full testing, maybe even use the makefile target that tests both the Intel and PPC path (the latter via Rosetta on an Intel system). What I'm not sure about at the moment is how much different in terms of testing such a setup might be (if, for example, any extra work is needed to be able to test a framework build while still local in the buildbot tree and not in a normal framework location), so how much energy is worth putting into it, especially if that might use resource among those able to resolve some of the open code issues. For my part, if there's free cycles I'd personally rather address the external library issue first, as opposed to the framework build stuff, since that feels more fragile to me (in terms of the buildbot environment for the Mac) at the moment. Over the past few weeks I'm also fairly sure that I'm finding stranded python test processes (not too dissimilar as on the Windows buildbots) that are bogging down the buildbot and thus more detrimental to ongoing buildbot operation than the lack of framework builds, so there may be other unknown issues more valuable to hit as we get more experience. -- David

On 29/04/2010 16:13, Ronald Oussoren wrote:
Well - I have nine failing tests on trunk for Mac OS X with Snow Leopard. 9 tests failed: test_cmd_line test_imp test_import test_posix test_pydoc test_runpy test_urllib2 test_urllib2_localnet test_warnings I believe that Victor has ensured that the buildbot failures have open issues against, I'm just going to check that the failures I see are the same. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Just built on Intel/Leopard with the trunk (almost vanilla, except for _ssl.c) and tested. Here's what I configured: ./configure --prefix=/local/python/trunk --disable-universalsdk --disable-framework --disable-toolbox-glue --with-pydebug And here's the results with "make test": 342 tests OK. 6 tests failed: test_argparse test_grp test_posix test_pwd test_socket_ssl test_urllib2_localnet 37 tests skipped: test_aepack test_al test_applesingle test_bsddb test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_epoll test_gdb test_gdbm test_gl test_imgfile test_largefile test_linuxaudiodev test_macos test_macostools test_ossaudiodev test_readline test_scriptpackages test_smtpnet test_socketserver test_startfile test_sunaudiodev test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 8 skips unexpected on darwin: test_aepack test_applesingle test_gdb test_macos test_macostools test_readline test_scriptpackages test_ttk_guionly Why is the skip of "test_readline" unexpected on darwin? The readline on Darwin isn't what Python wants. Bill

R. David Murray <rdmurray <at> bitdance.com> writes:
The machine is likely lacking the readline development headers. In any case, please don't focus on the skipped tests. What's important is the failed tests. Bill, Michael, it would be nice if you could investigate a bit more on these failures. Regards Antoine.

Antoine Pitrou <solipsis@pitrou.net> wrote:
Well, test_grp test is failing because it assumes that group IDs are unique, which is not guaranteed by getgrent(). And apparently not true at PARC. test_posix fails on some group-related thing, too: the use of initgroups(), which is supposed to fail, succeeds. test_pwd fails because the fake uid chosen for one test is a real UID for a PARC user. So these are really problems with the test being based on an insufficiently general model of the real world. Bill

On Thu, 29 Apr 2010 17:27:44 -0700, Bill Janssen <janssen@parc.com> wrote:
Could you file bugs for these, please? The first and last others should be able to replicate easily; the middle one will probably require your help on the machine in question to debug. -- R. David Murray www.bitdance.com

On 29 Apr, 2010, at 20:28, Bill Janssen wrote:
...
The skip of test_readline is unexpected because readline should have been build with the options above and tests should have worked. Ronald

Martin v. Löwis wrote:
I seem to recall there was some issue (aside from the current lack of a reliable OS X buildbot) that prevented us from regularly grabbing the head of the tree and using it to automatically build the Windows and Mac installers (to check that the installers could actually be created, preventing a repeat of the Mac situation with 2.7b1). Am I misremembering the discussion from last time this topic came up? Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On Thu, Apr 15, 2010 at 12:47:50AM +1000, Nick Coghlan wrote:
Making the Windows binary build process automatic and robust is challenging if you hate Windows (like I do). Martin also made the point that it's been broken forever, so people don't seem to care :). ISTR Martin just makes them manually. It's on my TODO list to robustify this process. OTOH, my Mac automated builds/tests are working just fine. I could produce nightly binaries from those easily enough: http://lyorn.idyll.org/ctb/pb-dev/p/python/ Just need to figure out the magic doohickey commands... will add to list. cheers, --titus -- C. Titus Brown, ctb@msu.edu

That's true. In particular, people had been asking that the MSI installers are code-signed, which I now also do. I wouldn't like to keep the private key for the PSF code signing certificate available to some machine that is readily accessible from the internet - so at least that part is manual, in the sense that I have to provide my password. Regards, Martin

I only remember a similar discussion, which was about why the Windows nightly builds had been dropped. The reason was that the process was too flaky and required too much manual fixing, and that the demand for these binaries was too low. I don't recall OSX being mentioned in that discussion, though. Regards, Martin

Martin v. Löwis wrote:
Yep, that sounds like the discussion I was thinking of. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On 14/04/2010 06:13, Ned Deily wrote:
The problem is the process that creates a new release with a 404 link to the Mac installer with no explanation. The 2.6.5 release (as always) caused several requests to webmaster from Mac users unable to download Python - which is a further waste of volunteer time as well as a cause of frustration for users. Building the Mac installer requires volunteer time which I'm not sure that more hardware will fix - compiling a full build of Python for Mac OS X (with all the Python modules like Tkinter etc) requires expertise which only a few people have. A Mac OS X machine (and location to keep it) for the buildbots is a *big* need however. All the best, Michael -- http://www.ironpythoninaction.com/

On Apr 14, 2010, at 10:56 AM, Michael Foord wrote:
As the RM, that's my fault then. When I start creating the page for a new release I will leave the Windows and Mac links commented out. I do (now <wink>) wait for Martin to upload the Windows installers before announcing the release, but generally don't wait for Ronald to produce the Mac images, so I leave those commented out too. When Ronald builds the Mac image and provides it to me, I'll upload them and tweak the page. -Barry

On 14/04/2010 13:46, Barry Warsaw wrote:
Can we amend that to having some placeholder text saying that the Mac installer is not yet available and a link to the previous available version please. That can then be replaced with the normal link once the Mac installer is uploaded. Thanks Michael Foord
-Barry

On Apr 14, 2010, at 02:45 PM, Michael Foord wrote:
You mean, on the x.y.z release page, or a separate page? I guess you're saying that it's better to include some "not available yet" text on the x.y.x release page than to just comment out the platform. That makes sense, and I'd be happy to do that, since I already make such a caveat in the release announcement. -Barry

On 14/04/2010 13:58, Barry Warsaw wrote:
Yes, I mean on the release page. The issue is that the download links on the sidebar / front page go straight to the latest release page. If there isn't yet a Mac installer available, and no alternative link to get the previous version, it leaves Mac users with no obvious way to get a Mac installer *at all*. (Those who know the site well can find the previous versions themselves, but for users not intimately familiar with our site layout it is 'a challenge'.) Thanks Michael
-Barry

Michael Foord wrote:
I would actually use the "source only" release pages as a guide here. They provide a link to the download page for the last version that provided Mac and Windows binaries. This makes it clear to the downloader that the binaries are for an older version, while still making a binary version easy to find. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

On 14/04/2010 17:36, Bill Janssen wrote:
Well - an XServe that we can run virtualisation on would be the *ideal* solution. I think the X serves are the only machines you are *allowed* to virtualise OS X on (?). All the best, Michael
Bill

On 14/04/2010 17:41, Michael Foord wrote:
An alternative that solves the problem of finding somewhere to put the machine would be to use a service like Mac Mini colocation hosting: http://www.macminicolo.net/ That could be used both for debugging OS X specific problems *and* as a buildbot. All the best, Michael

Much of it is, but it still takes expertise to run the script. It would take even more expertise to capture the remaining pieces in the script, too, and no living person has that much expertise to write the script (perhaps there are one or two people, and they don't have the time). Regards, Martin

Martin v. Löwis <martin@v.loewis.de> wrote:
Apparently not, according to Ronald's recent mail. I'd be happy to help with whatever remaining capture is necessary for OS X -- not automating this kind of thing is nuts. My buildbots build UpLib, and most of its optional packages (including ghostscript, xpdf, libtiff, openssl, t1lib, Lucene, PyLucene, etc.) from source every night, and whenever I do an UpLib check-in. On Linux and OS X, right now -- I'm working on Windows right now.
Well, God forbid they should ever be hit by a bus! This kind of thing needs to be written down. Bill

Bill Janssen wrote:
+1 regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Martin v. Löwis wrote:
I take it you don't mean to imply that there's a dead person somewhere with the necessary expertise? [ducks]. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

Michael Foord wrote:
How about as a first step the release build process include a check for broken links before committing the web content for a new release? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

On 14/04/2010 19:25, Steve Holden wrote:
Yes - needed but orthogonal. :-) Michael
regards Steve

How about as a first step the release build process include a check for broken links before committing the web content for a new release?
You'd have to convince the release manager to add a step to the release process. Given that the release process has already too many steps, he is probably skeptical wrt. such a proposal. Regards, Martin

Why is it unavoidable that the Mac build will languish behind others?
Because of lack of volunteers, and expertise (i.e. the experts lack time).
Are we supporting MacOs or aren't we?
We aren't. Strictly speaking, "we" (python-dev) "support" nothing (in the sense that "we" can promise a support hotline, service contracts, or anything in that respect). That includes timely availability of stuff. If you want Python "support" on MacOS, contact Apple, ActiveState, or any other company that provides actual support. In a wider sense of "to support", MacOS is certainly supported by Python. There is everything in the source code that you need to make Python run on a Mac. Just download the sources and compile them yourself.
Clearly it's not a priority given that nobody has seen fit to (or had time to) reply to this mail in three weeks.
That is not surprising: none of the webmaster people would be able to answer the question. python-dev is indeed the right place to ask. Regards, Martin

Martin v. Löwis wrote:
OK. I should confirm that I mean "support" in your wider sense below.
And yet we don't regard the Windows release as complete until you have built the binaries (for which service you deserve many thanks, by the way). Is the Mac platform one on which users will be happy to compile from source? I know its users are savvier than Windows users, and have a better tool set available to them, but they still seem to expect downloadable installers.
I thought I'd picked this thread off python-dev. What point am I not understanding here? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

This phenomenon exists for a lot of other systems, as well. For example, we also support Solaris, but stopped providing Solaris binaries since Python 1.5 (when I last built binaries for Das Python-Buch). People still can get Solaris binaries from ActiveState or Sunfreeware; Sun also ships Python as part of the system.
The major difference in the "do it yourself" attitude is that Mac user get a compiler for free, as part of the operating system release, whereas for Windows, they have to pay for it (leaving alone VS Express for the moment). However, the real difference is motivation for contribution to open source projects. You normally contribute to scratch an itch. Unfortunately, these binaries don't come out such a motiviation. So the release manager roles are either altruistic, or rely on extrinsic motivations (money, reputation).
Maybe I misunderstood. I thought you copied it from the webmaster list (as the original To indicated). I don't recall having it seen on python-dev, but I kill many messages to python-dev without reading them. Regards, Martin

On 14 April 2010 07:37, Paul Rudin <paul@rudin.co.uk> wrote:
I believe that the express editions don't include some of the advanced optimisations (profile guided optimisation rings a bell) which are used to build the official binaries. So if the binaries were built using Express Edition, they would be somewhat slower. That is just my recollection, however - it may be out of date or wrong. Paul.

Paul Moore wrote:
I spent some considerable effort last year ensuring the developer community was well-supplied with MS developer licenses that give access to any necessary tools. Was I wasting my time? regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

On Apr 14, 2010, at 01:36 PM, Steve Holden wrote:
At the time I didn't care because I had no access to anything Windows. Now I do, though it's a bit of a PITA (because I have to reboot my main dev box). How can I get a license and the tools? Installing the express version was pretty easy, though it has limitations of course. -Barry

On Wed, Apr 14, 2010 at 12:36, Steve Holden <steve@holdenweb.com> wrote:
I certainly hope not. I believe full MSDN licenses were involved there, which are a great resource to a Windows developer. I'll have a whack at building the Windows installer. It would be nice to see it integrated with the build process, including testing of an installed version. We already have one or more Linux buildbots which test an installed build, and that setup uncovered an additional bug in a recent fix of mine.

On 14 April 2010 18:36, Steve Holden <steve@holdenweb.com> wrote:
Definitely not - my offer is at least in part based on the fact that I have the full tools as a result and so can do (hopefully) useful work on assisting with any necessary build process improvements. My comment was in response to the question "why are we ignoring the express editions"? If having the full versions wasn't better than having the free ones, the developer licenses would be of less benefit (and that isn't the case, as I say). Paul.

Paul Moore <p.f.moore@gmail.com> writes:
I could be wrong too. However I think I had two things confused (it's been a while since I've done any windows development). There are two different suites of free compilers from MS. There's the windows sdk, which includes a command line toolchain and the express edition of VS which (essentially) is a cut down version of the commercial version of VS. Some info about the windows sdk versions is here <http://msdn.microsoft.com/en-us/windows/dd146047.aspx> and about versions - including Express - here (at least for VC++) <http://msdn.microsoft.com/en-us/library/hs24szh9%28VS.90%29.aspx> although that's a little old.

Paul Rudin wrote:
Primarily out of a historical perspective: I think "we" started providing Windows binaries primarily because some people bought the appropriate license (of Visual Studio, and Wise Installer). IMO, that's a stronger reason than mere convenience. As for available tools: I'm not sure how you would build an AMD64 release just with those tools. Regards, Martin

On Wed, Apr 14, 2010 at 14:03, "Martin v. Löwis" <martin@v.loewis.de> wrote:
Outside of hacking the registry and various config files, 64-bit builds can't be done with the express versions of Visual Studio, and I wouldn't be comfortable shipping a product built in that way.

On Wed, Apr 14, 2010 at 07:36:25AM +0200, "Martin v. L?wis" wrote:
I personally think the Mac is pretty important, as one of the big three consumer operating systems...
Actually, I think the more pernicious factor is that a version of Python comes pre-installed on Mac OS X, which means the up-front demand is lower for a pre-compiled version. This is problematic, though, because that version of Python only gets upgraded with full releases of Mac OS X (which are not very well correlated with releases of Python, of course). So we have lots of Python installs out there that, in the absence of a precompiled binary version, can't be upgraded without installing the developer tools.
I don't know what to do about motivation but if there are barriers that we can lower, please let me know. cheers, --titus -- C. Titus Brown, ctb@msu.edu

For example, you could volunteer to produce OSX binaries in a timely manner for the next two years. Now would be a good point to do so, since your next release would still be a beta release. Expect to mess up the first time you do it, so it's good that it would be only a beta. Regards, Martin

On 14/04/2010 07:17, Steve Holden wrote:
Mac users definitely *do* expect installers. Building Python requires, I believe, the XCode development tools to be installed. Even then, building a full version of Python - with *all* the C extensions that are part of a Python release - is not a trivial task. All the best, Michael Foord

Michael> Mac users definitely *do* expect installers. Building Python Michael> requires, I believe, the XCode development tools to be Michael> installed. XCode is free, and I suspect many people have it (I do). Michael> Even then, building a full version of Python - with *all* the C Michael> extensions that are part of a Python release - is not a trivial Michael> task. Isn't that just a matter of having the recipe written down somewhere? Skip

On 14/04/2010 18:01, skip@pobox.com wrote:
Sure - but probably not your average Python-on-Mac user. Or at least a good proportion of them, particularly newbies who we are keen to keep the experience of obtaining Python simple. First download and then install 1gigabyte of developer tools (seriously) requiring registration, then compile Python from source yourself, is not the way to go. (And yes the XCode tools are included in the OS disks - but not the latest versions, especially if you are still running Leopard.)
Yes, that would be nice. :-) Preferably a recipe that doesn't involve Macports or Fink which some of us are allergic to. Michael
Skip

Michael Foord <fuzzyman@voidspace.org.uk> wrote:
Yes, ditto the MacPorts/Fink allergy. All we need is a script, right? The released branches should be built automatically every night anyway, just for regression testing. Perhaps we should take this up on Mac-Python? Bill

Bill Janssen wrote:
Please do, and let me know if resources of some kind would help. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See PyCon Talks from Atlanta 2010 http://pycon.blip.tv/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/

On 14/04/2010 18:49, Steve Holden wrote:
A Mac buildbot would *definitely* be useful and I know some of the Python-dev crew would like access to a Mac OS X machine for trying out fixes when none of the core Mac maintainers are around. A couple of co-located, or otherwise hosted, Mac boxes would be very useful. The obvious question is who would hold the keys (maintain the buildbot)? I don't know if Ronald has the spare capacity to do this (?). If someone will help me with the initial setup I can otherwise volunteer. All the best, Michael
regards Steve

On Wed, Apr 14, 2010 at 06:52:46PM +0200, Michael Foord wrote:
We have some space in our machine room, and some sysadmins that like open source. I will ask them if they are willing to do physical maintenance (profs wisely aren't allowed access to the machine room). That would really be ideal... I will report back to interested people. --titus -- C. Titus Brown, ctb@msu.edu

On Wed, Apr 14, 2010 at 09:37:34AM -0700, Bill Janssen wrote:
Please pass it along to me, when you get it working... I build python2.7 nightly on Mac OS X, but just at the command line. see: http://lyorn.idyll.org/ctb/pb-dev/p/python/ http://lyorn.idyll.org/ctb/pb-dev/p/python/show_all (the Windows build is flaky for me, so the 'show_all' shows mostly Windows builds). --titus -- C. Titus Brown, ctb@msu.edu

On 14/04/2010 20:21, "Martin v. Löwis" wrote:
Right - but we were discussing this in the context of barrier to entry, particularly to new users. We don't impose this requirement for Windows users though - we provide binary installers. I *know* we're a volunteer organisation (etc), but it is good for us to be aware of our process weaknesses without excusing ourselves. Unfortunately the Mac installer build script doesn't seem to run at all on Mac OS X 10.6 (at least not on my machine), but hopefully the situation is clarified so that one of us who does still have Mac OS X 10.5 will be able to build the installer in a timely manner for the next release should Ronald be too busy. All the best, Michael
Regards, Martin

I think nobody here is questioning that it is desirable to have OSX binaries. And I readily admit that this is a weakness.
I'm not sure whether 10.5 would be sufficient - it may be that you need to go back to 10.4 (*). Unfortunately, Apple manages to break compatibility and portability with every release, which makes this particular build task soooo tricky. You have to make all kinds of decisions and compromises where are really difficult to keep track of. Regards, Martin (*) I know that this is the version I had to go back to when I created OSX binaries for 2.5.x.

On 14/04/2010 21:37, "Martin v. Löwis" wrote:
In an earlier email Ronald said:
I can't verify that this is correct, I can verify it that the build-installer script doesn't appear to work under 10.6. Michael

On Apr 14, 2010, at 3:37 PM, Martin v. Löwis wrote:
I'm not sure whether 10.5 would be sufficient - it may be that you need to go back to 10.4 (*).
I think you just need to supply to configure MACOSX_DEPLOYMENT_TARGET=10.4 and have the appropriate SDK installed with Xcode. I believe it is installed by default with Xcode on Leopard (10.5), but with Xcode on Snow Leopard (10.6) the 10.4 SDK is an optional install.
Unfortunately, Apple manages to break compatibility and portability with every release, which makes this particular build task soooo tricky. You have to make all kinds of decisions and compromises where are really difficult to keep track of.
Hmm. Apple provided compatibility SDK and documented it. The only compromise I see in this build process right now is that we are building a Panther (10.3) compatible installer, while Mac OS X is a certified UNIX starting with 10.5. So, we are deliberately using obsolete model to provide backward compatibility. However, I'm fine with that for the installer. Power users can compile their own build optimized for their platform. Regards, Zvezdan

On Apr 14, 2010, at 4:40 PM, Martin v. Löwis wrote:
I was replying to your point about 10.4 build. Naturally, if you want a 10.3 build you'd pass 10.3 as the target and would have to have appropriate Xcode SDK installed.
Yes. x86_64, i386, and ppc are supported even in the Xcode supplied with the latest Mac OS X 10.6. Only ppc64 is not. So, ppc is not an issue. The problem is that enforcing backward compatibility with 10.3 and 10.4 makes 64-bit Intel architecture not feasible. You are right, it is a compromise. We are making more users happy by providing a 32-bit installer for a wider range of OS releases. However, if we want a more modern certified UNIX, 64-bit installer, then we'll have to draw a line and stop supporting older OS releases. Just as we stop supporting older releases of Python. Regards, Zvezdan

On 15 Apr, 2010, at 0:12, Zvezdan Petkovic wrote:
You don't have to install an SDK to be able to build binaries that run on older versions. The reason the binary installer gets build with the 10.4u SDK is that the default compiler on OSX 10.4 cannot build universal binaries without that SDK.
No. It is possible to build a binary that supports ppc, ppc64, x86 and x86_64, and that's even possible using a single additional configure switch. That binary will require OSX 10.5 to run though due to using symbols that aren't available on earlier versions of OSX (thanks to the better UNIX API compatibility in 10.5). PPC64 is not supported on OSX 10.6 though.
I want to provide 2 installers for Python 2.7 and 3.2: 1) The current 32-bit only installer that runs on OSX 10.3 or later 2) An installer that supports ppc, x86 and x86_64 and requires OSX 10.5 or later The latter would be the one that most users would want to use. Note that the second installer does not support ppc64 and three reasons: (1) PPC64 is a dead end on OSX, (2) libffi has issues on darwin/ppc64 that probably affect ctypes and (3) I do not have regular access to ppc64 machines and can therefore not provide any support for that platform. Ronald

Martin v. Löwis wrote:
Even on Linux, it takes a bit of fiddling. I finally remembered to write the steps down for Kubuntu when I set up my current machine (you need to apt-get half a dozen or so additional dev packages): http://boredomandlaziness.blogspot.com/2010/01/kubuntu-dev-packages-to-build... Take away the convenience of apt-get and add installing the compiler and installer toolchains and you've got a fair amount of setup time on your hands (and lots of things that can go wrong). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Michael Foord wrote:
What's non-trivial about it? I usually find that the normal "./configure; make; make install" sequence works fine without any further intervention. If you want a framework installation you have to read the README and use a couple of extra options, but it still works very smoothly. -- Greg

On 14/04/2010 23:32, Greg Ewing wrote:
A build on my machine produces output similar to: Python build finished, but the necessary bits to build these modules were not found: _bsddb dl gdbm imageop linuxaudiodev ossaudiodev readline spwd sunaudiodev To find the necessary bits, look in setup.py in detect_modules() for the module's name. Failed to build these modules: _tkinter Obviously many of those are not meant to be built and I usually build Python for running the test suite - so I don't care about not having Tkinter. A new user of Python would most certainly care about not having Tkinter. Michael -- http://www.ironpythoninaction.com/

In article <4BC63599.5020005@voidspace.org.uk>, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
third-party (Sleepycat) library needed (see the installer script)
dl
only available for 32-bit
gdbm
third-party library needed (not supplied in the installer build)
imageop
only available for 32-bit (and removed in 3.x)
linuxaudiodev ossaudiodev
neither supported on OS X
readline
requires either GNU readline lib (not included with OS X) or (with 2.6.5, trunk, or py3k) can now use OS X editline (libedit) replacement when targeting for 10.5 or 10.6 (add MACOSX_DEPLOYMENT_TARGET=10.5 or 10.6 to ./configure arguments).
spwd sunaudiodev
neither supported on OS X
Failed to build these modules: _tkinter
currently only supported for 32-bit archs as there was no 64-bit (non-X) Tk on OS X prior to 10.6 and there are unresolved problems with the 10.6 Apple-supplied Tk 8.5 and 2.6.x IDLE (Issue6864). -- Ned Deily, nad@acm.org

Whilst making Python easier to build on the Mac is certainly a worthy goal, the point of my post was to demonstrate (in reply to an email by Greg Ewing) *why* building a *full* Python from source was non-trivial. I personally only build Python from source to test changes to core-Python and am happy with the Python binary installers for the Mac - once they arrive. :-) All the best, Michael On 15/04/2010 14:18, Ned Deily wrote:

On 18/04/2010 15:13, Ronald Oussoren wrote:
10.6.3 and yes I have Tcl and Tk in /Library/Frameworks. How do I determine which versions they are? All the best, Michael
Ronald
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

Michael Foord <fuzzyman@voidspace.org.uk> writes:
10.6.3 and yes I have Tcl and Tk in /Library/Frameworks. How do I determine which versions they are?
You can use "info patchlevel" in tclsh - assuming you're running a tclsh linked to your /Library version (a normal Tcl install puts this in /usr/local/bin I think). Or, tcl.h (in the Headers folder beneath the framework install) has TCL_VERSION and TCL_PATCH_LEVEL defines near the top of the file. Given that your error is a failure to build and not a skip, it sounds like setup is finding Tcl/Tk. From a quick glance, it looks like tkinter may also require the X11 headers (you'd have to have installed X11 separately) - do you have output in your log from what exactly is failing when that module attempts to build? -- David

On 24/04/2010 21:34, David Bolen wrote:
$ tclsh % info patchlevel 8.5.7
Hmmm... looks like a 32 / 64 bit issue, which I believe may be the expected result when trying to build on Snow Leopard (?). i686-apple-darwin10-gcc-4.2.1: -framework: linker input file unused because linking not done i686-apple-darwin10-gcc-4.2.1: Tk: linker input file unused because linking not done ld: warning: in /Library/Frameworks//Tcl.framework/Tcl, missing required architecture x86_64 in file ld: warning: in /Library/Frameworks//Tk.framework/Tk, missing required architecture x86_64 in file *** WARNING: renaming "_tkinter" since importing it failed: dlopen(build/lib.macosx-10.4-x86_64-2.7-pydebug/_tkinter.so, 2): Symbol not found: _TclFreeObj Referenced from: /compile/python-trunk/build/lib.macosx-10.4-x86_64-2.7-pydebug/_tkinter.so Expected in: dynamic lookup I think my Tk/Tcl install came from an Activestate installer. Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

Michael Foord <fuzzyman@voidspace.org.uk> writes:
Hmmm... looks like a 32 / 64 bit issue, which I believe may be the expected result when trying to build on Snow Leopard (?).
I think so - I haven't tried a 64-bit build myself, but there's a comment in setup.py indicating that none of the Tcl/Tk framework builds support 64-bit. So I suppose you'd have to build 32-bit if you wanted a working _tkinter. -- David

On 24/04/2010 21:50, David Bolen wrote:
Sorry for my ignorance - how do I force a 32 bit build? Michael
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

On 24/04/2010 22:16, Michael Foord wrote:
Ok, so the following configure command line works and successfully builds Tkinter: ./configure --prefix=/dev/null --with-pydebug --enable-universalsdk All the best, Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

In article <4BC697D2.4020200@v.loewis.de>, "Martin v. Lowis" <martin@v.loewis.de> wrote:
As Ronald pointed out, the installer build script does all of the dirty work of building the install disk image (the .dmg file), including downloading and building necessary third-party libraries. What isn't automatically checked at the moment is that the third-party Tk framework is in place during the build and that there isn't contamination from the build user's environment. There is a patch in Issue5651 to do much of that. It doesn't currently try to handle the possibility of MacPorts (/opt/local) and Fink (/sw) contamination. I believe that issue should be addressed by the resolution of Issue7713. Keep in mind that Python on OS X supports the standard OS X multi-architecture (Intel vs PPC and/or 32-bit vs 64-bit executables in the same file) and multi-version features (the current installer builds are fully supported on 10.6, 10.5, and 10.4 and "best-effort" supported on 10.3 as well although building is not supported on 10.3) as well as "Unix" shared library vs OS X framework shared library configurations. That leads to a rather imposing matrix of potential build systems vs potential target systems. For the most part, Apple provides excellent upward compatibility; downward is somewhat trickier. OS X 10.6 (Snow Leopard) has complicated matters by the change to preferring 64-bit building and executing where possible. For python builds, some build configurations that worked by default under 10.5 and earlier no longer do so on 10.6, primarily due to standard library modules that depend on APIs, in many cases deprecated ones, that are only available in 32-bit versions. The old Macintosh modules comprise the biggest group of these and, while they have been removed in 3.x, they still remain in 2.x. But there are some others as well, which explain most of the build issues Michael reported. There are also newer versions of some open source libraries in 10.6 which cause problems for multi-version builds: openssl, for one, and Apple's 64-bit version of Tk 8.5. None of these problems are insolvable but with the very limited resources (i.e. people time) we've had for working on these issues, and when there have been so many other more critical problems, it has been easier up to now to stick with building on 10.5 (or even on 10.4 which I test build occasionally). I think the single most important thing that can be done to help right now is to get a robust system of OS X buildbots going so that many of the critical problems can be detected up front rather than when one of us gets around to building and install testing the multi-arch and multi-version installers. Since there are so many potential configurations, some thought needs to go into which would be of the most value. I would say that the most important build configurations are: (1) 32-bit Intel/PPC framework on 10.5 (and eventually 10.6) targeted for 10.3 through 10.6 (the current installer config setup); and (2) 32-/64- Intel-only framework for 10.6; followed by (3) 32-/64- Intel/PPC framework for 10.5 and 10.6. Building the whole installer and testing on as many targeted systems as possible would be ideal but that adds more complexity to the automation (again, not insurmountable). But even just doing framework multi-arch builds, installs, and tests (using the appropriate options) on only the build systems themselves without building an installer or third-party libs would go a long way towards catching many, if not most, problems early. I'd be happy to supply more detailed suggestions for specific configuration parameters for those interested in setting up buildbots. (There may be some delay, though, as I will have limited time and Internet access for the next three weeks.) -- Ned Deily, nad@acm.org

Hmm. When I tried it last, it didn't do anything - it just crashed. I then had to fix it, and also arrange to meet a certain number of assumptions that it assumed but that had not been setup on my system. This is some years ago, so I don't recall details.
There is one build slave now up, contributed by David Bolen. Regards, Martin

On Thu, Apr 15, 2010 at 04:41, Ned Deily <nad@acm.org> wrote:
I know for me the reason I have never tried to help with building the binaries is I simply lack the expertise. I always build straight UNIX versions of Python under OS X, so I have no experience with framework builds (which is what the binary distribution is). Heck, I didn't even know about the build script until just now since it's such a pain to build. Plus I don't know if you can use a framework build from within your svn checkout for testing, so I REALLY don't bother building a framework. And to top it off I use the latest libraries of things to look for possible breakage. I am sure I am not the only person who has all of these barriers preventing them from using a framework build consistently.
And this is another reason why Ronald has always been the only person to build the OS X binary; it's complicated. Knowing how to get the proper universal framework with the right things is annoying. Plus making sure something works on multiple versions of OS X is a pain as I am sure very few of us have 10.6, 10.5, and 10.4 lying around. I mean if the build script was dead-simple run and forget I am sure many of us would be willing to create dmgs. But it will take someone who knows what is needed (sounds like Ned does and obviously Ronald does) to get it to that place. -Brett

On 15 Apr, 2010, at 6:36, Martin v. Löwis wrote:
That *is* trivial: use Mac/BuildScript/build-installer.py on OSX 10.5. Making sure that unrelated changes don't accidently break the OSX build can be non-trivial though... That doesn't work with the py3k trunk at the moment, I just noticed that framework builds are broken there. I've filed issue #8441 for that and am working on a fix. Ronald

Ronald Oussoren <ronaldoussoren@mac.com> writes:
For what it's worth, the trunk currently takes ~25 minutes on my relatively modest Mini, when run in parallel with some buildbot stuff, and with third-party sources already downloaded. More or less evenly split among third-party packages, the interpreter itself, and then docs/framework/disk image creation. -- David

On 14/04/2010 07:11, "Martin v. Löwis" wrote:
Why is it unavoidable that the Mac build will languish behind others?
Because of lack of volunteers, and expertise (i.e. the experts lack time).
That doesn't explain why we leave a broken link in place when we do major releases - for days usually (if not weeks) with no explanation to users. That part of the release process is broken and should be fixed. Putting some placeholder text on the release page instead of the broken link is barely any more effort at all. For the last 2.6.5 release I changed the text and removed the dead link myself. When the Mac installer was uploaded someone else put the link back. [snip...]
Which is why I sent the email onto python-dev. However, no-one responded until Steve. All the best, Michael

Steve> Why is it unavoidable that the Mac build will languish behind Steve> others? Are we supporting MacOs or aren't we? If we are, why Steve> isn't the creation of the build a part of the release process? Steve> Clearly it's not a priority given that nobody has seen fit to (or Steve> had time to) reply to this mail in three weeks. I'm not sure who normally creates the Mac distribution, perhaps Ronald Ousorren? It would appear that either Ronald (or whoever) has been unavailable or there was no coordination between the Mac and non-Mac folks involved in the release. I'm willing to give it a whirl (I have both a MacBook Pro and a PowerMac G5 at home - both running Max OSX 10.5.x (Leopard)) though I will almost certainly need a cheat sheet for the process. I normally treat my Macs as Unix boxes from a Python perspective so don't make framework builds. Skip

On Apr 14, 2010, at 10:51 AM, skip@pobox.com wrote:
From the RM perspective, what I would really like to see is updates to the release.py script to check dependencies and automate as much as possible, as well as updates to PEP 101 for any process steps that can't be automated. This goes for both Windows and OS X. -Barry
participants (23)
-
"Martin v. Löwis"
-
Antoine Pitrou
-
Barry Warsaw
-
Bill Janssen
-
Brett Cannon
-
Brian Curtin
-
C. Titus Brown
-
David Bolen
-
Eric Smith
-
Greg Ewing
-
Michael Foord
-
Ned Deily
-
Nick Coghlan
-
Paul Moore
-
Paul Rudin
-
R. David Murray
-
Ronald Oussoren
-
skip@pobox.com
-
Steve Holden
-
Trent Nelson
-
Tres Seaver
-
webmaster@python.org
-
Zvezdan Petkovic