Mailman 3 August 2006 - Python-Dev

Community buildbots (was Re: User's complaints)
by glyph＠divmod.com Sept. 10, 2006

Sept. 10, 2006

On Wed, 12 Jul 2006 10:30:17 +1000, Michael Ellerman <michael(a)ellerman.id.au> wrote: >Well here's one I stumbled across the other day. I don't know if it's >legit, but it's still bad PR: > >http://www.gbch.net/gjb/blog/software/discuss/python-sucks.html Having been exhorted (or maybe I mean "excoriated") by your friendly release manager earlier this week to post my comments and criticisms about Python here rather than vent in random IRC chatter, I feel morally compelled to comment. I see some responses to that post which indicate that the specific bug will be fixed, and that's good, but there is definitely a pattern he's talking about here, not just one issue. I think there is a general pattern of small, difficult to detect breakages in Python. Twisted (and the various other Divmod projects that I maintain) are thoroughly unit-tested, but there are still significant issues in each Python release that require changes. Unlike the jerk who posted that "python sucks" rant, I'm not leaving Python because one function changed in a major release; I do expect to have to maintain my projects myself, and major releases are "major" for a reason. I only wish that upgrading all my other dependencies were as easy as upgrading Python! I do see that the developers are working with some caution to avoid breaking code, and attempting to consider the cost of each change. However, although I've seen lots of discussions of what "average" code might break when exposed to new versions of Python, these discussions tend to be entirely hypothetical. Do the core Python developers typically run the test suites of major Python applications when considering major changes, such as Zope, Plone, TurboGears, and of course Twisted? Not that breakages would be considered unacceptable -- some gain is surely worth the cost -- but to establish some empirical level of burden on the community? I would like to propose, although I certainly don't have time to implement, a program by which Python-using projects could contribute buildslaves which would run their projects' tests with the latest Python trunk. This would provide two useful incentives: Python code would gain a reputation as generally well-tested (since there is a direct incentive to write tests for your project: get notified when core python changes might break it), and the core developers would have instant feedback when a "small" change breaks more code than it was expected to. I can see that certain Python developers expect that some of this work is the responsibility of the user community, and to some extent that's true, but at least half of the work needs to be done _before_ the changes are made. If some Python change breaks half of Twisted, I would like to know about it in time to complain about the implementation, rather than flailing around once the Python feature-freeze has set in and hoping that it's nothing too serious. For example, did anyone here know that the new-style exceptions stuff in 2.5 caused hundreds of unit-test failures in Twisted? I am glad the change was made, and one of our users did catch it, so the process isn't fatally broken, but it is still worrying. Another problem is simply that the Python developers don't have the same very public voice that other language developers do. It doesn't necessarily have to be a blog, but python-dev is fast-paced and intimidating, and a vehicle for discussion among those in the know, rather than dissimenation to the community at large. It's a constant source of anxiety to me that I might miss some key feature addition to Python which breaks or distorts some key bit of Twisted functionality (as the new-style exceptions, or recent ImportWarning almost did) because I don't have enough time to follow all the threads here. I really appreciate the work that Steve Bethard et. al. are doing on the python-dev summaries, but they're still pretty dry and low level. While the new python.org is very nice, I do note that there's no "blogs" entry on the front page, something which has become a fixture on almost every other website I visit regularly. The news page is not very personal, mainly a listing of releases and events. There's no forum for getting the community _excited_ about new features (especially new features which are explicitly enabled by potential breakages), and selling them on the cool uses. Who knows, maybe I'll even start using decorators syntax all over the place if I see them presented somewhere by someone who is very excited about the feature and thinks it's worthwhile, rather than as a defense on a mailing list against a criticism, or a simple announcement of the feature's existence. I've seen the other side of this problem as well, so I can appreciate that it is quite difficult to get this kind of thing right: lots of applications using Twisted break when we change broken or deprecated APIs. Twisted is lucky though; it has numerous subprojects, and I maintain a half-dozen unrelated projects in a different repository. Twisted itself is a fairly representative "application using Twisted", so when something breaks "average" Twisted code, I know about it right away. Python is less an application using Python: the standard library is rather passive (there is no "end-to-end" application path to test, only individual functions and classes), and the test coverage of included Python modules is rather spotty. I hope someone finds some of these suggestions helpful. I only wish I had the time and energy to act on some of them.

29 85

Interest in a Python 2.3.6?
by Barry Warsaw Sept. 9, 2006

Sept. 9, 2006

I am considering producing a Python 2.3.6 release, which would of course only be a bug fix maintenance release. The primary reason is that not all OS distributions have upgraded to Python 2.4 and I think it's worthwhile for us to bless a release that fixes known critical bugs. I'm willing to be the release manager for 2.3.6, but I'm hoping you all will be able to help identify the most important bugs that need fixing. First, is there interest in getting a 2.3.6 release out? I'd propose keeping things simple, by picking a date and releasing what we've got at that date (assuming of course all the unit tests pass). It's probably a good idea to add a wiki page tracking the fixes we want to get in there. 2.5 final is slated for 12-Sep-2006 and I know that Anthony is planning for a 2.4.4 soon after that. I'm thinking that we'd try to do a 2.3.6 a couple of weeks after 2.4.4 so that people who care about it aren't stacked up with fixing too many branches at once. My first thought was to shoot for Monday October 9th. What are the potential 2.3.6 fixes? Nothing on this page: http://www.python.org/download/releases/2.3.5/bugs/ seems critical to me. I know that I've added some important email package fixes that are already in Subversion. There are tons of bugs and patches reported against 2.3 in SF (lament: why won't SF just / tell/ me how many there are?), but I have no idea atm which, if any, are important enough to go into a 2.3.6. I haven't yet done an svn diff to see what changes are already in there. I don't have the cycles to fix things myself, so it would be up to everyone to help commit fixes. I'll ride herd a bit if necessary. Thoughts? I don't want to waste my time if nobody thinks a 2.3.6 would be useful, but I'm happy to do it if there's community support. I'll also need the usual help with Windows installers and documentation updates. Cheers, -Barry

4 6

new security doc using object-capabilities
by Brett Cannon Sept. 7, 2006

Sept. 7, 2006

After various people suggesting object-capabilities, takling with Mark S. Miller of the E programming language, and the people Mark works with at HP Labs (who have been giving talks every week during this month here at Google on object-capabilities), I have decided to go with object-capabilities for securing interpreters. I have rewritten my design doc from scratch and deleted the old one. The new doc is named securing_python.txt and can be found through the svn web interface at http://svn.python.org/view/python/branches/bcannon-sandboxing/securing_pyth…. I have pretty much ignored any concrete API and such and gone more with a conceptual doc to make sure the API does not get in the way of the core security model. Using object-capabilities should make the implementation much cleaner. There is much less work directly on the interpreter and more of it gets pushed up to extension modules. I also have the okay of my supervisor to use this approach in my dissertation so this will get done. Two things do fall out of all of this which will make development much more modular and easier. First, the memory cap work just becomes a special build on its own; no need to tie into the security work. So I will be cleaning up the bcannon-sandboxing branch code as it stands, and then either create a separate branch for the object-capabilities work, or create another branch for the memory cap stuff and shift the changes over there. I will most likely do the former so as to not lose the history on the checkins. I also plan to rewrite the import machinery in pure Python. This will make the code much more maintainable and make creating proxies for the import machinery much easier. I will be doing that in a directory in the sandbox initially since it needs to work from what Python has now (and possibly some new extension module code) before it can be integrated into the interpreter directly. Anyone who wants to help with that can. I already have some perliminary notes on the whole thing and I think it will be reasonably doable. Anyway, there you go. Here is to hoping I have thought this all through properly. =) -Brett

10 27

gcc 4.2 exposes signed integer overflows
by howarth＠bromo.msbb.uc.edu Sept. 5, 2006

Sept. 5, 2006

I discovered that gcc 4.2 exposes a flaw with signed integer overflows in python. This bug and the necessary fix has been discussed in detail on the gcc mailing list. I have filed a detailed bug report and the recommended patch proposed by the gcc developers. This problem should be addressed BEFORE python 2.5 is released. The bug report is... [ 1545668 ] gcc trunk (4.2) exposes a signed integer overflows in the python sourceforge bug tracker. Thanks in advance for attempting to fix this before Python 2.5 is released. Jack

7 13

Error while building 2.5rc1 pythoncore_pgo on VC8
by Muguntharaj Subramanian Sept. 4, 2006

Sept. 4, 2006

Hi All, I tried building python 2.5c1 using VC8. Getting the following errors while building pythoncore_pgo: Creating library pythoncore_pgo/python25.lib and object pythoncore_pgo/python25.exp config.obj : error LNK2001: unresolved external symbol _init_types .\pythoncore_pgo/python25.dll : fatal error LNK1120: 1 unresolved externals I checked pythoncore_pgo.vcproj file, it alreadu contains: RelativePath="..\Modules\_typesmodule.c" Can anyone give me some clue to solve this issue. Thanks & Regards, Mugunth http://mugunth.blogspot.com http://thamizha.com http://tamilblogs.com

3 4

Py2.5 issue: decimal context manager misimplemented, misdesigned, and misdocumented
by Raymond Hettinger Sept. 2, 2006

Sept. 2, 2006

I would like to see the changes to the decimal module reverted for the Py2.5 release. Currently, the code in the decimal module implements the context manager as a separate class instead of incorporating it directly in decimal.Context. This makes the API unnecessarily complex and is not pretty compared to the code it was intended to replace. Worse still, the implementation saves a reference to the context instead of making a copy of it. Remember decimal.Context objects are mutable -- the current implementation does not fulfill its contract to restore the context to its original state at the conclusion of the with-statement. The right way to do it was presented in PEP343. The implementation was correct and the API was simple. Additionally: * The examples in WhatsNew don't work because the implementation uses a different method name to fetch to context (this is a shallow error except that the name in WhatsNew is better and we don't really want to have a new method for this). It doesn't bode well that none of the release candidate end users noticed this discrepancy -- it means they are not trying out the examples. * The implementation's doc string examples were not tested and don't work (this is a deep error). One reads: with decimal.getcontext() as ctx: ctx.prec += 2 s = ... return +s To get this to work with the current implementation, it should read with decimal.getcontext().copy().get_manager() as ctx: ctx.prec += 2 s = ... return +s This is horrid. Please either revert the patch or fix it to match PEP-343. Raymond

4 14

That library reference, yet again
by Johann C. Rocholl Sept. 1, 2006

Sept. 1, 2006

What is the status of http://effbot.org/lib/ ? I think it's a step in the right direction. Is it still in progress? Can I have access to toolchain source code and/or conversion output? I would like to set up something like gotapi.com for the Python library reference, or help the gotapi people to include it. But gotapi needs link targets for functions etc., but the original Python library reference doesn't have them. I really like the simple linking idea here: http://online.effbot.org/2006_01_01_archive.htm#link-doc Here's another working example with (ugly) link targets: http://www.zvon.org/other/python/PHP/search.php Cheers, Johann

3 2

link to community buildbot?
by skip＠pobox.com Aug. 31, 2006

Aug. 31, 2006

There was a buildbot set up recently to test various external packages (like twisted) against Python's source code. I don't recall where it is, but it would be nice if relevant links to it existed on this page: http://www.python.org/dev/buildbot/ In addition, if a 2.3.6 release is going to happen it seems like it would be a good idea to set up the relevant bits on both buildbots. Skip

2 1

Windows build slave downtime
by Alan McIntyre Aug. 31, 2006

Aug. 31, 2006

Hi all, The Windows build slave ("x86 XP trunk") will be down for most of today while I move to a new apartment. As long as the phone company manages to provide me with an internet connection by the end of today, it should be available again some time this evening. Alan

1 0

Re: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8
by Kristján V. Jónsson Aug. 31, 2006

Aug. 31, 2006

I see. There is a file, called pythoncore_pgo_link.txt where you have to add the object too. But pythoncore_pgo is a bit broken in other ways at the moment. I am working on making it better in the trunk. (I also think that MS should improve their tools to make PGO building a one step process. the current recommended solution involves two project configurations...) Kristján ________________________________ From: python-dev-bounces+kristjan=ccpgames.com(a)python.org [mailto:python-dev-bounces+kristjan=ccpgames.com@python.org] On Behalf Of Muguntharaj Subramanian Sent: 31. ágúst 2006 11:46 To: Fredrik Lundh Cc: python-dev(a)python.org Subject: Re: [Python-Dev] Error while building 2.5rc1 pythoncore_pgo on VC8 > I checked pythoncore_pgo.vcproj file, it alreadu contains: > > RelativePath="..\Modules\_typesmodule.c" > > Can anyone give me some clue to solve this issue. googling for error messages is always a good idea: http://thread.gmane.org/gmane.comp.python.devel/83341 (see the last post in that thread for what I believe is a workaround) </F> Yes, I have already done a bit of searching before posting that message. I have seen that previous post. That error mentioned in that post was in "pythoncore" module. My error is while compiling "pythoncore_pgo" module. And the workaround solution mentioned there is already there in the pythoncore_pgo.vcproj file. Thats why i mentioned RelativePath="..\Modules\ _typesmodule.c" is already there. I am still getting the same error. I am not able to find the cause yet. Any clue regarding this. Thanks & Regards, Mugunth http://mugunth.blogspot.com http://thamizha.com http://tamilblogs.com

1 0