Mailman 3 March 2011 - Python-Dev

Non-code changes on "old" branches
by Mark Hammond 31 Mar '11

31 Mar '11

Hi, There are a couple of changes I'd like to make and would like some guidance on policy: http://bugs.python.org/issue6498 is a documentation bug which exists in Python 2.6 and later. The patch in that bug touches the docs and a comment in one source file. Is it acceptable to push that change to the 2.6 branch, or should I start with 2.7? My request re .hgignore from yesterday didn't get any complaints, so I intend opening a bug, asking for review here and if I don't get objections in a day or so, pushing the change. This really should go all the way back to 2.5 even though that release has long been closed. Is it acceptable to push a change to .hgignore to the 2.5 branch? If not, where should I start with such a change? I ask these questions primarily as the dev guide tells me I should forward-port all changes - thus, I need to know the earliest versions I can use before I can even start the process... Thanks, Mark

2 1

Security implications of pep 383
by Michael Foord 30 Mar '11

30 Mar '11

Hey all, Not sure how real the security risk is here: http://blog.omega-prime.co.uk/?p=107 Basically he is saying that if you store a list of blacklisted files with names encoded in big-5 (or some other non-utf8 compatible encoding) if those names are passed at the command line, or otherwise read in and decoded from an assumed-utf8 source with surrogate escaping, the surrogate escape decoded names will not match the properly decoded blacklisted names. All the best, Michael Foord -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

10 20

Notification for buildbot builder changes for slaves?
by David Bolen 30 Mar '11

30 Mar '11

I was wondering if it might be possible to have a channel (message here, email to a list of slave owners or whatever) to mention when the set of builders for the slaves is getting adjusted? A new builder tree can burn a good deal of disk space in some cases for example (each tree adds in the neighborhood of .5GB for each of my Windows buildbots), and while it's generally not an issue, the recent conversion to hg actually exhausted my disk space on several slaves, for example. I'm not looking to get in the way of any process - just have a way to hear about such changes in advance when possible in case I need to make adjustments. Thanks. -- David

1 0

Replace useless %.100s by %s in PyErr_Format()
by Victor Stinner 30 Mar '11

30 Mar '11

Hi, I plan to replace all %.100s (or any other size, %\.[0-9]+s regex) by %s in the whole source code, in all calls to PyErr_Format(). And I would like your opinion. When Guido added the function PyErr_Format(), 13 years ago, the function was implemented using a buffer of 500 bytes (allocated on the stack). The developer was responsible to limit the argument fit into a total of 500 bytes. But 3 years later (2000), PyErr_Format() was patched to use a dynamic buffer (allocated on the heap). But since this change, PyErr_Format() doesn't support %.100s anymore (the 100 bytes limitation is just ignored), and it becomes useless and so no, it's no more (since 10 years) a "protection" against segmentation fault. But I would like to know if I have to do in all branches (3.1-3.3, or worse: 2.5-3.3), or just in 3.3? Because it may make merge harder (like any change only done in default). I would like to replace %.100s because there are no more reason to truncate strings to an arbitrary length. => http://bugs.python.org/issue10833 --- ... at the same time, Ray Allen wrote a patch to implement %.100s in PyUnicode_FromFormatV() (so PyErr_Format() will support it too). I would like to replace %.100s in PyErr_Format(), and then commit its patch. http://bugs.python.org/issue7330 Victor

3 4

utf-8 encoding in checkins?
by skip＠pobox.com 30 Mar '11

30 Mar '11

It seems that all checkin mails are utf-8-encoded. This makes it challenging to view checkin mails if you have a text-based mail reader. (I use VM within XEmacs. XEmacs doesn't seem to support utf-8 out of the box. I believe you need to add MULE to it.) Is there some reason such messages can't be encoded in ASCII unless a non-ASCII message body is detected? Skip

13 20

.hgignore including site-packages and scripts directories?
by Mark Hammond 30 Mar '11

30 Mar '11

I'm wondering if it is a reasonable idea to have .hgignore exclude all files from 'Lib/site-packages' and 'Scripts'? As I install packages into my source builds, a 'hg status' lists *many* files in both those directories forcing me to scroll up a number of pages to see files which have actually changed. IIUC, listing a directory in .hgignore doesn't preclude files from that directory being added to hg, and doesn't prevent files in those directories already under hg from being detected as changed. The only downside I can see if that if new files are added to those directories which should be added to hg, a simple "hg st" will not show it - someone must remember and explicitly add it. However, ISTM those files are already likely to be missed given the large amount of noise 'hg st' shows in that directory - the files are likely to be in the middle of a very long list which my brain will be trained to habitually skip over. The number of new files which legitimately need to be added to those directories seem so small that this risk seems worthwhile. Any thoughts? Mark

3 5

Information about how cpython in benchmarked
by Tennessee Leeuwenburg 30 Mar '11

30 Mar '11

Hi all, Apologies for emailing this list with such an apparently trivial question. Is there some source of documentation or information on how Python is benchmarked? I am aware of the Python regression testing module, regrtest.py, which I presume, if profiled, would good be a good baseline test. PyPy maintains http://speed.pypy.org/, which provides very clear information about the relative performance of PyPy trunk against some version of cpython (presumably 2.6 or 2.7). I'm not aware of a similar site for cpython, but that could easily just be my ignorance speaking. My interest is that I'm looking at building a benchmarking solution at work. and I can't think of a better way to build something good and general than to try and write something that could potentially be released as open source and be useful to others. As such I thought that benchmarking cpython would be a great use case, but I want to find out as much as I can about how people currently go about benchmarking Python. Initially I'm just looking at CPU profiling since it's easiest. Anyway, if this is the wrong place to send this email, I'm very sorry for clogging up your inbox. Thanks very much, -Tennessee

4 5

Failed issue tracker submission
by Python tracker 30 Mar '11

30 Mar '11

The node specified by the designator in the subject of your message ("22663") does not exist. Subject was: "[issue22663]" Mail Gateway Help ================= Incoming messages are examined for multiple parts: . In a multipart/mixed message or part, each subpart is extracted and examined. The text/plain subparts are assembled to form the textual body of the message, to be stored in the file associated with a "msg" class node. Any parts of other types are each stored in separate files and given "file" class nodes that are linked to the "msg" node. . In a multipart/alternative message or part, we look for a text/plain subpart and ignore the other parts. Summary ------- The "summary" property on message nodes is taken from the first non-quoting section in the message body. The message body is divided into sections by blank lines. Sections where the second and all subsequent lines begin with a ">" or "|" character are considered "quoting sections". The first line of the first non-quoting section becomes the summary of the message. Addresses --------- All of the addresses in the To: and Cc: headers of the incoming message are looked up among the user nodes, and the corresponding users are placed in the "recipients" property on the new "msg" node. The address in the From: header similarly determines the "author" property of the new "msg" node. The default handling for addresses that don't have corresponding users is to create new users with no passwords and a username equal to the address. (The web interface does not permit logins for users with no passwords.) If we prefer to reject mail from outside sources, we can simply register an auditor on the "user" class that prevents the creation of user nodes with no passwords. Actions ------- The subject line of the incoming message is examined to determine whether the message is an attempt to create a new item or to discuss an existing item. A designator enclosed in square brackets is sought as the first thing on the subject line (after skipping any "Fwd:" or "Re:" prefixes). If an item designator (class name and id number) is found there, the newly created "msg" node is added to the "messages" property for that item, and any new "file" nodes are added to the "files" property for the item. If just an item class name is found there, we attempt to create a new item of that class with its "messages" property initialized to contain the new "msg" node and its "files" property initialized to contain any new "file" nodes. Triggers -------- Both cases may trigger detectors (in the first case we are calling the set() method to add the message to the item's spool; in the second case we are calling the create() method to create a new node). If an auditor raises an exception, the original message is bounced back to the sender with the explanatory message given in the exception. $Id: mailgw.py,v 1.196 2008-07-23 03:04:44 richard Exp $ Return-Path: <python-dev(a)python.org> X-Original-To: report(a)bugs.python.org Delivered-To: roundup+tracker(a)psf.upfronthosting.co.za Received: from mail.python.org (mail.python.org [82.94.164.166]) by psf.upfronthosting.co.za (Postfix) with ESMTPS id 7DCEE1DEB0 for <report(a)bugs.python.org>; Tue, 29 Mar 2011 22:10:55 +0200 (CEST) Received: from albatross.python.org (localhost [127.0.0.1]) by mail.python.org (Postfix) with ESMTP id 3PzjsW1q1lz7Lmy for <report(a)bugs.python.org>; Tue, 29 Mar 2011 22:10:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901; t=1301429455; bh=WYL3NF6gQtbDZ+R9KxXHGS2PSlCAxyY+EQEgb/Yw5jI=; h=Date:Message-Id:Content-Type:MIME-Version: Content-Transfer-Encoding:From:To:Subject; b=RiMAivS4Shae7bPg7E7SocheqYB9pzk/Svimv+qumX5arnUaaC8h9iIJo8MFDcDdi +Wk0XzTjTjKsbobrKgZnfZf9a8j6Fv4Ym1nTyTcPcyjCMritjq9xNUluVQvHv/Vn2e RhpV2FUWOdCtBx83eUopMPGEEEwABnbG5ZwgsDzM= Received: from localhost (HELO mail.python.org) (127.0.0.1) by albatross.python.org with SMTP; 29 Mar 2011 22:10:55 +0200 Received: from dinsdale.python.org (svn.python.org [IPv6:2001:888:2000:d::a4]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.python.org (Postfix) with ESMTPS for <report(a)bugs.python.org>; Tue, 29 Mar 2011 22:10:55 +0200 (CEST) Received: from localhost ([127.0.0.1] helo=dinsdale.python.org ident=hg) by dinsdale.python.org with esmtp (Exim 4.72) (envelope-from <python-dev(a)python.org>) id 1Q4fFf-00023G-4C for report(a)bugs.python.org; Tue, 29 Mar 2011 22:10:55 +0200 Date: Tue, 29 Mar 2011 22:10:55 +0200 Message-Id: <E1Q4fFf-00023G-4C(a)dinsdale.python.org> Content-Type: text/plain; charset="utf8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 From: python-dev(a)python.org To: report(a)bugs.python.org Subject: [issue22663] TmV3IGNoYW5nZXNldCBkZDg1MmEwZjkyZDYgYnkgZ3VpZG8gaW4gYnJhbmNoICcyLjUnOgpJc3N1 ZSAyMjY2MzogZml4IHJlZGlyZWN0IHZ1bG5lcmFiaWxpdHkgaW4gdXJsbGliL3VybGxpYjIuCmh0 dHA6Ly9oZy5weXRob24ub3JnL2NweXRob24vcmV2L2RkODUyYTBmOTJkNgo=

2 1

Proposed change to logging.basicConfig
by Vinay Sajip 29 Mar '11

29 Mar '11

I'm planning a change to logging.basicConfig to add an optional "handlers" keyword argument which defaults to None. If specified, this should be an iterable of already created handlers, which will be added to the root logger (if it doesn't already have any handlers). Any handler in the iterable which does not have a formatter assigned will be assigned the formatter created by basicConfig. If "handlers" is specified, the "stream", "filename" and "filemode" arguments will be ignored. If any of you can see any problems with this change, or can suggest any improvement to the approach, please respond. I expect to check this change in within the next few days. Regards, Vinay Sajip

4 5

Re: [Python-Dev] Security implications of pep 383
by Laura Creighton 29 Mar '11

29 Mar '11

In a message of Tue, 29 Mar 2011 19:23:25 BST, Michael Foord writes: >Hey all, > >Not sure how real the security risk is here: > > http://blog.omega-prime.co.uk/?p=107 > >Basically he is saying that if you store a list of blacklisted files >with names encoded in big-5 (or some other non-utf8 compatible encoding) >if those names are passed at the command line, or otherwise read in and >decoded from an assumed-utf8 source with surrogate escaping, the >surrogate escape decoded names will not match the properly decoded >blacklisted names. >All the best, > >Michael Foord > I am not sure there are any security related gotchas here. All he is saying is that if you decode the same bytestring using two different encodings, you will get two different unicode strings (which therefore will compare unequal). Where's the problem, except in that you might have unrealistic expectations? Laura

1 0