Mailman 3 April 2011 - Python-Dev

abstractmethod doesn't work in classes
by Eugene Toder 08 Apr '11

08 Apr '11

Hello, I've found that abstractmethod and similar decorators "don't work" in classes, inherited from built-in types other than object. For example: >>> import abc >>> class MyBase(metaclass=abc.ABCMeta): @abc.abstractmethod def foo(): pass >>> MyBase() Traceback (most recent call last): File "<pyshell#8>", line 1, in <module> MyBase() TypeError: Can't instantiate abstract class MyBase with abstract methods foo So far so good, but: >>> class MyList(list, MyBase): pass >>> MyList() [] >>> MyList.__abstractmethods__ frozenset({'foo'}) This is unexpected, since MyList still doesn't implement foo. Should this be considered a bug? I don't see this in documentation. The underlying reason is that __abstractmethods__ is checked in object_new, but built-in types typically call tp_alloc directly, thus skipping the check. Eugene

2 1

funky buildbot problems again...
by Bill Janssen 08 Apr '11

08 Apr '11

My Intel Snow Leopard 2 build slave has gone into outer-space again. When I look at it, I see buildslave taking up most of a CPU (80%), and nothing much else going on. The twistd log says: [... much omitted ...] 2011-04-04 08:35:47-0700 [-] sending app-level keepalive 2011-04-04 08:45:47-0700 [-] sending app-level keepalive 2011-04-04 08:55:47-0700 [-] sending app-level keepalive 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] Lost connection to dinsdale.python.org:9020 2011-04-04 09:03:15-0700 [Broker,client] <twisted.internet.tcp.Connector instance at 0x101629ab8> will retry in 3 seconds 2011-04-04 09:03:15-0700 [Broker,client] Stopping factory <buildslave.bot.BotFactory instance at 0x1016299e0> 2011-04-04 09:03:18-0700 [-] Starting factory <buildslave.bot.BotFactory instance at 0x1016299e0> 2011-04-04 09:03:18-0700 [-] Connecting to dinsdale.python.org:9020 2011-04-04 09:03:18-0700 [Uninitialized] Connection to dinsdale.python.org:9020 failed: Connection Refused 2011-04-04 09:03:18-0700 [Uninitialized] <twisted.internet.tcp.Connector instance at 0x101629ab8> will retry in 8 seconds 2011-04-04 09:03:18-0700 [Uninitialized] Stopping factory <buildslave.bot.BotFactory instance at 0x1016299e0> 2011-04-04 09:03:27-0700 [-] Starting factory <buildslave.bot.BotFactory instance at 0x1016299e0> 2011-04-04 09:03:27-0700 [-] Connecting to dinsdale.python.org:9020 So it's been spinning its wheels for 3 days. Sure looks like the connection attempt is failing, for some reason. I'm using the stock Twisted that comes with Snow Leopard -- tried to upgrade it but apparently can't. On my OS X 10.4 buildslave, I see a similar but more successful sequence: 2011-04-04 08:56:06-0700 [-] sending app-level keepalive 2011-04-04 09:04:39-0700 [Broker,client] lost remote 2011-04-04 09:04:39-0700 [Broker,client] lost remote 2011-04-04 09:04:39-0700 [Broker,client] lost remote 2011-04-04 09:04:39-0700 [Broker,client] lost remote 2011-04-04 09:04:39-0700 [Broker,client] lost remote 2011-04-04 09:04:39-0700 [Broker,client] <twisted.internet.tcp.Connector instance at 0x10352d8> will retry in 3 seconds 2011-04-04 09:04:39-0700 [Broker,client] Stopping factory <buildslave.bot.BotFactory instance at 0x133bd78> 2011-04-04 09:04:42-0700 [-] Starting factory <buildslave.bot.BotFactory instance at 0x133bd78> 2011-04-04 09:04:43-0700 [Uninitialized] <twisted.internet.tcp.Connector instance at 0x10352d8> will retry in 10 seconds 2011-04-04 09:04:43-0700 [Uninitialized] Stopping factory <buildslave.bot.BotFactory instance at 0x133bd78> 2011-04-04 09:04:53-0700 [-] Starting factory <buildslave.bot.BotFactory instance at 0x133bd78> 2011-04-04 09:04:57-0700 [Broker,client] message from master: attached Bill

4 6

Policy for making changes to the AST
by Eugene Toder 08 Apr '11

08 Apr '11

Hello, While working on a rewrite of peephole optimizer that works on AST (http://bugs.python.org/issue11549) I had to make a few changes to the structure of AST. The changes are described in the issue. This raises the question -- what is the policy for making changes to the AST? Documentation for ast module does not warn about possible changes, but obviously changes can occur, for example, when new constructs are introduced. What about other changes? Is there a policy for what's acceptable and how this should be handled? Assuming we do want to make changes (not just extensions for new language features), I see 3 options for handling them: 1. Do nothing. This will break code that currently uses AST, but doesn't add any complexity to cpython. 2. Write a pure-Python compatibility layer. This will convert AST between old and new formats, so that old code continues working. To do this a) Introduce ast.compile function (similar to ast.parse), which should be the recommended way of compiling to AST. b) Add ast_version argument to ast.parse and ast.compile defaulting to 1. c) Preserve old AST node classes and attributes in Python. d) If ast_version specified is not the latest, ast.parse and ast.compile would convert from/to latest version in Python using ast.NodeTransformer. This is not fully backward compatible, but allows to do all staging in Python. 3. Full backward compatibility (with Python code). This means conversion is done in compile(). It can either call Python conversion code from ast module, or actually implement conversion in C using AST visitors. Using my visitors generator this should not be very hard. Downsides here are a lot of C code and no clear separation of deprecated AST nodes (they will remain in Python.asdl). Otherwise it's similar to 2, with ast_version argument added to compile() and ast.parse. For 2 and 3 we can add a PendingDeprecationWarning when ast_version 1 is used. In any case, C extensions that manipulate AST will be broken, but 3 provides a simple way to update them -- insert calls to C conversion functions. Eugene

16 34

Force build form
by Antoine Pitrou 07 Apr '11

07 Apr '11

Hello, For the record, I've tried to make the force build form clearer on the buildbot Web UI. See e.g.: http://www.python.org/dev/buildbot/all/builders/x86%20OpenIndiana%20custom Regards Antoine.

4 3

Supporting Visual Studio 2010
by Brian Curtin 06 Apr '11

06 Apr '11

Would it be reasonable to begin supporting Visual Studio 2010 for Windows builds of 3.3? I now have a personal interest in this happening for some stuff at work, and there's been a lot of questions in the last few months about when we'll support it coming from python-list, #python-dev, and in person at PyCon. I wasn't around for the transition from 2005 to 2008, but I see we have a few sub-folders in PC/ for previous versions, so apparently we may support multiple versions at one time. Does it make sense to start this process now for a change to 2010? If it's not feasible to release 3.3 from a 2010 build, when might we be able to make the change? Keep in mind the 3.3 final release is almost a year and a half away, and we already know that Microsoft is likely to pull the cord on VS2008 Express at some point now that 2010 has been out for a while. I'm willing to do the work on this, but I just want to make sure it's a worthwhile effort.

9 18

clarification: subset vs equality Re: [Python-checkins] peps: Draft of PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements
by Jim Jewett 06 Apr '11

06 Apr '11

On 4/4/11, brett.cannon <python-checkins(a)python.org> wrote: > Draft of PEP 399: Pure Python/C Accelerator Module Compatibiilty > Requirements > +Abstract > +======== > + > +The Python standard library under CPython contains various instances > +of modules implemented in both pure Python and C. This PEP requires > +that in these instances that both the Python and C code *must* be > +semantically identical (except in cases where implementation details > +of a VM prevents it entirely). It is also required that new C-based > +modules lacking a pure Python equivalent implementation get special > +permissions to be added to the standard library. I think it is worth stating explicitly that the C version can be even a strict subset. It is OK for the accelerated C code to rely on the common python version; it is just the reverse that is not OK. -jJ

2 1

Buildbot status
by Antoine Pitrou 06 Apr '11

06 Apr '11

Hello, For the record, we have 9 stable buildbots, one of which is currently offline: 3 Windows, 2 OS X, 3 Linux and 1 Solaris. Paul Moore's XP buildbot is back in the stable stable. (http://www.python.org/dev/buildbot/all/waterfall?category=3.x.stable) We also have a new 64-bit FreeBSD 8.2 buildbot donated and managed by Stefan Krah. (http://www.python.org/dev/buildbot/all/buildslaves/krah-freebsd) Regards Antoine.

3 4

Re: [Python-Dev] [Python-checkins] cpython: Issue #11707: Fast C version of functools.cmp_to_key()
by Victor Stinner 05 Apr '11

05 Apr '11

I don't think that the following change conforms to the PEP 399: there is only a test for the C version. Victor Le mardi 05 avril 2011 à 11:34 +0200, raymond.hettinger a écrit : > http://hg.python.org/cpython/rev/a03fb2fc3ed8 > changeset: 69150:a03fb2fc3ed8 > user: Raymond Hettinger <python(a)rcn.com> > date: Tue Apr 05 02:33:54 2011 -0700 > summary: > Issue #11707: Fast C version of functools.cmp_to_key() > > diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py > --- a/Lib/test/test_functools.py > +++ b/Lib/test/test_functools.py > @@ -435,18 +435,81 @@ > self.assertEqual(self.func(add, d), "".join(d.keys())) > > class TestCmpToKey(unittest.TestCase): > + > def test_cmp_to_key(self): > + def cmp1(x, y): > + return (x > y) - (x < y) > + key = functools.cmp_to_key(cmp1) > + self.assertEqual(key(3), key(3)) > + self.assertGreater(key(3), key(1)) > + def cmp2(x, y): > + return int(x) - int(y) > + key = functools.cmp_to_key(cmp2) > + self.assertEqual(key(4.0), key('4')) > + self.assertLess(key(2), key('35')) > + > + def test_cmp_to_key_arguments(self): > + def cmp1(x, y): > + return (x > y) - (x < y) > + key = functools.cmp_to_key(mycmp=cmp1) > + self.assertEqual(key(obj=3), key(obj=3)) > + self.assertGreater(key(obj=3), key(obj=1)) > + with self.assertRaises((TypeError, AttributeError)): > + key(3) > 1 # rhs is not a K object > + with self.assertRaises((TypeError, AttributeError)): > + 1 < key(3) # lhs is not a K object > + with self.assertRaises(TypeError): > + key = functools.cmp_to_key() # too few args > + with self.assertRaises(TypeError): > + key = functools.cmp_to_key(cmp1, None) # too many args > + key = functools.cmp_to_key(cmp1) > + with self.assertRaises(TypeError): > + key() # too few args > + with self.assertRaises(TypeError): > + key(None, None) # too many args > + > + def test_bad_cmp(self): > + def cmp1(x, y): > + raise ZeroDivisionError > + key = functools.cmp_to_key(cmp1) > + with self.assertRaises(ZeroDivisionError): > + key(3) > key(1) > + > + class BadCmp: > + def __lt__(self, other): > + raise ZeroDivisionError > + def cmp1(x, y): > + return BadCmp() > + with self.assertRaises(ZeroDivisionError): > + key(3) > key(1) > + > + def test_obj_field(self): > + def cmp1(x, y): > + return (x > y) - (x < y) > + key = functools.cmp_to_key(mycmp=cmp1) > + self.assertEqual(key(50).obj, 50) > + > + def test_sort_int(self): > def mycmp(x, y): > return y - x > self.assertEqual(sorted(range(5), key=functools.cmp_to_key(mycmp)), > [4, 3, 2, 1, 0]) > > + def test_sort_int_str(self): > + def mycmp(x, y): > + x, y = int(x), int(y) > + return (x > y) - (x < y) > + values = [5, '3', 7, 2, '0', '1', 4, '10', 1] > + values = sorted(values, key=functools.cmp_to_key(mycmp)) > + self.assertEqual([int(value) for value in values], > + [0, 1, 1, 2, 3, 4, 5, 7, 10]) > + > def test_hash(self): > def mycmp(x, y): > return y - x > key = functools.cmp_to_key(mycmp) > k = key(10) > - self.assertRaises(TypeError, hash(k)) > + self.assertRaises(TypeError, hash, k)

1 0

Policy for versions of system python
by Eugene Toder 05 Apr '11

05 Apr '11

Hello, CPython source code currently contains a number of python scripts (e.g Python/makeopcodetargets.py, Objects/typeslots.py, Parser/asdl_c.py) that are used during the build of the python interpreter itself. For this reason they are run with system installed python. What is the policy regarding the range of python versions that they should support? I looked at some of the scripts and they seem to support both 2 and 3, starting from at most 2.4. Python/makeopcodetargets.py says at the top: # This code should stay compatible with Python 2.3, at least while # some of the buildbots have Python 2.3 as their system Python. Is this the official minimal version or do we have this spelled out more explicitly somewhere? Eugene

4 4

Re: [Python-Dev] [Python-checkins] cpython: Revert the Lib/test/test_bigmem.py changes from commit 17891566a478 (and a
by Éric Araujo 04 Apr '11

04 Apr '11

Hi, > changeset: 68921:11dc3f270594 > user: Thomas Wouters <thomas(a)python.org> > date: Fri Mar 25 11:42:37 2011 +0100 > summary: > Revert the Lib/test/test_bigmem.py changes from commit 17891566a478 (and a > few other assertEqual tests that snuck in), and expand the docstrings and > comments explaining why and how these tests are supposed to work. Your commit message does not explain why you reverted the changes. The specific assert* methods give more useful messages than assertEqual in case of failure. Regards

3 2