Mailman 3 February 2006 - Python-Dev

Current trunk test failures
by Tim Peters 27 Feb '06

27 Feb '06

The buildbot shows that the debug-build test_grammar is dying with a C assert failure on all boxes. In case it helps, in a Windows release build test_transformer is also failing: test_transformer test test_transformer failed -- Traceback (most recent call last): File "C:\Code\python\lib\test\test_transformer.py", line 16, in testMultipleLHS a = transformer.parse(s) File "C:\Code\python\lib\compiler\transformer.py", line 52, in parse return Transformer().parsesuite(buf) File "C:\Code\python\lib\compiler\transformer.py", line 129, in parsesuite return self.transform(parser.suite(text)) File "C:\Code\python\lib\compiler\transformer.py", line 125, in transform return self.compile_node(tree) File "C:\Code\python\lib\compiler\transformer.py", line 158, in compile_node return self.file_input(node[1:]) File "C:\Code\python\lib\compiler\transformer.py", line 189, in file_input self.com_append_stmt(stmts, node) File "C:\Code\python\lib\compiler\transformer.py", line 1036, in com_append_stmt result = self.lookup_node(node)(node[1:]) File "C:\Code\python\lib\compiler\transformer.py", line 305, in stmt return self.com_stmt(nodelist[0]) File "C:\Code\python\lib\compiler\transformer.py", line 1029, in com_stmt result = self.lookup_node(node)(node[1:]) File "C:\Code\python\lib\compiler\transformer.py", line 315, in simple_stmt self.com_append_stmt(stmts, nodelist[i]) File "C:\Code\python\lib\compiler\transformer.py", line 1036, in com_append_stmt result = self.lookup_node(node)(node[1:]) File "C:\Code\python\lib\compiler\transformer.py", line 305, in stmt return self.com_stmt(nodelist[0]) File "C:\Code\python\lib\compiler\transformer.py", line 1029, in com_stmt result = self.lookup_node(node)(node[1:]) File "C:\Code\python\lib\compiler\transformer.py", line 353, in expr_stmt exprNode = self.lookup_node(en)(en[1:]) File "C:\Code\python\lib\compiler\transformer.py", line 763, in lookup_node return self._dispatch[node[0]] KeyError: 324 Also test_parser: C:\Code\python\PCbuild>python -E -tt ../lib/test/regrtest.py -v test_parser test_parser test_assert (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_basic_import_statement (test.test_parser.RoundtripLegalSyntaxTestCase) ... ok test_class_defs (test.test_parser.RoundtripLegalSyntaxTestCase) ... ok test_expressions (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_function_defs (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_import_from_statement (test.test_parser.RoundtripLegalSyntaxTestCase) ... ok test_pep263 (test.test_parser.RoundtripLegalSyntaxTestCase) ... ok test_print (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_simple_assignments (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_simple_augmented_assignments (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_simple_expression (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_yield_statement (test.test_parser.RoundtripLegalSyntaxTestCase) ... FAIL test_a_comma_comma_c (test.test_parser.IllegalSyntaxTestCase) ... ok test_illegal_operator (test.test_parser.IllegalSyntaxTestCase) ... ok test_illegal_yield_1 (test.test_parser.IllegalSyntaxTestCase) ... ok test_illegal_yield_2 (test.test_parser.IllegalSyntaxTestCase) ... ok test_junk (test.test_parser.IllegalSyntaxTestCase) ... ok test_malformed_global (test.test_parser.IllegalSyntaxTestCase) ... ok test_print_chevron_comma (test.test_parser.IllegalSyntaxTestCase) ... ok test_compile_error (test.test_parser.CompileTestCase) ... ok test_compile_expr (test.test_parser.CompileTestCase) ... ok test_compile_suite (test.test_parser.CompileTestCase) ... ok ====================================================================== FAIL: test_assert (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 180, in test_assert self.check_suite("assert alo < ahi and blo < bhi\n") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'assert alo < ahi and blo < bhi\n': Expected node type 303, got 302. ====================================================================== FAIL: test_expressions (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 50, in test_expressions self.check_expr("foo(1)") File "C:\Code\python\lib\test\test_parser.py", line 25, in check_expr self.roundtrip(parser.expr, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'foo(1)': Expected node type 303, got 302. ====================================================================== FAIL: test_function_defs (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 119, in test_function_defs self.check_suite("def f(foo=bar): pass") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'def f(foo=bar): pass': Expected node type 303, got 302. ====================================================================== FAIL: test_print (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 86, in test_print self.check_suite("print 1") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'print 1': Expected node type 303, got 302. ====================================================================== FAIL: test_simple_assignments (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 97, in test_simple_assignments self.check_suite("a = b") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'a = b': Expected node type 303, got 302. ====================================================================== FAIL: test_simple_augmented_assignments (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 101, in test_simple_augmented_assignments self.check_suite("a += b") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'a += b': Expected node type 303, got 302. ====================================================================== FAIL: test_simple_expression (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 94, in test_simple_expression self.check_suite("a") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'a': Expected node type 303, got 302. ====================================================================== FAIL: test_yield_statement (test.test_parser.RoundtripLegalSyntaxTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Code\python\lib\test\test_parser.py", line 31, in test_yield_statement self.check_suite("def f(): yield 1") File "C:\Code\python\lib\test\test_parser.py", line 28, in check_suite self.roundtrip(parser.suite, s) File "C:\Code\python\lib\test\test_parser.py", line 19, in roundtrip self.fail("could not roundtrip %r: %s" % (s, why)) AssertionError: could not roundtrip 'def f(): yield 1': Expected node type 303, got 302. ---------------------------------------------------------------------- Ran 22 tests in 0.015s FAILED (failures=8) test test_parser failed -- errors occurred; run in verbose mode for details 1 test failed: test_parser and also test_compiler.

2 1

Re: [Python-Dev] PEP 332 revival in coordination with pep 349? [ Was:Re: release plan for 2.5 ?]
by Phillip J. Eby 27 Feb '06

27 Feb '06

At 04:29 PM 2/13/2006 -0800, Guido van Rossum wrote: >On 2/13/06, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > I didn't mean that it was the only purpose. In Python 2.x, practical code > > has to sometimes deal with "string-like" objects. That is, code that takes > > either strings or unicode. If such code calls bytes(), it's going to want > > to include an encoding so that unicode conversions won't fail. > >That sounds like a rather hypothetical example. Have you thought it >through? Presumably code that accepts both str and unicode either >doesn't care about encodings, but simply returns objects of the same >type as the arguments -- and then it's unlikely to want to convert the >arguments to bytes; or it *does* care about encodings, and then it >probably already has to special-case str vs. unicode because it has to >control how str objects are interpreted. Actually, it's the other way around. Code that wants to output uninterpreted bytes right now and accepts either strings or Unicode has to special-case *unicode* -- not str, because str is the only "bytes type" we currently have. This creates an interesting issue in WSGI for Jython, which of course only has one (unicode-based) string type now. Since there's no bytes type in Python in general, the only solution we could come up with was to treat such strings as latin-1: http://www.python.org/peps/pep-0333.html#unicode-issues This is why I'm biased towards latin-1 encoding of unicode to bytes; it's "the same thing" as an uninterpreted string of bytes. I think the difference in our viewpoints is that you're still thinking "string" thoughts, whereas I'm thinking "byte" thoughts. Bytes are just bytes; they don't *have* an encoding. So, if you think of "converting a string to bytes" as meaning "create an array of numerals corresponding to the characters in the string", then this leads to a uniform result whether the characters are in a str or a unicode object. In other words, to me, bytes(str_or_unicode) should be treated as: bytes(map(ord, str_or_unicode)) In other words, without an encoding, bytes() should simply treat str and unicode objects *as if they were a sequence of integers*, and produce an error when an integer is out of range. This is a logical and consistent interpretation in the absence of an encoding, because in that case you don't care about the encoding - it's just raw data. If, however, you include an encoding, then you're stating that you want to encode the *meaning* of the string, not merely its integer values. >What would bytes("abc\xf0", "latin-1") *mean*? Take the string >"abc\xf0", interpret it as being encoded in XXX, and then encode from >XXX to Latin-1. But what's XXX? As I showed in a previous post, >"abc\xf0".encode("latin-1") *fails* because the source for the >encoding is assumed to be ASCII. I'm saying that XXX would be the same encoding as you specified. i.e., including an encoding means you are encoding the *meaning* of the string. However, I believe I mainly proposed this as an alternative to having bytes(str_or_unicode) work like bytes(map(ord,str_or_unicode)), which I think is probably a saner default. >Your argument for symmetry would be a lot stronger if we used Latin-1 >for the conversion between str and Unicode. But we don't. But that's because we're dealing with its meaning *as a string*, not merely as ordinals in a sequence of bytes. > I like the >other interpretation (which I thought was yours too?) much better: str ><--> bytes conversions don't use encodings by simply change the type >without changing the bytes; I like it better too. The part you didn't like was where MAL and I believe this should be extended to Unicode characters in the 0-255 range also. :) >There's one property that bytes, str and unicode all share: type(x[0]) >== type(x), at least as long as len(x) >= 1. This is perhaps the >ultimate test for string-ness. > >Or should b[0] be an int, if b is a bytes object? That would change >things dramatically. +1 for it being an int. Heck, I'd want to at least consider the possibility of introducing a character type (chr?) in Python 3.0, and getting rid of the "iterating a string yields strings" characteristic. I've found it to be a bit of a pain when dealing with heterogeneous nested sequences that contain strings. >There's also the consideration for APIs that, informally, accept >either a string or a sequence of objects. Many of these exist, and >they are probably all being converted to support unicode as well as >str (if it makes sense at all). Should a bytes object be considered as >a sequence of things, or as a single thing, from the POV of these >types of APIs? Should we try to standardize how code tests for the >difference? (Currently all sorts of shortcuts are being taken, from >isinstance(x, (list, tuple)) to isinstance(x, basestring).) I'm inclined to think of certain features at least in terms of the buffer interface, but that's not something that's really exposed at the Python level.

9 10

PEP 308
by Thomas Wouters 26 Feb '06

26 Feb '06

Since I was on a streak of implementing not-quite-the-right-thing, I checked in my PEP 308 implementation *with* backward compatibility -- just to spite Guido's latest change to the PEP. It jumps through a minor hoop (two new grammar rules) in order to be backwardly compatible, but that hoop can go away in Python 3.0, and that shouldn't be too long from now. I apologize for the test failures of compile, transform and parser: they seem to all depend on the parsermodule being updated. If no one feels responsible for it, I'll do it later in the week (I'll be sprinting until Thursday anyway.) -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

1 0

Exposing the abstract syntax
by martin＠v.loewis.de 26 Feb '06

26 Feb '06

At PyCon, there was general reluctance for incorporating the ast-objects branch, primarily because people where concerned what the reference counting would do to maintainability, and what (potentially troublesome) options direct exposure of AST objects would do. OTOH, the approach of creating a shadow tree did not find opposition, so I implemented that. Currently, you can use compile() to create an AST out of source code, by passing PyCF_ONLY_AST (0x400) to compile. The mapping of AST to Python objects is as follows: - there is a Python type for every sum, product, and constructor. - The constructor types inherit from their sum types (e.g. ClassDef inherits from stmt) - Each constructor and product type has an _fields member, giving the names of the fields of the product. - Each node in the AST has members with the names given in _fields - If the field is optional, it might be None - if the field is zero-or-more, it is represented as a list. It might be reasonable to expose this through a separate module, in particular to provide access to the type objects. Regards, Martin

1 0

Re: [Python-Dev] Using and binding relative names (was Re: PEP for Better Control of Nested Lexical Scopes)
by Massimiliano Leoni 26 Feb '06

26 Feb '06

Why would you change the Python scoping rules, instead of using the function attributes, available from release 2.1 (PEP 232) ? For example, you may write: def incgen(start, inc): def incrementer(): incrementer.a += incrementer.b return incrementer.a incrementer.a = start - inc incrementer.b = inc return incrementer f = incgen(100, 2) g = incgen(200, 3) for i in range(5): print f(), g() The result is: 100 200 102 203 104 206 106 209 108 212

2 1

cProfile prints to stdout?
by skip＠pobox.com 26 Feb '06

26 Feb '06

I just noticed that cProfile (like profile) prints to stdout. Yuck. I guess that's to be expected because the pstats module does the actual printing and it's used by both modules. I'm willing to give up backward compatibility to achieve a little more sanity and flexibility here. I propose rewriting the necessary bits to att a stream= keyword argument where necessary and using stream.write(...) or print >> stream, ... instead of the current bare print. I'd prefer the default for the stream be sys.stderr as well. Thoughts? Skip

6 10

Fwd: Translating docs
by Facundo Batista 26 Feb '06

26 Feb '06

2006/2/25, martin(a)v.loewis.de <martin(a)v.loewis.de>: > Translating the library reference as such is more difficult, because > it can't be translated in small chunks very well. The SVN directory "python/dist/src/Doc/lib/" has 276 .tex's, with an average of 250 lines each. Maybe manage each file independently could work. > Some group of French translators once translated everything for 1.5.2, > and that translation never got updated. We're afraid of this. And that's why we think that it'd be necessary to have some automated system that tell us if the original file got updated, if there're new files to translate, to show the state of the translation (in process, finished, not even started, etc...). I think that a system like this is not so difficult, but if the wheel is already invented... . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

4 3

PEP 328
by Thomas Wouters 25 Feb '06

25 Feb '06

Since I implemented[*] PEP 328, Aahz suggested I take over editing the PEP, too, as there were some minor discussion points to add still. I haven't been around for the discussioons, though, and it's been a while for everone else, I think, so I'd like to rehash and ask for any other open points. The one open point that Aahz forwarded me, and is expressed somewhat in http://mail.python.org/pipermail/python-dev/2004-September/048695.html , is the case where you have a package that you want to transparently supply a particular version of a module for forward/backward compatibility, replacing a version elsewhere on sys.path (if any.) I see four distinct situations for this: 1) Replacing a stdlib module (or a set of them) with a newer version, if the stdlib module is too old, where you want the whole stdlib to use the newer version. 2) Same as 1), but private to your package; modules not in your package should get the stdlib version when they import the 'replaced' module. 3) Providing a module (or a set of them) that the stdlib might be missing (but which will be a new enough version if it's there) 1) and 3) are easy to solve: put the module in a separate directory, insert that into sys.path; at the front for 1), at the end for 3). Mailman, IIRC, does this, and I think it works fine. 2) is easy if it's a single module; include it in your package and import it relatively. If it's a package itself, it's again pretty easy; include the package and include it relatively. The package itself is hopefully already using relative imports to get sibling packages. If the package is using absolute imports to get sibling packages, well, crap. I don't think we can solve that issue whatever we do: that already breaks. The real problem with 2) is when you have tightly coupled modules that are not together in a package and not using relative imports, or perhaps when you want to *partially* override a package. I would argue that tightly coupled modules should always use relative imports, whether they are together in a package or not (even though they should probably be in a package anyway.) I'd also argue that having different modules import different versions of existing modules is a bad idea. It's workable if the modules are only used internally, but exposing anything is troublesome. for instance, an instance of a class defined in foo (1.0) imported by bar will not be an instance of the same class defined in foo (1.1) imported by feeble. Am I missing anything? ([*] incorrectly, to be sure, but I have a 'correct' version ready that I'll upload in a second; I was trying to confuse Guido into accepting my version, instead.) -- Thomas Wouters <thomas(a)xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

3 2

Dropping support for Win9x in 2.6
by Neal Norwitz 25 Feb '06

25 Feb '06

Martin and I were talking about dropping support for older versions of Windows (of the non-NT flavor). We both thought that it was reasonable to stop supporting Win9x (including WinME) in Python 2.6. I updated PEP 11 to reflect this. The Python 2.5 installer will present a warning message on the systems which will not be supported in Python 2.6. n

10 9

problem with genexp
by Neal Norwitz 24 Feb '06

24 Feb '06

There's a problem with genexp's that I think really needs to get fixed. See http://python.org/sf/1167751 the details are below. This code: >>> foo(a = i for i in range(10)) generates "NameError: name 'i' is not defined" when run because: 2 0 LOAD_GLOBAL 0 (foo) 3 LOAD_CONST 1 ('a') 6 LOAD_GLOBAL 1 (i) 9 CALL_FUNCTION 256 12 POP_TOP 13 LOAD_CONST 0 (None) 16 RETURN_VALUE If you add parens around the code: foo(a = i for i in range(10)) You get something quite different: 2 0 LOAD_GLOBAL 0 (foo) 3 LOAD_CONST 1 ('a') 6 LOAD_CONST 2 (<code object <generator expression> at 0x2a960baae8, file "<stdin>", line 2>) 9 MAKE_FUNCTION 0 12 LOAD_GLOBAL 1 (range) 15 LOAD_CONST 3 (10) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 CALL_FUNCTION 256 28 POP_TOP 29 LOAD_CONST 0 (None) 32 RETURN_VALUE I agree with the bug report that the code should either raise a SyntaxError or do the right thing. n

5 10

Python-Dev February 2006