Mailman 3 November 2000 - Python-Dev

XML runtime errors?
by Fredrik Lundh 05 Mar '01

05 Mar '01

stoopid question: why the heck is xmllib using "RuntimeError" to flag XML syntax errors? raise RuntimeError, 'Syntax error at line %d: %s' % (self.lineno, message) what's wrong with "SyntaxError"? </F>

3 3

[PEP 223] Change the Meaning of \x Escapes
by Tim Peters 15 Feb '01

15 Feb '01

An HTML version of the attached can be viewed at http://python.sourceforge.net/peps/pep-0223.html This will be adopted for 2.0 unless there's an uproar. Note that it *does* have potential for breaking existing code -- although no real-life instance of incompatibility has yet been reported. This is explained in detail in the PEP; check your code now. although-if-i-were-you-i-wouldn't-bother<0.5-wink>-ly y'rs - tim PEP: 223 Title: Change the Meaning of \x Escapes Version: $Revision: 1.4 $ Author: tpeters(a)beopen.com (Tim Peters) Status: Active Type: Standards Track Python-Version: 2.0 Created: 20-Aug-2000 Post-History: 23-Aug-2000 Abstract Change \x escapes, in both 8-bit and Unicode strings, to consume exactly the two hex digits following. The proposal views this as correcting an original design flaw, leading to clearer expression in all flavors of string, a cleaner Unicode story, better compatibility with Perl regular expressions, and with minimal risk to existing code. Syntax The syntax of \x escapes, in all flavors of non-raw strings, becomes \xhh where h is a hex digit (0-9, a-f, A-F). The exact syntax in 1.5.2 is not clearly specified in the Reference Manual; it says \xhh... implying "two or more" hex digits, but one-digit forms are also accepted by the 1.5.2 compiler, and a plain \x is "expanded" to itself (i.e., a backslash followed by the letter x). It's unclear whether the Reference Manual intended either of the 1-digit or 0-digit behaviors. Semantics In an 8-bit non-raw string, \xij expands to the character chr(int(ij, 16)) Note that this is the same as in 1.6 and before. In a Unicode string, \xij acts the same as \u00ij i.e. it expands to the obvious Latin-1 character from the initial segment of the Unicode space. An \x not followed by at least two hex digits is a compile-time error, specifically ValueError in 8-bit strings, and UnicodeError (a subclass of ValueError) in Unicode strings. Note that if an \x is followed by more than two hex digits, only the first two are "consumed". In 1.6 and before all but the *last* two were silently ignored. Example In 1.5.2: >>> "\x123465" # same as "\x65" 'e' >>> "\x65" 'e' >>> "\x1" '\001' >>> "\x\x" '\\x\\x' >>> In 2.0: >>> "\x123465" # \x12 -> \022, "3456" left alone '\0223456' >>> "\x65" 'e' >>> "\x1" [ValueError is raised] >>> "\x\x" [ValueError is raised] >>> History and Rationale \x escapes were introduced in C as a way to specify variable-width character encodings. Exactly which encodings those were, and how many hex digits they required, was left up to each implementation. The language simply stated that \x "consumed" *all* hex digits following, and left the meaning up to each implementation. So, in effect, \x in C is a standard hook to supply platform-defined behavior. Because Python explicitly aims at platform independence, the \x escape in Python (up to and including 1.6) has been treated the same way across all platforms: all *except* the last two hex digits were silently ignored. So the only actual use for \x escapes in Python was to specify a single byte using hex notation. Larry Wall appears to have realized that this was the only real use for \x escapes in a platform-independent language, as the proposed rule for Python 2.0 is in fact what Perl has done from the start (although you need to run in Perl -w mode to get warned about \x escapes with fewer than 2 hex digits following -- it's clearly more Pythonic to insist on 2 all the time). When Unicode strings were introduced to Python, \x was generalized so as to ignore all but the last *four* hex digits in Unicode strings. This caused a technical difficulty for the new regular expression engine: SRE tries very hard to allow mixing 8-bit and Unicode patterns and strings in intuitive ways, and it no longer had any way to guess what, for example, r"\x123456" should mean as a pattern: is it asking to match the 8-bit character \x56 or the Unicode character \u3456? There are hacky ways to guess, but it doesn't end there. The ISO C99 standard also introduces 8-digit \U12345678 escapes to cover the entire ISO 10646 character space, and it's also desired that Python 2 support that from the start. But then what are \x escapes supposed to mean? Do they ignore all but the last *eight* hex digits then? And if less than 8 following in a Unicode string, all but the last 4? And if less than 4, all but the last 2? This was getting messier by the minute, and the proposal cuts the Gordian knot by making \x simpler instead of more complicated. Note that the 4-digit generalization to \xijkl in Unicode strings was also redundant, because it meant exactly the same thing as \uijkl in Unicode strings. It's more Pythonic to have just one obvious way to specify a Unicode character via hex notation. Development and Discussion The proposal was worked out among Guido van Rossum, Fredrik Lundh and Tim Peters in email. It was subsequently explained and disussed on Python-Dev under subject "Go \x yourself", starting 2000-08-03. Response was overwhelmingly positive; no objections were raised. Backward Compatibility Changing the meaning of \x escapes does carry risk of breaking existing code, although no instances of incompabitility have yet been discovered. The risk is believed to be minimal. Tim Peters verified that, except for pieces of the standard test suite deliberately provoking end cases, there are no instances of \xabcdef... with fewer or more than 2 hex digits following, in either the Python CVS development tree, or in assorted Python packages sitting on his machine. It's unlikely there are any with fewer than 2, because the Reference Manual implied they weren't legal (although this is debatable!). If there are any with more than 2, Guido is ready to argue they were buggy anyway <0.9 wink>. Guido reported that the O'Reilly Python books *already* document that Python works the proposed way, likely due to their Perl editing heritage (as above, Perl worked (very close to) the proposed way from its start). Finn Bock reported that what JPython does with \x escapes is unpredictable today. This proposal gives a clear meaning that can be consistently and easily implemented across all Python implementations. Effects on Other Tools Believed to be none. The candidates for breakage would mostly be parsing tools, but the author knows of none that worry about the internal structure of Python strings beyond the approximation "when there's a backslash, swallow the next character". Tim Peters checked python-mode.el, the std tokenize.py and pyclbr.py, and the IDLE syntax coloring subsystem, and believes there's no need to change any of them. Tools like tabnanny.py and checkappend.py inherit their immunity from tokenize.py. Reference Implementation The code changes are so simple that a separate patch will not be produced. Fredrik Lundh is writing the code, is an expert in the area, and will simply check the changes in before 2.0b1 is released. BDFL Pronouncements Yes, ValueError, not SyntaxError. "Problems with literal interpretations traditionally raise 'runtime' exceptions rather than syntax errors." Copyright This document has been placed in the public domain.

9 16

Re: [Python-checkins] CVS: python/dist/src/Include pyport.h,2.20,2.21
by Gregor Hoffleit 29 Jan '01

29 Jan '01

FYI: This misdefinition with LONG_BIT was due to a bug in glibc's limits.h. It has been fixed in glibc 2.96. Gregor On Wed, Oct 04, 2000 at 06:42:32PM -0700, Tim Peters wrote: > Update of /cvsroot/python/python/dist/src/Include > In directory slayer.i.sourceforge.net:/tmp/cvs-serv5758/python/dist/src/Include > > Modified Files: > pyport.h > Log Message: > Move LONG_BIT from intobject.c to pyport.h. #error if it's already been > #define'd to an unreasonable value (several recent gcc systems have > misdefined it, causing bogus overflows in integer multiplication). Nuke > CHAR_BIT entirely.

3 2

Re: [Python-Dev] PEP-0218
by Moshe Zadka 21 Dec '00

21 Dec '00

> BTW, one lesson to take from SETL: a vital set operation in practice is a > mutating "pick a 'random' element E from the set, remove it from the set, > and return it". An enormous number of set algorithms are of the form > > while not S.empty(): > pick some element from S > deal with it, possibly mutating S > > This operation can't be done efficiently in Python code if the set is > represented by a dict (the best you can do is materialize the full list of > keys first, and pick one of those). That means my Set class often takes > quadratic time for what *should* be linear-time algorithms. Hmmm...actually, I've been wanting a method .key() for dictionaries a long time. So if we give dictionaries this one small method, then we *can* do this in Python. -- Moshe Zadka <sig(a)zadka.site.co.il> This is a signature anti-virus. Please stop the spread of signature viruses!

16 41

Re: [Python-Dev] A house upon the sand
by Michael Hudson 04 Dec '00

04 Dec '00

"Tim Peters" <tim.one(a)home.com> writes: > On the deprecation of the string module: where did this idea come from? > I've never seen anything saying that the string module is deprecated. I thought this, and went looking. I found on http://www.python.org/1.6/, about four fifths of the way down: Changed Modules string - most of this module is deprecated now that strings have methods. This no longer uses the built-in strop module, but takes advantage of the new string methods to provide transparent support for both Unicode and ordinary strings. I hope (and believe) this is Wrong. http://www.python.org/2.0/new-python.html says: The old string module is still around for backwards compatibility, but it mostly acts as a front-end to the new string methods. which is IMHO better. Cheers, M. -- Every day I send overnight packages filled with rabid weasels to people who use frames for no good reason. -- The Usenet Oracle, Oracularity #1017-1

12 32

Re: [Python-checkins] CVS: python/nondist/peps pep-0000.txt,1.50,1.51 pep-0207.txt,1.3,1.4
by Andrew Kuchling 01 Dec '00

01 Dec '00

On Tue, Nov 28, 2000 at 02:13:47PM -0800, Guido van Rossum wrote: >229 >is not sufficiently worked out and doesn't seem a priority. (This can >still make it into 2.1 if the author does the work.) I was waiting for further comments on the idea before actually doing the work. Anyone see gaping holes in the basic idea? (I *can* just charge ahead and work on it anyway...) --amk

5 6

Re: [Python-checkins] CVS: python/dist/src/Doc/lib libstdtypes.tex,1.42,1.43
by Fred L. Drake, Jr. 30 Nov '00

30 Nov '00

Moshe Zadka writes: > + \lineiii{\var{a}.firstitem()} > + {a (\var{key}, \var{value}) pair, the first one in \var{a}.items()} > + {(2)} So what happens if the mapping is empty? That's needed for all three. You should also update UserDict. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Digital Creations

3 3

Re: [Python-Dev] surprising bug in s.capitalize()?
by Michael Hudson 30 Nov '00

30 Nov '00

"M.-A. Lemburg" <mal(a)lemburg.com> writes: > > Oooh, here's something a bit more serious though: > > > > >>> u'aAaAaA'.capitalize() > > u'AAaAaA' > > > > Something obviously Needs To Be Done. My hunch is to change > > string_capitalize, but that may be just me (and probably Barry). > > Ouch. That's a bug. Yes. > Here's what the string.py module has to say about this BTW: I said "string_capitalize", i.e. "stringobject.c:string_capitalize", i.e I think >>> 'aAaA'.capitalize() should result in 'AAaA' We're not doing too well at understanding each other today, are we? > Note that .title() is very similar to the way string.capitalize() > works. unicode.title() also uses the title case information available > for Unicode characters. Ah yes. So in the interests of reducing redunancy, .capitalize should probably do something usefully different... Cheers, M. -- 59. In English every word can be verbed. Would that it were so in our programming languages. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

3 2

surprising bug in s.capitalize()?
by barry＠digicool.com 30 Nov '00

30 Nov '00

I was just surprised by something I've never observed before from string.capitalize(). There is clearly either a bug (really two) in the code or in the documentation. >>> '__Myfoo'.capitalize() '__myfoo' Huh? The documentation says: capitalize () Return a copy of the string with only its first character capitalized. So it's only doing one of the three things it's promising! For one thing, it's modifying something other than the first character, and for another, it's actually swapping the case of the character it /is/ changing. At least it returns a copy. :) Given the documentation, I would have expected the return value to be the same as the original string, i.e. unchanged. So which is it? Does the description of the method need to be made more complicated, or does the code need to be simplified <wink>? -Barry

3 3

Re: [Python-checkins] CVS: python/dist/src/Objects dictobject.c,2.65,2.66
by Greg Stein 30 Nov '00

30 Nov '00

On Thu, Nov 30, 2000 at 04:31:07AM -0800, Moshe Zadka wrote: >... > *** dictobject.c 2000/09/01 23:29:27 2.65 > --- dictobject.c 2000/11/30 12:31:00 2.66 > *************** > *** 710,713 **** > --- 710,782 ---- > > static PyObject * > + dict_firstkey(register dictobject *mp, PyObject *args) > + { > + register int i; > + > + if (!PyArg_NoArgs(args)) > + return NULL; PyArg_NoArgs() is deprecated because you cannot specify the method name. The code should be: if (!PyArg_ParseTuple(args, ":firstkey")) return NULL; (same in the next two funcs) >... > + if (mp->ma_table[i].me_value != NULL) { > + PyObject *key = mp->ma_table[i].me_key; > + PyObject *value = mp->ma_table[i].me_value; > + PyObject *item = PyTuple_New(2); > + if (item == NULL) { > + return NULL; > + } > + Py_INCREF(key); > + PyTuple_SetItem(item, 0, key); > + Py_INCREF(value); > + PyTuple_SetItem(item, 1, value); These can be PyTuple_SET_ITEM() since you know you have a tuple. Cheers, -g -- Greg Stein, http://www.lyra.org/

5 5