Mailman 3 August 2006 - Python-checkins

buildbot warnings in x86 XP trunk
by buildbot＠python.org 31 Aug '06

31 Aug '06

The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/1415 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in x86 OpenBSD trunk
by buildbot＠python.org 31 Aug '06

31 Aug '06

The Buildbot has detected a new failure of x86 OpenBSD trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520OpenBSD%2520trunk/builds/1245 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in x86 W2k trunk
by buildbot＠python.org 31 Aug '06

31 Aug '06

The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/1474 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in amd64 gentoo trunk
by buildbot＠python.org 31 Aug '06

31 Aug '06

The Buildbot has detected a new failure of amd64 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520gentoo%2520trunk/builds/14… Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in x86 gentoo trunk
by buildbot＠python.org 31 Aug '06

31 Aug '06

The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/1567 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,nick.coghlan Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot failure in ppc Debian unstable trunk
by buildbot＠python.org 31 Aug '06

31 Aug '06

The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trun… Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: armin.rigo,nick.coghlan BUILD FAILED: failed configure sincerely, -The Buildbot

1 0

r51664 - in python/trunk: Doc/lib/libdecimal.tex Lib/decimal.py Lib/test/test_decimal.py
by nick.coghlan 31 Aug '06

31 Aug '06

Author: nick.coghlan Date: Thu Aug 31 14:00:43 2006 New Revision: 51664 Modified: python/trunk/Doc/lib/libdecimal.tex python/trunk/Lib/decimal.py python/trunk/Lib/test/test_decimal.py Log: Fix the wrongheaded implementation of context management in the decimal module and add unit tests. (python-dev discussion is ongoing regarding what we do about Python 2.5) Modified: python/trunk/Doc/lib/libdecimal.tex ============================================================================== --- python/trunk/Doc/lib/libdecimal.tex (original) +++ python/trunk/Doc/lib/libdecimal.tex Thu Aug 31 14:00:43 2006 @@ -435,26 +435,34 @@ the \function{getcontext()} and \function{setcontext()} functions: \begin{funcdesc}{getcontext}{} - Return the current context for the active thread. + Return the current context for the active thread. \end{funcdesc} \begin{funcdesc}{setcontext}{c} - Set the current context for the active thread to \var{c}. + Set the current context for the active thread to \var{c}. \end{funcdesc} Beginning with Python 2.5, you can also use the \keyword{with} statement -to temporarily change the active context. For example the following code -increases the current decimal precision by 2 places, performs a -calculation, and then automatically restores the previous context: +to temporarily change the active context. -\begin{verbatim} -from __future__ import with_statement -import decimal - -with decimal.getcontext() as ctx: - ctx.prec += 2 # add 2 more digits of precision - calculate_something() +\begin{funcdesc}{localcontext}{\optional{c}} + Return a context manager that will set the current context for + the active thread to a copy of \var{c} on entry to the with statement + and restore the previous context when exiting the with statement. + + For example the following code increases the current decimal precision + by 2 places, performs a calculation, and then automatically restores + the previous context: +\begin{verbatim} + from __future__ import with_statement + import decimal + + with decimal.localcontext() as ctx: + ctx.prec += 2 # add 2 more digits of precision + s = calculate_something() + s = +s # Round the final result back to the default precision \end{verbatim} +\end{funcdesc} The context that's active in the body of the \keyword{with} statement is a \emph{copy} of the context you provided to the \keyword{with} Modified: python/trunk/Lib/decimal.py ============================================================================== --- python/trunk/Lib/decimal.py (original) +++ python/trunk/Lib/decimal.py Thu Aug 31 14:00:43 2006 @@ -130,8 +130,11 @@ 'ROUND_DOWN', 'ROUND_HALF_UP', 'ROUND_HALF_EVEN', 'ROUND_CEILING', 'ROUND_FLOOR', 'ROUND_UP', 'ROUND_HALF_DOWN', + # helper for context management + 'ContextManager', + # Functions for manipulating contexts - 'setcontext', 'getcontext' + 'setcontext', 'getcontext', 'localcontext' ] import copy as _copy @@ -458,6 +461,49 @@ del threading, local # Don't contaminate the namespace +def localcontext(ctx=None): + """Return a context manager for a copy of the supplied context + + Uses a copy of the current context if no context is specified + The returned context manager creates a local decimal context + in a with statement: + def sin(x): + with localcontext() as ctx: + ctx.prec += 2 + # Rest of sin calculation algorithm + # uses a precision 2 greater than normal + return +s # Convert result to normal precision + + def sin(x): + with localcontext(ExtendedContext): + # Rest of sin calculation algorithm + # uses the Extended Context from the + # General Decimal Arithmetic Specification + return +s # Convert result to normal context + + """ + # The below can't be included in the docstring until Python 2.6 + # as the doctest module doesn't understand __future__ statements + """ + >>> from __future__ import with_statement + >>> print getcontext().prec + 28 + >>> with localcontext(): + ... ctx = getcontext() + ... ctx.prec() += 2 + ... print ctx.prec + ... + 30 + >>> with localcontext(ExtendedContext): + ... print getcontext().prec + ... + 9 + >>> print getcontext().prec + 28 + """ + if ctx is None: ctx = getcontext().copy() + return ContextManager(ctx.copy()) + ##### Decimal class ########################################### @@ -2174,20 +2220,27 @@ del name, val, globalname, rounding_functions class ContextManager(object): - """Helper class to simplify Context management. + """Context manager class to support localcontext(). - Sample usage: - - with decimal.ExtendedContext: - s = ... - return +s # Convert result to normal precision - - with decimal.getcontext() as ctx: - ctx.prec += 2 - s = ... - return +s + Sets the supplied context in __enter__() and restores + the previous decimal context in __exit__() """ + # The below can't be included in the docstring until Python 2.6 + # as the doctest module doesn't understand __future__ statements + """ + Sample usage: + >>> from __future__ import with_statement + >>> print getcontext().prec + 28 + >>> ctx = Context(prec=15) + >>> with ContextManager(ctx): + ... print getcontext().prec + ... + 15 + >>> print getcontext().prec + 28 + """ def __init__(self, new_context): self.new_context = new_context def __enter__(self): @@ -2248,9 +2301,6 @@ s.append('traps=[' + ', '.join([t.__name__ for t, v in self.traps.items() if v]) + ']') return ', '.join(s) + ')' - def get_manager(self): - return ContextManager(self.copy()) - def clear_flags(self): """Reset all flags to zero""" for flag in self.flags: Modified: python/trunk/Lib/test/test_decimal.py ============================================================================== --- python/trunk/Lib/test/test_decimal.py (original) +++ python/trunk/Lib/test/test_decimal.py Thu Aug 31 14:00:43 2006 @@ -23,6 +23,7 @@ you're working through IDLE, you can import this test module and call test_main() with the corresponding argument. """ +from __future__ import with_statement import unittest import glob @@ -1064,6 +1065,43 @@ self.assertNotEqual(id(c.flags), id(d.flags)) self.assertNotEqual(id(c.traps), id(d.traps)) +class WithStatementTest(unittest.TestCase): + # Can't do these as docstrings until Python 2.6 + # as doctest can't handle __future__ statements + def test_ContextManager(self): + # The basic context manager uses the supplied context + # without making a copy of it + orig_ctx = getcontext() + new_ctx = Context() + with ContextManager(new_ctx) as enter_ctx: + set_ctx = getcontext() + final_ctx = getcontext() + self.assert_(orig_ctx is final_ctx, 'did not restore context correctly') + self.assert_(new_ctx is set_ctx, 'did not set correct context') + self.assert_(set_ctx is enter_ctx, '__enter__ returned wrong context') + + def test_localcontext(self): + # The helper function makes a copy of the supplied context + orig_ctx = getcontext() + with localcontext() as enter_ctx: + set_ctx = getcontext() + final_ctx = getcontext() + self.assert_(orig_ctx is final_ctx, 'did not restore context correctly') + self.assert_(orig_ctx is not set_ctx, 'did not copy the context') + self.assert_(set_ctx is enter_ctx, '__enter__ returned wrong context') + + def test_localcontextarg(self): + # The helper function makes a copy of the supplied context + orig_ctx = getcontext() + new_ctx = Context(prec=42) + with localcontext(new_ctx) as enter_ctx: + set_ctx = getcontext() + final_ctx = getcontext() + self.assert_(orig_ctx is final_ctx, 'did not restore context correctly') + self.assert_(set_ctx.prec == new_ctx.prec, 'did not set correct context') + self.assert_(new_ctx is not set_ctx, 'did not copy the context') + self.assert_(set_ctx is enter_ctx, '__enter__ returned wrong context') + def test_main(arith=False, verbose=None): """ Execute the tests. @@ -1084,6 +1122,7 @@ DecimalPythonAPItests, ContextAPItests, DecimalTest, + WithStatementTest, ] try:

1 0

r51663 - python/trunk/Doc/lib/libhashlib.tex
by armin.rigo 31 Aug '06

31 Aug '06

Author: armin.rigo Date: Thu Aug 31 10:51:06 2006 New Revision: 51663 Modified: python/trunk/Doc/lib/libhashlib.tex Log: Doc fix: hashlib objects don't always return a digest of 16 bytes. Backport candidate for 2.5. Modified: python/trunk/Doc/lib/libhashlib.tex ============================================================================== --- python/trunk/Doc/lib/libhashlib.tex (original) +++ python/trunk/Doc/lib/libhashlib.tex Thu Aug 31 10:51:06 2006 @@ -86,8 +86,8 @@ \begin{methoddesc}[hash]{digest}{} Return the digest of the strings passed to the \method{update()} -method so far. This is a 16-byte string which may contain -non-\ASCII{} characters, including null bytes. +method so far. This is a string of \member{digest_size} bytes which may +contain non-\ASCII{} characters, including null bytes. \end{methoddesc} \begin{methoddesc}[hash]{hexdigest}{}

1 0

r51657 - python/branches/bcannon-objcap/BRANCHNEWS python/branches/bcannon-objcap/securing_python.txt
by brett.cannon 30 Aug '06

30 Aug '06

Author: brett.cannon Date: Thu Aug 31 00:05:48 2006 New Revision: 51657 Modified: python/branches/bcannon-objcap/BRANCHNEWS python/branches/bcannon-objcap/securing_python.txt Log: Update status with 'file' initializer removed. Modified: python/branches/bcannon-objcap/BRANCHNEWS ============================================================================== --- python/branches/bcannon-objcap/BRANCHNEWS (original) +++ python/branches/bcannon-objcap/BRANCHNEWS Thu Aug 31 00:05:48 2006 @@ -5,5 +5,17 @@ Core and builtins ----------------- +* rev. 51656: Remove initializer from 'file'. By leaving tp_new alone you can + still subclass 'file' (although its usefulness as a subclass is doubtful when + its tp_init is empty and that is what actually opens the file descriptor). + Created a new function, PyFile_UnsafeOpen() which is what the built-in open() + function is now set to. Also changed the bz2 module to use it. + + Still need to decide how to handle subclasses of 'file' so they are not + totally useless (special function that calls the needed initializer on a + 'file' object?). Also need to come up with C API that opens files through + the built-in open() instead of doing it directly so as to not bypass + security. + * rev. 51392: Introduce objcap module to hold removed functions/methods. Begin with moving object.__subclasses__(). Modified: python/branches/bcannon-objcap/securing_python.txt ============================================================================== --- python/branches/bcannon-objcap/securing_python.txt (original) +++ python/branches/bcannon-objcap/securing_python.txt Thu Aug 31 00:05:48 2006 @@ -7,11 +7,11 @@ + Remove object.__subclasses__ (`Mutable Shared State`_) [done] + Dangerous constructors (`Constructors`_) - file - * Create PyFile_InsecureOpen() + * Create PyFile_UnsafeOpen() [done] * Switch current C-level uses of 'file' constructor to - PyFile_InsecureOpen(). - + built-in open() - + bz2 module + PyFile_UnsafeOpen(). [done] + + built-in open() [done] + + bz2 module [done] * Create PyFile_Safe*() version of C API that goes through open() built-in. + Convert C strings to Python objects and do a direct

1 0

r51656 - in python/branches/bcannon-objcap: Include/fileobject.h Lib/test/test_descr.py Modules/bz2module.c Objects/fileobject.c Python/bltinmodule.c
by brett.cannon 30 Aug '06

30 Aug '06

Author: brett.cannon Date: Wed Aug 30 23:52:22 2006 New Revision: 51656 Modified: python/branches/bcannon-objcap/Include/fileobject.h python/branches/bcannon-objcap/Lib/test/test_descr.py python/branches/bcannon-objcap/Modules/bz2module.c python/branches/bcannon-objcap/Objects/fileobject.c python/branches/bcannon-objcap/Python/bltinmodule.c Log: Remove tp_init from 'file'. Left tp_new so as to continue to allow subclassing, although mostly useless since without tp_init you cannot actually open any file. Added PyFile_UnsafeOpen() and set built-in open to that. Also changed bz2 module to use it as well. Still need to make C API calls go through built-in open() so that if it is overridden that is used instead of blindly opening any file. Will most likely require a new set of APIs for that. Modified: python/branches/bcannon-objcap/Include/fileobject.h ============================================================================== --- python/branches/bcannon-objcap/Include/fileobject.h (original) +++ python/branches/bcannon-objcap/Include/fileobject.h Wed Aug 30 23:52:22 2006 @@ -45,6 +45,8 @@ PyAPI_FUNC(int) PyFile_WriteString(const char *, PyObject *); PyAPI_FUNC(int) PyObject_AsFileDescriptor(PyObject *); +PyAPI_FUNC(PyObject *) PyFile_UnsafeOpen(PyObject *, PyObject *, PyObject *); + /* The default encoding used by the platform file system APIs If non-NULL, this is different than the default encoding for strings */ Modified: python/branches/bcannon-objcap/Lib/test/test_descr.py ============================================================================== --- python/branches/bcannon-objcap/Lib/test/test_descr.py (original) +++ python/branches/bcannon-objcap/Lib/test/test_descr.py Wed Aug 30 23:52:22 2006 @@ -2452,10 +2452,14 @@ lineno = 0 ateof = 0 + + def __init__(self, path): + self.__file = open(path) + def readline(self): if self.ateof: return "" - s = file.readline(self) + s = file.readline(self.__file) # Next line works too. # s = super(CountedInput, self).readline() self.lineno += 1 @@ -2500,7 +2504,7 @@ # note: as of Python 2.3, dict() no longer has an "items" keyword arg for constructor in (int, float, long, complex, str, unicode, - tuple, list, file): + tuple, list): try: constructor(bogus_keyword_arg=1) except TypeError: Modified: python/branches/bcannon-objcap/Modules/bz2module.c ============================================================================== --- python/branches/bcannon-objcap/Modules/bz2module.c (original) +++ python/branches/bcannon-objcap/Modules/bz2module.c Wed Aug 30 23:52:22 2006 @@ -1293,6 +1293,7 @@ static char *kwlist[] = {"filename", "mode", "buffering", "compresslevel", 0}; PyObject *name; + PyObject *file_open_args; char *mode = "r"; int buffering = -1; int compresslevel = 9; @@ -1353,8 +1354,10 @@ mode = (mode_char == 'r') ? "rb" : "wb"; - self->file = PyObject_CallFunction((PyObject*)&PyFile_Type, "(Osi)", - name, mode, buffering); + file_open_args = Py_BuildValue("Osi", name, mode, buffering); + if (!file_open_args) + goto error; + self->file = PyFile_UnsafeOpen(NULL, file_open_args, NULL); if (self->file == NULL) return -1; Modified: python/branches/bcannon-objcap/Objects/fileobject.c ============================================================================== --- python/branches/bcannon-objcap/Objects/fileobject.c (original) +++ python/branches/bcannon-objcap/Objects/fileobject.c Wed Aug 30 23:52:22 2006 @@ -273,6 +273,7 @@ return (PyObject *)f; } + PyObject * PyFile_FromFile(FILE *fp, char *name, char *mode, int (*close)(FILE *)) { @@ -1966,25 +1967,28 @@ return self; } -static int -file_init(PyObject *self, PyObject *args, PyObject *kwds) +/* + Open a file with no regards to whether it should be allowed. + + Used as the implementation of built-in open(). +*/ +PyObject * +PyFile_UnsafeOpen(PyObject *self, PyObject *args, PyObject *kwds) { - PyFileObject *foself = (PyFileObject *)self; - int ret = 0; + PyFileObject *foself; + PyObject *ret = NULL; static char *kwlist[] = {"name", "mode", "buffering", 0}; char *name = NULL; char *mode = "r"; int bufsize = -1; int wideargument = 0; + self = file_new(&PyFile_Type, args, kwds); + if (!self) + return NULL; + foself = (PyFileObject *)self; + assert(PyFile_Check(self)); - if (foself->f_fp != NULL) { - /* Have to close the existing file first. */ - PyObject *closeresult = file_close(foself); - if (closeresult == NULL) - return -1; - Py_DECREF(closeresult); - } #ifdef Py_WIN_WIDE_FILENAMES if (GetVersion() < 0x80000000) { /* On NT, so wide API available */ @@ -2010,13 +2014,13 @@ Py_FileSystemDefaultEncoding, &name, &mode, &bufsize)) - return -1; + return NULL; /* We parse again to get the name as a PyObject */ if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|si:file", kwlist, &o_name, &mode, &bufsize)) - return -1; + return NULL; if (fill_file_fields(foself, NULL, o_name, mode, fclose) == NULL) @@ -2026,10 +2030,11 @@ goto Error; foself->f_setbuf = NULL; PyFile_SetBufSize(self, bufsize); + ret = self; goto Done; Error: - ret = -1; + ret = NULL; /* fall through */ Done: PyMem_Free(name); /* free the encoded string */ @@ -2096,7 +2101,7 @@ 0, /* tp_descr_get */ 0, /* tp_descr_set */ 0, /* tp_dictoffset */ - file_init, /* tp_init */ + 0, /* tp_init */ PyType_GenericAlloc, /* tp_alloc */ file_new, /* tp_new */ PyObject_Del, /* tp_free */ Modified: python/branches/bcannon-objcap/Python/bltinmodule.c ============================================================================== --- python/branches/bcannon-objcap/Python/bltinmodule.c (original) +++ python/branches/bcannon-objcap/Python/bltinmodule.c Wed Aug 30 23:52:22 2006 @@ -1341,16 +1341,29 @@ Return the octal representation of an integer or long integer."); -static PyObject * -builtin_open(PyObject *self, PyObject *args, PyObject *kwds) -{ - return PyObject_Call((PyObject*)&PyFile_Type, args, kwds); -} +/* PyFile_UnsafeOpen() used as open(). */ -PyDoc_STRVAR(open_doc, -"open(name[, mode[, buffering]]) -> file object\n\ -\n\ -Open a file using the file() type, returns a file object."); +PyDoc_VAR(open_doc) = +PyDoc_STR( +"file(name[, mode[, buffering]]) -> file object\n" +"\n" +"Open a file. The mode can be 'r', 'w' or 'a' for reading (default),\n" +"writing or appending. The file will be created if it doesn't exist\n" +"when opened for writing or appending; it will be truncated when\n" +"opened for writing. Add a 'b' to the mode for binary files.\n" +"Add a '+' to the mode to allow simultaneous reading and writing.\n" +"If the buffering argument is given, 0 means unbuffered, 1 means line\n" +"buffered, and larger numbers specify the buffer size.\n" +) +PyDoc_STR( +"Add a 'U' to mode to open the file for input with universal newline\n" +"support. Any line ending in the input file will be seen as a '\\n'\n" +"in Python. Also, a file so opened gains the attribute 'newlines';\n" +"the value for this attribute is one of None (no newline read yet),\n" +"'\\r', '\\n', '\\r\\n' or a tuple containing all the newline types seen.\n" +"\n" +"'U' cannot be combined with 'w' or '+' mode.\n" +); static PyObject * @@ -2259,7 +2272,7 @@ {"max", (PyCFunction)builtin_max, METH_VARARGS | METH_KEYWORDS, max_doc}, {"min", (PyCFunction)builtin_min, METH_VARARGS | METH_KEYWORDS, min_doc}, {"oct", builtin_oct, METH_O, oct_doc}, - {"open", (PyCFunction)builtin_open, METH_VARARGS | METH_KEYWORDS, open_doc}, + {"open", (PyCFunction)PyFile_UnsafeOpen, METH_VARARGS | METH_KEYWORDS, open_doc}, {"ord", builtin_ord, METH_O, ord_doc}, {"pow", builtin_pow, METH_VARARGS, pow_doc}, {"range", builtin_range, METH_VARARGS, range_doc},

1 0