Mailman 3 August 2006 - Python-checkins

r51438 - in python/branches/p3yk: Doc/api/abstract.tex Doc/api/concrete.tex Doc/api/exceptions.tex Doc/api/init.tex Doc/api/intro.tex Doc/api/newtypes.tex Doc/commontex/boilerplate.tex Doc/dist/dist.tex Doc/lib/libctypes.tex Doc/lib/libexcs.tex Doc/lib/libwarnings.tex Doc/whatsnew/whatsnew25.tex Include/abstract.h Include/code.h Include/import.h Include/object.h Include/pyerrors.h Include/unicodeobject.h Lib/Cookie.py Lib/cgi.py Lib/compiler/ast.py Lib/compiler/pycodegen.py Lib/compiler/sy
by thomas.wouters 22 Aug '06

22 Aug '06

Author: thomas.wouters Date: Mon Aug 21 21:07:27 2006 New Revision: 51438 Added: python/branches/p3yk/Lib/ctypes/test/test_as_parameter.py - copied unchanged from r51433, python/trunk/Lib/ctypes/test/test_as_parameter.py python/branches/p3yk/Misc/RPM/python-2.6.spec - copied unchanged from r51433, python/trunk/Misc/RPM/python-2.6.spec python/branches/p3yk/Python/peephole.c - copied, changed from r51433, python/trunk/Python/peephole.c Removed: python/branches/p3yk/Misc/RPM/python-2.5.spec Modified: python/branches/p3yk/ (props changed) python/branches/p3yk/Doc/api/abstract.tex python/branches/p3yk/Doc/api/concrete.tex python/branches/p3yk/Doc/api/exceptions.tex python/branches/p3yk/Doc/api/init.tex python/branches/p3yk/Doc/api/intro.tex python/branches/p3yk/Doc/api/newtypes.tex python/branches/p3yk/Doc/commontex/boilerplate.tex python/branches/p3yk/Doc/dist/dist.tex python/branches/p3yk/Doc/lib/libctypes.tex python/branches/p3yk/Doc/lib/libexcs.tex python/branches/p3yk/Doc/lib/libwarnings.tex python/branches/p3yk/Doc/whatsnew/whatsnew25.tex python/branches/p3yk/Include/abstract.h python/branches/p3yk/Include/code.h python/branches/p3yk/Include/import.h python/branches/p3yk/Include/object.h python/branches/p3yk/Include/pyerrors.h python/branches/p3yk/Include/unicodeobject.h python/branches/p3yk/Lib/Cookie.py python/branches/p3yk/Lib/cgi.py python/branches/p3yk/Lib/compiler/ast.py python/branches/p3yk/Lib/compiler/pycodegen.py python/branches/p3yk/Lib/compiler/symbols.py python/branches/p3yk/Lib/ctypes/__init__.py python/branches/p3yk/Lib/ctypes/test/test_functions.py python/branches/p3yk/Lib/ctypes/test/test_numbers.py python/branches/p3yk/Lib/ctypes/test/test_prototypes.py python/branches/p3yk/Lib/ctypes/test/test_python_api.py python/branches/p3yk/Lib/distutils/__init__.py python/branches/p3yk/Lib/idlelib/Bindings.py python/branches/p3yk/Lib/idlelib/CREDITS.txt python/branches/p3yk/Lib/idlelib/CodeContext.py python/branches/p3yk/Lib/idlelib/NEWS.txt python/branches/p3yk/Lib/idlelib/PyShell.py python/branches/p3yk/Lib/idlelib/idlever.py python/branches/p3yk/Lib/logging/__init__.py python/branches/p3yk/Lib/logging/config.py python/branches/p3yk/Lib/site.py python/branches/p3yk/Lib/string.py python/branches/p3yk/Lib/tabnanny.py python/branches/p3yk/Lib/tarfile.py python/branches/p3yk/Lib/test/exception_hierarchy.txt python/branches/p3yk/Lib/test/inspect_fodder2.py python/branches/p3yk/Lib/test/output/test_cgi python/branches/p3yk/Lib/test/test_bytes.py python/branches/p3yk/Lib/test/test_bz2.py python/branches/p3yk/Lib/test/test_cgi.py python/branches/p3yk/Lib/test/test_compiler.py python/branches/p3yk/Lib/test/test_format.py python/branches/p3yk/Lib/test/test_index.py python/branches/p3yk/Lib/test/test_inspect.py python/branches/p3yk/Lib/test/test_logging.py python/branches/p3yk/Lib/test/test_mailbox.py python/branches/p3yk/Lib/test/test_signal.py python/branches/p3yk/Lib/test/test_socketserver.py python/branches/p3yk/Lib/test/test_tarfile.py python/branches/p3yk/Lib/test/test_threading.py python/branches/p3yk/Lib/test/test_threadsignals.py python/branches/p3yk/Lib/test/test_urllib2.py python/branches/p3yk/Lib/test/test_uuid.py python/branches/p3yk/Lib/test/test_xml_etree_c.py python/branches/p3yk/Lib/tokenize.py python/branches/p3yk/Lib/trace.py python/branches/p3yk/Lib/urllib2.py python/branches/p3yk/Lib/uuid.py python/branches/p3yk/Makefile.pre.in python/branches/p3yk/Misc/ACKS python/branches/p3yk/Misc/PURIFY.README python/branches/p3yk/Misc/README.klocwork python/branches/p3yk/Misc/build.sh python/branches/p3yk/Modules/_bsddb.c python/branches/p3yk/Modules/_codecsmodule.c python/branches/p3yk/Modules/_ctypes/_ctypes.c python/branches/p3yk/Modules/_ctypes/_ctypes_test.c python/branches/p3yk/Modules/_ctypes/callbacks.c python/branches/p3yk/Modules/_ctypes/callproc.c python/branches/p3yk/Modules/_ctypes/cfield.c python/branches/p3yk/Modules/_ctypes/ctypes.h python/branches/p3yk/Modules/_ctypes/libffi/configure python/branches/p3yk/Modules/_ctypes/libffi/configure.ac python/branches/p3yk/Modules/_ctypes/libffi/fficonfig.py.in python/branches/p3yk/Modules/_ctypes/stgdict.c python/branches/p3yk/Modules/_cursesmodule.c python/branches/p3yk/Modules/_elementtree.c python/branches/p3yk/Modules/_hotshot.c python/branches/p3yk/Modules/_sre.c python/branches/p3yk/Modules/_tkinter.c python/branches/p3yk/Modules/arraymodule.c python/branches/p3yk/Modules/bz2module.c python/branches/p3yk/Modules/collectionsmodule.c python/branches/p3yk/Modules/expat/xmlparse.c python/branches/p3yk/Modules/mmapmodule.c python/branches/p3yk/Modules/operator.c python/branches/p3yk/Modules/posixmodule.c python/branches/p3yk/Modules/rgbimgmodule.c python/branches/p3yk/Modules/socketmodule.c python/branches/p3yk/Modules/unicodedata.c python/branches/p3yk/Modules/zipimport.c python/branches/p3yk/Objects/abstract.c python/branches/p3yk/Objects/bytesobject.c python/branches/p3yk/Objects/exceptions.c python/branches/p3yk/Objects/intobject.c python/branches/p3yk/Objects/listobject.c python/branches/p3yk/Objects/longobject.c python/branches/p3yk/Objects/object.c python/branches/p3yk/Objects/sliceobject.c python/branches/p3yk/Objects/stringobject.c python/branches/p3yk/Objects/structseq.c python/branches/p3yk/Objects/tupleobject.c python/branches/p3yk/Objects/typeobject.c python/branches/p3yk/Objects/unicodeobject.c python/branches/p3yk/PC/pyconfig.h python/branches/p3yk/PCbuild/_ssl.mak python/branches/p3yk/PCbuild/_ssl.vcproj python/branches/p3yk/PCbuild/build_ssl.py python/branches/p3yk/PCbuild/pythoncore.vcproj python/branches/p3yk/PCbuild/readme.txt python/branches/p3yk/Parser/ (props changed) python/branches/p3yk/Parser/grammar.c python/branches/p3yk/Parser/parsetok.c python/branches/p3yk/Python/ast.c python/branches/p3yk/Python/bltinmodule.c python/branches/p3yk/Python/ceval.c python/branches/p3yk/Python/compile.c python/branches/p3yk/Python/import.c python/branches/p3yk/Python/marshal.c python/branches/p3yk/Python/pystate.c python/branches/p3yk/Python/pythonrun.c python/branches/p3yk/Python/symtable.c python/branches/p3yk/Tools/buildbot/ (props changed) python/branches/p3yk/Tools/buildbot/external.bat python/branches/p3yk/Tools/msi/msi.py Log: Merge current trunk into p3yk. This includes the PyNumber_Index API change, which unfortunately means the errors from the bytes type change somewhat: bytes([300]) still raises a ValueError, but bytes([10**100]) now raises a TypeError (either that, or bytes(1.0) also raises a ValueError -- PyNumber_AsSsize_t() can only raise one type of exception.) Merged revisions 51188-51433 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r51189 | kurt.kaiser | 2006-08-10 19:11:09 +0200 (Thu, 10 Aug 2006) | 4 lines Retrieval of previous shell command was not always preserving indentation since 1.2a1) Patch 1528468 Tal Einat. ........ r51190 | guido.van.rossum | 2006-08-10 19:41:07 +0200 (Thu, 10 Aug 2006) | 3 lines Chris McDonough's patch to defend against certain DoS attacks on FieldStorage. SF bug #1112549. ........ r51191 | guido.van.rossum | 2006-08-10 19:42:50 +0200 (Thu, 10 Aug 2006) | 2 lines News item for SF bug 1112549. ........ r51192 | guido.van.rossum | 2006-08-10 20:09:25 +0200 (Thu, 10 Aug 2006) | 2 lines Fix title -- it's rc1, not beta3. ........ r51194 | martin.v.loewis | 2006-08-10 21:04:00 +0200 (Thu, 10 Aug 2006) | 3 lines Update dangling references to the 3.2 database to mention that this is UCD 4.1 now. ........ r51195 | tim.peters | 2006-08-11 00:45:34 +0200 (Fri, 11 Aug 2006) | 6 lines Followup to bug #1069160. PyThreadState_SetAsyncExc(): internal correctness changes wrt refcount safety and deadlock avoidance. Also added a basic test case (relying on ctypes) and repaired the docs. ........ r51196 | tim.peters | 2006-08-11 00:48:45 +0200 (Fri, 11 Aug 2006) | 2 lines Whitespace normalization. ........ r51197 | tim.peters | 2006-08-11 01:22:13 +0200 (Fri, 11 Aug 2006) | 5 lines Whitespace normalization broke test_cgi, because a line of quoted test data relied on preserving a single trailing blank. Changed the string from raw to regular, and forced in the trailing blank via an explicit \x20 escape. ........ r51198 | tim.peters | 2006-08-11 02:49:01 +0200 (Fri, 11 Aug 2006) | 10 lines test_PyThreadState_SetAsyncExc(): This is failing on some 64-bit boxes. I have no idea what the ctypes docs mean by "integers", and blind-guessing here that it intended to mean the signed C "int" type, in which case perhaps I can repair this by feeding the thread id argument to type ctypes.c_long(). Also made the worker thread daemonic, so it doesn't hang Python shutdown if the test continues to fail. ........ r51199 | tim.peters | 2006-08-11 05:49:10 +0200 (Fri, 11 Aug 2006) | 6 lines force_test_exit(): This has been completely ineffective at stopping test_signal from hanging forever on the Tru64 buildbot. That could be because there's no such thing as signal.SIGALARM. Changed to the idiotic (but standard) signal.SIGALRM instead, and added some more debug output. ........ r51202 | neal.norwitz | 2006-08-11 08:09:41 +0200 (Fri, 11 Aug 2006) | 6 lines Fix the failures on cygwin (2006-08-10 fixed the actual locking issue). The first hunk changes the colon to an ! like other Windows variants. We need to always wait on the child so the lock gets released and no other tests fail. This is the try/finally in the second hunk. ........ r51205 | georg.brandl | 2006-08-11 09:15:38 +0200 (Fri, 11 Aug 2006) | 3 lines Add Chris McDonough (latest cgi.py patch) ........ r51206 | georg.brandl | 2006-08-11 09:26:10 +0200 (Fri, 11 Aug 2006) | 3 lines logging's atexit hook now runs even if the rest of the module has already been cleaned up. ........ r51212 | thomas.wouters | 2006-08-11 17:02:39 +0200 (Fri, 11 Aug 2006) | 4 lines Add ignore of *.pyc and *.pyo to Lib/xml/etree/. ........ r51215 | thomas.heller | 2006-08-11 21:55:35 +0200 (Fri, 11 Aug 2006) | 7 lines When a ctypes C callback function is called, zero out the result storage before converting the result to C data. See the comment in the code for details. Provide a better context for errors when the conversion of a callback function's result cannot be converted. ........ r51218 | neal.norwitz | 2006-08-12 03:43:40 +0200 (Sat, 12 Aug 2006) | 6 lines Klocwork made another run and found a bunch more problems. This is the first batch of fixes that should be easy to verify based on context. This fixes problem numbers: 220 (ast), 323-324 (symtable), 321-322 (structseq), 215 (array), 210 (hotshot), 182 (codecs), 209 (etree). ........ r51219 | neal.norwitz | 2006-08-12 03:45:47 +0200 (Sat, 12 Aug 2006) | 9 lines Even though _Py_Mangle() isn't truly public anyone can call it and there was no verification that privateobj was a PyString. If it wasn't a string, this could have allowed a NULL pointer to creep in below and crash. I wonder if this should be PyString_CheckExact? Must identifiers be strings or can they be subclasses? Klocwork #275 ........ r51220 | neal.norwitz | 2006-08-12 03:46:42 +0200 (Sat, 12 Aug 2006) | 5 lines It's highly unlikely, though possible for PyEval_Get*() to return NULLs. So be safe and do an XINCREF. Klocwork # 221-222. ........ r51221 | neal.norwitz | 2006-08-12 03:47:59 +0200 (Sat, 12 Aug 2006) | 7 lines This code is actually not used unless WITHOUT_COMPLEX is defined. However, there was no error checking that PyFloat_FromDouble returned a valid pointer. I believe this change is correct as it seemed to follow other code in the area. Klocwork # 292. ........ r51222 | neal.norwitz | 2006-08-12 03:49:12 +0200 (Sat, 12 Aug 2006) | 5 lines Handle NULL nodes while parsing. I'm not entirely sure this is correct. There might be something else that needs to be done to setup the error. Klocwork #295. ........ r51223 | neal.norwitz | 2006-08-12 03:50:38 +0200 (Sat, 12 Aug 2006) | 6 lines If _stat_float_times is false, we will try to INCREF ival which could be NULL. Return early in that case. The caller checks for PyErr_Occurred so this should be ok. Klocwork #297 ........ r51224 | neal.norwitz | 2006-08-12 03:51:12 +0200 (Sat, 12 Aug 2006) | 3 lines Move the assert which checks for a NULL pointer first. Klocwork #274. ........ r51225 | neal.norwitz | 2006-08-12 03:53:28 +0200 (Sat, 12 Aug 2006) | 5 lines Try to handle a malloc failure. I'm not entirely sure this is correct. There might be something else we need to do to handle the exception. Klocwork # 212-213 ........ r51226 | neal.norwitz | 2006-08-12 03:57:47 +0200 (Sat, 12 Aug 2006) | 6 lines I'm not sure why this code allocates this string for the error message. I think it would be better to always use snprintf and have the format limit the size of the name appropriately (like %.200s). Klocwork #340 ........ r51227 | neal.norwitz | 2006-08-12 04:06:34 +0200 (Sat, 12 Aug 2006) | 3 lines Check returned pointer is valid. Klocwork #233 ........ r51228 | neal.norwitz | 2006-08-12 04:12:30 +0200 (Sat, 12 Aug 2006) | 1 line Whoops, how did that get in there. :-) Revert all the parts of 51227 that were not supposed to go it. Only Modules/_ctypes/cfields.c was supposed to be changed ........ r51229 | neal.norwitz | 2006-08-12 04:33:36 +0200 (Sat, 12 Aug 2006) | 4 lines Don't deref v if it's NULL. Klocwork #214 ........ r51230 | neal.norwitz | 2006-08-12 05:16:54 +0200 (Sat, 12 Aug 2006) | 5 lines Check return of PyMem_MALLOC (garbage) is non-NULL. Check seq in both portions of if/else. Klocwork #289-290. ........ r51231 | neal.norwitz | 2006-08-12 05:17:41 +0200 (Sat, 12 Aug 2006) | 4 lines PyModule_GetDict() can fail, produce fatal errors if this happens on startup. Klocwork #298-299. ........ r51232 | neal.norwitz | 2006-08-12 05:18:50 +0200 (Sat, 12 Aug 2006) | 5 lines Verify verdat which is returned from malloc is not NULL. Ensure we don't pass NULL to free. Klocwork #306 (at least the first part, checking malloc) ........ r51233 | tim.peters | 2006-08-12 06:42:47 +0200 (Sat, 12 Aug 2006) | 35 lines test_signal: Signal handling on the Tru64 buildbot appears to be utterly insane. Plug some theoretical insecurities in the test script: - Verify that the SIGALRM handler was actually installed. - Don't call alarm() before the handler is installed. - Move everything that can fail inside the try/finally, so the test cleans up after itself more often. - Try sending all the expected signals in force_test_exit(), not just SIGALRM. Since that was fixed to actually send SIGALRM (instead of invisibly dying with an AttributeError), we've seen that sending SIGALRM alone does not stop this from hanging. - Move the "kill the child" business into the finally clause, so the child doesn't survive test failure to send SIGALRM to other tests later (there are also baffling SIGALRM-related failures in test_socket). - Cancel the alarm in the finally clause -- if the test dies early, we again don't want SIGALRM showing up to confuse a later test. Alas, this still relies on timing luck wrt the spawned script that sends the test signals, but it's hard to see how waiting for seconds can so often be so unlucky. test_threadedsignals: curiously, this test never fails on Tru64, but doesn't normally signal SIGALRM. Anyway, fixed an obvious (but probably inconsequential) logic error. ........ r51234 | tim.peters | 2006-08-12 07:17:41 +0200 (Sat, 12 Aug 2006) | 8 lines Ah, fudge. One of the prints here actually "shouldn't be" protected by "if verbose:", which caused the test to fail on all non-Windows boxes. Note that I deliberately didn't convert this to unittest yet, because I expect it would be even harder to debug this on Tru64 after conversion. ........ r51235 | georg.brandl | 2006-08-12 10:32:02 +0200 (Sat, 12 Aug 2006) | 3 lines Repair logging test spew caused by rev. 51206. ........ r51236 | neal.norwitz | 2006-08-12 19:03:09 +0200 (Sat, 12 Aug 2006) | 8 lines Patch #1538606, Patch to fix __index__() clipping. I modified this patch some by fixing style, some error checking, and adding XXX comments. This patch requires review and some changes are to be expected. I'm checking in now to get the greatest possible review and establish a baseline for moving forward. I don't want this to hold up release if possible. ........ r51238 | neal.norwitz | 2006-08-12 20:44:06 +0200 (Sat, 12 Aug 2006) | 10 lines Fix a couple of bugs exposed by the new __index__ code. The 64-bit buildbots were failing due to inappropriate clipping of numbers larger than 2**31 with new-style classes. (typeobject.c) In reviewing the code for classic classes, there were 2 problems. Any negative value return could be returned. Always return -1 if there was an error. Also make the checks similar with the new-style classes. I believe this is correct for 32 and 64 bit boxes, including Windows64. Add a test of classic classes too. ........ r51240 | neal.norwitz | 2006-08-13 02:20:49 +0200 (Sun, 13 Aug 2006) | 1 line SF bug #1539336, distutils example code missing ........ r51245 | neal.norwitz | 2006-08-13 20:10:10 +0200 (Sun, 13 Aug 2006) | 6 lines Move/copy assert for tstate != NULL before first use. Verify that PyEval_Get{Globals,Locals} returned valid pointers. Klocwork 231-232 ........ r51246 | neal.norwitz | 2006-08-13 20:10:28 +0200 (Sun, 13 Aug 2006) | 5 lines Handle a whole lot of failures from PyString_FromInternedString(). Should fix most of Klocwork 234-272. ........ r51247 | neal.norwitz | 2006-08-13 20:10:47 +0200 (Sun, 13 Aug 2006) | 8 lines cpathname could be NULL if it was longer than MAXPATHLEN. Don't try to write the .pyc to NULL. Check results of PyList_GetItem() and PyModule_GetDict() are not NULL. Klocwork 282, 283, 285 ........ r51248 | neal.norwitz | 2006-08-13 20:11:08 +0200 (Sun, 13 Aug 2006) | 6 lines Fix segfault when doing string formatting on subclasses of long if __oct__, __hex__ don't return a string. Klocwork 308 ........ r51250 | neal.norwitz | 2006-08-13 20:11:27 +0200 (Sun, 13 Aug 2006) | 5 lines Check return result of PyModule_GetDict(). Fix a bunch of refleaks in the init of the module. This would only be found when running python -v. ........ r51251 | neal.norwitz | 2006-08-13 20:11:43 +0200 (Sun, 13 Aug 2006) | 5 lines Handle malloc and fopen failures more gracefully. Klocwork 180-181 ........ r51252 | neal.norwitz | 2006-08-13 20:12:03 +0200 (Sun, 13 Aug 2006) | 7 lines It's very unlikely, though possible that source is not a string. Verify that PyString_AsString() returns a valid pointer. (The problem can arise when zlib.decompress doesn't return a string.) Klocwork 346 ........ r51253 | neal.norwitz | 2006-08-13 20:12:26 +0200 (Sun, 13 Aug 2006) | 5 lines Handle failures from lookup. Klocwork 341-342 ........ r51254 | neal.norwitz | 2006-08-13 20:12:45 +0200 (Sun, 13 Aug 2006) | 6 lines Handle failure from PyModule_GetDict() (Klocwork 208). Fix a bunch of refleaks in the init of the module. This would only be found when running python -v. ........ r51255 | neal.norwitz | 2006-08-13 20:13:02 +0200 (Sun, 13 Aug 2006) | 4 lines Really address the issue of where to place the assert for leftblock. (Followup of Klocwork 274) ........ r51256 | neal.norwitz | 2006-08-13 20:13:36 +0200 (Sun, 13 Aug 2006) | 4 lines Handle malloc failure. Klocwork 281 ........ r51258 | neal.norwitz | 2006-08-13 20:40:39 +0200 (Sun, 13 Aug 2006) | 4 lines Handle alloca failures. Klocwork 225-228 ........ r51259 | neal.norwitz | 2006-08-13 20:41:15 +0200 (Sun, 13 Aug 2006) | 1 line Get rid of compiler warning ........ r51261 | neal.norwitz | 2006-08-14 02:51:15 +0200 (Mon, 14 Aug 2006) | 1 line Ignore pgen.exe and kill_python.exe for cygwin ........ r51262 | neal.norwitz | 2006-08-14 02:59:03 +0200 (Mon, 14 Aug 2006) | 4 lines Can't return NULL from a void function. If there is a memory error, about the best we can do is call PyErr_WriteUnraisable and go on. We won't be able to do the call below either, so verify delstr is valid. ........ r51263 | neal.norwitz | 2006-08-14 03:49:54 +0200 (Mon, 14 Aug 2006) | 1 line Update purify doc some. ........ r51264 | thomas.heller | 2006-08-14 09:13:05 +0200 (Mon, 14 Aug 2006) | 2 lines Remove unused, buggy test function. Fixes klockwork issue #207. ........ r51265 | thomas.heller | 2006-08-14 09:14:09 +0200 (Mon, 14 Aug 2006) | 2 lines Check for NULL return value from new_CArgObject(). Fixes klockwork issues #183, #184, #185. ........ r51266 | thomas.heller | 2006-08-14 09:50:14 +0200 (Mon, 14 Aug 2006) | 2 lines Check for NULL return value of GenericCData_new(). Fixes klockwork issues #188, #189. ........ r51274 | thomas.heller | 2006-08-14 12:02:24 +0200 (Mon, 14 Aug 2006) | 2 lines Revert the change that tries to zero out a closure's result storage area because the size if unknown in source/callproc.c. ........ r51276 | marc-andre.lemburg | 2006-08-14 12:55:19 +0200 (Mon, 14 Aug 2006) | 11 lines Slightly revised version of patch #1538956: Replace UnicodeDecodeErrors raised during == and != compares of Unicode and other objects with a new UnicodeWarning. All other comparisons continue to raise exceptions. Exceptions other than UnicodeDecodeErrors are also left untouched. ........ r51277 | thomas.heller | 2006-08-14 13:17:48 +0200 (Mon, 14 Aug 2006) | 13 lines Apply the patch #1532975 plus ideas from the patch #1533481. ctypes instances no longer have the internal and undocumented '_as_parameter_' attribute which was used to adapt them to foreign function calls; this mechanism is replaced by a function pointer in the type's stgdict. In the 'from_param' class methods, try the _as_parameter_ attribute if other conversions are not possible. This makes the documented _as_parameter_ mechanism work as intended. Change the ctypes version number to 1.0.1. ........ r51278 | marc-andre.lemburg | 2006-08-14 13:44:34 +0200 (Mon, 14 Aug 2006) | 3 lines Readd NEWS items that were accidentally removed by r51276. ........ r51279 | georg.brandl | 2006-08-14 14:36:06 +0200 (Mon, 14 Aug 2006) | 3 lines Improve markup in PyUnicode_RichCompare. ........ r51280 | marc-andre.lemburg | 2006-08-14 14:57:27 +0200 (Mon, 14 Aug 2006) | 3 lines Correct an accidentally removed previous patch. ........ r51281 | thomas.heller | 2006-08-14 18:17:41 +0200 (Mon, 14 Aug 2006) | 3 lines Patch #1536908: Add support for AMD64 / OpenBSD. Remove the -no-stack-protector compiler flag for OpenBSD as it has been reported to be unneeded. ........ r51282 | thomas.heller | 2006-08-14 18:20:04 +0200 (Mon, 14 Aug 2006) | 1 line News item for rev 51281. ........ r51283 | georg.brandl | 2006-08-14 22:25:39 +0200 (Mon, 14 Aug 2006) | 3 lines Fix refleak introduced in rev. 51248. ........ r51284 | georg.brandl | 2006-08-14 23:34:08 +0200 (Mon, 14 Aug 2006) | 5 lines Make tabnanny recognize IndentationErrors raised by tokenize. Add a test to test_inspect to make sure indented source is recognized correctly. (fixes #1224621) ........ r51285 | georg.brandl | 2006-08-14 23:42:55 +0200 (Mon, 14 Aug 2006) | 3 lines Patch #1535500: fix segfault in BZ2File.writelines and make sure it raises the correct exceptions. ........ r51287 | georg.brandl | 2006-08-14 23:45:32 +0200 (Mon, 14 Aug 2006) | 3 lines Add an additional test: BZ2File write methods should raise IOError when file is read-only. ........ r51289 | georg.brandl | 2006-08-14 23:55:28 +0200 (Mon, 14 Aug 2006) | 3 lines Patch #1536071: trace.py should now find the full module name of a file correctly even on Windows. ........ r51290 | georg.brandl | 2006-08-15 00:01:24 +0200 (Tue, 15 Aug 2006) | 3 lines Cookie.py shouldn't "bogusly" use string._idmap. ........ r51291 | georg.brandl | 2006-08-15 00:10:24 +0200 (Tue, 15 Aug 2006) | 3 lines Patch #1511317: don't crash on invalid hostname info ........ r51292 | tim.peters | 2006-08-15 02:25:04 +0200 (Tue, 15 Aug 2006) | 2 lines Whitespace normalization. ........ r51293 | neal.norwitz | 2006-08-15 06:14:57 +0200 (Tue, 15 Aug 2006) | 3 lines Georg fixed one of my bugs, so I'll repay him with 2 NEWS entries. Now we're even. :-) ........ r51295 | neal.norwitz | 2006-08-15 06:58:28 +0200 (Tue, 15 Aug 2006) | 8 lines Fix the test for SocketServer so it should pass on cygwin and not fail sporadically on other platforms. This is really a band-aid that doesn't fix the underlying issue in SocketServer. It's not clear if it's worth it to fix SocketServer, however, I opened a bug to track it: http://python.org/sf/1540386 ........ r51296 | neal.norwitz | 2006-08-15 06:59:30 +0200 (Tue, 15 Aug 2006) | 3 lines Update the docstring to use a version a little newer than 1999. This was taken from a Debian patch. Should we update the version for each release? ........ r51298 | neal.norwitz | 2006-08-15 08:29:03 +0200 (Tue, 15 Aug 2006) | 2 lines Subclasses of int/long are allowed to define an __index__. ........ r51300 | thomas.heller | 2006-08-15 15:07:21 +0200 (Tue, 15 Aug 2006) | 1 line Check for NULL return value from new_CArgObject calls. ........ r51303 | kurt.kaiser | 2006-08-16 05:15:26 +0200 (Wed, 16 Aug 2006) | 2 lines The 'with' statement is now a Code Context block opener ........ r51304 | anthony.baxter | 2006-08-16 05:42:26 +0200 (Wed, 16 Aug 2006) | 1 line preparing for 2.5c1 ........ r51305 | anthony.baxter | 2006-08-16 05:58:37 +0200 (Wed, 16 Aug 2006) | 1 line preparing for 2.5c1 - no, really this time ........ r51306 | kurt.kaiser | 2006-08-16 07:01:42 +0200 (Wed, 16 Aug 2006) | 9 lines Patch #1540892: site.py Quitter() class attempts to close sys.stdin before raising SystemExit, allowing IDLE to honor quit() and exit(). M Lib/site.py M Lib/idlelib/PyShell.py M Lib/idlelib/CREDITS.txt M Lib/idlelib/NEWS.txt M Misc/NEWS ........ r51307 | ka-ping.yee | 2006-08-16 09:02:50 +0200 (Wed, 16 Aug 2006) | 6 lines Update code and tests to support the 'bytes_le' attribute (for little-endian byte order on Windows), and to work around clocks with low resolution yielding duplicate UUIDs. Anthony Baxter has approved this change. ........ r51308 | kurt.kaiser | 2006-08-16 09:04:17 +0200 (Wed, 16 Aug 2006) | 2 lines Get quit() and exit() to work cleanly when not using subprocess. ........ r51309 | marc-andre.lemburg | 2006-08-16 10:13:26 +0200 (Wed, 16 Aug 2006) | 2 lines Revert to having static version numbers again. ........ r51310 | martin.v.loewis | 2006-08-16 14:55:10 +0200 (Wed, 16 Aug 2006) | 2 lines Build _hashlib on Windows. Build OpenSSL with masm assembler code. Fixes #1535502. ........ r51311 | thomas.heller | 2006-08-16 15:03:11 +0200 (Wed, 16 Aug 2006) | 6 lines Add commented assert statements to check that the result of PyObject_stgdict() and PyType_stgdict() calls are non-NULL before dereferencing the result. Hopefully this fixes what klocwork is complaining about. Fix a few other nits as well. ........ r51312 | anthony.baxter | 2006-08-16 15:08:25 +0200 (Wed, 16 Aug 2006) | 1 line news entry for 51307 ........ r51313 | andrew.kuchling | 2006-08-16 15:22:20 +0200 (Wed, 16 Aug 2006) | 1 line Add UnicodeWarning ........ r51314 | andrew.kuchling | 2006-08-16 15:41:52 +0200 (Wed, 16 Aug 2006) | 1 line Bump document version to 1.0; remove pystone paragraph ........ r51315 | andrew.kuchling | 2006-08-16 15:51:32 +0200 (Wed, 16 Aug 2006) | 1 line Link to docs; remove an XXX comment ........ r51316 | martin.v.loewis | 2006-08-16 15:58:51 +0200 (Wed, 16 Aug 2006) | 1 line Make cl build step compile-only (/c). Remove libs from source list. ........ r51317 | thomas.heller | 2006-08-16 16:07:44 +0200 (Wed, 16 Aug 2006) | 5 lines The __repr__ method of a NULL py_object does no longer raise an exception. Remove a stray '?' character from the exception text when the value is retrieved of such an object. Includes tests. ........ r51318 | andrew.kuchling | 2006-08-16 16:18:23 +0200 (Wed, 16 Aug 2006) | 1 line Update bug/patch counts ........ r51319 | andrew.kuchling | 2006-08-16 16:21:14 +0200 (Wed, 16 Aug 2006) | 1 line Wording/typo fixes ........ r51320 | thomas.heller | 2006-08-16 17:10:12 +0200 (Wed, 16 Aug 2006) | 9 lines Remove the special casing of Py_None when converting the return value of the Python part of a callback function to C. If it cannot be converted, call PyErr_WriteUnraisable with the exception we got. Before, arbitrary data has been passed to the calling C code in this case. (I'm not really sure the NEWS entry is understandable, but I cannot find better words) ........ r51321 | marc-andre.lemburg | 2006-08-16 18:11:01 +0200 (Wed, 16 Aug 2006) | 2 lines Add NEWS item mentioning the reverted distutils version number patch. ........ r51322 | fredrik.lundh | 2006-08-16 18:47:07 +0200 (Wed, 16 Aug 2006) | 5 lines SF#1534630 ignore data that arrives before the opening start tag ........ r51324 | andrew.kuchling | 2006-08-16 19:11:18 +0200 (Wed, 16 Aug 2006) | 1 line Grammar fix ........ r51328 | thomas.heller | 2006-08-16 20:02:11 +0200 (Wed, 16 Aug 2006) | 12 lines Tutorial: Clarify somewhat how parameters are passed to functions (especially explain what integer means). Correct the table - Python integers and longs can both be used. Further clarification to the table comparing ctypes types, Python types, and C types. Reference: Replace integer by C ``int`` where it makes sense. ........ r51329 | kurt.kaiser | 2006-08-16 23:45:59 +0200 (Wed, 16 Aug 2006) | 8 lines File menu hotkeys: there were three 'p' assignments. Reassign the 'Save Copy As' and 'Print' hotkeys to 'y' and 't'. Change the Shell menu hotkey from 's' to 'l'. M Bindings.py M PyShell.py M NEWS.txt ........ r51330 | neil.schemenauer | 2006-08-17 01:38:05 +0200 (Thu, 17 Aug 2006) | 3 lines Fix a bug in the ``compiler`` package that caused invalid code to be generated for generator expressions. ........ r51342 | martin.v.loewis | 2006-08-17 21:19:32 +0200 (Thu, 17 Aug 2006) | 3 lines Merge 51340 and 51341 from 2.5 branch: Leave tk build directory to restore original path. Invoke debug mk1mf.pl after running Configure. ........ r51354 | martin.v.loewis | 2006-08-18 05:47:18 +0200 (Fri, 18 Aug 2006) | 3 lines Bug #1541863: uuid.uuid1 failed to generate unique identifiers on systems with low clock resolution. ........ r51355 | neal.norwitz | 2006-08-18 05:57:54 +0200 (Fri, 18 Aug 2006) | 1 line Add template for 2.6 on HEAD ........ r51356 | neal.norwitz | 2006-08-18 06:01:38 +0200 (Fri, 18 Aug 2006) | 1 line More post-release wibble ........ r51357 | neal.norwitz | 2006-08-18 06:58:33 +0200 (Fri, 18 Aug 2006) | 1 line Try to get Windows bots working again ........ r51358 | neal.norwitz | 2006-08-18 07:10:00 +0200 (Fri, 18 Aug 2006) | 1 line Try to get Windows bots working again. Take 2 ........ r51359 | neal.norwitz | 2006-08-18 07:39:20 +0200 (Fri, 18 Aug 2006) | 1 line Try to get Unix bots install working again. ........ r51360 | neal.norwitz | 2006-08-18 07:41:46 +0200 (Fri, 18 Aug 2006) | 1 line Set version to 2.6a0, seems more consistent. ........ r51362 | neal.norwitz | 2006-08-18 08:14:52 +0200 (Fri, 18 Aug 2006) | 1 line More version wibble ........ r51364 | georg.brandl | 2006-08-18 09:27:59 +0200 (Fri, 18 Aug 2006) | 4 lines Bug #1541682: Fix example in the "Refcount details" API docs. Additionally, remove a faulty example showing PySequence_SetItem applied to a newly created list object and add notes that this isn't a good idea. ........ r51366 | anthony.baxter | 2006-08-18 09:29:02 +0200 (Fri, 18 Aug 2006) | 3 lines Updating IDLE's version number to match Python's (as per python-dev discussion). ........ r51367 | anthony.baxter | 2006-08-18 09:30:07 +0200 (Fri, 18 Aug 2006) | 1 line RPM specfile updates ........ r51368 | georg.brandl | 2006-08-18 09:35:47 +0200 (Fri, 18 Aug 2006) | 2 lines Typo in tp_clear docs. ........ r51378 | andrew.kuchling | 2006-08-18 15:57:13 +0200 (Fri, 18 Aug 2006) | 1 line Minor edits ........ r51379 | thomas.heller | 2006-08-18 16:38:46 +0200 (Fri, 18 Aug 2006) | 6 lines Add asserts to check for 'impossible' NULL values, with comments. In one place where I'n not 1000% sure about the non-NULL, raise a RuntimeError for safety. This should fix the klocwork issues that Neal sent me. If so, it should be applied to the release25-maint branch also. ........ r51400 | neal.norwitz | 2006-08-19 06:22:33 +0200 (Sat, 19 Aug 2006) | 5 lines Move initialization of interned strings to before allocating the object so we don't leak op. (Fixes an earlier patch to this code) Klockwork #350 ........ r51401 | neal.norwitz | 2006-08-19 06:23:04 +0200 (Sat, 19 Aug 2006) | 4 lines Move assert to after NULL check, otherwise we deref NULL in the assert. Klocwork #307 ........ r51402 | neal.norwitz | 2006-08-19 06:25:29 +0200 (Sat, 19 Aug 2006) | 2 lines SF #1542693: Remove semi-colon at end of PyImport_ImportModuleEx macro ........ r51403 | neal.norwitz | 2006-08-19 06:28:55 +0200 (Sat, 19 Aug 2006) | 6 lines Move initialization to after the asserts for non-NULL values. Klocwork 286-287. (I'm not backporting this, but if someone wants to, feel free.) ........ r51404 | neal.norwitz | 2006-08-19 06:52:03 +0200 (Sat, 19 Aug 2006) | 6 lines Handle PyString_FromInternedString() failing (unlikely, but possible). Klocwork #325 (I'm not backporting this, but if someone wants to, feel free.) ........ r51416 | georg.brandl | 2006-08-20 15:15:39 +0200 (Sun, 20 Aug 2006) | 2 lines Patch #1542948: fix urllib2 header casing issue. With new test. ........ r51428 | jeremy.hylton | 2006-08-21 18:19:37 +0200 (Mon, 21 Aug 2006) | 3 lines Move peephole optimizer to separate file. ........ r51429 | jeremy.hylton | 2006-08-21 18:20:29 +0200 (Mon, 21 Aug 2006) | 2 lines Move peephole optimizer to separate file. (Forgot .h in previous checkin.) ........ r51432 | neal.norwitz | 2006-08-21 19:59:46 +0200 (Mon, 21 Aug 2006) | 5 lines Fix bug #1543303, tarfile adds padding that breaks gunzip. Patch # 1543897. Will backport to 2.5 ........ r51433 | neal.norwitz | 2006-08-21 20:01:30 +0200 (Mon, 21 Aug 2006) | 2 lines Add assert to make Klocwork happy (#276) ........ Modified: python/branches/p3yk/Doc/api/abstract.tex ============================================================================== --- python/branches/p3yk/Doc/api/abstract.tex (original) +++ python/branches/p3yk/Doc/api/abstract.tex Mon Aug 21 21:07:27 2006 @@ -5,6 +5,10 @@ numerical types, or all sequence types). When used on object types for which they do not apply, they will raise a Python exception. +It is not possible to use these functions on objects that are not properly +initialized, such as a list object that has been created by +\cfunction{PyList_New()}, but whose items have not been set to some +non-\code{NULL} value yet. \section{Object Protocol \label{object}} @@ -673,12 +677,31 @@ \samp{float(\var{o})}.\bifuncindex{float} \end{cfuncdesc} -\begin{cfuncdesc}{Py_ssize_t}{PyNumber_Index}{PyObject *o} - Returns the \var{o} converted to a Py_ssize_t integer on success, or - -1 with an exception raised on failure. +\begin{cfuncdesc}{PyObject*}{PyNumber_Index}{PyObject *o} + Returns the \var{o} converted to a Python int or long on success or \NULL{} + with a TypeError exception raised on failure. \versionadded{2.5} \end{cfuncdesc} +\begin{cfuncdesc}{Py_ssize_t}{PyNumber_AsSsize_t}{PyObject *o, PyObject *exc} + Returns \var{o} converted to a Py_ssize_t value if \var{o} + can be interpreted as an integer. If \var{o} can be converted to a Python + int or long but the attempt to convert to a Py_ssize_t value + would raise an \exception{OverflowError}, then the \var{exc} argument + is the type of exception that will be raised (usually \exception{IndexError} + or \exception{OverflowError}). If \var{exc} is \NULL{}, then the exception + is cleared and the value is clipped to \var{PY_SSIZE_T_MIN} + for a negative integer or \var{PY_SSIZE_T_MAX} for a positive integer. + \versionadded{2.5} +\end{cfuncdesc} + +\begin{cfuncdesc}{int}{PyIndex_Check}{PyObject *o} + Returns True if \var{o} is an index integer (has the nb_index slot of + the tp_as_number structure filled in). + \versionadded{2.5} +\end{cfuncdesc} + + \section{Sequence Protocol \label{sequence}} \begin{cfuncdesc}{int}{PySequence_Check}{PyObject *o} Modified: python/branches/p3yk/Doc/api/concrete.tex ============================================================================== --- python/branches/p3yk/Doc/api/concrete.tex (original) +++ python/branches/p3yk/Doc/api/concrete.tex Mon Aug 21 21:07:27 2006 @@ -1560,6 +1560,26 @@ greater than, respectively. \end{cfuncdesc} +\begin{cfuncdesc}{int}{PyUnicode_RichCompare}{PyObject *left, + PyObject *right, + int op} + + Rich compare two unicode strings and return one of the following: + \begin{itemize} + \item \code{NULL} in case an exception was raised + \item \constant{Py_True} or \constant{Py_False} for successful comparisons + \item \constant{Py_NotImplemented} in case the type combination is unknown + \end{itemize} + + Note that \constant{Py_EQ} and \constant{Py_NE} comparisons can cause a + \exception{UnicodeWarning} in case the conversion of the arguments to + Unicode fails with a \exception{UnicodeDecodeError}. + + Possible values for \var{op} are + \constant{Py_GT}, \constant{Py_GE}, \constant{Py_EQ}, + \constant{Py_NE}, \constant{Py_LT}, and \constant{Py_LE}. +\end{cfuncdesc} + \begin{cfuncdesc}{PyObject*}{PyUnicode_Format}{PyObject *format, PyObject *args} Return a new string object from \var{format} and \var{args}; this @@ -1820,6 +1840,11 @@ \begin{cfuncdesc}{PyObject*}{PyList_New}{Py_ssize_t len} Return a new list of length \var{len} on success, or \NULL{} on failure. + \note{If \var{length} is greater than zero, the returned list object's + items are set to \code{NULL}. Thus you cannot use abstract + API functions such as \cfunction{PySequence_SetItem()} + or expose the object to Python code before setting all items to a + real object with \cfunction{PyList_SetItem()}.} \end{cfuncdesc} \begin{cfuncdesc}{Py_ssize_t}{PyList_Size}{PyObject *list} Modified: python/branches/p3yk/Doc/api/exceptions.tex ============================================================================== --- python/branches/p3yk/Doc/api/exceptions.tex (original) +++ python/branches/p3yk/Doc/api/exceptions.tex Mon Aug 21 21:07:27 2006 @@ -285,10 +285,11 @@ names are \samp{PyExc_} followed by the Python exception name. These have the type \ctype{PyObject*}; they are all class objects. Their names are \cdata{PyExc_Warning}, \cdata{PyExc_UserWarning}, - \cdata{PyExc_DeprecationWarning}, \cdata{PyExc_SyntaxWarning}, - \cdata{PyExc_RuntimeWarning}, and \cdata{PyExc_FutureWarning}. - \cdata{PyExc_Warning} is a subclass of \cdata{PyExc_Exception}; the - other warning categories are subclasses of \cdata{PyExc_Warning}. + \cdata{PyExc_UnicodeWarning}, \cdata{PyExc_DeprecationWarning}, + \cdata{PyExc_SyntaxWarning}, \cdata{PyExc_RuntimeWarning}, and + \cdata{PyExc_FutureWarning}. \cdata{PyExc_Warning} is a subclass of + \cdata{PyExc_Exception}; the other warning categories are subclasses + of \cdata{PyExc_Warning}. For information about warning control, see the documentation for the \module{warnings} module and the \programopt{-W} option in the Modified: python/branches/p3yk/Doc/api/init.tex ============================================================================== --- python/branches/p3yk/Doc/api/init.tex (original) +++ python/branches/p3yk/Doc/api/init.tex Mon Aug 21 21:07:27 2006 @@ -696,15 +696,15 @@ \end{cfuncdesc} \begin{cfuncdesc}{int}{PyThreadState_SetAsyncExc}{long id, PyObject *exc} - Asynchronously raise an exception in a thread. + Asynchronously raise an exception in a thread. The \var{id} argument is the thread id of the target thread; \var{exc} is the exception object to be raised. This function does not steal any references to \var{exc}. - To prevent naive misuse, you must write your own C extension - to call this. Must be called with the GIL held. - Returns the number of thread states modified; if it returns a number - greater than one, you're in trouble, and you should call it again - with \var{exc} set to \constant{NULL} to revert the effect. + To prevent naive misuse, you must write your own C extension + to call this. Must be called with the GIL held. + Returns the number of thread states modified; this is normally one, but + will be zero if the thread id isn't found. If \var{exc} is + \constant{NULL}, the pending exception (if any) for the thread is cleared. This raises no exceptions. \versionadded{2.3} \end{cfuncdesc} Modified: python/branches/p3yk/Doc/api/intro.tex ============================================================================== --- python/branches/p3yk/Doc/api/intro.tex (original) +++ python/branches/p3yk/Doc/api/intro.tex Mon Aug 21 21:07:27 2006 @@ -225,25 +225,10 @@ \cfunction{PyTuple_SetItem()} for tuples that you are creating yourself. -Equivalent code for populating a list can be written using -\cfunction{PyList_New()} and \cfunction{PyList_SetItem()}. Such code -can also use \cfunction{PySequence_SetItem()}; this illustrates the -difference between the two (the extra \cfunction{Py_DECREF()} calls): +Equivalent code for populating a list can be written using +\cfunction{PyList_New()} and \cfunction{PyList_SetItem()}. -\begin{verbatim} -PyObject *l, *x; - -l = PyList_New(3); -x = PyInt_FromLong(1L); -PySequence_SetItem(l, 0, x); Py_DECREF(x); -x = PyInt_FromLong(2L); -PySequence_SetItem(l, 1, x); Py_DECREF(x); -x = PyString_FromString("three"); -PySequence_SetItem(l, 2, x); Py_DECREF(x); -\end{verbatim} - -You might find it strange that the ``recommended'' approach takes more -code. However, in practice, you will rarely use these ways of +However, in practice, you will rarely use these ways of creating and populating a tuple or list. There's a generic function, \cfunction{Py_BuildValue()}, that can create most common objects from C values, directed by a \dfn{format string}. For example, the @@ -251,10 +236,10 @@ also takes care of the error checking): \begin{verbatim} -PyObject *t, *l; +PyObject *tuple, *list; -t = Py_BuildValue("(iis)", 1, 2, "three"); -l = Py_BuildValue("[iis]", 1, 2, "three"); +tuple = Py_BuildValue("(iis)", 1, 2, "three"); +list = Py_BuildValue("[iis]", 1, 2, "three"); \end{verbatim} It is much more common to use \cfunction{PyObject_SetItem()} and @@ -276,8 +261,12 @@ if (n < 0) return -1; for (i = 0; i < n; i++) { - if (PyObject_SetItem(target, i, item) < 0) + PyObject *index = PyInt_FromLong(i); + if (!index) + return -1; + if (PyObject_SetItem(target, index, item) < 0) return -1; + Py_DECREF(index); } return 0; } Modified: python/branches/p3yk/Doc/api/newtypes.tex ============================================================================== --- python/branches/p3yk/Doc/api/newtypes.tex (original) +++ python/branches/p3yk/Doc/api/newtypes.tex Mon Aug 21 21:07:27 2006 @@ -979,7 +979,7 @@ More information about Python's garbage collection scheme can be found in section \ref{supporting-cycle-detection}. - This field is inherited by subtypes together with \member{tp_clear} + This field is inherited by subtypes together with \member{tp_traverse} and the \constant{Py_TPFLAGS_HAVE_GC} flag bit: the flag bit, \member{tp_traverse}, and \member{tp_clear} are all inherited from the base type if they are all zero in the subtype \emph{and} the Modified: python/branches/p3yk/Doc/commontex/boilerplate.tex ============================================================================== --- python/branches/p3yk/Doc/commontex/boilerplate.tex (original) +++ python/branches/p3yk/Doc/commontex/boilerplate.tex Mon Aug 21 21:07:27 2006 @@ -5,5 +5,5 @@ Email: \email{docs(a)python.org} } -\date{3rd August, 2006} % XXX update before final release! +\date{\today} % XXX update before final release! \input{patchlevel} % include Python version information Modified: python/branches/p3yk/Doc/dist/dist.tex ============================================================================== --- python/branches/p3yk/Doc/dist/dist.tex (original) +++ python/branches/p3yk/Doc/dist/dist.tex Mon Aug 21 21:07:27 2006 @@ -1992,6 +1992,7 @@ script for this could be \begin{verbatim} from distutils.core import setup +from distutils.extension import Extension setup(name='foobar', version='1.0', ext_modules=[Extension('foo', ['foo.c'])], @@ -2006,6 +2007,7 @@ extension: \begin{verbatim} from distutils.core import setup +from distutils.extension import Extension setup(name='foobar', version='1.0', ext_modules=[Extension('foopkg.foo', ['foo.c'])], Modified: python/branches/p3yk/Doc/lib/libctypes.tex ============================================================================== --- python/branches/p3yk/Doc/lib/libctypes.tex (original) +++ python/branches/p3yk/Doc/lib/libctypes.tex Mon Aug 21 21:07:27 2006 @@ -199,8 +199,13 @@ There are, however, enough ways to crash Python with \code{ctypes}, so you should be careful anyway. -Python integers, strings and unicode strings are the only objects that -can directly be used as parameters in these function calls. +\code{None}, integers, longs, byte strings and unicode strings are the +only native Python objects that can directly be used as parameters in +these function calls. \code{None} is passed as a C \code{NULL} pointer, +byte strings and unicode strings are passed as pointer to the memory +block that contains their data (\code{char *} or \code{wchar{\_}t *}). Python +integers and Python longs are passed as the platforms default C +\code{int} type, their value is masked to fit into the C type. Before we move on calling functions with other parameter types, we have to learn more about \code{ctypes} data types. @@ -227,7 +232,18 @@ \code{char} } { -character +1-character +string +} +\lineiii{ +\class{c{\_}wchar} +} +{ +\code{wchar{\_}t} +} +{ +1-character +unicode string } \lineiii{ \class{c{\_}byte} @@ -236,7 +252,7 @@ \code{char} } { -integer +int/long } \lineiii{ \class{c{\_}ubyte} @@ -245,7 +261,7 @@ \code{unsigned char} } { -integer +int/long } \lineiii{ \class{c{\_}short} @@ -254,7 +270,7 @@ \code{short} } { -integer +int/long } \lineiii{ \class{c{\_}ushort} @@ -263,7 +279,7 @@ \code{unsigned short} } { -integer +int/long } \lineiii{ \class{c{\_}int} @@ -272,7 +288,7 @@ \code{int} } { -integer +int/long } \lineiii{ \class{c{\_}uint} @@ -281,7 +297,7 @@ \code{unsigned int} } { -integer +int/long } \lineiii{ \class{c{\_}long} @@ -290,7 +306,7 @@ \code{long} } { -integer +int/long } \lineiii{ \class{c{\_}ulong} @@ -299,7 +315,7 @@ \code{unsigned long} } { -long +int/long } \lineiii{ \class{c{\_}longlong} @@ -309,7 +325,7 @@ \code{long long} } { -long +int/long } \lineiii{ \class{c{\_}ulonglong} @@ -319,7 +335,7 @@ \code{unsigned long long} } { -long +int/long } \lineiii{ \class{c{\_}float} @@ -368,8 +384,8 @@ \code{void *} } { -integer or -\code{None} +int/long +or \code{None} } \end{tableiii} \end{quote} @@ -554,11 +570,11 @@ \subsubsection{Return types\label{ctypes-return-types}} -By default functions are assumed to return integers. Other return -types can be specified by setting the \member{restype} attribute of the -function object. +By default functions are assumed to return the C \code{int} type. Other +return types can be specified by setting the \member{restype} attribute of +the function object. -Here is a more advanced example, it uses the strchr function, which +Here is a more advanced example, it uses the \code{strchr} function, which expects a string pointer and a char, and returns a pointer to a string: \begin{verbatim} @@ -1611,8 +1627,8 @@ \begin{datadescni}{pythonapi} An instance of \class{PyDLL} that exposes Python C api functions as -attributes. Note that all these functions are assumed to return -integers, which is of course not always the truth, so you have to +attributes. Note that all these functions are assumed to return C +\code{int}, which is of course not always the truth, so you have to assign the correct \member{restype} attribute to use these functions. \end{datadescni} @@ -1642,8 +1658,8 @@ anything. It is possible to assign a callable Python object that is not a -ctypes type, in this case the function is assumed to return an -integer, and the callable will be called with this integer, +ctypes type, in this case the function is assumed to return a +C \code{int}, and the callable will be called with this integer, allowing to do further processing or error checking. Using this is deprecated, for more flexible postprocessing or error checking use a ctypes data type as \member{restype} and assign a callable to the @@ -2283,9 +2299,12 @@ or error information for a function or method call. \end{classdesc*} -\begin{classdesc*}{py_object} -Represents the C \code{PyObject *} datatype. -\end{classdesc*} +\code{py{\_}object} : classdesc* +\begin{quote} + +Represents the C \code{PyObject *} datatype. Calling this with an +without an argument creates a \code{NULL} \code{PyObject *} pointer. +\end{quote} The \code{ctypes.wintypes} module provides quite some other Windows specific data types, for example \code{HWND}, \code{WPARAM}, or \code{DWORD}. @@ -2324,9 +2343,9 @@ the second item specifies the type of the field; it can be any ctypes data type. -For integer type fields, a third optional item can be given. It -must be a small positive integer defining the bit width of the -field. +For integer type fields like \class{c{\_}int}, a third optional item can +be given. It must be a small positive integer defining the bit +width of the field. Field names must be unique within one structure or union. This is not checked, only one field can be accessed when names are Modified: python/branches/p3yk/Doc/lib/libexcs.tex ============================================================================== --- python/branches/p3yk/Doc/lib/libexcs.tex (original) +++ python/branches/p3yk/Doc/lib/libexcs.tex Mon Aug 21 21:07:27 2006 @@ -451,6 +451,11 @@ \versionadded{2.5} \end{excdesc} +\begin{excdesc}{UnicodeWarning} +Base class for warnings related to Unicode. +\versionadded{2.5} +\end{excdesc} + The class hierarchy for built-in exceptions is: \verbatiminput{../../Lib/test/exception_hierarchy.txt} Modified: python/branches/p3yk/Doc/lib/libwarnings.tex ============================================================================== --- python/branches/p3yk/Doc/lib/libwarnings.tex (original) +++ python/branches/p3yk/Doc/lib/libwarnings.tex Mon Aug 21 21:07:27 2006 @@ -76,6 +76,9 @@ \lineii{ImportWarning}{Base category for warnings triggered during the process of importing a module (ignored by default).} + +\lineii{UnicodeWarning}{Base category for warnings related to Unicode.} + \end{tableii} While these are technically built-in exceptions, they are documented Modified: python/branches/p3yk/Doc/whatsnew/whatsnew25.tex ============================================================================== --- python/branches/p3yk/Doc/whatsnew/whatsnew25.tex (original) +++ python/branches/p3yk/Doc/whatsnew/whatsnew25.tex Mon Aug 21 21:07:27 2006 @@ -5,7 +5,7 @@ % Fix XXX comments \title{What's New in Python 2.5} -\release{0.9} +\release{1.0} \author{A.M. Kuchling} \authoraddress{\email{amk(a)amk.ca}} @@ -40,15 +40,14 @@ As well as the language and library additions, other improvements and bugfixes were made throughout the source tree. A search through the -SVN change logs finds there were 334 patches applied and 443 bugs +SVN change logs finds there were 353 patches applied and 458 bugs fixed between Python 2.4 and 2.5. (Both figures are likely to be underestimates.) This article doesn't try to be a complete specification of the new features; instead changes are briefly introduced using helpful examples. For full details, you should always refer to the -documentation for Python 2.5. -% XXX add hyperlink when the documentation becomes available online. +documentation for Python 2.5 at \url{http://docs.python.org}. If you want to understand the complete implementation and design rationale, refer to the PEP for a particular new feature. @@ -751,7 +750,6 @@ database, or rolled back, meaning that the changes are all discarded and the database is unchanged. See any database textbook for more information.) -% XXX find a shorter reference? Let's assume there's an object representing a database connection. Our goal will be to let the user write code like this: @@ -1184,6 +1182,35 @@ # -*- coding: latin1 -*- \end{verbatim} +\item A new warning, \class{UnicodeWarning}, is triggered when +you attempt to compare a Unicode string and an 8-bit string +that can't be converted to Unicode using the default ASCII encoding. +The result of the comparison is false: + +\begin{verbatim} +>>> chr(128) == unichr(128) # Can't convert chr(128) to Unicode +__main__:1: UnicodeWarning: Unicode equal comparison failed + to convert both arguments to Unicode - interpreting them + as being unequal +False +>>> chr(127) == unichr(127) # chr(127) can be converted +True +\end{verbatim} + +Previously this would raise a \class{UnicodeDecodeError} exception, +but in 2.5 this could result in puzzling problems when accessing a +dictionary. If you looked up \code{unichr(128)} and \code{chr(128)} +was being used as a key, you'd get a \class{UnicodeDecodeError} +exception. Other changes in 2.5 resulted in this exception being +raised instead of suppressed by the code in \file{dictobject.c} that +implements dictionaries. + +Raising an exception for such a comparison is strictly correct, but +the change might have broken code, so instead +\class{UnicodeWarning} was introduced. + +(Implemented by Marc-Andr\'e Lemburg.) + \item One error that Python programmers sometimes make is forgetting to include an \file{__init__.py} module in a package directory. Debugging this mistake can be confusing, and usually requires running @@ -1305,9 +1332,6 @@ \end{itemize} -The net result of the 2.5 optimizations is that Python 2.5 runs the -pystone benchmark around XXX\% faster than Python 2.4. - %====================================================================== \section{New, Improved, and Removed Modules\label{modules}} @@ -2423,6 +2447,11 @@ described in section~\ref{pep-342}, it's now possible for \member{gi_frame} to be \code{None}. +\item A new warning, \class{UnicodeWarning}, is triggered when +you attempt to compare a Unicode string and an 8-bit string that can't +be converted to Unicode using the default ASCII encoding. Previously +such comparisons would raise a \class{UnicodeDecodeError} exception. + \item Library: the \module{csv} module is now stricter about multi-line quoted fields. If your files contain newlines embedded within fields, the input should be split into lines in a manner which preserves the Modified: python/branches/p3yk/Include/abstract.h ============================================================================== --- python/branches/p3yk/Include/abstract.h (original) +++ python/branches/p3yk/Include/abstract.h Mon Aug 21 21:07:27 2006 @@ -694,13 +694,26 @@ expression: o1|o2. */ - PyAPI_FUNC(Py_ssize_t) PyNumber_Index(PyObject *); +#define PyIndex_Check(obj) \ + ((obj)->ob_type->tp_as_number != NULL && \ + (obj)->ob_type->tp_as_number->nb_index != NULL) + + PyAPI_FUNC(PyObject *) PyNumber_Index(PyObject *o); /* - Returns the object converted to Py_ssize_t on success - or -1 with an error raised on failure. + Returns the object converted to a Python long or int + or NULL with an error raised on failure. */ + PyAPI_FUNC(Py_ssize_t) PyNumber_AsSsize_t(PyObject *o, PyObject *exc); + + /* + Returns the object converted to Py_ssize_t by going through + PyNumber_Index first. If an overflow error occurs while + converting the int-or-long to Py_ssize_t, then the second argument + is the error-type to return. If it is NULL, then the overflow error + is cleared and the value is clipped. + */ PyAPI_FUNC(PyObject *) PyNumber_Int(PyObject *o); Modified: python/branches/p3yk/Include/code.h ============================================================================== --- python/branches/p3yk/Include/code.h (original) +++ python/branches/p3yk/Include/code.h Mon Aug 21 21:07:27 2006 @@ -88,6 +88,9 @@ PyAPI_FUNC(int) PyCode_CheckLineNumber(PyCodeObject* co, int lasti, PyAddrPair *bounds); +PyAPI_FUNC(PyObject*) PyCode_Optimize(PyObject *code, PyObject* consts, + PyObject *names, PyObject *lineno_obj); + #ifdef __cplusplus } #endif Modified: python/branches/p3yk/Include/import.h ============================================================================== --- python/branches/p3yk/Include/import.h (original) +++ python/branches/p3yk/Include/import.h Mon Aug 21 21:07:27 2006 @@ -22,7 +22,7 @@ PyAPI_FUNC(PyObject *) PyImport_ImportModuleEx( char *name, PyObject *globals, PyObject *locals, PyObject *fromlist); #define PyImport_ImportModuleEx(n, g, l, f) \ - PyImport_ImportModuleLevel(n, g, l, f, -1); + PyImport_ImportModuleLevel(n, g, l, f, -1) PyAPI_FUNC(PyObject *) PyImport_Import(PyObject *name); PyAPI_FUNC(PyObject *) PyImport_ReloadModule(PyObject *m); Modified: python/branches/p3yk/Include/object.h ============================================================================== --- python/branches/p3yk/Include/object.h (original) +++ python/branches/p3yk/Include/object.h Mon Aug 21 21:07:27 2006 @@ -190,7 +190,7 @@ binaryfunc nb_inplace_floor_divide; binaryfunc nb_inplace_true_divide; - lenfunc nb_index; + unaryfunc nb_index; } PyNumberMethods; typedef struct { Modified: python/branches/p3yk/Include/pyerrors.h ============================================================================== --- python/branches/p3yk/Include/pyerrors.h (original) +++ python/branches/p3yk/Include/pyerrors.h Mon Aug 21 21:07:27 2006 @@ -166,6 +166,7 @@ PyAPI_DATA(PyObject *) PyExc_RuntimeWarning; PyAPI_DATA(PyObject *) PyExc_FutureWarning; PyAPI_DATA(PyObject *) PyExc_ImportWarning; +PyAPI_DATA(PyObject *) PyExc_UnicodeWarning; /* Convenience functions */ Modified: python/branches/p3yk/Include/unicodeobject.h ============================================================================== --- python/branches/p3yk/Include/unicodeobject.h (original) +++ python/branches/p3yk/Include/unicodeobject.h Mon Aug 21 21:07:27 2006 @@ -189,6 +189,7 @@ # define PyUnicode_RSplit PyUnicodeUCS2_RSplit # define PyUnicode_Replace PyUnicodeUCS2_Replace # define PyUnicode_Resize PyUnicodeUCS2_Resize +# define PyUnicode_RichCompare PyUnicodeUCS2_RichCompare # define PyUnicode_SetDefaultEncoding PyUnicodeUCS2_SetDefaultEncoding # define PyUnicode_Split PyUnicodeUCS2_Split # define PyUnicode_Splitlines PyUnicodeUCS2_Splitlines @@ -266,6 +267,7 @@ # define PyUnicode_RSplit PyUnicodeUCS4_RSplit # define PyUnicode_Replace PyUnicodeUCS4_Replace # define PyUnicode_Resize PyUnicodeUCS4_Resize +# define PyUnicode_RichCompare PyUnicodeUCS4_RichCompare # define PyUnicode_SetDefaultEncoding PyUnicodeUCS4_SetDefaultEncoding # define PyUnicode_Split PyUnicodeUCS4_Split # define PyUnicode_Splitlines PyUnicodeUCS4_Splitlines @@ -1139,6 +1141,28 @@ PyObject *right /* Right string */ ); +/* Rich compare two strings and return one of the following: + + - NULL in case an exception was raised + - Py_True or Py_False for successfuly comparisons + - Py_NotImplemented in case the type combination is unknown + + Note that Py_EQ and Py_NE comparisons can cause a UnicodeWarning in + case the conversion of the arguments to Unicode fails with a + UnicodeDecodeError. + + Possible values for op: + + Py_GT, Py_GE, Py_EQ, Py_NE, Py_LT, Py_LE + +*/ + +PyAPI_FUNC(PyObject *) PyUnicode_RichCompare( + PyObject *left, /* Left string */ + PyObject *right, /* Right string */ + int op /* Operation: Py_EQ, Py_NE, Py_GT, etc. */ + ); + /* Apply a argument tuple or dictionary to a format string and return the resulting Unicode string. */ Modified: python/branches/p3yk/Lib/Cookie.py ============================================================================== --- python/branches/p3yk/Lib/Cookie.py (original) +++ python/branches/p3yk/Lib/Cookie.py Mon Aug 21 21:07:27 2006 @@ -305,8 +305,10 @@ '\375' : '\\375', '\376' : '\\376', '\377' : '\\377' } +_idmap = ''.join(chr(x) for x in xrange(256)) + def _quote(str, LegalChars=_LegalChars, - idmap=string._idmap, translate=string.translate): + idmap=_idmap, translate=string.translate): # # If the string does not need to be double-quoted, # then just return the string. Otherwise, surround @@ -440,7 +442,7 @@ def set(self, key, val, coded_val, LegalChars=_LegalChars, - idmap=string._idmap, translate=string.translate ): + idmap=_idmap, translate=string.translate): # First we verify that the key isn't a reserved word # Second we make sure it only contains legal characters if key.lower() in self._reserved: Modified: python/branches/p3yk/Lib/cgi.py ============================================================================== --- python/branches/p3yk/Lib/cgi.py (original) +++ python/branches/p3yk/Lib/cgi.py Mon Aug 21 21:07:27 2006 @@ -251,6 +251,10 @@ XXX This should really be subsumed by FieldStorage altogether -- no point in having two implementations of the same parsing algorithm. + Also, FieldStorage protects itself better against certain DoS attacks + by limiting the size of the data read in one chunk. The API here + does not support that kind of protection. This also affects parse() + since it can call parse_multipart(). """ boundary = "" @@ -691,7 +695,7 @@ def read_lines_to_eof(self): """Internal: read lines until EOF.""" while 1: - line = self.fp.readline() + line = self.fp.readline(1<<16) if not line: self.done = -1 break @@ -702,12 +706,13 @@ next = "--" + self.outerboundary last = next + "--" delim = "" + last_line_lfend = True while 1: - line = self.fp.readline() + line = self.fp.readline(1<<16) if not line: self.done = -1 break - if line[:2] == "--": + if line[:2] == "--" and last_line_lfend: strippedline = line.strip() if strippedline == next: break @@ -718,11 +723,14 @@ if line[-2:] == "\r\n": delim = "\r\n" line = line[:-2] + last_line_lfend = True elif line[-1] == "\n": delim = "\n" line = line[:-1] + last_line_lfend = True else: delim = "" + last_line_lfend = False self.__write(odelim + line) def skip_lines(self): @@ -731,18 +739,20 @@ return next = "--" + self.outerboundary last = next + "--" + last_line_lfend = True while 1: - line = self.fp.readline() + line = self.fp.readline(1<<16) if not line: self.done = -1 break - if line[:2] == "--": + if line[:2] == "--" and last_line_lfend: strippedline = line.strip() if strippedline == next: break if strippedline == last: self.done = 1 break + last_line_lfend = line.endswith('\n') def make_file(self, binary=None): """Overridable: return a readable & writable file. Modified: python/branches/p3yk/Lib/compiler/ast.py ============================================================================== --- python/branches/p3yk/Lib/compiler/ast.py (original) +++ python/branches/p3yk/Lib/compiler/ast.py Mon Aug 21 21:07:27 2006 @@ -583,11 +583,9 @@ def __init__(self, code, lineno=None): self.code = code self.lineno = lineno - self.argnames = ['[outmost-iterable]'] + self.argnames = ['.0'] self.varargs = self.kwargs = None - - def getChildren(self): return self.code, Modified: python/branches/p3yk/Lib/compiler/pycodegen.py ============================================================================== --- python/branches/p3yk/Lib/compiler/pycodegen.py (original) +++ python/branches/p3yk/Lib/compiler/pycodegen.py Mon Aug 21 21:07:27 2006 @@ -656,18 +656,19 @@ stack = [] for i, for_ in zip(range(len(node.quals)), node.quals): - start, anchor = self.visit(for_) + start, anchor, end = self.visit(for_) cont = None for if_ in for_.ifs: if cont is None: cont = self.newBlock() self.visit(if_, cont) - stack.insert(0, (start, cont, anchor)) + stack.insert(0, (start, cont, anchor, end)) self.visit(node.expr) self.emit('YIELD_VALUE') + self.emit('POP_TOP') - for start, cont, anchor in stack: + for start, cont, anchor, end in stack: if cont: skip_one = self.newBlock() self.emit('JUMP_FORWARD', skip_one) @@ -676,14 +677,22 @@ self.nextBlock(skip_one) self.emit('JUMP_ABSOLUTE', start) self.startBlock(anchor) + self.emit('POP_BLOCK') + self.setups.pop() + self.startBlock(end) + self.emit('LOAD_CONST', None) def visitGenExprFor(self, node): start = self.newBlock() anchor = self.newBlock() + end = self.newBlock() + + self.setups.push((LOOP, start)) + self.emit('SETUP_LOOP', end) if node.is_outmost: - self.loadName('[outmost-iterable]') + self.loadName('.0') else: self.visit(node.iter) self.emit('GET_ITER') @@ -693,7 +702,7 @@ self.emit('FOR_ITER', anchor) self.nextBlock() self.visit(node.assign) - return start, anchor + return start, anchor, end def visitGenExprIf(self, node, branch): self.set_lineno(node, force=True) Modified: python/branches/p3yk/Lib/compiler/symbols.py ============================================================================== --- python/branches/p3yk/Lib/compiler/symbols.py (original) +++ python/branches/p3yk/Lib/compiler/symbols.py Mon Aug 21 21:07:27 2006 @@ -188,7 +188,7 @@ i = self.__counter self.__counter += 1 self.__super_init("generator expression<%d>"%i, module, klass) - self.add_param('[outmost-iterable]') + self.add_param('.0') def get_names(self): keys = Scope.get_names(self) Modified: python/branches/p3yk/Lib/ctypes/__init__.py ============================================================================== --- python/branches/p3yk/Lib/ctypes/__init__.py (original) +++ python/branches/p3yk/Lib/ctypes/__init__.py Mon Aug 21 21:07:27 2006 @@ -5,7 +5,7 @@ import os as _os, sys as _sys -__version__ = "1.0.0" +__version__ = "1.0.1" from _ctypes import Union, Structure, Array from _ctypes import _Pointer @@ -135,6 +135,11 @@ class py_object(_SimpleCData): _type_ = "O" + def __repr__(self): + try: + return super(py_object, self).__repr__() + except ValueError: + return "%s(<NULL>)" % type(self).__name__ class c_short(_SimpleCData): _type_ = "h" Modified: python/branches/p3yk/Lib/ctypes/test/test_functions.py ============================================================================== --- python/branches/p3yk/Lib/ctypes/test/test_functions.py (original) +++ python/branches/p3yk/Lib/ctypes/test/test_functions.py Mon Aug 21 21:07:27 2006 @@ -222,6 +222,7 @@ def callback(v): args.append(v) + return v CallBack = CFUNCTYPE(c_int, c_int) Modified: python/branches/p3yk/Lib/ctypes/test/test_numbers.py ============================================================================== --- python/branches/p3yk/Lib/ctypes/test/test_numbers.py (original) +++ python/branches/p3yk/Lib/ctypes/test/test_numbers.py Mon Aug 21 21:07:27 2006 @@ -19,7 +19,7 @@ result.append((min(a, b, c, d), max(a, b, c, d))) return result -ArgType = type(c_int(0)._as_parameter_) +ArgType = type(byref(c_int(0))) unsigned_types = [c_ubyte, c_ushort, c_uint, c_ulong] signed_types = [c_byte, c_short, c_int, c_long, c_longlong] @@ -80,19 +80,6 @@ for t in signed_types + unsigned_types + float_types: self.failUnlessEqual(ArgType, type(t.from_param(0))) - def test_as_parameter(self): - # The _as_parameter_ property must also - # be a PyCArgObject instance - for t in signed_types + unsigned_types + float_types: - parm = t()._as_parameter_ - self.failUnlessEqual(ArgType, type(parm)) - - # _as_parameter_ is readonly! - # - # Python 2.3 and 2.4 raise a TypeError when trying to set - # a readonly attribute, 2.5 raises an AttributeError. - self.assertRaises((AttributeError, TypeError), setattr, t(), "_as_parameter_", None) - def test_byref(self): # calling byref returns also a PyCArgObject instance for t in signed_types + unsigned_types + float_types: Modified: python/branches/p3yk/Lib/ctypes/test/test_prototypes.py ============================================================================== --- python/branches/p3yk/Lib/ctypes/test/test_prototypes.py (original) +++ python/branches/p3yk/Lib/ctypes/test/test_prototypes.py Mon Aug 21 21:07:27 2006 @@ -125,13 +125,18 @@ self.failUnlessEqual(None, func(c_wchar_p(None))) self.failUnlessEqual(u"123", func(c_wchar_p(u"123"))) -## def test_instance(self): -## func = testdll._testfunc_p_p + def test_instance(self): + func = testdll._testfunc_p_p + func.restype = c_void_p -## class X: -## _as_parameter_ = 0 + class X: + _as_parameter_ = None -## self.failUnlessEqual(0, func(X())) + func.argtypes = c_void_p, + self.failUnlessEqual(None, func(X())) + + func.argtypes = None + self.failUnlessEqual(None, func(X())) try: c_wchar Modified: python/branches/p3yk/Lib/ctypes/test/test_python_api.py ============================================================================== --- python/branches/p3yk/Lib/ctypes/test/test_python_api.py (original) +++ python/branches/p3yk/Lib/ctypes/test/test_python_api.py Mon Aug 21 21:07:27 2006 @@ -78,5 +78,10 @@ # not enough arguments self.failUnlessRaises(TypeError, PyOS_snprintf, buf) + def test_pyobject_repr(self): + self.failUnlessEqual(repr(py_object()), "py_object(<NULL>)") + self.failUnlessEqual(repr(py_object(42)), "py_object(42)") + self.failUnlessEqual(repr(py_object(object)), "py_object(%r)" % object) + if __name__ == "__main__": unittest.main() Modified: python/branches/p3yk/Lib/distutils/__init__.py ============================================================================== --- python/branches/p3yk/Lib/distutils/__init__.py (original) +++ python/branches/p3yk/Lib/distutils/__init__.py Mon Aug 21 21:07:27 2006 @@ -12,6 +12,12 @@ __revision__ = "$Id$" -import sys -__version__ = "%d.%d.%d" % sys.version_info[:3] -del sys +# Distutils version +# +# Please coordinate with Marc-Andre Lemburg <mal(a)egenix.com> when adding +# new features to distutils that would warrant bumping the version number. +# +# In general, major and minor version should loosely follow the Python +# version number the distutils code was shipped with. +# +__version__ = "2.5.0" Modified: python/branches/p3yk/Lib/idlelib/Bindings.py ============================================================================== --- python/branches/p3yk/Lib/idlelib/Bindings.py (original) +++ python/branches/p3yk/Lib/idlelib/Bindings.py Mon Aug 21 21:07:27 2006 @@ -22,9 +22,9 @@ None, ('_Save', '<<save-window>>'), ('Save _As...', '<<save-window-as-file>>'), - ('Save Co_py As...', '<<save-copy-of-window-as-file>>'), + ('Save Cop_y As...', '<<save-copy-of-window-as-file>>'), None, - ('_Print Window', '<<print-window>>'), + ('Prin_t Window', '<<print-window>>'), None, ('_Close', '<<close-window>>'), ('E_xit', '<<close-all-windows>>'), Modified: python/branches/p3yk/Lib/idlelib/CREDITS.txt ============================================================================== --- python/branches/p3yk/Lib/idlelib/CREDITS.txt (original) +++ python/branches/p3yk/Lib/idlelib/CREDITS.txt Mon Aug 21 21:07:27 2006 @@ -24,8 +24,8 @@ integration, debugger integration and persistent breakpoints). Scott David Daniels, Tal Einat, Hernan Foffani, Christos Georgiou, -Martin v. L�wis, Jason Orendorff, Josh Robb, Nigel Rowe, Bruce Sherwood, -and Jeff Shute have submitted useful patches. Thanks, guys! +Jim Jewett, Martin v. L�wis, Jason Orendorff, Josh Robb, Nigel Rowe, +Bruce Sherwood, and Jeff Shute have submitted useful patches. Thanks, guys! For additional details refer to NEWS.txt and Changelog. Modified: python/branches/p3yk/Lib/idlelib/CodeContext.py ============================================================================== --- python/branches/p3yk/Lib/idlelib/CodeContext.py (original) +++ python/branches/p3yk/Lib/idlelib/CodeContext.py Mon Aug 21 21:07:27 2006 @@ -15,7 +15,7 @@ from sys import maxint as INFINITY BLOCKOPENERS = set(["class", "def", "elif", "else", "except", "finally", "for", - "if", "try", "while"]) + "if", "try", "while", "with"]) UPDATEINTERVAL = 100 # millisec FONTUPDATEINTERVAL = 1000 # millisec Modified: python/branches/p3yk/Lib/idlelib/NEWS.txt ============================================================================== --- python/branches/p3yk/Lib/idlelib/NEWS.txt (original) +++ python/branches/p3yk/Lib/idlelib/NEWS.txt Mon Aug 21 21:07:27 2006 @@ -1,7 +1,29 @@ +What's New in IDLE 2.6a1? +========================= + +*Release date: XX-XXX-200X* + +- IDLE's version number takes a big jump to match the version number of + the Python release of which it's a part. + + What's New in IDLE 1.2c1? ========================= -*Release date: XX-AUG-2006* +*Release date: 17-AUG-2006* + +- File menu hotkeys: there were three 'p' assignments. Reassign the + 'Save Copy As' and 'Print' hotkeys to 'y' and 't'. Change the + Shell hotkey from 's' to 'l'. + +- IDLE honors new quit() and exit() commands from site.py Quitter() object. + Patch 1540892, Jim Jewett + +- The 'with' statement is now a Code Context block opener. + Patch 1540851, Jim Jewett + +- Retrieval of previous shell command was not always preserving indentation + (since 1.2a1) Patch 1528468 Tal Einat. - Changing tokenize (39046) to detect dedent broke tabnanny check (since 1.2a1) Modified: python/branches/p3yk/Lib/idlelib/PyShell.py ============================================================================== --- python/branches/p3yk/Lib/idlelib/PyShell.py (original) +++ python/branches/p3yk/Lib/idlelib/PyShell.py Mon Aug 21 21:07:27 2006 @@ -478,9 +478,6 @@ import sys as _sys _sys.path = %r del _sys - _msg = 'Use File/Exit or your end-of-file key to quit IDLE' - __builtins__.quit = __builtins__.exit = _msg - del _msg \n""" % (sys.path,)) active_seq = None @@ -514,7 +511,10 @@ print >>sys.__stderr__, errmsg, what print >>console, errmsg, what # we received a response to the currently active seq number: - self.tkconsole.endexecuting() + try: + self.tkconsole.endexecuting() + except AttributeError: # shell may have closed + pass # Reschedule myself if not self.tkconsole.closing: self.tkconsole.text.after(self.tkconsole.pollinterval, @@ -593,7 +593,7 @@ source = source.encode(IOBinding.encoding) except UnicodeError: self.tkconsole.resetoutput() - self.write("Unsupported characters in input") + self.write("Unsupported characters in input\n") return try: # InteractiveInterpreter.runsource() calls its runcode() method, @@ -713,14 +713,17 @@ else: exec code in self.locals except SystemExit: - if tkMessageBox.askyesno( - "Exit?", - "Do you want to exit altogether?", - default="yes", - master=self.tkconsole.text): - raise + if not self.tkconsole.closing: + if tkMessageBox.askyesno( + "Exit?", + "Do you want to exit altogether?", + default="yes", + master=self.tkconsole.text): + raise + else: + self.showtraceback() else: - self.showtraceback() + raise except: if use_subprocess: print >> self.tkconsole.stderr, \ @@ -730,7 +733,10 @@ self.tkconsole.endexecuting() finally: if not use_subprocess: - self.tkconsole.endexecuting() + try: + self.tkconsole.endexecuting() + except AttributeError: # shell may have closed + pass def write(self, s): "Override base class method" @@ -794,7 +800,7 @@ if use_subprocess: ms = self.menu_specs if ms[2][0] != "shell": - ms.insert(2, ("shell", "_Shell")) + ms.insert(2, ("shell", "She_ll")) self.interp = ModifiedInterpreter(self) if flist is None: root = Tk() @@ -804,9 +810,6 @@ # OutputWindow.__init__(self, flist, None, None) # - import __builtin__ - __builtin__.quit = __builtin__.exit = "To exit, type Ctrl-D." - # ## self.config(usetabs=1, indentwidth=8, context_use_ps1=1) self.usetabs = True # indentwidth must be 8 when using tabs. See note in EditorWindow: @@ -1138,21 +1141,27 @@ return "break" def recall(self, s, event): + # remove leading and trailing empty or whitespace lines + s = re.sub(r'^\s*\n', '' , s) + s = re.sub(r'\n\s*$', '', s) + lines = s.split('\n') self.text.undo_block_start() try: self.text.tag_remove("sel", "1.0", "end") self.text.mark_set("insert", "end-1c") - s = s.strip() - lines = s.split('\n') - prefix = self.text.get("insert linestart","insert").rstrip() - if prefix and prefix[-1]==':': + prefix = self.text.get("insert linestart", "insert") + if prefix.rstrip().endswith(':'): self.newline_and_indent_event(event) - self.text.insert("insert",lines[0].strip()) + prefix = self.text.get("insert linestart", "insert") + self.text.insert("insert", lines[0].strip()) if len(lines) > 1: - self.newline_and_indent_event(event) + orig_base_indent = re.search(r'^([ \t]*)', lines[0]).group(0) + new_base_indent = re.search(r'^([ \t]*)', prefix).group(0) for line in lines[1:]: - self.text.insert("insert", line.strip()) - self.newline_and_indent_event(event) + if line.startswith(orig_base_indent): + # replace orig base indentation with new indentation + line = new_base_indent + line[len(orig_base_indent):] + self.text.insert('insert', '\n'+line.rstrip()) finally: self.text.see("insert") self.text.undo_block_stop() Modified: python/branches/p3yk/Lib/idlelib/idlever.py ============================================================================== --- python/branches/p3yk/Lib/idlelib/idlever.py (original) +++ python/branches/p3yk/Lib/idlelib/idlever.py Mon Aug 21 21:07:27 2006 @@ -1 +1 @@ -IDLE_VERSION = "1.2b3" +IDLE_VERSION = "2.6a0" Modified: python/branches/p3yk/Lib/logging/__init__.py ============================================================================== --- python/branches/p3yk/Lib/logging/__init__.py (original) +++ python/branches/p3yk/Lib/logging/__init__.py Mon Aug 21 21:07:27 2006 @@ -1337,14 +1337,14 @@ """ root.manager.disable = level -def shutdown(): +def shutdown(handlerList=_handlerList): """ Perform any cleanup actions in the logging system (e.g. flushing buffers). Should be called at application exit. """ - for h in _handlerList[:]: # was _handlers.keys(): + for h in handlerList[:]: #errors might occur, for example, if files are locked #we just ignore them if raiseExceptions is not set try: Modified: python/branches/p3yk/Lib/logging/config.py ============================================================================== --- python/branches/p3yk/Lib/logging/config.py (original) +++ python/branches/p3yk/Lib/logging/config.py Mon Aug 21 21:07:27 2006 @@ -79,7 +79,7 @@ logging._acquireLock() try: logging._handlers.clear() - logging._handlerList = [] + del logging._handlerList[:] # Handlers add themselves to logging._handlers handlers = _install_handlers(cp, formatters) _install_loggers(cp, handlers) Modified: python/branches/p3yk/Lib/site.py ============================================================================== --- python/branches/p3yk/Lib/site.py (original) +++ python/branches/p3yk/Lib/site.py Mon Aug 21 21:07:27 2006 @@ -27,7 +27,7 @@ '#' are skipped. Lines starting with 'import' are executed. For example, suppose sys.prefix and sys.exec_prefix are set to -/usr/local and there is a directory /usr/local/lib/python1.5/site-packages +/usr/local and there is a directory /usr/local/lib/python2.5/site-packages with three subdirectories, foo, bar and spam, and two path configuration files, foo.pth and bar.pth. Assume foo.pth contains the following: @@ -44,8 +44,8 @@ Then the following directories are added to sys.path, in this order: - /usr/local/lib/python1.5/site-packages/bar - /usr/local/lib/python1.5/site-packages/foo + /usr/local/lib/python2.5/site-packages/bar + /usr/local/lib/python2.5/site-packages/foo Note that bletch is omitted because it doesn't exist; bar precedes foo because bar.pth comes alphabetically before foo.pth; and spam is @@ -242,6 +242,12 @@ def __repr__(self): return 'Use %s() or %s to exit' % (self.name, eof) def __call__(self, code=None): + # Shells like IDLE catch the SystemExit, but listen when their + # stdin wrapper is closed. + try: + sys.stdin.close() + except: + pass raise SystemExit(code) __builtin__.quit = Quitter('quit') __builtin__.exit = Quitter('exit') Modified: python/branches/p3yk/Lib/string.py ============================================================================== --- python/branches/p3yk/Lib/string.py (original) +++ python/branches/p3yk/Lib/string.py Mon Aug 21 21:07:27 2006 @@ -35,7 +35,6 @@ # Case conversion helpers # Use str to convert Unicode literal in case of -U -# Note that Cookie.py bogusly uses _idmap :( l = map(chr, xrange(256)) _idmap = str('').join(l) del l Modified: python/branches/p3yk/Lib/tabnanny.py ============================================================================== --- python/branches/p3yk/Lib/tabnanny.py (original) +++ python/branches/p3yk/Lib/tabnanny.py Mon Aug 21 21:07:27 2006 @@ -109,6 +109,10 @@ errprint("%r: Token Error: %s" % (file, msg)) return + except IndentationError, msg: + errprint("%r: Indentation Error: %s" % (file, msg)) + return + except NannyNag, nag: badline = nag.get_lineno() line = nag.get_line() Modified: python/branches/p3yk/Lib/tarfile.py ============================================================================== --- python/branches/p3yk/Lib/tarfile.py (original) +++ python/branches/p3yk/Lib/tarfile.py Mon Aug 21 21:07:27 2006 @@ -411,9 +411,6 @@ self.buf += self.cmp.flush() if self.mode == "w" and self.buf: - blocks, remainder = divmod(len(self.buf), self.bufsize) - if remainder > 0: - self.buf += NUL * (self.bufsize - remainder) self.fileobj.write(self.buf) self.buf = "" if self.comptype == "gz": Modified: python/branches/p3yk/Lib/test/exception_hierarchy.txt ============================================================================== --- python/branches/p3yk/Lib/test/exception_hierarchy.txt (original) +++ python/branches/p3yk/Lib/test/exception_hierarchy.txt Mon Aug 21 21:07:27 2006 @@ -45,3 +45,4 @@ +-- UserWarning +-- FutureWarning +-- ImportWarning + +-- UnicodeWarning Modified: python/branches/p3yk/Lib/test/inspect_fodder2.py ============================================================================== --- python/branches/p3yk/Lib/test/inspect_fodder2.py (original) +++ python/branches/p3yk/Lib/test/inspect_fodder2.py Mon Aug 21 21:07:27 2006 @@ -88,3 +88,12 @@ def func88(): # comment return 90 + +# line 92 +def f(): + class X: + def g(): + "doc" + return 42 + return X +method_in_dynamic_class = f().g.im_func Modified: python/branches/p3yk/Lib/test/output/test_cgi ============================================================================== --- python/branches/p3yk/Lib/test/output/test_cgi (original) +++ python/branches/p3yk/Lib/test/output/test_cgi Mon Aug 21 21:07:27 2006 @@ -38,3 +38,5 @@ Testing log Testing initlog 1 Testing log 2 +Test FieldStorage methods that use readline +Test basic FieldStorage multipart parsing Modified: python/branches/p3yk/Lib/test/test_bytes.py ============================================================================== --- python/branches/p3yk/Lib/test/test_bytes.py (original) +++ python/branches/p3yk/Lib/test/test_bytes.py Mon Aug 21 21:07:27 2006 @@ -60,13 +60,13 @@ self.assertRaises(ValueError, bytes, [-1]) self.assertRaises(ValueError, bytes, [-sys.maxint]) self.assertRaises(ValueError, bytes, [-sys.maxint-1]) - self.assertRaises(ValueError, bytes, [-sys.maxint-2]) - self.assertRaises(ValueError, bytes, [-10**100]) + self.assertRaises(TypeError, bytes, [-sys.maxint-2]) + self.assertRaises(TypeError, bytes, [-10**100]) self.assertRaises(ValueError, bytes, [256]) self.assertRaises(ValueError, bytes, [257]) self.assertRaises(ValueError, bytes, [sys.maxint]) - self.assertRaises(ValueError, bytes, [sys.maxint+1]) - self.assertRaises(ValueError, bytes, [10**100]) + self.assertRaises(TypeError, bytes, [sys.maxint+1]) + self.assertRaises(TypeError, bytes, [10**100]) def test_repr(self): self.assertEqual(repr(bytes()), "bytes()") Modified: python/branches/p3yk/Lib/test/test_bz2.py ============================================================================== --- python/branches/p3yk/Lib/test/test_bz2.py (original) +++ python/branches/p3yk/Lib/test/test_bz2.py Mon Aug 21 21:07:27 2006 @@ -158,10 +158,21 @@ sio = StringIO(self.TEXT) bz2f.writelines(sio.readlines()) bz2f.close() + # patch #1535500 + self.assertRaises(ValueError, bz2f.writelines, ["a"]) f = open(self.filename, 'rb') self.assertEqual(self.decompress(f.read()), self.TEXT) f.close() + def testWriteMethodsOnReadOnlyFile(self): + bz2f = BZ2File(self.filename, "w") + bz2f.write("abc") + bz2f.close() + + bz2f = BZ2File(self.filename, "r") + self.assertRaises(IOError, bz2f.write, "a") + self.assertRaises(IOError, bz2f.writelines, ["a"]) + def testSeekForward(self): # "Test BZ2File.seek(150, 0)" self.createTempFile() Modified: python/branches/p3yk/Lib/test/test_cgi.py ============================================================================== --- python/branches/p3yk/Lib/test/test_cgi.py (original) +++ python/branches/p3yk/Lib/test/test_cgi.py Mon Aug 21 21:07:27 2006 @@ -2,6 +2,8 @@ import cgi import os import sys +import tempfile +from StringIO import StringIO class HackedSysModule: # The regression test will have real values in sys.argv, which @@ -203,4 +205,71 @@ cgi.initlog("%s", "Testing log 3") cgi.log("Testing log 4") + print "Test FieldStorage methods that use readline" + # FieldStorage uses readline, which has the capacity to read all + # contents of the input file into memory; we use readline's size argument + # to prevent that for files that do not contain any newlines in + # non-GET/HEAD requests + class TestReadlineFile: + def __init__(self, file): + self.file = file + self.numcalls = 0 + + def readline(self, size=None): + self.numcalls += 1 + if size: + return self.file.readline(size) + else: + return self.file.readline() + + def __getattr__(self, name): + file = self.__dict__['file'] + a = getattr(file, name) + if not isinstance(a, int): + setattr(self, name, a) + return a + + f = TestReadlineFile(tempfile.TemporaryFile()) + f.write('x' * 256 * 1024) + f.seek(0) + env = {'REQUEST_METHOD':'PUT'} + fs = cgi.FieldStorage(fp=f, environ=env) + # if we're not chunking properly, readline is only called twice + # (by read_binary); if we are chunking properly, it will be called 5 times + # as long as the chunksize is 1 << 16. + verify(f.numcalls > 2) + + print "Test basic FieldStorage multipart parsing" + env = {'REQUEST_METHOD':'POST', 'CONTENT_TYPE':'multipart/form-data; boundary=---------------------------721837373350705526688164684', 'CONTENT_LENGTH':'558'} + postdata = """-----------------------------721837373350705526688164684 +Content-Disposition: form-data; name="id" + +1234 +-----------------------------721837373350705526688164684 +Content-Disposition: form-data; name="title" + + +-----------------------------721837373350705526688164684 +Content-Disposition: form-data; name="file"; filename="test.txt" +Content-Type: text/plain + +Testing 123. + +-----------------------------721837373350705526688164684 +Content-Disposition: form-data; name="submit" + + Add\x20 +-----------------------------721837373350705526688164684-- +""" + fs = cgi.FieldStorage(fp=StringIO(postdata), environ=env) + verify(len(fs.list) == 4) + expect = [{'name':'id', 'filename':None, 'value':'1234'}, + {'name':'title', 'filename':None, 'value':''}, + {'name':'file', 'filename':'test.txt','value':'Testing 123.\n'}, + {'name':'submit', 'filename':None, 'value':' Add '}] + for x in range(len(fs.list)): + for k, exp in expect[x].items(): + got = getattr(fs.list[x], k) + verify(got == exp) + main() Modified: python/branches/p3yk/Lib/test/test_compiler.py ============================================================================== --- python/branches/p3yk/Lib/test/test_compiler.py (original) +++ python/branches/p3yk/Lib/test/test_compiler.py Mon Aug 21 21:07:27 2006 @@ -116,6 +116,13 @@ exec c in dct self.assertEquals(dct.get('result'), 3) + def testGenExp(self): + c = compiler.compile('list((i,j) for i in range(3) if i < 3' + ' for j in range(4) if j > 2)', + '<string>', + 'eval') + self.assertEquals(eval(c), [(0, 3), (1, 3), (2, 3)]) + NOLINENO = (compiler.ast.Module, compiler.ast.Stmt, compiler.ast.Discard) Modified: python/branches/p3yk/Lib/test/test_format.py ============================================================================== --- python/branches/p3yk/Lib/test/test_format.py (original) +++ python/branches/p3yk/Lib/test/test_format.py Mon Aug 21 21:07:27 2006 @@ -230,6 +230,14 @@ test_exc(u'no format', u'1', TypeError, "not all arguments converted during string formatting") +class Foobar(long): + def __oct__(self): + # Returning a non-string should not blow up. + return self + 1 + +test_exc('%o', Foobar(), TypeError, + "expected string or Unicode object, long found") + if sys.maxint == 2**31-1: # crashes 2.2.1 and earlier: try: Modified: python/branches/p3yk/Lib/test/test_index.py ============================================================================== --- python/branches/p3yk/Lib/test/test_index.py (original) +++ python/branches/p3yk/Lib/test/test_index.py Mon Aug 21 21:07:27 2006 @@ -1,6 +1,7 @@ import unittest from test import test_support import operator +from sys import maxint class oldstyle: def __index__(self): @@ -10,68 +11,116 @@ def __index__(self): return self.ind +class TrapInt(int): + def __index__(self): + return self + +class TrapLong(long): + def __index__(self): + return self + class BaseTestCase(unittest.TestCase): def setUp(self): self.o = oldstyle() self.n = newstyle() - self.o2 = oldstyle() - self.n2 = newstyle() def test_basic(self): self.o.ind = -2 self.n.ind = 2 - assert(self.seq[self.n] == self.seq[2]) - assert(self.seq[self.o] == self.seq[-2]) - assert(operator.index(self.o) == -2) - assert(operator.index(self.n) == 2) + self.assertEqual(operator.index(self.o), -2) + self.assertEqual(operator.index(self.n), 2) + + def test_slice(self): + self.o.ind = 1 + self.n.ind = 2 + slc = slice(self.o, self.o, self.o) + check_slc = slice(1, 1, 1) + self.assertEqual(slc.indices(self.o), check_slc.indices(1)) + slc = slice(self.n, self.n, self.n) + check_slc = slice(2, 2, 2) + self.assertEqual(slc.indices(self.n), check_slc.indices(2)) + + def test_wrappers(self): + self.o.ind = 4 + self.n.ind = 5 + self.assertEqual(6 .__index__(), 6) + self.assertEqual(-7L.__index__(), -7) + self.assertEqual(self.o.__index__(), 4) + self.assertEqual(self.n.__index__(), 5) + + def test_subclasses(self): + r = range(10) + self.assertEqual(r[TrapInt(5):TrapInt(10)], r[5:10]) + self.assertEqual(r[TrapLong(5):TrapLong(10)], r[5:10]) + self.assertEqual(slice(TrapInt()).indices(0), (0,0,1)) + self.assertEqual(slice(TrapLong(0)).indices(0), (0,0,1)) def test_error(self): self.o.ind = 'dumb' self.n.ind = 'bad' - myfunc = lambda x, obj: obj.seq[x] self.failUnlessRaises(TypeError, operator.index, self.o) self.failUnlessRaises(TypeError, operator.index, self.n) - self.failUnlessRaises(TypeError, myfunc, self.o, self) - self.failUnlessRaises(TypeError, myfunc, self.n, self) + self.failUnlessRaises(TypeError, slice(self.o).indices, 0) + self.failUnlessRaises(TypeError, slice(self.n).indices, 0) + + +class SeqTestCase(unittest.TestCase): + # This test case isn't run directly. It just defines common tests + # to the different sequence types below + def setUp(self): + self.o = oldstyle() + self.n = newstyle() + self.o2 = oldstyle() + self.n2 = newstyle() + + def test_index(self): + self.o.ind = -2 + self.n.ind = 2 + self.assertEqual(self.seq[self.n], self.seq[2]) + self.assertEqual(self.seq[self.o], self.seq[-2]) def test_slice(self): self.o.ind = 1 self.o2.ind = 3 self.n.ind = 2 self.n2.ind = 4 - assert(self.seq[self.o:self.o2] == self.seq[1:3]) - assert(self.seq[self.n:self.n2] == self.seq[2:4]) + self.assertEqual(self.seq[self.o:self.o2], self.seq[1:3]) + self.assertEqual(self.seq[self.n:self.n2], self.seq[2:4]) def test_repeat(self): self.o.ind = 3 self.n.ind = 2 - assert(self.seq * self.o == self.seq * 3) - assert(self.seq * self.n == self.seq * 2) - assert(self.o * self.seq == self.seq * 3) - assert(self.n * self.seq == self.seq * 2) + self.assertEqual(self.seq * self.o, self.seq * 3) + self.assertEqual(self.seq * self.n, self.seq * 2) + self.assertEqual(self.o * self.seq, self.seq * 3) + self.assertEqual(self.n * self.seq, self.seq * 2) def test_wrappers(self): - n = self.n - n.ind = 5 - assert n.__index__() == 5 - assert 6 .__index__() == 6 - assert -7L.__index__() == -7 - assert self.seq.__getitem__(n) == self.seq[5] - assert self.seq.__mul__(n) == self.seq * 5 - assert self.seq.__rmul__(n) == self.seq * 5 - - def test_infinite_recusion(self): - class Trap1(int): - def __index__(self): - return self - class Trap2(long): - def __index__(self): - return self - self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap1()) - self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap2()) + self.o.ind = 4 + self.n.ind = 5 + self.assertEqual(self.seq.__getitem__(self.o), self.seq[4]) + self.assertEqual(self.seq.__mul__(self.o), self.seq * 4) + self.assertEqual(self.seq.__rmul__(self.o), self.seq * 4) + self.assertEqual(self.seq.__getitem__(self.n), self.seq[5]) + self.assertEqual(self.seq.__mul__(self.n), self.seq * 5) + self.assertEqual(self.seq.__rmul__(self.n), self.seq * 5) + + def test_subclasses(self): + self.assertEqual(self.seq[TrapInt()], self.seq[0]) + self.assertEqual(self.seq[TrapLong()], self.seq[0]) + def test_error(self): + self.o.ind = 'dumb' + self.n.ind = 'bad' + indexobj = lambda x, obj: obj.seq[x] + self.failUnlessRaises(TypeError, indexobj, self.o, self) + self.failUnlessRaises(TypeError, indexobj, self.n, self) + sliceobj = lambda x, obj: obj.seq[x:] + self.failUnlessRaises(TypeError, sliceobj, self.o, self) + self.failUnlessRaises(TypeError, sliceobj, self.n, self) -class ListTestCase(BaseTestCase): + +class ListTestCase(SeqTestCase): seq = [0,10,20,30,40,50] def test_setdelitem(self): @@ -82,36 +131,36 @@ del lst[self.n] lst[self.o] = 'X' lst[self.n] = 'Y' - assert lst == list('abYdefghXj') + self.assertEqual(lst, list('abYdefghXj')) lst = [5, 6, 7, 8, 9, 10, 11] lst.__setitem__(self.n, "here") - assert lst == [5, 6, "here", 8, 9, 10, 11] + self.assertEqual(lst, [5, 6, "here", 8, 9, 10, 11]) lst.__delitem__(self.n) - assert lst == [5, 6, 8, 9, 10, 11] + self.assertEqual(lst, [5, 6, 8, 9, 10, 11]) def test_inplace_repeat(self): self.o.ind = 2 self.n.ind = 3 lst = [6, 4] lst *= self.o - assert lst == [6, 4, 6, 4] + self.assertEqual(lst, [6, 4, 6, 4]) lst *= self.n - assert lst == [6, 4, 6, 4] * 3 + self.assertEqual(lst, [6, 4, 6, 4] * 3) lst = [5, 6, 7, 8, 9, 11] l2 = lst.__imul__(self.n) - assert l2 is lst - assert lst == [5, 6, 7, 8, 9, 11] * 3 + self.assert_(l2 is lst) + self.assertEqual(lst, [5, 6, 7, 8, 9, 11] * 3) -class TupleTestCase(BaseTestCase): +class TupleTestCase(SeqTestCase): seq = (0,10,20,30,40,50) -class StringTestCase(BaseTestCase): +class StringTestCase(SeqTestCase): seq = "this is a test" -class UnicodeTestCase(BaseTestCase): +class UnicodeTestCase(SeqTestCase): seq = u"this is a test" @@ -120,17 +169,54 @@ def test_xrange(self): n = newstyle() n.ind = 5 - assert xrange(1, 20)[n] == 6 - assert xrange(1, 20).__getitem__(n) == 6 + self.assertEqual(xrange(1, 20)[n], 6) + self.assertEqual(xrange(1, 20).__getitem__(n), 6) + +class OverflowTestCase(unittest.TestCase): + + def setUp(self): + self.pos = 2**100 + self.neg = -self.pos + + def test_large_longs(self): + self.assertEqual(self.pos.__index__(), self.pos) + self.assertEqual(self.neg.__index__(), self.neg) + + def _getitem_helper(self, base): + class GetItem(base): + def __len__(self): + return maxint + def __getitem__(self, key): + return key + def __getslice__(self, i, j): + return i, j + x = GetItem() + self.assertEqual(x[self.pos], self.pos) + self.assertEqual(x[self.neg], self.neg) + self.assertEqual(x[self.neg:self.pos], (-1, maxint)) + self.assertEqual(x[self.neg:self.pos:1].indices(maxint), (0, maxint, 1)) + + def test_getitem(self): + self._getitem_helper(object) + + def test_getitem_classic(self): + class Empty: pass + self._getitem_helper(Empty) + + def test_sequence_repeat(self): + self.failUnlessRaises(OverflowError, lambda: "a" * self.pos) + self.failUnlessRaises(OverflowError, lambda: "a" * self.neg) def test_main(): test_support.run_unittest( + BaseTestCase, ListTestCase, TupleTestCase, StringTestCase, UnicodeTestCase, XRangeTestCase, + OverflowTestCase, ) if __name__ == "__main__": Modified: python/branches/p3yk/Lib/test/test_inspect.py ============================================================================== --- python/branches/p3yk/Lib/test/test_inspect.py (original) +++ python/branches/p3yk/Lib/test/test_inspect.py Mon Aug 21 21:07:27 2006 @@ -276,6 +276,9 @@ def test_with_comment_instead_of_docstring(self): self.assertSourceEqual(mod2.func88, 88, 90) + def test_method_in_dynamic_class(self): + self.assertSourceEqual(mod2.method_in_dynamic_class, 95, 97) + # Helper for testing classify_class_attrs. def attrs_wo_objs(cls): return [t[:3] for t in inspect.classify_class_attrs(cls)] Modified: python/branches/p3yk/Lib/test/test_logging.py ============================================================================== --- python/branches/p3yk/Lib/test/test_logging.py (original) +++ python/branches/p3yk/Lib/test/test_logging.py Mon Aug 21 21:07:27 2006 @@ -493,7 +493,7 @@ try: logging._handlers.clear() logging._handlers.update(saved_handlers) - logging._handlerList = saved_handler_list + logging._handlerList[:] = saved_handler_list loggerDict = logging.getLogger().manager.loggerDict loggerDict.clear() loggerDict.update(saved_loggers) @@ -560,7 +560,7 @@ try: logging._handlers.clear() logging._handlers.update(saved_handlers) - logging._handlerList = saved_handler_list + logging._handlerList[:] = saved_handler_list loggerDict = logging.getLogger().manager.loggerDict loggerDict.clear() loggerDict.update(saved_loggers) Modified: python/branches/p3yk/Lib/test/test_mailbox.py ============================================================================== --- python/branches/p3yk/Lib/test/test_mailbox.py (original) +++ python/branches/p3yk/Lib/test/test_mailbox.py Mon Aug 21 21:07:27 2006 @@ -1,4 +1,5 @@ import os +import sys import time import stat import socket @@ -454,7 +455,7 @@ def setUp(self): TestMailbox.setUp(self) - if os.name in ('nt', 'os2'): + if os.name in ('nt', 'os2') or sys.platform == 'cygwin': self._box.colon = '!' def test_add_MM(self): @@ -729,11 +730,13 @@ # In the parent, sleep a bit to give the child time to acquire # the lock. time.sleep(0.5) - self.assertRaises(mailbox.ExternalClashError, - self._box.lock) + try: + self.assertRaises(mailbox.ExternalClashError, + self._box.lock) + finally: + # Wait for child to exit. Locking should now succeed. + exited_pid, status = os.waitpid(pid, 0) - # Wait for child to exit. Locking should now succeed. - exited_pid, status = os.waitpid(pid, 0) self._box.lock() self._box.unlock() Modified: python/branches/p3yk/Lib/test/test_signal.py ============================================================================== --- python/branches/p3yk/Lib/test/test_signal.py (original) +++ python/branches/p3yk/Lib/test/test_signal.py Mon Aug 21 21:07:27 2006 @@ -6,11 +6,16 @@ if sys.platform[:3] in ('win', 'os2') or sys.platform=='riscos': raise TestSkipped, "Can't test signal on %s" % sys.platform +MAX_DURATION = 20 # Entire test should last at most 20 sec. + if verbose: x = '-x' else: x = '+x' + pid = os.getpid() +if verbose: + print "test runner's pid is", pid # Shell script that will send us asynchronous signals script = """ @@ -31,7 +36,7 @@ global a_called a_called = True if verbose: - print "handlerA", args + print "handlerA invoked", args class HandlerBCalled(Exception): pass @@ -40,87 +45,110 @@ global b_called b_called = True if verbose: - print "handlerB", args + print "handlerB invoked", args raise HandlerBCalled, args -MAX_DURATION = 20 -signal.alarm(MAX_DURATION) # Entire test should last at most 20 sec. +# Set up a child to send signals to us (the parent) after waiting long +# enough to receive the alarm. It seems we miss the alarm for some +# reason. This will hopefully stop the hangs on Tru64/Alpha. +# Alas, it doesn't. Tru64 appears to miss all the signals at times, or +# seemingly random subsets of them, and nothing done in force_test_exit +# so far has actually helped. +def force_test_exit(): + # Sigh, both imports seem necessary to avoid errors. + import os + fork_pid = os.fork() + if fork_pid: + # In parent. + return fork_pid + + # In child. + import os, time + try: + # Wait 5 seconds longer than the expected alarm to give enough + # time for the normal sequence of events to occur. This is + # just a stop-gap to try to prevent the test from hanging. + time.sleep(MAX_DURATION + 5) + print >> sys.__stdout__, ' child should not have to kill parent' + for signame in "SIGHUP", "SIGUSR1", "SIGUSR2", "SIGALRM": + os.kill(pid, getattr(signal, signame)) + print >> sys.__stdout__, " child sent", signame, "to", pid + time.sleep(1) + finally: + os._exit(0) + +# Install handlers. hup = signal.signal(signal.SIGHUP, handlerA) usr1 = signal.signal(signal.SIGUSR1, handlerB) usr2 = signal.signal(signal.SIGUSR2, signal.SIG_IGN) alrm = signal.signal(signal.SIGALRM, signal.default_int_handler) -vereq(signal.getsignal(signal.SIGHUP), handlerA) -vereq(signal.getsignal(signal.SIGUSR1), handlerB) -vereq(signal.getsignal(signal.SIGUSR2), signal.SIG_IGN) - try: - signal.signal(4242, handlerB) - raise TestFailed, 'expected ValueError for invalid signal # to signal()' -except ValueError: - pass -try: - signal.getsignal(4242) - raise TestFailed, 'expected ValueError for invalid signal # to getsignal()' -except ValueError: - pass + signal.alarm(MAX_DURATION) + vereq(signal.getsignal(signal.SIGHUP), handlerA) + vereq(signal.getsignal(signal.SIGUSR1), handlerB) + vereq(signal.getsignal(signal.SIGUSR2), signal.SIG_IGN) + vereq(signal.getsignal(signal.SIGALRM), signal.default_int_handler) -try: - signal.signal(signal.SIGUSR1, None) - raise TestFailed, 'expected TypeError for non-callable' -except TypeError: - pass + # Try to ensure this test exits even if there is some problem with alarm. + # Tru64/Alpha often hangs and is ultimately killed by the buildbot. + fork_pid = force_test_exit() -# Set up a child to send an alarm signal to us (the parent) after waiting -# long enough to receive the alarm. It seems we miss the alarm for some -# reason. This will hopefully stop the hangs on Tru64/Alpha. -def force_test_exit(): - # Sigh, both imports seem necessary to avoid errors. - import os - fork_pid = os.fork() - if fork_pid == 0: - # In child - import os, time - try: - # Wait 5 seconds longer than the expected alarm to give enough - # time for the normal sequence of events to occur. This is - # just a stop-gap to prevent the test from hanging. - time.sleep(MAX_DURATION + 5) - print >> sys.__stdout__, ' child should not have to kill parent' - for i in range(3): - os.kill(pid, signal.SIGALARM) - finally: - os._exit(0) - # In parent (or error) - return fork_pid + try: + signal.getsignal(4242) + raise TestFailed('expected ValueError for invalid signal # to ' + 'getsignal()') + except ValueError: + pass -try: - os.system(script) + try: + signal.signal(4242, handlerB) + raise TestFailed('expected ValueError for invalid signal # to ' + 'signal()') + except ValueError: + pass - # Try to ensure this test exits even if there is some problem with alarm. - # Tru64/Alpha sometimes hangs and is ultimately killed by the buildbot. - fork_pid = force_test_exit() - print "starting pause() loop..." + try: + signal.signal(signal.SIGUSR1, None) + raise TestFailed('expected TypeError for non-callable') + except TypeError: + pass + # Launch an external script to send us signals. + # We expect the external script to: + # send HUP, which invokes handlerA to set a_called + # send USR1, which invokes handlerB to set b_called and raise + # HandlerBCalled + # send USR2, which is ignored + # + # Then we expect the alarm to go off, and its handler raises + # KeyboardInterrupt, finally getting us out of the loop. + os.system(script) try: + print "starting pause() loop..." while 1: - if verbose: - print "call pause()..." try: + if verbose: + print "call pause()..." signal.pause() if verbose: print "pause() returned" except HandlerBCalled: if verbose: print "HandlerBCalled exception caught" - else: - pass except KeyboardInterrupt: if verbose: - print "KeyboardInterrupt (assume the alarm() went off)" + print "KeyboardInterrupt (the alarm() went off)" + if not a_called: + print 'HandlerA not called' + + if not b_called: + print 'HandlerB not called' + +finally: # Forcibly kill the child we created to ping us if there was a test error. try: # Make sure we don't kill ourself if there was a fork error. @@ -128,16 +156,11 @@ os.kill(fork_pid, signal.SIGKILL) except: # If the child killed us, it has probably exited. Killing a - # non-existant process will raise an error which we don't care about. + # non-existent process will raise an error which we don't care about. pass - if not a_called: - print 'HandlerA not called' - - if not b_called: - print 'HandlerB not called' - -finally: + # Restore handlers. + signal.alarm(0) # cancel alarm in case we died early signal.signal(signal.SIGHUP, hup) signal.signal(signal.SIGUSR1, usr1) signal.signal(signal.SIGUSR2, usr2) Modified: python/branches/p3yk/Lib/test/test_socketserver.py ============================================================================== --- python/branches/p3yk/Lib/test/test_socketserver.py (original) +++ python/branches/p3yk/Lib/test/test_socketserver.py Mon Aug 21 21:07:27 2006 @@ -181,10 +181,19 @@ if hasattr(os, 'fork') and os.name not in ('os2',): dgramservers.append(ForkingUnixDatagramServer) +def sloppy_cleanup(): + # See http://python.org/sf/1540386 + # We need to reap children here otherwise a child from one server + # can be left running for the next server and cause a test failure. + time.sleep(DELAY) + reap_children() + def testall(): testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream) + sloppy_cleanup() testloop(socket.AF_INET, udpservers, MyDatagramHandler, testdgram) if hasattr(socket, 'AF_UNIX'): + sloppy_cleanup() testloop(socket.AF_UNIX, streamservers, MyStreamHandler, teststream) # Alas, on Linux (at least) recvfrom() doesn't return a meaningful # client address so this cannot work: Modified: python/branches/p3yk/Lib/test/test_tarfile.py ============================================================================== --- python/branches/p3yk/Lib/test/test_tarfile.py (original) +++ python/branches/p3yk/Lib/test/test_tarfile.py Mon Aug 21 21:07:27 2006 @@ -324,6 +324,27 @@ class WriteStreamTest(WriteTest): sep = '|' + def test_padding(self): + self.dst.close() + + if self.comp == "gz": + f = gzip.GzipFile(self.dstname) + s = f.read() + f.close() + elif self.comp == "bz2": + f = bz2.BZ2Decompressor() + s = file(self.dstname).read() + s = f.decompress(s) + self.assertEqual(len(f.unused_data), 0, "trailing data") + else: + f = file(self.dstname) + s = f.read() + f.close() + + self.assertEqual(s.count("\0"), tarfile.RECORDSIZE, + "incorrect zero padding") + + class WriteGNULongTest(unittest.TestCase): """This testcase checks for correct creation of GNU Longname and Longlink extensions. Modified: python/branches/p3yk/Lib/test/test_threading.py ============================================================================== --- python/branches/p3yk/Lib/test/test_threading.py (original) +++ python/branches/p3yk/Lib/test/test_threading.py Mon Aug 21 21:07:27 2006 @@ -131,6 +131,76 @@ threading._DummyThread)) del threading._active[tid] + # PyThreadState_SetAsyncExc() is a CPython-only gimmick, not (currently) + # exposed at the Python level. This test relies on ctypes to get at it. + def test_PyThreadState_SetAsyncExc(self): + try: + import ctypes + except ImportError: + if verbose: + print "test_PyThreadState_SetAsyncExc can't import ctypes" + return # can't do anything + + set_async_exc = ctypes.pythonapi.PyThreadState_SetAsyncExc + + class AsyncExc(Exception): + pass + + exception = ctypes.py_object(AsyncExc) + + # `worker_started` is set by the thread when it's inside a try/except + # block waiting to catch the asynchronously set AsyncExc exception. + # `worker_saw_exception` is set by the thread upon catching that + # exception. + worker_started = threading.Event() + worker_saw_exception = threading.Event() + + class Worker(threading.Thread): + def run(self): + self.id = thread.get_ident() + self.finished = False + + try: + while True: + worker_started.set() + time.sleep(0.1) + except AsyncExc: + self.finished = True + worker_saw_exception.set() + + t = Worker() + t.setDaemon(True) # so if this fails, we don't hang Python at shutdown + t.start() + if verbose: + print " started worker thread" + + # Try a thread id that doesn't make sense. + if verbose: + print " trying nonsensical thread id" + result = set_async_exc(ctypes.c_long(-1), exception) + self.assertEqual(result, 0) # no thread states modified + + # Now raise an exception in the worker thread. + if verbose: + print " waiting for worker thread to get started" + worker_started.wait() + if verbose: + print " verifying worker hasn't exited" + self.assert_(not t.finished) + if verbose: + print " attempting to raise asynch exception in worker" + result = set_async_exc(ctypes.c_long(t.id), exception) + self.assertEqual(result, 1) # one thread state modified + if verbose: + print " waiting for worker to say it caught the exception" + worker_saw_exception.wait(timeout=10) + self.assert_(t.finished) + if verbose: + print " all OK -- joining worker" + if t.finished: + t.join() + # else the thread is still running, and we have no way to kill it + def test_main(): test.test_support.run_unittest(ThreadTests) Modified: python/branches/p3yk/Lib/test/test_threadsignals.py ============================================================================== --- python/branches/p3yk/Lib/test/test_threadsignals.py (original) +++ python/branches/p3yk/Lib/test/test_threadsignals.py Mon Aug 21 21:07:27 2006 @@ -49,7 +49,7 @@ # and might be out of order.) If we haven't seen # the signals yet, send yet another signal and # wait for it return. - if signal_blackboard[signal.SIGUSR2]['tripped'] == 0 \ + if signal_blackboard[signal.SIGUSR1]['tripped'] == 0 \ or signal_blackboard[signal.SIGUSR2]['tripped'] == 0: signal.alarm(1) signal.pause() Modified: python/branches/p3yk/Lib/test/test_urllib2.py ============================================================================== --- python/branches/p3yk/Lib/test/test_urllib2.py (original) +++ python/branches/p3yk/Lib/test/test_urllib2.py Mon Aug 21 21:07:27 2006 @@ -46,6 +46,69 @@ self.assertEquals(urllib2.parse_http_list(string), list) +def test_request_headers_dict(): + """ + The Request.headers dictionary is not a documented interface. It should + stay that way, because the complete set of headers are only accessible + through the .get_header(), .has_header(), .header_items() interface. + However, .headers pre-dates those methods, and so real code will be using + the dictionary. + + The introduction in 2.4 of those methods was a mistake for the same reason: + code that previously saw all (urllib2 user)-provided headers in .headers + now sees only a subset (and the function interface is ugly and incomplete). + A better change would have been to replace .headers dict with a dict + subclass (or UserDict.DictMixin instance?) that preserved the .headers + interface and also provided access to the "unredirected" headers. It's + probably too late to fix that, though. + + + Check .capitalize() case normalization: + + >>> url = "http://example.com" + >>> Request(url, headers={"Spam-eggs": "blah"}).headers["Spam-eggs"] + 'blah' + >>> Request(url, headers={"spam-EggS": "blah"}).headers["Spam-eggs"] + 'blah' + + Currently, Request(url, "Spam-eggs").headers["Spam-Eggs"] raises KeyError, + but that could be changed in future. + + """ + +def test_request_headers_methods(): + """ + Note the case normalization of header names here, to .capitalize()-case. + This should be preserved for backwards-compatibility. (In the HTTP case, + normalization to .title()-case is done by urllib2 before sending headers to + httplib). + + >>> url = "http://example.com" + >>> r = Request(url, headers={"Spam-eggs": "blah"}) + >>> r.has_header("Spam-eggs") + True + >>> r.header_items() + [('Spam-eggs', 'blah')] + >>> r.add_header("Foo-Bar", "baz") + >>> items = r.header_items() + >>> items.sort() + >>> items + [('Foo-bar', 'baz'), ('Spam-eggs', 'blah')] + + Note that e.g. r.has_header("spam-EggS") is currently False, and + r.get_header("spam-EggS") returns None, but that could be changed in + future. + + >>> r.has_header("Not-there") + False + >>> print r.get_header("Not-there") + None + >>> r.get_header("Not-there", "default") + 'default' + + """ + + def test_password_manager(self): """ >>> mgr = urllib2.HTTPPasswordMgr() @@ -676,11 +739,11 @@ r = MockResponse(200, "OK", {}, "") newreq = h.do_request_(req) if data is None: # GET - self.assert_("Content-Length" not in req.unredirected_hdrs) - self.assert_("Content-Type" not in req.unredirected_hdrs) + self.assert_("Content-length" not in req.unredirected_hdrs) + self.assert_("Content-type" not in req.unredirected_hdrs) else: # POST - self.assertEqual(req.unredirected_hdrs["Content-Length"], "0") - self.assertEqual(req.unredirected_hdrs["Content-Type"], + self.assertEqual(req.unredirected_hdrs["Content-length"], "0") + self.assertEqual(req.unredirected_hdrs["Content-type"], "application/x-www-form-urlencoded") # XXX the details of Host could be better tested self.assertEqual(req.unredirected_hdrs["Host"], "example.com") @@ -692,8 +755,8 @@ req.add_unredirected_header("Host", "baz") req.add_unredirected_header("Spam", "foo") newreq = h.do_request_(req) - self.assertEqual(req.unredirected_hdrs["Content-Length"], "foo") - self.assertEqual(req.unredirected_hdrs["Content-Type"], "bar") + self.assertEqual(req.unredirected_hdrs["Content-length"], "foo") + self.assertEqual(req.unredirected_hdrs["Content-type"], "bar") self.assertEqual(req.unredirected_hdrs["Host"], "baz") self.assertEqual(req.unredirected_hdrs["Spam"], "foo") @@ -847,7 +910,7 @@ 407, 'Proxy-Authenticate: Basic realm="%s"\r\n\r\n' % realm) opener.add_handler(auth_handler) opener.add_handler(http_handler) - self._test_basic_auth(opener, auth_handler, "Proxy-Authorization", + self._test_basic_auth(opener, auth_handler, "Proxy-authorization", realm, http_handler, password_manager, "http://acme.example.com:3128/protected", "proxy.example.com:3128", Modified: python/branches/p3yk/Lib/test/test_uuid.py ============================================================================== --- python/branches/p3yk/Lib/test/test_uuid.py (original) +++ python/branches/p3yk/Lib/test/test_uuid.py Mon Aug 21 21:07:27 2006 @@ -16,12 +16,13 @@ def test_UUID(self): equal = self.assertEqual ascending = [] - for (string, curly, hex, bytes, fields, integer, urn, + for (string, curly, hex, bytes, bytes_le, fields, integer, urn, time, clock_seq, variant, version) in [ ('00000000-0000-0000-0000-000000000000', '{00000000-0000-0000-0000-000000000000}', '00000000000000000000000000000000', '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', + '\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0', (0, 0, 0, 0, 0, 0), 0, 'urn:uuid:00000000-0000-0000-0000-000000000000', @@ -30,6 +31,7 @@ '{00010203-0405-0607-0809-0a0b0c0d0e0f}', '000102030405060708090a0b0c0d0e0f', '\0\x01\x02\x03\x04\x05\x06\x07\x08\t\n\x0b\x0c\x0d\x0e\x0f', + '\x03\x02\x01\0\x05\x04\x07\x06\x08\t\n\x0b\x0c\x0d\x0e\x0f', (0x00010203L, 0x0405, 0x0607, 8, 9, 0x0a0b0c0d0e0fL), 0x000102030405060708090a0b0c0d0e0fL, 'urn:uuid:00010203-0405-0607-0809-0a0b0c0d0e0f', @@ -38,6 +40,7 @@ '{02d9e6d5-9467-382e-8f9b-9300a64ac3cd}', '02d9e6d59467382e8f9b9300a64ac3cd', '\x02\xd9\xe6\xd5\x94\x67\x38\x2e\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', + '\xd5\xe6\xd9\x02\x67\x94\x2e\x38\x8f\x9b\x93\x00\xa6\x4a\xc3\xcd', (0x02d9e6d5L, 0x9467, 0x382e, 0x8f, 0x9b, 0x9300a64ac3cdL), 0x02d9e6d59467382e8f9b9300a64ac3cdL, 'urn:uuid:02d9e6d5-9467-382e-8f9b-9300a64ac3cd', @@ -46,6 +49,7 @@ '{12345678-1234-5678-1234-567812345678}', '12345678123456781234567812345678', '\x12\x34\x56\x78'*4, + '\x78\x56\x34\x12\x34\x12\x78\x56\x12\x34\x56\x78\x12\x34\x56\x78', (0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678), 0x12345678123456781234567812345678, 'urn:uuid:12345678-1234-5678-1234-567812345678', @@ -54,6 +58,7 @@ '{6ba7b810-9dad-11d1-80b4-00c04fd430c8}', '6ba7b8109dad11d180b400c04fd430c8', '\x6b\xa7\xb8\x10\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + '\x10\xb8\xa7\x6b\xad\x9d\xd1\x11\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', (0x6ba7b810L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), 0x6ba7b8109dad11d180b400c04fd430c8L, 'urn:uuid:6ba7b810-9dad-11d1-80b4-00c04fd430c8', @@ -62,6 +67,7 @@ '{6ba7b811-9dad-11d1-80b4-00c04fd430c8}', '6ba7b8119dad11d180b400c04fd430c8', '\x6b\xa7\xb8\x11\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + '\x11\xb8\xa7\x6b\xad\x9d\xd1\x11\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', (0x6ba7b811L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), 0x6ba7b8119dad11d180b400c04fd430c8L, 'urn:uuid:6ba7b811-9dad-11d1-80b4-00c04fd430c8', @@ -70,6 +76,7 @@ '{6ba7b812-9dad-11d1-80b4-00c04fd430c8}', '6ba7b8129dad11d180b400c04fd430c8', '\x6b\xa7\xb8\x12\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + '\x12\xb8\xa7\x6b\xad\x9d\xd1\x11\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', (0x6ba7b812L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), 0x6ba7b8129dad11d180b400c04fd430c8L, 'urn:uuid:6ba7b812-9dad-11d1-80b4-00c04fd430c8', @@ -78,6 +85,7 @@ '{6ba7b814-9dad-11d1-80b4-00c04fd430c8}', '6ba7b8149dad11d180b400c04fd430c8', '\x6b\xa7\xb8\x14\x9d\xad\x11\xd1\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', + '\x14\xb8\xa7\x6b\xad\x9d\xd1\x11\x80\xb4\x00\xc0\x4f\xd4\x30\xc8', (0x6ba7b814L, 0x9dad, 0x11d1, 0x80, 0xb4, 0x00c04fd430c8L), 0x6ba7b8149dad11d180b400c04fd430c8L, 'urn:uuid:6ba7b814-9dad-11d1-80b4-00c04fd430c8', @@ -86,6 +94,7 @@ '{7d444840-9dc0-11d1-b245-5ffdce74fad2}', '7d4448409dc011d1b2455ffdce74fad2', '\x7d\x44\x48\x40\x9d\xc0\x11\xd1\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', + '\x40\x48\x44\x7d\xc0\x9d\xd1\x11\xb2\x45\x5f\xfd\xce\x74\xfa\xd2', (0x7d444840L, 0x9dc0, 0x11d1, 0xb2, 0x45, 0x5ffdce74fad2L), 0x7d4448409dc011d1b2455ffdce74fad2L, 'urn:uuid:7d444840-9dc0-11d1-b245-5ffdce74fad2', @@ -94,6 +103,7 @@ '{e902893a-9d22-3c7e-a7b8-d6e313b71d9f}', 'e902893a9d223c7ea7b8d6e313b71d9f', '\xe9\x02\x89\x3a\x9d\x22\x3c\x7e\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', + '\x3a\x89\x02\xe9\x22\x9d\x7e\x3c\xa7\xb8\xd6\xe3\x13\xb7\x1d\x9f', (0xe902893aL, 0x9d22, 0x3c7e, 0xa7, 0xb8, 0xd6e313b71d9fL), 0xe902893a9d223c7ea7b8d6e313b71d9fL, 'urn:uuid:e902893a-9d22-3c7e-a7b8-d6e313b71d9f', @@ -102,6 +112,7 @@ '{eb424026-6f54-4ef8-a4d0-bb658a1fc6cf}', 'eb4240266f544ef8a4d0bb658a1fc6cf', '\xeb\x42\x40\x26\x6f\x54\x4e\xf8\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', + '\x26\x40\x42\xeb\x54\x6f\xf8\x4e\xa4\xd0\xbb\x65\x8a\x1f\xc6\xcf', (0xeb424026L, 0x6f54, 0x4ef8, 0xa4, 0xd0, 0xbb658a1fc6cfL), 0xeb4240266f544ef8a4d0bb658a1fc6cfL, 'urn:uuid:eb424026-6f54-4ef8-a4d0-bb658a1fc6cf', @@ -110,6 +121,7 @@ '{f81d4fae-7dec-11d0-a765-00a0c91e6bf6}', 'f81d4fae7dec11d0a76500a0c91e6bf6', '\xf8\x1d\x4f\xae\x7d\xec\x11\xd0\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', + '\xae\x4f\x1d\xf8\xec\x7d\xd0\x11\xa7\x65\x00\xa0\xc9\x1e\x6b\xf6', (0xf81d4faeL, 0x7dec, 0x11d0, 0xa7, 0x65, 0x00a0c91e6bf6L), 0xf81d4fae7dec11d0a76500a0c91e6bf6L, 'urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6', @@ -118,6 +130,7 @@ '{fffefdfc-fffe-fffe-fffe-fffefdfcfbfa}', 'fffefdfcfffefffefffefffefdfcfbfa', '\xff\xfe\xfd\xfc\xff\xfe\xff\xfe\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', + '\xfc\xfd\xfe\xff\xfe\xff\xfe\xff\xff\xfe\xff\xfe\xfd\xfc\xfb\xfa', (0xfffefdfcL, 0xfffe, 0xfffe, 0xff, 0xfe, 0xfffefdfcfbfaL), 0xfffefdfcfffefffefffefffefdfcfbfaL, 'urn:uuid:fffefdfc-fffe-fffe-fffe-fffefdfcfbfa', @@ -126,6 +139,7 @@ '{ffffffff-ffff-ffff-ffff-ffffffffffff}', 'ffffffffffffffffffffffffffffffff', '\xff'*16, + '\xff'*16, (0xffffffffL, 0xffffL, 0xffffL, 0xff, 0xff, 0xffffffffffffL), 0xffffffffffffffffffffffffffffffffL, 'urn:uuid:ffffffff-ffff-ffff-ffff-ffffffffffff', @@ -134,12 +148,14 @@ equivalents = [] # Construct each UUID in several different ways. for u in [uuid.UUID(string), uuid.UUID(curly), uuid.UUID(hex), - uuid.UUID(bytes=bytes), uuid.UUID(fields=fields), - uuid.UUID(int=integer), uuid.UUID(urn)]: + uuid.UUID(bytes=bytes), uuid.UUID(bytes_le=bytes_le), + uuid.UUID(fields=fields), uuid.UUID(int=integer), + uuid.UUID(urn)]: # Test all conversions and properties of the UUID object. equal(str(u), string) equal(int(u), integer) equal(u.bytes, bytes) + equal(u.bytes_le, bytes_le) equal(u.fields, fields) equal(u.time_low, fields[0]) equal(u.time_mid, fields[1]) @@ -189,6 +205,11 @@ badvalue(lambda: uuid.UUID(bytes='\0'*15)) badvalue(lambda: uuid.UUID(bytes='\0'*17)) + # Badly formed bytes_le. + badvalue(lambda: uuid.UUID(bytes_le='abc')) + badvalue(lambda: uuid.UUID(bytes_le='\0'*15)) + badvalue(lambda: uuid.UUID(bytes_le='\0'*17)) + # Badly formed fields. badvalue(lambda: uuid.UUID(fields=(1,))) badvalue(lambda: uuid.UUID(fields=(1, 2, 3, 4, 5))) @@ -221,51 +242,43 @@ uuid.UUID(h) uuid.UUID(hex=h) uuid.UUID(bytes=b) + uuid.UUID(bytes_le=b) uuid.UUID(fields=f) uuid.UUID(int=i) # Wrong number of arguments (positional). badtype(lambda: uuid.UUID()) badtype(lambda: uuid.UUID(h, b)) - badtype(lambda: uuid.UUID(h, b, f)) - badtype(lambda: uuid.UUID(h, b, f, i)) - - # Duplicate arguments (named). - badtype(lambda: uuid.UUID(hex=h, bytes=b)) - badtype(lambda: uuid.UUID(hex=h, fields=f)) - badtype(lambda: uuid.UUID(hex=h, int=i)) - badtype(lambda: uuid.UUID(bytes=b, fields=f)) - badtype(lambda: uuid.UUID(bytes=b, int=i)) - badtype(lambda: uuid.UUID(fields=f, int=i)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(hex=h, fields=f, int=i)) - badtype(lambda: uuid.UUID(bytes=b, int=i, fields=f)) - badtype(lambda: uuid.UUID(hex=h, bytes=b, int=i, fields=f)) - - # Duplicate arguments (positional and named). - badtype(lambda: uuid.UUID(h, hex=h)) - badtype(lambda: uuid.UUID(h, bytes=b)) - badtype(lambda: uuid.UUID(h, fields=f)) - badtype(lambda: uuid.UUID(h, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b)) - badtype(lambda: uuid.UUID(h, hex=h, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, int=i)) - badtype(lambda: uuid.UUID(h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(h, fields=f, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i)) - badtype(lambda: uuid.UUID(h, hex=h, fields=f, int=i)) - badtype(lambda: uuid.UUID(h, bytes=b, int=i, fields=f)) - badtype(lambda: uuid.UUID(h, hex=h, bytes=b, int=i, fields=f)) + badtype(lambda: uuid.UUID(h, b, b)) + badtype(lambda: uuid.UUID(h, b, b, f)) + badtype(lambda: uuid.UUID(h, b, b, f, i)) + + # Duplicate arguments. + for hh in [[], [('hex', h)]]: + for bb in [[], [('bytes', b)]]: + for bble in [[], [('bytes_le', b)]]: + for ii in [[], [('int', i)]]: + for ff in [[], [('fields', f)]]: + args = dict(hh + bb + bble + ii + ff) + if len(args) != 0: + badtype(lambda: uuid.UUID(h, **args)) + if len(args) != 1: + badtype(lambda: uuid.UUID(**args)) # Immutability. u = uuid.UUID(h) badtype(lambda: setattr(u, 'hex', h)) badtype(lambda: setattr(u, 'bytes', b)) + badtype(lambda: setattr(u, 'bytes_le', b)) badtype(lambda: setattr(u, 'fields', f)) badtype(lambda: setattr(u, 'int', i)) + badtype(lambda: setattr(u, 'time_low', 0)) + badtype(lambda: setattr(u, 'time_mid', 0)) + badtype(lambda: setattr(u, 'time_hi_version', 0)) + badtype(lambda: setattr(u, 'time_hi_version', 0)) + badtype(lambda: setattr(u, 'clock_seq_hi_variant', 0)) + badtype(lambda: setattr(u, 'clock_seq_low', 0)) + badtype(lambda: setattr(u, 'node', 0)) def check_node(self, node, source): individual_group_bit = (node >> 40L) & 1 @@ -356,11 +369,17 @@ def test_uuid1(self): equal = self.assertEqual - # Make sure uuid4() generates UUIDs that are actually version 1. + # Make sure uuid1() generates UUIDs that are actually version 1. for u in [uuid.uuid1() for i in range(10)]: equal(u.variant, uuid.RFC_4122) equal(u.version, 1) + # Make sure the generated UUIDs are actually unique. + uuids = {} + for u in [uuid.uuid1() for i in range(1000)]: + uuids[u] = 1 + equal(len(uuids.keys()), 1000) + # Make sure the supplied node ID appears in the UUID. u = uuid.uuid1(0) equal(u.node, 0) @@ -408,6 +427,12 @@ equal(u.variant, uuid.RFC_4122) equal(u.version, 4) + # Make sure the generated UUIDs are actually unique. + uuids = {} + for u in [uuid.uuid4() for i in range(1000)]: + uuids[u] = 1 + equal(len(uuids.keys()), 1000) + def test_uuid5(self): equal = self.assertEqual Modified: python/branches/p3yk/Lib/test/test_xml_etree_c.py ============================================================================== --- python/branches/p3yk/Lib/test/test_xml_etree_c.py (original) +++ python/branches/p3yk/Lib/test/test_xml_etree_c.py Mon Aug 21 21:07:27 2006 @@ -204,6 +204,17 @@ "<?xml version='1.0' encoding='%s'?><xml />" % encoding ) +def bug_1534630(): + """ + >>> bob = ET.TreeBuilder() + >>> e = bob.data("data") + >>> e = bob.start("tag", {}) + >>> e = bob.end("tag") + >>> e = bob.close() + >>> serialize(ET, e) + '<tag />' + """ + def test_main(): from test import test_xml_etree_c test_support.run_doctest(test_xml_etree_c, verbosity=True) Modified: python/branches/p3yk/Lib/tokenize.py ============================================================================== --- python/branches/p3yk/Lib/tokenize.py (original) +++ python/branches/p3yk/Lib/tokenize.py Mon Aug 21 21:07:27 2006 @@ -273,7 +273,8 @@ while column < indents[-1]: if column not in indents: raise IndentationError( - "unindent does not match any outer indentation level") + "unindent does not match any outer indentation level", + ("<tokenize>", lnum, pos, line)) indents = indents[:-1] yield (DEDENT, '', (lnum, pos), (lnum, pos), line) Modified: python/branches/p3yk/Lib/trace.py ============================================================================== --- python/branches/p3yk/Lib/trace.py (original) +++ python/branches/p3yk/Lib/trace.py Mon Aug 21 21:07:27 2006 @@ -179,9 +179,11 @@ # looking in sys.path for the longest matching prefix. We'll # assume that the rest is the package name. + comparepath = os.path.normcase(path) longest = "" for dir in sys.path: - if path.startswith(dir) and path[len(dir)] == os.path.sep: + dir = os.path.normcase(dir) + if comparepath.startswith(dir) and comparepath[len(dir)] == os.sep: if len(dir) > len(longest): longest = dir Modified: python/branches/p3yk/Lib/urllib2.py ============================================================================== --- python/branches/p3yk/Lib/urllib2.py (original) +++ python/branches/p3yk/Lib/urllib2.py Mon Aug 21 21:07:27 2006 @@ -263,11 +263,11 @@ def add_header(self, key, val): # useful for something like authentication - self.headers[key.title()] = val + self.headers[key.capitalize()] = val def add_unredirected_header(self, key, val): # will not be added to a redirected request - self.unredirected_hdrs[key.title()] = val + self.unredirected_hdrs[key.capitalize()] = val def has_header(self, header_name): return (header_name in self.headers or @@ -286,7 +286,7 @@ class OpenerDirector: def __init__(self): client_version = "Python-urllib/%s" % __version__ - self.addheaders = [('User-Agent', client_version)] + self.addheaders = [('User-agent', client_version)] # manage the individual handlers self.handlers = [] self.handle_open = {} @@ -675,7 +675,7 @@ if user and password: user_pass = '%s:%s' % (unquote(user), unquote(password)) creds = base64.encodestring(user_pass).strip() - req.add_header('Proxy-Authorization', 'Basic ' + creds) + req.add_header('Proxy-authorization', 'Basic ' + creds) hostport = unquote(hostport) req.set_proxy(hostport, proxy_type) if orig_type == proxy_type: @@ -819,7 +819,7 @@ class ProxyBasicAuthHandler(AbstractBasicAuthHandler, BaseHandler): - auth_header = 'Proxy-Authorization' + auth_header = 'Proxy-authorization' def http_error_407(self, req, fp, code, msg, headers): # http_error_auth_reqed requires that there is no userinfo component in @@ -1022,20 +1022,20 @@ if request.has_data(): # POST data = request.get_data() - if not request.has_header('Content-Type'): + if not request.has_header('Content-type'): request.add_unredirected_header( - 'Content-Type', + 'Content-type', 'application/x-www-form-urlencoded') - if not request.has_header('Content-Length'): + if not request.has_header('Content-length'): request.add_unredirected_header( - 'Content-Length', '%d' % len(data)) + 'Content-length', '%d' % len(data)) scheme, sel = splittype(request.get_selector()) sel_host, sel_path = splithost(sel) if not request.has_header('Host'): request.add_unredirected_header('Host', sel_host or host) for name, value in self.parent.addheaders: - name = name.title() + name = name.capitalize() if not request.has_header(name): request.add_unredirected_header(name, value) @@ -1067,6 +1067,8 @@ # So make sure the connection gets closed after the (only) # request. headers["Connection"] = "close" + headers = dict( + (name.title(), val) for name, val in headers.items()) try: h.request(req.get_method(), req.get_selector(), req.data, headers) r = h.getresponse() @@ -1217,7 +1219,7 @@ modified = email.Utils.formatdate(stats.st_mtime, usegmt=True) mtype = mimetypes.guess_type(file)[0] headers = mimetools.Message(StringIO( - 'Content-Type: %s\nContent-Length: %d\nLast-Modified: %s\n' % + 'Content-type: %s\nContent-length: %d\nLast-modified: %s\n' % (mtype or 'text/plain', size, modified))) if host: host, port = splitport(host) @@ -1272,9 +1274,9 @@ headers = "" mtype = mimetypes.guess_type(req.get_full_url())[0] if mtype: - headers += "Content-Type: %s\n" % mtype + headers += "Content-type: %s\n" % mtype if retrlen is not None and retrlen >= 0: - headers += "Content-Length: %d\n" % retrlen + headers += "Content-length: %d\n" % retrlen sf = StringIO(headers) headers = mimetools.Message(sf) return addinfourl(fp, headers, req.get_full_url()) Modified: python/branches/p3yk/Lib/uuid.py ============================================================================== --- python/branches/p3yk/Lib/uuid.py (original) +++ python/branches/p3yk/Lib/uuid.py Mon Aug 21 21:07:27 2006 @@ -45,8 +45,6 @@ """ __author__ = 'Ka-Ping Yee <ping(a)zesty.ca>' -__date__ = '$Date: 2006/06/12 23:15:40 $'.split()[1].replace('/', '-') -__version__ = '$Revision: 1.30 $'.split()[1] RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, RESERVED_FUTURE = [ 'reserved for NCS compatibility', 'specified in RFC 4122', @@ -57,15 +55,21 @@ UUID objects are immutable, hashable, and usable as dictionary keys. Converting a UUID to a string with str() yields something in the form '12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts - four possible forms: a similar string of hexadecimal digits, or a - string of 16 raw bytes as an argument named 'bytes', or a tuple of - six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and - 48-bit values respectively) as an argument named 'fields', or a single - 128-bit integer as an argument named 'int'. + five possible forms: a similar string of hexadecimal digits, or a tuple + of six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and + 48-bit values respectively) as an argument named 'fields', or a string + of 16 bytes (with all the integer fields in big-endian order) as an + argument named 'bytes', or a string of 16 bytes (with the first three + fields in little-endian order) as an argument named 'bytes_le', or a + single 128-bit integer as an argument named 'int'. UUIDs have these read-only attributes: - bytes the UUID as a 16-byte string + bytes the UUID as a 16-byte string (containing the six + integer fields in big-endian byte order) + + bytes_le the UUID as a 16-byte string (with time_low, time_mid, + and time_hi_version in little-endian byte order) fields a tuple of the six integer fields of the UUID, which are also available as six individual attributes @@ -94,10 +98,11 @@ when the variant is RFC_4122) """ - def __init__(self, hex=None, bytes=None, fields=None, int=None, - version=None): + def __init__(self, hex=None, bytes=None, bytes_le=None, fields=None, + int=None, version=None): r"""Create a UUID from either a string of 32 hexadecimal digits, - a string of 16 bytes as the 'bytes' argument, a tuple of six + a string of 16 bytes as the 'bytes' argument, a string of 16 bytes + in little-endian order as the 'bytes_le' argument, a tuple of six integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as the 'fields' argument, or a single 128-bit integer as the 'int' @@ -109,23 +114,31 @@ UUID('12345678123456781234567812345678') UUID('urn:uuid:12345678-1234-5678-1234-567812345678') UUID(bytes='\x12\x34\x56\x78'*4) + UUID(bytes_le='\x78\x56\x34\x12\x34\x12\x78\x56' + + '\x12\x34\x56\x78\x12\x34\x56\x78') UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) UUID(int=0x12345678123456781234567812345678) - Exactly one of 'hex', 'bytes', 'fields', or 'int' must be given. - The 'version' argument is optional; if given, the resulting UUID - will have its variant and version number set according to RFC 4122, - overriding bits in the given 'hex', 'bytes', 'fields', or 'int'. + Exactly one of 'hex', 'bytes', 'bytes_le', 'fields', or 'int' must + be given. The 'version' argument is optional; if given, the resulting + UUID will have its variant and version set according to RFC 4122, + overriding the given 'hex', 'bytes', 'bytes_le', 'fields', or 'int'. """ - if [hex, bytes, fields, int].count(None) != 3: - raise TypeError('need just one of hex, bytes, fields, or int') + if [hex, bytes, bytes_le, fields, int].count(None) != 4: + raise TypeError('need one of hex, bytes, bytes_le, fields, or int') if hex is not None: hex = hex.replace('urn:', '').replace('uuid:', '') hex = hex.strip('{}').replace('-', '') if len(hex) != 32: raise ValueError('badly formed hexadecimal UUID string') int = long(hex, 16) + if bytes_le is not None: + if len(bytes_le) != 16: + raise ValueError('bytes_le is not a 16-char string') + bytes = (bytes_le[3] + bytes_le[2] + bytes_le[1] + bytes_le[0] + + bytes_le[5] + bytes_le[4] + bytes_le[7] + bytes_le[6] + + bytes_le[8:]) if bytes is not None: if len(bytes) != 16: raise ValueError('bytes is not a 16-char string') @@ -194,6 +207,13 @@ bytes = property(get_bytes) + def get_bytes_le(self): + bytes = self.bytes + return (bytes[3] + bytes[2] + bytes[1] + bytes[0] + + bytes[5] + bytes[4] + bytes[7] + bytes[6] + bytes[8:]) + + bytes_le = property(get_bytes_le) + def get_fields(self): return (self.time_low, self.time_mid, self.time_hi_version, self.clock_seq_hi_variant, self.clock_seq_low, self.node) @@ -448,6 +468,8 @@ if _node is not None: return _node +_last_timestamp = None + def uuid1(node=None, clock_seq=None): """Generate a UUID from a host ID, sequence number, and the current time. If 'node' is not given, getnode() is used to obtain the hardware @@ -460,11 +482,15 @@ _uuid_generate_time(_buffer) return UUID(bytes=_buffer.raw) + global _last_timestamp import time nanoseconds = int(time.time() * 1e9) # 0x01b21dd213814000 is the number of 100-ns intervals between the # UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00. timestamp = int(nanoseconds/100) + 0x01b21dd213814000L + if timestamp <= _last_timestamp: + timestamp = _last_timestamp + 1 + _last_timestamp = timestamp if clock_seq is None: import random clock_seq = random.randrange(1<<14L) # instead of stable storage Modified: python/branches/p3yk/Makefile.pre.in ============================================================================== --- python/branches/p3yk/Makefile.pre.in (original) +++ python/branches/p3yk/Makefile.pre.in Mon Aug 21 21:07:27 2006 @@ -260,6 +260,7 @@ Python/modsupport.o \ Python/mystrtoul.o \ Python/mysnprintf.o \ + Python/peephole.o \ Python/pyarena.o \ Python/pyfpe.o \ Python/pystate.o \ Modified: python/branches/p3yk/Misc/ACKS ============================================================================== --- python/branches/p3yk/Misc/ACKS (original) +++ python/branches/p3yk/Misc/ACKS Mon Aug 21 21:07:27 2006 @@ -412,6 +412,7 @@ Michael McLay Gordon McMillan Jay T. Miller +Chris McDonough Andrew McNamara Caolan McNamara Craig McPheeters Modified: python/branches/p3yk/Misc/PURIFY.README ============================================================================== --- python/branches/p3yk/Misc/PURIFY.README (original) +++ python/branches/p3yk/Misc/PURIFY.README Mon Aug 21 21:07:27 2006 @@ -1,6 +1,6 @@ Purify (tm) and Quantify (tm) are commercial software quality -assurance tools available from Rational Software Corporation -<http://www.rational.com/>. Purify is essentially a memory access +assurance tools available from IBM <http://www.ibm.com/software/rational/>. +Purify is essentially a memory access verifier and leak detector; Quantify is a C level profiler. The rest of this file assumes you generally know how to use Purify and Quantify, and that you have installed valid licenses for these @@ -20,6 +20,17 @@ make PURIFY=quantify +Starting with Python 2.3, pymalloc is enabled by default. This +will cause many supurious warnings. Modify Objects/obmalloc.c +and enable Py_USING_MEMORY_DEBUGGER by uncommenting it. +README.valgrind has more details about why this is necessary. +See below about setting up suppressions. Some tests may not +run well with Purify due to heavy memory or CPU usage. These +tests may include: test_largefile, test_import, and test_long. + +Please report any findings (problems or no warnings) to python-dev(a)python.org. +It may be useful to submit a bug report for any problems. + When running the regression test (make test), I have found it useful to set my PURIFYOPTIONS environment variable using the following (bash) shell function. Check out the Purify documentation for @@ -52,6 +63,11 @@ suppress umr ...; "nismodule.c" suppress umr ...; "pwdmodule.c" +Note: this list is very old and may not be accurate any longer. +It's possible some of these no longer need to be suppressed. +You will also need to suppress warnings (at least umr) +from Py_ADDRESS_IN_RANGE. + This will still leave you with just a few UMR, mostly in the readline library, which you can safely ignore. A lot of work has gone into Python 1.5 to plug as many leaks as possible. Modified: python/branches/p3yk/Misc/README.klocwork ============================================================================== --- python/branches/p3yk/Misc/README.klocwork (original) +++ python/branches/p3yk/Misc/README.klocwork Mon Aug 21 21:07:27 2006 @@ -23,4 +23,8 @@ False positives were also annotated so that the comments can be reviewed and reversed if the analysis was incorrect. +A second run was performed on 10-Aug-2006. The tool was tuned to remove +some false positives and perform some additional checks. ~150 new +warnings were produced, primarily related to dereferencing NULL pointers. + Contact python-dev(a)python.org for more information. Deleted: /python/branches/p3yk/Misc/RPM/python-2.5.spec ============================================================================== --- /python/branches/p3yk/Misc/RPM/python-2.5.spec Mon Aug 21 21:07:27 2006 +++ (empty file) @@ -1,385 +0,0 @@ -########################## -# User-modifiable configs -########################## - -# Is the resulting package and the installed binary named "python" or -# "python2"? -#WARNING: Commenting out doesn't work. Last line is what's used. -%define config_binsuffix none -%define config_binsuffix 2.5 - -# Build tkinter? "auto" enables it if /usr/bin/wish exists. -#WARNING: Commenting out doesn't work. Last line is what's used. -%define config_tkinter no -%define config_tkinter yes -%define config_tkinter auto - -# Use pymalloc? The last line (commented or not) determines wether -# pymalloc is used. -#WARNING: Commenting out doesn't work. Last line is what's used. -%define config_pymalloc no -%define config_pymalloc yes - -# Enable IPV6? -#WARNING: Commenting out doesn't work. Last line is what's used. -%define config_ipv6 yes -%define config_ipv6 no - -# Location of the HTML directory. -%define config_htmldir /var/www/html/python - -################################# -# End of user-modifiable configs -################################# - -%define name python -%define version 2.5b3 -%define libvers 2.5 -%define release 1pydotorg -%define __prefix /usr - -# kludge to get around rpm <percent>define weirdness -%define ipv6 %(if [ "%{config_ipv6}" = yes ]; then echo --enable-ipv6; else echo --disable-ipv6; fi) -%define pymalloc %(if [ "%{config_pymalloc}" = yes ]; then echo --with-pymalloc; else echo --without-pymalloc; fi) -%define binsuffix %(if [ "%{config_binsuffix}" = none ]; then echo ; else echo "%{config_binsuffix}"; fi) -%define include_tkinter %(if [ \$ "%{config_tkinter}" = auto -a -f /usr/bin/wish \$ -o "%{config_tkinter}" = yes ]; then echo 1; else echo 0; fi) -%define libdirname %(( uname -m | egrep -q '_64$' && [ -d /usr/lib64 ] && echo lib64 ) || echo lib) - -# detect if documentation is available -%define include_docs %(if [ -f "%{_sourcedir}/html-%{version}.tar.bz2" ]; then echo 1; else echo 0; fi) - -Summary: An interpreted, interactive, object-oriented programming language. -Name: %{name}%{binsuffix} -Version: %{version} -Release: %{release} -Copyright: Modified CNRI Open Source License -Group: Development/Languages -Source: Python-%{version}.tar.bz2 -%if %{include_docs} -Source1: html-%{version}.tar.bz2 -%endif -BuildRoot: %{_tmppath}/%{name}-%{version}-root -BuildPrereq: expat-devel -BuildPrereq: db4-devel -BuildPrereq: gdbm-devel -BuildPrereq: sqlite-devel -Prefix: %{__prefix} -Packager: Sean Reifschneider <jafo-rpms(a)tummy.com> - -%description -Python is an interpreted, interactive, object-oriented programming -language. It incorporates modules, exceptions, dynamic typing, very high -level dynamic data types, and classes. Python combines remarkable power -with very clear syntax. It has interfaces to many system calls and -libraries, as well as to various window systems, and is extensible in C or -C++. It is also usable as an extension language for applications that need -a programmable interface. Finally, Python is portable: it runs on many -brands of UNIX, on PCs under Windows, MS-DOS, and OS/2, and on the -Mac. - -%package devel -Summary: The libraries and header files needed for Python extension development. -Prereq: python%{binsuffix} = %{PACKAGE_VERSION} -Group: Development/Libraries - -%description devel -The Python programming language's interpreter can be extended with -dynamically loaded extensions and can be embedded in other programs. -This package contains the header files and libraries needed to do -these types of tasks. - -Install python-devel if you want to develop Python extensions. The -python package will also need to be installed. You'll probably also -want to install the python-docs package, which contains Python -documentation. - -%if %{include_tkinter} -%package tkinter -Summary: A graphical user interface for the Python scripting language. -Group: Development/Languages -Prereq: python%{binsuffix} = %{PACKAGE_VERSION}-%{release} - -%description tkinter -The Tkinter (Tk interface) program is an graphical user interface for -the Python scripting language. - -You should install the tkinter package if you'd like to use a graphical -user interface for Python programming. -%endif - -%package tools -Summary: A collection of development tools included with Python. -Group: Development/Tools -Prereq: python%{binsuffix} = %{PACKAGE_VERSION}-%{release} - -%description tools -The Python package includes several development tools that are used -to build python programs. This package contains a selection of those -tools, including the IDLE Python IDE. - -Install python-tools if you want to use these tools to develop -Python programs. You will also need to install the python and -tkinter packages. - -%if %{include_docs} -%package docs -Summary: Python-related documentation. -Group: Development/Documentation - -%description docs -Documentation relating to the Python programming language in HTML and info -formats. -%endif - -%changelog -* Mon Dec 20 2004 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.4-2pydotorg] -- Changing the idle wrapper so that it passes arguments to idle. - -* Tue Oct 19 2004 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.4b1-1pydotorg] -- Updating to 2.4. - -* Thu Jul 22 2004 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3.4-3pydotorg] -- Paul Tiemann fixes for %{prefix}. -- Adding permission changes for directory as suggested by reimeika.ca -- Adding code to detect when it should be using lib64. -- Adding a define for the location of /var/www/html for docs. - -* Thu May 27 2004 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3.4-2pydotorg] -- Including changes from Ian Holsman to build under Red Hat 7.3. -- Fixing some problems with the /usr/local path change. - -* Sat Mar 27 2004 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3.2-3pydotorg] -- Being more agressive about finding the paths to fix for - #!/usr/local/bin/python. - -* Sat Feb 07 2004 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3.3-2pydotorg] -- Adding code to remove "#!/usr/local/bin/python" from particular files and - causing the RPM build to terminate if there are any unexpected files - which have that line in them. - -* Mon Oct 13 2003 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3.2-1pydotorg] -- Adding code to detect wether documentation is available to build. - -* Fri Sep 19 2003 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3.1-1pydotorg] -- Updating to the 2.3.1 release. - -* Mon Feb 24 2003 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3b1-1pydotorg] -- Updating to 2.3b1 release. - -* Mon Feb 17 2003 Sean Reifschneider <jafo-rpms(a)tummy.com> [2.3a1-1] -- Updating to 2.3 release. - -* Sun Dec 23 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2-2] -- Added -docs package. -- Added "auto" config_tkinter setting which only enables tk if - /usr/bin/wish exists. - -* Sat Dec 22 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2-1] -- Updated to 2.2. -- Changed the extension to "2" from "2.2". - -* Tue Nov 18 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2c1-1] -- Updated to 2.2c1. - -* Thu Nov 1 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2b1-3] -- Changed the way the sed for fixing the #! in pydoc works. - -* Wed Oct 24 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2b1-2] -- Fixed missing "email" package, thanks to anonymous report on sourceforge. -- Fixed missing "compiler" package. - -* Mon Oct 22 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2b1-1] -- Updated to 2.2b1. - -* Mon Oct 9 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2a4-4] -- otto(a)balinor.mat.unimi.it mentioned that the license file is missing. - -* Sun Sep 30 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2a4-3] -- Ignacio Vazquez-Abrams pointed out that I had a spruious double-quote in - the spec files. Thanks. - -* Wed Jul 25 2001 Sean Reifschneider <jafo-rpms(a)tummy.com> -[Release 2.2a1-1] -- Updated to 2.2a1 release. -- Changed idle and pydoc to use binsuffix macro - -####### -# PREP -####### -%prep -%setup -n Python-%{version} - -######## -# BUILD -######## -%build -./configure --enable-unicode=ucs4 %{ipv6} %{pymalloc} --prefix=%{__prefix} -make - -########## -# INSTALL -########## -%install -# set the install path -echo '[install_scripts]' >setup.cfg -echo 'install_dir='"${RPM_BUILD_ROOT}%{__prefix}/bin" >>setup.cfg - -[ -d "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT -mkdir -p $RPM_BUILD_ROOT%{__prefix}/%{libdirname}/python%{libvers}/lib-dynload -make prefix=$RPM_BUILD_ROOT%{__prefix} install - -# REPLACE PATH IN PYDOC -if [ ! -z "%{binsuffix}" ] -then - ( - cd $RPM_BUILD_ROOT%{__prefix}/bin - mv pydoc pydoc.old - sed 's|#!.*|#!%{__prefix}/bin/env python'%{binsuffix}'|' \ - pydoc.old >pydoc - chmod 755 pydoc - rm -f pydoc.old - ) -fi - -# add the binsuffix -if [ ! -z "%{binsuffix}" ] -then - ( cd $RPM_BUILD_ROOT%{__prefix}/bin; rm -f python[0-9a-zA-Z]*; - mv -f python python"%{binsuffix}" ) - ( cd $RPM_BUILD_ROOT%{__prefix}/man/man1; mv python.1 python%{binsuffix}.1 ) - ( cd $RPM_BUILD_ROOT%{__prefix}/bin; mv -f pydoc pydoc"%{binsuffix}" ) - ( cd $RPM_BUILD_ROOT%{__prefix}/bin; mv -f idle idle"%{binsuffix}" ) -fi - -######## -# Tools -echo '#!%{__prefix}/bin/env python%{binsuffix}' >${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} -echo 'import os, sys' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} -echo 'os.execvp("%{__prefix}/bin/python%{binsuffix}", ["%{__prefix}/bin/python%{binsuffix}", "%{__prefix}/lib/python%{libvers}/idlelib/idle.py"] + sys.argv[1:])' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} -echo 'print "Failed to exec Idle"' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} -echo 'sys.exit(1)' >>${RPM_BUILD_ROOT}%{__prefix}/bin/idle%{binsuffix} -chmod 755 $RPM_BUILD_ROOT%{__prefix}/bin/idle%{binsuffix} -cp -a Tools $RPM_BUILD_ROOT%{__prefix}/%{libdirname}/python%{libvers} - -# MAKE FILE LISTS -rm -f mainpkg.files -find "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/lib-dynload -type f | - sed "s|^${RPM_BUILD_ROOT}|/|" | - grep -v -e '_tkinter.so$' >mainpkg.files -find "$RPM_BUILD_ROOT""%{__prefix}"/bin -type f | - sed "s|^${RPM_BUILD_ROOT}|/|" | - grep -v -e '/bin/idle%{binsuffix}$' >>mainpkg.files - -rm -f tools.files -find "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/idlelib \ - "$RPM_BUILD_ROOT""%{__prefix}"/%{libdirname}/python%{libvers}/Tools -type f | - sed "s|^${RPM_BUILD_ROOT}|/|" >tools.files -echo "%{__prefix}"/bin/idle%{binsuffix} >>tools.files - -###### -# Docs -%if %{include_docs} -mkdir -p "$RPM_BUILD_ROOT"%{config_htmldir} -( - cd "$RPM_BUILD_ROOT"%{config_htmldir} - bunzip2 < %{SOURCE1} | tar x -) -%endif - -# fix the #! line in installed files -find "$RPM_BUILD_ROOT" -type f -print0 | - xargs -0 grep -l /usr/local/bin/python | while read file -do - FIXFILE="$file" - sed 's|^#!.*python|#!%{__prefix}/bin/env python'"%{binsuffix}"'|' \ - "$FIXFILE" >/tmp/fix-python-path.$$ - cat /tmp/fix-python-path.$$ >"$FIXFILE" - rm -f /tmp/fix-python-path.$$ -done - -# check to see if there are any straggling #! lines -find "$RPM_BUILD_ROOT" -type f | xargs egrep -n '^#! */usr/local/bin/python' \ - | grep ':1:#!' >/tmp/python-rpm-files.$$ || true -if [ -s /tmp/python-rpm-files.$$ ] -then - echo '*****************************************************' - cat /tmp/python-rpm-files.$$ - cat <<@EOF - ***************************************************** - There are still files referencing /usr/local/bin/python in the - install directory. They are listed above. Please fix the .spec - file and try again. If you are an end-user, you probably want - to report this to jafo-rpms(a)tummy.com as well. - ***************************************************** -@EOF - rm -f /tmp/python-rpm-files.$$ - exit 1 -fi -rm -f /tmp/python-rpm-files.$$ - -######## -# CLEAN -######## -%clean -[ -n "$RPM_BUILD_ROOT" -a "$RPM_BUILD_ROOT" != / ] && rm -rf $RPM_BUILD_ROOT -rm -f mainpkg.files tools.files - -######## -# FILES -######## -%files -f mainpkg.files -%defattr(-,root,root) -%doc Misc/README Misc/cheatsheet Misc/Porting -%doc LICENSE Misc/ACKS Misc/HISTORY Misc/NEWS -%{__prefix}/man/man1/python%{binsuffix}.1* - -%attr(755,root,root) %dir %{__prefix}/include/python%{libvers} -%attr(755,root,root) %dir %{__prefix}/%{libdirname}/python%{libvers}/ -%{__prefix}/%{libdirname}/python%{libvers}/*.txt -%{__prefix}/%{libdirname}/python%{libvers}/*.py* -%{__prefix}/%{libdirname}/python%{libvers}/pdb.doc -%{__prefix}/%{libdirname}/python%{libvers}/profile.doc -%{__prefix}/%{libdirname}/python%{libvers}/curses -%{__prefix}/%{libdirname}/python%{libvers}/distutils -%{__prefix}/%{libdirname}/python%{libvers}/encodings -%{__prefix}/%{libdirname}/python%{libvers}/plat-linux2 -%{__prefix}/%{libdirname}/python%{libvers}/site-packages -%{__prefix}/%{libdirname}/python%{libvers}/test -%{__prefix}/%{libdirname}/python%{libvers}/xml -%{__prefix}/%{libdirname}/python%{libvers}/email -%{__prefix}/%{libdirname}/python%{libvers}/email/mime -%{__prefix}/%{libdirname}/python%{libvers}/sqlite3 -%{__prefix}/%{libdirname}/python%{libvers}/compiler -%{__prefix}/%{libdirname}/python%{libvers}/bsddb -%{__prefix}/%{libdirname}/python%{libvers}/hotshot -%{__prefix}/%{libdirname}/python%{libvers}/logging -%{__prefix}/%{libdirname}/python%{libvers}/lib-old - -%files devel -%defattr(-,root,root) -%{__prefix}/include/python%{libvers}/*.h -%{__prefix}/%{libdirname}/python%{libvers}/config - -%files -f tools.files tools -%defattr(-,root,root) - -%if %{include_tkinter} -%files tkinter -%defattr(-,root,root) -%{__prefix}/%{libdirname}/python%{libvers}/lib-tk -%{__prefix}/%{libdirname}/python%{libvers}/lib-dynload/_tkinter.so* -%endif - -%if %{include_docs} -%files docs -%defattr(-,root,root) -%{config_htmldir}/* -%endif Modified: python/branches/p3yk/Misc/build.sh ============================================================================== --- python/branches/p3yk/Misc/build.sh (original) +++ python/branches/p3yk/Misc/build.sh Mon Aug 21 21:07:27 2006 @@ -58,7 +58,7 @@ PYTHON=$INSTALL_DIR/bin/python # Python options and regression test program that should always be run. -REGRTEST_ARGS="-E -tt $INSTALL_DIR/lib/python2.5/test/regrtest.py" +REGRTEST_ARGS="-E -tt $INSTALL_DIR/lib/python2.6/test/regrtest.py" REFLOG="build/reflog.txt.out" # These tests are not stable and falsely report leaks sometimes. Modified: python/branches/p3yk/Modules/_bsddb.c ============================================================================== --- python/branches/p3yk/Modules/_bsddb.c (original) +++ python/branches/p3yk/Modules/_bsddb.c Mon Aug 21 21:07:27 2006 @@ -1797,7 +1797,6 @@ DBC** cursors; DBC* dbc; - if (!PyArg_ParseTuple(args,"O|i:join", &cursorsObj, &flags)) return NULL; @@ -1811,6 +1810,11 @@ length = PyObject_Length(cursorsObj); cursors = malloc((length+1) * sizeof(DBC*)); + if (!cursors) { + PyErr_NoMemory(); + return NULL; + } + cursors[length] = NULL; for (x=0; x<length; x++) { PyObject* item = PySequence_GetItem(cursorsObj, x); @@ -2622,11 +2626,13 @@ CHECK_DB_NOT_CLOSED(self); if (outFileName) outFile = fopen(outFileName, "w"); + /* XXX(nnorwitz): it should probably be an exception if outFile + can't be opened. */ MYDB_BEGIN_ALLOW_THREADS; err = self->db->verify(self->db, fileName, dbName, outFile, flags); MYDB_END_ALLOW_THREADS; - if (outFileName) + if (outFile) fclose(outFile); /* DB.verify acts as a DB handle destructor (like close); this was Modified: python/branches/p3yk/Modules/_codecsmodule.c ============================================================================== --- python/branches/p3yk/Modules/_codecsmodule.c (original) +++ python/branches/p3yk/Modules/_codecsmodule.c Mon Aug 21 21:07:27 2006 @@ -192,7 +192,8 @@ buf = PyString_AS_STRING (str); len = PyString_GET_SIZE (str); memmove(buf, buf+1, len-2); - _PyString_Resize(&str, len-2); + if (_PyString_Resize(&str, len-2) < 0) + return NULL; return codec_tuple(str, PyString_Size(str)); } Modified: python/branches/p3yk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/_ctypes.c (original) +++ python/branches/p3yk/Modules/_ctypes/_ctypes.c Mon Aug 21 21:07:27 2006 @@ -140,6 +140,31 @@ accessible fields somehow. */ +static PyCArgObject * +StructUnionType_paramfunc(CDataObject *self) +{ + PyCArgObject *parg; + StgDictObject *stgdict; + + parg = new_CArgObject(); + if (parg == NULL) + return NULL; + + parg->tag = 'V'; + stgdict = PyObject_stgdict((PyObject *)self); + assert(stgdict); /* Cannot be NULL for structure/union instances */ + parg->pffi_type = &stgdict->ffi_type_pointer; + /* For structure parameters (by value), parg->value doesn't contain the structure + data itself, instead parg->value.p *points* to the structure's data + See also _ctypes.c, function _call_function_pointer(). + */ + parg->value.p = self->b_ptr; + parg->size = self->b_size; + Py_INCREF(self); + parg->obj = (PyObject *)self; + return parg; +} + static PyObject * StructUnionType_new(PyTypeObject *type, PyObject *args, PyObject *kwds, int isStruct) { @@ -172,6 +197,8 @@ Py_DECREF(result->tp_dict); result->tp_dict = (PyObject *)dict; + dict->paramfunc = StructUnionType_paramfunc; + fields = PyDict_GetItemString((PyObject *)dict, "_fields_"); if (!fields) { StgDictObject *basedict = PyType_stgdict((PyObject *)result->tp_base); @@ -287,6 +314,7 @@ static PyObject * CDataType_from_param(PyObject *type, PyObject *value) { + PyObject *as_parameter; if (1 == PyObject_IsInstance(value, type)) { Py_INCREF(value); return value; @@ -300,7 +328,6 @@ /* If we got a PyCArgObject, we must check if the object packed in it is an instance of the type's dict->proto */ -// if(dict && ob && dict->proto == (PyObject *)ob->ob_type){ if(dict && ob && PyObject_IsInstance(ob, dict->proto)) { Py_INCREF(value); @@ -330,6 +357,13 @@ } /* ... and leave the rest */ #endif + + as_parameter = PyObject_GetAttrString(value, "_as_parameter_"); + if (as_parameter) { + value = CDataType_from_param(type, as_parameter); + Py_DECREF(as_parameter); + return value; + } PyErr_Format(PyExc_TypeError, "expected %s instance instead of %s", ((PyTypeObject *)type)->tp_name, @@ -540,6 +574,23 @@ return 0; } +static PyCArgObject * +PointerType_paramfunc(CDataObject *self) +{ + PyCArgObject *parg; + + parg = new_CArgObject(); + if (parg == NULL) + return NULL; + + parg->tag = 'P'; + parg->pffi_type = &ffi_type_pointer; + Py_INCREF(self); + parg->obj = (PyObject *)self; + parg->value.p = *(void **)self->b_ptr; + return parg; +} + static PyObject * PointerType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { @@ -563,6 +614,7 @@ stgdict->align = getentry("P")->pffi_type->alignment; stgdict->length = 1; stgdict->ffi_type_pointer = ffi_type_pointer; + stgdict->paramfunc = PointerType_paramfunc; proto = PyDict_GetItemString(typedict, "_type_"); /* Borrowed ref */ if (proto && -1 == PointerType_SetProto(stgdict, proto)) { @@ -620,6 +672,7 @@ return PyInt_FromLong(0); /* NULL pointer */ typedict = PyType_stgdict(type); + assert(typedict); /* Cannot be NULL for pointer types */ /* If we expect POINTER(<type>), but receive a <type> instance, accept it by calling byref(<type>). @@ -640,6 +693,7 @@ the item types are the same. */ StgDictObject *v = PyObject_stgdict(value); + assert(v); /* Cannot be NULL for pointer or array objects */ if (PyObject_IsSubclass(v->proto, typedict->proto)) { Py_INCREF(value); return value; @@ -906,6 +960,19 @@ return 0; } +static PyCArgObject * +ArrayType_paramfunc(CDataObject *self) +{ + PyCArgObject *p = new_CArgObject(); + if (p == NULL) + return NULL; + p->tag = 'P'; + p->pffi_type = &ffi_type_pointer; + p->value.p = (char *)self->b_ptr; + Py_INCREF(self); + p->obj = (PyObject *)self; + return p; +} static PyObject * ArrayType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) @@ -961,6 +1028,8 @@ Py_INCREF(proto); stgdict->proto = proto; + stgdict->paramfunc = &ArrayType_paramfunc; + /* Arrays are passed as pointers to function calls. */ stgdict->ffi_type_pointer = ffi_type_pointer; @@ -1055,6 +1124,7 @@ static PyObject * c_wchar_p_from_param(PyObject *type, PyObject *value) { + PyObject *as_parameter; #if (PYTHON_API_VERSION < 1012) # error not supported #endif @@ -1067,6 +1137,8 @@ struct fielddesc *fd = getentry("Z"); parg = new_CArgObject(); + if (parg == NULL) + return NULL; parg->pffi_type = &ffi_type_pointer; parg->tag = 'Z'; parg->obj = fd->setfunc(&parg->value, value, 0); @@ -1083,7 +1155,9 @@ if (ArrayObject_Check(value) || PointerObject_Check(value)) { /* c_wchar array instance or pointer(c_wchar(...)) */ StgDictObject *dt = PyObject_stgdict(value); - StgDictObject *dict = dt && dt->proto ? PyType_stgdict(dt->proto) : NULL; + StgDictObject *dict; + assert(dt); /* Cannot be NULL for pointer or array objects */ + dict = dt && dt->proto ? PyType_stgdict(dt->proto) : NULL; if (dict && (dict->setfunc == getentry("u")->setfunc)) { Py_INCREF(value); return value; @@ -1098,6 +1172,13 @@ return value; } } + + as_parameter = PyObject_GetAttrString(value, "_as_parameter_"); + if (as_parameter) { + value = c_wchar_p_from_param(type, as_parameter); + Py_DECREF(as_parameter); + return value; + } /* XXX better message */ PyErr_SetString(PyExc_TypeError, "wrong type"); @@ -1107,6 +1188,7 @@ static PyObject * c_char_p_from_param(PyObject *type, PyObject *value) { + PyObject *as_parameter; #if (PYTHON_API_VERSION < 1012) # error not supported #endif @@ -1119,6 +1201,8 @@ struct fielddesc *fd = getentry("z"); parg = new_CArgObject(); + if (parg == NULL) + return NULL; parg->pffi_type = &ffi_type_pointer; parg->tag = 'z'; parg->obj = fd->setfunc(&parg->value, value, 0); @@ -1135,7 +1219,9 @@ if (ArrayObject_Check(value) || PointerObject_Check(value)) { /* c_char array instance or pointer(c_char(...)) */ StgDictObject *dt = PyObject_stgdict(value); - StgDictObject *dict = dt && dt->proto ? PyType_stgdict(dt->proto) : NULL; + StgDictObject *dict; + assert(dt); /* Cannot be NULL for pointer or array objects */ + dict = dt && dt->proto ? PyType_stgdict(dt->proto) : NULL; if (dict && (dict->setfunc == getentry("c")->setfunc)) { Py_INCREF(value); return value; @@ -1150,6 +1236,13 @@ return value; } } + + as_parameter = PyObject_GetAttrString(value, "_as_parameter_"); + if (as_parameter) { + value = c_char_p_from_param(type, as_parameter); + Py_DECREF(as_parameter); + return value; + } /* XXX better message */ PyErr_SetString(PyExc_TypeError, "wrong type"); @@ -1160,6 +1253,7 @@ c_void_p_from_param(PyObject *type, PyObject *value) { StgDictObject *stgd; + PyObject *as_parameter; #if (PYTHON_API_VERSION < 1012) # error not supported #endif @@ -1176,6 +1270,8 @@ struct fielddesc *fd = getentry("P"); parg = new_CArgObject(); + if (parg == NULL) + return NULL; parg->pffi_type = &ffi_type_pointer; parg->tag = 'P'; parg->obj = fd->setfunc(&parg->value, value, 0); @@ -1191,6 +1287,8 @@ struct fielddesc *fd = getentry("z"); parg = new_CArgObject(); + if (parg == NULL) + return NULL; parg->pffi_type = &ffi_type_pointer; parg->tag = 'z'; parg->obj = fd->setfunc(&parg->value, value, 0); @@ -1206,6 +1304,8 @@ struct fielddesc *fd = getentry("Z"); parg = new_CArgObject(); + if (parg == NULL) + return NULL; parg->pffi_type = &ffi_type_pointer; parg->tag = 'Z'; parg->obj = fd->setfunc(&parg->value, value, 0); @@ -1242,6 +1342,8 @@ CFuncPtrObject *func; func = (CFuncPtrObject *)value; parg = new_CArgObject(); + if (parg == NULL) + return NULL; parg->pffi_type = &ffi_type_pointer; parg->tag = 'P'; Py_INCREF(value); @@ -1269,6 +1371,13 @@ return (PyObject *)parg; } } + + as_parameter = PyObject_GetAttrString(value, "_as_parameter_"); + if (as_parameter) { + value = c_void_p_from_param(type, as_parameter); + Py_DECREF(as_parameter); + return value; + } /* XXX better message */ PyErr_SetString(PyExc_TypeError, "wrong type"); @@ -1355,6 +1464,33 @@ return (PyObject *)result; } +static PyCArgObject * +SimpleType_paramfunc(CDataObject *self) +{ + StgDictObject *dict; + char *fmt; + PyCArgObject *parg; + struct fielddesc *fd; + + dict = PyObject_stgdict((PyObject *)self); + assert(dict); /* Cannot be NULL for CDataObject instances */ + fmt = PyString_AsString(dict->proto); + assert(fmt); + + fd = getentry(fmt); + assert(fd); + + parg = new_CArgObject(); + if (parg == NULL) + return NULL; + + parg->tag = fmt[0]; + parg->pffi_type = fd->pffi_type; + Py_INCREF(self); + parg->obj = (PyObject *)self; + memcpy(&parg->value, self->b_ptr, self->b_size); + return parg; +} static PyObject * SimpleType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) @@ -1404,6 +1540,8 @@ stgdict->size = fmt->pffi_type->size; stgdict->setfunc = fmt->setfunc; stgdict->getfunc = fmt->getfunc; + + stgdict->paramfunc = SimpleType_paramfunc; /* if (result->tp_base != &Simple_Type) { stgdict->setfunc = NULL; @@ -1502,23 +1640,6 @@ /* * This is a *class method*. * Convert a parameter into something that ConvParam can handle. - * - * This is either an instance of the requested type, a Python integer, or a - * 'magic' 3-tuple. - * - * (These are somewhat related to Martin v. Loewis 'Enhanced Argument Tuples', - * described in PEP 286.) - * - * The tuple must contain - * - * - a format character, currently 'ifdqc' are understood - * which will inform ConvParam about how to push the argument on the stack. - * - * - a corresponding Python object: i - integer, f - float, d - float, - * q - longlong, c - integer - * - * - any object which can be used to keep the original parameter alive - * as long as the tuple lives. */ static PyObject * SimpleType_from_param(PyObject *type, PyObject *value) @@ -1527,6 +1648,7 @@ char *fmt; PyCArgObject *parg; struct fielddesc *fd; + PyObject *as_parameter; /* If the value is already an instance of the requested type, we can use it as is */ @@ -1552,11 +1674,20 @@ parg->tag = fmt[0]; parg->pffi_type = fd->pffi_type; parg->obj = fd->setfunc(&parg->value, value, 0); - if (parg->obj == NULL) { - Py_DECREF(parg); - return NULL; + if (parg->obj) + return (PyObject *)parg; + PyErr_Clear(); + Py_DECREF(parg); + + as_parameter = PyObject_GetAttrString(value, "_as_parameter_"); + if (as_parameter) { + value = SimpleType_from_param(type, as_parameter); + Py_DECREF(as_parameter); + return value; } - return (PyObject *)parg; + PyErr_SetString(PyExc_TypeError, + "wrong type"); + return NULL; } static PyMethodDef SimpleType_methods[] = { @@ -1721,6 +1852,23 @@ } +static PyCArgObject * +CFuncPtrType_paramfunc(CDataObject *self) +{ + PyCArgObject *parg; + + parg = new_CArgObject(); + if (parg == NULL) + return NULL; + + parg->tag = 'P'; + parg->pffi_type = &ffi_type_pointer; + Py_INCREF(self); + parg->obj = (PyObject *)self; + parg->value.p = *(void **)self->b_ptr; + return parg; +} + static PyObject * CFuncPtrType_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { @@ -1732,6 +1880,8 @@ if (!stgdict) return NULL; + stgdict->paramfunc = CFuncPtrType_paramfunc; + /* create the new instance (which is a class, since we are a metatype!) */ result = (PyTypeObject *)PyType_Type.tp_new(type, args, kwds); @@ -1921,6 +2071,7 @@ CData_clear(CDataObject *self) { StgDictObject *dict = PyObject_stgdict((PyObject *)self); + assert(dict); /* Cannot be NULL for CDataObject instances */ Py_CLEAR(self->b_objects); if ((self->b_needsfree) && ((size_t)dict->size > sizeof(self->b_value))) @@ -2218,7 +2369,9 @@ StgDictObject *p1, *p2; PyObject *keep; p1 = PyObject_stgdict(value); + assert(p1); /* Cannot be NULL for array instances */ p2 = PyType_stgdict(type); + assert(p2); /* Cannot be NULL for pointer types */ if (p1->proto != p2->proto) { PyErr_Format(PyExc_TypeError, @@ -2308,23 +2461,6 @@ CFuncPtr_Type */ -static PyObject * -CFuncPtr_as_parameter(CDataObject *self) -{ - PyCArgObject *parg; - - parg = new_CArgObject(); - if (parg == NULL) - return NULL; - - parg->tag = 'P'; - parg->pffi_type = &ffi_type_pointer; - Py_INCREF(self); - parg->obj = (PyObject *)self; - parg->value.p = *(void **)self->b_ptr; - return (PyObject *)parg; -} - static int CFuncPtr_set_errcheck(CFuncPtrObject *self, PyObject *ob) { @@ -2384,7 +2520,7 @@ return self->restype; } dict = PyObject_stgdict((PyObject *)self); - assert(dict); + assert(dict); /* Cannot be NULL for CFuncPtrObject instances */ if (dict->restype) { Py_INCREF(dict->restype); return dict->restype; @@ -2426,7 +2562,7 @@ return self->argtypes; } dict = PyObject_stgdict((PyObject *)self); - assert(dict); + assert(dict); /* Cannot be NULL for CFuncPtrObject instances */ if (dict->argtypes) { Py_INCREF(dict->argtypes); return dict->argtypes; @@ -2444,9 +2580,6 @@ { "argtypes", (getter)CFuncPtr_get_argtypes, (setter)CFuncPtr_set_argtypes, "specify the argument types", NULL }, - { "_as_parameter_", (getter)CFuncPtr_as_parameter, NULL, - "return a magic value so that this can be converted to a C parameter (readonly)", - NULL }, { NULL, NULL } }; @@ -2475,6 +2608,8 @@ where n is 0, 4, 8, 12, ..., 128 */ mangled_name = alloca(strlen(name) + 1 + 1 + 1 + 3); /* \0 _ @ %d */ + if (!mangled_name) + return NULL; for (i = 0; i < 32; ++i) { sprintf(mangled_name, "_%s@%d", name, i*4); address = (PPROC)GetProcAddress(handle, mangled_name); @@ -2520,8 +2655,12 @@ _validate_paramflags(PyTypeObject *type, PyObject *paramflags) { int i, len; - StgDictObject *dict = PyType_stgdict((PyObject *)type); - PyObject *argtypes = dict->argtypes; + StgDictObject *dict; + PyObject *argtypes; + + dict = PyType_stgdict((PyObject *)type); + assert(dict); /* Cannot be NULL. 'type' is a CFuncPtr type. */ + argtypes = dict->argtypes; if (paramflags == NULL || dict->argtypes == NULL) return 1; @@ -2744,6 +2883,8 @@ if (ptr == NULL) return NULL; ob = (CDataObject *)GenericCData_new(type, args, kwds); + if (ob == NULL) + return NULL; *(void **)ob->b_ptr = ptr; return (PyObject *)ob; } @@ -2791,6 +2932,8 @@ return NULL; self = (CFuncPtrObject *)GenericCData_new(type, args, kwds); + if (self == NULL) + return NULL; Py_INCREF(callable); self->callable = callable; @@ -2987,6 +3130,13 @@ } ob = PyTuple_GET_ITEM(argtypes, i); dict = PyType_stgdict(ob); + if (dict == NULL) { + /* Cannot happen: _validate_paramflags() + would not accept such an object */ + PyErr_Format(PyExc_RuntimeError, + "NULL stgdict unexpected"); + goto error; + } if (PyString_Check(dict->proto)) { PyErr_Format( PyExc_TypeError, @@ -3129,7 +3279,7 @@ int outmask; unsigned int numretvals; - assert(dict); /* if not, it's a bug */ + assert(dict); /* Cannot be NULL for CFuncPtrObject instances */ restype = self->restype ? self->restype : dict->restype; converters = self->converters ? self->converters : dict->converters; checker = self->checker ? self->checker : dict->checker; @@ -3349,30 +3499,6 @@ return -1; } -static PyObject * -Struct_as_parameter(CDataObject *self) -{ - PyCArgObject *parg; - StgDictObject *stgdict; - - parg = new_CArgObject(); - if (parg == NULL) - return NULL; - - parg->tag = 'V'; - stgdict = PyObject_stgdict((PyObject *)self); - parg->pffi_type = &stgdict->ffi_type_pointer; - /* For structure parameters (by value), parg->value doesn't contain the structure - data itself, instead parg->value.p *points* to the structure's data - See also _ctypes.c, function _call_function_pointer(). - */ - parg->value.p = self->b_ptr; - parg->size = self->b_size; - Py_INCREF(self); - parg->obj = (PyObject *)self; - return (PyObject *)parg; -} - static int Struct_init(PyObject *self, PyObject *args, PyObject *kwds) { @@ -3447,13 +3573,6 @@ return 0; } -static PyGetSetDef Struct_getsets[] = { - { "_as_parameter_", (getter)Struct_as_parameter, NULL, - "return a magic value so that this can be converted to a C parameter (readonly)", - NULL }, - { NULL, NULL } -}; - static PyTypeObject Struct_Type = { PyObject_HEAD_INIT(NULL) 0, @@ -3485,7 +3604,7 @@ 0, /* tp_iternext */ 0, /* tp_methods */ 0, /* tp_members */ - Struct_getsets, /* tp_getset */ + 0, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ 0, /* tp_descr_get */ @@ -3528,7 +3647,7 @@ 0, /* tp_iternext */ 0, /* tp_methods */ 0, /* tp_members */ - Struct_getsets, /* tp_getset */ + 0, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ 0, /* tp_descr_get */ @@ -3581,7 +3700,7 @@ } stgdict = PyObject_stgdict((PyObject *)self); - assert(stgdict); + assert(stgdict); /* Cannot be NULL for array instances */ /* Would it be clearer if we got the item size from stgdict->proto's stgdict? */ @@ -3612,8 +3731,11 @@ len = ihigh - ilow; stgdict = PyObject_stgdict((PyObject *)self); + assert(stgdict); /* Cannot be NULL for array object instances */ proto = stgdict->proto; itemdict = PyType_stgdict(proto); + assert(itemdict); /* proto is the item type of the array, a ctypes + type, so this cannot be NULL */ if (itemdict->getfunc == getentry("c")->getfunc) { char *ptr = (char *)self->b_ptr; return PyString_FromStringAndSize(ptr + ilow, len); @@ -3650,6 +3772,7 @@ } stgdict = PyObject_stgdict((PyObject *)self); + assert(stgdict); /* Cannot be NULL for array object instances */ if (index < 0 || index >= stgdict->length) { PyErr_SetString(PyExc_IndexError, "invalid index"); @@ -3726,26 +3849,6 @@ 0, /* sq_inplace_repeat; */ }; -static PyObject * -Array_as_parameter(CDataObject *self) -{ - PyCArgObject *p = new_CArgObject(); - if (p == NULL) - return NULL; - p->tag = 'P'; - p->pffi_type = &ffi_type_pointer; - p->value.p = (char *)self->b_ptr; - Py_INCREF(self); - p->obj = (PyObject *)self; - return (PyObject *)p; -} - -static PyGetSetDef Array_getsets[] = { - { "_as_parameter_", (getter)Array_as_parameter, - (setter)NULL, "convert to a parameter", NULL }, - { NULL }, -}; - PyTypeObject Array_Type = { PyObject_HEAD_INIT(NULL) 0, @@ -3777,7 +3880,7 @@ 0, /* tp_iternext */ 0, /* tp_methods */ 0, /* tp_members */ - Array_getsets, /* tp_getset */ + 0, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ 0, /* tp_descr_get */ @@ -3861,6 +3964,7 @@ PyObject *result; StgDictObject *dict = PyObject_stgdict((PyObject *)self); + assert(dict); /* Cannot be NULL for CDataObject instances */ assert(dict->setfunc); result = dict->setfunc(self->b_ptr, value, dict->size); if (!result) @@ -3886,40 +3990,14 @@ { StgDictObject *dict; dict = PyObject_stgdict((PyObject *)self); + assert(dict); /* Cannot be NULL for CDataObject instances */ assert(dict->getfunc); - dict = PyObject_stgdict((PyObject *)self); return dict->getfunc(self->b_ptr, self->b_size); } -static PyObject * -Simple_as_parameter(CDataObject *self) -{ - StgDictObject *dict = PyObject_stgdict((PyObject *)self); - char *fmt = PyString_AsString(dict->proto); - PyCArgObject *parg; - struct fielddesc *fd; - - fd = getentry(fmt); - assert(fd); - - parg = new_CArgObject(); - if (parg == NULL) - return NULL; - - parg->tag = fmt[0]; - parg->pffi_type = fd->pffi_type; - Py_INCREF(self); - parg->obj = (PyObject *)self; - memcpy(&parg->value, self->b_ptr, self->b_size); - return (PyObject *)parg; -} - static PyGetSetDef Simple_getsets[] = { { "value", (getter)Simple_get_value, (setter)Simple_set_value, "current value", NULL }, - { "_as_parameter_", (getter)Simple_as_parameter, NULL, - "return a magic value so that this can be converted to a C parameter (readonly)", - NULL }, { NULL, NULL } }; @@ -4085,12 +4163,14 @@ } stgdict = PyObject_stgdict((PyObject *)self); - assert(stgdict); - assert(stgdict->proto); + assert(stgdict); /* Cannot be NULL for pointer object instances */ proto = stgdict->proto; - /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ + assert(proto); itemdict = PyType_stgdict(proto); + assert(itemdict); /* proto is the item type of the pointer, a ctypes + type, so this cannot be NULL */ + size = itemdict->size; offset = index * itemdict->size; @@ -4120,12 +4200,15 @@ } stgdict = PyObject_stgdict((PyObject *)self); - assert(stgdict); - assert(stgdict->proto); + assert(stgdict); /* Cannot be NULL fr pointer instances */ proto = stgdict->proto; - /* XXXXXX MAKE SURE PROTO IS NOT NULL! */ + assert(proto); + itemdict = PyType_stgdict(proto); + assert(itemdict); /* Cannot be NULL because the itemtype of a pointer + is always a ctypes type */ + size = itemdict->size; offset = index * itemdict->size; @@ -4145,7 +4228,7 @@ } stgdict = PyObject_stgdict((PyObject *)self); - assert(stgdict); + assert(stgdict); /* Cannot be NULL fr pointer instances */ return CData_FromBaseObj(stgdict->proto, (PyObject *)self, 0, *(void **)self->b_ptr); @@ -4164,7 +4247,7 @@ return -1; } stgdict = PyObject_stgdict((PyObject *)self); - /* should have been catched in Pointer_new() */ + assert(stgdict); /* Cannot be NULL fr pointer instances */ assert(stgdict->proto); if (!CDataObject_Check(value) || 0 == PyObject_IsInstance(value, stgdict->proto)) { @@ -4193,30 +4276,10 @@ return KeepRef(self, 0, keep); } -static PyObject * -Pointer_as_parameter(CDataObject *self) -{ - PyCArgObject *parg; - - parg = new_CArgObject(); - if (parg == NULL) - return NULL; - - parg->tag = 'P'; - parg->pffi_type = &ffi_type_pointer; - Py_INCREF(self); - parg->obj = (PyObject *)self; - parg->value.p = *(void **)self->b_ptr; - return (PyObject *)parg; -} - static PyGetSetDef Pointer_getsets[] = { { "contents", (getter)Pointer_get_contents, (setter)Pointer_set_contents, "the object this pointer points to (read-write)", NULL }, - { "_as_parameter_", (getter)Pointer_as_parameter, NULL, - "return a magic value so that this can be converted to a C parameter (readonly)", - NULL }, { NULL, NULL } }; @@ -4260,8 +4323,11 @@ len = ihigh - ilow; stgdict = PyObject_stgdict((PyObject *)self); + assert(stgdict); /* Cannot be NULL fr pointer instances */ proto = stgdict->proto; + assert(proto); itemdict = PyType_stgdict(proto); + assert(itemdict); if (itemdict->getfunc == getentry("c")->getfunc) { char *ptr = *(char **)self->b_ptr; return PyString_FromStringAndSize(ptr + ilow, len); @@ -4676,7 +4742,7 @@ #endif PyModule_AddObject(m, "FUNCFLAG_CDECL", PyInt_FromLong(FUNCFLAG_CDECL)); PyModule_AddObject(m, "FUNCFLAG_PYTHONAPI", PyInt_FromLong(FUNCFLAG_PYTHONAPI)); - PyModule_AddStringConstant(m, "__version__", "1.0.0"); + PyModule_AddStringConstant(m, "__version__", "1.0.1"); PyModule_AddObject(m, "_memmove_addr", PyLong_FromVoidPtr(memmove)); PyModule_AddObject(m, "_memset_addr", PyLong_FromVoidPtr(memset)); Modified: python/branches/p3yk/Modules/_ctypes/_ctypes_test.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/_ctypes_test.c (original) +++ python/branches/p3yk/Modules/_ctypes/_ctypes_test.c Mon Aug 21 21:07:27 2006 @@ -197,18 +197,6 @@ #endif -EXPORT(int) _testfunc_ppp(char ***p) -{ - static char message[] = "Hello, World"; - if (p) { - *p = (char **)malloc(sizeof(char *)); - printf("malloc returned %p\n", *p); - **p = message; - return 1; - } - return 0; -} - typedef struct { char *name; char *value; Modified: python/branches/p3yk/Modules/_ctypes/callbacks.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/callbacks.c (original) +++ python/branches/p3yk/Modules/_ctypes/callbacks.c Mon Aug 21 21:07:27 2006 @@ -205,7 +205,7 @@ result = PyObject_CallObject(callable, arglist); CHECK("'calling callback function'", result); - if ((restype != &ffi_type_void) && result && result != Py_None) { + if ((restype != &ffi_type_void) && result) { PyObject *keep; assert(setfunc); #ifdef WORDS_BIGENDIAN @@ -225,13 +225,13 @@ itself knows how to manage the refcount of these objects. */ if (keep == NULL) /* Could not convert callback result. */ - PyErr_WriteUnraisable(Py_None); + PyErr_WriteUnraisable(callable); else if (keep == Py_None) /* Nothing to keep */ Py_DECREF(keep); else if (setfunc != getentry("O")->setfunc) { if (-1 == PyErr_Warn(PyExc_RuntimeWarning, "memory leak in callback function.")) - PyErr_WriteUnraisable(Py_None); + PyErr_WriteUnraisable(callable); } } Py_XDECREF(result); Modified: python/branches/p3yk/Modules/_ctypes/callproc.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/callproc.c (original) +++ python/branches/p3yk/Modules/_ctypes/callproc.c Mon Aug 21 21:07:27 2006 @@ -465,7 +465,21 @@ */ static int ConvParam(PyObject *obj, int index, struct argument *pa) { + StgDictObject *dict; pa->keep = NULL; /* so we cannot forget it later */ + + dict = PyObject_stgdict(obj); + if (dict) { + PyCArgObject *carg; + assert(dict->paramfunc); + /* If it has an stgdict, it is a CDataObject */ + carg = dict->paramfunc((CDataObject *)obj); + pa->ffi_type = carg->pffi_type; + memcpy(&pa->value, &carg->value, sizeof(pa->value)); + pa->keep = (PyObject *)carg; + return 0; + } + if (PyCArg_CheckExact(obj)) { PyCArgObject *carg = (PyCArgObject *)obj; pa->ffi_type = carg->pffi_type; @@ -548,25 +562,12 @@ as parameters (they have to expose the '_as_parameter_' attribute) */ - if (arg == 0) { - PyErr_Format(PyExc_TypeError, - "Don't know how to convert parameter %d", index); - return -1; + if (arg) { + int result; + result = ConvParam(arg, index, pa); + Py_DECREF(arg); + return result; } - if (PyCArg_CheckExact(arg)) { - PyCArgObject *carg = (PyCArgObject *)arg; - pa->ffi_type = carg->pffi_type; - memcpy(&pa->value, &carg->value, sizeof(pa->value)); - pa->keep = arg; - return 0; - } - if (PyInt_Check(arg)) { - pa->ffi_type = &ffi_type_sint; - pa->value.i = PyInt_AS_LONG(arg); - pa->keep = arg; - return 0; - } - Py_DECREF(arg); PyErr_Format(PyExc_TypeError, "Don't know how to convert parameter %d", index); return -1; @@ -915,6 +916,10 @@ #endif args = (struct argument *)alloca(sizeof(struct argument) * argcount); + if (!args) { + PyErr_NoMemory(); + return NULL; + } memset(args, 0, sizeof(struct argument) * argcount); argtype_count = argtypes ? PyTuple_GET_SIZE(argtypes) : 0; #ifdef MS_WIN32 @@ -968,6 +973,10 @@ avalues = (void **)alloca(sizeof(void *) * argcount); atypes = (ffi_type **)alloca(sizeof(ffi_type *) * argcount); + if (!resbuf || !avalues || !atypes) { + PyErr_NoMemory(); + goto cleanup; + } for (i = 0; i < argcount; ++i) { atypes[i] = args[i].ffi_type; if (atypes[i]->type == FFI_TYPE_STRUCT) @@ -1068,6 +1077,11 @@ return NULL; #ifdef _UNICODE name = alloca((PyString_Size(nameobj) + 1) * sizeof(WCHAR)); + if (!name) { + PyErr_NoMemory(); + return NULL; + } + { int r; char *aname = PyString_AsString(nameobj); Modified: python/branches/p3yk/Modules/_ctypes/cfield.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/cfield.c (original) +++ python/branches/p3yk/Modules/_ctypes/cfield.c Mon Aug 21 21:07:27 2006 @@ -105,6 +105,12 @@ StgDictObject *idict; if (adict && adict->proto) { idict = PyType_stgdict(adict->proto); + if (!idict) { + PyErr_SetString(PyExc_TypeError, + "has no _stginfo_"); + Py_DECREF(self); + return NULL; + } if (idict->getfunc == getentry("c")->getfunc) { struct fielddesc *fd = getentry("s"); getfunc = fd->getfunc; @@ -1094,7 +1100,7 @@ if (!PyErr_Occurred()) /* Set an error if not yet set */ PyErr_SetString(PyExc_ValueError, - "PyObject is NULL?"); + "PyObject is NULL"); return NULL; } Py_INCREF(ob); Modified: python/branches/p3yk/Modules/_ctypes/ctypes.h ============================================================================== --- python/branches/p3yk/Modules/_ctypes/ctypes.h (original) +++ python/branches/p3yk/Modules/_ctypes/ctypes.h Mon Aug 21 21:07:27 2006 @@ -23,9 +23,11 @@ #define PY_LONG_LONG LONG_LONG #endif +typedef struct tagPyCArgObject PyCArgObject; typedef struct tagCDataObject CDataObject; typedef PyObject *(* GETFUNC)(void *, unsigned size); typedef PyObject *(* SETFUNC)(void *, PyObject *value, unsigned size); +typedef PyCArgObject *(* PARAMFUNC)(CDataObject *obj); /* A default buffer in CDataObject, which can be used for small C types. If this buffer is too small, PyMem_Malloc will be called to create a larger one, @@ -205,6 +207,7 @@ PyObject *proto; /* Only for Pointer/ArrayObject */ SETFUNC setfunc; /* Only for simple objects */ GETFUNC getfunc; /* Only for simple objects */ + PARAMFUNC paramfunc; /* Following fields only used by CFuncPtrType_Type instances */ PyObject *argtypes; /* tuple of CDataObjects */ @@ -283,7 +286,7 @@ #define DICTFLAG_FINAL 0x1000 -typedef struct { +struct tagPyCArgObject { PyObject_HEAD ffi_type *pffi_type; char tag; @@ -302,7 +305,7 @@ } value; PyObject *obj; int size; /* for the 'V' tag */ -} PyCArgObject; +}; extern PyTypeObject PyCArg_Type; extern PyCArgObject *new_CArgObject(void); Modified: python/branches/p3yk/Modules/_ctypes/libffi/configure ============================================================================== --- python/branches/p3yk/Modules/_ctypes/libffi/configure (original) +++ python/branches/p3yk/Modules/_ctypes/libffi/configure Mon Aug 21 21:07:27 2006 @@ -3470,6 +3470,7 @@ TARGETDIR="unknown" case "$host" in +x86_64-*-openbsd*) TARGET=X86_64; TARGETDIR=x86;; mips*-*-openbsd*) TARGET=MIPS; TARGETDIR=mips;; sparc-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; sparc64-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; Modified: python/branches/p3yk/Modules/_ctypes/libffi/configure.ac ============================================================================== --- python/branches/p3yk/Modules/_ctypes/libffi/configure.ac (original) +++ python/branches/p3yk/Modules/_ctypes/libffi/configure.ac Mon Aug 21 21:07:27 2006 @@ -21,6 +21,7 @@ TARGETDIR="unknown" case "$host" in +x86_64-*-openbsd*) TARGET=X86_64; TARGETDIR=x86;; mips*-*-openbsd*) TARGET=MIPS; TARGETDIR=mips;; sparc-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; sparc64-*-openbsd*) TARGET=SPARC; TARGETDIR=sparc;; Modified: python/branches/p3yk/Modules/_ctypes/libffi/fficonfig.py.in ============================================================================== --- python/branches/p3yk/Modules/_ctypes/libffi/fficonfig.py.in (original) +++ python/branches/p3yk/Modules/_ctypes/libffi/fficonfig.py.in Mon Aug 21 21:07:27 2006 @@ -43,6 +43,3 @@ ffi_sources = [os.path.join('@srcdir@', f) for f in ffi_sources] ffi_cflags = '@CFLAGS@' -# I think this may no longer be needed: -if sys.platform == "openbsd3": - ffi_cflags += " -fno-stack-protector" Modified: python/branches/p3yk/Modules/_ctypes/stgdict.c ============================================================================== --- python/branches/p3yk/Modules/_ctypes/stgdict.c (original) +++ python/branches/p3yk/Modules/_ctypes/stgdict.c Mon Aug 21 21:07:27 2006 @@ -204,12 +204,12 @@ continue; } new_descr = (CFieldObject *)PyObject_CallObject((PyObject *)&CField_Type, NULL); - assert(new_descr->ob_type == &CField_Type); if (new_descr == NULL) { Py_DECREF(fdescr); Py_DECREF(fieldlist); return -1; } + assert(new_descr->ob_type == &CField_Type); new_descr->size = fdescr->size; new_descr->offset = fdescr->offset + offset; new_descr->index = fdescr->index + index; Modified: python/branches/p3yk/Modules/_cursesmodule.c ============================================================================== --- python/branches/p3yk/Modules/_cursesmodule.c (original) +++ python/branches/p3yk/Modules/_cursesmodule.c Mon Aug 21 21:07:27 2006 @@ -1784,7 +1784,6 @@ PyCurses_InitScr(PyObject *self) { WINDOW *win; - PyObject *nlines, *cols; if (initialised == TRUE) { wrefresh(stdscr); @@ -1803,7 +1802,12 @@ /* This was moved from initcurses() because it core dumped on SGI, where they're not defined until you've called initscr() */ #define SetDictInt(string,ch) \ - PyDict_SetItemString(ModDict,string,PyInt_FromLong((long) (ch))); + do { \ + PyObject *o = PyInt_FromLong((long) (ch)); \ + if (o && PyDict_SetItemString(ModDict, string, o) == 0) { \ + Py_DECREF(o); \ + } \ + } while (0) /* Here are some graphic symbols you can use */ SetDictInt("ACS_ULCORNER", (ACS_ULCORNER)); @@ -1872,12 +1876,8 @@ SetDictInt("ACS_STERLING", (ACS_STERLING)); #endif - nlines = PyInt_FromLong((long) LINES); - PyDict_SetItemString(ModDict, "LINES", nlines); - Py_DECREF(nlines); - cols = PyInt_FromLong((long) COLS); - PyDict_SetItemString(ModDict, "COLS", cols); - Py_DECREF(cols); + SetDictInt("LINES", LINES); + SetDictInt("COLS", COLS); return (PyObject *)PyCursesWindow_New(win); } @@ -2554,6 +2554,8 @@ /* Add some symbolic constants to the module */ d = PyModule_GetDict(m); + if (d == NULL) + return; ModDict = d; /* For PyCurses_InitScr to use later */ /* Add a CObject for the C API */ @@ -2667,6 +2669,10 @@ if (strncmp(key_n,"KEY_F(",6)==0) { char *p1, *p2; key_n2 = malloc(strlen(key_n)+1); + if (!key_n2) { + PyErr_NoMemory(); + break; + } p1 = key_n; p2 = key_n2; while (*p1) { @@ -2679,7 +2685,7 @@ *p2 = (char)0; } else key_n2 = key_n; - PyDict_SetItemString(d,key_n2,PyInt_FromLong((long) key)); + SetDictInt(key_n2,key); if (key_n2 != key_n) free(key_n2); } Modified: python/branches/p3yk/Modules/_elementtree.c ============================================================================== --- python/branches/p3yk/Modules/_elementtree.c (original) +++ python/branches/p3yk/Modules/_elementtree.c Mon Aug 21 21:07:27 2006 @@ -48,7 +48,7 @@ #include "Python.h" -#define VERSION "1.0.6-snapshot" +#define VERSION "1.0.6" /* -------------------------------------------------------------------- */ /* configuration */ @@ -809,7 +809,7 @@ PyObject* text = element_get_text(item); if (text == Py_None) return PyString_FromString(""); - Py_INCREF(text); + Py_XINCREF(text); return text; } } @@ -1599,6 +1599,10 @@ treebuilder_handle_data(TreeBuilderObject* self, PyObject* data) { if (!self->data) { + if (self->last == (ElementObject*) Py_None) { + /* ignore calls to data before the first call to start */ + Py_RETURN_NONE; + } /* store the first item as is */ Py_INCREF(data); self->data = data; } else { Modified: python/branches/p3yk/Modules/_hotshot.c ============================================================================== --- python/branches/p3yk/Modules/_hotshot.c (original) +++ python/branches/p3yk/Modules/_hotshot.c Mon Aug 21 21:07:27 2006 @@ -313,6 +313,11 @@ return err; buf = (char *)malloc(len); + if (!buf) { + PyErr_NoMemory(); + return ERR_EXCEPTION; + } + for (i=0; i < len; i++) { ch = fgetc(self->logfp); buf[i] = ch; Modified: python/branches/p3yk/Modules/_sre.c ============================================================================== --- python/branches/p3yk/Modules/_sre.c (original) +++ python/branches/p3yk/Modules/_sre.c Mon Aug 21 21:07:27 2006 @@ -1166,6 +1166,9 @@ /* install new repeat context */ ctx->u.rep = (SRE_REPEAT*) PyObject_MALLOC(sizeof(*ctx->u.rep)); + /* XXX(nnorwitz): anything else we need to do on error? */ + if (!ctx->u.rep) + RETURN_FAILURE; ctx->u.rep->count = -1; ctx->u.rep->pattern = ctx->pattern; ctx->u.rep->prev = state->repeat; Modified: python/branches/p3yk/Modules/_tkinter.c ============================================================================== --- python/branches/p3yk/Modules/_tkinter.c (original) +++ python/branches/p3yk/Modules/_tkinter.c Mon Aug 21 21:07:27 2006 @@ -2493,8 +2493,10 @@ } v = Tktt_New(func); - v->token = Tcl_CreateTimerHandler(milliseconds, TimerHandler, - (ClientData)v); + if (v) { + v->token = Tcl_CreateTimerHandler(milliseconds, TimerHandler, + (ClientData)v); + } return (PyObject *) v; } Modified: python/branches/p3yk/Modules/arraymodule.c ============================================================================== --- python/branches/p3yk/Modules/arraymodule.c (original) +++ python/branches/p3yk/Modules/arraymodule.c Mon Aug 21 21:07:27 2006 @@ -702,6 +702,8 @@ /* Special case "a[i:j] = a" -- copy b first */ int ret; v = array_slice(b, 0, n); + if (!v) + return -1; ret = array_ass_slice(a, ilow, ihigh, v); Py_DECREF(v); return ret; @@ -1573,9 +1575,8 @@ static PyObject* array_subscr(arrayobject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i==-1 && PyErr_Occurred()) { return NULL; } @@ -1623,9 +1624,8 @@ static int array_ass_subscr(arrayobject* self, PyObject* item, PyObject* value) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i==-1 && PyErr_Occurred()) return -1; if (i < 0) @@ -1706,6 +1706,8 @@ if (self == av) { value = array_slice(av, 0, av->ob_size); av = (arrayobject*)value; + if (!av) + return -1; } else { Py_INCREF(value); Modified: python/branches/p3yk/Modules/bz2module.c ============================================================================== --- python/branches/p3yk/Modules/bz2module.c (original) +++ python/branches/p3yk/Modules/bz2module.c Mon Aug 21 21:07:27 2006 @@ -805,12 +805,12 @@ case MODE_CLOSED: PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); - goto cleanup;; + goto cleanup; default: PyErr_SetString(PyExc_IOError, "file is not ready for writing"); - goto cleanup;; + goto cleanup; } self->f_softspace = 0; @@ -854,6 +854,21 @@ int bzerror; ACQUIRE_LOCK(self); + switch (self->mode) { + case MODE_WRITE: + break; + + case MODE_CLOSED: + PyErr_SetString(PyExc_ValueError, + "I/O operation on closed file"); + goto error; + + default: + PyErr_SetString(PyExc_IOError, + "file is not ready for writing"); + goto error; + } + islist = PyList_Check(seq); if (!islist) { iter = PyObject_GetIter(seq); Modified: python/branches/p3yk/Modules/collectionsmodule.c ============================================================================== --- python/branches/p3yk/Modules/collectionsmodule.c (original) +++ python/branches/p3yk/Modules/collectionsmodule.c Mon Aug 21 21:07:27 2006 @@ -211,6 +211,7 @@ PyErr_SetString(PyExc_IndexError, "pop from an empty deque"); return NULL; } + assert(deque->leftblock != NULL); item = deque->leftblock->data[deque->leftindex]; deque->leftindex++; deque->len--; @@ -226,7 +227,6 @@ } else { assert(deque->leftblock != deque->rightblock); prevblock = deque->leftblock->rightlink; - assert(deque->leftblock != NULL); PyMem_Free(deque->leftblock); assert(prevblock != NULL); prevblock->leftlink = NULL; Modified: python/branches/p3yk/Modules/expat/xmlparse.c ============================================================================== --- python/branches/p3yk/Modules/expat/xmlparse.c (original) +++ python/branches/p3yk/Modules/expat/xmlparse.c Mon Aug 21 21:07:27 2006 @@ -2795,6 +2795,8 @@ unsigned long uriHash = 0; ((XML_Char *)s)[-1] = 0; /* clear flag */ id = (ATTRIBUTE_ID *)lookup(&dtd->attributeIds, s, 0); + if (!id) + return XML_ERROR_NO_MEMORY; b = id->prefix->binding; if (!b) return XML_ERROR_UNBOUND_PREFIX; @@ -5340,6 +5342,8 @@ return NULL; id->prefix = (PREFIX *)lookup(&dtd->prefixes, poolStart(&dtd->pool), sizeof(PREFIX)); + if (!id->prefix) + return NULL; if (id->prefix->name == poolStart(&dtd->pool)) poolFinish(&dtd->pool); else Modified: python/branches/p3yk/Modules/mmapmodule.c ============================================================================== --- python/branches/p3yk/Modules/mmapmodule.c (original) +++ python/branches/p3yk/Modules/mmapmodule.c Mon Aug 21 21:07:27 2006 @@ -815,31 +815,19 @@ static Py_ssize_t _GetMapSize(PyObject *o) { - PyNumberMethods *nb = o->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(o); + if (PyIndex_Check(o)) { + Py_ssize_t i = PyNumber_AsSsize_t(o, PyExc_OverflowError); if (i==-1 && PyErr_Occurred()) return -1; - if (i < 0) - goto onnegoverflow; - if (i==PY_SSIZE_T_MAX) - goto onposoverflow; + if (i < 0) { + PyErr_SetString(PyExc_OverflowError, + "memory mapped size must be positive"); + return -1; + } return i; } - else { - PyErr_SetString(PyExc_TypeError, - "map size must be an integral value"); - return -1; - } - - onnegoverflow: - PyErr_SetString(PyExc_OverflowError, - "memory mapped size must be positive"); - return -1; - onposoverflow: - PyErr_SetString(PyExc_OverflowError, - "memory mapped size is too large (limited by C int)"); + PyErr_SetString(PyExc_TypeError, "map size must be an integral value"); return -1; } @@ -1138,6 +1126,15 @@ {NULL, NULL} /* Sentinel */ }; +static void +setint(PyObject *d, const char *name, long value) +{ + PyObject *o = PyInt_FromLong(value); + if (o && PyDict_SetItemString(d, name, o) == 0) { + Py_DECREF(o); + } +} + PyMODINIT_FUNC initmmap(void) { @@ -1150,47 +1147,40 @@ if (module == NULL) return; dict = PyModule_GetDict(module); + if (!dict) + return; mmap_module_error = PyExc_EnvironmentError; - Py_INCREF(mmap_module_error); PyDict_SetItemString(dict, "error", mmap_module_error); #ifdef PROT_EXEC - PyDict_SetItemString(dict, "PROT_EXEC", PyInt_FromLong(PROT_EXEC) ); + setint(dict, "PROT_EXEC", PROT_EXEC); #endif #ifdef PROT_READ - PyDict_SetItemString(dict, "PROT_READ", PyInt_FromLong(PROT_READ) ); + setint(dict, "PROT_READ", PROT_READ); #endif #ifdef PROT_WRITE - PyDict_SetItemString(dict, "PROT_WRITE", PyInt_FromLong(PROT_WRITE) ); + setint(dict, "PROT_WRITE", PROT_WRITE); #endif #ifdef MAP_SHARED - PyDict_SetItemString(dict, "MAP_SHARED", PyInt_FromLong(MAP_SHARED) ); + setint(dict, "MAP_SHARED", MAP_SHARED); #endif #ifdef MAP_PRIVATE - PyDict_SetItemString(dict, "MAP_PRIVATE", - PyInt_FromLong(MAP_PRIVATE) ); + setint(dict, "MAP_PRIVATE", MAP_PRIVATE); #endif #ifdef MAP_DENYWRITE - PyDict_SetItemString(dict, "MAP_DENYWRITE", - PyInt_FromLong(MAP_DENYWRITE) ); + setint(dict, "MAP_DENYWRITE", MAP_DENYWRITE); #endif #ifdef MAP_EXECUTABLE - PyDict_SetItemString(dict, "MAP_EXECUTABLE", - PyInt_FromLong(MAP_EXECUTABLE) ); + setint(dict, "MAP_EXECUTABLE", MAP_EXECUTABLE); #endif #ifdef MAP_ANONYMOUS - PyDict_SetItemString(dict, "MAP_ANON", PyInt_FromLong(MAP_ANONYMOUS) ); - PyDict_SetItemString(dict, "MAP_ANONYMOUS", - PyInt_FromLong(MAP_ANONYMOUS) ); + setint(dict, "MAP_ANON", MAP_ANONYMOUS); + setint(dict, "MAP_ANONYMOUS", MAP_ANONYMOUS); #endif - PyDict_SetItemString(dict, "PAGESIZE", - PyInt_FromLong((long)my_getpagesize())); - - PyDict_SetItemString(dict, "ACCESS_READ", - PyInt_FromLong(ACCESS_READ)); - PyDict_SetItemString(dict, "ACCESS_WRITE", - PyInt_FromLong(ACCESS_WRITE)); - PyDict_SetItemString(dict, "ACCESS_COPY", - PyInt_FromLong(ACCESS_COPY)); + setint(dict, "PAGESIZE", (long)my_getpagesize()); + + setint(dict, "ACCESS_READ", ACCESS_READ); + setint(dict, "ACCESS_WRITE", ACCESS_WRITE); + setint(dict, "ACCESS_COPY", ACCESS_COPY); } Modified: python/branches/p3yk/Modules/operator.c ============================================================================== --- python/branches/p3yk/Modules/operator.c (original) +++ python/branches/p3yk/Modules/operator.c Mon Aug 21 21:07:27 2006 @@ -137,15 +137,7 @@ static PyObject * op_index(PyObject *s, PyObject *a) { - Py_ssize_t i; - PyObject *a1; - if (!PyArg_UnpackTuple(a,"index", 1, 1, &a1)) - return NULL; - i = PyNumber_Index(a1); - if (i == -1 && PyErr_Occurred()) - return NULL; - else - return PyInt_FromSsize_t(i); + return PyNumber_Index(a); } static PyObject* @@ -247,7 +239,7 @@ spam1(is_, "is_(a, b) -- Same as a is b.") spam1(is_not, "is_not(a, b) -- Same as a is not b.") -spam2(index, __index__, "index(a) -- Same as a.__index__()") +spam2o(index, __index__, "index(a) -- Same as a.__index__()") spam2(add,__add__, "add(a, b) -- Same as a + b.") spam2(sub,__sub__, "sub(a, b) -- Same as a - b.") spam2(mul,__mul__, "mul(a, b) -- Same as a * b.") Modified: python/branches/p3yk/Modules/posixmodule.c ============================================================================== --- python/branches/p3yk/Modules/posixmodule.c (original) +++ python/branches/p3yk/Modules/posixmodule.c Mon Aug 21 21:07:27 2006 @@ -1121,6 +1121,8 @@ #else ival = PyInt_FromLong((long)sec); #endif + if (!ival) + return; if (_stat_float_times) { fval = PyFloat_FromDouble(sec + 1e-9*nsec); } else { Modified: python/branches/p3yk/Modules/rgbimgmodule.c ============================================================================== --- python/branches/p3yk/Modules/rgbimgmodule.c (original) +++ python/branches/p3yk/Modules/rgbimgmodule.c Mon Aug 21 21:07:27 2006 @@ -410,6 +410,11 @@ addlongimgtag(base, xsize, ysize); #endif verdat = (unsigned char *)malloc(xsize); + if (!verdat) { + Py_CLEAR(rv); + goto finally; + } + fseek(inf, 512, SEEK_SET); for (z = 0; z < zsize; z++) { lptr = base; @@ -431,10 +436,14 @@ copybw((Py_Int32 *) base, xsize * ysize); } finally: - free(starttab); - free(lengthtab); - free(rledat); - free(verdat); + if (starttab) + free(starttab); + if (lengthtab) + free(lengthtab); + if (rledat) + free(rledat); + if (verdat) + free(verdat); fclose(inf); return rv; } Modified: python/branches/p3yk/Modules/socketmodule.c ============================================================================== --- python/branches/p3yk/Modules/socketmodule.c (original) +++ python/branches/p3yk/Modules/socketmodule.c Mon Aug 21 21:07:27 2006 @@ -3041,17 +3041,20 @@ if ((addr_list = PyList_New(0)) == NULL) goto err; - for (pch = h->h_aliases; *pch != NULL; pch++) { - int status; - tmp = PyString_FromString(*pch); - if (tmp == NULL) - goto err; + /* SF #1511317: h_aliases can be NULL */ + if (h->h_aliases) { + for (pch = h->h_aliases; *pch != NULL; pch++) { + int status; + tmp = PyString_FromString(*pch); + if (tmp == NULL) + goto err; - status = PyList_Append(name_list, tmp); - Py_DECREF(tmp); + status = PyList_Append(name_list, tmp); + Py_DECREF(tmp); - if (status) - goto err; + if (status) + goto err; + } } for (pch = h->h_addr_list; *pch != NULL; pch++) { Modified: python/branches/p3yk/Modules/unicodedata.c ============================================================================== --- python/branches/p3yk/Modules/unicodedata.c (original) +++ python/branches/p3yk/Modules/unicodedata.c Mon Aug 21 21:07:27 2006 @@ -1,8 +1,8 @@ /* ------------------------------------------------------------------------ - unicodedata -- Provides access to the Unicode 3.2 data base. + unicodedata -- Provides access to the Unicode 4.1 data base. - Data was extracted from the Unicode 3.2 UnicodeData.txt file. + Data was extracted from the Unicode 4.1 UnicodeData.txt file. Written by Marc-Andre Lemburg (mal(a)lemburg.com). Modified for Python 2.0 by Fredrik Lundh (fredrik(a)pythonware.com) @@ -1078,6 +1078,7 @@ { Py_UCS4 code; Py_UNICODE str[1]; + char errbuf[256]; char* name; int namelen; @@ -1085,11 +1086,19 @@ return NULL; if (!_getcode(self, name, namelen, &code)) { + /* XXX(nnorwitz): why are we allocating for the error msg? + Why not always use snprintf? */ char fmt[] = "undefined character name '%s'"; char *buf = PyMem_MALLOC(sizeof(fmt) + namelen); - sprintf(buf, fmt, name); + if (buf) + sprintf(buf, fmt, name); + else { + buf = errbuf; + PyOS_snprintf(buf, sizeof(errbuf), fmt, name); + } PyErr_SetString(PyExc_KeyError, buf); - PyMem_FREE(buf); + if (buf != errbuf) + PyMem_FREE(buf); return NULL; } @@ -1173,11 +1182,11 @@ "This module provides access to the Unicode Character Database which\n\ defines character properties for all Unicode characters. The data in\n\ this database is based on the UnicodeData.txt file version\n\ -3.2.0 which is publically available from ftp://ftp.unicode.org/.\n\ +4.1.0 which is publically available from ftp://ftp.unicode.org/.\n\ \n\ The module uses the same names and symbols as defined by the\n\ -UnicodeData File Format 3.2.0 (see\n\ -http://www.unicode.org/Public/3.2-Update/UnicodeData-3.2.0.html)."); +UnicodeData File Format 4.1.0 (see\n\ +http://www.unicode.org/Public/4.1.0/ucd/UCD.html)."); PyMODINIT_FUNC initunicodedata(void) Modified: python/branches/p3yk/Modules/zipimport.c ============================================================================== --- python/branches/p3yk/Modules/zipimport.c (original) +++ python/branches/p3yk/Modules/zipimport.c Mon Aug 21 21:07:27 2006 @@ -950,6 +950,9 @@ char *buf, *q, *p = PyString_AsString(source); PyObject *fixed_source; + if (!p) + return NULL; + /* one char extra for trailing \n and one for terminating \0 */ buf = (char *)PyMem_Malloc(PyString_Size(source) + 2); if (buf == NULL) { Modified: python/branches/p3yk/Objects/abstract.c ============================================================================== --- python/branches/p3yk/Objects/abstract.c (original) +++ python/branches/p3yk/Objects/abstract.c Mon Aug 21 21:07:27 2006 @@ -6,6 +6,7 @@ #include "longintrepr.h" + /* Shorthands to return certain errors */ static PyObject * @@ -117,9 +118,9 @@ return m->mp_subscript(o, key); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return NULL; return PySequence_GetItem(o, key_value); @@ -146,9 +147,9 @@ return m->mp_ass_subscript(o, key, value); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return -1; return PySequence_SetItem(o, key_value, value); @@ -178,9 +179,9 @@ return m->mp_ass_subscript(o, key, (PyObject*)NULL); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return -1; return PySequence_DelItem(o, key_value); @@ -530,9 +531,8 @@ sequence_repeat(ssizeargfunc repeatfunc, PyObject *seq, PyObject *n) { Py_ssize_t count; - PyNumberMethods *nb = n->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - count = nb->nb_index(n); + if (PyIndex_Check(n)) { + count = PyNumber_AsSsize_t(n, PyExc_OverflowError); if (count == -1 && PyErr_Occurred()) return NULL; } @@ -809,23 +809,87 @@ return x; } -/* Return a Py_ssize_t integer from the object item */ -Py_ssize_t +/* Return a Python Int or Long from the object item + Raise TypeError if the result is not an int-or-long + or if the object cannot be interpreted as an index. +*/ +PyObject * PyNumber_Index(PyObject *item) { - Py_ssize_t value = -1; - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - value = nb->nb_index(item); + PyObject *result = NULL; + if (item == NULL) + return null_error(); + if (PyInt_Check(item) || PyLong_Check(item)) { + Py_INCREF(item); + return item; + } + if (PyIndex_Check(item)) { + result = item->ob_type->tp_as_number->nb_index(item); + if (result && + !PyInt_Check(result) && !PyLong_Check(result)) { + PyErr_Format(PyExc_TypeError, + "__index__ returned non-(int,long) " \ + "(type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } } else { PyErr_Format(PyExc_TypeError, "'%.200s' object cannot be interpreted " "as an index", item->ob_type->tp_name); } - return value; + return result; +} + +/* Return an error on Overflow only if err is not NULL*/ + +Py_ssize_t +PyNumber_AsSsize_t(PyObject *item, PyObject *err) +{ + Py_ssize_t result; + PyObject *runerr; + PyObject *value = PyNumber_Index(item); + if (value == NULL) + return -1; + + /* We're done if PyInt_AsSsize_t() returns without error. */ + result = PyInt_AsSsize_t(value); + if (result != -1 || !(runerr = PyErr_Occurred())) + goto finish; + + /* Error handling code -- only manage OverflowError differently */ + if (!PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError)) + goto finish; + + PyErr_Clear(); + /* If no error-handling desired then the default clipping + is sufficient. + */ + if (!err) { + assert(PyLong_Check(value)); + /* Whether or not it is less than or equal to + zero is determined by the sign of ob_size + */ + if (_PyLong_Sign(value) < 0) + result = PY_SSIZE_T_MIN; + else + result = PY_SSIZE_T_MAX; + } + else { + /* Otherwise replace the error with caller's error object. */ + PyErr_Format(err, + "cannot fit '%.200s' into an index-sized integer", + item->ob_type->tp_name); + } + + finish: + Py_DECREF(value); + return result; } + PyObject * PyNumber_Int(PyObject *o) { Modified: python/branches/p3yk/Objects/bytesobject.c ============================================================================== --- python/branches/p3yk/Objects/bytesobject.c (original) +++ python/branches/p3yk/Objects/bytesobject.c Mon Aug 21 21:07:27 2006 @@ -245,10 +245,9 @@ if (PyBytes_Check(value)) return bytes_substring(self, (PyBytesObject *)value); - ival = PyNumber_Index(value); + ival = PyNumber_AsSsize_t(value, PyExc_TypeError); if (ival == -1 && PyErr_Occurred()) return -1; - if (ival < 0 || ival >= 256) { PyErr_SetString(PyExc_ValueError, "byte must be in range(0, 256)"); return -1; @@ -366,7 +365,7 @@ if (value == NULL) return bytes_setslice(self, i, i+1, NULL); - ival = PyNumber_Index(value); + ival = PyNumber_AsSsize_t(value, PyExc_TypeError); if (ival == -1 && PyErr_Occurred()) return -1; @@ -449,7 +448,7 @@ } /* Is it an int? */ - count = PyNumber_Index(arg); + count = PyNumber_AsSsize_t(arg, PyExc_TypeError); if (count == -1 && PyErr_Occurred()) PyErr_Clear(); else { @@ -501,7 +500,7 @@ } /* Interpret it as an int (__index__) */ - value = PyNumber_Index(item); + value = PyNumber_AsSsize_t(item, PyExc_TypeError); Py_DECREF(item); if (value == -1 && PyErr_Occurred()) goto error; Modified: python/branches/p3yk/Objects/exceptions.c ============================================================================== --- python/branches/p3yk/Objects/exceptions.c (original) +++ python/branches/p3yk/Objects/exceptions.c Mon Aug 21 21:07:27 2006 @@ -1948,6 +1948,14 @@ "Base class for warnings about probable mistakes in module imports"); +/* + * UnicodeWarning extends Warning + */ +SimpleExtendsException(PyExc_Warning, UnicodeWarning, + "Base class for warnings about Unicode related problems, mostly\n" + "related to conversion problems."); + + /* Pre-computed MemoryError instance. Best to create this as early as * possible and not wait until a MemoryError is actually raised! */ @@ -2048,6 +2056,7 @@ PRE_INIT(RuntimeWarning) PRE_INIT(FutureWarning) PRE_INIT(ImportWarning) + PRE_INIT(UnicodeWarning) m = Py_InitModule4("exceptions", functions, exceptions_doc, (PyObject *)NULL, PYTHON_API_VERSION); @@ -2113,6 +2122,7 @@ POST_INIT(RuntimeWarning) POST_INIT(FutureWarning) POST_INIT(ImportWarning) + POST_INIT(UnicodeWarning) PyExc_MemoryErrorInst = BaseException_new(&_PyExc_MemoryError, NULL, NULL); if (!PyExc_MemoryErrorInst) Modified: python/branches/p3yk/Objects/intobject.c ============================================================================== --- python/branches/p3yk/Objects/intobject.c (original) +++ python/branches/p3yk/Objects/intobject.c Mon Aug 21 21:07:27 2006 @@ -193,16 +193,21 @@ PyIntObject *io; Py_ssize_t val; #endif - if (op && !PyInt_CheckExact(op) && PyLong_Check(op)) + + if (op == NULL) { + PyErr_SetString(PyExc_TypeError, "an integer is required"); + return -1; + } + + if (PyInt_Check(op)) + return PyInt_AS_LONG((PyIntObject*) op); + if (PyLong_Check(op)) return _PyLong_AsSsize_t(op); #if SIZEOF_SIZE_T == SIZEOF_LONG return PyInt_AsLong(op); #else - if (op && PyInt_Check(op)) - return PyInt_AS_LONG((PyIntObject*) op); - - if (op == NULL || (nb = op->ob_type->tp_as_number) == NULL || + if ((nb = op->ob_type->tp_as_number) == NULL || (nb->nb_int == NULL && nb->nb_long == 0)) { PyErr_SetString(PyExc_TypeError, "an integer is required"); return -1; @@ -1045,7 +1050,7 @@ int_true_divide, /* nb_true_divide */ 0, /* nb_inplace_floor_divide */ 0, /* nb_inplace_true_divide */ - PyInt_AsSsize_t, /* nb_index */ + (unaryfunc)int_int, /* nb_index */ }; PyTypeObject PyInt_Type = { Modified: python/branches/p3yk/Objects/listobject.c ============================================================================== --- python/branches/p3yk/Objects/listobject.c (original) +++ python/branches/p3yk/Objects/listobject.c Mon Aug 21 21:07:27 2006 @@ -1398,7 +1398,7 @@ PyObject *compare; PyObject **dest; int result = -1; /* guilty until proved innocent */ - Py_ssize_t min_gallop = ms->min_gallop; + Py_ssize_t min_gallop; assert(ms && pa && pb && na > 0 && nb > 0 && pa + na == pb); if (MERGE_GETMEM(ms, na) < 0) @@ -1414,6 +1414,7 @@ if (na == 1) goto CopyB; + min_gallop = ms->min_gallop; compare = ms->compare; for (;;) { Py_ssize_t acount = 0; /* # of times A won in a row */ @@ -1531,7 +1532,7 @@ int result = -1; /* guilty until proved innocent */ PyObject **basea; PyObject **baseb; - Py_ssize_t min_gallop = ms->min_gallop; + Py_ssize_t min_gallop; assert(ms && pa && pb && na > 0 && nb > 0 && pa + na == pb); if (MERGE_GETMEM(ms, nb) < 0) @@ -1550,6 +1551,7 @@ if (nb == 1) goto CopyA; + min_gallop = ms->min_gallop; compare = ms->compare; for (;;) { Py_ssize_t acount = 0; /* # of times A won in a row */ @@ -2445,9 +2447,9 @@ static PyObject * list_subscript(PyListObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i; + i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) @@ -2494,9 +2496,8 @@ static int list_ass_subscript(PyListObject* self, PyObject* item, PyObject* value) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return -1; if (i < 0) @@ -2531,6 +2532,10 @@ garbage = (PyObject**) PyMem_MALLOC(slicelength*sizeof(PyObject*)); + if (!garbage) { + PyErr_NoMemory(); + return -1; + } /* drawing pictures might help understand these for loops */ @@ -2579,9 +2584,9 @@ else { seq = PySequence_Fast(value, "must assign iterable to extended slice"); - if (!seq) - return -1; } + if (!seq) + return -1; if (PySequence_Fast_GET_SIZE(seq) != slicelength) { PyErr_Format(PyExc_ValueError, Modified: python/branches/p3yk/Objects/longobject.c ============================================================================== --- python/branches/p3yk/Objects/longobject.c (original) +++ python/branches/p3yk/Objects/longobject.c Mon Aug 21 21:07:27 2006 @@ -240,8 +240,11 @@ return -1; } -static Py_ssize_t -_long_as_ssize_t(PyObject *vv) { +/* Get a Py_ssize_t from a long int object. + Returns -1 and sets an error condition if overflow occurs. */ + +Py_ssize_t +_PyLong_AsSsize_t(PyObject *vv) { register PyLongObject *v; size_t x, prev; Py_ssize_t i; @@ -277,45 +280,7 @@ overflow: PyErr_SetString(PyExc_OverflowError, "long int too large to convert to int"); - if (sign > 0) - return PY_SSIZE_T_MAX; - else - return PY_SSIZE_T_MIN; -} - -/* Get a Py_ssize_t from a long int object. - Returns -1 and sets an error condition if overflow occurs. */ - -Py_ssize_t -_PyLong_AsSsize_t(PyObject *vv) -{ - Py_ssize_t x; - - x = _long_as_ssize_t(vv); - if (PyErr_Occurred()) return -1; - return x; -} - - -/* Get a Py_ssize_t from a long int object. - Silently reduce values larger than PY_SSIZE_T_MAX to PY_SSIZE_T_MAX, - and silently boost values less than -PY_SSIZE_T_MAX-1 to -PY_SSIZE_T_MAX-1. - On error, return -1 with an exception set. -*/ - -static Py_ssize_t -long_index(PyObject *vv) -{ - Py_ssize_t x; - - x = _long_as_ssize_t(vv); - if (PyErr_Occurred()) { - /* If overflow error, ignore the error */ - if (x != -1) { - PyErr_Clear(); - } - } - return x; + return -1; } /* Get a C unsigned long int from a long int object. @@ -3379,7 +3344,7 @@ long_true_divide, /* nb_true_divide */ 0, /* nb_inplace_floor_divide */ 0, /* nb_inplace_true_divide */ - long_index, /* nb_index */ + long_long, /* nb_index */ }; PyTypeObject PyLong_Type = { Modified: python/branches/p3yk/Objects/object.c ============================================================================== --- python/branches/p3yk/Objects/object.c (original) +++ python/branches/p3yk/Objects/object.c Mon Aug 21 21:07:27 2006 @@ -725,23 +725,6 @@ return (vv < ww) ? -1 : (vv > ww) ? 1 : 0; } -#ifdef Py_USING_UNICODE - /* Special case for Unicode */ - if (PyUnicode_Check(v) || PyUnicode_Check(w)) { - c = PyUnicode_Compare(v, w); - if (!PyErr_Occurred()) - return c; - /* TypeErrors are ignored: if Unicode coercion fails due - to one of the arguments not having the right type, we - continue as defined by the coercion protocol (see - above). Luckily, decoding errors are reported as - ValueErrors and are not masked by this technique. */ - if (!PyErr_ExceptionMatches(PyExc_TypeError)) - return -2; - PyErr_Clear(); - } -#endif - /* None is smaller than anything */ if (v == Py_None) return -1; Modified: python/branches/p3yk/Objects/sliceobject.c ============================================================================== --- python/branches/p3yk/Objects/sliceobject.c (original) +++ python/branches/p3yk/Objects/sliceobject.c Mon Aug 21 21:07:27 2006 @@ -252,7 +252,7 @@ { Py_ssize_t ilen, start, stop, step, slicelength; - ilen = PyInt_AsSsize_t(len); + ilen = PyNumber_AsSsize_t(len, PyExc_OverflowError); if (ilen == -1 && PyErr_Occurred()) { return NULL; Modified: python/branches/p3yk/Objects/stringobject.c ============================================================================== --- python/branches/p3yk/Objects/stringobject.c (original) +++ python/branches/p3yk/Objects/stringobject.c Mon Aug 21 21:07:27 2006 @@ -1187,9 +1187,8 @@ static PyObject* string_subscript(PyStringObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) @@ -4226,12 +4225,17 @@ if (!result) return NULL; + buf = PyString_AsString(result); + if (!buf) { + Py_DECREF(result); + return NULL; + } + /* To modify the string in-place, there can only be one reference. */ if (result->ob_refcnt != 1) { PyErr_BadInternalCall(); return NULL; } - buf = PyString_AsString(result); llen = PyString_Size(result); if (llen > PY_SSIZE_T_MAX) { PyErr_SetString(PyExc_ValueError, "string too large in _PyString_FormatLong"); Modified: python/branches/p3yk/Objects/structseq.c ============================================================================== --- python/branches/p3yk/Objects/structseq.c (original) +++ python/branches/p3yk/Objects/structseq.c Mon Aug 21 21:07:27 2006 @@ -215,6 +215,8 @@ PyObject *tup; int result; tup = make_tuple(obj); + if (!tup) + return -1; result = PySequence_Contains(tup, o); Py_DECREF(tup); return result; @@ -226,6 +228,8 @@ PyObject *tup; long result; tup = make_tuple((PyStructSequence*) obj); + if (!tup) + return -1; result = PyObject_Hash(tup); Py_DECREF(tup); return result; Modified: python/branches/p3yk/Objects/tupleobject.c ============================================================================== --- python/branches/p3yk/Objects/tupleobject.c (original) +++ python/branches/p3yk/Objects/tupleobject.c Mon Aug 21 21:07:27 2006 @@ -580,9 +580,8 @@ static PyObject* tuplesubscript(PyTupleObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/branches/p3yk/Objects/typeobject.c ============================================================================== --- python/branches/p3yk/Objects/typeobject.c (original) +++ python/branches/p3yk/Objects/typeobject.c Mon Aug 21 21:07:27 2006 @@ -3373,7 +3373,7 @@ if (!PyArg_UnpackTuple(args, "", 1, 1, &o)) return NULL; - i = PyNumber_Index(o); + i = PyNumber_AsSsize_t(o, PyExc_OverflowError); if (i == -1 && PyErr_Occurred()) return NULL; return (*func)(self, i); @@ -3384,7 +3384,7 @@ { Py_ssize_t i; - i = PyNumber_Index(arg); + i = PyNumber_AsSsize_t(arg, PyExc_OverflowError); if (i == -1 && PyErr_Occurred()) return -1; if (i < 0) { @@ -3956,19 +3956,17 @@ { static PyObject *len_str; PyObject *res = call_method(self, "__len__", &len_str, "()"); - Py_ssize_t temp; Py_ssize_t len; if (res == NULL) return -1; - temp = PyInt_AsSsize_t(res); - len = (int)temp; + len = PyInt_AsSsize_t(res); Py_DECREF(res); if (len == -1 && PyErr_Occurred()) return -1; -#if SIZEOF_SIZE_T < SIZEOF_LONG +#if SIZEOF_SIZE_T < SIZEOF_INT /* Overflow check -- range of PyInt is more than C ssize_t */ - if (len != temp) { + if (len != (int)len) { PyErr_SetString(PyExc_OverflowError, "__len__() should return 0 <= outcome < 2**31"); return -1; @@ -4189,26 +4187,11 @@ } -static Py_ssize_t +static PyObject * slot_nb_index(PyObject *self) { static PyObject *index_str; - PyObject *temp = call_method(self, "__index__", &index_str, "()"); - Py_ssize_t result; - - if (temp == NULL) - return -1; - if (PyInt_CheckExact(temp) || PyLong_CheckExact(temp)) { - result = temp->ob_type->tp_as_number->nb_index(temp); - } - else { - PyErr_Format(PyExc_TypeError, - "__index__ must return an int or a long, " - "not '%.200s'", temp->ob_type->tp_name); - result = -1; - } - Py_DECREF(temp); - return result; + return call_method(self, "__index__", &index_str, "()"); } @@ -4882,7 +4865,7 @@ "oct(x)"), UNSLOT("__hex__", nb_hex, slot_nb_hex, wrap_unaryfunc, "hex(x)"), - NBSLOT("__index__", nb_index, slot_nb_index, wrap_lenfunc, + NBSLOT("__index__", nb_index, slot_nb_index, wrap_unaryfunc, "x[y:z] <==> x[y.__index__():z.__index__()]"), IBSLOT("__iadd__", nb_inplace_add, slot_nb_inplace_add, wrap_binaryfunc, "+"), Modified: python/branches/p3yk/Objects/unicodeobject.c ============================================================================== --- python/branches/p3yk/Objects/unicodeobject.c (original) +++ python/branches/p3yk/Objects/unicodeobject.c Mon Aug 21 21:07:27 2006 @@ -5405,6 +5405,82 @@ return -1; } +PyObject *PyUnicode_RichCompare(PyObject *left, + PyObject *right, + int op) +{ + int result; + + result = PyUnicode_Compare(left, right); + if (result == -1 && PyErr_Occurred()) + goto onError; + + /* Convert the return value to a Boolean */ + switch (op) { + case Py_EQ: + result = (result == 0); + break; + case Py_NE: + result = (result != 0); + break; + case Py_LE: + result = (result <= 0); + break; + case Py_GE: + result = (result >= 0); + break; + case Py_LT: + result = (result == -1); + break; + case Py_GT: + result = (result == 1); + break; + } + return PyBool_FromLong(result); + + onError: + + /* Standard case + + Type errors mean that PyUnicode_FromObject() could not convert + one of the arguments (usually the right hand side) to Unicode, + ie. we can't handle the comparison request. However, it is + possible that the other object knows a comparison method, which + is why we return Py_NotImplemented to give the other object a + chance. + + */ + if (PyErr_ExceptionMatches(PyExc_TypeError)) { + PyErr_Clear(); + Py_INCREF(Py_NotImplemented); + return Py_NotImplemented; + } + if (op != Py_EQ && op != Py_NE) + return NULL; + + /* Equality comparison. + + This is a special case: we silence any PyExc_UnicodeDecodeError + and instead turn it into a PyErr_UnicodeWarning. + + */ + if (!PyErr_ExceptionMatches(PyExc_UnicodeDecodeError)) + return NULL; + PyErr_Clear(); + if (PyErr_Warn(PyExc_UnicodeWarning, + (op == Py_EQ) ? + "Unicode equal comparison " + "failed to convert both arguments to Unicode - " + "interpreting them as being unequal" : + "Unicode unequal comparison " + "failed to convert both arguments to Unicode - " + "interpreting them as being unequal" + ) < 0) + return NULL; + result = (op == Py_NE); + return PyBool_FromLong(result); +} + int PyUnicode_Contains(PyObject *container, PyObject *element) { @@ -6987,9 +7063,8 @@ static PyObject* unicode_subscript(PyUnicodeObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) @@ -7861,7 +7936,7 @@ 0, /* tp_print */ 0, /* tp_getattr */ 0, /* tp_setattr */ - (cmpfunc) unicode_compare, /* tp_compare */ + 0, /* tp_compare */ unicode_repr, /* tp_repr */ &unicode_as_number, /* tp_as_number */ &unicode_as_sequence, /* tp_as_sequence */ @@ -7876,7 +7951,7 @@ unicode_doc, /* tp_doc */ 0, /* tp_traverse */ 0, /* tp_clear */ - 0, /* tp_richcompare */ + PyUnicode_RichCompare, /* tp_richcompare */ 0, /* tp_weaklistoffset */ unicode_iter, /* tp_iter */ 0, /* tp_iternext */ Modified: python/branches/p3yk/PC/pyconfig.h ============================================================================== --- python/branches/p3yk/PC/pyconfig.h (original) +++ python/branches/p3yk/PC/pyconfig.h Mon Aug 21 21:07:27 2006 @@ -280,9 +280,9 @@ their Makefile (other compilers are generally taken care of by distutils.) */ # ifdef _DEBUG -# pragma comment(lib,"python25_d.lib") +# pragma comment(lib,"python26_d.lib") # else -# pragma comment(lib,"python25.lib") +# pragma comment(lib,"python26.lib") # endif /* _DEBUG */ # endif /* _MSC_VER */ # endif /* Py_BUILD_CORE */ Modified: python/branches/p3yk/PCbuild/_ssl.mak ============================================================================== --- python/branches/p3yk/PCbuild/_ssl.mak (original) +++ python/branches/p3yk/PCbuild/_ssl.mak Mon Aug 21 21:07:27 2006 @@ -1,21 +1,37 @@ !IFDEF DEBUG -MODULE=_ssl_d.pyd -TEMP_DIR=x86-temp-debug/_ssl +SUFFIX=_d.pyd +TEMP=x86-temp-debug/ CFLAGS=/Od /Zi /MDd /LDd /DDEBUG /D_DEBUG /DWIN32 SSL_LIB_DIR=$(SSL_DIR)/out32.dbg !ELSE -MODULE=_ssl.pyd -TEMP_DIR=x86-temp-release/_ssl +SUFFIX=.pyd +TEMP=x86-temp-release/ CFLAGS=/Ox /MD /LD /DWIN32 SSL_LIB_DIR=$(SSL_DIR)/out32 !ENDIF INCLUDES=-I ../Include -I ../PC -I $(SSL_DIR)/inc32 -LIBS=gdi32.lib wsock32.lib user32.lib advapi32.lib /libpath:$(SSL_LIB_DIR) libeay32.lib ssleay32.lib -SOURCE=../Modules/_ssl.c $(SSL_LIB_DIR)/libeay32.lib $(SSL_LIB_DIR)/ssleay32.lib +SSL_LIBS=gdi32.lib wsock32.lib user32.lib advapi32.lib /LIBPATH:$(SSL_LIB_DIR) libeay32.lib ssleay32.lib +SSL_SOURCE=../Modules/_ssl.c -$(MODULE): $(SOURCE) ../PC/*.h ../Include/*.h - @if not exist "$(TEMP_DIR)/." mkdir "$(TEMP_DIR)" - cl /nologo $(SOURCE) $(CFLAGS) /Fo$(TEMP_DIR)\$*.obj $(INCLUDES) /link /out:$(MODULE) $(LIBS) +HASH_LIBS=gdi32.lib user32.lib advapi32.lib /libpath:$(SSL_LIB_DIR) libeay32.lib +HASH_SOURCE=../Modules/_hashopenssl.c + +all: _ssl$(SUFFIX) _hashlib$(SUFFIX) + +# Split compile/link into two steps to better support VSExtComp +_ssl$(SUFFIX): $(SSL_SOURCE) $(SSL_LIB_DIR)/libeay32.lib $(SSL_LIB_DIR)/ssleay32.lib ../PC/*.h ../Include/*.h + @if not exist "$(TEMP)/_ssl/." mkdir "$(TEMP)/_ssl" + cl /nologo /c $(SSL_SOURCE) $(CFLAGS) /Fo$(TEMP)\_ssl\$*.obj $(INCLUDES) + link /nologo @<< + /dll /out:_ssl$(SUFFIX) $(TEMP)\_ssl\$*.obj $(SSL_LIBS) +<< + +_hashlib$(SUFFIX): $(HASH_SOURCE) $(SSL_LIB_DIR)/libeay32.lib ../PC/*.h ../Include/*.h + @if not exist "$(TEMP)/_hashlib/." mkdir "$(TEMP)/_hashlib" + cl /nologo /c $(HASH_SOURCE) $(CFLAGS) /Fo$(TEMP)\_hashlib\$*.obj $(INCLUDES) + link /nologo @<< + /dll /out:_hashlib$(SUFFIX) $(HASH_LIBS) $(TEMP)\_hashlib\$*.obj +<< Modified: python/branches/p3yk/PCbuild/_ssl.vcproj ============================================================================== --- python/branches/p3yk/PCbuild/_ssl.vcproj (original) +++ python/branches/p3yk/PCbuild/_ssl.vcproj Mon Aug 21 21:07:27 2006 @@ -75,6 +75,9 @@ <File RelativePath="..\Modules\_ssl.c"> </File> + <File + RelativePath="..\Modules\_hashopenssl.c"> + </File> </Files> <Globals> </Globals> Modified: python/branches/p3yk/PCbuild/build_ssl.py ============================================================================== --- python/branches/p3yk/PCbuild/build_ssl.py (original) +++ python/branches/p3yk/PCbuild/build_ssl.py Mon Aug 21 21:07:27 2006 @@ -1,7 +1,7 @@ -# Script for building the _ssl module for Windows. +# Script for building the _ssl and _hashlib modules for Windows. # Uses Perl to setup the OpenSSL environment correctly # and build OpenSSL, then invokes a simple nmake session -# for _ssl.pyd itself. +# for the actual _ssl.pyd and _hashlib.pyd DLLs. # THEORETICALLY, you can: # * Unpack the latest SSL release one level above your main Python source @@ -10,8 +10,8 @@ # * Install ActivePerl and ensure it is somewhere on your path. # * Run this script from the PCBuild directory. # -# it should configure and build SSL, then build the ssl Python extension -# without intervention. +# it should configure and build SSL, then build the _ssl and _hashlib +# Python extensions without intervention. import os, sys, re @@ -59,7 +59,8 @@ candidates = [] for s in sources: try: - s = os.path.abspath(s) + # note: do not abspath s; the build will fail if any + # higher up directory name has spaces in it. fnames = os.listdir(s) except os.error: fnames = [] @@ -82,31 +83,9 @@ print "Found an SSL directory at '%s'" % (best_name,) else: print "Could not find an SSL directory in '%s'" % (sources,) + sys.stdout.flush() return best_name -def run_32all_py(): - # ms\32all.bat will reconfigure OpenSSL and then try to build - # all outputs (debug/nondebug/dll/lib). So we filter the file - # to exclude any "nmake" commands and then execute. - tempname = "ms\\32all_py.bat" - - in_bat = open("ms\\32all.bat") - temp_bat = open(tempname,"w") - while 1: - cmd = in_bat.readline() - print 'cmd', repr(cmd) - if not cmd: break - if cmd.strip()[:5].lower() == "nmake": - continue - temp_bat.write(cmd) - in_bat.close() - temp_bat.close() - os.system(tempname) - try: - os.remove(tempname) - except: - pass - def run_configure(configure, do_script): os.system("perl Configure "+configure) os.system(do_script) @@ -117,12 +96,14 @@ arch = "x86" debug = False configure = "VC-WIN32" - makefile = "32.mak" + do_script = "ms\\do_masm" + makefile = "ms\\nt.mak" elif sys.argv[1] == "Debug": arch = "x86" debug = True configure = "VC-WIN32" - makefile="d32.mak" + do_script = "ms\\do_masm" + makefile="ms\\d32.mak" elif sys.argv[1] == "ReleaseItanium": arch = "ia64" debug = False @@ -148,8 +129,9 @@ sys.exit(1) print "Found a working perl at '%s'" % (perl,) + sys.stdout.flush() # Look for SSL 2 levels up from pcbuild - ie, same place zlib etc all live. - ssl_dir = find_best_ssl_dir(("../..",)) + ssl_dir = find_best_ssl_dir(("..\\..",)) if ssl_dir is None: sys.exit(1) @@ -157,31 +139,40 @@ try: os.chdir(ssl_dir) # If the ssl makefiles do not exist, we invoke Perl to generate them. - if not os.path.isfile(makefile): + # Due to a bug in this script, the makefile sometimes ended up empty + # Force a regeneration if it is. + if not os.path.isfile(makefile) or os.path.getsize(makefile)==0: print "Creating the makefiles..." + sys.stdout.flush() # Put our working Perl at the front of our path - os.environ["PATH"] = os.path.split(perl)[0] + \ + os.environ["PATH"] = os.path.dirname(perl) + \ os.pathsep + \ os.environ["PATH"] - if arch=="x86": - run_32all_py() - else: - run_configure(configure, do_script) + run_configure(configure, do_script) + if arch=="x86" and debug: + # the do_masm script in openssl doesn't generate a debug + # build makefile so we generate it here: + os.system("perl util\mk1mf.pl debug "+configure+" >"+makefile) # Now run make. - print "Executing nmake over the ssl makefiles..." - rc = os.system("nmake /nologo -f "+makefile) + makeCommand = "nmake /nologo PERL=\"%s\" -f \"%s\"" %(perl, makefile) + print "Executing ssl makefiles:", makeCommand + sys.stdout.flush() + rc = os.system(makeCommand) if rc: - print "Executing d32.mak failed" + print "Executing "+makefile+" failed" print rc sys.exit(rc) finally: os.chdir(old_cd) # And finally, we can build the _ssl module itself for Python. - defs = "SSL_DIR=%s" % (ssl_dir,) + defs = "SSL_DIR=\"%s\"" % (ssl_dir,) if debug: defs = defs + " " + "DEBUG=1" - rc = os.system('nmake /nologo -f _ssl.mak ' + defs + " " + make_flags) + makeCommand = 'nmake /nologo -f _ssl.mak ' + defs + " " + make_flags + print "Executing:", makeCommand + sys.stdout.flush() + rc = os.system(makeCommand) sys.exit(rc) if __name__=='__main__': Modified: python/branches/p3yk/PCbuild/pythoncore.vcproj ============================================================================== --- python/branches/p3yk/PCbuild/pythoncore.vcproj (original) +++ python/branches/p3yk/PCbuild/pythoncore.vcproj Mon Aug 21 21:07:27 2006 @@ -39,15 +39,15 @@ <Tool Name="VCLinkerTool" AdditionalDependencies="getbuildinfo.o" - OutputFile="./python25.dll" + OutputFile="./python26.dll" LinkIncremental="1" SuppressStartupBanner="TRUE" IgnoreDefaultLibraryNames="libc" GenerateDebugInformation="TRUE" - ProgramDatabaseFile=".\./python25.pdb" + ProgramDatabaseFile=".\./python26.pdb" SubSystem="2" BaseAddress="0x1e000000" - ImportLibrary=".\./python25.lib" + ImportLibrary=".\./python26.lib" TargetMachine="1"/> <Tool Name="VCMIDLTool"/> @@ -99,15 +99,15 @@ <Tool Name="VCLinkerTool" AdditionalDependencies="getbuildinfo.o" - OutputFile="./python25_d.dll" + OutputFile="./python26_d.dll" LinkIncremental="1" SuppressStartupBanner="TRUE" IgnoreDefaultLibraryNames="libc" GenerateDebugInformation="TRUE" - ProgramDatabaseFile=".\./python25_d.pdb" + ProgramDatabaseFile=".\./python26_d.pdb" SubSystem="2" BaseAddress="0x1e000000" - ImportLibrary=".\./python25_d.lib" + ImportLibrary=".\./python26_d.lib" TargetMachine="1"/> <Tool Name="VCMIDLTool"/> @@ -166,15 +166,15 @@ Name="VCLinkerTool" AdditionalOptions=" /MACHINE:IA64 /USELINK:MS_SDK" AdditionalDependencies="getbuildinfo.o" - OutputFile="./python25.dll" + OutputFile="./python26.dll" LinkIncremental="1" SuppressStartupBanner="FALSE" IgnoreDefaultLibraryNames="libc" GenerateDebugInformation="TRUE" - ProgramDatabaseFile=".\./python25.pdb" + ProgramDatabaseFile=".\./python26.pdb" SubSystem="2" BaseAddress="0x1e000000" - ImportLibrary=".\./python25.lib" + ImportLibrary=".\./python26.lib" TargetMachine="0"/> <Tool Name="VCMIDLTool"/> @@ -233,15 +233,15 @@ Name="VCLinkerTool" AdditionalOptions=" /MACHINE:AMD64 /USELINK:MS_SDK" AdditionalDependencies="getbuildinfo.o" - OutputFile="./python25.dll" + OutputFile="./python26.dll" LinkIncremental="1" SuppressStartupBanner="TRUE" IgnoreDefaultLibraryNames="libc" GenerateDebugInformation="TRUE" - ProgramDatabaseFile=".\./python25.pdb" + ProgramDatabaseFile=".\./python26.pdb" SubSystem="2" BaseAddress="0x1e000000" - ImportLibrary=".\./python25.lib" + ImportLibrary=".\./python26.lib" TargetMachine="0"/> <Tool Name="VCMIDLTool"/> Modified: python/branches/p3yk/PCbuild/readme.txt ============================================================================== --- python/branches/p3yk/PCbuild/readme.txt (original) +++ python/branches/p3yk/PCbuild/readme.txt Mon Aug 21 21:07:27 2006 @@ -12,7 +12,7 @@ The proper order to build subprojects: 1) pythoncore (this builds the main Python DLL and library files, - python25.{dll, lib} in Release mode) + python26.{dll, lib} in Release mode) NOTE: in previous releases, this subproject was named after the release number, e.g. python20. @@ -26,7 +26,7 @@ test slave; see SUBPROJECTS below) When using the Debug setting, the output files have a _d added to -their name: python25_d.dll, python_d.exe, parser_d.pyd, and so on. +their name: python26_d.dll, python_d.exe, parser_d.pyd, and so on. SUBPROJECTS ----------- Modified: python/branches/p3yk/Parser/grammar.c ============================================================================== --- python/branches/p3yk/Parser/grammar.c (original) +++ python/branches/p3yk/Parser/grammar.c Mon Aug 21 21:07:27 2006 @@ -199,6 +199,10 @@ else name_len = strlen(src); dest = (char *)malloc(name_len + 1); + if (!dest) { + printf("Can't alloc dest '%s'\n", src); + return; + } strncpy(dest, src, name_len); dest[name_len] = '\0'; free(lb->lb_str); Modified: python/branches/p3yk/Parser/parsetok.c ============================================================================== --- python/branches/p3yk/Parser/parsetok.c (original) +++ python/branches/p3yk/Parser/parsetok.c Mon Aug 21 21:07:27 2006 @@ -225,6 +225,11 @@ } } else if (tok->encoding != NULL) { node* r = PyNode_New(encoding_decl); + if (!r) { + err_ret->error = E_NOMEM; + n = NULL; + goto done; + } r->n_str = tok->encoding; r->n_nchildren = 1; r->n_child = n; @@ -232,6 +237,7 @@ n = r; } +done: PyTokenizer_Free(tok); return n; Modified: python/branches/p3yk/Python/ast.c ============================================================================== --- python/branches/p3yk/Python/ast.c (original) +++ python/branches/p3yk/Python/ast.c Mon Aug 21 21:07:27 2006 @@ -2197,6 +2197,8 @@ } else { alias_ty a = alias_for_import_name(c, CHILD(n, 0)); + if (!a) + return NULL; if (strcmp(STR(CHILD(n, 1)), "as") != 0) { ast_error(n, "must use 'as' in import"); return NULL; Modified: python/branches/p3yk/Python/bltinmodule.c ============================================================================== --- python/branches/p3yk/Python/bltinmodule.c (original) +++ python/branches/p3yk/Python/bltinmodule.c Mon Aug 21 21:07:27 2006 @@ -691,7 +691,7 @@ PyObject *d; d = PyEval_GetGlobals(); - Py_INCREF(d); + Py_XINCREF(d); return d; } @@ -1084,7 +1084,7 @@ PyObject *d; d = PyEval_GetLocals(); - Py_INCREF(d); + Py_XINCREF(d); return d; } Modified: python/branches/p3yk/Python/ceval.c ============================================================================== --- python/branches/p3yk/Python/ceval.c (original) +++ python/branches/p3yk/Python/ceval.c Mon Aug 21 21:07:27 2006 @@ -2597,6 +2597,7 @@ return NULL; } + assert(tstate != NULL); assert(globals != NULL); f = PyFrame_New(tstate, co, globals, locals); if (f == NULL) @@ -3589,6 +3590,7 @@ PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ + assert(tstate != NULL); f = PyFrame_New(tstate, co, globals, NULL); if (f == NULL) return NULL; @@ -3601,7 +3603,6 @@ fastlocals[i] = *stack++; } retval = PyEval_EvalFrameEx(f,0); - assert(tstate != NULL); ++tstate->recursion_depth; Py_DECREF(f); --tstate->recursion_depth; @@ -3819,11 +3820,14 @@ if (v != NULL) { Py_ssize_t x; if (PyInt_Check(v)) { - x = PyInt_AsSsize_t(v); + /* XXX(nnorwitz): I think PyInt_AS_LONG is correct, + however, it looks like it should be AsSsize_t. + There should be a comment here explaining why. + */ + x = PyInt_AS_LONG(v); } - else if (v->ob_type->tp_as_number && - v->ob_type->tp_as_number->nb_index) { - x = v->ob_type->tp_as_number->nb_index(v); + else if (PyIndex_Check(v)) { + x = PyNumber_AsSsize_t(v, NULL); if (x == -1 && PyErr_Occurred()) return 0; } @@ -3839,9 +3843,8 @@ } #undef ISINDEX -#define ISINDEX(x) ((x) == NULL || PyInt_Check(x) || PyLong_Check(x) || \ - ((x)->ob_type->tp_as_number && \ - (x)->ob_type->tp_as_number->nb_index)) +#define ISINDEX(x) ((x) == NULL || \ + PyInt_Check(x) || PyLong_Check(x) || PyIndex_Check(x)) static PyObject * apply_slice(PyObject *u, PyObject *v, PyObject *w) /* return u[v:w] */ @@ -4081,6 +4084,11 @@ locals = PyEval_GetLocals(); plain = 1; } + if (!globals || !locals) { + PyErr_SetString(PyExc_SystemError, + "globals and locals cannot be NULL"); + return -1; + } } else if (locals == Py_None) locals = globals; Modified: python/branches/p3yk/Python/compile.c ============================================================================== --- python/branches/p3yk/Python/compile.c (original) +++ python/branches/p3yk/Python/compile.c Mon Aug 21 21:07:27 2006 @@ -204,8 +204,8 @@ const char *p, *name = PyString_AsString(ident); char *buffer; size_t nlen, plen; - if (privateobj == NULL || name == NULL || name[0] != '_' || - name[1] != '_') { + if (privateobj == NULL || !PyString_Check(privateobj) || + name == NULL || name[0] != '_' || name[1] != '_') { Py_INCREF(ident); return ident; } @@ -394,608 +394,6 @@ return dest; } -/* Begin: Peephole optimizations ----------------------------------------- */ - -#define GETARG(arr, i) ((int)((arr[i+2]<<8) + arr[i+1])) -#define UNCONDITIONAL_JUMP(op) (op==JUMP_ABSOLUTE || op==JUMP_FORWARD) -#define ABSOLUTE_JUMP(op) (op==JUMP_ABSOLUTE || op==CONTINUE_LOOP) -#define GETJUMPTGT(arr, i) (GETARG(arr,i) + (ABSOLUTE_JUMP(arr[i]) ? 0 : i+3)) -#define SETARG(arr, i, val) arr[i+2] = val>>8; arr[i+1] = val & 255 -#define CODESIZE(op) (HAS_ARG(op) ? 3 : 1) -#define ISBASICBLOCK(blocks, start, bytes) \ - (blocks[start]==blocks[start+bytes-1]) - -/* Replace LOAD_CONST c1. LOAD_CONST c2 ... LOAD_CONST cn BUILD_TUPLE n - with LOAD_CONST (c1, c2, ... cn). - The consts table must still be in list form so that the - new constant (c1, c2, ... cn) can be appended. - Called with codestr pointing to the first LOAD_CONST. - Bails out with no change if one or more of the LOAD_CONSTs is missing. - Also works for BUILD_LIST when followed by an "in" or "not in" test. -*/ -static int -tuple_of_constants(unsigned char *codestr, int n, PyObject *consts) -{ - PyObject *newconst, *constant; - Py_ssize_t i, arg, len_consts; - - /* Pre-conditions */ - assert(PyList_CheckExact(consts)); - assert(codestr[n*3] == BUILD_TUPLE || codestr[n*3] == BUILD_LIST); - assert(GETARG(codestr, (n*3)) == n); - for (i=0 ; i<n ; i++) - assert(codestr[i*3] == LOAD_CONST); - - /* Buildup new tuple of constants */ - newconst = PyTuple_New(n); - if (newconst == NULL) - return 0; - len_consts = PyList_GET_SIZE(consts); - for (i=0 ; i<n ; i++) { - arg = GETARG(codestr, (i*3)); - assert(arg < len_consts); - constant = PyList_GET_ITEM(consts, arg); - Py_INCREF(constant); - PyTuple_SET_ITEM(newconst, i, constant); - } - - /* Append folded constant onto consts */ - if (PyList_Append(consts, newconst)) { - Py_DECREF(newconst); - return 0; - } - Py_DECREF(newconst); - - /* Write NOPs over old LOAD_CONSTS and - add a new LOAD_CONST newconst on top of the BUILD_TUPLE n */ - memset(codestr, NOP, n*3); - codestr[n*3] = LOAD_CONST; - SETARG(codestr, (n*3), len_consts); - return 1; -} - -/* Replace LOAD_CONST c1. LOAD_CONST c2 BINOP - with LOAD_CONST binop(c1,c2) - The consts table must still be in list form so that the - new constant can be appended. - Called with codestr pointing to the first LOAD_CONST. - Abandons the transformation if the folding fails (i.e. 1+'a'). - If the new constant is a sequence, only folds when the size - is below a threshold value. That keeps pyc files from - becoming large in the presence of code like: (None,)*1000. -*/ -static int -fold_binops_on_constants(unsigned char *codestr, PyObject *consts) -{ - PyObject *newconst, *v, *w; - Py_ssize_t len_consts, size; - int opcode; - - /* Pre-conditions */ - assert(PyList_CheckExact(consts)); - assert(codestr[0] == LOAD_CONST); - assert(codestr[3] == LOAD_CONST); - - /* Create new constant */ - v = PyList_GET_ITEM(consts, GETARG(codestr, 0)); - w = PyList_GET_ITEM(consts, GETARG(codestr, 3)); - opcode = codestr[6]; - switch (opcode) { - case BINARY_POWER: - newconst = PyNumber_Power(v, w, Py_None); - break; - case BINARY_MULTIPLY: - newconst = PyNumber_Multiply(v, w); - break; - case BINARY_TRUE_DIVIDE: - newconst = PyNumber_TrueDivide(v, w); - break; - case BINARY_FLOOR_DIVIDE: - newconst = PyNumber_FloorDivide(v, w); - break; - case BINARY_MODULO: - newconst = PyNumber_Remainder(v, w); - break; - case BINARY_ADD: - newconst = PyNumber_Add(v, w); - break; - case BINARY_SUBTRACT: - newconst = PyNumber_Subtract(v, w); - break; - case BINARY_SUBSCR: - newconst = PyObject_GetItem(v, w); - break; - case BINARY_LSHIFT: - newconst = PyNumber_Lshift(v, w); - break; - case BINARY_RSHIFT: - newconst = PyNumber_Rshift(v, w); - break; - case BINARY_AND: - newconst = PyNumber_And(v, w); - break; - case BINARY_XOR: - newconst = PyNumber_Xor(v, w); - break; - case BINARY_OR: - newconst = PyNumber_Or(v, w); - break; - default: - /* Called with an unknown opcode */ - PyErr_Format(PyExc_SystemError, - "unexpected binary operation %d on a constant", - opcode); - return 0; - } - if (newconst == NULL) { - PyErr_Clear(); - return 0; - } - size = PyObject_Size(newconst); - if (size == -1) - PyErr_Clear(); - else if (size > 20) { - Py_DECREF(newconst); - return 0; - } - - /* Append folded constant into consts table */ - len_consts = PyList_GET_SIZE(consts); - if (PyList_Append(consts, newconst)) { - Py_DECREF(newconst); - return 0; - } - Py_DECREF(newconst); - - /* Write NOP NOP NOP NOP LOAD_CONST newconst */ - memset(codestr, NOP, 4); - codestr[4] = LOAD_CONST; - SETARG(codestr, 4, len_consts); - return 1; -} - -static int -fold_unaryops_on_constants(unsigned char *codestr, PyObject *consts) -{ - PyObject *newconst=NULL, *v; - Py_ssize_t len_consts; - int opcode; - - /* Pre-conditions */ - assert(PyList_CheckExact(consts)); - assert(codestr[0] == LOAD_CONST); - - /* Create new constant */ - v = PyList_GET_ITEM(consts, GETARG(codestr, 0)); - opcode = codestr[3]; - switch (opcode) { - case UNARY_NEGATIVE: - /* Preserve the sign of -0.0 */ - if (PyObject_IsTrue(v) == 1) - newconst = PyNumber_Negative(v); - break; - case UNARY_CONVERT: - newconst = PyObject_Repr(v); - break; - case UNARY_INVERT: - newconst = PyNumber_Invert(v); - break; - default: - /* Called with an unknown opcode */ - PyErr_Format(PyExc_SystemError, - "unexpected unary operation %d on a constant", - opcode); - return 0; - } - if (newconst == NULL) { - PyErr_Clear(); - return 0; - } - - /* Append folded constant into consts table */ - len_consts = PyList_GET_SIZE(consts); - if (PyList_Append(consts, newconst)) { - Py_DECREF(newconst); - return 0; - } - Py_DECREF(newconst); - - /* Write NOP LOAD_CONST newconst */ - codestr[0] = NOP; - codestr[1] = LOAD_CONST; - SETARG(codestr, 1, len_consts); - return 1; -} - -static unsigned int * -markblocks(unsigned char *code, int len) -{ - unsigned int *blocks = (unsigned int *)PyMem_Malloc(len*sizeof(int)); - int i,j, opcode, blockcnt = 0; - - if (blocks == NULL) { - PyErr_NoMemory(); - return NULL; - } - memset(blocks, 0, len*sizeof(int)); - - /* Mark labels in the first pass */ - for (i=0 ; i<len ; i+=CODESIZE(opcode)) { - opcode = code[i]; - switch (opcode) { - case FOR_ITER: - case JUMP_FORWARD: - case JUMP_IF_FALSE: - case JUMP_IF_TRUE: - case JUMP_ABSOLUTE: - case CONTINUE_LOOP: - case SETUP_LOOP: - case SETUP_EXCEPT: - case SETUP_FINALLY: - j = GETJUMPTGT(code, i); - blocks[j] = 1; - break; - } - } - /* Build block numbers in the second pass */ - for (i=0 ; i<len ; i++) { - blockcnt += blocks[i]; /* increment blockcnt over labels */ - blocks[i] = blockcnt; - } - return blocks; -} - -/* Perform basic peephole optimizations to components of a code object. - The consts object should still be in list form to allow new constants - to be appended. - - To keep the optimizer simple, it bails out (does nothing) for code - containing extended arguments or that has a length over 32,700. That - allows us to avoid overflow and sign issues. Likewise, it bails when - the lineno table has complex encoding for gaps >= 255. - - Optimizations are restricted to simple transformations occuring within a - single basic block. All transformations keep the code size the same or - smaller. For those that reduce size, the gaps are initially filled with - NOPs. Later those NOPs are removed and the jump addresses retargeted in - a single pass. Line numbering is adjusted accordingly. */ - -static PyObject * -optimize_code(PyObject *code, PyObject* consts, PyObject *names, - PyObject *lineno_obj) -{ - Py_ssize_t i, j, codelen; - int nops, h, adj; - int tgt, tgttgt, opcode; - unsigned char *codestr = NULL; - unsigned char *lineno; - int *addrmap = NULL; - int new_line, cum_orig_line, last_line, tabsiz; - int cumlc=0, lastlc=0; /* Count runs of consecutive LOAD_CONSTs */ - unsigned int *blocks = NULL; - char *name; - - /* Bail out if an exception is set */ - if (PyErr_Occurred()) - goto exitUnchanged; - - /* Bypass optimization when the lineno table is too complex */ - assert(PyString_Check(lineno_obj)); - lineno = (unsigned char*)PyString_AS_STRING(lineno_obj); - tabsiz = PyString_GET_SIZE(lineno_obj); - if (memchr(lineno, 255, tabsiz) != NULL) - goto exitUnchanged; - - /* Avoid situations where jump retargeting could overflow */ - assert(PyString_Check(code)); - codelen = PyString_Size(code); - if (codelen > 32700) - goto exitUnchanged; - - /* Make a modifiable copy of the code string */ - codestr = (unsigned char *)PyMem_Malloc(codelen); - if (codestr == NULL) - goto exitUnchanged; - codestr = (unsigned char *)memcpy(codestr, - PyString_AS_STRING(code), codelen); - - /* Verify that RETURN_VALUE terminates the codestring. This allows - the various transformation patterns to look ahead several - instructions without additional checks to make sure they are not - looking beyond the end of the code string. - */ - if (codestr[codelen-1] != RETURN_VALUE) - goto exitUnchanged; - - /* Mapping to new jump targets after NOPs are removed */ - addrmap = (int *)PyMem_Malloc(codelen * sizeof(int)); - if (addrmap == NULL) - goto exitUnchanged; - - blocks = markblocks(codestr, codelen); - if (blocks == NULL) - goto exitUnchanged; - assert(PyList_Check(consts)); - - for (i=0 ; i<codelen ; i += CODESIZE(codestr[i])) { - opcode = codestr[i]; - - lastlc = cumlc; - cumlc = 0; - - switch (opcode) { - - /* Replace UNARY_NOT JUMP_IF_FALSE POP_TOP with - with JUMP_IF_TRUE POP_TOP */ - case UNARY_NOT: - if (codestr[i+1] != JUMP_IF_FALSE || - codestr[i+4] != POP_TOP || - !ISBASICBLOCK(blocks,i,5)) - continue; - tgt = GETJUMPTGT(codestr, (i+1)); - if (codestr[tgt] != POP_TOP) - continue; - j = GETARG(codestr, i+1) + 1; - codestr[i] = JUMP_IF_TRUE; - SETARG(codestr, i, j); - codestr[i+3] = POP_TOP; - codestr[i+4] = NOP; - break; - - /* not a is b --> a is not b - not a in b --> a not in b - not a is not b --> a is b - not a not in b --> a in b - */ - case COMPARE_OP: - j = GETARG(codestr, i); - if (j < 6 || j > 9 || - codestr[i+3] != UNARY_NOT || - !ISBASICBLOCK(blocks,i,4)) - continue; - SETARG(codestr, i, (j^1)); - codestr[i+3] = NOP; - break; - - /* Replace LOAD_GLOBAL/LOAD_NAME None - with LOAD_CONST None */ - case LOAD_NAME: - case LOAD_GLOBAL: - j = GETARG(codestr, i); - name = PyString_AsString(PyTuple_GET_ITEM(names, j)); - if (name == NULL || strcmp(name, "None") != 0) - continue; - for (j=0 ; j < PyList_GET_SIZE(consts) ; j++) { - if (PyList_GET_ITEM(consts, j) == Py_None) { - codestr[i] = LOAD_CONST; - SETARG(codestr, i, j); - cumlc = lastlc + 1; - break; - } - } - break; - - /* Skip over LOAD_CONST trueconst - JUMP_IF_FALSE xx POP_TOP */ - case LOAD_CONST: - cumlc = lastlc + 1; - j = GETARG(codestr, i); - if (codestr[i+3] != JUMP_IF_FALSE || - codestr[i+6] != POP_TOP || - !ISBASICBLOCK(blocks,i,7) || - !PyObject_IsTrue(PyList_GET_ITEM(consts, j))) - continue; - memset(codestr+i, NOP, 7); - cumlc = 0; - break; - - /* Try to fold tuples of constants (includes a case for lists - which are only used for "in" and "not in" tests). - Skip over BUILD_SEQN 1 UNPACK_SEQN 1. - Replace BUILD_SEQN 2 UNPACK_SEQN 2 with ROT2. - Replace BUILD_SEQN 3 UNPACK_SEQN 3 with ROT3 ROT2. */ - case BUILD_TUPLE: - case BUILD_LIST: - j = GETARG(codestr, i); - h = i - 3 * j; - if (h >= 0 && - j <= lastlc && - ((opcode == BUILD_TUPLE && - ISBASICBLOCK(blocks, h, 3*(j+1))) || - (opcode == BUILD_LIST && - codestr[i+3]==COMPARE_OP && - ISBASICBLOCK(blocks, h, 3*(j+2)) && - (GETARG(codestr,i+3)==6 || - GETARG(codestr,i+3)==7))) && - tuple_of_constants(&codestr[h], j, consts)) { - assert(codestr[i] == LOAD_CONST); - cumlc = 1; - break; - } - if (codestr[i+3] != UNPACK_SEQUENCE || - !ISBASICBLOCK(blocks,i,6) || - j != GETARG(codestr, i+3)) - continue; - if (j == 1) { - memset(codestr+i, NOP, 6); - } else if (j == 2) { - codestr[i] = ROT_TWO; - memset(codestr+i+1, NOP, 5); - } else if (j == 3) { - codestr[i] = ROT_THREE; - codestr[i+1] = ROT_TWO; - memset(codestr+i+2, NOP, 4); - } - break; - - /* Fold binary ops on constants. - LOAD_CONST c1 LOAD_CONST c2 BINOP --> LOAD_CONST binop(c1,c2) */ - case BINARY_POWER: - case BINARY_MULTIPLY: - case BINARY_TRUE_DIVIDE: - case BINARY_FLOOR_DIVIDE: - case BINARY_MODULO: - case BINARY_ADD: - case BINARY_SUBTRACT: - case BINARY_SUBSCR: - case BINARY_LSHIFT: - case BINARY_RSHIFT: - case BINARY_AND: - case BINARY_XOR: - case BINARY_OR: - if (lastlc >= 2 && - ISBASICBLOCK(blocks, i-6, 7) && - fold_binops_on_constants(&codestr[i-6], consts)) { - i -= 2; - assert(codestr[i] == LOAD_CONST); - cumlc = 1; - } - break; - - /* Fold unary ops on constants. - LOAD_CONST c1 UNARY_OP --> LOAD_CONST unary_op(c) */ - case UNARY_NEGATIVE: - case UNARY_CONVERT: - case UNARY_INVERT: - if (lastlc >= 1 && - ISBASICBLOCK(blocks, i-3, 4) && - fold_unaryops_on_constants(&codestr[i-3], consts)) { - i -= 2; - assert(codestr[i] == LOAD_CONST); - cumlc = 1; - } - break; - - /* Simplify conditional jump to conditional jump where the - result of the first test implies the success of a similar - test or the failure of the opposite test. - Arises in code like: - "if a and b:" - "if a or b:" - "a and b or c" - "(a and b) and c" - x:JUMP_IF_FALSE y y:JUMP_IF_FALSE z --> x:JUMP_IF_FALSE z - x:JUMP_IF_FALSE y y:JUMP_IF_TRUE z --> x:JUMP_IF_FALSE y+3 - where y+3 is the instruction following the second test. - */ - case JUMP_IF_FALSE: - case JUMP_IF_TRUE: - tgt = GETJUMPTGT(codestr, i); - j = codestr[tgt]; - if (j == JUMP_IF_FALSE || j == JUMP_IF_TRUE) { - if (j == opcode) { - tgttgt = GETJUMPTGT(codestr, tgt) - i - 3; - SETARG(codestr, i, tgttgt); - } else { - tgt -= i; - SETARG(codestr, i, tgt); - } - break; - } - /* Intentional fallthrough */ - - /* Replace jumps to unconditional jumps */ - case FOR_ITER: - case JUMP_FORWARD: - case JUMP_ABSOLUTE: - case CONTINUE_LOOP: - case SETUP_LOOP: - case SETUP_EXCEPT: - case SETUP_FINALLY: - tgt = GETJUMPTGT(codestr, i); - if (!UNCONDITIONAL_JUMP(codestr[tgt])) - continue; - tgttgt = GETJUMPTGT(codestr, tgt); - if (opcode == JUMP_FORWARD) /* JMP_ABS can go backwards */ - opcode = JUMP_ABSOLUTE; - if (!ABSOLUTE_JUMP(opcode)) - tgttgt -= i + 3; /* Calc relative jump addr */ - if (tgttgt < 0) /* No backward relative jumps */ - continue; - codestr[i] = opcode; - SETARG(codestr, i, tgttgt); - break; - - case EXTENDED_ARG: - goto exitUnchanged; - - /* Replace RETURN LOAD_CONST None RETURN with just RETURN */ - case RETURN_VALUE: - if (i+4 >= codelen || - codestr[i+4] != RETURN_VALUE || - !ISBASICBLOCK(blocks,i,5)) - continue; - memset(codestr+i+1, NOP, 4); - break; - } - } - - /* Fixup linenotab */ - for (i=0, nops=0 ; i<codelen ; i += CODESIZE(codestr[i])) { - addrmap[i] = i - nops; - if (codestr[i] == NOP) - nops++; - } - cum_orig_line = 0; - last_line = 0; - for (i=0 ; i < tabsiz ; i+=2) { - cum_orig_line += lineno[i]; - new_line = addrmap[cum_orig_line]; - assert (new_line - last_line < 255); - lineno[i] =((unsigned char)(new_line - last_line)); - last_line = new_line; - } - - /* Remove NOPs and fixup jump targets */ - for (i=0, h=0 ; i<codelen ; ) { - opcode = codestr[i]; - switch (opcode) { - case NOP: - i++; - continue; - - case JUMP_ABSOLUTE: - case CONTINUE_LOOP: - j = addrmap[GETARG(codestr, i)]; - SETARG(codestr, i, j); - break; - - case FOR_ITER: - case JUMP_FORWARD: - case JUMP_IF_FALSE: - case JUMP_IF_TRUE: - case SETUP_LOOP: - case SETUP_EXCEPT: - case SETUP_FINALLY: - j = addrmap[GETARG(codestr, i) + i + 3] - addrmap[i] - 3; - SETARG(codestr, i, j); - break; - } - adj = CODESIZE(opcode); - while (adj--) - codestr[h++] = codestr[i++]; - } - assert(h + nops == codelen); - - code = PyString_FromStringAndSize((char *)codestr, h); - PyMem_Free(addrmap); - PyMem_Free(codestr); - PyMem_Free(blocks); - return code; - - exitUnchanged: - if (blocks != NULL) - PyMem_Free(blocks); - if (addrmap != NULL) - PyMem_Free(addrmap); - if (codestr != NULL) - PyMem_Free(codestr); - Py_INCREF(code); - return code; -} - -/* End: Peephole optimizations ----------------------------------------- */ - /* Leave this debugging code for just a little longer. @@ -1170,6 +568,7 @@ if (n >= 0) { wrapper = PyList_GET_ITEM(c->c_stack, n); c->u = (struct compiler_unit *)PyCObject_AsVoidPtr(wrapper); + assert(c->u); /* we are deleting from a list so this really shouldn't fail */ if (PySequence_DelItem(c->c_stack, n) < 0) Py_FatalError("compiler_exit_scope()"); @@ -4400,7 +3799,7 @@ if (flags < 0) goto error; - bytecode = optimize_code(a->a_bytecode, consts, names, a->a_lnotab); + bytecode = PyCode_Optimize(a->a_bytecode, consts, names, a->a_lnotab); if (!bytecode) goto error; Modified: python/branches/p3yk/Python/import.c ============================================================================== --- python/branches/p3yk/Python/import.c (original) +++ python/branches/p3yk/Python/import.c Mon Aug 21 21:07:27 2006 @@ -934,7 +934,8 @@ if (Py_VerboseFlag) PySys_WriteStderr("import %s # from %s\n", name, pathname); - write_compiled_module(co, cpathname, mtime); + if (cpathname) + write_compiled_module(co, cpathname, mtime); } m = PyImport_ExecCodeModuleEx(name, (PyObject *)co, pathname); Py_DECREF(co); @@ -1233,6 +1234,8 @@ for (i = 0; i < npath; i++) { PyObject *copy = NULL; PyObject *v = PyList_GetItem(path, i); + if (!v) + return NULL; #ifdef Py_USING_UNICODE if (PyUnicode_Check(v)) { copy = PyUnicode_Encode(PyUnicode_AS_UNICODE(v), @@ -3045,6 +3048,8 @@ if (m == NULL) goto failure; d = PyModule_GetDict(m); + if (d == NULL) + goto failure; if (setint(d, "SEARCH_ERROR", SEARCH_ERROR) < 0) goto failure; if (setint(d, "PY_SOURCE", PY_SOURCE) < 0) goto failure; Modified: python/branches/p3yk/Python/marshal.c ============================================================================== --- python/branches/p3yk/Python/marshal.c (original) +++ python/branches/p3yk/Python/marshal.c Mon Aug 21 21:07:27 2006 @@ -213,6 +213,10 @@ w_byte(TYPE_COMPLEX, p); temp = (PyFloatObject*)PyFloat_FromDouble( PyComplex_RealAsDouble(v)); + if (!temp) { + p->error = 1; + return; + } PyFloat_AsReprString(buf, temp); Py_DECREF(temp); n = strlen(buf); @@ -220,6 +224,10 @@ w_string(buf, (int)n, p); temp = (PyFloatObject*)PyFloat_FromDouble( PyComplex_ImagAsDouble(v)); + if (!temp) { + p->error = 1; + return; + } PyFloat_AsReprString(buf, temp); Py_DECREF(temp); n = strlen(buf); Copied: python/branches/p3yk/Python/peephole.c (from r51433, python/trunk/Python/peephole.c) ============================================================================== --- python/trunk/Python/peephole.c (original) +++ python/branches/p3yk/Python/peephole.c Mon Aug 21 21:07:27 2006 @@ -102,11 +102,6 @@ case BINARY_MULTIPLY: newconst = PyNumber_Multiply(v, w); break; - case BINARY_DIVIDE: - /* Cannot fold this operation statically since - the result can depend on the run-time presence - of the -Qnew flag */ - return 0; case BINARY_TRUE_DIVIDE: newconst = PyNumber_TrueDivide(v, w); break; Modified: python/branches/p3yk/Python/pystate.c ============================================================================== --- python/branches/p3yk/Python/pystate.c (original) +++ python/branches/p3yk/Python/pystate.c Mon Aug 21 21:07:27 2006 @@ -342,28 +342,43 @@ /* Asynchronously raise an exception in a thread. Requested by Just van Rossum and Alex Martelli. To prevent naive misuse, you must write your own extension - to call this. Must be called with the GIL held. - Returns the number of tstates modified; if it returns a number - greater than one, you're in trouble, and you should call it again - with exc=NULL to revert the effect. This raises no exceptions. */ + to call this, or use ctypes. Must be called with the GIL held. + Returns the number of tstates modified (normally 1, but 0 if `id` didn't + match any known thread id). Can be called with exc=NULL to clear an + existing async exception. This raises no exceptions. */ int PyThreadState_SetAsyncExc(long id, PyObject *exc) { PyThreadState *tstate = PyThreadState_GET(); PyInterpreterState *interp = tstate->interp; PyThreadState *p; - int count = 0; + + /* Although the GIL is held, a few C API functions can be called + * without the GIL held, and in particular some that create and + * destroy thread and interpreter states. Those can mutate the + * list of thread states we're traversing, so to prevent that we lock + * head_mutex for the duration. + */ HEAD_LOCK(); for (p = interp->tstate_head; p != NULL; p = p->next) { - if (p->thread_id != id) - continue; - Py_CLEAR(p->async_exc); - Py_XINCREF(exc); - p->async_exc = exc; - count += 1; + if (p->thread_id == id) { + /* Tricky: we need to decref the current value + * (if any) in p->async_exc, but that can in turn + * allow arbitrary Python code to run, including + * perhaps calls to this function. To prevent + * deadlock, we need to release head_mutex before + * the decref. + */ + PyObject *old_exc = p->async_exc; + Py_XINCREF(exc); + p->async_exc = exc; + HEAD_UNLOCK(); + Py_XDECREF(old_exc); + return 1; + } } HEAD_UNLOCK(); - return count; + return 0; } Modified: python/branches/p3yk/Python/pythonrun.c ============================================================================== --- python/branches/p3yk/Python/pythonrun.c (original) +++ python/branches/p3yk/Python/pythonrun.c Mon Aug 21 21:07:27 2006 @@ -199,12 +199,16 @@ if (bimod == NULL) Py_FatalError("Py_Initialize: can't initialize __builtin__"); interp->builtins = PyModule_GetDict(bimod); + if (interp->builtins == NULL) + Py_FatalError("Py_Initialize: can't initialize builtins dict"); Py_INCREF(interp->builtins); sysmod = _PySys_Init(); if (sysmod == NULL) Py_FatalError("Py_Initialize: can't initialize sys"); interp->sysdict = PyModule_GetDict(sysmod); + if (interp->sysdict == NULL) + Py_FatalError("Py_Initialize: can't initialize sys dict"); Py_INCREF(interp->sysdict); _PyImport_FixupExtension("sys", "sys"); PySys_SetPath(Py_GetPath()); Modified: python/branches/p3yk/Python/symtable.c ============================================================================== --- python/branches/p3yk/Python/symtable.c (original) +++ python/branches/p3yk/Python/symtable.c Mon Aug 21 21:07:27 2006 @@ -221,8 +221,8 @@ return st; st->st_filename = filename; st->st_future = future; - if (!symtable_enter_block(st, GET_IDENTIFIER(top), ModuleBlock, - (void *)mod, 0)) { + if (!GET_IDENTIFIER(top) || + !symtable_enter_block(st, top, ModuleBlock, (void *)mod, 0)) { PySymtable_Free(st); return NULL; } @@ -915,6 +915,8 @@ PyOS_snprintf(tmpname, sizeof(tmpname), "_[%d]", ++st->st_cur->ste_tmpname); tmp = PyString_InternFromString(tmpname); + if (!tmp) + return 0; if (!symtable_add_def(st, tmp, DEF_LOCAL)) return 0; Py_DECREF(tmp); @@ -1121,12 +1123,13 @@ VISIT(st, expr, e->v.UnaryOp.operand); break; case Lambda_kind: { - if (!symtable_add_def(st, GET_IDENTIFIER(lambda), DEF_LOCAL)) + if (!GET_IDENTIFIER(lambda) || + !symtable_add_def(st, lambda, DEF_LOCAL)) return 0; if (e->v.Lambda.args->defaults) VISIT_SEQ(st, expr, e->v.Lambda.args->defaults); /* XXX how to get line numbers for expressions */ - if (!symtable_enter_block(st, GET_IDENTIFIER(lambda), + if (!symtable_enter_block(st, lambda, FunctionBlock, (void *)e, 0)) return 0; VISIT_IN_BLOCK(st, arguments, e->v.Lambda.args, (void*)e); @@ -1323,8 +1326,11 @@ PyObject *name = (a->asname == NULL) ? a->name : a->asname; const char *base = PyString_AS_STRING(name); char *dot = strchr(base, '.'); - if (dot) + if (dot) { store_name = PyString_FromStringAndSize(base, dot - base); + if (!store_name) + return 0; + } else { store_name = name; Py_INCREF(store_name); @@ -1399,8 +1405,8 @@ /* Outermost iterator is evaluated in current scope */ VISIT(st, expr, outermost->iter); /* Create generator scope for the rest */ - if (!symtable_enter_block(st, GET_IDENTIFIER(genexpr), - FunctionBlock, (void *)e, 0)) { + if (!GET_IDENTIFIER(genexpr) || + !symtable_enter_block(st, genexpr, FunctionBlock, (void *)e, 0)) { return 0; } st->st_cur->ste_generator = 1; @@ -1414,7 +1420,5 @@ VISIT_SEQ_TAIL_IN_BLOCK(st, comprehension, e->v.GeneratorExp.generators, 1, (void*)e); VISIT_IN_BLOCK(st, expr, e->v.GeneratorExp.elt, (void*)e); - if (!symtable_exit_block(st, (void *)e)) - return 0; - return 1; + return symtable_exit_block(st, (void *)e); } Modified: python/branches/p3yk/Tools/buildbot/external.bat ============================================================================== --- python/branches/p3yk/Tools/buildbot/external.bat (original) +++ python/branches/p3yk/Tools/buildbot/external.bat Mon Aug 21 21:07:27 2006 @@ -28,6 +28,7 @@ cd tk8.4.12\win nmake -f makefile.vc TCLDIR=..\..\tcl8.4.12 nmake -f makefile.vc TCLDIR=..\..\tcl8.4.12 INSTALLDIR=..\..\tcltk install + cd ..\.. ) @rem sqlite Modified: python/branches/p3yk/Tools/msi/msi.py ============================================================================== --- python/branches/p3yk/Tools/msi/msi.py (original) +++ python/branches/p3yk/Tools/msi/msi.py Mon Aug 21 21:07:27 2006 @@ -89,7 +89,8 @@ '_msi.pyd', '_ctypes.pyd', '_ctypes_test.pyd', - '_sqlite3.pyd' + '_sqlite3.pyd', + '_hashlib.pyd' ] # Well-known component UUIDs

2 1

buildbot warnings in alpha Debian 2.4
by buildbot＠python.org 22 Aug '06

22 Aug '06

The Buildbot has detected a new failure of alpha Debian 2.4. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%25202.4/builds/11 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release24-maint] HEAD Blamelist: georg.brandl Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in PPC64 Debian trunk
by buildbot＠python.org 22 Aug '06

22 Aug '06

The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/431 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot

1 0

r51466 - in python/branches/release24-maint: Lib/test/test_unicode.py Misc/ACKS Misc/NEWS Objects/unicodeobject.c
by georg.brandl 22 Aug '06

22 Aug '06

Author: georg.brandl Date: Tue Aug 22 10:25:33 2006 New Revision: 51466 Modified: python/branches/release24-maint/Lib/test/test_unicode.py python/branches/release24-maint/Misc/ACKS python/branches/release24-maint/Misc/NEWS python/branches/release24-maint/Objects/unicodeobject.c Log: Backport rev 51448: - Patch #1541585: fix buffer overrun when performing repr() on a unicode string in a build with wide unicode (UCS-4) support. Modified: python/branches/release24-maint/Lib/test/test_unicode.py ============================================================================== --- python/branches/release24-maint/Lib/test/test_unicode.py (original) +++ python/branches/release24-maint/Lib/test/test_unicode.py Tue Aug 22 10:25:33 2006 @@ -92,6 +92,10 @@ "\\xfe\\xff'") testrepr = repr(u''.join(map(unichr, xrange(256)))) self.assertEqual(testrepr, latin1repr) + # Test repr works on wide unicode escapes without overflow. + self.assertEqual(repr(u"\U00010000" * 39 + u"\uffff" * 4096), + repr(u"\U00010000" * 39 + u"\uffff" * 4096)) + def test_count(self): string_tests.CommonTest.test_count(self) Modified: python/branches/release24-maint/Misc/ACKS ============================================================================== --- python/branches/release24-maint/Misc/ACKS (original) +++ python/branches/release24-maint/Misc/ACKS Tue Aug 22 10:25:33 2006 @@ -348,6 +348,7 @@ Soren Larsen Piers Lauder Ben Laurie +Simon Law Chris Lawrence Christopher Lee Inyeol Lee Modified: python/branches/release24-maint/Misc/NEWS ============================================================================== --- python/branches/release24-maint/Misc/NEWS (original) +++ python/branches/release24-maint/Misc/NEWS Tue Aug 22 10:25:33 2006 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Patch #1541585: fix buffer overrun when performing repr() on + a unicode string in a build with wide unicode (UCS-4) support. + - Bug #1536786: buffer comparison could emit a RuntimeWarning. - Bug #1535165: fixed a segfault in input() and raw_input() when @@ -33,6 +36,7 @@ - Patch #1488312, Fix memory alignment problem on SPARC in unicode + Extension Modules ----------------- @@ -72,6 +76,7 @@ methods now allow their database parameter to be None as the sleepycat API allows. + Library ------- Modified: python/branches/release24-maint/Objects/unicodeobject.c ============================================================================== --- python/branches/release24-maint/Objects/unicodeobject.c (original) +++ python/branches/release24-maint/Objects/unicodeobject.c Tue Aug 22 10:25:33 2006 @@ -1970,7 +1970,28 @@ static const char *hexdigit = "0123456789abcdef"; - repr = PyString_FromStringAndSize(NULL, 2 + 6*size + 1); + /* Initial allocation is based on the longest-possible unichr + escape. + + In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source + unichr, so in this case it's the longest unichr escape. In + narrow (UTF-16) builds this is five chars per source unichr + since there are two unichrs in the surrogate pair, so in narrow + (UTF-16) builds it's not the longest unichr escape. + + In wide or narrow builds '\uxxxx' is 6 chars per source unichr, + so in the narrow (UTF-16) build case it's the longest unichr + escape. + */ + + repr = PyString_FromStringAndSize(NULL, + 2 +#ifdef Py_UNICODE_WIDE + + 10*size +#else + + 6*size +#endif + + 1); if (repr == NULL) return NULL; @@ -1995,15 +2016,6 @@ #ifdef Py_UNICODE_WIDE /* Map 21-bit characters to '\U00xxxxxx' */ else if (ch >= 0x10000) { - int offset = p - PyString_AS_STRING(repr); - - /* Resize the string if necessary */ - if (offset + 12 > PyString_GET_SIZE(repr)) { - if (_PyString_Resize(&repr, PyString_GET_SIZE(repr) + 100)) - return NULL; - p = PyString_AS_STRING(repr) + offset; - } - *p++ = '\\'; *p++ = 'U'; *p++ = hexdigit[(ch >> 28) & 0x0000000F]; @@ -2016,8 +2028,8 @@ *p++ = hexdigit[ch & 0x0000000F]; continue; } -#endif - /* Map UTF-16 surrogate pairs to Unicode \UXXXXXXXX escapes */ +#else + /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */ else if (ch >= 0xD800 && ch < 0xDC00) { Py_UNICODE ch2; Py_UCS4 ucs; @@ -2042,6 +2054,7 @@ s--; size++; } +#endif /* Map 16-bit characters to '\uxxxx' */ if (ch >= 256) {

1 0

r51461 - python/trunk/Doc/lib/libuuid.tex
by anthony.baxter 22 Aug '06

22 Aug '06

Author: anthony.baxter Date: Tue Aug 22 09:36:59 2006 New Revision: 51461 Modified: python/trunk/Doc/lib/libuuid.tex Log: patch for documentation for recent uuid changes (from ping) Modified: python/trunk/Doc/lib/libuuid.tex ============================================================================== --- python/trunk/Doc/lib/libuuid.tex (original) +++ python/trunk/Doc/lib/libuuid.tex Tue Aug 22 09:36:59 2006 @@ -18,20 +18,11 @@ network address. \function{uuid4()} creates a random UUID. \begin{classdesc}{UUID}{\optional{hex\optional{, bytes\optional{, -fields\optional{, int\optional{, version}}}}}} - -%Instances of the UUID class represent UUIDs as specified in RFC 4122. -%UUID objects are immutable, hashable, and usable as dictionary keys. -%Converting a UUID to a string with str() yields something in the form -%'12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts -%four possible forms: a similar string of hexadecimal digits, or a -%string of 16 raw bytes as an argument named 'bytes', or a tuple of -%six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and -%48-bit values respectively) as an argument named 'fields', or a single -%128-bit integer as an argument named 'int'. +bytes_le\optional{, fields\optional{, int\optional{, version}}}}}}} Create a UUID from either a string of 32 hexadecimal digits, -a string of 16 bytes as the \var{bytes} argument, a tuple of six +a string of 16 bytes as the \var{bytes} argument, a string of 16 bytes +in little-endian order as the \var{bytes_le} argument, a tuple of six integers (32-bit \var{time_low}, 16-bit \var{time_mid}, 16-bit \var{time_hi_version}, 8-bit \var{clock_seq_hi_variant}, 8-bit \var{clock_seq_low}, 48-bit \var{node}) @@ -45,22 +36,31 @@ UUID('12345678123456781234567812345678') UUID('urn:uuid:12345678-1234-5678-1234-567812345678') UUID(bytes='\x12\x34\x56\x78'*4) +UUID(bytes_le='\x78\x56\x34\x12\x34\x12\x78\x56' + + '\x12\x34\x56\x78\x12\x34\x56\x78') UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) UUID(int=0x12345678123456781234567812345678) \end{verbatim} -Exactly one of \var{hex}, \var{bytes}, \var{fields}, or \var{int} must +Exactly one of \var{hex}, \var{bytes}, \var{bytes_le}, \var{fields}, +or \var{int} must be given. The \var{version} argument is optional; if given, the resulting UUID will have its variant and version number set according to RFC 4122, overriding bits in the given \var{hex}, \var{bytes}, -\var{fields}, or \var{int}. +\var{bytes_le}, \var{fields}, or \var{int}. \end{classdesc} \class{UUID} instances have these read-only attributes: \begin{memberdesc}{bytes} -The UUID as a 16-byte string. +The UUID as a 16-byte string (containing the six +integer fields in big-endian byte order). +\end{memberdesc} + +\begin{memberdesc}{bytes_le} +The UUID as a 16-byte string (with \var{time_low}, \var{time_mid}, +and \var{time_hi_version} in little-endian byte order). \end{memberdesc} \begin{memberdesc}{fields}

1 0

r51460 - python/branches/release25-maint/Doc/lib/libuuid.tex
by anthony.baxter 22 Aug '06

22 Aug '06

Author: anthony.baxter Date: Tue Aug 22 09:36:06 2006 New Revision: 51460 Modified: python/branches/release25-maint/Doc/lib/libuuid.tex Log: patch for documentation for recent uuid changes (from ping) Modified: python/branches/release25-maint/Doc/lib/libuuid.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/libuuid.tex (original) +++ python/branches/release25-maint/Doc/lib/libuuid.tex Tue Aug 22 09:36:06 2006 @@ -18,20 +18,11 @@ network address. \function{uuid4()} creates a random UUID. \begin{classdesc}{UUID}{\optional{hex\optional{, bytes\optional{, -fields\optional{, int\optional{, version}}}}}} - -%Instances of the UUID class represent UUIDs as specified in RFC 4122. -%UUID objects are immutable, hashable, and usable as dictionary keys. -%Converting a UUID to a string with str() yields something in the form -%'12345678-1234-1234-1234-123456789abc'. The UUID constructor accepts -%four possible forms: a similar string of hexadecimal digits, or a -%string of 16 raw bytes as an argument named 'bytes', or a tuple of -%six integer fields (with 32-bit, 16-bit, 16-bit, 8-bit, 8-bit, and -%48-bit values respectively) as an argument named 'fields', or a single -%128-bit integer as an argument named 'int'. +bytes_le\optional{, fields\optional{, int\optional{, version}}}}}}} Create a UUID from either a string of 32 hexadecimal digits, -a string of 16 bytes as the \var{bytes} argument, a tuple of six +a string of 16 bytes as the \var{bytes} argument, a string of 16 bytes +in little-endian order as the \var{bytes_le} argument, a tuple of six integers (32-bit \var{time_low}, 16-bit \var{time_mid}, 16-bit \var{time_hi_version}, 8-bit \var{clock_seq_hi_variant}, 8-bit \var{clock_seq_low}, 48-bit \var{node}) @@ -45,22 +36,31 @@ UUID('12345678123456781234567812345678') UUID('urn:uuid:12345678-1234-5678-1234-567812345678') UUID(bytes='\x12\x34\x56\x78'*4) +UUID(bytes_le='\x78\x56\x34\x12\x34\x12\x78\x56' + + '\x12\x34\x56\x78\x12\x34\x56\x78') UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678)) UUID(int=0x12345678123456781234567812345678) \end{verbatim} -Exactly one of \var{hex}, \var{bytes}, \var{fields}, or \var{int} must +Exactly one of \var{hex}, \var{bytes}, \var{bytes_le}, \var{fields}, +or \var{int} must be given. The \var{version} argument is optional; if given, the resulting UUID will have its variant and version number set according to RFC 4122, overriding bits in the given \var{hex}, \var{bytes}, -\var{fields}, or \var{int}. +\var{bytes_le}, \var{fields}, or \var{int}. \end{classdesc} \class{UUID} instances have these read-only attributes: \begin{memberdesc}{bytes} -The UUID as a 16-byte string. +The UUID as a 16-byte string (containing the six +integer fields in big-endian byte order). +\end{memberdesc} + +\begin{memberdesc}{bytes_le} +The UUID as a 16-byte string (with \var{time_low}, \var{time_mid}, +and \var{time_hi_version} in little-endian byte order). \end{memberdesc} \begin{memberdesc}{fields}

1 0

buildbot warnings in hppa Ubuntu dapper trunk
by buildbot＠python.org 21 Aug '06

21 Aug '06

The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk… Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot

1 0

buildbot warnings in x86 Ubuntu dapper (icc) trunk
by buildbot＠python.org 21 Aug '06

21 Aug '06

The Buildbot has detected a new failure of x86 Ubuntu dapper (icc) trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520Ubuntu%2520dapper%2520%2528i… Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings failed slave lost sincerely, -The Buildbot

1 0

buildbot warnings in alpha Debian trunk
by buildbot＠python.org 21 Aug '06

21 Aug '06

The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/524 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build Had Warnings: warnings test sincerely, -The Buildbot

1 0