Mailman 3 September 2011 - Python-checkins

cpython: Complete What's New in 3.3 about PEP 393
by victor.stinner Sept. 29, 2011

Sept. 29, 2011

http://hg.python.org/cpython/rev/991ef9253263 changeset: 72515:991ef9253263 user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Thu Sep 29 02:56:16 2011 +0200 summary: Complete What's New in 3.3 about PEP 393 files: Doc/whatsnew/3.3.rst | 28 ++++++++++++++++++++++++++++ 1 files changed, 28 insertions(+), 0 deletions(-) diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -65,6 +65,28 @@ either ``0xFFFF`` or ``0x10FFFF`` for backward compatibility, and it should not be used with the new Unicode API (see :issue:`13054`). +* Non-BMP characters (U+10000-U+10FFFF range) are no more special cases. + ``'\U0010FFFF'[0]`` is now ``'\U0010FFFF'`` on any platform, instead of + ``'\uDFFF'`` on narrow build or ``'\U0010FFFF'`` on wide build. And + ``len('\U0010FFFF')`` is now ``1`` on any platform, instead of ``2`` on + narrow build or ``1`` on wide build. More generally, most bugs related to + non-BMP characters are now fixed. For example, :func:`unicodedata.normalize` + handles correctly non-BMP characters on all platforms. + +* The storage of Unicode string is now adapted on the content of the string. + Pure ASCII and Latin1 strings (U+0000-U+00FF) use 1 byte per character, BMP + strings (U+0000-U+FFFF) use 2 bytes per character, and non-BMP characters + (U+10000-U+10FFFF range) use 4 bytes per characters. The memory usage of + Python 3.3 is two to three times smaller than Python 3.2, and a little bit + better than Python 2.7, on a `Django benchmark + <http://mail.python.org/pipermail/python-dev/2011-September/113714.html>`_. + +* The PEP 393 is fully backward compatible. The legacy API should remain + available at least five years. Applications using the legacy API will not + fully benefit of the memory reduction, or worse may use a little bit more + memory, because Python may have to maintain two versions of each string (in + the legacy format and in the new efficient storage). + Other Language Changes ====================== @@ -334,3 +356,9 @@ .. Issue #10998: -Q command-line flags are related artifacts have been removed. Code checking sys.flags.division_warning will need updating. Contributed by Éric Araujo. + +* :pep:`393`: The :c:type:`Py_UNICODE` type and all functions using this type + are deprecated. To fully benefit of the memory footprint reduction provided + by the PEP 393, you have to convert your code to the new Unicode API. Read + the porting guide: XXX. + -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Move UCS4-specific tests with the "normal" tests.
by ezio.melotti Sept. 29, 2011

Sept. 29, 2011

http://hg.python.org/cpython/rev/9f41acdb3150 changeset: 72514:9f41acdb3150 parent: 72512:36fc514de7f0 user: Ezio Melotti <ezio.melotti(a)gmail.com> date: Thu Sep 29 03:14:56 2011 +0300 summary: Move UCS4-specific tests with the "normal" tests. files: Lib/test/test_unicodedata.py | 12 ++++-------- 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/Lib/test/test_unicodedata.py b/Lib/test/test_unicodedata.py --- a/Lib/test/test_unicodedata.py +++ b/Lib/test/test_unicodedata.py @@ -108,6 +108,7 @@ self.assertEqual(self.db.digit('\u215b', None), None) self.assertEqual(self.db.digit('\u2468'), 9) self.assertEqual(self.db.digit('\U00020000', None), None) + self.assertEqual(self.db.digit('\U0001D7FD'), 7) self.assertRaises(TypeError, self.db.digit) self.assertRaises(TypeError, self.db.digit, 'xx') @@ -120,6 +121,7 @@ self.assertEqual(self.db.numeric('\u2468'), 9.0) self.assertEqual(self.db.numeric('\ua627'), 7.0) self.assertEqual(self.db.numeric('\U00020000', None), None) + self.assertEqual(self.db.numeric('\U0001012A'), 9000) self.assertRaises(TypeError, self.db.numeric) self.assertRaises(TypeError, self.db.numeric, 'xx') @@ -131,6 +133,7 @@ self.assertEqual(self.db.decimal('\u215b', None), None) self.assertEqual(self.db.decimal('\u2468', None), None) self.assertEqual(self.db.decimal('\U00020000', None), None) + self.assertEqual(self.db.decimal('\U0001D7FD'), 7) self.assertRaises(TypeError, self.db.decimal) self.assertRaises(TypeError, self.db.decimal, 'xx') @@ -141,6 +144,7 @@ self.assertEqual(self.db.category('a'), 'Ll') self.assertEqual(self.db.category('A'), 'Lu') self.assertEqual(self.db.category('\U00020000'), 'Lo') + self.assertEqual(self.db.category('\U0001012A'), 'No') self.assertRaises(TypeError, self.db.category) self.assertRaises(TypeError, self.db.category, 'xx') @@ -308,14 +312,6 @@ self.assertEqual(len(lines), 1, r"\u%.4x should not be a linebreak" % i) - def test_UCS4(self): - # unicodedata should work with code points outside the BMP - # even on a narrow Unicode build - self.assertEqual(self.db.category("\U0001012A"), "No") - self.assertEqual(self.db.numeric("\U0001012A"), 9000) - self.assertEqual(self.db.decimal("\U0001D7FD"), 7) - self.assertEqual(self.db.digit("\U0001D7FD"), 7) - def test_main(): test.support.run_unittest( UnicodeMiscTest, -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Enhance Py_ARRAY_LENGTH(): fail at build time if the argument is not an array
by victor.stinner Sept. 29, 2011

Sept. 29, 2011

http://hg.python.org/cpython/rev/36fc514de7f0 changeset: 72512:36fc514de7f0 user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Thu Sep 29 01:12:24 2011 +0200 summary: Enhance Py_ARRAY_LENGTH(): fail at build time if the argument is not an array Move other various macros to pymcacro.h Thanks Rusty Russell for having written these amazing C macros! files: Include/Python.h | 19 +-------- Include/pymacro.h | 57 +++++++++++++++++++++++++++ Makefile.pre.in | 3 +- Misc/ACKS | 1 + PCbuild/pythoncore.vcproj | 4 + 5 files changed, 65 insertions(+), 19 deletions(-) diff --git a/Include/Python.h b/Include/Python.h --- a/Include/Python.h +++ b/Include/Python.h @@ -48,6 +48,7 @@ #include <assert.h> #include "pyport.h" +#include "pymacro.h" #include "pyatomic.h" @@ -126,24 +127,6 @@ #include "pystrcmp.h" #include "dtoa.h" #include "fileutils.h" - -/* Argument must be a char or an int in [-128, 127] or [0, 255]. */ -#define Py_CHARMASK(c) ((unsigned char)((c) & 0xff)) - #include "pyfpe.h" -/* Define macros for inline documentation. */ -#define PyDoc_VAR(name) static char name[] -#define PyDoc_STRVAR(name,str) PyDoc_VAR(name) = PyDoc_STR(str) -#ifdef WITH_DOC_STRINGS -#define PyDoc_STR(str) str -#else -#define PyDoc_STR(str) "" -#endif - -#define Py_ARRAY_LENGTH(array) (sizeof(array) / sizeof((array)[0])) - -#define Py_MIN(x, y) (((x) > (y)) ? (y) : (x)) -#define Py_MAX(x, y) (((x) > (y)) ? (x) : (y)) - #endif /* !Py_PYTHON_H */ diff --git a/Include/pymacro.h b/Include/pymacro.h new file mode 100644 --- /dev/null +++ b/Include/pymacro.h @@ -0,0 +1,57 @@ +#ifndef Py_PYMACRO_H +#define Py_PYMACRO_H + +#define Py_MIN(x, y) (((x) > (y)) ? (y) : (x)) +#define Py_MAX(x, y) (((x) > (y)) ? (x) : (y)) + +/* Argument must be a char or an int in [-128, 127] or [0, 255]. */ +#define Py_CHARMASK(c) ((unsigned char)((c) & 0xff)) + + +/* Assert a build-time dependency, as an expression. + + Your compile will fail if the condition isn't true, or can't be evaluated + by the compiler. This can be used in an expression: its value is 0. + + Example: + + #define foo_to_char(foo) \ + ((char *)(foo) \ + + Py_BUILD_ASSERT(offsetof(struct foo, string) == 0)) + + Written by Rusty Russell, public domain. */ +#define Py_BUILD_ASSERT(cond) \ + (sizeof(char [1 - 2*!(cond)]) - 1) + +#if defined(__GNUC__) +/* Two gcc extensions. + &a[0] degrades to a pointer: a different type from an array */ +#define _Py_ARRAY_LENGTH_CHECK(array) \ + Py_BUILD_ASSERT(!__builtin_types_compatible_p(typeof(array), \ + typeof(&(array)[0]))) +#else +#define _Py_ARRAY_LENGTH_CHECK(array) 0 +#endif + + +/* Get the number of elements in a visible array + + This does not work on pointers, or arrays declared as [], or function + parameters. With correct compiler support, such usage will cause a build + error (see Py_BUILD_ASSERT). + + Written by Rusty Russell, public domain. */ +#define Py_ARRAY_LENGTH(array) \ + (sizeof(array) / sizeof((array)[0]) + _Py_ARRAY_LENGTH_CHECK(array)) + + +/* Define macros for inline documentation. */ +#define PyDoc_VAR(name) static char name[] +#define PyDoc_STRVAR(name,str) PyDoc_VAR(name) = PyDoc_STR(str) +#ifdef WITH_DOC_STRINGS +#define PyDoc_STR(str) str +#else +#define PyDoc_STR(str) "" +#endif + +#endif /* Py_PYMACRO_H */ diff --git a/Makefile.pre.in b/Makefile.pre.in --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -493,7 +493,7 @@ -install_name $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK) \ -compatibility_version $(VERSION) \ -current_version $(VERSION) \ - -framework CoreFoundation $(LIBS); + -framework CoreFoundation $(LIBS); $(INSTALL) -d -m $(DIRMODE) \ $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/Resources/English.lproj $(INSTALL_DATA) $(RESSRCDIR)/Info.plist \ @@ -718,6 +718,7 @@ Include/pyfpe.h \ Include/pymath.h \ Include/pygetopt.h \ + Include/pymacro.h \ Include/pymem.h \ Include/pyport.h \ Include/pystate.h \ diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -825,6 +825,7 @@ Jeff Rush Sam Rushing Mark Russell +Rusty Russell Nick Russo Patrick Sabin Sébastien Sablé diff --git a/PCbuild/pythoncore.vcproj b/PCbuild/pythoncore.vcproj --- a/PCbuild/pythoncore.vcproj +++ b/PCbuild/pythoncore.vcproj @@ -887,6 +887,10 @@ > </File> <File + RelativePath="..\Include\pymacro.h" + > + </File> + <File RelativePath="..\Include\pymem.h" > </File> -- Repository URL: http://hg.python.org/cpython

2 1

cpython (2.7): Doc fix. Mathematically correct sentence.
by senthil.kumaran Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/f612b783bc17 changeset: 72513:f612b783bc17 branch: 2.7 parent: 72472:1fb5b0cc6367 user: Senthil Kumaran <senthil(a)uthcode.com> date: Thu Sep 29 07:52:46 2011 +0800 summary: Doc fix. Mathematically correct sentence. files: Doc/tutorial/datastructures.rst | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/tutorial/datastructures.rst b/Doc/tutorial/datastructures.rst --- a/Doc/tutorial/datastructures.rst +++ b/Doc/tutorial/datastructures.rst @@ -170,8 +170,8 @@ ``filter(function, sequence)`` returns a sequence consisting of those items from the sequence for which ``function(item)`` is true. If *sequence* is a :class:`string` or :class:`tuple`, the result will be of the same type; -otherwise, it is always a :class:`list`. For example, to compute primes up -to 25:: +otherwise, it is always a :class:`list`. For example, to compute a sequence of +numbers not divisible by 2 and 3:: >>> def f(x): return x % 2 != 0 and x % 3 != 0 ... -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Move code related to compile from Python.h to compile.h
by victor.stinner Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/775977c601cd changeset: 72511:775977c601cd user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Thu Sep 29 01:04:08 2011 +0200 summary: Move code related to compile from Python.h to compile.h files: Include/Python.h | 19 ------------------- Include/compile.h | 14 ++++++++++++-- 2 files changed, 12 insertions(+), 21 deletions(-) diff --git a/Include/Python.h b/Include/Python.h --- a/Include/Python.h +++ b/Include/Python.h @@ -127,30 +127,11 @@ #include "dtoa.h" #include "fileutils.h" -#ifdef __cplusplus -extern "C" { -#endif - -/* _Py_Mangle is defined in compile.c */ -#ifndef Py_LIMITED_API -PyAPI_FUNC(PyObject*) _Py_Mangle(PyObject *p, PyObject *name); -#endif - -#ifdef __cplusplus -} -#endif - /* Argument must be a char or an int in [-128, 127] or [0, 255]. */ #define Py_CHARMASK(c) ((unsigned char)((c) & 0xff)) #include "pyfpe.h" -/* These definitions must match corresponding definitions in graminit.h. - There's code in compile.c that checks that they are the same. */ -#define Py_single_input 256 -#define Py_file_input 257 -#define Py_eval_input 258 - /* Define macros for inline documentation. */ #define PyDoc_VAR(name) static char name[] #define PyDoc_STRVAR(name,str) PyDoc_VAR(name) = PyDoc_STR(str) diff --git a/Include/compile.h b/Include/compile.h --- a/Include/compile.h +++ b/Include/compile.h @@ -1,7 +1,7 @@ -#ifndef Py_LIMITED_API #ifndef Py_COMPILE_H #define Py_COMPILE_H +#ifndef Py_LIMITED_API #include "code.h" #ifdef __cplusplus @@ -38,9 +38,19 @@ PyArena *arena); PyAPI_FUNC(PyFutureFeatures *) PyFuture_FromAST(struct _mod *, const char *); +/* _Py_Mangle is defined in compile.c */ +PyAPI_FUNC(PyObject*) _Py_Mangle(PyObject *p, PyObject *name); #ifdef __cplusplus } #endif + +#endif /* !Py_LIMITED_API */ + +/* These definitions must match corresponding definitions in graminit.h. + There's code in compile.c that checks that they are the same. */ +#define Py_single_input 256 +#define Py_file_input 257 +#define Py_eval_input 258 + #endif /* !Py_COMPILE_H */ -#endif /* !Py_LIMITED_API */ -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Use the new Py_ARRAY_LENGTH macro
by victor.stinner Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/b534b56379cb changeset: 72510:b534b56379cb user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Thu Sep 29 00:42:28 2011 +0200 summary: Use the new Py_ARRAY_LENGTH macro files: Modules/_testcapimodule.c | 2 +- Modules/faulthandler.c | 2 +- Modules/mathmodule.c | 2 +- Modules/ossaudiodev.c | 6 +++--- Modules/posixmodule.c | 10 +++++----- Modules/socketmodule.c | 4 ++-- Modules/unicodedata.c | 2 +- Objects/longobject.c | 3 +-- Objects/typeobject.c | 4 ++-- Objects/unicodeobject.c | 2 +- PC/frozen_dllmain.c | 4 ++-- PC/getpathp.c | 6 +++--- Python/codecs.c | 2 +- Python/dynload_aix.c | 3 +-- Python/sysmodule.c | 2 +- 15 files changed, 26 insertions(+), 28 deletions(-) diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -1593,7 +1593,7 @@ {-0xfffffffL, 28, -1}}; int i; - for (i = 0; i < sizeof(testcases) / sizeof(struct triple); ++i) { + for (i = 0; i < Py_ARRAY_LENGTH(testcases); ++i) { PyObject *plong = PyLong_FromLong(testcases[i].input); size_t nbits = _PyLong_NumBits(plong); int sign = _PyLong_Sign(plong); diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -112,7 +112,7 @@ {SIGSEGV, 0, "Segmentation fault", } }; static const unsigned char faulthandler_nsignals = \ - sizeof(faulthandler_handlers) / sizeof(faulthandler_handlers[0]); + Py_ARRAY_LENGTH(faulthandler_handlers); #ifdef HAVE_SIGALTSTACK static stack_t stack; diff --git a/Modules/mathmodule.c b/Modules/mathmodule.c --- a/Modules/mathmodule.c +++ b/Modules/mathmodule.c @@ -1435,7 +1435,7 @@ } /* use lookup table if x is small */ - if (x < (long)(sizeof(SmallFactorials)/sizeof(SmallFactorials[0]))) + if (x < (long)Py_ARRAY_LENGTH(SmallFactorials)) return PyLong_FromUnsignedLong(SmallFactorials[x]); /* else express in the form odd_part * 2**two_valuation, and compute as diff --git a/Modules/ossaudiodev.c b/Modules/ossaudiodev.c --- a/Modules/ossaudiodev.c +++ b/Modules/ossaudiodev.c @@ -530,7 +530,7 @@ return self; } -static PyObject * +static PyObject * oss_exit(PyObject *self, PyObject *unused) { PyObject *ret = PyObject_CallMethod(self, "close", NULL); @@ -1061,8 +1061,8 @@ int num_controls; int i; - num_controls = sizeof(control_labels) / sizeof(control_labels[0]); - assert(num_controls == sizeof(control_names) / sizeof(control_names[0])); + num_controls = Py_ARRAY_LENGTH(control_labels); + assert(num_controls == Py_ARRAY_LENGTH(control_names)); labels = PyList_New(num_controls); names = PyList_New(num_controls); diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -2895,9 +2895,9 @@ DWORD result; PyObject *v; result = GetFullPathNameW(wpath, - sizeof(woutbuf)/sizeof(woutbuf[0]), + Py_ARRAY_LENGTH(woutbuf), woutbuf, &wtemp); - if (result > sizeof(woutbuf)/sizeof(woutbuf[0])) { + if (result > Py_ARRAY_LENGTH(woutbuf)) { woutbufp = malloc(result * sizeof(Py_UNICODE)); if (!woutbufp) return PyErr_NoMemory(); @@ -2920,7 +2920,7 @@ PyUnicode_FSConverter, &opath)) return NULL; path = PyBytes_AsString(opath); - if (!GetFullPathName(path, sizeof(outbuf)/sizeof(outbuf[0]), + if (!GetFullPathName(path, Py_ARRAY_LENGTH(outbuf), outbuf, &temp)) { win32_error("GetFullPathName", path); Py_DECREF(opath); @@ -4903,7 +4903,7 @@ cpu_set_repr(Py_cpu_set *set) { return PyUnicode_FromFormat("<cpu_set with %li entries>", set->ncpus); -} +} static Py_ssize_t cpu_set_len(Py_cpu_set *set) @@ -5656,7 +5656,7 @@ PyObject *result = NULL; #ifdef MS_WINDOWS wchar_t user_name[UNLEN + 1]; - DWORD num_chars = sizeof(user_name)/sizeof(user_name[0]); + DWORD num_chars = Py_ARRAY_LENGTH(user_name); if (GetUserNameW(user_name, &num_chars)) { /* num_chars is the number of unicode chars plus null terminator */ diff --git a/Modules/socketmodule.c b/Modules/socketmodule.c --- a/Modules/socketmodule.c +++ b/Modules/socketmodule.c @@ -3812,7 +3812,7 @@ version of the hostname, whereas we need a Unicode string. Otherwise, gethostname apparently also returns the DNS name. */ wchar_t buf[MAX_COMPUTERNAME_LENGTH + 1]; - DWORD size = sizeof(buf) / sizeof(wchar_t); + DWORD size = Py_ARRAY_LENGTH(buf); PyObject *result; if (!GetComputerNameExW(ComputerNamePhysicalDnsHostname, buf, &size)) { if (GetLastError() == ERROR_MORE_DATA) { @@ -6281,7 +6281,7 @@ DWORD codes[] = {SIO_RCVALL, SIO_KEEPALIVE_VALS}; const char *names[] = {"SIO_RCVALL", "SIO_KEEPALIVE_VALS"}; int i; - for(i = 0; i<sizeof(codes)/sizeof(*codes); ++i) { + for(i = 0; i<Py_ARRAY_LENGTH(codes); ++i) { PyObject *tmp; tmp = PyLong_FromUnsignedLong(codes[i]); if (tmp == NULL) diff --git a/Modules/unicodedata.c b/Modules/unicodedata.c --- a/Modules/unicodedata.c +++ b/Modules/unicodedata.c @@ -439,7 +439,7 @@ from Tools/unicode/makeunicodedata.py, it should not be possible to overflow decomp_prefix. */ prefix_index = decomp_data[index] & 255; - assert(prefix_index < (sizeof(decomp_prefix)/sizeof(*decomp_prefix))); + assert(prefix_index < Py_ARRAY_LENGTH(decomp_prefix)); /* copy prefix */ i = strlen(decomp_prefix[prefix_index]); diff --git a/Objects/longobject.c b/Objects/longobject.c --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -2444,8 +2444,7 @@ break; } } - assert(1 <= x_size && - x_size <= (Py_ssize_t)(sizeof(x_digits)/sizeof(digit))); + assert(1 <= x_size && x_size <= (Py_ssize_t)Py_ARRAY_LENGTH(x_digits)); /* Round, and convert to double. */ x_digits[0] += half_even_correction[x_digits[0] & 7]; diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -2355,7 +2355,7 @@ res->ht_type.tp_flags = spec->flags | Py_TPFLAGS_HEAPTYPE; for (slot = spec->slots; slot->slot; slot++) { - if (slot->slot >= sizeof(slotoffsets)/sizeof(slotoffsets[0])) { + if (slot->slot >= Py_ARRAY_LENGTH(slotoffsets)) { PyErr_SetString(PyExc_RuntimeError, "invalid slot offset"); goto fail; } @@ -2583,7 +2583,7 @@ return PyDict_New(); } -/* +/* Merge the __dict__ of aclass into dict, and recursively also all the __dict__s of aclass's base classes. The order of merging isn't defined, as it's expected that only the final set of dict keys is diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -12552,7 +12552,7 @@ /* initialize the linebreak bloom filter */ bloom_linebreak = make_bloom_mask( PyUnicode_2BYTE_KIND, linebreak, - sizeof(linebreak) / sizeof(linebreak[0])); + Py_ARRAY_LENGTH(linebreak)); PyType_Ready(&EncodingMapType); } diff --git a/PC/frozen_dllmain.c b/PC/frozen_dllmain.c --- a/PC/frozen_dllmain.c +++ b/PC/frozen_dllmain.c @@ -77,7 +77,7 @@ { // Must go backwards char **modName; - for (modName = possibleModules+(sizeof(possibleModules) / sizeof(char *))-2; + for (modName = possibleModules+Py_ARRAY_LENGTH(possibleModules)-2; modName >= possibleModules; *modName--) { /* printf("Terminating '%s'\n", *modName);*/ @@ -103,7 +103,7 @@ { // Must go backwards char **modName; - for (modName = possibleModules+(sizeof(possibleModules) / sizeof(char *))-2; + for (modName = possibleModules+Py_ARRAY_LENGTH(possibleModules)-2; modName >= possibleModules; *modName--) CallModuleDllMain(*modName, DLL_PROCESS_DETACH); diff --git a/PC/getpathp.c b/PC/getpathp.c --- a/PC/getpathp.c +++ b/PC/getpathp.c @@ -251,7 +251,7 @@ if (keyBuf==NULL) goto done; memcpy(keyBufPtr, keyPrefix, sizeof(keyPrefix)-sizeof(WCHAR)); - keyBufPtr += sizeof(keyPrefix)/sizeof(WCHAR) - 1; + keyBufPtr += Py_ARRAY_LENGTH(keyPrefix) - 1; mbstowcs(keyBufPtr, PyWin_DLLVersionString, versionLen); keyBufPtr += versionLen; /* NULL comes with this one! */ @@ -708,8 +708,8 @@ return progpath; } -/* Load python3.dll before loading any extension module that might refer - to it. That way, we can be sure that always the python3.dll corresponding +/* Load python3.dll before loading any extension module that might refer + to it. That way, we can be sure that always the python3.dll corresponding to this python DLL is loaded, not a python3.dll that might be on the path by chance. Return whether the DLL was found. diff --git a/Python/codecs.c b/Python/codecs.c --- a/Python/codecs.c +++ b/Python/codecs.c @@ -1044,7 +1044,7 @@ interp->codec_error_registry = PyDict_New(); if (interp->codec_error_registry) { - for (i = 0; i < sizeof(methods)/sizeof(methods[0]); ++i) { + for (i = 0; i < Py_ARRAY_LENGTH(methods); ++i) { PyObject *func = PyCFunction_New(&methods[i].def, NULL); int res; if (!func) diff --git a/Python/dynload_aix.c b/Python/dynload_aix.c --- a/Python/dynload_aix.c +++ b/Python/dynload_aix.c @@ -129,7 +129,6 @@ {L_ERROR_ERRNO, NULL} }; -#define LOAD_ERRTAB_LEN (sizeof(load_errtab)/sizeof(load_errtab[0])) #define ERRBUF_APPEND(s) strncat(errbuf, s, sizeof(errbuf)-strlen(errbuf)-1) PyOS_snprintf(errbuf, sizeof(errbuf), "from module %.200s ", pathname); @@ -140,7 +139,7 @@ } for(i = 0; message[i] && *message[i]; i++) { int nerr = atoi(message[i]); - for (j=0; j<LOAD_ERRTAB_LEN ; j++) { + for (j=0; j < Py_ARRAY_LENGTH(load_errtab); j++) { if (nerr == load_errtab[j].errNo && load_errtab[j].errstr) ERRBUF_APPEND(load_errtab[j].errstr); } diff --git a/Python/sysmodule.c b/Python/sysmodule.c --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -1771,7 +1771,7 @@ the argument must be the full path anyway. */ wchar_t *ptemp; if (GetFullPathNameW(argv0, - sizeof(fullpath)/sizeof(fullpath[0]), + Py_ARRAY_LENGTH(fullpath), fullpath, &ptemp)) { argv0 = fullpath; -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Fix 'c' format of PyUnicode_Format()
by victor.stinner Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/876a36feb508 changeset: 72509:876a36feb508 user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Thu Sep 29 00:39:24 2011 +0200 summary: Fix 'c' format of PyUnicode_Format() formatbuf is now an array of Py_UCS4, not of Py_UNICODE files: Objects/unicodeobject.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -12254,7 +12254,7 @@ case 'c': pbuf = formatbuf; kind = PyUnicode_4BYTE_KIND; - len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v); + len = formatchar(pbuf, Py_ARRAY_LENGTH(formatbuf), v); if (len < 0) goto onError; break; -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Oops, fix my previous commit: unicode => to
by victor.stinner Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/8d4252f0bf14 changeset: 72508:8d4252f0bf14 user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Thu Sep 29 00:16:58 2011 +0200 summary: Oops, fix my previous commit: unicode => to files: Objects/unicodeobject.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -642,7 +642,7 @@ "Cannot modify a string having more than 1 reference"); return -1; } - _PyUnicode_DIRTY(unicode); + _PyUnicode_DIRTY(to); from_kind = PyUnicode_KIND(from); to_kind = PyUnicode_KIND(to); @@ -4881,7 +4881,7 @@ PyObject * PyUnicode_DecodeUnicodeEscape(const char *s, Py_ssize_t size, - const char *errors) + const char *errors) { const char *starts = s; Py_ssize_t startinpos; -- Repository URL: http://hg.python.org/cpython

1 0

cpython: PyUnicode_CopyCharacters() marks the string as dirty (reset the hash)
by victor.stinner Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/29ef50337bdb changeset: 72507:29ef50337bdb user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Wed Sep 28 23:59:20 2011 +0200 summary: PyUnicode_CopyCharacters() marks the string as dirty (reset the hash) files: Objects/unicodeobject.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -117,6 +117,9 @@ (assert(PyUnicode_Check(op)), \ ((PyASCIIObject *)(op))->length) +/* The Unicode string has been modified: reset the hash */ +#define _PyUnicode_DIRTY(op) do { _PyUnicode_HASH(op) = -1; } while (0) + /* This dictionary holds all interned unicode strings. Note that references to strings in this dictionary are *not* counted in the string's ob_refcnt. @@ -356,7 +359,7 @@ _PyUnicode_STATE(unicode).interned = _PyUnicode_STATE(unicode).interned; _PyUnicode_STATE(unicode).kind = PyUnicode_WCHAR_KIND; } - _PyUnicode_HASH(unicode) = -1; + _PyUnicode_DIRTY(unicode); return 0; } @@ -639,6 +642,7 @@ "Cannot modify a string having more than 1 reference"); return -1; } + _PyUnicode_DIRTY(unicode); from_kind = PyUnicode_KIND(from); to_kind = PyUnicode_KIND(to); -- Repository URL: http://hg.python.org/cpython

1 0

cpython: PyUnicode_CopyCharacters() fails if 'to' has more than 1 reference
by victor.stinner Sept. 28, 2011

Sept. 28, 2011

http://hg.python.org/cpython/rev/9f0e1f44ebac changeset: 72506:9f0e1f44ebac user: Victor Stinner <victor.stinner(a)haypocalc.com> date: Wed Sep 28 23:54:59 2011 +0200 summary: PyUnicode_CopyCharacters() fails if 'to' has more than 1 reference files: Include/unicodeobject.h | 3 ++- Objects/unicodeobject.c | 8 ++++++++ 2 files changed, 10 insertions(+), 1 deletions(-) diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -522,7 +522,8 @@ character conversion when necessary and falls back to memcpy if possible. Fail if 'to' is smaller than how_many or smaller than len(from)-from_start, - or if kind(from[from_start:from_start+how_many]) > kind(to). + or if kind(from[from_start:from_start+how_many]) > kind(to), or if to has + more than 1 reference. Return the number of written character, or return -1 and raise an exception on error. diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -631,6 +631,14 @@ how_many, to_start, PyUnicode_GET_LENGTH(to)); return -1; } + if (how_many == 0) + return 0; + + if (Py_REFCNT(to) != 1) { + PyErr_SetString(PyExc_ValueError, + "Cannot modify a string having more than 1 reference"); + return -1; + } from_kind = PyUnicode_KIND(from); to_kind = PyUnicode_KIND(to); -- Repository URL: http://hg.python.org/cpython

1 0