Mailman 3 December 2015 - Python-checkins

Benchmark Results for Python 2.7 2015-12-24
by lp_benchmark_robot＠intel.com 24 Dec '15

24 Dec '15

No new revisions. Here are the previous results: Results for project Python 2.7, build date 2015-12-24 03:58:09 +0000 commit: d7b5c2f99a99414eb8698f38444de4b6f5463eb2 previous commit: b87fe3e4a3d87e855e8e640da0d9a99ec3b7bd95 revision date: 2015-12-21 17:43:03 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v2.7.10, with hash 15c95b7d81dcf821daade360741e00714667653f from 2015-05-23 16:02:14+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.45% -0.49% 3.86% 5.58% :-) pybench 0.15% 0.08% 6.33% 3.74% :-( regex_v8 1.05% -0.11% -2.25% 10.71% :-) nbody 0.57% 1.55% 8.71% 1.90% :-) json_dump_v2 0.22% 0.13% 5.52% 9.61% :-( normal_startup 1.86% 0.17% -5.21% 1.90% :-| ssbench 0.07% -0.03% 1.34% 0.77% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results for ssbench are measured in requests/second while all other are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration.

1 0

Benchmark Results for Python Default 2015-12-24
by lp_benchmark_robot＠intel.com 24 Dec '15

24 Dec '15

Results for project Python default, build date 2015-12-24 03:04:45 +0000 commit: 92760d2edc9ec8407f844320bfe4988e8ca8e1de previous commit: dcf9e9ae53936e779b99b2d120f2f004f902bc62 revision date: 2015-12-24 02:19:53 +0000 environment: Haswell-EP cpu: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz 2x18 cores, stepping 2, LLC 45 MB mem: 128 GB os: CentOS 7.1 kernel: Linux 3.10.0-229.4.2.el7.x86_64 Baseline results were generated using release v3.4.3, with hash b4cbecbc0781e89a309d03b60a1f75f8499250e6 from 2015-02-25 12:15:33+00:00 ---------------------------------------------------------------------------------- benchmark relative change since change since current rev run std_dev* last run baseline with PGO ---------------------------------------------------------------------------------- :-) django_v2 0.43% -0.18% 10.53% 14.90% :-| pybench 0.12% -0.07% -1.53% 5.96% :-( regex_v8 2.77% -0.01% -3.92% 4.65% :-| nbody 0.61% 0.05% 1.55% 3.24% :-| json_dump_v2 0.27% 0.12% -1.42% 11.07% :-| normal_startup 1.07% -0.00% -1.90% 5.15% ---------------------------------------------------------------------------------- * Relative Standard Deviation (Standard Deviation/Average) Note: Benchmark results are measured in seconds. Our lab does a nightly source pull and build of the Python project and measures performance changes against the previous stable version and the previous nightly measurement. This is provided as a service to the community so that quality issues with current hardware can be identified quickly. Intel technologies' features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration.

1 0

cpython (3.5): Issue #24103: Fixed possible use after free in ElementTree.XMLPullParser.
by serhiy.storchaka 24 Dec '15

24 Dec '15

https://hg.python.org/cpython/rev/ed62cf0cf256 changeset: 99672:ed62cf0cf256 branch: 3.5 parent: 99669:fd36d72f6030 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 24 11:51:57 2015 +0200 summary: Issue #24103: Fixed possible use after free in ElementTree.XMLPullParser. files: Misc/NEWS | 2 + Modules/_elementtree.c | 32 ++++++++++++----------------- 2 files changed, 15 insertions(+), 19 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -31,6 +31,8 @@ Library ------- +- Issue #24103: Fixed possible use after free in ElementTree.XMLPullParser. + - Issue #25860: os.fwalk() no longer skips remaining directories when error occurs. Original patch by Samson Lee. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3598,7 +3598,7 @@ /*[clinic end generated code: output=1440092922b13ed1 input=59db9742910c6174]*/ { /* activate element event reporting */ - Py_ssize_t i, seqlen; + Py_ssize_t i; TreeBuilderObject *target; PyObject *events_seq; @@ -3614,8 +3614,7 @@ target = (TreeBuilderObject*) self->target; Py_INCREF(events_queue); - Py_XDECREF(target->events); - target->events = events_queue; + Py_SETREF(target->events, events_queue); /* clear out existing events */ Py_CLEAR(target->start_event_obj); @@ -3634,46 +3633,41 @@ return NULL; } - seqlen = PySequence_Size(events_seq); - for (i = 0; i < seqlen; ++i) { + for (i = 0; i < PySequence_Size(events_seq); ++i) { PyObject *event_name_obj = PySequence_Fast_GET_ITEM(events_seq, i); char *event_name = NULL; if (PyUnicode_Check(event_name_obj)) { - event_name = _PyUnicode_AsString(event_name_obj); + event_name = PyUnicode_AsUTF8(event_name_obj); } else if (PyBytes_Check(event_name_obj)) { event_name = PyBytes_AS_STRING(event_name_obj); } - if (event_name == NULL) { Py_DECREF(events_seq); PyErr_Format(PyExc_ValueError, "invalid events sequence"); return NULL; - } else if (strcmp(event_name, "start") == 0) { - Py_INCREF(event_name_obj); - target->start_event_obj = event_name_obj; + } + + Py_INCREF(event_name_obj); + if (strcmp(event_name, "start") == 0) { + Py_SETREF(target->start_event_obj, event_name_obj); } else if (strcmp(event_name, "end") == 0) { - Py_INCREF(event_name_obj); - Py_XDECREF(target->end_event_obj); - target->end_event_obj = event_name_obj; + Py_SETREF(target->end_event_obj, event_name_obj); } else if (strcmp(event_name, "start-ns") == 0) { - Py_INCREF(event_name_obj); - Py_XDECREF(target->start_ns_event_obj); - target->start_ns_event_obj = event_name_obj; + Py_SETREF(target->start_ns_event_obj, event_name_obj); EXPAT(SetNamespaceDeclHandler)( self->parser, (XML_StartNamespaceDeclHandler) expat_start_ns_handler, (XML_EndNamespaceDeclHandler) expat_end_ns_handler ); } else if (strcmp(event_name, "end-ns") == 0) { - Py_INCREF(event_name_obj); - Py_XDECREF(target->end_ns_event_obj); - target->end_ns_event_obj = event_name_obj; + Py_SETREF(target->end_ns_event_obj, event_name_obj); EXPAT(SetNamespaceDeclHandler)( self->parser, (XML_StartNamespaceDeclHandler) expat_start_ns_handler, (XML_EndNamespaceDeclHandler) expat_end_ns_handler ); } else { + Py_DECREF(event_name_obj); Py_DECREF(events_seq); PyErr_Format(PyExc_ValueError, "unknown event '%s'", event_name); return NULL; -- Repository URL: https://hg.python.org/cpython

1 0

cpython (2.7): Issue #24103: Fixed possible use after free in ElementTree.iterparse().
by serhiy.storchaka 24 Dec '15

24 Dec '15

https://hg.python.org/cpython/rev/deda5b5160d2 changeset: 99671:deda5b5160d2 branch: 2.7 parent: 99668:23296440b654 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 24 11:51:24 2015 +0200 summary: Issue #24103: Fixed possible use after free in ElementTree.iterparse(). files: Misc/NEWS | 2 ++ Modules/_elementtree.c | 20 +++++++------------- 2 files changed, 9 insertions(+), 13 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -29,6 +29,8 @@ Library ------- +- Issue #24103: Fixed possible use after free in ElementTree.iterparse(). + - Issue #20954: _args_from_interpreter_flags used by multiprocessing and some tests no longer behaves incorrectly in the presence of the PYTHONHASHSEED environment variable. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -2751,8 +2751,7 @@ target = (TreeBuilderObject*) self->target; Py_INCREF(events); - Py_XDECREF(target->events); - target->events = events; + Py_SETREF(target->events, events); /* clear out existing events */ Py_CLEAR(target->start_event_obj); @@ -2774,33 +2773,28 @@ char* event; if (!PyString_Check(item)) goto error; + Py_INCREF(item); event = PyString_AS_STRING(item); if (strcmp(event, "start") == 0) { - Py_INCREF(item); - target->start_event_obj = item; + Py_SETREF(target->start_event_obj, item); } else if (strcmp(event, "end") == 0) { - Py_INCREF(item); - Py_XDECREF(target->end_event_obj); - target->end_event_obj = item; + Py_SETREF(target->end_event_obj, item); } else if (strcmp(event, "start-ns") == 0) { - Py_INCREF(item); - Py_XDECREF(target->start_ns_event_obj); - target->start_ns_event_obj = item; + Py_SETREF(target->start_ns_event_obj, item); EXPAT(SetNamespaceDeclHandler)( self->parser, (XML_StartNamespaceDeclHandler) expat_start_ns_handler, (XML_EndNamespaceDeclHandler) expat_end_ns_handler ); } else if (strcmp(event, "end-ns") == 0) { - Py_INCREF(item); - Py_XDECREF(target->end_ns_event_obj); - target->end_ns_event_obj = item; + Py_SETREF(target->end_ns_event_obj, item); EXPAT(SetNamespaceDeclHandler)( self->parser, (XML_StartNamespaceDeclHandler) expat_start_ns_handler, (XML_EndNamespaceDeclHandler) expat_end_ns_handler ); } else { + Py_DECREF(item); PyErr_Format( PyExc_ValueError, "unknown event '%s'", event -- Repository URL: https://hg.python.org/cpython

1 0

cpython (merge 3.5 -> default): Issue #24103: Fixed possible use after free in ElementTree.XMLPullParser.
by serhiy.storchaka 24 Dec '15

24 Dec '15

https://hg.python.org/cpython/rev/8a14af800f96 changeset: 99673:8a14af800f96 parent: 99670:c4e8751ce637 parent: 99672:ed62cf0cf256 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 24 11:53:16 2015 +0200 summary: Issue #24103: Fixed possible use after free in ElementTree.XMLPullParser. files: Misc/NEWS | 2 + Modules/_elementtree.c | 32 ++++++++++++----------------- 2 files changed, 15 insertions(+), 19 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -118,6 +118,8 @@ Library ------- +- Issue #24103: Fixed possible use after free in ElementTree.XMLPullParser. + - Issue #25860: os.fwalk() no longer skips remaining directories when error occurs. Original patch by Samson Lee. diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3581,7 +3581,7 @@ /*[clinic end generated code: output=1440092922b13ed1 input=abf90830a1c3b0fc]*/ { /* activate element event reporting */ - Py_ssize_t i, seqlen; + Py_ssize_t i; TreeBuilderObject *target; PyObject *events_append, *events_seq; @@ -3599,8 +3599,7 @@ events_append = PyObject_GetAttrString(events_queue, "append"); if (events_append == NULL) return NULL; - Py_XDECREF(target->events_append); - target->events_append = events_append; + Py_SETREF(target->events_append, events_append); /* clear out existing events */ Py_CLEAR(target->start_event_obj); @@ -3619,46 +3618,41 @@ return NULL; } - seqlen = PySequence_Size(events_seq); - for (i = 0; i < seqlen; ++i) { + for (i = 0; i < PySequence_Size(events_seq); ++i) { PyObject *event_name_obj = PySequence_Fast_GET_ITEM(events_seq, i); char *event_name = NULL; if (PyUnicode_Check(event_name_obj)) { - event_name = _PyUnicode_AsString(event_name_obj); + event_name = PyUnicode_AsUTF8(event_name_obj); } else if (PyBytes_Check(event_name_obj)) { event_name = PyBytes_AS_STRING(event_name_obj); } - if (event_name == NULL) { Py_DECREF(events_seq); PyErr_Format(PyExc_ValueError, "invalid events sequence"); return NULL; - } else if (strcmp(event_name, "start") == 0) { - Py_INCREF(event_name_obj); - target->start_event_obj = event_name_obj; + } + + Py_INCREF(event_name_obj); + if (strcmp(event_name, "start") == 0) { + Py_SETREF(target->start_event_obj, event_name_obj); } else if (strcmp(event_name, "end") == 0) { - Py_INCREF(event_name_obj); - Py_XDECREF(target->end_event_obj); - target->end_event_obj = event_name_obj; + Py_SETREF(target->end_event_obj, event_name_obj); } else if (strcmp(event_name, "start-ns") == 0) { - Py_INCREF(event_name_obj); - Py_XDECREF(target->start_ns_event_obj); - target->start_ns_event_obj = event_name_obj; + Py_SETREF(target->start_ns_event_obj, event_name_obj); EXPAT(SetNamespaceDeclHandler)( self->parser, (XML_StartNamespaceDeclHandler) expat_start_ns_handler, (XML_EndNamespaceDeclHandler) expat_end_ns_handler ); } else if (strcmp(event_name, "end-ns") == 0) { - Py_INCREF(event_name_obj); - Py_XDECREF(target->end_ns_event_obj); - target->end_ns_event_obj = event_name_obj; + Py_SETREF(target->end_ns_event_obj, event_name_obj); EXPAT(SetNamespaceDeclHandler)( self->parser, (XML_StartNamespaceDeclHandler) expat_start_ns_handler, (XML_EndNamespaceDeclHandler) expat_end_ns_handler ); } else { + Py_DECREF(event_name_obj); Py_DECREF(events_seq); PyErr_Format(PyExc_ValueError, "unknown event '%s'", event_name); return NULL; -- Repository URL: https://hg.python.org/cpython

1 0

Daily reference leaks (92760d2edc9e): sum=4
by solipsis＠pitrou.net 24 Dec '15

24 Dec '15

results for 92760d2edc9e on branch "default" -------------------------------------------- test_functools leaked [0, 2, 2] memory blocks, sum=4 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/psf-users/antoine/refleaks/reflogxOz39J', '--timeout', '7200']

1 0

cpython (2.7): Issue #20440: Massive replacing unsafe attribute setting code with special
by serhiy.storchaka 24 Dec '15

24 Dec '15

https://hg.python.org/cpython/rev/23296440b654 changeset: 99668:23296440b654 branch: 2.7 parent: 99657:d7b5c2f99a99 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 24 10:35:35 2015 +0200 summary: Issue #20440: Massive replacing unsafe attribute setting code with special macro Py_SETREF. files: Include/object.h | 23 ++++++++++++++++ Misc/NEWS | 3 ++ Modules/_bsddb.c | 15 +++------- Modules/_csv.c | 6 +-- Modules/_ctypes/_ctypes.c | 33 ++++++++---------------- Modules/_curses_panel.c | 5 +-- Modules/_json.c | 3 +- Modules/_sqlite/connection.c | 13 +++----- Modules/_sqlite/cursor.c | 13 +++----- Modules/_sre.c | 3 +- Modules/_ssl.c | 3 +- Modules/bz2module.c | 5 +-- Modules/cPickle.c | 24 +++++----------- Modules/cdmodule.c | 6 +-- Modules/itertoolsmodule.c | 3 +- Modules/signalmodule.c | 3 +- Modules/zlibmodule.c | 6 +-- Objects/descrobject.c | 3 +- Objects/exceptions.c | 8 ++--- Objects/fileobject.c | 6 +-- Objects/frameobject.c | 3 +- Objects/funcobject.c | 9 ++---- Objects/stringobject.c | 6 +-- Objects/typeobject.c | 6 +-- Objects/unicodeobject.c | 3 +- Python/_warnings.c | 3 +- Python/ceval.c | 3 +- Python/compile.c | 5 +-- 28 files changed, 94 insertions(+), 128 deletions(-) diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -824,6 +824,29 @@ #define Py_XINCREF(op) do { if ((op) == NULL) ; else Py_INCREF(op); } while (0) #define Py_XDECREF(op) do { if ((op) == NULL) ; else Py_DECREF(op); } while (0) +/* Safely decref `op` and set `op` to `op2`. + * + * As in case of Py_CLEAR "the obvious" code can be deadly: + * + * Py_XDECREF(op); + * op = op2; + * + * The safe way is: + * + * Py_SETREF(op, op2); + * + * That arranges to set `op` to `op2` _before_ decref'ing, so that any code + * triggered as a side-effect of `op` getting torn down no longer believes + * `op` points to a valid object. + */ + +#define Py_SETREF(op, op2) \ + do { \ + PyObject *_py_tmp = (PyObject *)(op); \ + (op) = (op2); \ + Py_XDECREF(_py_tmp); \ + } while (0) + /* These are provided as conveniences to Python runtime embedders, so that they can have object code that is not dependent on Python compilation flags. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #20440: Massive replacing unsafe attribute setting code with special + macro Py_SETREF. + - Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size. This allows sys.getsize() to work correctly with their subclasses with __slots__ defined. diff --git a/Modules/_bsddb.c b/Modules/_bsddb.c --- a/Modules/_bsddb.c +++ b/Modules/_bsddb.c @@ -1607,9 +1607,8 @@ } /* Save a reference to the callback in the secondary DB. */ - Py_XDECREF(secondaryDB->associateCallback); Py_XINCREF(callback); - secondaryDB->associateCallback = callback; + Py_SETREF(secondaryDB->associateCallback, callback); secondaryDB->primaryDBType = _DB_get_type(self); /* PyEval_InitThreads is called here due to a quirk in python 1.5 @@ -2498,9 +2497,8 @@ DB_set_private(DBObject* self, PyObject* private_obj) { /* We can set the private field even if db is closed */ - Py_DECREF(self->private_obj); Py_INCREF(private_obj); - self->private_obj = private_obj; + Py_SETREF(self->private_obj, private_obj); RETURN_NONE(); } @@ -6998,9 +6996,8 @@ DBEnv_set_private(DBEnvObject* self, PyObject* private_obj) { /* We can set the private field even if dbenv is closed */ - Py_DECREF(self->private_obj); Py_INCREF(private_obj); - self->private_obj = private_obj; + Py_SETREF(self->private_obj, private_obj); RETURN_NONE(); } @@ -7253,9 +7250,8 @@ return NULL; } - Py_XDECREF(self->event_notifyCallback); Py_INCREF(notifyFunc); - self->event_notifyCallback = notifyFunc; + Py_SETREF(self->event_notifyCallback, notifyFunc); /* This is to workaround a problem with un-initialized threads (see comment in DB_associate) */ @@ -7413,9 +7409,8 @@ MYDB_END_ALLOW_THREADS; RETURN_IF_ERR(); - Py_DECREF(self->rep_transport); Py_INCREF(rep_transport); - self->rep_transport = rep_transport; + Py_SETREF(self->rep_transport, rep_transport); RETURN_NONE(); } diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -276,9 +276,8 @@ return -1; } else { - Py_XDECREF(*target); Py_INCREF(src); - *target = src; + Py_SETREF(*target, src); } } return 0; @@ -770,8 +769,7 @@ static int parse_reset(ReaderObj *self) { - Py_XDECREF(self->fields); - self->fields = PyList_New(0); + Py_SETREF(self->fields, PyList_New(0)); if (self->fields == NULL) return -1; self->field_len = 0; diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -424,8 +424,7 @@ Py_DECREF((PyObject *)dict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)dict; + Py_SETREF(result->tp_dict, (PyObject *)dict); dict->format = _ctypes_alloc_format_string(NULL, "B"); if (dict->format == NULL) { Py_DECREF(result); @@ -903,8 +902,7 @@ return -1; } Py_INCREF(proto); - Py_XDECREF(stgdict->proto); - stgdict->proto = proto; + Py_SETREF(stgdict->proto, proto); return 0; } @@ -994,8 +992,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); return (PyObject *)result; } @@ -1460,8 +1457,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); /* Special case for character arrays. A permanent annoyance: char arrays are also strings! @@ -1884,8 +1880,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); return (PyObject *)result; } @@ -2393,8 +2388,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); if (-1 == make_funcptrtype_dict(stgdict)) { Py_DECREF(result); @@ -2536,8 +2530,7 @@ } ob = PyCData_GetContainer(target); if (ob->b_objects == NULL || !PyDict_CheckExact(ob->b_objects)) { - Py_XDECREF(ob->b_objects); - ob->b_objects = keep; /* refcount consumed */ + Py_SETREF(ob->b_objects, keep); /* refcount consumed */ return 0; } key = unique_key(target, index); @@ -3059,9 +3052,8 @@ "the errcheck attribute must be callable"); return -1; } - Py_XDECREF(self->errcheck); Py_XINCREF(ob); - self->errcheck = ob; + Py_SETREF(self->errcheck, ob); return 0; } @@ -3090,9 +3082,8 @@ return -1; } Py_XDECREF(self->checker); - Py_XDECREF(self->restype); Py_INCREF(ob); - self->restype = ob; + Py_SETREF(self->restype, ob); self->checker = PyObject_GetAttrString(ob, "_check_retval_"); if (self->checker == NULL) PyErr_Clear(); @@ -3130,11 +3121,9 @@ converters = converters_from_argtypes(ob); if (!converters) return -1; - Py_XDECREF(self->converters); - self->converters = converters; - Py_XDECREF(self->argtypes); + Py_SETREF(self->converters, converters); Py_INCREF(ob); - self->argtypes = ob; + Py_SETREF(self->argtypes, ob); } return 0; } diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -283,9 +283,8 @@ PyErr_SetString(PyCursesError, "replace_panel() returned ERR"); return NULL; } - Py_DECREF(po->wo); - po->wo = temp; - Py_INCREF(po->wo); + Py_INCREF(temp); + Py_SETREF(po->wo, temp); Py_INCREF(Py_None); return Py_None; } diff --git a/Modules/_json.c b/Modules/_json.c --- a/Modules/_json.c +++ b/Modules/_json.c @@ -1717,8 +1717,7 @@ } else if (PyUnicode_Check(s->encoding)) { PyObject *tmp = PyUnicode_AsEncodedString(s->encoding, NULL, NULL); - Py_DECREF(s->encoding); - s->encoding = tmp; + Py_SETREF(s->encoding, tmp); } if (s->encoding == NULL) goto bail; diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -228,8 +228,8 @@ node = node->next; } - Py_DECREF(self->statement_cache); - self->statement_cache = (pysqlite_Cache*)PyObject_CallFunction((PyObject*)&pysqlite_CacheType, "O", self); + Py_SETREF(self->statement_cache, + (pysqlite_Cache *)PyObject_CallFunction((PyObject *)&pysqlite_CacheType, "O", self)); Py_DECREF(self); self->statement_cache->decref_factory = 0; } @@ -346,9 +346,8 @@ _pysqlite_drop_unused_cursor_references(self); if (cursor && self->row_factory != Py_None) { - Py_XDECREF(((pysqlite_Cursor*)cursor)->row_factory); Py_INCREF(self->row_factory); - ((pysqlite_Cursor*)cursor)->row_factory = self->row_factory; + Py_SETREF(((pysqlite_Cursor *)cursor)->row_factory, self->row_factory); } return cursor; @@ -816,8 +815,7 @@ } } - Py_DECREF(self->statements); - self->statements = new_list; + Py_SETREF(self->statements, new_list); } static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self) @@ -848,8 +846,7 @@ } } - Py_DECREF(self->cursors); - self->cursors = new_list; + Py_SETREF(self->cursors, new_list); } PyObject* pysqlite_connection_create_function(pysqlite_Connection* self, PyObject* args, PyObject* kwargs) diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -172,8 +172,7 @@ return 0; } - Py_XDECREF(self->row_cast_map); - self->row_cast_map = PyList_New(0); + Py_SETREF(self->row_cast_map, PyList_New(0)); for (i = 0; i < sqlite3_column_count(self->statement->st); i++) { converter = NULL; @@ -544,9 +543,8 @@ } /* reset description and rowcount */ - Py_DECREF(self->description); Py_INCREF(Py_None); - self->description = Py_None; + Py_SETREF(self->description, Py_None); self->rowcount = -1L; func_args = PyTuple_New(1); @@ -571,8 +569,8 @@ } if (self->statement->in_use) { - Py_DECREF(self->statement); - self->statement = PyObject_New(pysqlite_Statement, &pysqlite_StatementType); + Py_SETREF(self->statement, + PyObject_New(pysqlite_Statement, &pysqlite_StatementType)); if (!self->statement) { goto error; } @@ -683,8 +681,7 @@ numcols = sqlite3_column_count(self->statement->st); Py_END_ALLOW_THREADS - Py_DECREF(self->description); - self->description = PyTuple_New(numcols); + Py_SETREF(self->description, PyTuple_New(numcols)); if (!self->description) { goto error; } diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2054,8 +2054,7 @@ if (!copy) return 0; - Py_DECREF(*object); - *object = copy; + Py_SETREF(*object, copy); return 1; /* success */ } diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1467,8 +1467,7 @@ return -1; #else Py_INCREF(value); - Py_DECREF(self->ctx); - self->ctx = (PySSLContext *) value; + Py_SETREF(self->ctx, (PySSLContext *)value); SSL_set_SSL_CTX(self->ssl, self->ctx->ctx); #endif } else { diff --git a/Modules/bz2module.c b/Modules/bz2module.c --- a/Modules/bz2module.c +++ b/Modules/bz2module.c @@ -1953,9 +1953,8 @@ self->running = 0; input_left += bzs->avail_in; if (input_left != 0) { - Py_DECREF(self->unused_data); - self->unused_data = - PyString_FromStringAndSize(bzs->next_in, input_left); + Py_SETREF(self->unused_data, + PyString_FromStringAndSize(bzs->next_in, input_left)); if (self->unused_data == NULL) goto error; } diff --git a/Modules/cPickle.c b/Modules/cPickle.c --- a/Modules/cPickle.c +++ b/Modules/cPickle.c @@ -689,8 +689,7 @@ } if (! str) return -1; - Py_XDECREF(self->last_string); - self->last_string = str; + Py_SETREF(self->last_string, str); if (! (*s = PyString_AsString(str))) return -1; @@ -716,8 +715,7 @@ if ((str_size = PyString_Size(str)) < 0) return -1; - Py_XDECREF(self->last_string); - self->last_string = str; + Py_SETREF(self->last_string, str); if (! (*s = PyString_AsString(str))) return -1; @@ -3228,9 +3226,8 @@ "attribute deletion is not supported"); return -1; } - Py_XDECREF(p->pers_func); Py_INCREF(v); - p->pers_func = v; + Py_SETREF(p->pers_func, v); return 0; } @@ -3242,9 +3239,8 @@ "attribute deletion is not supported"); return -1; } - Py_XDECREF(p->inst_pers_func); Py_INCREF(v); - p->inst_pers_func = v; + Py_SETREF(p->inst_pers_func, v); return 0; } @@ -3270,9 +3266,8 @@ PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); return -1; } - Py_XDECREF(p->memo); Py_INCREF(v); - p->memo = v; + Py_SETREF(p->memo, v); return 0; } @@ -5667,16 +5662,14 @@ { if (!strcmp(name, "persistent_load")) { - Py_XDECREF(self->pers_func); - self->pers_func = value; Py_XINCREF(value); + Py_SETREF(self->pers_func, value); return 0; } if (!strcmp(name, "find_global")) { - Py_XDECREF(self->find_class); - self->find_class = value; Py_XINCREF(value); + Py_SETREF(self->find_class, value); return 0; } @@ -5692,9 +5685,8 @@ "memo must be a dictionary"); return -1; } - Py_XDECREF(self->memo); - self->memo = value; Py_INCREF(value); + Py_SETREF(self->memo, value); return 0; } diff --git a/Modules/cdmodule.c b/Modules/cdmodule.c --- a/Modules/cdmodule.c +++ b/Modules/cdmodule.c @@ -628,12 +628,10 @@ CDsetcallback(self->ob_cdparser, (CDDATATYPES) type, CD_callback, (void *) self); #endif - Py_XDECREF(self->ob_cdcallbacks[type].ob_cdcallback); Py_INCREF(func); - self->ob_cdcallbacks[type].ob_cdcallback = func; - Py_XDECREF(self->ob_cdcallbacks[type].ob_cdcallbackarg); + Py_SETREF(self->ob_cdcallbacks[type].ob_cdcallback, func); Py_INCREF(funcarg); - self->ob_cdcallbacks[type].ob_cdcallbackarg = funcarg; + Py_SETREF(self->ob_cdcallbacks[type].ob_cdcallbackarg, funcarg); /* if (type == cd_audio) { diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -494,8 +494,7 @@ link = teedataobject_jumplink(to->dataobj); if (link == NULL) return NULL; - Py_DECREF(to->dataobj); - to->dataobj = (teedataobject *)link; + Py_SETREF(to->dataobj, (teedataobject *)link); to->index = 0; } value = teedataobject_getitem(to->dataobj, to->index); diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -621,8 +621,7 @@ if (Handlers[SIGINT].func == DefaultHandler) { /* Install default int handler */ Py_INCREF(IntHandler); - Py_DECREF(Handlers[SIGINT].func); - Handlers[SIGINT].func = IntHandler; + Py_SETREF(Handlers[SIGINT].func, IntHandler); old_siginthandler = PyOS_setsig(SIGINT, signal_handler); } diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -491,8 +491,7 @@ PyString_AS_STRING(self->unused_data), old_size); Py_MEMCPY(PyString_AS_STRING(new_data) + old_size, self->zst.next_in, self->zst.avail_in); - Py_DECREF(self->unused_data); - self->unused_data = new_data; + Py_SETREF(self->unused_data, new_data); self->zst.avail_in = 0; } } @@ -504,8 +503,7 @@ (char *)self->zst.next_in, self->zst.avail_in); if (new_data == NULL) return -1; - Py_DECREF(self->unconsumed_tail); - self->unconsumed_tail = new_data; + Py_SETREF(self->unconsumed_tail, new_data); } return 0; } diff --git a/Objects/descrobject.c b/Objects/descrobject.c --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1332,8 +1332,7 @@ PyObject *get_doc = PyObject_GetAttrString(get, "__doc__"); if (get_doc) { if (Py_TYPE(self) == &PyProperty_Type) { - Py_XDECREF(prop->prop_doc); - prop->prop_doc = get_doc; + Py_SETREF(prop->prop_doc, get_doc); } else { /* If this is a property subclass, put __doc__ diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -58,9 +58,8 @@ if (!_PyArg_NoKeywords(Py_TYPE(self)->tp_name, kwds)) return -1; - Py_DECREF(self->args); - self->args = args; - Py_INCREF(self->args); + Py_INCREF(args); + Py_SETREF(self->args, args); if (PyTuple_GET_SIZE(self->args) == 1) { Py_CLEAR(self->message); @@ -627,8 +626,7 @@ if (!subslice) return -1; - Py_DECREF(self->args); /* replacing args */ - self->args = subslice; + Py_SETREF(self->args, subslice); } return 0; } diff --git a/Objects/fileobject.c b/Objects/fileobject.c --- a/Objects/fileobject.c +++ b/Objects/fileobject.c @@ -574,10 +574,8 @@ oerrors = Py_None; Py_INCREF(Py_None); } - Py_DECREF(file->f_encoding); - file->f_encoding = str; - Py_DECREF(file->f_errors); - file->f_errors = oerrors; + Py_SETREF(file->f_encoding, str); + Py_SETREF(file->f_errors, oerrors); return 1; } diff --git a/Objects/frameobject.c b/Objects/frameobject.c --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -859,8 +859,7 @@ } } else if (values[j] != value) { Py_XINCREF(value); - Py_XDECREF(values[j]); - values[j] = value; + Py_SETREF(values[j], value); } Py_XDECREF(value); } diff --git a/Objects/funcobject.c b/Objects/funcobject.c --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -116,8 +116,7 @@ PyErr_SetString(PyExc_SystemError, "non-tuple default args"); return -1; } - Py_XDECREF(((PyFunctionObject *) op) -> func_defaults); - ((PyFunctionObject *) op) -> func_defaults = defaults; + Py_SETREF(((PyFunctionObject *)op)->func_defaults, defaults); return 0; } @@ -149,8 +148,7 @@ closure->ob_type->tp_name); return -1; } - Py_XDECREF(((PyFunctionObject *) op) -> func_closure); - ((PyFunctionObject *) op) -> func_closure = closure; + Py_SETREF(((PyFunctionObject *)op)->func_closure, closure); return 0; } @@ -430,8 +428,7 @@ if (name != Py_None) { Py_INCREF(name); - Py_DECREF(newfunc->func_name); - newfunc->func_name = name; + Py_SETREF(newfunc->func_name, name); } if (defaults != Py_None) { Py_INCREF(defaults); diff --git a/Objects/stringobject.c b/Objects/stringobject.c --- a/Objects/stringobject.c +++ b/Objects/stringobject.c @@ -3865,8 +3865,7 @@ return; } v = string_concat((PyStringObject *) *pv, w); - Py_DECREF(*pv); - *pv = v; + Py_SETREF(*pv, v); } void @@ -4751,8 +4750,7 @@ t = PyDict_GetItem(interned, (PyObject *)s); if (t) { Py_INCREF(t); - Py_DECREF(*p); - *p = t; + Py_SETREF(*p, t); return; } diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -166,9 +166,8 @@ are borrowed reference */ for (i = 0; i < (1 << MCACHE_SIZE_EXP); i++) { method_cache[i].value = NULL; - Py_XDECREF(method_cache[i].name); - method_cache[i].name = Py_None; Py_INCREF(Py_None); + Py_SETREF(method_cache[i].name, Py_None); } /* mark all version tags as invalid */ PyType_Modified(&PyBaseObject_Type); @@ -2527,8 +2526,7 @@ method_cache[h].version = type->tp_version_tag; method_cache[h].value = res; /* borrowed */ Py_INCREF(name); - Py_DECREF(method_cache[h].name); - method_cache[h].name = name; + Py_SETREF(method_cache[h].name, name); } return res; } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -436,8 +436,7 @@ return -1; Py_UNICODE_COPY(w->str, v->str, length < v->length ? length : v->length); - Py_DECREF(*unicode); - *unicode = w; + Py_SETREF(*unicode, w); return 0; } diff --git a/Python/_warnings.c b/Python/_warnings.c --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -528,8 +528,7 @@ goto handle_error; } else if (!is_true) { - Py_DECREF(*filename); - *filename = PyString_FromString("__main__"); + Py_SETREF(*filename, PyString_FromString("__main__")); if (*filename == NULL) goto handle_error; } diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -4360,8 +4360,7 @@ Py_INCREF(self); func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); - Py_DECREF(*pfunc); - *pfunc = self; + Py_SETREF(*pfunc, self); na++; n++; } else diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -1455,9 +1455,8 @@ if (!compiler_enter_scope(c, s->v.ClassDef.name, (void *)s, s->lineno)) return 0; - Py_XDECREF(c->u->u_private); - c->u->u_private = s->v.ClassDef.name; - Py_INCREF(c->u->u_private); + Py_INCREF(s->v.ClassDef.name); + Py_SETREF(c->u->u_private, s->v.ClassDef.name); str = PyString_InternFromString("__name__"); if (!str || !compiler_nameop(c, str, Load)) { Py_XDECREF(str); -- Repository URL: https://hg.python.org/cpython

1 0

cpython (3.5): Issue #20440: Massive replacing unsafe attribute setting code with special
by serhiy.storchaka 24 Dec '15

24 Dec '15

https://hg.python.org/cpython/rev/fd36d72f6030 changeset: 99669:fd36d72f6030 branch: 3.5 parent: 99666:105bf5dd93b8 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 24 10:35:59 2015 +0200 summary: Issue #20440: Massive replacing unsafe attribute setting code with special macro Py_SETREF. files: Include/object.h | 26 +++++++++++++++ Misc/NEWS | 3 + Modules/_csv.c | 6 +-- Modules/_ctypes/_ctypes.c | 41 ++++++++--------------- Modules/_curses_panel.c | 5 +- Modules/_io/bytesio.c | 3 +- Modules/_sqlite/connection.c | 13 ++---- Modules/_sqlite/cursor.c | 13 ++---- Modules/_sre.c | 3 +- Modules/_ssl.c | 6 +-- Modules/faulthandler.c | 9 +--- Modules/itertoolsmodule.c | 3 +- Modules/signalmodule.c | 3 +- Modules/zipimport.c | 3 +- Modules/zlibmodule.c | 6 +-- Objects/bytesobject.c | 3 +- Objects/descrobject.c | 3 +- Objects/exceptions.c | 3 +- Objects/frameobject.c | 3 +- Objects/funcobject.c | 15 ++----- Objects/moduleobject.c | 3 +- Objects/rangeobject.c | 3 +- Objects/typeobject.c | 9 +--- Objects/unicodeobject.c | 12 ++---- Parser/tokenizer.c | 3 +- Python/_warnings.c | 3 +- Python/ceval.c | 6 +-- Python/compile.c | 3 +- 28 files changed, 92 insertions(+), 120 deletions(-) diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -846,6 +846,32 @@ Py_DECREF(_py_xdecref_tmp); \ } while (0) +#ifndef Py_LIMITED_API +/* Safely decref `op` and set `op` to `op2`. + * + * As in case of Py_CLEAR "the obvious" code can be deadly: + * + * Py_XDECREF(op); + * op = op2; + * + * The safe way is: + * + * Py_SETREF(op, op2); + * + * That arranges to set `op` to `op2` _before_ decref'ing, so that any code + * triggered as a side-effect of `op` getting torn down no longer believes + * `op` points to a valid object. + */ + +#define Py_SETREF(op, op2) \ + do { \ + PyObject *_py_tmp = (PyObject *)(op); \ + (op) = (op2); \ + Py_XDECREF(_py_tmp); \ + } while (0) + +#endif /* ifndef Py_LIMITED_API */ + /* These are provided as conveniences to Python runtime embedders, so that they can have object code that is not dependent on Python compilation flags. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #20440: Massive replacing unsafe attribute setting code with special + macro Py_SETREF. + - Issue #25766: Special method __bytes__() now works in str subclasses. - Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -276,9 +276,8 @@ else { if (PyUnicode_READY(src) == -1) return -1; - Py_XDECREF(*target); Py_INCREF(src); - *target = src; + Py_SETREF(*target, src); } } return 0; @@ -784,8 +783,7 @@ static int parse_reset(ReaderObj *self) { - Py_XDECREF(self->fields); - self->fields = PyList_New(0); + Py_SETREF(self->fields, PyList_New(0)); if (self->fields == NULL) return -1; self->field_len = 0; diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -391,8 +391,7 @@ Py_DECREF((PyObject *)dict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)dict; + Py_SETREF(result->tp_dict, (PyObject *)dict); dict->format = _ctypes_alloc_format_string(NULL, "B"); if (dict->format == NULL) { Py_DECREF(result); @@ -871,8 +870,7 @@ return -1; } Py_INCREF(proto); - Py_XDECREF(stgdict->proto); - stgdict->proto = proto; + Py_SETREF(stgdict->proto, proto); return 0; } @@ -962,8 +960,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); return (PyObject *)result; } @@ -1406,8 +1403,7 @@ /* replace the class dict by our updated spam dict */ if (-1 == PyDict_Update((PyObject *)stgdict, result->tp_dict)) goto error; - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; /* steal the reference */ + Py_SETREF(result->tp_dict, (PyObject *)stgdict); /* steal the reference */ stgdict = NULL; /* Special case for character arrays. @@ -1820,8 +1816,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); return (PyObject *)result; } @@ -1949,8 +1944,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); /* Install from_param class methods in ctypes base classes. Overrides the PyCSimpleType_from_param generic method. @@ -2313,8 +2307,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); if (-1 == make_funcptrtype_dict(stgdict)) { Py_DECREF(result); @@ -2458,8 +2451,7 @@ return -1; } if (ob->b_objects == NULL || !PyDict_CheckExact(ob->b_objects)) { - Py_XDECREF(ob->b_objects); - ob->b_objects = keep; /* refcount consumed */ + Py_SETREF(ob->b_objects, keep); /* refcount consumed */ return 0; } key = unique_key(target, index); @@ -2962,9 +2954,8 @@ "the errcheck attribute must be callable"); return -1; } - Py_XDECREF(self->errcheck); Py_XINCREF(ob); - self->errcheck = ob; + Py_SETREF(self->errcheck, ob); return 0; } @@ -2993,9 +2984,8 @@ return -1; } Py_XDECREF(self->checker); - Py_XDECREF(self->restype); Py_INCREF(ob); - self->restype = ob; + Py_SETREF(self->restype, ob); self->checker = PyObject_GetAttrString(ob, "_check_retval_"); if (self->checker == NULL) PyErr_Clear(); @@ -3033,11 +3023,9 @@ converters = converters_from_argtypes(ob); if (!converters) return -1; - Py_XDECREF(self->converters); - self->converters = converters; - Py_XDECREF(self->argtypes); + Py_SETREF(self->converters, converters); Py_INCREF(ob); - self->argtypes = ob; + Py_SETREF(self->argtypes, ob); } return 0; } @@ -5164,9 +5152,8 @@ return -1; bself = (PyBaseExceptionObject *)self; - Py_DECREF(bself->args); - bself->args = args; - Py_INCREF(bself->args); + Py_INCREF(args); + Py_SETREF(bself->args, args); return 0; } diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -312,9 +312,8 @@ PyErr_SetString(_curses_panelstate_global->PyCursesError, "replace_panel() returned ERR"); return NULL; } - Py_DECREF(po->wo); - po->wo = temp; - Py_INCREF(po->wo); + Py_INCREF(temp); + Py_SETREF(po->wo, temp); Py_INCREF(Py_None); return Py_None; } diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -969,8 +969,7 @@ if (initvalue && initvalue != Py_None) { if (PyBytes_CheckExact(initvalue)) { Py_INCREF(initvalue); - Py_XDECREF(self->buf); - self->buf = initvalue; + Py_SETREF(self->buf, initvalue); self->string_size = PyBytes_GET_SIZE(initvalue); } else { diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -204,8 +204,8 @@ node = node->next; } - Py_DECREF(self->statement_cache); - self->statement_cache = (pysqlite_Cache*)PyObject_CallFunction((PyObject*)&pysqlite_CacheType, "O", self); + Py_SETREF(self->statement_cache, + (pysqlite_Cache *)PyObject_CallFunction((PyObject *)&pysqlite_CacheType, "O", self)); Py_DECREF(self); self->statement_cache->decref_factory = 0; } @@ -318,9 +318,8 @@ _pysqlite_drop_unused_cursor_references(self); if (cursor && self->row_factory != Py_None) { - Py_XDECREF(((pysqlite_Cursor*)cursor)->row_factory); Py_INCREF(self->row_factory); - ((pysqlite_Cursor*)cursor)->row_factory = self->row_factory; + Py_SETREF(((pysqlite_Cursor *)cursor)->row_factory, self->row_factory); } return cursor; @@ -795,8 +794,7 @@ } } - Py_DECREF(self->statements); - self->statements = new_list; + Py_SETREF(self->statements, new_list); } static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self) @@ -827,8 +825,7 @@ } } - Py_DECREF(self->cursors); - self->cursors = new_list; + Py_SETREF(self->cursors, new_list); } PyObject* pysqlite_connection_create_function(pysqlite_Connection* self, PyObject* args, PyObject* kwargs) diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -170,8 +170,7 @@ return 0; } - Py_XDECREF(self->row_cast_map); - self->row_cast_map = PyList_New(0); + Py_SETREF(self->row_cast_map, PyList_New(0)); for (i = 0; i < sqlite3_column_count(self->statement->st); i++) { converter = NULL; @@ -510,9 +509,8 @@ goto error; /* reset description and rowcount */ - Py_DECREF(self->description); Py_INCREF(Py_None); - self->description = Py_None; + Py_SETREF(self->description, Py_None); self->rowcount = -1L; func_args = PyTuple_New(1); @@ -537,8 +535,8 @@ } if (self->statement->in_use) { - Py_DECREF(self->statement); - self->statement = PyObject_New(pysqlite_Statement, &pysqlite_StatementType); + Py_SETREF(self->statement, + PyObject_New(pysqlite_Statement, &pysqlite_StatementType)); if (!self->statement) { goto error; } @@ -654,8 +652,7 @@ numcols = sqlite3_column_count(self->statement->st); Py_END_ALLOW_THREADS - Py_DECREF(self->description); - self->description = PyTuple_New(numcols); + Py_SETREF(self->description, PyTuple_New(numcols)); if (!self->description) { goto error; } diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -756,8 +756,7 @@ if (!copy) return 0; - Py_DECREF(*object); - *object = copy; + Py_SETREF(*object, copy); return 1; /* success */ } diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1589,8 +1589,7 @@ return -1; #else Py_INCREF(value); - Py_DECREF(self->ctx); - self->ctx = (PySSLContext *) value; + Py_SETREF(self->ctx, (PySSLContext *)value); SSL_set_SSL_CTX(self->ssl, self->ctx->ctx); #endif } else { @@ -1647,8 +1646,7 @@ static int PySSL_set_owner(PySSLSocket *self, PyObject *value, void *c) { - Py_XDECREF(self->owner); - self->owner = PyWeakref_NewRef(value, NULL); + Py_SETREF(self->owner, PyWeakref_NewRef(value, NULL)); if (self->owner == NULL) return -1; return 0; diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -380,9 +380,8 @@ if (tstate == NULL) return NULL; - Py_XDECREF(fatal_error.file); Py_XINCREF(file); - fatal_error.file = file; + Py_SETREF(fatal_error.file, file); fatal_error.fd = fd; fatal_error.all_threads = all_threads; fatal_error.interp = tstate->interp; @@ -599,9 +598,8 @@ /* Cancel previous thread, if running */ cancel_dump_traceback_later(); - Py_XDECREF(thread.file); Py_XINCREF(file); - thread.file = file; + Py_SETREF(thread.file, file); thread.fd = fd; thread.timeout_us = timeout_us; thread.repeat = repeat; @@ -778,9 +776,8 @@ user->previous = previous; } - Py_XDECREF(user->file); Py_XINCREF(file); - user->file = file; + Py_SETREF(user->file, file); user->fd = fd; user->all_threads = all_threads; user->chain = chain; diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -636,8 +636,7 @@ link = teedataobject_jumplink(to->dataobj); if (link == NULL) return NULL; - Py_DECREF(to->dataobj); - to->dataobj = (teedataobject *)link; + Py_SETREF(to->dataobj, (teedataobject *)link); to->index = 0; } value = teedataobject_getitem(to->dataobj, to->index); diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -1266,8 +1266,7 @@ if (Handlers[SIGINT].func == DefaultHandler) { /* Install default int handler */ Py_INCREF(IntHandler); - Py_DECREF(Handlers[SIGINT].func); - Handlers[SIGINT].func = IntHandler; + Py_SETREF(Handlers[SIGINT].func, IntHandler); old_siginthandler = PyOS_setsig(SIGINT, signal_handler); } diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -155,8 +155,7 @@ tmp = PyUnicode_FromFormat("%U%c", self->prefix, SEP); if (tmp == NULL) goto error; - Py_DECREF(self->prefix); - self->prefix = tmp; + Py_SETREF(self->prefix, tmp); } } else diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -667,8 +667,7 @@ PyBytes_AS_STRING(self->unused_data), old_size); Py_MEMCPY(PyBytes_AS_STRING(new_data) + old_size, self->zst.next_in, self->zst.avail_in); - Py_DECREF(self->unused_data); - self->unused_data = new_data; + Py_SETREF(self->unused_data, new_data); self->zst.avail_in = 0; } } @@ -680,8 +679,7 @@ (char *)self->zst.next_in, self->zst.avail_in); if (new_data == NULL) return -1; - Py_DECREF(self->unconsumed_tail); - self->unconsumed_tail = new_data; + Py_SETREF(self->unconsumed_tail, new_data); } return 0; } diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -3520,8 +3520,7 @@ /* Multiple references, need to create new object */ PyObject *v; v = bytes_concat(*pv, w); - Py_DECREF(*pv); - *pv = v; + Py_SETREF(*pv, v); } } diff --git a/Objects/descrobject.c b/Objects/descrobject.c --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1509,8 +1509,7 @@ PyObject *get_doc = _PyObject_GetAttrId(get, &PyId___doc__); if (get_doc) { if (Py_TYPE(self) == &PyProperty_Type) { - Py_XDECREF(prop->prop_doc); - prop->prop_doc = get_doc; + Py_SETREF(prop->prop_doc, get_doc); } else { /* If this is a property subclass, put __doc__ diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -236,8 +236,7 @@ } Py_XINCREF(tb); - Py_XDECREF(self->traceback); - self->traceback = tb; + Py_SETREF(self->traceback, tb); return 0; } diff --git a/Objects/frameobject.c b/Objects/frameobject.c --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -857,8 +857,7 @@ } } else if (values[j] != value) { Py_XINCREF(value); - Py_XDECREF(values[j]); - values[j] = value; + Py_SETREF(values[j], value); } Py_XDECREF(value); } diff --git a/Objects/funcobject.c b/Objects/funcobject.c --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -127,8 +127,7 @@ PyErr_SetString(PyExc_SystemError, "non-tuple default args"); return -1; } - Py_XDECREF(((PyFunctionObject *) op) -> func_defaults); - ((PyFunctionObject *) op) -> func_defaults = defaults; + Py_SETREF(((PyFunctionObject *)op)->func_defaults, defaults); return 0; } @@ -159,8 +158,7 @@ "non-dict keyword only default args"); return -1; } - Py_XDECREF(((PyFunctionObject *)op) -> func_kwdefaults); - ((PyFunctionObject *) op) -> func_kwdefaults = defaults; + Py_SETREF(((PyFunctionObject *)op)->func_kwdefaults, defaults); return 0; } @@ -192,8 +190,7 @@ closure->ob_type->tp_name); return -1; } - Py_XDECREF(((PyFunctionObject *) op) -> func_closure); - ((PyFunctionObject *) op) -> func_closure = closure; + Py_SETREF(((PyFunctionObject *)op)->func_closure, closure); return 0; } @@ -224,8 +221,7 @@ "non-dict annotations"); return -1; } - Py_XDECREF(((PyFunctionObject *)op) -> func_annotations); - ((PyFunctionObject *) op) -> func_annotations = annotations; + Py_SETREF(((PyFunctionObject *)op)->func_annotations, annotations); return 0; } @@ -531,8 +527,7 @@ if (name != Py_None) { Py_INCREF(name); - Py_DECREF(newfunc->func_name); - newfunc->func_name = name; + Py_SETREF(newfunc->func_name, name); } if (defaults != Py_None) { Py_INCREF(defaults); diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -69,8 +69,7 @@ return -1; if (PyUnicode_CheckExact(name)) { Py_INCREF(name); - Py_XDECREF(mod->md_name); - mod->md_name = name; + Py_SETREF(mod->md_name, name); } return 0; diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -1066,8 +1066,7 @@ result = PyNumber_Add(r->start, product); Py_DECREF(product); if (result) { - Py_DECREF(r->index); - r->index = new_index; + Py_SETREF(r->index, new_index); } else { Py_DECREF(new_index); diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -315,9 +315,8 @@ are borrowed reference */ for (i = 0; i < (1 << MCACHE_SIZE_EXP); i++) { method_cache[i].value = NULL; - Py_XDECREF(method_cache[i].name); - method_cache[i].name = Py_None; Py_INCREF(Py_None); + Py_SETREF(method_cache[i].name, Py_None); } /* mark all version tags as invalid */ PyType_Modified(&PyBaseObject_Type); @@ -462,8 +461,7 @@ et = (PyHeapTypeObject*)type; Py_INCREF(value); - Py_DECREF(et->ht_qualname); - et->ht_qualname = value; + Py_SETREF(et->ht_qualname, value); return 0; } @@ -2910,8 +2908,7 @@ else method_cache_misses++; #endif - Py_DECREF(method_cache[h].name); - method_cache[h].name = name; + Py_SETREF(method_cache[h].name, name); } return res; } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -1665,8 +1665,7 @@ _Py_INCREF_UNICODE_EMPTY(); if (!unicode_empty) return -1; - Py_DECREF(*p_unicode); - *p_unicode = unicode_empty; + Py_SETREF(*p_unicode, unicode_empty); return 0; } @@ -1674,8 +1673,7 @@ PyObject *copy = resize_copy(unicode, length); if (copy == NULL) return -1; - Py_DECREF(*p_unicode); - *p_unicode = copy; + Py_SETREF(*p_unicode, copy); return 0; } @@ -13322,8 +13320,7 @@ return -1; _PyUnicode_FastCopyCharacters(newbuffer, 0, writer->buffer, 0, writer->pos); - Py_DECREF(writer->buffer); - writer->buffer = newbuffer; + Py_SETREF(writer->buffer, newbuffer); } _PyUnicodeWriter_Update(writer); return 0; @@ -15009,8 +15006,7 @@ if (t) { Py_INCREF(t); - Py_DECREF(*p); - *p = t; + Py_SETREF(*p, t); return; } diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -524,9 +524,8 @@ if (stream == NULL) goto cleanup; - Py_XDECREF(tok->decoding_readline); readline = _PyObject_GetAttrId(stream, &PyId_readline); - tok->decoding_readline = readline; + Py_SETREF(tok->decoding_readline, readline); if (pos > 0) { if (PyObject_CallObject(readline, NULL) == NULL) { readline = NULL; diff --git a/Python/_warnings.c b/Python/_warnings.c --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -680,8 +680,7 @@ goto handle_error; } else if (!is_true) { - Py_DECREF(*filename); - *filename = PyUnicode_FromString("__main__"); + Py_SETREF(*filename, PyUnicode_FromString("__main__")); if (*filename == NULL) goto handle_error; } diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3214,8 +3214,7 @@ Py_INCREF(self); func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); - Py_DECREF(*pfunc); - *pfunc = self; + Py_SETREF(*pfunc, self); na++; /* n++; */ } else @@ -4670,8 +4669,7 @@ Py_INCREF(self); func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); - Py_DECREF(*pfunc); - *pfunc = self; + Py_SETREF(*pfunc, self); na++; n++; } else diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -1795,8 +1795,7 @@ { /* use the class name for name mangling */ Py_INCREF(s->v.ClassDef.name); - Py_XDECREF(c->u->u_private); - c->u->u_private = s->v.ClassDef.name; + Py_SETREF(c->u->u_private, s->v.ClassDef.name); /* load (global) __name__ ... */ str = PyUnicode_InternFromString("__name__"); if (!str || !compiler_nameop(c, str, Load)) { -- Repository URL: https://hg.python.org/cpython

1 0

cpython (merge 3.5 -> default): Issue #20440: Massive replacing unsafe attribute setting code with special
by serhiy.storchaka 24 Dec '15

24 Dec '15

https://hg.python.org/cpython/rev/c4e8751ce637 changeset: 99670:c4e8751ce637 parent: 99667:92760d2edc9e parent: 99669:fd36d72f6030 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Dec 24 10:39:57 2015 +0200 summary: Issue #20440: Massive replacing unsafe attribute setting code with special macro Py_SETREF. files: Include/object.h | 26 +++++++++++++++ Misc/NEWS | 3 + Modules/_csv.c | 6 +-- Modules/_ctypes/_ctypes.c | 41 ++++++++--------------- Modules/_curses_panel.c | 5 +- Modules/_io/bytesio.c | 3 +- Modules/_sqlite/connection.c | 13 ++---- Modules/_sqlite/cursor.c | 13 ++---- Modules/_sre.c | 3 +- Modules/_ssl.c | 6 +-- Modules/faulthandler.c | 9 +--- Modules/itertoolsmodule.c | 3 +- Modules/signalmodule.c | 3 +- Modules/zipimport.c | 3 +- Modules/zlibmodule.c | 6 +-- Objects/bytesobject.c | 3 +- Objects/descrobject.c | 3 +- Objects/exceptions.c | 3 +- Objects/frameobject.c | 3 +- Objects/funcobject.c | 15 ++----- Objects/moduleobject.c | 3 +- Objects/rangeobject.c | 3 +- Objects/typeobject.c | 9 +--- Objects/unicodeobject.c | 12 ++---- Parser/tokenizer.c | 3 +- Python/_warnings.c | 3 +- Python/ceval.c | 6 +-- Python/compile.c | 3 +- 28 files changed, 92 insertions(+), 120 deletions(-) diff --git a/Include/object.h b/Include/object.h --- a/Include/object.h +++ b/Include/object.h @@ -846,6 +846,32 @@ Py_DECREF(_py_xdecref_tmp); \ } while (0) +#ifndef Py_LIMITED_API +/* Safely decref `op` and set `op` to `op2`. + * + * As in case of Py_CLEAR "the obvious" code can be deadly: + * + * Py_XDECREF(op); + * op = op2; + * + * The safe way is: + * + * Py_SETREF(op, op2); + * + * That arranges to set `op` to `op2` _before_ decref'ing, so that any code + * triggered as a side-effect of `op` getting torn down no longer believes + * `op` points to a valid object. + */ + +#define Py_SETREF(op, op2) \ + do { \ + PyObject *_py_tmp = (PyObject *)(op); \ + (op) = (op2); \ + Py_XDECREF(_py_tmp); \ + } while (0) + +#endif /* ifndef Py_LIMITED_API */ + /* These are provided as conveniences to Python runtime embedders, so that they can have object code that is not dependent on Python compilation flags. diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #20440: Massive replacing unsafe attribute setting code with special + macro Py_SETREF. + - Issue #25766: Special method __bytes__() now works in str subclasses. - Issue #25421: __sizeof__ methods of builtin types now use dynamic basic size. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -276,9 +276,8 @@ else { if (PyUnicode_READY(src) == -1) return -1; - Py_XDECREF(*target); Py_INCREF(src); - *target = src; + Py_SETREF(*target, src); } } return 0; @@ -784,8 +783,7 @@ static int parse_reset(ReaderObj *self) { - Py_XDECREF(self->fields); - self->fields = PyList_New(0); + Py_SETREF(self->fields, PyList_New(0)); if (self->fields == NULL) return -1; self->field_len = 0; diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c --- a/Modules/_ctypes/_ctypes.c +++ b/Modules/_ctypes/_ctypes.c @@ -391,8 +391,7 @@ Py_DECREF((PyObject *)dict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)dict; + Py_SETREF(result->tp_dict, (PyObject *)dict); dict->format = _ctypes_alloc_format_string(NULL, "B"); if (dict->format == NULL) { Py_DECREF(result); @@ -871,8 +870,7 @@ return -1; } Py_INCREF(proto); - Py_XDECREF(stgdict->proto); - stgdict->proto = proto; + Py_SETREF(stgdict->proto, proto); return 0; } @@ -962,8 +960,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); return (PyObject *)result; } @@ -1406,8 +1403,7 @@ /* replace the class dict by our updated spam dict */ if (-1 == PyDict_Update((PyObject *)stgdict, result->tp_dict)) goto error; - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; /* steal the reference */ + Py_SETREF(result->tp_dict, (PyObject *)stgdict); /* steal the reference */ stgdict = NULL; /* Special case for character arrays. @@ -1820,8 +1816,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); return (PyObject *)result; } @@ -1949,8 +1944,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); /* Install from_param class methods in ctypes base classes. Overrides the PyCSimpleType_from_param generic method. @@ -2313,8 +2307,7 @@ Py_DECREF((PyObject *)stgdict); return NULL; } - Py_DECREF(result->tp_dict); - result->tp_dict = (PyObject *)stgdict; + Py_SETREF(result->tp_dict, (PyObject *)stgdict); if (-1 == make_funcptrtype_dict(stgdict)) { Py_DECREF(result); @@ -2458,8 +2451,7 @@ return -1; } if (ob->b_objects == NULL || !PyDict_CheckExact(ob->b_objects)) { - Py_XDECREF(ob->b_objects); - ob->b_objects = keep; /* refcount consumed */ + Py_SETREF(ob->b_objects, keep); /* refcount consumed */ return 0; } key = unique_key(target, index); @@ -2962,9 +2954,8 @@ "the errcheck attribute must be callable"); return -1; } - Py_XDECREF(self->errcheck); Py_XINCREF(ob); - self->errcheck = ob; + Py_SETREF(self->errcheck, ob); return 0; } @@ -2993,9 +2984,8 @@ return -1; } Py_XDECREF(self->checker); - Py_XDECREF(self->restype); Py_INCREF(ob); - self->restype = ob; + Py_SETREF(self->restype, ob); self->checker = PyObject_GetAttrString(ob, "_check_retval_"); if (self->checker == NULL) PyErr_Clear(); @@ -3033,11 +3023,9 @@ converters = converters_from_argtypes(ob); if (!converters) return -1; - Py_XDECREF(self->converters); - self->converters = converters; - Py_XDECREF(self->argtypes); + Py_SETREF(self->converters, converters); Py_INCREF(ob); - self->argtypes = ob; + Py_SETREF(self->argtypes, ob); } return 0; } @@ -5164,9 +5152,8 @@ return -1; bself = (PyBaseExceptionObject *)self; - Py_DECREF(bself->args); - bself->args = args; - Py_INCREF(bself->args); + Py_INCREF(args); + Py_SETREF(bself->args, args); return 0; } diff --git a/Modules/_curses_panel.c b/Modules/_curses_panel.c --- a/Modules/_curses_panel.c +++ b/Modules/_curses_panel.c @@ -312,9 +312,8 @@ PyErr_SetString(_curses_panelstate_global->PyCursesError, "replace_panel() returned ERR"); return NULL; } - Py_DECREF(po->wo); - po->wo = temp; - Py_INCREF(po->wo); + Py_INCREF(temp); + Py_SETREF(po->wo, temp); Py_INCREF(Py_None); return Py_None; } diff --git a/Modules/_io/bytesio.c b/Modules/_io/bytesio.c --- a/Modules/_io/bytesio.c +++ b/Modules/_io/bytesio.c @@ -969,8 +969,7 @@ if (initvalue && initvalue != Py_None) { if (PyBytes_CheckExact(initvalue)) { Py_INCREF(initvalue); - Py_XDECREF(self->buf); - self->buf = initvalue; + Py_SETREF(self->buf, initvalue); self->string_size = PyBytes_GET_SIZE(initvalue); } else { diff --git a/Modules/_sqlite/connection.c b/Modules/_sqlite/connection.c --- a/Modules/_sqlite/connection.c +++ b/Modules/_sqlite/connection.c @@ -204,8 +204,8 @@ node = node->next; } - Py_DECREF(self->statement_cache); - self->statement_cache = (pysqlite_Cache*)PyObject_CallFunction((PyObject*)&pysqlite_CacheType, "O", self); + Py_SETREF(self->statement_cache, + (pysqlite_Cache *)PyObject_CallFunction((PyObject *)&pysqlite_CacheType, "O", self)); Py_DECREF(self); self->statement_cache->decref_factory = 0; } @@ -318,9 +318,8 @@ _pysqlite_drop_unused_cursor_references(self); if (cursor && self->row_factory != Py_None) { - Py_XDECREF(((pysqlite_Cursor*)cursor)->row_factory); Py_INCREF(self->row_factory); - ((pysqlite_Cursor*)cursor)->row_factory = self->row_factory; + Py_SETREF(((pysqlite_Cursor *)cursor)->row_factory, self->row_factory); } return cursor; @@ -795,8 +794,7 @@ } } - Py_DECREF(self->statements); - self->statements = new_list; + Py_SETREF(self->statements, new_list); } static void _pysqlite_drop_unused_cursor_references(pysqlite_Connection* self) @@ -827,8 +825,7 @@ } } - Py_DECREF(self->cursors); - self->cursors = new_list; + Py_SETREF(self->cursors, new_list); } PyObject* pysqlite_connection_create_function(pysqlite_Connection* self, PyObject* args, PyObject* kwargs) diff --git a/Modules/_sqlite/cursor.c b/Modules/_sqlite/cursor.c --- a/Modules/_sqlite/cursor.c +++ b/Modules/_sqlite/cursor.c @@ -170,8 +170,7 @@ return 0; } - Py_XDECREF(self->row_cast_map); - self->row_cast_map = PyList_New(0); + Py_SETREF(self->row_cast_map, PyList_New(0)); for (i = 0; i < sqlite3_column_count(self->statement->st); i++) { converter = NULL; @@ -510,9 +509,8 @@ goto error; /* reset description and rowcount */ - Py_DECREF(self->description); Py_INCREF(Py_None); - self->description = Py_None; + Py_SETREF(self->description, Py_None); self->rowcount = -1L; func_args = PyTuple_New(1); @@ -537,8 +535,8 @@ } if (self->statement->in_use) { - Py_DECREF(self->statement); - self->statement = PyObject_New(pysqlite_Statement, &pysqlite_StatementType); + Py_SETREF(self->statement, + PyObject_New(pysqlite_Statement, &pysqlite_StatementType)); if (!self->statement) { goto error; } @@ -654,8 +652,7 @@ numcols = sqlite3_column_count(self->statement->st); Py_END_ALLOW_THREADS - Py_DECREF(self->description); - self->description = PyTuple_New(numcols); + Py_SETREF(self->description, PyTuple_New(numcols)); if (!self->description) { goto error; } diff --git a/Modules/_sre.c b/Modules/_sre.c --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -753,8 +753,7 @@ if (!copy) return 0; - Py_DECREF(*object); - *object = copy; + Py_SETREF(*object, copy); return 1; /* success */ } diff --git a/Modules/_ssl.c b/Modules/_ssl.c --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@ -1589,8 +1589,7 @@ return -1; #else Py_INCREF(value); - Py_DECREF(self->ctx); - self->ctx = (PySSLContext *) value; + Py_SETREF(self->ctx, (PySSLContext *)value); SSL_set_SSL_CTX(self->ssl, self->ctx->ctx); #endif } else { @@ -1647,8 +1646,7 @@ static int PySSL_set_owner(PySSLSocket *self, PyObject *value, void *c) { - Py_XDECREF(self->owner); - self->owner = PyWeakref_NewRef(value, NULL); + Py_SETREF(self->owner, PyWeakref_NewRef(value, NULL)); if (self->owner == NULL) return -1; return 0; diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -380,9 +380,8 @@ if (tstate == NULL) return NULL; - Py_XDECREF(fatal_error.file); Py_XINCREF(file); - fatal_error.file = file; + Py_SETREF(fatal_error.file, file); fatal_error.fd = fd; fatal_error.all_threads = all_threads; fatal_error.interp = tstate->interp; @@ -599,9 +598,8 @@ /* Cancel previous thread, if running */ cancel_dump_traceback_later(); - Py_XDECREF(thread.file); Py_XINCREF(file); - thread.file = file; + Py_SETREF(thread.file, file); thread.fd = fd; thread.timeout_us = timeout_us; thread.repeat = repeat; @@ -778,9 +776,8 @@ user->previous = previous; } - Py_XDECREF(user->file); Py_XINCREF(file); - user->file = file; + Py_SETREF(user->file, file); user->fd = fd; user->all_threads = all_threads; user->chain = chain; diff --git a/Modules/itertoolsmodule.c b/Modules/itertoolsmodule.c --- a/Modules/itertoolsmodule.c +++ b/Modules/itertoolsmodule.c @@ -634,8 +634,7 @@ link = teedataobject_jumplink(to->dataobj); if (link == NULL) return NULL; - Py_DECREF(to->dataobj); - to->dataobj = (teedataobject *)link; + Py_SETREF(to->dataobj, (teedataobject *)link); to->index = 0; } value = teedataobject_getitem(to->dataobj, to->index); diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -1266,8 +1266,7 @@ if (Handlers[SIGINT].func == DefaultHandler) { /* Install default int handler */ Py_INCREF(IntHandler); - Py_DECREF(Handlers[SIGINT].func); - Handlers[SIGINT].func = IntHandler; + Py_SETREF(Handlers[SIGINT].func, IntHandler); old_siginthandler = PyOS_setsig(SIGINT, signal_handler); } diff --git a/Modules/zipimport.c b/Modules/zipimport.c --- a/Modules/zipimport.c +++ b/Modules/zipimport.c @@ -155,8 +155,7 @@ tmp = PyUnicode_FromFormat("%U%c", self->prefix, SEP); if (tmp == NULL) goto error; - Py_DECREF(self->prefix); - self->prefix = tmp; + Py_SETREF(self->prefix, tmp); } } else diff --git a/Modules/zlibmodule.c b/Modules/zlibmodule.c --- a/Modules/zlibmodule.c +++ b/Modules/zlibmodule.c @@ -667,8 +667,7 @@ PyBytes_AS_STRING(self->unused_data), old_size); Py_MEMCPY(PyBytes_AS_STRING(new_data) + old_size, self->zst.next_in, self->zst.avail_in); - Py_DECREF(self->unused_data); - self->unused_data = new_data; + Py_SETREF(self->unused_data, new_data); self->zst.avail_in = 0; } } @@ -680,8 +679,7 @@ (char *)self->zst.next_in, self->zst.avail_in); if (new_data == NULL) return -1; - Py_DECREF(self->unconsumed_tail); - self->unconsumed_tail = new_data; + Py_SETREF(self->unconsumed_tail, new_data); } return 0; } diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -3688,8 +3688,7 @@ /* Multiple references, need to create new object */ PyObject *v; v = bytes_concat(*pv, w); - Py_DECREF(*pv); - *pv = v; + Py_SETREF(*pv, v); } } diff --git a/Objects/descrobject.c b/Objects/descrobject.c --- a/Objects/descrobject.c +++ b/Objects/descrobject.c @@ -1509,8 +1509,7 @@ PyObject *get_doc = _PyObject_GetAttrId(get, &PyId___doc__); if (get_doc) { if (Py_TYPE(self) == &PyProperty_Type) { - Py_XDECREF(prop->prop_doc); - prop->prop_doc = get_doc; + Py_SETREF(prop->prop_doc, get_doc); } else { /* If this is a property subclass, put __doc__ diff --git a/Objects/exceptions.c b/Objects/exceptions.c --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -236,8 +236,7 @@ } Py_XINCREF(tb); - Py_XDECREF(self->traceback); - self->traceback = tb; + Py_SETREF(self->traceback, tb); return 0; } diff --git a/Objects/frameobject.c b/Objects/frameobject.c --- a/Objects/frameobject.c +++ b/Objects/frameobject.c @@ -857,8 +857,7 @@ } } else if (values[j] != value) { Py_XINCREF(value); - Py_XDECREF(values[j]); - values[j] = value; + Py_SETREF(values[j], value); } Py_XDECREF(value); } diff --git a/Objects/funcobject.c b/Objects/funcobject.c --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -127,8 +127,7 @@ PyErr_SetString(PyExc_SystemError, "non-tuple default args"); return -1; } - Py_XDECREF(((PyFunctionObject *) op) -> func_defaults); - ((PyFunctionObject *) op) -> func_defaults = defaults; + Py_SETREF(((PyFunctionObject *)op)->func_defaults, defaults); return 0; } @@ -159,8 +158,7 @@ "non-dict keyword only default args"); return -1; } - Py_XDECREF(((PyFunctionObject *)op) -> func_kwdefaults); - ((PyFunctionObject *) op) -> func_kwdefaults = defaults; + Py_SETREF(((PyFunctionObject *)op)->func_kwdefaults, defaults); return 0; } @@ -192,8 +190,7 @@ closure->ob_type->tp_name); return -1; } - Py_XDECREF(((PyFunctionObject *) op) -> func_closure); - ((PyFunctionObject *) op) -> func_closure = closure; + Py_SETREF(((PyFunctionObject *)op)->func_closure, closure); return 0; } @@ -224,8 +221,7 @@ "non-dict annotations"); return -1; } - Py_XDECREF(((PyFunctionObject *)op) -> func_annotations); - ((PyFunctionObject *) op) -> func_annotations = annotations; + Py_SETREF(((PyFunctionObject *)op)->func_annotations, annotations); return 0; } @@ -531,8 +527,7 @@ if (name != Py_None) { Py_INCREF(name); - Py_DECREF(newfunc->func_name); - newfunc->func_name = name; + Py_SETREF(newfunc->func_name, name); } if (defaults != Py_None) { Py_INCREF(defaults); diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c --- a/Objects/moduleobject.c +++ b/Objects/moduleobject.c @@ -69,8 +69,7 @@ return -1; if (PyUnicode_CheckExact(name)) { Py_INCREF(name); - Py_XDECREF(mod->md_name); - mod->md_name = name; + Py_SETREF(mod->md_name, name); } return 0; diff --git a/Objects/rangeobject.c b/Objects/rangeobject.c --- a/Objects/rangeobject.c +++ b/Objects/rangeobject.c @@ -1066,8 +1066,7 @@ result = PyNumber_Add(r->start, product); Py_DECREF(product); if (result) { - Py_DECREF(r->index); - r->index = new_index; + Py_SETREF(r->index, new_index); } else { Py_DECREF(new_index); diff --git a/Objects/typeobject.c b/Objects/typeobject.c --- a/Objects/typeobject.c +++ b/Objects/typeobject.c @@ -315,9 +315,8 @@ are borrowed reference */ for (i = 0; i < (1 << MCACHE_SIZE_EXP); i++) { method_cache[i].value = NULL; - Py_XDECREF(method_cache[i].name); - method_cache[i].name = Py_None; Py_INCREF(Py_None); + Py_SETREF(method_cache[i].name, Py_None); } /* mark all version tags as invalid */ PyType_Modified(&PyBaseObject_Type); @@ -462,8 +461,7 @@ et = (PyHeapTypeObject*)type; Py_INCREF(value); - Py_DECREF(et->ht_qualname); - et->ht_qualname = value; + Py_SETREF(et->ht_qualname, value); return 0; } @@ -2918,8 +2916,7 @@ else method_cache_misses++; #endif - Py_DECREF(method_cache[h].name); - method_cache[h].name = name; + Py_SETREF(method_cache[h].name, name); } return res; } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -1828,8 +1828,7 @@ _Py_INCREF_UNICODE_EMPTY(); if (!unicode_empty) return -1; - Py_DECREF(*p_unicode); - *p_unicode = unicode_empty; + Py_SETREF(*p_unicode, unicode_empty); return 0; } @@ -1837,8 +1836,7 @@ PyObject *copy = resize_copy(unicode, length); if (copy == NULL) return -1; - Py_DECREF(*p_unicode); - *p_unicode = copy; + Py_SETREF(*p_unicode, copy); return 0; } @@ -13543,8 +13541,7 @@ return -1; _PyUnicode_FastCopyCharacters(newbuffer, 0, writer->buffer, 0, writer->pos); - Py_DECREF(writer->buffer); - writer->buffer = newbuffer; + Py_SETREF(writer->buffer, newbuffer); } _PyUnicodeWriter_Update(writer); return 0; @@ -15261,8 +15258,7 @@ if (t) { Py_INCREF(t); - Py_DECREF(*p); - *p = t; + Py_SETREF(*p, t); return; } diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -524,9 +524,8 @@ if (stream == NULL) goto cleanup; - Py_XDECREF(tok->decoding_readline); readline = _PyObject_GetAttrId(stream, &PyId_readline); - tok->decoding_readline = readline; + Py_SETREF(tok->decoding_readline, readline); if (pos > 0) { if (PyObject_CallObject(readline, NULL) == NULL) { readline = NULL; diff --git a/Python/_warnings.c b/Python/_warnings.c --- a/Python/_warnings.c +++ b/Python/_warnings.c @@ -680,8 +680,7 @@ goto handle_error; } else if (!is_true) { - Py_DECREF(*filename); - *filename = PyUnicode_FromString("__main__"); + Py_SETREF(*filename, PyUnicode_FromString("__main__")); if (*filename == NULL) goto handle_error; } diff --git a/Python/ceval.c b/Python/ceval.c --- a/Python/ceval.c +++ b/Python/ceval.c @@ -3221,8 +3221,7 @@ Py_INCREF(self); func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); - Py_DECREF(*pfunc); - *pfunc = self; + Py_SETREF(*pfunc, self); na++; /* n++; */ } else @@ -4734,8 +4733,7 @@ Py_INCREF(self); func = PyMethod_GET_FUNCTION(func); Py_INCREF(func); - Py_DECREF(*pfunc); - *pfunc = self; + Py_SETREF(*pfunc, self); na++; n++; } else diff --git a/Python/compile.c b/Python/compile.c --- a/Python/compile.c +++ b/Python/compile.c @@ -1800,8 +1800,7 @@ { /* use the class name for name mangling */ Py_INCREF(s->v.ClassDef.name); - Py_XDECREF(c->u->u_private); - c->u->u_private = s->v.ClassDef.name; + Py_SETREF(c->u->u_private, s->v.ClassDef.name); /* load (global) __name__ ... */ str = PyUnicode_InternFromString("__name__"); if (!str || !compiler_nameop(c, str, Load)) { -- Repository URL: https://hg.python.org/cpython

1 0

cpython (merge 3.5 -> default): Merge: #1753718: clarify RFC compliance and bytes/string argument types.
by r.david.murray 23 Dec '15

23 Dec '15

https://hg.python.org/cpython/rev/92760d2edc9e changeset: 99667:92760d2edc9e parent: 99665:dcf9e9ae5393 parent: 99666:105bf5dd93b8 user: R David Murray <rdmurray(a)bitdance.com> date: Wed Dec 23 21:19:53 2015 -0500 summary: Merge: #1753718: clarify RFC compliance and bytes/string argument types. files: Doc/library/base64.rst | 128 ++++++++++++++++------------ 1 files changed, 71 insertions(+), 57 deletions(-) diff --git a/Doc/library/base64.rst b/Doc/library/base64.rst --- a/Doc/library/base64.rst +++ b/Doc/library/base64.rst @@ -21,13 +21,19 @@ POST request. The encoding algorithm is not the same as the :program:`uuencode` program. -There are two :rfc:`3548` interfaces provided by this module. The modern -interface supports encoding and decoding ASCII byte string objects using all -three :rfc:`3548` defined alphabets (normal, URL-safe, and filesystem-safe). -Additionally, the decoding functions of the modern interface also accept -Unicode strings containing only ASCII characters. The legacy interface provides -for encoding and decoding to and from file-like objects as well as byte -strings, but only using the Base64 standard alphabet. +There are two interfaces provided by this module. The modern interface +supports encoding :term:`bytes-like objects <bytes-like object>` to ASCII +:class:`bytes`, and decoding :term:`bytes-like objects <bytes-like object>` or +strings containing ASCII to :class:`bytes`. All three :rfc:`3548` defined +alphabets (normal, URL-safe, and filesystem-safe) are supported. + +The legacy interface does not support decoding from strings, but it does +provide functions for encoding and decoding to and from :term:`file objects +<file object>`. It only supports the Base64 standard alphabet, and it adds +newlines every 76 characters as per :rfc:`2045`. Note that if you are looking +for :rfc:`2045` support you probably want to be looking at the :mod:`email` +package instead. + .. versionchanged:: 3.3 ASCII-only Unicode strings are now accepted by the decoding functions of @@ -41,26 +47,26 @@ .. function:: b64encode(s, altchars=None) - Encode a byte string using Base64. + Encode the :term:`bytes-like object` *s* using Base64 and return the encoded + :class:`bytes`. - *s* is the string to encode. Optional *altchars* must be a string of at least + Optional *altchars* must be a :term:`bytes-like object` of at least length 2 (additional characters are ignored) which specifies an alternative alphabet for the ``+`` and ``/`` characters. This allows an application to e.g. generate URL or filesystem safe Base64 strings. The default is ``None``, for which the standard Base64 alphabet is used. - The encoded byte string is returned. - .. function:: b64decode(s, altchars=None, validate=False) - Decode a Base64 encoded byte string. + Decode the Base64 encoded :term:`bytes-like object` or ASCII string + *s* and return the decoded :class:`bytes`. - *s* is the byte string to decode. Optional *altchars* must be a string of + Optional *altchars* must be a :term:`bytes-like object` or ASCII string of at least length 2 (additional characters are ignored) which specifies the alternative alphabet used instead of the ``+`` and ``/`` characters. - The decoded string is returned. A :exc:`binascii.Error` exception is raised + A :exc:`binascii.Error` exception is raised if *s* is incorrectly padded. If *validate* is ``False`` (the default), non-base64-alphabet characters are @@ -71,38 +77,44 @@ .. function:: standard_b64encode(s) - Encode byte string *s* using the standard Base64 alphabet. + Encode :term:`bytes-like object` *s* using the standard Base64 alphabet + and return the encoded :class:`bytes`. .. function:: standard_b64decode(s) - Decode byte string *s* using the standard Base64 alphabet. + Decode :term:`bytes-like object` or ASCII string *s* using the standard + Base64 alphabet and return the decoded :class:`bytes`. .. function:: urlsafe_b64encode(s) - Encode byte string *s* using a URL-safe alphabet, which substitutes ``-`` instead of - ``+`` and ``_`` instead of ``/`` in the standard Base64 alphabet. The result + Encode :term:`bytes-like object` *s* using a URL-safe alphabet, which + substitutes ``-`` instead of ``+`` and ``_`` instead of ``/`` in the + standard Base64 alphabet, and return the encoded :class:`bytes`. The result can still contain ``=``. .. function:: urlsafe_b64decode(s) - Decode byte string *s* using a URL-safe alphabet, which substitutes ``-`` instead of - ``+`` and ``_`` instead of ``/`` in the standard Base64 alphabet. + Decode :term:`bytes-like object` or ASCII string *s* using a URL-safe + alphabet, which substitutes ``-`` instead of ``+`` and ``_`` instead of + ``/`` in the standard Base64 alphabet, and return the decoded + :class:`bytes`. .. function:: b32encode(s) - Encode a byte string using Base32. *s* is the string to encode. The encoded string - is returned. + Encode the :term:`bytes-like object` *s* using Base32 and return the + encoded :class:`bytes`. .. function:: b32decode(s, casefold=False, map01=None) - Decode a Base32 encoded byte string. + Decode the Base32 encoded :term:`bytes-like object` or ASCII string *s* and + return the decoded :class:`bytes`. - *s* is the byte string to decode. Optional *casefold* is a flag specifying + Optional *casefold* is a flag specifying whether a lowercase alphabet is acceptable as input. For security purposes, the default is ``False``. @@ -113,46 +125,45 @@ digit 0 is always mapped to the letter O). For security purposes the default is ``None``, so that 0 and 1 are not allowed in the input. - The decoded byte string is returned. A :exc:`binascii.Error` is raised if *s* is + A :exc:`binascii.Error` is raised if *s* is incorrectly padded or if there are non-alphabet characters present in the - string. + input. .. function:: b16encode(s) - Encode a byte string using Base16. - - *s* is the string to encode. The encoded byte string is returned. + Encode the :term:`bytes-like object` *s* using Base16 and return the + encoded :class:`bytes`. .. function:: b16decode(s, casefold=False) - Decode a Base16 encoded byte string. + Decode the Base16 encoded :term:`bytes-like object` or ASCII string *s* and + return the decoded :class:`bytes`. - *s* is the string to decode. Optional *casefold* is a flag specifying whether a + Optional *casefold* is a flag specifying whether a lowercase alphabet is acceptable as input. For security purposes, the default is ``False``. - The decoded byte string is returned. A :exc:`TypeError` is raised if *s* were + A :exc:`TypeError` is raised if *s* is incorrectly padded or if there are non-alphabet characters present in the - string. + input. .. function:: a85encode(s, *, foldspaces=False, wrapcol=0, pad=False, adobe=False) - Encode a byte string using Ascii85. - - *s* is the string to encode. The encoded byte string is returned. + Encode the :term:`bytes-like object` *s* using Ascii85 and return the + encoded :class:`bytes`. *foldspaces* is an optional flag that uses the special short sequence 'y' instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This feature is not supported by the "standard" Ascii85 encoding. - *wrapcol* controls whether the output should have newline (``'\n'``) + *wrapcol* controls whether the output should have newline (``b'\n'``) characters added to it. If this is non-zero, each output line will be at most this many characters long. - *pad* controls whether the input string is padded to a multiple of 4 + *pad* controls whether the input is padded to a multiple of 4 before encoding. Note that the ``btoa`` implementation always pads. *adobe* controls whether the encoded byte sequence is framed with ``<~`` @@ -163,9 +174,8 @@ .. function:: a85decode(s, *, foldspaces=False, adobe=False, ignorechars=b' \\t\\n\\r\\v') - Decode an Ascii85 encoded byte string. - - *s* is the byte string to decode. + Decode the Ascii85 encoded :term:`bytes-like object` or ASCII string *s* and + return the decoded :class:`bytes`. *foldspaces* is a flag that specifies whether the 'y' short sequence should be accepted as shorthand for 4 consecutive spaces (ASCII 0x20). @@ -174,7 +184,8 @@ *adobe* controls whether the input sequence is in Adobe Ascii85 format (i.e. is framed with <~ and ~>). - *ignorechars* should be a byte string containing characters to ignore + *ignorechars* should be a :term:`bytes-like object` or ASCII string + containing characters to ignore from the input. This should only contain whitespace characters, and by default contains all whitespace characters in ASCII. @@ -183,18 +194,19 @@ .. function:: b85encode(s, pad=False) - Encode a byte string using base85, as used in e.g. git-style binary - diffs. + Encode the :term:`bytes-like object` *s* using base85 (as used in e.g. + git-style binary diffs) and return the encoded :class:`bytes`. - If *pad* is true, the input is padded with "\\0" so its length is a - multiple of 4 characters before encoding. + If *pad* is true, the input is padded with ``b'\0'`` so its length is a + multiple of 4 bytes before encoding. .. versionadded:: 3.4 .. function:: b85decode(b) - Decode base85-encoded byte string. Padding is implicitly removed, if + Decode the base85-encoded :term:`bytes-like object` or ASCII string *b* and + return the decoded :class:`bytes`. Padding is implicitly removed, if necessary. .. versionadded:: 3.4 @@ -214,15 +226,15 @@ Decode the contents of the binary *input* file and write the resulting binary data to the *output* file. *input* and *output* must be :term:`file objects - <file object>`. *input* will be read until ``input.read()`` returns an empty - bytes object. + <file object>`. *input* will be read until ``input.readline()`` returns an + empty bytes object. .. function:: decodebytes(s) decodestring(s) - Decode the byte string *s*, which must contain one or more lines of base64 - encoded data, and return a byte string containing the resulting binary data. + Decode the :term:`bytes-like object` *s*, which must contain one or more + lines of base64 encoded data, and return the decoded :class:`bytes`. ``decodestring`` is a deprecated alias. .. versionadded:: 3.1 @@ -233,17 +245,19 @@ Encode the contents of the binary *input* file and write the resulting base64 encoded data to the *output* file. *input* and *output* must be :term:`file objects <file object>`. *input* will be read until ``input.read()`` returns - an empty bytes object. :func:`encode` returns the encoded data plus a trailing - newline character (``b'\n'``). + an empty bytes object. :func:`encode` inserts a newline character (``b'\n'``) + after every 76 bytes of the output, as well as ensuring that the output + always ends with a newline, as per :rfc:`2045` (MIME). .. function:: encodebytes(s) encodestring(s) - Encode the byte string *s*, which can contain arbitrary binary data, and - return a byte string containing one or more lines of base64-encoded data. - :func:`encodebytes` returns a string containing one or more lines of - base64-encoded data always including an extra trailing newline (``b'\n'``). + Encode the :term:`bytes-like object` *s*, which can contain arbitrary binary + data, and return :class:`bytes` containing the base64-encoded data, with newlines + (``b'\n'``) inserted after every 76 bytes of output, and ensuring that + there is a trailing newline, as per :rfc:`2045` (MIME). + ``encodestring`` is a deprecated alias. -- Repository URL: https://hg.python.org/cpython

1 0