Mailman 3 September 2018 - Python-Dev

Request for review: binary op dispatch rules for subclasses
by Stephan Hoyer Sept. 17, 2018

Sept. 17, 2018

Over a year ago, I made a pull request ( https://github.com/python/cpython/pull/1325) to fix a long-standing issue with how Python handles dispatch for arithmetic binary operations involving subclasses (https://bugs.python.org/issue30140). I pinged the bug several times, but I'm still waiting for a review, which would be greatly appreciated! Best, Stephan

2 1

Re: [Python-Dev] SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register
by Wes Turner Sept. 17, 2018

Sept. 17, 2018

Are all current Python builds and C extensions vulnerable to Spectre variants {1, 2, *}? There are now multiple threads: "SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register" - https://mail.python.org/mm3/archives/list/distutils-sig@python.org/thread/4… - https://mail.python.org/pipermail/python-ideas/2018-September/053473.html - https://mail.python.org/pipermail/python-dev/2018-September/155199.html Original thread (that I forwarded to security@): "[Python-ideas] Executable space protection: NX bit," https://mail.python.org/pipermail/python-ideas/2018-September/053175.html > ~ Do trampolines / nested functions in C extensions switch off the NX bit? On Sunday, September 16, 2018, Nathaniel Smith <njs(a)pobox.com> wrote: > On Wed, Sep 12, 2018, 12:29 Joni Orponen <j.orponen(a)4teamwork.ch> wrote: > >> On Wed, Sep 12, 2018 at 8:48 PM Wes Turner <wes.turner(a)gmail.com> wrote: >> >>> Should C extensions that compile all add >>> `-mindirect-branch=thunk -mindirect-branch-register` [1] to mitigate the >>> risk of Spectre variant 2 (which does indeed affect user space applications >>> as well as kernels)? >>> >> >> Are those available on GCC <= 4.2.0 as per PEP 513? >> > > Pretty sure no manylinux1 compiler is ever going to get these mitigations. > > For manylinux2010 on x86-64, we can easily use a much newer compiler: RH > maintains a recent compiler, currently gcc 7.3, or if that doesn't work for > some reason then the conda folks have be apparently figured out how to > build the equivalent from gcc upstream releases. > Are there different CFLAGS and/or gcc compatibility flags in conda builds of Python and C extensions? Where are those set in conda builds? What's the best way to set CFLAGS in Python builds and C extensions? export CFLAGS="-mindirect-branch=thunk -mindirect-branch-register" ./configure make ? Why are we supposed to use an old version of GCC that doesn't have the retpoline patches that only mitigate Spectre variant 2? > > Unfortunately, the manylinux2010 infrastructure is not quite ready... I'm > pretty sure it needs some volunteers to push it to the finish line, though > unfortunately I haven't had enough time to keep track. > "PEP 571 -- The manylinux2010 Platform Tag" https://www.python.org/dev/peps/pep-0571/ "Tracking issue for manylinux2010 rollout" https://github.com/pypa/manylinux/issues/179 Are all current Python builds and C extensions vulnerable to Spectre variants {1, 2, *}? >

2 1

Summary of Python tracker Issues
by Python tracker Sept. 14, 2018

Sept. 14, 2018

ACTIVITY SUMMARY (2018-09-07 - 2018-09-14) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 6819 (-22) closed 39612 (+95) total 46431 (+73) Open issues with patches: 2716 Issues opened (38) ================== #23354: Loading 2 GiLOC file which raises exception causes wrong trace https://bugs.python.org/issue23354 reopened by benjamin.peterson #26544: platform.libc_ver() returns incorrect version number https://bugs.python.org/issue26544 reopened by vstinner #33083: math.factorial accepts non-integral Decimal instances https://bugs.python.org/issue33083 reopened by vstinner #34609: Idle Unitest https://bugs.python.org/issue34609 opened by piscvau(a)yahoo.fr #34610: Incorrect iteration of Manager.dict() method of the multiproce https://bugs.python.org/issue34610 opened by deltaclock #34613: asyncio.StreamReader initialization documentation incorrectly https://bugs.python.org/issue34613 opened by psycojoker #34616: implement "Async exec" https://bugs.python.org/issue34616 opened by mbussonn #34617: socket.recvfrom(): docs should warn about packet truncation wh https://bugs.python.org/issue34617 opened by Anees Ahmed #34620: Octal byte literals with a decimal value > 255 are silently tr https://bugs.python.org/issue34620 opened by bup #34623: _elementtree.c doesn't call XML_SetHashSalt() https://bugs.python.org/issue34623 opened by christian.heimes #34624: -W option does not accept module regexes https://bugs.python.org/issue34624 opened by coldfix #34626: PEP 384's PyType_Spec and PyType_Slot are not documented https://bugs.python.org/issue34626 opened by petr.viktorin #34628: urllib.request.urlopen fails when userinfo is present in URL https://bugs.python.org/issue34628 opened by ytvwld #34629: Python3 regression for urllib(2).urlopen(...).fp for chunked h https://bugs.python.org/issue34629 opened by tkruse #34630: Don't log ssl cert errors in asyncio https://bugs.python.org/issue34630 opened by asvetlov #34631: Upgrade to OpenSSL 1.1.1 https://bugs.python.org/issue34631 opened by christian.heimes #34632: Port importlib_metadata to Python 3.8 https://bugs.python.org/issue34632 opened by barry #34634: New asyncio streams API https://bugs.python.org/issue34634 opened by asvetlov #34639: PYTHONCOERCECLOCALE is ignored when using -E or -I option https://bugs.python.org/issue34639 opened by vstinner #34643: How to build Release Version of Python in Windows? https://bugs.python.org/issue34643 opened by Valentin Zhao #34648: Confirm the types of parameters of traceback.format_list and t https://bugs.python.org/issue34648 opened by Nathaniel Manista #34651: Disallow fork in a subinterpreter. https://bugs.python.org/issue34651 opened by eric.snow #34655: Support sendfile in asyncio streams API https://bugs.python.org/issue34655 opened by asvetlov #34656: memory exhaustion in Modules/_pickle.c:1393 https://bugs.python.org/issue34656 opened by shuoz #34659: Inconsistency between functools.reduce & itertools.accumulate https://bugs.python.org/issue34659 opened by lycantropos #34662: tarfile.TarFile may write corrupt files if not closed https://bugs.python.org/issue34662 opened by lordmauve #34663: Support POSIX_SPAWN_USEVFORK flag in posix_spawn https://bugs.python.org/issue34663 opened by pablogsal #34665: Py_FinalizeEx() - Bugs & caveats - Add info that NumPy and Pan https://bugs.python.org/issue34665 opened by jcmuel #34667: Review documentation section by section https://bugs.python.org/issue34667 opened by willingc #34669: test_ssl fails if SSLv2 is enabled https://bugs.python.org/issue34669 opened by benjamin.peterson #34670: Add set_post_handshake_auth for TLS 1.3 https://bugs.python.org/issue34670 opened by christian.heimes #34672: '%Z' strftime specifier never works with musl https://bugs.python.org/issue34672 opened by benjamin.peterson #34673: make the eval loop more editable https://bugs.python.org/issue34673 opened by benjamin.peterson #34676: Guarantie that divmod() and PyNumber_Divmod() return a 2-tuple https://bugs.python.org/issue34676 opened by serhiy.storchaka #34677: Event scheduler page example https://bugs.python.org/issue34677 opened by mediator42 #34678: need to remove the term "white space" https://bugs.python.org/issue34678 opened by dwaygig #34679: asyncio.add_signal_handler call fails if not on main thread https://bugs.python.org/issue34679 opened by jnwatson #34680: asyncio event_loop close fails off main thread if signal handl https://bugs.python.org/issue34680 opened by jnwatson Most recent 15 issues with no replies (15) ========================================== #34680: asyncio event_loop close fails off main thread if signal handl https://bugs.python.org/issue34680 #34679: asyncio.add_signal_handler call fails if not on main thread https://bugs.python.org/issue34679 #34677: Event scheduler page example https://bugs.python.org/issue34677 #34676: Guarantie that divmod() and PyNumber_Divmod() return a 2-tuple https://bugs.python.org/issue34676 #34673: make the eval loop more editable https://bugs.python.org/issue34673 #34672: '%Z' strftime specifier never works with musl https://bugs.python.org/issue34672 #34670: Add set_post_handshake_auth for TLS 1.3 https://bugs.python.org/issue34670 #34667: Review documentation section by section https://bugs.python.org/issue34667 #34665: Py_FinalizeEx() - Bugs & caveats - Add info that NumPy and Pan https://bugs.python.org/issue34665 #34655: Support sendfile in asyncio streams API https://bugs.python.org/issue34655 #34634: New asyncio streams API https://bugs.python.org/issue34634 #34632: Port importlib_metadata to Python 3.8 https://bugs.python.org/issue34632 #34631: Upgrade to OpenSSL 1.1.1 https://bugs.python.org/issue34631 #34629: Python3 regression for urllib(2).urlopen(...).fp for chunked h https://bugs.python.org/issue34629 #34626: PEP 384's PyType_Spec and PyType_Slot are not documented https://bugs.python.org/issue34626 Most recent 15 issues waiting for review (15) ============================================= #34676: Guarantie that divmod() and PyNumber_Divmod() return a 2-tuple https://bugs.python.org/issue34676 #34673: make the eval loop more editable https://bugs.python.org/issue34673 #34672: '%Z' strftime specifier never works with musl https://bugs.python.org/issue34672 #34663: Support POSIX_SPAWN_USEVFORK flag in posix_spawn https://bugs.python.org/issue34663 #34656: memory exhaustion in Modules/_pickle.c:1393 https://bugs.python.org/issue34656 #34651: Disallow fork in a subinterpreter. https://bugs.python.org/issue34651 #34634: New asyncio streams API https://bugs.python.org/issue34634 #34630: Don't log ssl cert errors in asyncio https://bugs.python.org/issue34630 #34626: PEP 384's PyType_Spec and PyType_Slot are not documented https://bugs.python.org/issue34626 #34623: _elementtree.c doesn't call XML_SetHashSalt() https://bugs.python.org/issue34623 #34613: asyncio.StreamReader initialization documentation incorrectly https://bugs.python.org/issue34613 #34610: Incorrect iteration of Manager.dict() method of the multiproce https://bugs.python.org/issue34610 #34604: Possible mojibake in pwd.getpwnam and grp.getgrnam https://bugs.python.org/issue34604 #34603: ctypes on Windows: error calling C function that returns a str https://bugs.python.org/issue34603 #34596: [unittest] raise error if @skip is used with an argument that https://bugs.python.org/issue34596 Top 10 most discussed issues (10) ================================= #34580: sqlite doc: clarify the scope of the context manager https://bugs.python.org/issue34580 10 msgs #34589: Py_Initialize() and Py_Main() should not enable C locale coerc https://bugs.python.org/issue34589 10 msgs #34595: PyUnicode_FromFormat(): add %T format for an object type name https://bugs.python.org/issue34595 10 msgs #34421: Cannot install package with unicode module names on Windows https://bugs.python.org/issue34421 9 msgs #34604: Possible mojibake in pwd.getpwnam and grp.getgrnam https://bugs.python.org/issue34604 9 msgs #31577: crash in os.utime() in case of a bad ns argument https://bugs.python.org/issue31577 8 msgs #34597: Python needs to check existence of functions at runtime for ta https://bugs.python.org/issue34597 8 msgs #33649: asyncio docs overhaul https://bugs.python.org/issue33649 7 msgs #1621: Do not assume signed integer overflow behavior https://bugs.python.org/issue1621 6 msgs #34600: python3 regression ElementTree.iterparse() unable to capture c https://bugs.python.org/issue34600 6 msgs Issues closed (89) ================== #7713: implement ability to disable automatic search path additions https://bugs.python.org/issue7713 closed by willingc #21258: Add __iter__ support for mock_open https://bugs.python.org/issue21258 closed by berker.peksag #23855: Missing Sanity Check for malloc() in PC/_msi.c https://bugs.python.org/issue23855 closed by berker.peksag #24696: Don't use None as sentinel for traceback https://bugs.python.org/issue24696 closed by benjamin.peterson #25041: document AF_PACKET socket address format https://bugs.python.org/issue25041 closed by benjamin.peterson #25083: Python can sometimes create incorrect .pyc files https://bugs.python.org/issue25083 closed by petr.viktorin #29051: Improve error reporting involving f-strings (PEP 498) https://bugs.python.org/issue29051 closed by eric.smith #29386: select.epoll.poll may behave differently if timeout = -1 vs ti https://bugs.python.org/issue29386 closed by berker.peksag #29832: Don't refer to getsockaddrarg in error messages https://bugs.python.org/issue29832 closed by benjamin.peterson #30576: http.server should support HTTP compression (gzip) https://bugs.python.org/issue30576 closed by brett.cannon #31132: test_prlimit from test_resource fails when building python3 in https://bugs.python.org/issue31132 closed by benjamin.peterson #31141: Start should be a keyword argument of the built-in sum https://bugs.python.org/issue31141 closed by rhettinger #31608: crash in methods of a subclass of _collections.deque with a ba https://bugs.python.org/issue31608 closed by benjamin.peterson #31704: HTTP check lowercase response from proxy https://bugs.python.org/issue31704 closed by benjamin.peterson #31734: crash or SystemError in sqlite3.Cache in case it is uninitiali https://bugs.python.org/issue31734 closed by berker.peksag #31801: vars() manipulation encounters problems with Enum https://bugs.python.org/issue31801 closed by ethan.furman #31902: Fix col_offset for ast nodes: AsyncFor, AsyncFunctionDef, Asyn https://bugs.python.org/issue31902 closed by miss-islington #31903: `_scproxy` calls SystemConfiguration functions in a way that c https://bugs.python.org/issue31903 closed by benjamin.peterson #32270: subprocess closes redirected fds even if they are in pass_fds https://bugs.python.org/issue32270 closed by izbyshev #32490: subprocess: duplicate filename in exception message https://bugs.python.org/issue32490 closed by benjamin.peterson #32933: mock_open does not support iteration around text files. https://bugs.python.org/issue32933 closed by berker.peksag #33032: Mention implicit cache in struct.Struct docs https://bugs.python.org/issue33032 closed by gregory.p.smith #33073: Add as_integer_ratio() to int() objects https://bugs.python.org/issue33073 closed by rhettinger #33217: x in enum.Flag member is True when x is not a Flag https://bugs.python.org/issue33217 closed by ethan.furman #33487: BZ2File(buffering=None) does not emit deprecation warning, dep https://bugs.python.org/issue33487 closed by gregory.p.smith #33604: HMAC default to MD5 marked as to be removed in 3.6 https://bugs.python.org/issue33604 closed by gregory.p.smith #33625: Release GIL for grp.getgr{nam,gid} and pwd.getpw{nam,uid} https://bugs.python.org/issue33625 closed by serhiy.storchaka #33774: Document that @lru_cache caches based on exactly how the funct https://bugs.python.org/issue33774 closed by rhettinger #33883: doc Mention mypy, pyrex, pytype and PyAnnotate in FAQ https://bugs.python.org/issue33883 closed by benjamin.peterson #33986: asyncio: Typo in documentation: BaseSubprocessTransport -> Sub https://bugs.python.org/issue33986 closed by asvetlov #34004: Acquiring locks not interrupted by signals on musl libc https://bugs.python.org/issue34004 closed by benjamin.peterson #34082: EnumMeta.__new__ should use enum_class.__new__ https://bugs.python.org/issue34082 closed by ethan.furman #34194: test_ssl, AIX, and defaults for _ssl connections https://bugs.python.org/issue34194 closed by Michael.Felt #34200: importlib: python -m test test_pkg fails semi-randomly https://bugs.python.org/issue34200 closed by gregory.p.smith #34213: Frozen dataclass __init__ fails for "object" property" https://bugs.python.org/issue34213 closed by eric.smith #34246: Gentoo Refleaks 3.7: test_smtplib has dangling threads https://bugs.python.org/issue34246 closed by pablogsal #34286: lib2to3 tests fail on the 3.7 branch (used to work with 3.7.0) https://bugs.python.org/issue34286 closed by ned.deily #34365: datetime's documentation refers to "comparison [...] falling b https://bugs.python.org/issue34365 closed by Mariatta #34405: Upgrade to OpenSSL 1.1.0i / 1.0.2p https://bugs.python.org/issue34405 closed by ned.deily #34409: Add a way to customize iteration over fields in asdict() for t https://bugs.python.org/issue34409 closed by eric.smith #34455: Tkinter crashing when pressing Command + ^ (OSX) https://bugs.python.org/issue34455 closed by ned.deily #34487: enum _sunder_ names mix metaclass and enum class attributes https://bugs.python.org/issue34487 closed by ethan.furman #34490: transport.get_extra_info('sockname') of test_asyncio fails on https://bugs.python.org/issue34490 closed by asvetlov #34525: smtplib's authobject return value wrongly documented https://bugs.python.org/issue34525 closed by benjamin.peterson #34546: Add encryption support to zipfile https://bugs.python.org/issue34546 closed by serhiy.storchaka #34578: Pipenv lock : ModuleNotFoundError: No module named '_ctypes' https://bugs.python.org/issue34578 closed by ned.deily #34586: collections.ChainMap should have a get_where method https://bugs.python.org/issue34586 closed by rhettinger #34588: traceback formatting can drop a frame https://bugs.python.org/issue34588 closed by benjamin.peterson #34598: How to fix? Error in Kali linux python 2.7 - Collecting pip Fr https://bugs.python.org/issue34598 closed by benjamin.peterson #34605: Avoid master/slave terminology https://bugs.python.org/issue34605 closed by gvanrossum #34606: Unable to read zip file with extra https://bugs.python.org/issue34606 closed by serhiy.storchaka #34608: gc.get_referrers behavior change 3.6 to 3.7 https://bugs.python.org/issue34608 closed by inada.naoki #34611: some examples in 'itertools' modules docs are inaccuracy. https://bugs.python.org/issue34611 closed by rhettinger #34612: doc Some classes are treated as functions in Built-in Function https://bugs.python.org/issue34612 closed by rhettinger #34614: Builtin `abs(Path)` should return `Path.absolute()`. https://bugs.python.org/issue34614 closed by pitrou #34615: subprocess.call wrong exit code https://bugs.python.org/issue34615 closed by kayhayen #34618: Encoding error running in subprocess with captured output https://bugs.python.org/issue34618 closed by eryksun #34619: Typo in docs.python.jp https://bugs.python.org/issue34619 closed by brett.cannon #34621: uuid.UUID objects can't be unpickled in older Python versions https://bugs.python.org/issue34621 closed by taleinat #34622: Extract asyncio exceptions into a separate file https://bugs.python.org/issue34622 closed by asvetlov #34625: update to expat 2.2.6 https://bugs.python.org/issue34625 closed by benjamin.peterson #34627: Python incorrect execution order https://bugs.python.org/issue34627 closed by badrussians #34633: Simplify __reduce__ for ordered dict iterators https://bugs.python.org/issue34633 closed by pablogsal #34635: inspect: add tools for inspecting subclasses https://bugs.python.org/issue34635 closed by bmintz #34636: re module microoptimization: speed up bytes \w \s \d matching https://bugs.python.org/issue34636 closed by gregory.p.smith #34637: Make *start* usable as a keyword argument for sum(). https://bugs.python.org/issue34637 closed by rhettinger #34638: Avoid circular references in asyncio streams https://bugs.python.org/issue34638 closed by asvetlov #34640: remove the configure check TANH_PRESERVES_ZERO_SIGN https://bugs.python.org/issue34640 closed by benjamin.peterson #34641: Curiosity: f((a)=1) is not a syntax error -- why? https://bugs.python.org/issue34641 closed by benjamin.peterson #34642: time.ctime() uses %3d instead of %.2d to format. https://bugs.python.org/issue34642 closed by martin.panter #34644: Bug in reverse method https://bugs.python.org/issue34644 closed by steven.daprano #34645: math and numpy yield different results (nan) https://bugs.python.org/issue34645 closed by steven.daprano #34646: remove PyAPI_* from function definitions https://bugs.python.org/issue34646 closed by benjamin.peterson #34647: print sys.thread_info in regrtest header https://bugs.python.org/issue34647 closed by benjamin.peterson #34649: Modules/_json.c: Missing NULL checks in _encoded_const() https://bugs.python.org/issue34649 closed by berker.peksag #34650: test_posix fails with musl https://bugs.python.org/issue34650 closed by benjamin.peterson #34652: never enable lchmod on Linux https://bugs.python.org/issue34652 closed by benjamin.peterson #34653: PyParser_SimpleParseStringFilename should be deleted https://bugs.python.org/issue34653 closed by eric.smith #34654: test_time needs to handle '+' at the beginning of large years https://bugs.python.org/issue34654 closed by benjamin.peterson #34657: pyconfig.h macro "timezone" name clashes with user source are https://bugs.python.org/issue34657 closed by zach.ware #34658: subprocess with preexec_fn when fork() fails could corrupt PyE https://bugs.python.org/issue34658 closed by gregory.p.smith #34660: Remove ableist terms and pejoratives from source code and docs https://bugs.python.org/issue34660 closed by willingc #34661: test_shutil fails with busybox unzip https://bugs.python.org/issue34661 closed by benjamin.peterson #34664: test.test_os.MakedirTests.test_mode is too strict https://bugs.python.org/issue34664 closed by benjamin.peterson #34666: Implement async write and async close in asyncio StreamWriter https://bugs.python.org/issue34666 closed by asvetlov #34668: test_resource fails if test has CAP_SYS_RESOURCE but isn't roo https://bugs.python.org/issue34668 closed by petr.viktorin #34671: Remove references to Benevolent Dictator https://bugs.python.org/issue34671 closed by Mariatta #34674: assume unistd.h exists https://bugs.python.org/issue34674 closed by benjamin.peterson #34675: Avoid terminology related to slavery https://bugs.python.org/issue34675 closed by Mariatta

1 0

bpo-34595: How to format a type name?
by Victor Stinner Sept. 14, 2018

Sept. 14, 2018

Hi, Last week, I opened an issue to propose to add a new %T formatter to PyUnicode_FromFormatV() and so indirectly to PyUnicode_FromFormat() and PyErr_Format(): https://bugs.python.org/issue34595 I merged my change, but then Serhiy Storchaka asked if we can add something to get the "fully qualified name" (FQN) of a type, ex "datetime.timedelta" (FQN) vs "timedelta" (what I call "short" name). I proposed a second pull request to add %t (short) in addition to %T (FQN). But then Petr Viktorin asked me to open a thread on python-dev to get a wider discussion. So here I am. The rationale for this change is to fix multiple issues: * C extensions use Py_TYPE(obj)->tp_name which returns a fully qualified name for C types, but the name (without the module) for Python name. Python modules use type(obj).__name__ which always return the short name. * currently, many C extensions truncate the type name: use "%.80s" instead of "%s" to format a type name * "%s" with Py_TYPE(obj)->tp_name is used more than 200 times in the C code, and I dislike this complex pattern. IMHO "%t" with obj would be simpler to read, write and maintain. * I want C extensions and Python modules to have the same behavior: respect the PEP 399. Petr considers that error messages are not part of the PEP 399, but the issue is wider than only error messages. The main issue is that at the C level, Py_TYPE(obj)->tp_name is "usually" the fully qualified name for types defined in C, but it's only the "short" name for types defined in Python. For example, if you get the C accelerator "_datetime", PyTYPE(obj)->tp_name of a datetime.timedelta object gives you "datetime.timedelta", but if you don't have the accelerator, tp_name is just "timedelta". Another example, this script displays "mytimedelta(0)" if you have the C accelerator, but "__main__.mytimedelta(0)" if you use the Python implementation: --- import sys #sys.modules['_datetime'] = None import datetime class mytimedelta(datetime.timedelta): pass print(repr(mytimedelta())) --- So I would like to fix this kind of issue. Type names are mainly used for two purposes: * format an error message * obj.__repr__() It's unclear to me if we should use the "short" or the "fully qualified" name. It should maybe be decided on a case by case basis. There is also a 3rd usage: to implement __reduce__, here backward compatibility matters. Note: The discussion evolved since my first implementation of %T which just used the not well defined Py_TYPE(obj)->tp_name. -- Petr asked me why not exposing functions to get these names. For example, with my second PR (not merged), there are 3 (private) functions: /* type.__name__ */ const char* _PyType_Name(PyTypeObject *type); /* type.__qualname__ */ PyObject* _PyType_QualName(PyTypeObject *type); * type.__module__ "." type.__qualname__ (but type.__qualname__ for builtin types) */ PyObject * _PyType_FullName(PyTypeObject *type); My concern here is that each caller has to handler error: PyErr_Format(PyExc_TypeError, "must be str, not %.100s", Py_TYPE(obj)->tp_name); would become: PyObject *type_name = _PyType_FullName(Py_TYPE(obj)); if (name == NULL) { /* do something with this error ... */ PyErr_Format(PyExc_TypeError, "must be str, not %U", type_name); Py_DECREF(name); When I report an error, I dislike having to handle *new* errors... I prefer that the error handling is done inside PyErr_Format() for me, to reduce the risk of additional bugs. -- Serhiy also asked if we could expose the same feature at the *Python* level: provide something to get the fully qualified name of a type. It's not just f"{type(obj).__module}.{type(obj).__name__}", but you have to skip the module for builtin types like "str" (not return "builtins.str"). Maybe we can have "name: {0:t}, FQN: {0:T}".format(type(obj)). "t" for name and "T" for fully qualfied name. We would only have to modify type.__format__(). I'm not sure if we need to add new formatters to str % args. Example of Python code: raise TypeError("must be str, not %s" % type(fmt).__name__) I'm not sure about Python changes. My first concern was just to avoid Py_TYPE(obj)->tp_name at the C level. But again, we should keep C and Python consistent. If the behavior of C extensions change, Python modules should be adapted as well, to get the same behavior. Note: I reverted my change which added the %T formatter from PyUnicode_FromFormatV() to clarify the status of this issue. Victor

11 18

Re: [Python-Dev] PEP 579 and PEP 580: refactoring C functions and methods
by Jeroen Demeyer Sept. 13, 2018

Sept. 13, 2018

On 2018-09-13 02:26, Petr Viktorin wrote: > The reference to PEP 573 is premature. It seems to me that PEP 580 helps with the use case of PEP 573. In fact, it implements part of what PEP 573 proposes. So I don't see the problem with the reference to PEP 573. Even if the implementation of PEP 573 changes, the problem statement will remain and that's what I'm referring to. > If you agree I can > summarize rationale for "parent", as much as it concerns 580. Sure. I still think that we should refer to PEP 573, but maybe we can summarize it also in PEP 580. > # Using tp_print > > The tp_print gimmick is my biggest worry. > AFAIK there's no guarantee that a function pointer and Py_ssize_t are > the same size. I'm not actually claiming anywhere that it is the same size. > # Descriptor behavior > > I'd say "SHOULD" rather than "MUST" here. The section describes how to > implement expected/reasonable behavior, but I see no need to limit that. There *is* actually an important reason to limit it: it allows code to make assumptions on what __get__ does. This enables optimizations which wouldn't be possible otherwise. If you cannot be sure what __get__ does, then you cannot optimize obj.method(x) to type(obj).method(obj, x) > "if func supports the C call protocol, then func.__set__ must not be > implemented." -- also, __delete__ should not be implemented, right?. Indeed. I write Python but I think C API, so for me these are both really tp_descr_set. > PyCCall_FASTCALL is not a macro, shouldn't it be named PyCCall_FastCall? What's the convention for that anyway? I assumed that capital letters meant a "really know what you are doing" function which could segfault if used badly. For me, whether something is a function or macro is just an implementation detail (which can change between Python versions) which should not affect the naming. > # C API functions > > The function PyCFunction_GetFlags is, for better or worse, part of the > stable ABI. We shouldn't just give up on it. I'm fine with documenting > that it shouldn't be used, but for functions defined using > PyCFunction_New etc. it should continue behaving as before. > One solution could be to preserve the "definition time" METH_* flags in > the 0xFFF bits of cc_flags and use the other bits for CCALL_*. I'm fine with that if you insist. However, it would be a silly solution to formally satisfy the "stable ABI" requirement without actually helping. I agree with your other points that I didn't reply to and will make some edits to PEP 580. Jeroen.

3 2

We cannot fix all issues: let's close XML security issues (not fix them)
by Victor Stinner Sept. 13, 2018

Sept. 13, 2018

Hi, The Python bug tracker is full of bugs, and sadly we don't have enough people to take care of all of them. There are 3 open bugs about security issues in XML and I simply propose to close it: https://bugs.python.org/issue17318 https://bugs.python.org/issue17239 https://bugs.python.org/issue24238 The XML documentation already starts with a red warning explaining the security limitations of the Python implementation and points to defusedxml and defusedexpat which are existing and working counter-measures: https://docs.python.org/dev/library/xml.html Note: Christian Heimes, author of these 2 packages, told me that these modules may not work on Python 3.7, he didn't have time to maintain them recently. Maybe someone might want to help him? I suggest to close the 3 Python bugs without doing anything. Are you ok with that? Keeping the issue open for 3 years doesn't help anyone, and there is already a security warning in all supported version (I checked 2.7 and 3.4). It seems like XML is getting less popular because of JSON becoming more popular (even if JSON obviously comes with its own set of security issues...). It seems like less core developers care about XML (today than 3 years ago). We should just accept that core developers have limited availability and that documenting security issues is an *acceptable* trade-off. I don't see any value of keeping these 3 issues open. Victor

13 21

Request for review
by Mark Shannon Sept. 13, 2018

Sept. 13, 2018

Hi, Can I request a review of https://github.com/python/cpython/pull/6641. It has been open for a few months now. Cheers, Mark.

1 0

PEP 579 and PEP 580: refactoring C functions and methods
by Jeroen Demeyer Sept. 13, 2018

Sept. 13, 2018

Hello, Let me present PEP 579 and PEP 580. PEP 579 is an informational meta-PEP, listing some of the issues with functions/methods implemented in C. The idea is to create several PEPs each fix some part of the issues mentioned in PEP 579. PEP 580 is a standards track PEP to introduce a new "C call" protocol, which is an important part of PEP 579. In the reference implementation (which is work in progress), this protocol will be used by built-in functions and methods. However, it should be used by more classes in the future. You find the texts at https://www.python.org/dev/peps/pep-0579 https://www.python.org/dev/peps/pep-0580 Jeroen.

6 7

Stop automerging
by Serhiy Storchaka Sept. 12, 2018

Sept. 12, 2018

12.09.18 01:34, Miss Islington (bot) пише: > https://github.com/python/cpython/commit/d13e59c1b512069d90efe7ee9b613d3913… > commit: d13e59c1b512069d90efe7ee9b613d3913e79c56 > branch: master > author: Benjamin Peterson <benjamin(a)python.org> > committer: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> > date: 2018-09-11T15:29:57-07:00 > summary: > > Make sure the line comes from the same node as the col offset. (GH-9189) > > Followup to 90fc8980bbcc5c7dcced3627fe172b0bfd193a3b. > >  This commit message looks awful (and it is duplicated in maintained branches). Please stop automerging to master by bots. The reason of automating merging before is that the core dev that performs merging is responsible for editing the commit message. There were mistakes from time to time, but usually regular commiters did care of this. I often use "git log", and such commit messages spoil the history.

5 5

AES cipher implementation in standard library
by 大野隆弘 Sept. 10, 2018

Sept. 10, 2018

Dear all, Have we tried cipher implementation includes AES as a standard library in the past? https://docs.python.org/3.6/library/crypto.html if possible I want to try to implement AES because famous 3rd party library is not maintained and general cipher programs should be used for multiple purpose.Though the implementation is tough, I believe this should be worth to it. In my case, I want to use AES implementation for zipfile module. Thanks and Regards, --------------- Takahiro Ono

3 5