Mailman 3 January 2020 - Python-Dev

Steering Council Update (November through December)
by Ewa Jodlowska Jan. 6, 2020

Jan. 6, 2020

Hello! Below is an update from the Steering Council. It covers November and December. The update has been added to the Steering Council repo: https://github.com/python/steering-council/blob/master/updates/2020-01-06-s… (added to README as well). ------------------------------------------------------------------------ # Steering Council Community Update for November - December 2019 ## November recap: - Steering Council assigned one another tasks to check-in on and clean up 29 open PEPs in November (Nov 5 and Nov 12 meetings). See [PEP repo]( https://github.com/python/peps/) for details. - Steering Council decided that going forward, only one person will need to approve the minutes and the group will try to rotate that responsibility. Additionally, to keep meetings efficient we will: - 1. have a person responsible for the agenda to make sure the agenda is accurate and timed appropriately - 2. have a person run the actual meeting to make sure we stay on topic and cut discussions when topics go over time. - Steering Council discussed ideas on how to lower the count of PRs. It was decided that Brett would start a discussion on Discourse with the idea to close enhancement PRs. This has been done [here]( https://discuss.python.org/t/automatically-close-all-enhancement-prs-for-co…). At a later time, it was decided the idea would be shut down since many folks did not approve. - Steering Council discussed how a hiring plan can coincide with the Vision Deck and how it would work with sponsor/corporate collaborations. The Vision Deck is an overview document the Steering Council is drafting, which will help develop Python's roadmap for the next 5 years. Goal is to have the Vision Deck complete by PyCon 2020. - Steering Council had a brief discussion about encouraging more candidates to run for the next steering council and that everyone would work on raising awareness for this. ## December recap: - The Steering Council reviewed [PEP 584]( https://www.python.org/dev/peps/pep-0584/) (“Add + and += operators to the built-in dict class”) and decided that Guido would let the PEP authors know | and |= was preferred and that the PEP needed some editing. [This was done]( https://mail.python.org/archives/list/python-ideas@python.org/message/P46F7…). At a later time it was decided that Guido will be BDFL-Delegate. - The Steering Council decided it would meet after the 2020 Steering Council vote ended. The Steering Council will meet December 10 and December 17th. The hand-off meeting will happen the first week of January. Ewa will send out a Doodle for the first full week of January once the new Steering Council members are known. It was decided that on January 8th, 2020 the Steering Council will have a hand-off meeting between the 2019 team and 2020 team. - Steering Council decided that we would send out ballots to Marc-Andre Lemburg, Alex Martelli, and Kurt B. Kaiser. [Brett responded to thread]( https://mail.python.org/archives/list/python-committers@python.org/message/…) noting this decision and future plans for [PEP 13]( https://www.python.org/dev/peps/pep-0013/). - Steering Council discussed [PEP 611]( https://www.python.org/dev/peps/pep-0611/). Barry [emailed the python-dev@ list]( https://mail.python.org/archives/list/python-dev@python.org/message/KY46EXG…) with the outcome of the discussion. - Steering Council discussed PyPI’s typosquatting issues. - Group discussed the status of the GitHub migration plan. Ewa drafted a job description for the Project Manager role to help with the GitHub migration for the Dec 17th SC meeting. The Steering Council reviewed it and Ewa scheduled a call with the GitHub team for late January for next steps. - Steering Council decided to keep Zulip since the Packaging-wg plans on using it. We will re-evaluate usage next year. Guido informed the [Discourse thread]( https://discuss.python.org/t/should-we-continue-using-zulip/2816/7). Brett [posted an announcement]( https://python.zulipchat.com/#narrow/stream/116742-core.2Fhelp/topic/Engage…) in Zulip. Thank you, Ewa ----------------------- *As a non-profit organization, the PSF depends on sponsorships and donations to support the Python community. Check out our Annual Impact Report for more details: https://www.python.org/psf/annual-report/2019/ <https://www.python.org/psf/annual-report/2019/>* *Please contribute to PSF; we can't continue our work without your support! https://www.python.org/psf/donations/ <https://www.python.org/psf/donations/>*

1 0

Adding a scarier warning to object.__del__?
by Yonatan Zunger Jan. 5, 2020

Jan. 5, 2020

Hey everyone, I just encountered yet another reason to beware of __del__: when it's called during interpreter shutdown, for reasons which are kind of obvious in retrospect, if it calls notify() on a threading.Condition, the waiting thread may or may not ever actually receive it, and so if it does that and then tries to join() the thread the interpreter may hang in a hard-to-debug way. This isn't something that can reasonably be fixed, and (like in most cases involving __del__) there's a very simple fix of using weakref.finalize instead. My question for the dev list: How would people feel about changing the documentation for the method <https://docs.python.org/3/reference/datamodel.html#object.__del__> to more bluntly warn people against using it, and refer them to weakref.finalize and/or atexit.register as an alternative? The text already has an undertone of "lasciate ogni speranza, voi ch'entrate" but it may be helpful to be more explicit to avoid people getting caught in unexpected pitfalls. Yonatan

10 16

Summary of Python tracker Issues
by Python tracker Jan. 3, 2020

Jan. 3, 2020

ACTIVITY SUMMARY (2019-12-27 - 2020-01-03) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7227 (+24) closed 43726 (+38) total 50953 (+62) Open issues with patches: 2842 Issues opened (48) ================== #27973: urllib.urlretrieve() fails on second ftp transfer https://bugs.python.org/issue27973 reopened by pablogsal #39114: Python 3.9.0a2 changed how finally/return is traced https://bugs.python.org/issue39114 reopened by nedbat #39141: IDLE Clear function returns 256 on Mac OS Catalina https://bugs.python.org/issue39141 opened by twister68(a)gmail.com #39143: Implementing sub-generation steps in the gc https://bugs.python.org/issue39143 opened by pablogsal #39146: too much memory consumption in re.compile unicode https://bugs.python.org/issue39146 opened by Zhipeng Xie #39147: using zipfile with root privilege shows FileNotFoundError https://bugs.python.org/issue39147 opened by Patrick Liu #39148: DatagramProtocol + IPv6 does not work with ProactorEventLoop https://bugs.python.org/issue39148 opened by alex.gronholm #39150: See if PyToken_OneChar would be faster as a lookup table https://bugs.python.org/issue39150 opened by petdance #39151: Simplify the deep-first-search of the assembler https://bugs.python.org/issue39151 opened by pablogsal #39152: Faulty override of tkinter.Misc.configure in tkinter.ttk.Scale https://bugs.python.org/issue39152 opened by glombardo #39153: Clarify refcounting semantics of PyDict_SetItem[String] https://bugs.python.org/issue39153 opened by ncoghlan #39154: "utf8" not always a synonym for "utf-8" in lib2to3 https://bugs.python.org/issue39154 opened by Peter Ludemann #39155: "utf8-sig" missing from codecs (inconsistency) https://bugs.python.org/issue39155 opened by Peter Ludemann #39156: Break up COMPARE_OP into logically distinct operations. https://bugs.python.org/issue39156 opened by Mark.Shannon #39157: test_pidfd_send_signal can fail on some systems with Permissio https://bugs.python.org/issue39157 opened by pablogsal #39159: Ideas for making ast.literal_eval() usable https://bugs.python.org/issue39159 opened by rhettinger #39160: ./configure --help has inconsistencies in style https://bugs.python.org/issue39160 opened by anthony shaw #39161: Py_NewInterpreter docs need updating for multi-phase initializ https://bugs.python.org/issue39161 opened by ncoghlan #39162: setup.py not picking up tkinter headers https://bugs.python.org/issue39162 opened by anthonypjshaw #39164: PyErr_GetExcInfo does not allow to retrieve for an arbitrary t https://bugs.python.org/issue39164 opened by jd #39165: Completeness and symmetry in RE, avoid `findall(...)[0]` https://bugs.python.org/issue39165 opened by apalala #39166: Python 3.9.0a2 changed how "async for" traces its final iterat https://bugs.python.org/issue39166 opened by nedbat #39168: Generic type subscription is a huge toll on Python performance https://bugs.python.org/issue39168 opened by Ruslan Dautkhanov #39169: TypeError: 'int' object is not callable if the signal handler https://bugs.python.org/issue39169 opened by Ronald Li #39170: Sqlite3 row_factory for attribute access: NamedRow https://bugs.python.org/issue39170 opened by jidn #39171: Missing default root in tkinter simpledialog.py https://bugs.python.org/issue39171 opened by dominic108 #39173: _AttributeHolder of argparse should support the sort function https://bugs.python.org/issue39173 opened by shihai1991 #39174: unicodedata.normalize failing with NFD and NFKD for some chara https://bugs.python.org/issue39174 opened by Lee Collins #39177: In tkinter, simple dialogs, askstrings, etc. with flexible coo https://bugs.python.org/issue39177 opened by dominic108 #39179: pandas tz_convert() seems to report incorrect date conversion https://bugs.python.org/issue39179 opened by Seeking.that #39180: Missing getlines func documentation from linecache module https://bugs.python.org/issue39180 opened by khalidmammadov #39181: Add `os.makedirs()` as `Path.mkdir()` equivalent in correspond https://bugs.python.org/issue39181 opened by jack1142 #39182: sys.addaudithook(hook) loops indefinitely on mismatch for hook https://bugs.python.org/issue39182 opened by Dutcho #39184: Many command execution functions are not raising auditing even https://bugs.python.org/issue39184 opened by Saiyang Gou #39185: Add quiet and detailed verbosity levels to build.bat https://bugs.python.org/issue39185 opened by anthonypjshaw #39187: urllib.robotparser does not respect the longest match for the https://bugs.python.org/issue39187 opened by gallicrooster #39188: recent change when passing a Union to a function https://bugs.python.org/issue39188 opened by Kevin Schlossser #39189: Use io.DEFAULT_BUFFER_SIZE for filecmp BUFSIZE variable https://bugs.python.org/issue39189 opened by Bahram Aghaei #39190: _result_handler dies on raised exceptions [multiprocessing] https://bugs.python.org/issue39190 opened by sindrig #39191: Coroutine is awaited despite an exception in run_until_complet https://bugs.python.org/issue39191 opened by dkaveshnikov #39194: asyncio.open_connection returns a closed client when server fa https://bugs.python.org/issue39194 opened by Jonathan Martin #39196: json fails to encode dictionary view types https://bugs.python.org/issue39196 opened by jogo #39197: Support the title and description arguments for mutually exclu https://bugs.python.org/issue39197 opened by signing_agreement #39198: Lock may not be released in Logger.isEnabledFor https://bugs.python.org/issue39198 opened by derektbrown #39199: Improve the AST documentation https://bugs.python.org/issue39199 opened by pablogsal #39200: Fix inaccurate TypeError messages when calling with insufficie https://bugs.python.org/issue39200 opened by jferard #39201: Threading.timer leaks memory in 3.8.0/3.8.1 https://bugs.python.org/issue39201 opened by mneerup #39202: Python shelve __del__ ignored exception https://bugs.python.org/issue39202 opened by libbkmz Most recent 15 issues with no replies (15) ========================================== #39202: Python shelve __del__ ignored exception https://bugs.python.org/issue39202 #39199: Improve the AST documentation https://bugs.python.org/issue39199 #39197: Support the title and description arguments for mutually exclu https://bugs.python.org/issue39197 #39194: asyncio.open_connection returns a closed client when server fa https://bugs.python.org/issue39194 #39190: _result_handler dies on raised exceptions [multiprocessing] https://bugs.python.org/issue39190 #39189: Use io.DEFAULT_BUFFER_SIZE for filecmp BUFSIZE variable https://bugs.python.org/issue39189 #39188: recent change when passing a Union to a function https://bugs.python.org/issue39188 #39185: Add quiet and detailed verbosity levels to build.bat https://bugs.python.org/issue39185 #39184: Many command execution functions are not raising auditing even https://bugs.python.org/issue39184 #39182: sys.addaudithook(hook) loops indefinitely on mismatch for hook https://bugs.python.org/issue39182 #39181: Add `os.makedirs()` as `Path.mkdir()` equivalent in correspond https://bugs.python.org/issue39181 #39180: Missing getlines func documentation from linecache module https://bugs.python.org/issue39180 #39177: In tkinter, simple dialogs, askstrings, etc. with flexible coo https://bugs.python.org/issue39177 #39170: Sqlite3 row_factory for attribute access: NamedRow https://bugs.python.org/issue39170 #39169: TypeError: 'int' object is not callable if the signal handler https://bugs.python.org/issue39169 Most recent 15 issues waiting for review (15) ============================================= #39200: Fix inaccurate TypeError messages when calling with insufficie https://bugs.python.org/issue39200 #39199: Improve the AST documentation https://bugs.python.org/issue39199 #39198: Lock may not be released in Logger.isEnabledFor https://bugs.python.org/issue39198 #39191: Coroutine is awaited despite an exception in run_until_complet https://bugs.python.org/issue39191 #39190: _result_handler dies on raised exceptions [multiprocessing] https://bugs.python.org/issue39190 #39187: urllib.robotparser does not respect the longest match for the https://bugs.python.org/issue39187 #39185: Add quiet and detailed verbosity levels to build.bat https://bugs.python.org/issue39185 #39180: Missing getlines func documentation from linecache module https://bugs.python.org/issue39180 #39171: Missing default root in tkinter simpledialog.py https://bugs.python.org/issue39171 #39170: Sqlite3 row_factory for attribute access: NamedRow https://bugs.python.org/issue39170 #39166: Python 3.9.0a2 changed how "async for" traces its final iterat https://bugs.python.org/issue39166 #39164: PyErr_GetExcInfo does not allow to retrieve for an arbitrary t https://bugs.python.org/issue39164 #39162: setup.py not picking up tkinter headers https://bugs.python.org/issue39162 #39160: ./configure --help has inconsistencies in style https://bugs.python.org/issue39160 #39157: test_pidfd_send_signal can fail on some systems with Permissio https://bugs.python.org/issue39157 Top 10 most discussed issues (10) ================================= #39143: Implementing sub-generation steps in the gc https://bugs.python.org/issue39143 15 msgs #39168: Generic type subscription is a huge toll on Python performance https://bugs.python.org/issue39168 12 msgs #36051: Drop the GIL during large bytes.join operations? https://bugs.python.org/issue36051 10 msgs #32615: Inconsistent behavior if globals is a dict subclass https://bugs.python.org/issue32615 9 msgs #27973: urllib.urlretrieve() fails on second ftp transfer https://bugs.python.org/issue27973 8 msgs #39114: Python 3.9.0a2 changed how finally/return is traced https://bugs.python.org/issue39114 6 msgs #39134: can't construct dataclass as ABC (or runtime check as data pro https://bugs.python.org/issue39134 6 msgs #39166: Python 3.9.0a2 changed how "async for" traces its final iterat https://bugs.python.org/issue39166 6 msgs #34118: Fix some class entries in 'Built-in Functions' https://bugs.python.org/issue34118 5 msgs #39156: Break up COMPARE_OP into logically distinct operations. https://bugs.python.org/issue39156 5 msgs Issues closed (38) ================== #5851: Add a stream parameter to gc.set_debug https://bugs.python.org/issue5851 closed by pablogsal #13601: sys.stderr should be line-buffered when stderr is not a TTY https://bugs.python.org/issue13601 closed by pitrou #15273: Remove unnecessarily random behavior from test_unparse.py https://bugs.python.org/issue15273 closed by pablogsal #15723: Python breaks OS' append guarantee on file writes https://bugs.python.org/issue15723 closed by gvanrossum #18858: dummy_threading lacks threading.get_ident() equivalent https://bugs.python.org/issue18858 closed by inada.naoki #20443: __code__. co_filename should always be an absolute path https://bugs.python.org/issue20443 closed by ncoghlan #24174: Python crash on exit https://bugs.python.org/issue24174 closed by SilentGhost #25087: Type variable substitution in type instances https://bugs.python.org/issue25087 closed by gvanrossum #30427: isinstance checks in os.path.normcase redundant with os.fspath https://bugs.python.org/issue30427 closed by brett.cannon #32438: PyLong_ API cleanup https://bugs.python.org/issue32438 closed by mark.dickinson #33387: Simplify bytecodes for try-finally, try-except and with blocks https://bugs.python.org/issue33387 closed by Mark.Shannon #33777: dummy_threading: .is_alive method returns True after execution https://bugs.python.org/issue33777 closed by inada.naoki #35143: `from __future__ import annotations` has no effect inside `ast https://bugs.python.org/issue35143 closed by eric.smith #36220: LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses https://bugs.python.org/issue36220 closed by pablogsal #37446: Undefined behavior in Python/hamt.c https://bugs.python.org/issue37446 closed by benjamin.peterson #38532: Missing decrefs in the _ctypes module https://bugs.python.org/issue38532 closed by twouters #38588: Use-after-free in dict/list https://bugs.python.org/issue38588 closed by pablogsal #38610: use-after-free in list object function https://bugs.python.org/issue38610 closed by pablogsal #38879: Reordered error checking in PyArena_New(). https://bugs.python.org/issue38879 closed by pitrou #39037: Fix the trial order of the __exit__ and __enter__ methods in t https://bugs.python.org/issue39037 closed by maggyero #39112: Misleading documentation for tuple https://bugs.python.org/issue39112 closed by terry.reedy #39136: Typos in whatsnew file and docs https://bugs.python.org/issue39136 closed by terry.reedy #39142: logging.config.dictConfig will convert namedtuple to Convertin https://bugs.python.org/issue39142 closed by vinay.sajip #39144: Align ctags and etags targets and include Python stdlib https://bugs.python.org/issue39144 closed by pablogsal #39145: Innocuous parent class changes multiple inheritance MRO https://bugs.python.org/issue39145 closed by steven.daprano #39149: False positive using operator 'AND' while checking keys on dic https://bugs.python.org/issue39149 closed by serhiy.storchaka #39158: ast.literal_eval() doesn't support empty sets https://bugs.python.org/issue39158 closed by rhettinger #39163: spam https://bugs.python.org/issue39163 closed by xtreak #39167: argparse boolean type bug https://bugs.python.org/issue39167 closed by josh.r #39172: Transformation of "string.find('asd', 'sd')" to Python 3 using https://bugs.python.org/issue39172 closed by serhiy.storchaka #39175: Funkness with issubset https://bugs.python.org/issue39175 closed by rhettinger #39176: Syntax error message uses strange term: "named assignment" https://bugs.python.org/issue39176 closed by rhettinger #39178: Should we make dict not accept a sequence of sets? https://bugs.python.org/issue39178 closed by steven.daprano #39183: Text divided in two strings due to wrong formatting https://bugs.python.org/issue39183 closed by rffontenelle #39186: Windows installer instructions refer to mercurial https://bugs.python.org/issue39186 closed by zach.ware #39192: relationlist module https://bugs.python.org/issue39192 closed by mark.dickinson #39193: Out-of-bound write in ceval.c:_PyEval_EvalFrameDefault https://bugs.python.org/issue39193 closed by pablogsal #39195: re._compile should check if the argument is a compiled pattern https://bugs.python.org/issue39195 closed by Recursing

1 0

PEP 558: Defined semantics for locals() (December 2019 edition)
by Nick Coghlan Jan. 2, 2020

Jan. 2, 2020

Hi folks, I've finally updated PEP 558 and it's reference implementation based on Nathaniel's feedback back in May. The latest version of the PEP can now be found at https://www.python.org/dev/peps/pep-0558/, and I've created a discussion thread on Discourse: https://discuss.python.org/t/pep-558-defined-semantics-for-locals/2936 The latest version implements Nathaniel's "independent snapshot" proposal, and I like how that has turned out. The one thing that changed from the May discussion thread is that the refcount semantics of PyEval_GetLocals() (it returns a borrowed reference) meant that it had to keep the old behaviour of returning a reference to the internal dynamically updated shared "snapshot" at function scope, with a new API, PyEval_GetPyLocals(), providing the C equivalent of the locals() builtin. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 3

Use of AT_SECURE on Linux
by Max Mazurov Jan. 1, 2020

Jan. 1, 2020

Hello, list. I noticed that CPython does not sanitize environment when transition between SELinux contexts or AppArmor profiles happens, which I think is a good thing to do in cases when one sandboxed application runs a Python script with different (potentially less strict) sandbox configuration. On Linux, such transition is indicated by LSMs (SELinux, AppArmor) using the AT_SECURE bit in the aux. vector. Consult getauxval(3) in the glibc manual for details. Here is the paragraph describing AT_SECURE: > Has a nonzero value if this executable should be treated securely. Most > commonly, a nonzero value indicates that the process is executing a > set-user-ID or set-group-ID binary (so that its real and effective UIDs or > GIDs differ from one another), or that it gained capabilities by executing a > binary file that has capabilities (see capabilities(7)). Alternatively, a > nonzero value may be triggered by a Linux Security Module. When this value is > nonzero, the dynamic linker disables the use of certain environment variables > (see ld-linux.so(8)) and glibc changes other aspects of its behavior. (See > also secure_getenv(3).) set-user-ID and capabilities are not directly relevant to Python since it is not possible to apply setuid bit or file capabilities to the script (with #!). Though, it might be relevant for other applications embedding CPython. The case with LSMs is directly relevant for standalone scripts since both SELinux and AppArmor allow applying per-script restrictions. There is the -I flag for the CPython interpreter that in my opinion does exactly what should be done when AT_SECURE bit is set. Since isolated flag is already checked by all relevant code paths in CPython and standard library, I believe it is a good idea to simply make AT_SECURE imply -I. Though I am not sure whether the behavior for other applications embedding CPython should be same. I can put together a patch to implement that if there is an interest and somebody can point me to an appropriate place to change. -- Cheers, Max Mazurov https://foxcpp.dev

1 0