docs January 2020

docs@python.org

27 participants
286 discussions

[issue39375] Document os.environ[x] = y and os.putenv() as thread unsafe

by Gregory P. Smith

New submission from Gregory P. Smith <greg(a)krypto.org>: The underlying API calls made by os.putenv() and os.environ[name] = value syntax are not thread safe on POSIX systems. POSIX _does not have_ any thread safe way to access the process global environment. In a pure Python program, the GIL prevents this from being an issue. But when embedded in a C/C++ program or using extension modules that launch their own threads from C, those threads could also make the invalid assumption that they can safely _read_ the environment. Which is a race condition when a Python thread is doing a putenv() at the same time. We should document the danger. CPython's os module snapshots a copy of the environment into a dict at import time (during CPython startup). But os.environ[] assignment and os.putenv() modify the actual process global environment in addition to updating this dict. (If an embedded interpreter is launched from a process with other threads already running, even that initial environment reading could be unsafe if the larger application has a thread that wrongly assumes it has exclusive environment access) For people modifying os.environ so that the change is visible to child processes, we can recommend using the env= parameter on subprocess API calls to supply a new environment. A broader issue of should we be modifying the process global environment state at all from os.putenv() and os.environ[] assignment still exists. I'll track that in another issue (to be opened). ---------- assignee: docs@python components: Documentation messages: 360221 nosy: docs@python, gregory.p.smith priority: normal severity: normal status: open title: Document os.environ[x] = y and os.putenv() as thread unsafe versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue39375> _______________________________________

3 weeks, 4 days

[issue35276] Document thread safety

by STINNER Victor

New submission from STINNER Victor <vstinner(a)redhat.com>: Many developers only discover that a Python function/module is not thread safe because they have a bug in production... Some examples: * bpo-7672: ssl * bpo-8865: select.poll is not thread safe * bpo-539175, bpo-21216: socket.gethostbyname() * bpo-7980: time.strptime() * bpo-6647: warnings.catch_warnings() * bpo-11077, bpo-33479: Tkinter * bpo-1336, bpo-19809: subprocess on Python 2 * bpo-15329: deque * bpo-35275: os.umask() Hopefully, sometimes it was possible to fix it: * bpo-3139: bytearray, buffer protocol * bpo-28969: @functools.lru_cache * bpo-21291: subprocess.Popen.wait() In the asyncio documentation, I explicitly documented that, by design, most classes are not thread-safe. For example, asyncio.Lock() is *NOT* thread-safe: https://docs.python.org/dev/library/asyncio-sync.html#asyncio.Lock Maybe we should start to use a standard way to describe "thread safety". See "POSIX Safety Concepts" of the GNU libc: https://www.gnu.org/software/libc/manual/html_node/POSIX-Safety-Concepts.ht… Example with setlocale, "MT-Unsafe": https://www.gnu.org/software/libc/manual/html_node/Setting-the-Locale.html -- My own (incomplete) list of "process-wide states": https://vstinner.readthedocs.io/threads.html#process-wide ---------- assignee: docs@python components: Documentation messages: 330093 nosy: docs@python, vstinner priority: normal severity: normal status: open title: Document thread safety versions: Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue35276> _______________________________________

3 weeks, 6 days

[issue13828] Further improve casefold documentation

by Jim Jewett

New submission from Jim Jewett <jimjjewett(a)gmail.com>: > http://hg.python.org/cpython/rev/0b5ce36a7a24 > changeset: 74515:0b5ce36a7a24 > + Casefolding is similar to lowercasing but more aggressive because it is > + intended to remove all case distinctions in a string. For example, the German > + lowercase letter ``'ß'`` is equivalent to ``"ss"``. Since it is already > + lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold` > + converts it to ``"ss"``. Perhaps add the recommendation to canonicalize as well. A complete, but possibly too long, try is below: Casefolding is similar to lowercasing but more aggressive because it is intended to remove all case distinctions in a string. For example, the German lowercase letter ``'ß'`` is equivalent to ``"ss"``. Since it is already lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold` converts it to ``"ss"``. Note that most case-insensitive matches should also match compatibility equivalent characters. The casefolding algorithm is described in section 3.13 of the Unicode Standard. Per D146, a compatibility caseless match can be achieved by from unicodedata import normalize def caseless_compat(string): nfd_string = normalize("NFD", string) nfkd1_string = normalize("NFKD", nfd_string.casefold()) return normalize("NFKD", nfkd1_string.casefold()) ---------- assignee: docs@python components: Documentation messages: 151644 nosy: Jim.Jewett, benjamin.peterson, docs@python priority: normal severity: normal status: open title: Further improve casefold documentation versions: Python 3.3 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue13828> _______________________________________

4 weeks

[issue23802] patch: __deepcopy__ memo dict argument usage

by Daniel Shahaf

New submission from Daniel Shahaf: In the 'copy' module documentation, it wasn't fully clear to me how __deepcopy__ implementations should treat the memo dict argument. The attached patch clarifies that __deepcopy__ implementations should treat the memo dict argument as an opaque type. ---------- assignee: docs@python components: Documentation files: opaque.diff keywords: patch messages: 239473 nosy: danielsh, docs@python priority: normal severity: normal status: open title: patch: __deepcopy__ memo dict argument usage type: enhancement Added file: http://bugs.python.org/file38723/opaque.diff _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue23802> _______________________________________

1 month

[issue30757] pyinstaller can be added to docs, py2exe ref can be updated

by Denis Akhiyarov

New submission from Denis Akhiyarov: https://github.com/python/cpython/pull/1158 It is not clear why this FAQ item is written in addition to this document: https://github.com/python/cpython/blob/master/Doc/faq/windows.rst#how-do-i-… https://github.com/python/cpython/blob/master/Doc/faq/programming.rst#how-c… ---------- assignee: docs@python components: Documentation messages: 296844 nosy: denfromufa, docs@python priority: normal severity: normal status: open title: pyinstaller can be added to docs, py2exe ref can be updated type: enhancement versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue30757> _______________________________________

1 month

[issue35786] get_lock() method is not present for Values created using multiprocessing.Manager()

by Lorenzo Persichetti

New submission from Lorenzo Persichetti <lpersichetti(a)gmail.com>: According to the documentation of the multiprocessing.Value() class available here https://docs.python.org/3.6/library/multiprocessing.html#multiprocessing.Va… Operations like += which involve a read and write are not atomic. So if, for instance, you want to atomically increment a shared value it is insufficient to just do counter.value += 1 Assuming the associated lock is recursive (which it is by default) you can instead do with counter.get_lock(): counter.value += 1 What happens is that when running the following snippet import multiprocessing manager = multiprocessing.Manager() value = manager.Value('i', 0) value.get_lock() the result is AttributeError: 'ValueProxy' object has no attribute 'get_lock' ---------- assignee: docs@python components: Documentation, Library (Lib), Windows messages: 334070 nosy: Lorenzo Persichetti, docs@python, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: get_lock() method is not present for Values created using multiprocessing.Manager() versions: Python 3.6 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue35786> _______________________________________

1 month, 1 week

[issue36700] base64 has old references that should be updated

by Paul Hoffman

New submission from Paul Hoffman <phoffman(a)proper.com>: The documentation for base64 library has an RFC that is obsolete. ---------- assignee: docs@python components: Documentation messages: 340668 nosy: docs@python, paulehoffman priority: normal pull_requests: 12839 severity: normal status: open title: base64 has old references that should be updated _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue36700> _______________________________________

1 month, 1 week

[issue36661] Missing import in docs

by Merlin Fisher-Levine

New submission from Merlin Fisher-Levine <merlin.fisherlevine(a)gmail.com>: Dataclasses docs don't mention needing import for @dataclass decorator https://docs.python.org/3/library/dataclasses.html ---------- assignee: docs@python components: Documentation messages: 340510 nosy: docs@python, mfisherlevine priority: normal severity: normal status: open title: Missing import in docs type: enhancement versions: Python 3.7 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue36661> _______________________________________

1 month, 3 weeks

[issue37082] Assignment expression operator (walrus) not in built-in help()

by Matthias Bussonnier

New submission from Matthias Bussonnier <bussonniermatthias(a)gmail.com>: Do the following at a Python prompt: ``` >>> help() Welcome to Python 3.8's help utility! [...]SNIP help> symbols Here is a list of the punctuation symbols which Python assigns special meaning to. Enter any symbol to get more help. != + <= __ " += <> ` """ , == b" % - > b' %= -= >= f" & . >> f' &= ... >>= j ' / @ r" ''' // J r' ( //= [ u" ) /= \ u' * : ] | ** < ^ |= **= << ^= ~ *= <<= _ ``` Oh no ! Our favorite `:=` is missing :-( ---------- assignee: docs@python components: Documentation messages: 343813 nosy: docs@python, mbussonn priority: normal severity: normal status: open title: Assignment expression operator (walrus) not in built-in help() versions: Python 3.8, Python 3.9 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue37082> _______________________________________

1 month, 4 weeks

[issue39417] Link to "Python Packaging User Guide: Creating and using virtual environments" is broken

by Angel Cervera Claudio

New submission from Angel Cervera Claudio <angelcervera(a)silyan.com>: The link "See also: Python Packaging User Guide: Creating and using virtual environments" Is broken. The problem is in line 30 of https://github.com/python/cpython/blob/3.8/Doc/library/venv.rst I don't know the right link, so I can not fix it. I any one provive me the right link, I can create a PR. ---------- assignee: docs@python components: Documentation messages: 360454 nosy: angelcervera, docs@python priority: normal severity: normal status: open title: Link to "Python Packaging User Guide: Creating and using virtual environments" is broken versions: Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue39417> _______________________________________

2 months, 2 weeks

Jump to page:

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

docs January 2020