Python-Dev May 2019

python-dev@python.org

105 participants
73 discussions

Plan to accept PEP 544, PEP 586, PEP 589, PEP 591 -- last call for discussion
by Guido van Rossum 20 May '19

20 May '19

Please send all discussions about these PEPs to the typing-sig mailing list, so we have the archives all in one place. Read the full message here: https://mail.python.org/archives/list/typing-sig@python.org/thread/GCI2TF43… -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

2 1

Feature request: Change a Dependency Package Version During Package Initiation
by Q 20 May '19

20 May '19

A lot of the Python code we use in production are used directly as imports in other python distributions (such as the python comes with the finite element software Abaqus and MSC Marc), many packages (such as matplotlib, numpy) that may have varying versioned dependencies. I was wondering if this could be expanded to allow a version to be set within a package and have that propagate to all modules in that package. For example in the root init.py if I set multiversion(tornado, 2.2.1) then all modules in that package will use tornado 2.2.1 when I import tornado. See a relevant issue on github: https://github.com/mitsuhiko/multiversion/issues/1 Thank you! Qiang

5 7

PEP 581 (Using GitHub issues for CPython) is accepted
by Barry Warsaw 18 May '19

18 May '19

As the BDFL-Delegate for PEP 581, and with the unanimous backing of the rest of the Steering Council, I hereby Accept this PEP. We would like to thank Mariatta for championing PEP 581, and to all the contributors to the discussion, both pro and con. We appreciate your candor and respect for your fellow developers. The SC believes that this migration is in the best interest of the Python community, and we look forward to the elaboration of the detailed migration plan (PEP 588). We also extend our heartfelt thanks Berker, Ezio, and everyone who has helped develop and maintain Roundup and bugs.python.org over the years. bpo has been a critical component for Python development for a very long time, and we all greatly appreciate the time, effort, and devotion you have put into this resource. https://www.python.org/dev/peps/pep-0581/ Onward we go! The migration will be a large effort, with much planning, development, and testing, and we welcome volunteers who wish to help make it a reality. I look forward to your contributions on PEP 588 and the actual work of migrating issues to GitHub. Cheers, -Barry (BDFL-Delegate, and on behalf of the Python Steering Council)

14 23

Merge ?
by Srinivas Reddy Thatiparthy 18 May '19

18 May '19

Pls take a look here https://github.com/python/cpython/pull/9882 If it not okay to document that >>> mock.assert_has_calls([]) does not raise an exception; then i will go ahead and close the pr. Best Regards, Srinivas Reddy Thatiparthy.

1 0

Summary of Python tracker Issues
by Python tracker 17 May '19

17 May '19

ACTIVITY SUMMARY (2019-05-10 - 2019-05-17) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7067 (-17) closed 41633 (+91) total 48700 (+74) Open issues with patches: 2861 Issues opened (56) ================== #36084: Threading: add builtin TID attribute to Thread objects https://bugs.python.org/issue36084 reopened by vstinner #36801: Wait for connection_lost in StreamWriter.drain https://bugs.python.org/issue36801 reopened by asvetlov #36802: Revert back StreamWriter awrite/aclose but provide await write https://bugs.python.org/issue36802 reopened by vstinner #36817: Add = to f-strings for easier debugging. https://bugs.python.org/issue36817 reopened by pablogsal #36876: Global C variables are a problem. https://bugs.python.org/issue36876 opened by eric.snow #36877: [meta] Move fields from _PyRuntimeState to PyInterpreterState. https://bugs.python.org/issue36877 opened by eric.snow #36878: ast.parse with type_comments=True should allow extra text afte https://bugs.python.org/issue36878 opened by msullivan #36879: bug with round() and "numpy floats" https://bugs.python.org/issue36879 opened by Scaler #36880: Returning None from a callback with restype py_object decremen https://bugs.python.org/issue36880 opened by dgelessus #36881: isinstance raises TypeError for metaclass with metaclass=ABCMe https://bugs.python.org/issue36881 opened by emanspeaks #36882: 在ctypes里调用C库返回c_char_p类型时的问题 https://bugs.python.org/issue36882 opened by fy2099 #36885: Make makeunicode.py script more readable https://bugs.python.org/issue36885 opened by scoder #36887: Add integer square root, math.isqrt https://bugs.python.org/issue36887 opened by mark.dickinson #36888: Create a way to check that the parent process is alive for dea https://bugs.python.org/issue36888 opened by tomMoral #36889: Merge StreamWriter and StreamReader into just asyncio.Stream https://bugs.python.org/issue36889 opened by asvetlov #36890: python-3.7.3-macosx10.6.pkg verification error on macOS 10.6 S https://bugs.python.org/issue36890 opened by steve_314 #36891: Additional startup plugin for vendors https://bugs.python.org/issue36891 opened by vx1920 #36892: "Modules" section in Tutorial contains incorrect description a https://bugs.python.org/issue36892 opened by musou1500 #36893: email.headerregistry.Address blocks Unicode local part addr_sp https://bugs.python.org/issue36893 opened by dracos #36894: test_multiprocessing_spawn regression on Windows https://bugs.python.org/issue36894 opened by pitrou #36895: time.clock() marked for removal in 3.8 – still there. https://bugs.python.org/issue36895 opened by mbussonn #36896: clarify in types.rst that FunctionTypes & co constructors don https://bugs.python.org/issue36896 opened by mbussonn #36897: shlex doesn't differentiate escaped characters in output https://bugs.python.org/issue36897 opened by Matthew Gamble #36898: Add parameter @case_sensitive to glob and rglob in pathlib https://bugs.python.org/issue36898 opened by Chuang Men #36901: Fix leaks in /PC/bdist_wininst/install.c https://bugs.python.org/issue36901 opened by cheryl.sabella #36904: Implement _PyStack_UnpackDict() with a single allocation https://bugs.python.org/issue36904 opened by jdemeyer #36905: test_typing.GetTypeHintTests.test_get_type_hints_modules_forwa https://bugs.python.org/issue36905 opened by xtreak #36906: Compile time textwrap.dedent() equivalent for str or bytes lit https://bugs.python.org/issue36906 opened by gregory.p.smith #36907: Crash due to borrowed references in _PyStack_UnpackDict() https://bugs.python.org/issue36907 opened by jdemeyer #36908: "This module is always available" is confusing https://bugs.python.org/issue36908 opened by nedbat #36910: Certain Malformed email causes email.parser to throw Attribute https://bugs.python.org/issue36910 opened by msapiro #36913: Missing documentation for decorators https://bugs.python.org/issue36913 opened by tomerv #36917: ast.NodeVisitor no longer calls visit_Str https://bugs.python.org/issue36917 opened by Anthony Sottile #36918: ValueError warning in test_urllib due to io.IOBase destructor https://bugs.python.org/issue36918 opened by xtreak #36919: Exception from 'compile' reports a newline char not present in https://bugs.python.org/issue36919 opened by BCSharp #36921: Deprecate yield from and @coroutine in asyncio https://bugs.python.org/issue36921 opened by asvetlov #36922: Implement Py_TPFLAGS_METHOD_DESCRIPTOR https://bugs.python.org/issue36922 opened by jdemeyer #36924: Simplify implementation of classmethod_descriptor.__call__ https://bugs.python.org/issue36924 opened by jdemeyer #36926: Implement methoddescr_call without _PyMethodDef_RawFastCallDic https://bugs.python.org/issue36926 opened by jdemeyer #36927: traceback docstrings should explicitly state return values ins https://bugs.python.org/issue36927 opened by Aaron Hall #36929: Other Python _io implementations may not expose _io in their t https://bugs.python.org/issue36929 opened by tekknolagi #36932: asyncio-task.rst could use proper deprecated-removed directive https://bugs.python.org/issue36932 opened by mbussonn #36933: sys.set_coroutine_wrapper documented as to be removed in 3.8 ( https://bugs.python.org/issue36933 opened by mbussonn #36935: bpo-35813 introduced usage of the deprecated PyErr_SetFromWind https://bugs.python.org/issue36935 opened by ZackerySpytz #36936: CALL_FUNCTION_KW opcode: keyword names must be non-empty https://bugs.python.org/issue36936 opened by jdemeyer #36937: New _PyObject_MakeTpCall() function https://bugs.python.org/issue36937 opened by jdemeyer #36940: Update Py_FrozenMain() for _PyCoreConfig (PEP 587) https://bugs.python.org/issue36940 opened by vstinner #36941: Windows build changes for Windows ARM64 https://bugs.python.org/issue36941 opened by Paul Monson #36942: Windows code changes for Windows ARM64 https://bugs.python.org/issue36942 opened by Paul Monson #36943: Windows test changes for Windows ARM64 https://bugs.python.org/issue36943 opened by Paul Monson #36944: Add support for ARM64 to libffi https://bugs.python.org/issue36944 opened by Paul Monson #36945: Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unc https://bugs.python.org/issue36945 opened by vstinner #36946: Possible signed integer overflow in slice handling https://bugs.python.org/issue36946 opened by ZackerySpytz #36947: [Good first issue] Fix 3.3.3.1 Metaclasses Documentation https://bugs.python.org/issue36947 opened by Nicholas Matthews #36948: NameError in urllib.request.URLopener.retrieve https://bugs.python.org/issue36948 opened by xtreak #36949: WeakSet.__repr__ and __str__ do not show contents of the set https://bugs.python.org/issue36949 opened by steve.dower Most recent 15 issues with no replies (15) ========================================== #36949: WeakSet.__repr__ and __str__ do not show contents of the set https://bugs.python.org/issue36949 #36948: NameError in urllib.request.URLopener.retrieve https://bugs.python.org/issue36948 #36944: Add support for ARM64 to libffi https://bugs.python.org/issue36944 #36943: Windows test changes for Windows ARM64 https://bugs.python.org/issue36943 #36940: Update Py_FrozenMain() for _PyCoreConfig (PEP 587) https://bugs.python.org/issue36940 #36933: sys.set_coroutine_wrapper documented as to be removed in 3.8 ( https://bugs.python.org/issue36933 #36932: asyncio-task.rst could use proper deprecated-removed directive https://bugs.python.org/issue36932 #36926: Implement methoddescr_call without _PyMethodDef_RawFastCallDic https://bugs.python.org/issue36926 #36924: Simplify implementation of classmethod_descriptor.__call__ https://bugs.python.org/issue36924 #36922: Implement Py_TPFLAGS_METHOD_DESCRIPTOR https://bugs.python.org/issue36922 #36921: Deprecate yield from and @coroutine in asyncio https://bugs.python.org/issue36921 #36905: test_typing.GetTypeHintTests.test_get_type_hints_modules_forwa https://bugs.python.org/issue36905 #36901: Fix leaks in /PC/bdist_wininst/install.c https://bugs.python.org/issue36901 #36896: clarify in types.rst that FunctionTypes & co constructors don https://bugs.python.org/issue36896 #36889: Merge StreamWriter and StreamReader into just asyncio.Stream https://bugs.python.org/issue36889 Most recent 15 issues waiting for review (15) ============================================= #36948: NameError in urllib.request.URLopener.retrieve https://bugs.python.org/issue36948 #36946: Possible signed integer overflow in slice handling https://bugs.python.org/issue36946 #36945: Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unc https://bugs.python.org/issue36945 #36943: Windows test changes for Windows ARM64 https://bugs.python.org/issue36943 #36942: Windows code changes for Windows ARM64 https://bugs.python.org/issue36942 #36941: Windows build changes for Windows ARM64 https://bugs.python.org/issue36941 #36937: New _PyObject_MakeTpCall() function https://bugs.python.org/issue36937 #36936: CALL_FUNCTION_KW opcode: keyword names must be non-empty https://bugs.python.org/issue36936 #36935: bpo-35813 introduced usage of the deprecated PyErr_SetFromWind https://bugs.python.org/issue36935 #36932: asyncio-task.rst could use proper deprecated-removed directive https://bugs.python.org/issue36932 #36929: Other Python _io implementations may not expose _io in their t https://bugs.python.org/issue36929 #36927: traceback docstrings should explicitly state return values ins https://bugs.python.org/issue36927 #36926: Implement methoddescr_call without _PyMethodDef_RawFastCallDic https://bugs.python.org/issue36926 #36924: Simplify implementation of classmethod_descriptor.__call__ https://bugs.python.org/issue36924 #36922: Implement Py_TPFLAGS_METHOD_DESCRIPTOR https://bugs.python.org/issue36922 Top 10 most discussed issues (10) ================================= #36867: Make semaphore_tracker track other system resources https://bugs.python.org/issue36867 11 msgs #36887: Add integer square root, math.isqrt https://bugs.python.org/issue36887 11 msgs #36763: PEP 587: Rework initialization API to prepare second version o https://bugs.python.org/issue36763 10 msgs #36917: ast.NodeVisitor no longer calls visit_Str https://bugs.python.org/issue36917 10 msgs #36870: test_asyncio: test_drain_raises() fails randomly on Windows https://bugs.python.org/issue36870 9 msgs #36880: Returning None from a callback with restype py_object decremen https://bugs.python.org/issue36880 9 msgs #36895: time.clock() marked for removal in 3.8 – still there. https://bugs.python.org/issue36895 9 msgs #36908: "This module is always available" is confusing https://bugs.python.org/issue36908 8 msgs #36945: Add _PyPreConfig.configure_locale: allow to leave LC_CTYPE unc https://bugs.python.org/issue36945 8 msgs #35495: argparse does not honor default argument for nargs=argparse.RE https://bugs.python.org/issue35495 7 msgs Issues closed (89) ================== #1257: atexit errors should result in nonzero exit code https://bugs.python.org/issue1257 closed by vstinner #2921: enable embedding: declare/#define only py* symbols in #include https://bugs.python.org/issue2921 closed by vstinner #3020: doctest should have lib2to3 integration https://bugs.python.org/issue3020 closed by scoder #3620: test_smtplib is flaky https://bugs.python.org/issue3620 closed by SilentGhost #6584: gzip module has no custom exception https://bugs.python.org/issue6584 closed by serhiy.storchaka #10991: trace fails when test imported a temporary file https://bugs.python.org/issue10991 closed by brett.cannon #13320: _remove_visual_c_ref in distutils.msvc9compiler causes DLL loa https://bugs.python.org/issue13320 closed by vstinner #13493: Import error with embedded python on AIX 6.1 https://bugs.python.org/issue13493 closed by vstinner #14228: Don't display traceback when import site is interrupted by CTR https://bugs.python.org/issue14228 closed by vstinner #14353: Proper gettext support in locale module https://bugs.python.org/issue14353 closed by benjamin.peterson #16100: Compiling vim with Python 3.3 support fails https://bugs.python.org/issue16100 closed by zach.ware #16961: No regression tests for -E and individual environment vars https://bugs.python.org/issue16961 closed by vstinner #17679: sysconfig generation uses some env variables multiple times https://bugs.python.org/issue17679 closed by vstinner #19175: Erroneous reference to "integer" in format string grammar https://bugs.python.org/issue19175 closed by cheryl.sabella #19216: stat cache for import bootstrap https://bugs.python.org/issue19216 closed by vstinner #21202: Naming a file` io.py` causes cryptic error message https://bugs.python.org/issue21202 closed by vstinner #21563: Segv during call to builtin_execfile in application embedding https://bugs.python.org/issue21563 closed by vstinner #22747: Interpreter fails in initialize on systems where HAVE_LANGINFO https://bugs.python.org/issue22747 closed by vstinner #23603: Embedding Python3.4 - PyUnicode_Check fails (MinGW-W64) https://bugs.python.org/issue23603 closed by vstinner #24280: Unable to install Python https://bugs.python.org/issue24280 closed by vstinner #24758: unittest.mock.Mock's new "unsafe" feature needs a better error https://bugs.python.org/issue24758 closed by xtreak #26122: Isolated mode doesn't ignore PYTHONHASHSEED https://bugs.python.org/issue26122 closed by vstinner #26480: add a flag that will not give the set a sys.stdin https://bugs.python.org/issue26480 closed by vstinner #26707: plistlib fails to parse bplist with 0x80 UID values https://bugs.python.org/issue26707 closed by serhiy.storchaka #26891: CPython doesn't work when you disable refcounting https://bugs.python.org/issue26891 closed by vstinner #29818: Py_SetStandardStreamEncoding leads to a memory error in debug https://bugs.python.org/issue29818 closed by vstinner #30464: gammavariate has a wrong comment https://bugs.python.org/issue30464 closed by mark.dickinson #30633: Python 3.6.1 installation issues on OpenSuse 42.1: ModuleNotFo https://bugs.python.org/issue30633 closed by vstinner #31473: Debug hooks on memory allocators are not thread safe (serialno https://bugs.python.org/issue31473 closed by vstinner #32177: spammers mine emails from bugs.python.org https://bugs.python.org/issue32177 closed by SilentGhost #32231: -bb option should override -W options https://bugs.python.org/issue32231 closed by vstinner #32832: doctest should support custom ps1/ps2 prompts https://bugs.python.org/issue32832 closed by cheryl.sabella #32995: Add a glossary entry for context variables https://bugs.python.org/issue32995 closed by cheryl.sabella #33123: Path.unlink should have a missing_ok parameter https://bugs.python.org/issue33123 closed by pitrou #33303: ElementTree Comment text isn't escaped https://bugs.python.org/issue33303 closed by scoder #33882: doc Mention breakpoint() in debugger-related FAQ https://bugs.python.org/issue33882 closed by matrixise #34152: performance of some list slice assignment margin cases can be https://bugs.python.org/issue34152 closed by pablogsal #34408: possible null pointer dereference in pystate.c https://bugs.python.org/issue34408 closed by pablogsal #34424: Unicode names break email header https://bugs.python.org/issue34424 closed by cheryl.sabella #34484: Unicode HOWTO incorrectly refers to Private Use Area for surro https://bugs.python.org/issue34484 closed by akuchling #34509: Starting to use gcc-8 on CI https://bugs.python.org/issue34509 closed by zach.ware #34579: test_embed.InitConfigTests fail on AIX https://bugs.python.org/issue34579 closed by vstinner #34682: Typo reports on docs@ https://bugs.python.org/issue34682 closed by cheryl.sabella #35138: timeit documentation should have example with function argumen https://bugs.python.org/issue35138 closed by vstinner #35589: BaseSelectorEventLoop.sock_sendall() performance regression: e https://bugs.python.org/issue35589 closed by asvetlov #36008: [good first issue] Update documentation for 3.8 https://bugs.python.org/issue36008 closed by cheryl.sabella #36471: PEP 432, PEP 587: Add _Py_RunMain() https://bugs.python.org/issue36471 closed by vstinner #36474: RecursionError resets trace function set via sys.settrace https://bugs.python.org/issue36474 closed by blueyed #36594: Undefined behavior due to incorrect usage of %p in format stri https://bugs.python.org/issue36594 closed by brett.cannon #36618: clang expects memory aligned on 16 bytes, but pymalloc aligns https://bugs.python.org/issue36618 closed by vstinner #36620: Documentation missing parameter for Itertools.zip_longest https://bugs.python.org/issue36620 closed by rhettinger #36684: codecov.io code coverage has not updated since 2019-04-13 https://bugs.python.org/issue36684 closed by inada.naoki #36725: Reference leak regression with Python3.8a3 https://bugs.python.org/issue36725 closed by vstinner #36728: Remove PyEval_ReInitThreads() from the public C API https://bugs.python.org/issue36728 closed by vstinner #36748: Optimize textio write buffering https://bugs.python.org/issue36748 closed by inada.naoki #36749: PPC64 AIX 3.x: compilation issue, linker fails to locate symbo https://bugs.python.org/issue36749 closed by vstinner #36751: Changes in the inspect module for PEP 570 https://bugs.python.org/issue36751 closed by pablogsal #36760: subprocess.run fails with capture_output=True and stderr=STDOU https://bugs.python.org/issue36760 closed by gregory.p.smith #36789: Unicode HOWTO incorrectly states that UTF-8 contains no zero b https://bugs.python.org/issue36789 closed by cheryl.sabella #36791: sum() relies on C signed overflow behaviour https://bugs.python.org/issue36791 closed by serhiy.storchaka #36799: Typo in ctypes documentation https://bugs.python.org/issue36799 closed by matrixise #36807: IDLE doesn't call os.fsync() https://bugs.python.org/issue36807 closed by terry.reedy #36818: Add PyInterpreterState.runtime. https://bugs.python.org/issue36818 closed by vstinner #36822: Minor grammatical fix in glossary.rst https://bugs.python.org/issue36822 closed by pablogsal #36845: ipaddres.IPv4Network and ipaddress.IPv6Network tuple construct https://bugs.python.org/issue36845 closed by inada.naoki #36875: argparse does not ship with translations https://bugs.python.org/issue36875 closed by terry.reedy #36883: 在ctypes里调用C库返回c_char_p类型时的问题 https://bugs.python.org/issue36883 closed by anthonypjshaw #36884: DeprecationWarning in test_asyncio.test_pep492.StreamReaderTes https://bugs.python.org/issue36884 closed by asvetlov #36886: Failed to construct CodeType on Python-3.8.0a4 https://bugs.python.org/issue36886 closed by SilentGhost #36899: datetime utcfromtimestamp ignores astimezone https://bugs.python.org/issue36899 closed by SilentGhost #36900: Use _PyCoreConfig rather than global configuration variables https://bugs.python.org/issue36900 closed by vstinner #36902: spam https://bugs.python.org/issue36902 closed by SilentGhost #36903: ResourceWarning in test_logging.HandlerTest.test_post_fork_chi https://bugs.python.org/issue36903 closed by xtreak #36909: LastUpdatedOrderedDict recipe uses super() unnecessarily https://bugs.python.org/issue36909 closed by rhettinger #36911: ast.parse outputs ast.Strs which do not differentiate between https://bugs.python.org/issue36911 closed by eric.smith #36912: Can't left-align strings using f-strings or format() https://bugs.python.org/issue36912 closed by gvanrossum #36914: zipimport does not load ZIP files produced by GitHub https://bugs.python.org/issue36914 closed by SilentGhost #36915: regrtest: when interrupted, temporary directory is not removed https://bugs.python.org/issue36915 closed by vstinner #36916: Swallow unhandled exception report introduced by 36802 https://bugs.python.org/issue36916 closed by asvetlov #36920: inspect.getcallargs sees optional arg to builtin as required https://bugs.python.org/issue36920 closed by brett.cannon #36923: Implemented __str__ for zip and map objects https://bugs.python.org/issue36923 closed by SilentGhost #36925: python3.7.3 can't work after MacOS upgrade from v10.14.4 to v1 https://bugs.python.org/issue36925 closed by ned.deily #36928: linkt threading.settrace to sys.settrace https://bugs.python.org/issue36928 closed by SilentGhost #36930: Windows proxy settings automatically used ... partly https://bugs.python.org/issue36930 closed by iwetzel #36931: json lib doesnt want to load from file https://bugs.python.org/issue36931 closed by zach.ware #36934: C API Function PyLong_AsDouble Returning Wrong Value https://bugs.python.org/issue36934 closed by sajjadfx #36938: Py_XDECREF on PyUnicodeobject raises SIGSEGV signal https://bugs.python.org/issue36938 closed by serhiy.storchaka #36939: Allow to use shutil.copytree for existing destination director https://bugs.python.org/issue36939 closed by jack1142 #1465646: test_grp & test_pwd fail https://bugs.python.org/issue1465646 closed by matrixise

1 0

bpo-36829: Add sys.unraisablehook()
by Victor Stinner 16 May '19

16 May '19

Hi, I recently modified Python 3.8 to start logging I/O error in io destructors when using the development mode (-X dev) or when Python is built in debug mode. This change allows to debug bpo-18748 very strange crash in PyThread_exit_thread(), when a thread closes an arbitrary file descriptor by mistake. The problem is that exceptions raised in a destructor cannot be passed to the current function call: the exception is logged into sys.stderr with a message like "Exception ignored in: ...". It's easy to miss such message in stderr. Thomas Grainger opened 36829 to propose to add -X abortunraisable command line option: abort immediately the current process (by calling Py_FatalError()) at the first PyErr_WriteUnraisable() call. Thomas had like a way to easily detect these in CI. I'm not satisfied by his proposal, since it only gives little control to the user on how "unraisable exceptions" are handled: the process dies, that's all. https://bugs.python.org/issue36829 I proposed a different approach: add a new sys.unraisablehook hook which is called to handle an "unraisable exception". To handle them differently, replace the hook. For example, I wrote a custom hook to log these exceptions into a file (the output on the Python test suite is interesting!). It also becomes trivial to reimplement Thomas's idea (kill the process): import signal def hook(unraisable): signal.raise_signal(signal.SIGABRT) sys.unraisablehook = hook I plan to merge my implementation soon, are you fine with that? https://github.com/python/cpython/pull/13187 -- The first implementation of my API used sys.unraisablehook(exc_type, exc_value, exc_tb, obj). The problem is that Serhiy Storchaka asked me to add a new error message field which breaks the API: the API is not future-proof. I modified my API to create an object to pack arguments. The new API becomes sys.unraisablehook(unraisable) where unraisable has 4 fields: exc_type, exc_value, exc_tb, obj. This API is now future-proof: adding a new field will not break existing custom hooks! By the way, I like this idea of adding an optional error message, and I plan to implement it once sys.unraisablehook is merged ;-) I already implemented it previously, but I reverted my changes to make the PR simpler to review. Extract of the documentation: """ Add new :func:`sys.unraisablehook` function which can be overriden to control how "unraisable exceptions" are handled. It is called when an exception has occurred but there is no way for Python to handle it. For example, when a destructor raises an exception or during garbage collection (:func:`gc.collect`). """ My implementation has a limitation: if PyErr_WriteUnraisable() is called after the Python finalization cleared sys attributes (almost the last function call of Py_Finalize), the default hook is called instead of the custom hook. In this case, sys.stderr is None and so the default hook does nothing. These late calls to PyErr_WriteUnraisable() cannot be catched with my approached, whereas Thomas Grainger's command line option "-X abortunraisable" allows that. My concern with Thomas's approach is that if the problem is killed with SIGABRT by such late PyErr_WriteUnraisable(), a low-level debugger like gdb is needed to investigate the crash, since Python is already finalized and so cannot be used to investigate. I prefer to allow arbitrary hook with the limitation, rather than always kill the process with SIGABRT at the first PyErr_WriteUnraisable() and require to use a low-level debugger. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

8 24

Need review
by Antoine Pitrou 16 May '19

16 May '19

Hello, I need a review on the PEP 574 implementation: https://github.com/python/cpython/pull/7076 I would really like it to be in 3.8 and so ideally it should be merged within two weeks. Regards Antoine.

1 0

bpo-9584: Added brace expressions to the glob and fnmatch.
by matus valo 16 May '19

16 May '19

Hi All, I have implemented PR fixing bpo-9584 few months ago. Recently, I was encouraged that conversation on DEV mailing list should be started here on DEV mailing list. From the conversation in bug report, I understood that the main problem is that straightforward implementation can break a backward compatibility. E.g.: * Patched python: >>> import os >>> import glob >>> os.makedirs('a{b,c}d/e') >>> os.listdir('a{b,c}d') ['e'] >>> glob.glob('a{b,c}d/*') [] >>> * Unpatched python: >>> import os, glob >>> os.makedirs('a{b,c}d/e') >>> os.listdir('a{b,c}d') ['e'] >>> glob.glob('a{b,c}d/*') ['a{b,c}d/e'] How can I proceed with the PR? Please advise. Thanks, Matus Valo

1 0

Python Language Summit 2019 blog posts
by Mariatta 15 May '19

15 May '19

If you have FOMO (fear of missing out) of Python Language Summit 2019, worry no more. We invited A. Jesse Jiryu Davis to cover for the language summit, and the blog posts are starting to appear in The PSF's official blog. Starts here: http://pyfound.blogspot.com/2019/05/the-2019-python-language-summit.html More links will be added as they become available. Currently available posts: - Russell Keith-Magee: Python On Other Platforms https://pyfound.blogspot.com/2019/05/russell-keith-magee-python-on-other.ht… - Paul Ganssle: Time Zones In The Standard Library https://pyfound.blogspot.com/2019/05/paul-ganssle-time-zones-in-standard.ht… Thank you Jesse for writing up these posts, and thanks to all language summit presenters and attendees. ᐧ

2 1

RFC: PEP 587 "Python Initialization Configuration": 2nd version
by Victor Stinner 15 May '19

15 May '19

Hi, Thanks to Steve Dower's feedback, I enhanced and completed my PEP 587. Main changes: * It is now possible to read the configuration and then modify the read configuration. For example, new directories can be added to PyConfig.module_search_paths (see the example below and the example in the PEP) * PyConfig is now "dynamic" by default: strings are duplicated and PyConfig_Clear() must be called to release memory * PyConfig now only uses wchar_t* for strings (unicode): char* (bytes) is no longer used. I had to hack CPython internals for that :-) * I added a "_config_version" private field to PyPreConfig and PyConfig to prepare the backward compatibility for future changes. * I removed the Open Question section: all known issues have been fixed. During the Language Summit, Brett Cannon said that Steve Dower declined the offer to be the BDFL-delegate for this PEP. Thomas Wouters proposed himself to be the new BDFL-delegate. Example to read the configuration, append a directory to sys.path (module_search_paths) and then initialize Python with this configuration: void init_python(void) { PyInitError err; PyConfig config = PyConfig_INIT; err = PyConfig_Read(&config); if (_Py_INIT_FAILED(err)) { goto fail; } err = PyWideStringList_Append(&config.module_search_paths, L"/path/to/more/modules"); if (_Py_INIT_FAILED(err)) { goto fail; } err = Py_InitializeFromConfig(&config); if (_Py_INIT_FAILED(err)) { goto fail; } PyConfig_Clear(&config); return; fail: PyConfig_Clear(&config); Py_ExitInitError(err); } The HTML version will be online shortly: https://www.python.org/dev/peps/pep-0587/ Full text below. Victor PEP: 587 Title: Python Initialization Configuration Author: Nick Coghlan <ncoghlan(a)gmail.com>, Victor Stinner <vstinner(a)redhat.com> Discussions-To: python-dev(a)python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Mar-2019 Python-Version: 3.8 Abstract ======== Add a new C API to configure the Python Initialization providing finer control on the whole configuration and better error reporting. Rationale ========= Python is highly configurable but its configuration evolved organically: configuration parameters is scattered all around the code using different ways to set them (mostly global configuration variables and functions). A straightforward and reliable way to configure Python is needed. Some configuration parameters are not accessible from the C API, or not easily. The C API of Python 3.7 Initialization takes ``wchar_t*`` strings as input whereas the Python filesystem encoding is set during the initialization. This PEP is a partial implementation of PEP 432 which is the overall design. New fields can be added later to ``PyConfig`` structure to finish the implementation of the PEP 432 (add a new partial initialization which allows to configure Python using Python objects to finish the full initialization). Python Initialization C API =========================== This PEP proposes to add the following new structures, functions and macros. New structures (4): * ``PyConfig`` * ``PyInitError`` * ``PyPreConfig`` * ``PyWideStringList`` New functions (16): * ``Py_PreInitialize(config)`` * ``Py_PreInitializeFromArgs(config, argc, argv)`` * ``Py_PreInitializeFromWideArgs(config, argc, argv)`` * ``PyWideStringList_Append(list, item)`` * ``PyConfig_DecodeLocale(config_str, str)`` * ``PyConfig_SetString(config_str, str)`` * ``PyConfig_Read(config)`` * ``PyConfig_SetArgv(config, argc, argv)`` * ``PyConfig_SetWideArgv(config, argc, argv)`` * ``PyConfig_Clear(config)`` * ``Py_InitializeFromConfig(config)`` * ``Py_InitializeFromArgs(config, argc, argv)`` * ``Py_InitializeFromWideArgs(config, argc, argv)`` * ``Py_UnixMain(argc, argv)`` * ``Py_ExitInitError(err)`` * ``Py_RunMain()`` New macros (9): * ``PyPreConfig_INIT`` * ``PyConfig_INIT`` * ``Py_INIT_OK()`` * ``Py_INIT_ERR(MSG)`` * ``Py_INIT_NO_MEMORY()`` * ``Py_INIT_EXIT(EXITCODE)`` * ``Py_INIT_IS_ERROR(err)`` * ``Py_INIT_IS_EXIT(err)`` * ``Py_INIT_FAILED(err)`` This PEP also adds ``_PyRuntimeState.preconfig`` (``PyPreConfig`` type) and ``PyInterpreterState.config`` (``PyConfig`` type) fields to these internal structures. ``PyInterpreterState.config`` becomes the new reference configuration, replacing global configuration variables and other private variables. PyWideStringList ---------------- ``PyWideStringList`` is a list of ``wchar_t*`` strings. Example to initialize a string from C static array:: static wchar_t* argv[2] = { L"-c", L"pass", }; PyWideStringList config_argv = PyWideStringList_INIT; config_argv.length = Py_ARRAY_LENGTH(argv); config_argv.items = argv; ``PyWideStringList`` structure fields: * ``length`` (``Py_ssize_t``) * ``items`` (``wchar_t**``) Methods: * ``PyInitError PyWideStringList_Append(PyWideStringList *list, const wchar_t *item)``: Append *item* to *list*. If *length* is non-zero, *items* must be non-NULL and all strings must be non-NULL. PyInitError ----------- ``PyInitError`` is a structure to store an error message or an exit code for the Python Initialization. For an error, it stores the C function name which created the error. Example:: PyInitError alloc(void **ptr, size_t size) { *ptr = PyMem_RawMalloc(size); if (*ptr == NULL) { return Py_INIT_NO_MEMORY(); } return Py_INIT_OK(); } int main(int argc, char **argv) { void *ptr; PyInitError err = alloc(&ptr, 16); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } PyMem_Free(ptr); return 0; } ``PyInitError`` fields: * ``exitcode`` (``int``): argument passed to ``exit()`` on Unix and to ``ExitProcess()`` on Windows. Only set by ``Py_INIT_EXIT()``. * ``err_msg`` (``const char*``): error message * private ``_func`` field: used by ``Py_INIT_ERR()`` to store the C function name which created the error. * private ``_type`` field: for internal usage only. Macro to create an error: * ``Py_INIT_OK()``: success * ``Py_INIT_ERR(err_msg)``: initialization error with a message * ``Py_INIT_NO_MEMORY()``: memory allocation failure (out of memory) * ``Py_INIT_EXIT(exitcode)``: exit Python with the specified exit code Other macros and functions: * ``Py_INIT_IS_ERROR(err)``: Is the result an error? * ``Py_INIT_IS_EXIT(err)``: Is the result an exit? * ``Py_INIT_FAILED(err)``: Is the result an error or an exit? Similar to ``Py_INIT_IS_ERROR(err) || Py_INIT_IS_EXIT(err)``. * ``Py_ExitInitError(err)``: Call ``exit(exitcode)`` on Unix or ``ExitProcess(exitcode)`` if the result is an exit, call ``Py_FatalError(err_msg)`` if the result is an error. Must not be called if the result is a success. Pre-Initialization with PyPreConfig ----------------------------------- ``PyPreConfig`` structure is used to pre-initialize Python: * Set the memory allocator * Configure the LC_CTYPE locale * Set the UTF-8 mode Example using the pre-initialization to enable the UTF-8 Mode:: PyPreConfig preconfig = PyPreConfig_INIT; preconfig.utf8_mode = 1; PyInitError err = Py_PreInitialize(&preconfig); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } /* at this point, Python will speak UTF-8 */ Py_Initialize(); /* ... use Python API here ... */ Py_Finalize(); Functions to pre-initialize Python: * ``PyInitError Py_PreInitialize(const PyPreConfig *config)`` * ``PyInitError Py_PreInitializeFromArgs(const PyPreConfig *config, int argc, char **argv)`` * ``PyInitError Py_PreInitializeFromWideArgs(const PyPreConfig *config, int argc, wchar_t **argv)`` If Python should be pre-initialized explicitly first and then initialized with command line arguments, it is possible to pass these command line arguments to the pre-initialization since they impact the encodings. For example, ``-X utf8`` enables the UTF-8 Mode. These functions can be called with *config* set to ``NULL``. The caller is responsible to handle error using ``Py_INIT_FAILED()`` and ``Py_ExitInitError()``. ``PyPreConfig`` fields: * ``allocator`` (``char*``): name of the memory allocator (ex: ``"malloc"``) * ``coerce_c_locale_warn`` (``int``): if non-zero, emit a warning if the C locale is coerced. * ``coerce_c_locale`` (``int``): if equals to 2, coerce the C locale; if equals to 1, read the LC_CTYPE to decide if it should be coerced. * ``dev_mode`` (``int``): see ``PyConfig.dev_mode`` * ``isolated`` (``int``): see ``PyConfig.isolated`` * ``legacy_windows_fs_encoding`` (``int``, Windows only): if non-zero, set the Python filesystem encoding to ``"mbcs"``. * ``use_environment`` (``int``): see ``PyConfig.use_environment`` * ``utf8_mode`` (``int``): if non-zero, enable the UTF-8 mode There is also a private field which is for internal-usage only: * ``_config_version`` (``int``): Configuration version, used for ABI compatibility The C locale coercion (PEP 538) and the UTF-8 Mode (PEP 540) are disabled by default in ``PyPreConfig``. Set ``coerce_c_locale``, ``coerce_c_locale_warn`` and ``utf8_mode`` to ``-1`` to let Python enable them depending on the user configuration. Initialization with PyConfig ---------------------------- The ``PyConfig`` structure contains all parameters to configure Python. Example:: PyInitError err; PyConfig config = PyConfig_INIT; err = PyConfig_SetString(&config.program_name, L"my_program"); if (_Py_INIT_FAILED(err)) { Py_ExitInitError(err); } err = Py_InitializeFromConfig(&config); PyConfig_Clear(&config); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } ``PyConfig`` methods: * ``PyInitError PyConfig_SetString(wchar_t **config_str, const wchar_t *str)``: Set a config wide string field from *str* (copy the string) * ``PyInitError PyConfig_DecodeLocale(wchar_t **config_str, const char *str)``: Decode *str* using ``Py_DecodeLocale()`` and set the result into ``*config_str``. Pre-initialize Python if needed to ensure that encodings are properly configured. * ``PyInitError PyConfig_SetArgv(PyConfig *config, int argc, char **argv)``: Set command line arguments (decode bytes). Pre-initialize Python if needed to ensure that encodings are properly configured. * ``PyInitError PyConfig_SetWideArgv(PyConfig *config, int argc, wchar_t **argv)``: Set command line arguments (wide characters). * ``PyInitError PyConfig_Read(PyConfig *config)``: Read all Python configuration * ``void PyConfig_Clear(PyConfig *config)``: Release memory Functions to initialize Python: * ``PyInitError Py_InitializeFromConfig(const PyConfig *config)`` These functions can be called with *config* set to ``NULL``. The caller is responsible to handler error using ``Py_INIT_FAILED()`` and ``Py_ExitInitError()``. PyConfig fields: * ``argv`` (``PyWideStringList``): ``sys.argv`` * ``base_exec_prefix`` (``wchar_t*``): ``sys.base_exec_prefix`` * ``base_prefix`` (``wchar_t*``): ``sys.base_prefix`` * ``buffered_stdio`` (``int``): if equals to 0, enable unbuffered mode, make stdout and stderr streams to be unbuffered. * ``bytes_warning`` (``int``): if equals to 1, issue a warning when comparing ``bytes`` or ``bytearray`` with ``str``, or comparing ``bytes`` with ``int``. If equal or greater to 2, raise a ``BytesWarning`` exception. * ``check_hash_pycs_mode`` (``wchar_t*``): ``--check-hash-based-pycs`` command line option value (see PEP 552) * ``dev_mode`` (``int``): Development mode * ``dll_path`` (``wchar_t*``, Windows only): Windows DLL path * ``dump_refs`` (``int``): if non-zero, display all objects still alive at exit * ``exec_prefix`` (``wchar_t*``): ``sys.exec_prefix`` * ``executable`` (``wchar_t*``): ``sys.executable`` * ``faulthandler`` (``int``): if non-zero, call ``faulthandler.enable()`` * ``filesystem_encoding`` (``wchar_t*``): Filesystem encoding, ``sys.getfilesystemencoding()`` * ``filesystem_errors`` (``wchar_t*``): Filesystem encoding errors, ``sys.getfilesystemencodeerrors()`` * ``use_hash_seed`` (``int``), ``hash_seed`` (``unsigned long``): randomized hash function seed * ``home`` (``wchar_t*``): Python home * ``import_time`` (``int``): if non-zero, profile import time * ``inspect`` (``int``): enter interactive mode after executing a script or a command * ``install_signal_handlers`` (``int``): install signal handlers? * ``interactive`` (``int``): interactive mode * ``legacy_windows_stdio`` (``int``, Windows only): if non-zero, use ``io.FileIO`` instead of ``WindowsConsoleIO`` for ``sys.stdin``, ``sys.stdout`` and ``sys.stderr``. * ``malloc_stats`` (``int``): if non-zero, dump memory allocation statistics at exit * ``module_search_path_env`` (``wchar_t*``): ``PYTHONPATH`` environment variale value * ``use_module_search_paths`` (``int``), ``module_search_paths`` (``PyWideStringList``): ``sys.path`` * ``optimization_level`` (``int``): compilation optimization level * ``parser_debug`` (``int``): if non-zero, turn on parser debugging output (for expert only, depending on compilation options). * ``prefix`` (``wchar_t*``): ``sys.prefix`` * ``program_name`` (``wchar_t*``): Program name * ``program`` (``wchar_t*``): ``argv[0]`` or an empty string * ``pycache_prefix`` (``wchar_t*``): ``.pyc`` cache prefix * ``quiet`` (``int``): quiet mode (ex: don't display the copyright and version messages even in interactive mode) * ``run_command`` (``wchar_t*``): ``-c COMMAND`` argument * ``run_filename`` (``wchar_t*``): ``python3 SCRIPT`` argument * ``run_module`` (``wchar_t*``): ``python3 -m MODULE`` argument * ``show_alloc_count`` (``int``): show allocation counts at exit? * ``show_ref_count`` (``int``): show total reference count at exit? * ``site_import`` (``int``): import the ``site`` module at startup? * ``skip_source_first_line`` (``int``): skip the first line of the source * ``stdio_encoding`` (``wchar_t*``), ``stdio_errors`` (``wchar_t*``): encoding and encoding errors of ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` * ``tracemalloc`` (``int``): if non-zero, call ``tracemalloc.start(value)`` * ``user_site_directory`` (``int``): if non-zero, add user site directory to ``sys.path`` * ``verbose`` (``int``): if non-zero, enable verbose mode * ``warnoptions`` (``PyWideStringList``): options of the ``warnings`` module to build filters * ``write_bytecode`` (``int``): if non-zero, write ``.pyc`` files * ``xoptions`` (``PyWideStringList``): ``sys._xoptions`` There are also private fields which are for internal-usage only: * ``_config_version`` (``int``): Configuration version, used for ABI compatibility * ``_frozen`` (``int``): Emit warning when computing the path configuration? * ``_install_importlib`` (``int``): Install importlib? More complete commented example modifying the configuration before calling ``PyConfig_Read()`` and then modify the read configuration:: PyInitError init_python(const char *program_name) { PyInitError err; PyConfig config = PyConfig_INIT; /* Set the program name before reading the configuraton (decode byte string from the locale encoding) */ err = PyConfig_DecodeLocale(&config.program_name, program_name); if (_Py_INIT_FAILED(err)) { goto fail; } /* Read all configuration at once */ err = PyConfig_Read(&config); if (_Py_INIT_FAILED(err)) { goto fail; } /* Append our custom search path to sys.path */ err = PyWideStringList_Append(&config.module_search_paths, L"/path/to/more/modules"); if (_Py_INIT_FAILED(err)) { goto fail; } /* Override executable computed by PyConfig_Read() */ err = PyConfig_SetString(&config.executable, L"my_executable"); if (_Py_INIT_FAILED(err)) { goto fail; } err = Py_InitializeFromConfig(&config); /* Py_InitializeFromConfig() copied config which must now be cleared to release memory */ PyConfig_Clear(&config); return err; fail: PyConfig_Clear(&config); Py_ExitInitError(err); } .. note:: ``PyConfig`` does not have any field for extra inittab functions: ``PyImport_AppendInittab()`` and ``PyImport_ExtendInittab()`` functions are still relevant. Initialization with static PyConfig ----------------------------------- When no ``PyConfig`` method is used but only ``Py_InitializeFromConfig()``, the caller is responsible for managing ``PyConfig`` memory which means that static strings and static string lists can be used rather than using dynamically allocated memory. It can be used for most simple configurations. Example of Python initialization enabling the isolated mode:: PyConfig config = PyConfig_INIT; config.isolated = 1; PyInitError err = Py_InitializeFromConfig(&config); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } /* ... use Python API here ... */ Py_Finalize(); In this example, ``PyConfig_Clear()`` is not needed since ``config`` does not contain any dynamically allocated string: ``Py_InitializeFromConfig`` is responsible for filling other fields and manage the memory. For convenience, two other functions are provided: * ``PyInitError Py_InitializeFromArgs(const PyConfig *config, int argc, char **argv)`` * ``PyInitError Py_InitializeFromWideArgs(const PyConfig *config, int argc, wchar_t **argv)`` These functions can be used with static ``PyConfig``. Pseudo-code of ``Py_InitializeFromArgs()``:: PyInitError init_with_args(const PyConfig *src_config, int argc, char **argv) { PyInitError err; PyConfig config = PyConfig_INIT; /* Copy strings and string lists * (memory dynamically allocated on the heap) */ err = _PyConfig_Copy(&config, src_config); if (Py_INIT_FAILED(err)) { goto exit; } /* Set config.argv: decode argv bytes. Pre-initialize Python if needed to ensure that the encodings are properly configured. */ err = PyConfig_SetArgv(&config, argc, argv); if (Py_INIT_FAILED(err)) { goto exit; } err = Py_InitializeFromConfig(&config); exit: PyConfig_Clear(&config); return err; } where ``_PyConfig_Copy()`` is an internal function. The actual implementation of ``Py_InitializeFromArgs()`` is more complex. Py_UnixMain() ------------- Python 3.7 provides a high-level ``Py_Main()`` function which requires to pass command line arguments as ``wchar_t*`` strings. It is non-trivial to use the correct encoding to decode bytes. Python has its own set of issues with C locale coercion and UTF-8 Mode. This PEP adds a new ``Py_UnixMain()`` function which takes command line arguments as bytes:: int Py_UnixMain(int argc, char **argv) Py_RunMain() ------------ The new ``Py_RunMain()`` function executes the command (``PyConfig.run_command``), the script (``PyConfig.run_filename``) or the module (``PyConfig.run_module``) specified on the command line or in the configuration, and then finalizes Python. It returns an exit status that can be passed to the ``exit()`` function. Example of custom Python executable always running in isolated mode:: #include <Python.h> int main(int argc, char *argv[]) { PyConfig config = PyConfig_INIT; config.isolated = 1; PyInitError err = Py_InitializeFromArgs(&config, argc, argv); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } /* put more configuration code here if needed */ return Py_RunMain(); } The example is a basic implementation of the "System Python Executable" discussed in PEP 432. Memory allocations and Py_DecodeLocale() ---------------------------------------- Python memory allocation functions like ``PyMem_RawMalloc()`` must not be used before Python pre-initialization. Calling directly ``malloc()`` and ``free()`` is always safe. For ``PyPreConfig`` and static ``PyConfig``, the caller is responsible to manage dynamically allocated strings, but static strings and static string lists are fine. Dynamic ``PyConfig`` requires to call ``PyConfig_Clear()`` to release memory. ``Py_DecodeLocale()`` must not be called before the pre-initialization. When using dynanic configuration, ``PyConfig_DecodeLocale()`` must be used instead of ``Py_DecodeLocale()``. Backwards Compatibility ======================= This PEP only adds a new API: it leaves the existing API unchanged and has no impact on the backwards compatibility. Annex: Python Configuration =========================== Priority and Rules ------------------ Priority of configuration parameters, highest to lowest: * ``PyConfig`` * ``PyPreConfig`` * Configuration files * Command line options * Environment variables * Global configuration variables Priority of warning options, highest to lowest: * ``PyConfig.warnoptions`` * ``PyConfig.dev_mode`` (add ``"default"``) * ``PYTHONWARNINGS`` environment variables * ``-W WARNOPTION`` command line argument * ``PyConfig.bytes_warning`` (add ``"error::BytesWarning"`` if greater than 1, or add ``"default::BytesWarning``) Rules on ``PyConfig`` and ``PyPreConfig`` parameters: * If ``isolated`` is non-zero, ``use_environment`` and ``user_site_directory`` are set to 0 * If ``legacy_windows_fs_encoding`` is non-zero, ``utf8_mode`` is set to 0 * If ``dev_mode`` is non-zero, ``allocator`` is set to ``"debug"``, ``faulthandler`` is set to 1, and ``"default"`` filter is added to ``warnoptions``. But ``PYTHONMALLOC`` has the priority over ``dev_mode`` to set the memory allocator. Configuration Files ------------------- Python configuration files: * ``pyvenv.cfg`` * ``python._pth`` (Windows only) * ``pybuilddir.txt`` (Unix only) Global Configuration Variables ------------------------------ Global configuration variables mapped to ``PyPreConfig`` fields: ======================================== ================================ Variable Field ======================================== ================================ ``Py_LegacyWindowsFSEncodingFlag`` ``legacy_windows_fs_encoding`` ``Py_LegacyWindowsFSEncodingFlag`` ``legacy_windows_fs_encoding`` ``Py_UTF8Mode`` ``utf8_mode`` ``Py_UTF8Mode`` ``utf8_mode`` ======================================== ================================ Global configuration variables mapped to ``PyConfig`` fields: ======================================== ================================ Variable Field ======================================== ================================ ``Py_BytesWarningFlag`` ``bytes_warning`` ``Py_DebugFlag`` ``parser_debug`` ``Py_DontWriteBytecodeFlag`` ``write_bytecode`` ``Py_FileSystemDefaultEncodeErrors`` ``filesystem_errors`` ``Py_FileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_FrozenFlag`` ``_frozen`` ``Py_HasFileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_HashRandomizationFlag`` ``use_hash_seed``, ``hash_seed`` ``Py_IgnoreEnvironmentFlag`` ``use_environment`` ``Py_InspectFlag`` ``inspect`` ``Py_InteractiveFlag`` ``interactive`` ``Py_IsolatedFlag`` ``isolated`` ``Py_LegacyWindowsStdioFlag`` ``legacy_windows_stdio`` ``Py_NoSiteFlag`` ``site_import`` ``Py_NoUserSiteDirectory`` ``user_site_directory`` ``Py_OptimizeFlag`` ``optimization_level`` ``Py_QuietFlag`` ``quiet`` ``Py_UnbufferedStdioFlag`` ``buffered_stdio`` ``Py_VerboseFlag`` ``verbose`` ``_Py_HasFileSystemDefaultEncodeErrors`` ``filesystem_errors`` ``Py_BytesWarningFlag`` ``bytes_warning`` ``Py_DebugFlag`` ``parser_debug`` ``Py_DontWriteBytecodeFlag`` ``write_bytecode`` ``Py_FileSystemDefaultEncodeErrors`` ``filesystem_errors`` ``Py_FileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_FrozenFlag`` ``_frozen`` ``Py_HasFileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_HashRandomizationFlag`` ``use_hash_seed``, ``hash_seed`` ``Py_IgnoreEnvironmentFlag`` ``use_environment`` ``Py_InspectFlag`` ``inspect`` ``Py_InteractiveFlag`` ``interactive`` ``Py_IsolatedFlag`` ``isolated`` ``Py_LegacyWindowsStdioFlag`` ``legacy_windows_stdio`` ``Py_NoSiteFlag`` ``site_import`` ``Py_NoUserSiteDirectory`` ``user_site_directory`` ``Py_OptimizeFlag`` ``optimization_level`` ``Py_QuietFlag`` ``quiet`` ``Py_UnbufferedStdioFlag`` ``buffered_stdio`` ``Py_VerboseFlag`` ``verbose`` ``_Py_HasFileSystemDefaultEncodeErrors`` ``filesystem_errors`` ======================================== ================================ ``Py_LegacyWindowsFSEncodingFlag`` and ``Py_LegacyWindowsStdioFlag`` are only available on Windows. Command Line Arguments ---------------------- Usage:: python3 [options] python3 [options] -c COMMAND python3 [options] -m MODULE python3 [options] SCRIPT Command line options mapped to pseudo-action on ``PyConfig`` fields: ================================ ================================ Option ``PyPreConfig`` field ================================ ================================ ``-X dev`` ``dev_mode = 1`` ``-X utf8=N`` ``utf8_mode = N`` ================================ ================================ Command line options mapped to pseudo-action on ``PyConfig`` fields: ================================ ================================ Option ``PyConfig`` field ================================ ================================ ``-b`` ``bytes_warning++`` ``-B`` ``write_bytecode = 0`` ``-c COMMAND`` ``run_module = COMMAND`` ``--check-hash-based-pycs=MODE`` ``_check_hash_pycs_mode = MODE`` ``-d`` ``parser_debug++`` ``-E`` ``use_environment = 0`` ``-i`` ``inspect++`` and ``interactive++`` ``-I`` ``isolated = 1`` ``-m MODULE`` ``run_module = MODULE`` ``-O`` ``optimization_level++`` ``-q`` ``quiet++`` ``-R`` ``use_hash_seed = 0`` ``-s`` ``user_site_directory = 0`` ``-S`` ``site_import`` ``-t`` ignored (kept for backwards compatibility) ``-u`` ``buffered_stdio = 0`` ``-v`` ``verbose++`` ``-W WARNING`` add ``WARNING`` to ``warnoptions`` ``-x`` ``skip_source_first_line = 1`` ``-X XOPTION`` add ``XOPTION`` to ``xoptions`` ``-X dev`` ``dev_mode = 1`` ``-X faulthandler`` ``faulthandler = 1`` ``-X importtime`` ``import_time = 1`` ``-X pycache_prefix=PREFIX`` ``pycache_prefix = PREFIX`` ``-X show_alloc_count`` ``show_alloc_count = 1`` ``-X show_ref_count`` ``show_ref_count = 1`` ``-X tracemalloc=N`` ``tracemalloc = N`` ================================ ================================ ``-h``, ``-?`` and ``-V`` options are handled outside ``PyConfig``. Environment Variables --------------------- Environment variables mapped to ``PyPreConfig`` fields: ================================= ============================================= Variable ``PyPreConfig`` field ================================= ============================================= ``PYTHONCOERCECLOCALE`` ``coerce_c_locale``, ``coerce_c_locale_warn`` ``PYTHONDEVMODE`` ``dev_mode`` ``PYTHONLEGACYWINDOWSFSENCODING`` ``legacy_windows_fs_encoding`` ``PYTHONMALLOC`` ``allocator`` ``PYTHONUTF8`` ``utf8_mode`` ================================= ============================================= Environment variables mapped to ``PyConfig`` fields: ================================= ==================================== Variable ``PyConfig`` field ================================= ==================================== ``PYTHONDEBUG`` ``parser_debug`` ``PYTHONDEVMODE`` ``dev_mode`` ``PYTHONDONTWRITEBYTECODE`` ``write_bytecode`` ``PYTHONDUMPREFS`` ``dump_refs`` ``PYTHONEXECUTABLE`` ``program_name`` ``PYTHONFAULTHANDLER`` ``faulthandler`` ``PYTHONHASHSEED`` ``use_hash_seed``, ``hash_seed`` ``PYTHONHOME`` ``home`` ``PYTHONINSPECT`` ``inspect`` ``PYTHONIOENCODING`` ``stdio_encoding``, ``stdio_errors`` ``PYTHONLEGACYWINDOWSSTDIO`` ``legacy_windows_stdio`` ``PYTHONMALLOCSTATS`` ``malloc_stats`` ``PYTHONNOUSERSITE`` ``user_site_directory`` ``PYTHONOPTIMIZE`` ``optimization_level`` ``PYTHONPATH`` ``module_search_path_env`` ``PYTHONPROFILEIMPORTTIME`` ``import_time`` ``PYTHONPYCACHEPREFIX,`` ``pycache_prefix`` ``PYTHONTRACEMALLOC`` ``tracemalloc`` ``PYTHONUNBUFFERED`` ``buffered_stdio`` ``PYTHONVERBOSE`` ``verbose`` ``PYTHONWARNINGS`` ``warnoptions`` ================================= ==================================== ``PYTHONLEGACYWINDOWSFSENCODING`` and ``PYTHONLEGACYWINDOWSSTDIO`` are specific to Windows. ``PYTHONDEVMODE`` is mapped to ``PyPreConfig.dev_mode`` and ``PyConfig.dev_mode``. Annex: Python 3.7 API ===================== Python 3.7 has 4 functions in its C API to initialize and finalize Python: * ``Py_Initialize()``, ``Py_InitializeEx()``: initialize Python * ``Py_Finalize()``, ``Py_FinalizeEx()``: finalize Python Python can be configured using scattered global configuration variables (like ``Py_IgnoreEnvironmentFlag``) and using the following functions: * ``PyImport_AppendInittab()`` * ``PyImport_ExtendInittab()`` * ``PyMem_SetAllocator()`` * ``PyMem_SetupDebugHooks()`` * ``PyObject_SetArenaAllocator()`` * ``Py_SetPath()`` * ``Py_SetProgramName()`` * ``Py_SetPythonHome()`` * ``Py_SetStandardStreamEncoding()`` * ``PySys_AddWarnOption()`` * ``PySys_AddXOption()`` * ``PySys_ResetWarnOptions()`` There is also a high-level ``Py_Main()`` function. Copyright ========= This document has been placed in the public domain.

5 10

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev May 2019