Mailman 3 May 2019 - Python-Dev

RFC: PEP 587 "Python Initialization Configuration": 2nd version
by Victor Stinner 15 May '19

15 May '19

Hi, Thanks to Steve Dower's feedback, I enhanced and completed my PEP 587. Main changes: * It is now possible to read the configuration and then modify the read configuration. For example, new directories can be added to PyConfig.module_search_paths (see the example below and the example in the PEP) * PyConfig is now "dynamic" by default: strings are duplicated and PyConfig_Clear() must be called to release memory * PyConfig now only uses wchar_t* for strings (unicode): char* (bytes) is no longer used. I had to hack CPython internals for that :-) * I added a "_config_version" private field to PyPreConfig and PyConfig to prepare the backward compatibility for future changes. * I removed the Open Question section: all known issues have been fixed. During the Language Summit, Brett Cannon said that Steve Dower declined the offer to be the BDFL-delegate for this PEP. Thomas Wouters proposed himself to be the new BDFL-delegate. Example to read the configuration, append a directory to sys.path (module_search_paths) and then initialize Python with this configuration: void init_python(void) { PyInitError err; PyConfig config = PyConfig_INIT; err = PyConfig_Read(&config); if (_Py_INIT_FAILED(err)) { goto fail; } err = PyWideStringList_Append(&config.module_search_paths, L"/path/to/more/modules"); if (_Py_INIT_FAILED(err)) { goto fail; } err = Py_InitializeFromConfig(&config); if (_Py_INIT_FAILED(err)) { goto fail; } PyConfig_Clear(&config); return; fail: PyConfig_Clear(&config); Py_ExitInitError(err); } The HTML version will be online shortly: https://www.python.org/dev/peps/pep-0587/ Full text below. Victor PEP: 587 Title: Python Initialization Configuration Author: Nick Coghlan <ncoghlan(a)gmail.com>, Victor Stinner <vstinner(a)redhat.com> Discussions-To: python-dev(a)python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 27-Mar-2019 Python-Version: 3.8 Abstract ======== Add a new C API to configure the Python Initialization providing finer control on the whole configuration and better error reporting. Rationale ========= Python is highly configurable but its configuration evolved organically: configuration parameters is scattered all around the code using different ways to set them (mostly global configuration variables and functions). A straightforward and reliable way to configure Python is needed. Some configuration parameters are not accessible from the C API, or not easily. The C API of Python 3.7 Initialization takes ``wchar_t*`` strings as input whereas the Python filesystem encoding is set during the initialization. This PEP is a partial implementation of PEP 432 which is the overall design. New fields can be added later to ``PyConfig`` structure to finish the implementation of the PEP 432 (add a new partial initialization which allows to configure Python using Python objects to finish the full initialization). Python Initialization C API =========================== This PEP proposes to add the following new structures, functions and macros. New structures (4): * ``PyConfig`` * ``PyInitError`` * ``PyPreConfig`` * ``PyWideStringList`` New functions (16): * ``Py_PreInitialize(config)`` * ``Py_PreInitializeFromArgs(config, argc, argv)`` * ``Py_PreInitializeFromWideArgs(config, argc, argv)`` * ``PyWideStringList_Append(list, item)`` * ``PyConfig_DecodeLocale(config_str, str)`` * ``PyConfig_SetString(config_str, str)`` * ``PyConfig_Read(config)`` * ``PyConfig_SetArgv(config, argc, argv)`` * ``PyConfig_SetWideArgv(config, argc, argv)`` * ``PyConfig_Clear(config)`` * ``Py_InitializeFromConfig(config)`` * ``Py_InitializeFromArgs(config, argc, argv)`` * ``Py_InitializeFromWideArgs(config, argc, argv)`` * ``Py_UnixMain(argc, argv)`` * ``Py_ExitInitError(err)`` * ``Py_RunMain()`` New macros (9): * ``PyPreConfig_INIT`` * ``PyConfig_INIT`` * ``Py_INIT_OK()`` * ``Py_INIT_ERR(MSG)`` * ``Py_INIT_NO_MEMORY()`` * ``Py_INIT_EXIT(EXITCODE)`` * ``Py_INIT_IS_ERROR(err)`` * ``Py_INIT_IS_EXIT(err)`` * ``Py_INIT_FAILED(err)`` This PEP also adds ``_PyRuntimeState.preconfig`` (``PyPreConfig`` type) and ``PyInterpreterState.config`` (``PyConfig`` type) fields to these internal structures. ``PyInterpreterState.config`` becomes the new reference configuration, replacing global configuration variables and other private variables. PyWideStringList ---------------- ``PyWideStringList`` is a list of ``wchar_t*`` strings. Example to initialize a string from C static array:: static wchar_t* argv[2] = { L"-c", L"pass", }; PyWideStringList config_argv = PyWideStringList_INIT; config_argv.length = Py_ARRAY_LENGTH(argv); config_argv.items = argv; ``PyWideStringList`` structure fields: * ``length`` (``Py_ssize_t``) * ``items`` (``wchar_t**``) Methods: * ``PyInitError PyWideStringList_Append(PyWideStringList *list, const wchar_t *item)``: Append *item* to *list*. If *length* is non-zero, *items* must be non-NULL and all strings must be non-NULL. PyInitError ----------- ``PyInitError`` is a structure to store an error message or an exit code for the Python Initialization. For an error, it stores the C function name which created the error. Example:: PyInitError alloc(void **ptr, size_t size) { *ptr = PyMem_RawMalloc(size); if (*ptr == NULL) { return Py_INIT_NO_MEMORY(); } return Py_INIT_OK(); } int main(int argc, char **argv) { void *ptr; PyInitError err = alloc(&ptr, 16); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } PyMem_Free(ptr); return 0; } ``PyInitError`` fields: * ``exitcode`` (``int``): argument passed to ``exit()`` on Unix and to ``ExitProcess()`` on Windows. Only set by ``Py_INIT_EXIT()``. * ``err_msg`` (``const char*``): error message * private ``_func`` field: used by ``Py_INIT_ERR()`` to store the C function name which created the error. * private ``_type`` field: for internal usage only. Macro to create an error: * ``Py_INIT_OK()``: success * ``Py_INIT_ERR(err_msg)``: initialization error with a message * ``Py_INIT_NO_MEMORY()``: memory allocation failure (out of memory) * ``Py_INIT_EXIT(exitcode)``: exit Python with the specified exit code Other macros and functions: * ``Py_INIT_IS_ERROR(err)``: Is the result an error? * ``Py_INIT_IS_EXIT(err)``: Is the result an exit? * ``Py_INIT_FAILED(err)``: Is the result an error or an exit? Similar to ``Py_INIT_IS_ERROR(err) || Py_INIT_IS_EXIT(err)``. * ``Py_ExitInitError(err)``: Call ``exit(exitcode)`` on Unix or ``ExitProcess(exitcode)`` if the result is an exit, call ``Py_FatalError(err_msg)`` if the result is an error. Must not be called if the result is a success. Pre-Initialization with PyPreConfig ----------------------------------- ``PyPreConfig`` structure is used to pre-initialize Python: * Set the memory allocator * Configure the LC_CTYPE locale * Set the UTF-8 mode Example using the pre-initialization to enable the UTF-8 Mode:: PyPreConfig preconfig = PyPreConfig_INIT; preconfig.utf8_mode = 1; PyInitError err = Py_PreInitialize(&preconfig); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } /* at this point, Python will speak UTF-8 */ Py_Initialize(); /* ... use Python API here ... */ Py_Finalize(); Functions to pre-initialize Python: * ``PyInitError Py_PreInitialize(const PyPreConfig *config)`` * ``PyInitError Py_PreInitializeFromArgs(const PyPreConfig *config, int argc, char **argv)`` * ``PyInitError Py_PreInitializeFromWideArgs(const PyPreConfig *config, int argc, wchar_t **argv)`` If Python should be pre-initialized explicitly first and then initialized with command line arguments, it is possible to pass these command line arguments to the pre-initialization since they impact the encodings. For example, ``-X utf8`` enables the UTF-8 Mode. These functions can be called with *config* set to ``NULL``. The caller is responsible to handle error using ``Py_INIT_FAILED()`` and ``Py_ExitInitError()``. ``PyPreConfig`` fields: * ``allocator`` (``char*``): name of the memory allocator (ex: ``"malloc"``) * ``coerce_c_locale_warn`` (``int``): if non-zero, emit a warning if the C locale is coerced. * ``coerce_c_locale`` (``int``): if equals to 2, coerce the C locale; if equals to 1, read the LC_CTYPE to decide if it should be coerced. * ``dev_mode`` (``int``): see ``PyConfig.dev_mode`` * ``isolated`` (``int``): see ``PyConfig.isolated`` * ``legacy_windows_fs_encoding`` (``int``, Windows only): if non-zero, set the Python filesystem encoding to ``"mbcs"``. * ``use_environment`` (``int``): see ``PyConfig.use_environment`` * ``utf8_mode`` (``int``): if non-zero, enable the UTF-8 mode There is also a private field which is for internal-usage only: * ``_config_version`` (``int``): Configuration version, used for ABI compatibility The C locale coercion (PEP 538) and the UTF-8 Mode (PEP 540) are disabled by default in ``PyPreConfig``. Set ``coerce_c_locale``, ``coerce_c_locale_warn`` and ``utf8_mode`` to ``-1`` to let Python enable them depending on the user configuration. Initialization with PyConfig ---------------------------- The ``PyConfig`` structure contains all parameters to configure Python. Example:: PyInitError err; PyConfig config = PyConfig_INIT; err = PyConfig_SetString(&config.program_name, L"my_program"); if (_Py_INIT_FAILED(err)) { Py_ExitInitError(err); } err = Py_InitializeFromConfig(&config); PyConfig_Clear(&config); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } ``PyConfig`` methods: * ``PyInitError PyConfig_SetString(wchar_t **config_str, const wchar_t *str)``: Set a config wide string field from *str* (copy the string) * ``PyInitError PyConfig_DecodeLocale(wchar_t **config_str, const char *str)``: Decode *str* using ``Py_DecodeLocale()`` and set the result into ``*config_str``. Pre-initialize Python if needed to ensure that encodings are properly configured. * ``PyInitError PyConfig_SetArgv(PyConfig *config, int argc, char **argv)``: Set command line arguments (decode bytes). Pre-initialize Python if needed to ensure that encodings are properly configured. * ``PyInitError PyConfig_SetWideArgv(PyConfig *config, int argc, wchar_t **argv)``: Set command line arguments (wide characters). * ``PyInitError PyConfig_Read(PyConfig *config)``: Read all Python configuration * ``void PyConfig_Clear(PyConfig *config)``: Release memory Functions to initialize Python: * ``PyInitError Py_InitializeFromConfig(const PyConfig *config)`` These functions can be called with *config* set to ``NULL``. The caller is responsible to handler error using ``Py_INIT_FAILED()`` and ``Py_ExitInitError()``. PyConfig fields: * ``argv`` (``PyWideStringList``): ``sys.argv`` * ``base_exec_prefix`` (``wchar_t*``): ``sys.base_exec_prefix`` * ``base_prefix`` (``wchar_t*``): ``sys.base_prefix`` * ``buffered_stdio`` (``int``): if equals to 0, enable unbuffered mode, make stdout and stderr streams to be unbuffered. * ``bytes_warning`` (``int``): if equals to 1, issue a warning when comparing ``bytes`` or ``bytearray`` with ``str``, or comparing ``bytes`` with ``int``. If equal or greater to 2, raise a ``BytesWarning`` exception. * ``check_hash_pycs_mode`` (``wchar_t*``): ``--check-hash-based-pycs`` command line option value (see PEP 552) * ``dev_mode`` (``int``): Development mode * ``dll_path`` (``wchar_t*``, Windows only): Windows DLL path * ``dump_refs`` (``int``): if non-zero, display all objects still alive at exit * ``exec_prefix`` (``wchar_t*``): ``sys.exec_prefix`` * ``executable`` (``wchar_t*``): ``sys.executable`` * ``faulthandler`` (``int``): if non-zero, call ``faulthandler.enable()`` * ``filesystem_encoding`` (``wchar_t*``): Filesystem encoding, ``sys.getfilesystemencoding()`` * ``filesystem_errors`` (``wchar_t*``): Filesystem encoding errors, ``sys.getfilesystemencodeerrors()`` * ``use_hash_seed`` (``int``), ``hash_seed`` (``unsigned long``): randomized hash function seed * ``home`` (``wchar_t*``): Python home * ``import_time`` (``int``): if non-zero, profile import time * ``inspect`` (``int``): enter interactive mode after executing a script or a command * ``install_signal_handlers`` (``int``): install signal handlers? * ``interactive`` (``int``): interactive mode * ``legacy_windows_stdio`` (``int``, Windows only): if non-zero, use ``io.FileIO`` instead of ``WindowsConsoleIO`` for ``sys.stdin``, ``sys.stdout`` and ``sys.stderr``. * ``malloc_stats`` (``int``): if non-zero, dump memory allocation statistics at exit * ``module_search_path_env`` (``wchar_t*``): ``PYTHONPATH`` environment variale value * ``use_module_search_paths`` (``int``), ``module_search_paths`` (``PyWideStringList``): ``sys.path`` * ``optimization_level`` (``int``): compilation optimization level * ``parser_debug`` (``int``): if non-zero, turn on parser debugging output (for expert only, depending on compilation options). * ``prefix`` (``wchar_t*``): ``sys.prefix`` * ``program_name`` (``wchar_t*``): Program name * ``program`` (``wchar_t*``): ``argv[0]`` or an empty string * ``pycache_prefix`` (``wchar_t*``): ``.pyc`` cache prefix * ``quiet`` (``int``): quiet mode (ex: don't display the copyright and version messages even in interactive mode) * ``run_command`` (``wchar_t*``): ``-c COMMAND`` argument * ``run_filename`` (``wchar_t*``): ``python3 SCRIPT`` argument * ``run_module`` (``wchar_t*``): ``python3 -m MODULE`` argument * ``show_alloc_count`` (``int``): show allocation counts at exit? * ``show_ref_count`` (``int``): show total reference count at exit? * ``site_import`` (``int``): import the ``site`` module at startup? * ``skip_source_first_line`` (``int``): skip the first line of the source * ``stdio_encoding`` (``wchar_t*``), ``stdio_errors`` (``wchar_t*``): encoding and encoding errors of ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` * ``tracemalloc`` (``int``): if non-zero, call ``tracemalloc.start(value)`` * ``user_site_directory`` (``int``): if non-zero, add user site directory to ``sys.path`` * ``verbose`` (``int``): if non-zero, enable verbose mode * ``warnoptions`` (``PyWideStringList``): options of the ``warnings`` module to build filters * ``write_bytecode`` (``int``): if non-zero, write ``.pyc`` files * ``xoptions`` (``PyWideStringList``): ``sys._xoptions`` There are also private fields which are for internal-usage only: * ``_config_version`` (``int``): Configuration version, used for ABI compatibility * ``_frozen`` (``int``): Emit warning when computing the path configuration? * ``_install_importlib`` (``int``): Install importlib? More complete commented example modifying the configuration before calling ``PyConfig_Read()`` and then modify the read configuration:: PyInitError init_python(const char *program_name) { PyInitError err; PyConfig config = PyConfig_INIT; /* Set the program name before reading the configuraton (decode byte string from the locale encoding) */ err = PyConfig_DecodeLocale(&config.program_name, program_name); if (_Py_INIT_FAILED(err)) { goto fail; } /* Read all configuration at once */ err = PyConfig_Read(&config); if (_Py_INIT_FAILED(err)) { goto fail; } /* Append our custom search path to sys.path */ err = PyWideStringList_Append(&config.module_search_paths, L"/path/to/more/modules"); if (_Py_INIT_FAILED(err)) { goto fail; } /* Override executable computed by PyConfig_Read() */ err = PyConfig_SetString(&config.executable, L"my_executable"); if (_Py_INIT_FAILED(err)) { goto fail; } err = Py_InitializeFromConfig(&config); /* Py_InitializeFromConfig() copied config which must now be cleared to release memory */ PyConfig_Clear(&config); return err; fail: PyConfig_Clear(&config); Py_ExitInitError(err); } .. note:: ``PyConfig`` does not have any field for extra inittab functions: ``PyImport_AppendInittab()`` and ``PyImport_ExtendInittab()`` functions are still relevant. Initialization with static PyConfig ----------------------------------- When no ``PyConfig`` method is used but only ``Py_InitializeFromConfig()``, the caller is responsible for managing ``PyConfig`` memory which means that static strings and static string lists can be used rather than using dynamically allocated memory. It can be used for most simple configurations. Example of Python initialization enabling the isolated mode:: PyConfig config = PyConfig_INIT; config.isolated = 1; PyInitError err = Py_InitializeFromConfig(&config); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } /* ... use Python API here ... */ Py_Finalize(); In this example, ``PyConfig_Clear()`` is not needed since ``config`` does not contain any dynamically allocated string: ``Py_InitializeFromConfig`` is responsible for filling other fields and manage the memory. For convenience, two other functions are provided: * ``PyInitError Py_InitializeFromArgs(const PyConfig *config, int argc, char **argv)`` * ``PyInitError Py_InitializeFromWideArgs(const PyConfig *config, int argc, wchar_t **argv)`` These functions can be used with static ``PyConfig``. Pseudo-code of ``Py_InitializeFromArgs()``:: PyInitError init_with_args(const PyConfig *src_config, int argc, char **argv) { PyInitError err; PyConfig config = PyConfig_INIT; /* Copy strings and string lists * (memory dynamically allocated on the heap) */ err = _PyConfig_Copy(&config, src_config); if (Py_INIT_FAILED(err)) { goto exit; } /* Set config.argv: decode argv bytes. Pre-initialize Python if needed to ensure that the encodings are properly configured. */ err = PyConfig_SetArgv(&config, argc, argv); if (Py_INIT_FAILED(err)) { goto exit; } err = Py_InitializeFromConfig(&config); exit: PyConfig_Clear(&config); return err; } where ``_PyConfig_Copy()`` is an internal function. The actual implementation of ``Py_InitializeFromArgs()`` is more complex. Py_UnixMain() ------------- Python 3.7 provides a high-level ``Py_Main()`` function which requires to pass command line arguments as ``wchar_t*`` strings. It is non-trivial to use the correct encoding to decode bytes. Python has its own set of issues with C locale coercion and UTF-8 Mode. This PEP adds a new ``Py_UnixMain()`` function which takes command line arguments as bytes:: int Py_UnixMain(int argc, char **argv) Py_RunMain() ------------ The new ``Py_RunMain()`` function executes the command (``PyConfig.run_command``), the script (``PyConfig.run_filename``) or the module (``PyConfig.run_module``) specified on the command line or in the configuration, and then finalizes Python. It returns an exit status that can be passed to the ``exit()`` function. Example of custom Python executable always running in isolated mode:: #include <Python.h> int main(int argc, char *argv[]) { PyConfig config = PyConfig_INIT; config.isolated = 1; PyInitError err = Py_InitializeFromArgs(&config, argc, argv); if (Py_INIT_FAILED(err)) { Py_ExitInitError(err); } /* put more configuration code here if needed */ return Py_RunMain(); } The example is a basic implementation of the "System Python Executable" discussed in PEP 432. Memory allocations and Py_DecodeLocale() ---------------------------------------- Python memory allocation functions like ``PyMem_RawMalloc()`` must not be used before Python pre-initialization. Calling directly ``malloc()`` and ``free()`` is always safe. For ``PyPreConfig`` and static ``PyConfig``, the caller is responsible to manage dynamically allocated strings, but static strings and static string lists are fine. Dynamic ``PyConfig`` requires to call ``PyConfig_Clear()`` to release memory. ``Py_DecodeLocale()`` must not be called before the pre-initialization. When using dynanic configuration, ``PyConfig_DecodeLocale()`` must be used instead of ``Py_DecodeLocale()``. Backwards Compatibility ======================= This PEP only adds a new API: it leaves the existing API unchanged and has no impact on the backwards compatibility. Annex: Python Configuration =========================== Priority and Rules ------------------ Priority of configuration parameters, highest to lowest: * ``PyConfig`` * ``PyPreConfig`` * Configuration files * Command line options * Environment variables * Global configuration variables Priority of warning options, highest to lowest: * ``PyConfig.warnoptions`` * ``PyConfig.dev_mode`` (add ``"default"``) * ``PYTHONWARNINGS`` environment variables * ``-W WARNOPTION`` command line argument * ``PyConfig.bytes_warning`` (add ``"error::BytesWarning"`` if greater than 1, or add ``"default::BytesWarning``) Rules on ``PyConfig`` and ``PyPreConfig`` parameters: * If ``isolated`` is non-zero, ``use_environment`` and ``user_site_directory`` are set to 0 * If ``legacy_windows_fs_encoding`` is non-zero, ``utf8_mode`` is set to 0 * If ``dev_mode`` is non-zero, ``allocator`` is set to ``"debug"``, ``faulthandler`` is set to 1, and ``"default"`` filter is added to ``warnoptions``. But ``PYTHONMALLOC`` has the priority over ``dev_mode`` to set the memory allocator. Configuration Files ------------------- Python configuration files: * ``pyvenv.cfg`` * ``python._pth`` (Windows only) * ``pybuilddir.txt`` (Unix only) Global Configuration Variables ------------------------------ Global configuration variables mapped to ``PyPreConfig`` fields: ======================================== ================================ Variable Field ======================================== ================================ ``Py_LegacyWindowsFSEncodingFlag`` ``legacy_windows_fs_encoding`` ``Py_LegacyWindowsFSEncodingFlag`` ``legacy_windows_fs_encoding`` ``Py_UTF8Mode`` ``utf8_mode`` ``Py_UTF8Mode`` ``utf8_mode`` ======================================== ================================ Global configuration variables mapped to ``PyConfig`` fields: ======================================== ================================ Variable Field ======================================== ================================ ``Py_BytesWarningFlag`` ``bytes_warning`` ``Py_DebugFlag`` ``parser_debug`` ``Py_DontWriteBytecodeFlag`` ``write_bytecode`` ``Py_FileSystemDefaultEncodeErrors`` ``filesystem_errors`` ``Py_FileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_FrozenFlag`` ``_frozen`` ``Py_HasFileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_HashRandomizationFlag`` ``use_hash_seed``, ``hash_seed`` ``Py_IgnoreEnvironmentFlag`` ``use_environment`` ``Py_InspectFlag`` ``inspect`` ``Py_InteractiveFlag`` ``interactive`` ``Py_IsolatedFlag`` ``isolated`` ``Py_LegacyWindowsStdioFlag`` ``legacy_windows_stdio`` ``Py_NoSiteFlag`` ``site_import`` ``Py_NoUserSiteDirectory`` ``user_site_directory`` ``Py_OptimizeFlag`` ``optimization_level`` ``Py_QuietFlag`` ``quiet`` ``Py_UnbufferedStdioFlag`` ``buffered_stdio`` ``Py_VerboseFlag`` ``verbose`` ``_Py_HasFileSystemDefaultEncodeErrors`` ``filesystem_errors`` ``Py_BytesWarningFlag`` ``bytes_warning`` ``Py_DebugFlag`` ``parser_debug`` ``Py_DontWriteBytecodeFlag`` ``write_bytecode`` ``Py_FileSystemDefaultEncodeErrors`` ``filesystem_errors`` ``Py_FileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_FrozenFlag`` ``_frozen`` ``Py_HasFileSystemDefaultEncoding`` ``filesystem_encoding`` ``Py_HashRandomizationFlag`` ``use_hash_seed``, ``hash_seed`` ``Py_IgnoreEnvironmentFlag`` ``use_environment`` ``Py_InspectFlag`` ``inspect`` ``Py_InteractiveFlag`` ``interactive`` ``Py_IsolatedFlag`` ``isolated`` ``Py_LegacyWindowsStdioFlag`` ``legacy_windows_stdio`` ``Py_NoSiteFlag`` ``site_import`` ``Py_NoUserSiteDirectory`` ``user_site_directory`` ``Py_OptimizeFlag`` ``optimization_level`` ``Py_QuietFlag`` ``quiet`` ``Py_UnbufferedStdioFlag`` ``buffered_stdio`` ``Py_VerboseFlag`` ``verbose`` ``_Py_HasFileSystemDefaultEncodeErrors`` ``filesystem_errors`` ======================================== ================================ ``Py_LegacyWindowsFSEncodingFlag`` and ``Py_LegacyWindowsStdioFlag`` are only available on Windows. Command Line Arguments ---------------------- Usage:: python3 [options] python3 [options] -c COMMAND python3 [options] -m MODULE python3 [options] SCRIPT Command line options mapped to pseudo-action on ``PyConfig`` fields: ================================ ================================ Option ``PyPreConfig`` field ================================ ================================ ``-X dev`` ``dev_mode = 1`` ``-X utf8=N`` ``utf8_mode = N`` ================================ ================================ Command line options mapped to pseudo-action on ``PyConfig`` fields: ================================ ================================ Option ``PyConfig`` field ================================ ================================ ``-b`` ``bytes_warning++`` ``-B`` ``write_bytecode = 0`` ``-c COMMAND`` ``run_module = COMMAND`` ``--check-hash-based-pycs=MODE`` ``_check_hash_pycs_mode = MODE`` ``-d`` ``parser_debug++`` ``-E`` ``use_environment = 0`` ``-i`` ``inspect++`` and ``interactive++`` ``-I`` ``isolated = 1`` ``-m MODULE`` ``run_module = MODULE`` ``-O`` ``optimization_level++`` ``-q`` ``quiet++`` ``-R`` ``use_hash_seed = 0`` ``-s`` ``user_site_directory = 0`` ``-S`` ``site_import`` ``-t`` ignored (kept for backwards compatibility) ``-u`` ``buffered_stdio = 0`` ``-v`` ``verbose++`` ``-W WARNING`` add ``WARNING`` to ``warnoptions`` ``-x`` ``skip_source_first_line = 1`` ``-X XOPTION`` add ``XOPTION`` to ``xoptions`` ``-X dev`` ``dev_mode = 1`` ``-X faulthandler`` ``faulthandler = 1`` ``-X importtime`` ``import_time = 1`` ``-X pycache_prefix=PREFIX`` ``pycache_prefix = PREFIX`` ``-X show_alloc_count`` ``show_alloc_count = 1`` ``-X show_ref_count`` ``show_ref_count = 1`` ``-X tracemalloc=N`` ``tracemalloc = N`` ================================ ================================ ``-h``, ``-?`` and ``-V`` options are handled outside ``PyConfig``. Environment Variables --------------------- Environment variables mapped to ``PyPreConfig`` fields: ================================= ============================================= Variable ``PyPreConfig`` field ================================= ============================================= ``PYTHONCOERCECLOCALE`` ``coerce_c_locale``, ``coerce_c_locale_warn`` ``PYTHONDEVMODE`` ``dev_mode`` ``PYTHONLEGACYWINDOWSFSENCODING`` ``legacy_windows_fs_encoding`` ``PYTHONMALLOC`` ``allocator`` ``PYTHONUTF8`` ``utf8_mode`` ================================= ============================================= Environment variables mapped to ``PyConfig`` fields: ================================= ==================================== Variable ``PyConfig`` field ================================= ==================================== ``PYTHONDEBUG`` ``parser_debug`` ``PYTHONDEVMODE`` ``dev_mode`` ``PYTHONDONTWRITEBYTECODE`` ``write_bytecode`` ``PYTHONDUMPREFS`` ``dump_refs`` ``PYTHONEXECUTABLE`` ``program_name`` ``PYTHONFAULTHANDLER`` ``faulthandler`` ``PYTHONHASHSEED`` ``use_hash_seed``, ``hash_seed`` ``PYTHONHOME`` ``home`` ``PYTHONINSPECT`` ``inspect`` ``PYTHONIOENCODING`` ``stdio_encoding``, ``stdio_errors`` ``PYTHONLEGACYWINDOWSSTDIO`` ``legacy_windows_stdio`` ``PYTHONMALLOCSTATS`` ``malloc_stats`` ``PYTHONNOUSERSITE`` ``user_site_directory`` ``PYTHONOPTIMIZE`` ``optimization_level`` ``PYTHONPATH`` ``module_search_path_env`` ``PYTHONPROFILEIMPORTTIME`` ``import_time`` ``PYTHONPYCACHEPREFIX,`` ``pycache_prefix`` ``PYTHONTRACEMALLOC`` ``tracemalloc`` ``PYTHONUNBUFFERED`` ``buffered_stdio`` ``PYTHONVERBOSE`` ``verbose`` ``PYTHONWARNINGS`` ``warnoptions`` ================================= ==================================== ``PYTHONLEGACYWINDOWSFSENCODING`` and ``PYTHONLEGACYWINDOWSSTDIO`` are specific to Windows. ``PYTHONDEVMODE`` is mapped to ``PyPreConfig.dev_mode`` and ``PyConfig.dev_mode``. Annex: Python 3.7 API ===================== Python 3.7 has 4 functions in its C API to initialize and finalize Python: * ``Py_Initialize()``, ``Py_InitializeEx()``: initialize Python * ``Py_Finalize()``, ``Py_FinalizeEx()``: finalize Python Python can be configured using scattered global configuration variables (like ``Py_IgnoreEnvironmentFlag``) and using the following functions: * ``PyImport_AppendInittab()`` * ``PyImport_ExtendInittab()`` * ``PyMem_SetAllocator()`` * ``PyMem_SetupDebugHooks()`` * ``PyObject_SetArenaAllocator()`` * ``Py_SetPath()`` * ``Py_SetProgramName()`` * ``Py_SetPythonHome()`` * ``Py_SetStandardStreamEncoding()`` * ``PySys_AddWarnOption()`` * ``PySys_AddXOption()`` * ``PySys_ResetWarnOptions()`` There is also a high-level ``Py_Main()`` function. Copyright ========= This document has been placed in the public domain.

5 10

Online Devguide mostly not working
by Terry Reedy 14 May '19

14 May '19

https://devguide.python.org gives the intro page with TOC on sidebar and at end. Clicking anything, such as Getting Started, which tries to display https://devguide.python.org/setup/, returns a Read the Docs page "Sorry This page does not exist yet." 'Down for everyone' site also cannot access. -- Terry Jan Reedy

5 5

Adding a tzidx cache to datetime
by Paul Ganssle 14 May '19

14 May '19

Greetings all, I have one last feature request that I'd like added to datetime for Python 3.8, and this one I think could use some more discussion, the addition of a "time zone index cache" to the /datetime/ object. The rationale is laid out in detail in bpo-35723 <https://bugs.python.org/issue35723>. The general problem is that currently, /every/ invocation of utcoffset, tzname and dst needs to do full, independent calculations of the time zone offsets, even for time zones where the mapping is guaranteed to be stable because datetimes are immutable. I have a proof of concept implementation: PR #11529 <https://github.com/python/cpython/pull/11529/>. I'm envisioning that the `datetime` class will add a private `_tzidx` single-byte member (it seems that this does not increase the size of the datetime object, because it's just using an unused alignment byte). `datetime` will also add a `tzidx()` method, which will return `_tzidx` if it's been set and otherwise it will call `self.tzinfo.tzidx()`. If `self.tzinfo.tzidx()` returns a number between 0 and 254 (inclusive), it sets `_tzidx` to this value. tzidx() then returns whatever self.tzinfo.tzidx() returned. The value of this is that as far as I can tell, nearly all non-trivial tzinfo implementations construct a list of possible offsets, and implement utcoffset(), tzname() and dst() by calculating an index into that list and returning it. There are almost always less than 255 distinct offsets. By adding this cache /on the datetime/, we're using a small amount of currently-unused memory to prevent unnecessary calculations about a given datetime. The feature is entirely opt-in, and has no downsides if it goes unused, and it makes it possible to write tzinfo implementations that are both lazy and as fast as the "eager calculation" mode that pytz uses (and that causes many problems for pytz's users). I have explored the idea of using an lru cache of some sort on the tzinfo object itself, but there are two problems with this: 1. Calculating the hash of a datetime calls .utcoffset(), which means that it is necessary to, at minimum, do a `replace` on the datetime (and constructing a new datetime is a pretty considerable speed hit) 2. It will be a much bigger memory cost, since my current proposal uses approximately zero additional memory (not sure if the alignment stuff is platform-dependent or something, but it doesn't use additional memory on my linux computer). I realize this proposal is somewhat difficult to wrap your head around, so if anyone would like to chat with me about it in person, I'll be at PyCon sprints until Thursday morning. Best, Paul

3 5

Online Devguide Status: Working Again
by Carol Willing 12 May '19

12 May '19

Sorry for the inconvenience yesterday.

1 0

Ignore spurious 3.4 checkin messages!
by Ned Deily 11 May '19

11 May '19

Eek! I was just doing a bit of branch cleanup in the cpython repo and managed to trigger a bunch (30-ish) of duplicate checkin messages to bugs.python.org for old commits. I will remove them from b.p.o. Please ignore any 3.4 spam email. Sorry! -- Ned Deily nad(a)python.org -- []

2 1

Summary of Python tracker Issues
by Python tracker 10 May '19

10 May '19

ACTIVITY SUMMARY (2019-05-03 - 2019-05-10) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7084 ( +3) closed 41542 (+88) total 48626 (+91) Open issues with patches: 2851 Issues opened (65) ================== #15987: Provide a way to compare AST nodes for equality recursively https://bugs.python.org/issue15987 reopened by vstinner #36785: Implement PEP 574 https://bugs.python.org/issue36785 opened by pitrou #36786: "make install" should run compileall in parallel https://bugs.python.org/issue36786 opened by pitrou #36788: Add clamp() function to builtins https://bugs.python.org/issue36788 opened by TheComet #36789: Unicode HOWTO incorrectly states that UTF-8 contains no zero b https://bugs.python.org/issue36789 opened by mbiggs #36790: test_asyncio fails with application verifier! https://bugs.python.org/issue36790 opened by Alexander Riccio #36791: sum() relies on C signed overflow behaviour https://bugs.python.org/issue36791 opened by serhiy.storchaka #36792: [Windows] time: crash on formatting time with de_DE locale https://bugs.python.org/issue36792 opened by CharlieClark #36793: Do not define unneeded __str__ equal to __repr__ https://bugs.python.org/issue36793 opened by serhiy.storchaka #36794: asyncio.Lock documentation in Py3.8 lacks parts presented in d https://bugs.python.org/issue36794 opened by germn #36795: "make venv" failed in Docs https://bugs.python.org/issue36795 opened by pitrou #36796: Error handling cleanup in _testcapimodule.c https://bugs.python.org/issue36796 opened by ZackerySpytz #36797: Cull more oudated distutils information https://bugs.python.org/issue36797 opened by ncoghlan #36799: Typo in ctypes documentation https://bugs.python.org/issue36799 opened by rrt #36803: Getting a lot of runtime misaligned address error while buildi https://bugs.python.org/issue36803 opened by dibya ranjan mishra #36804: Python for Windows installer Repair option does not repair PIP https://bugs.python.org/issue36804 opened by Mark Green #36805: Don't close subprocess stream if it's stdin is closed https://bugs.python.org/issue36805 opened by asvetlov #36807: IDLE doesn't call os.fsync() https://bugs.python.org/issue36807 opened by gvanrossum #36809: Crash for test test_importlib https://bugs.python.org/issue36809 opened by furzoom #36810: Recursive type annotations do not work in documentation tests https://bugs.python.org/issue36810 opened by lovasoa #36813: QueueListener not calling task_done upon termination https://bugs.python.org/issue36813 opened by bar.harel #36816: self-signed.pythontest.net TLS certificate key is too weak https://bugs.python.org/issue36816 opened by gregory.p.smith #36818: Add PyInterpreterState.runtime. https://bugs.python.org/issue36818 opened by eric.snow #36819: Crash during encoding using UTF-16/32 and custom error handler https://bugs.python.org/issue36819 opened by atalaba #36820: Captured exceptions are keeping user objects alive unnecessari https://bugs.python.org/issue36820 opened by mariocj89 #36821: Termios module largely untested https://bugs.python.org/issue36821 opened by anthonypjshaw #36822: Minor grammatical fix in glossary.rst https://bugs.python.org/issue36822 opened by CuriousLearner #36823: shutil.copytree copies directories and files but fails with th https://bugs.python.org/issue36823 opened by rbavery #36824: Refactor str tests to reflect that str and unicode are merged https://bugs.python.org/issue36824 opened by dfortunov #36825: Make TestCase aware of the command line arguments given to Tes https://bugs.python.org/issue36825 opened by remi.lapeyre #36826: ast_unparser.c doesn't handle := expressions https://bugs.python.org/issue36826 opened by eric.smith #36827: Overriding __new__ method with itself changes behaviour of the https://bugs.python.org/issue36827 opened by alexey-muranov #36829: CLI option to make PyErr_WriteUnraisable abort the current pro https://bugs.python.org/issue36829 opened by graingert #36833: Add tests for Datetime C API Macros https://bugs.python.org/issue36833 opened by p-ganssle #36834: mock.patch.object does not persist __module__ name for functio https://bugs.python.org/issue36834 opened by spark #36837: Make il8n tools available from `python -m` https://bugs.python.org/issue36837 opened by bbkane #36839: Support the buffer protocol in code objects https://bugs.python.org/issue36839 opened by dino.viehland #36840: Add stream.abort() async method https://bugs.python.org/issue36840 opened by asvetlov #36841: Supporting customization of float encoding in JSON https://bugs.python.org/issue36841 opened by mitar #36842: Implement PEP 578 https://bugs.python.org/issue36842 opened by steve.dower #36843: AIX build fails with failure to get random numbers https://bugs.python.org/issue36843 opened by Robert Boehne #36845: ipaddres.IPv4Network and ipaddress.IPv6Network tuple construct https://bugs.python.org/issue36845 opened by niconorsk #36848: autospec fails with AttributeError when mocked class has __sig https://bugs.python.org/issue36848 opened by xtreak #36849: Native libcurses on HP-UX not properly detected https://bugs.python.org/issue36849 opened by michael-o #36850: shutil.copy2 fails with even with source network filesystem no https://bugs.python.org/issue36850 opened by hans-meine #36852: Python3.7.2 fails to cross-compile (yocto / openembedded) when https://bugs.python.org/issue36852 opened by mschoepf #36853: inconsistencies in docs builds (Sphinx 2) https://bugs.python.org/issue36853 opened by jaraco #36854: GC operates out of global runtime state. https://bugs.python.org/issue36854 opened by eric.snow #36855: update tests relying on a copy of a TLS certificate (pem) to s https://bugs.python.org/issue36855 opened by gregory.p.smith #36856: faulthandler._stack_overflow doesn't work on x86-linux with KP https://bugs.python.org/issue36856 opened by xry111 #36858: f-string '=' debugging output needs to be documented https://bugs.python.org/issue36858 opened by eric.smith #36859: sqlite3 dml statement detection does not account for CTEs https://bugs.python.org/issue36859 opened by coleifer #36860: Inconsistent/Undocumented behavior with pathlib resolve and ab https://bugs.python.org/issue36860 opened by alexjacobson95 #36862: Add Visual Studio 2019 support to PCBuild/env.bat https://bugs.python.org/issue36862 opened by anthonypjshaw #36863: argparse doesn't like options in the middle of arguments https://bugs.python.org/issue36863 opened by rhubarbdog x #36865: FileInput does not allow 'rt' mode, but all its existing deleg https://bugs.python.org/issue36865 opened by natgaertner #36866: Certificate verification errors in urllib.request become URLEr https://bugs.python.org/issue36866 opened by sivel #36867: Make semaphore_tracker track other system resources https://bugs.python.org/issue36867 opened by pierreglaser #36868: New behavior of OpenSSL hostname verification not exposed, inc https://bugs.python.org/issue36868 opened by josh.r #36869: Avoid warning of unused variables https://bugs.python.org/issue36869 opened by eamanu #36870: test_asyncio: test_drain_raises() fails randomly on Windows https://bugs.python.org/issue36870 opened by vstinner #36871: Misleading error from unittest.mock's assert_has_calls https://bugs.python.org/issue36871 opened by gregory.p.smith #36873: http.server: Document explicitly that symbolic links are follo https://bugs.python.org/issue36873 opened by vstinner #36874: Support CDATA by xml.etree.(c)ElementTree https://bugs.python.org/issue36874 opened by Pierre van de Laar #36875: argparse does not ship with translations https://bugs.python.org/issue36875 opened by carmenbianca Most recent 15 issues with no replies (15) ========================================== #36873: http.server: Document explicitly that symbolic links are follo https://bugs.python.org/issue36873 #36870: test_asyncio: test_drain_raises() fails randomly on Windows https://bugs.python.org/issue36870 #36868: New behavior of OpenSSL hostname verification not exposed, inc https://bugs.python.org/issue36868 #36867: Make semaphore_tracker track other system resources https://bugs.python.org/issue36867 #36866: Certificate verification errors in urllib.request become URLEr https://bugs.python.org/issue36866 #36865: FileInput does not allow 'rt' mode, but all its existing deleg https://bugs.python.org/issue36865 #36862: Add Visual Studio 2019 support to PCBuild/env.bat https://bugs.python.org/issue36862 #36855: update tests relying on a copy of a TLS certificate (pem) to s https://bugs.python.org/issue36855 #36854: GC operates out of global runtime state. https://bugs.python.org/issue36854 #36850: shutil.copy2 fails with even with source network filesystem no https://bugs.python.org/issue36850 #36849: Native libcurses on HP-UX not properly detected https://bugs.python.org/issue36849 #36842: Implement PEP 578 https://bugs.python.org/issue36842 #36839: Support the buffer protocol in code objects https://bugs.python.org/issue36839 #36833: Add tests for Datetime C API Macros https://bugs.python.org/issue36833 #36826: ast_unparser.c doesn't handle := expressions https://bugs.python.org/issue36826 Most recent 15 issues waiting for review (15) ============================================= #36867: Make semaphore_tracker track other system resources https://bugs.python.org/issue36867 #36865: FileInput does not allow 'rt' mode, but all its existing deleg https://bugs.python.org/issue36865 #36862: Add Visual Studio 2019 support to PCBuild/env.bat https://bugs.python.org/issue36862 #36859: sqlite3 dml statement detection does not account for CTEs https://bugs.python.org/issue36859 #36856: faulthandler._stack_overflow doesn't work on x86-linux with KP https://bugs.python.org/issue36856 #36854: GC operates out of global runtime state. https://bugs.python.org/issue36854 #36852: Python3.7.2 fails to cross-compile (yocto / openembedded) when https://bugs.python.org/issue36852 #36850: shutil.copy2 fails with even with source network filesystem no https://bugs.python.org/issue36850 #36849: Native libcurses on HP-UX not properly detected https://bugs.python.org/issue36849 #36842: Implement PEP 578 https://bugs.python.org/issue36842 #36841: Supporting customization of float encoding in JSON https://bugs.python.org/issue36841 #36839: Support the buffer protocol in code objects https://bugs.python.org/issue36839 #36829: CLI option to make PyErr_WriteUnraisable abort the current pro https://bugs.python.org/issue36829 #36825: Make TestCase aware of the command line arguments given to Tes https://bugs.python.org/issue36825 #36824: Refactor str tests to reflect that str and unicode are merged https://bugs.python.org/issue36824 Top 10 most discussed issues (10) ================================= #36792: [Windows] time: crash on formatting time with de_DE locale https://bugs.python.org/issue36792 31 msgs #27639: UserList.__getitem__ doesn't account for slices https://bugs.python.org/issue27639 12 msgs #36829: CLI option to make PyErr_WriteUnraisable abort the current pro https://bugs.python.org/issue36829 11 msgs #36843: AIX build fails with failure to get random numbers https://bugs.python.org/issue36843 11 msgs #36807: IDLE doesn't call os.fsync() https://bugs.python.org/issue36807 9 msgs #36816: self-signed.pythontest.net TLS certificate key is too weak https://bugs.python.org/issue36816 8 msgs #35925: test_httplib test_nntplib test_ssl fail on ARMv7 Debian buster https://bugs.python.org/issue35925 7 msgs #36395: Add deferred single-threaded/fake executor to concurrent.futur https://bugs.python.org/issue36395 7 msgs #36848: autospec fails with AttributeError when mocked class has __sig https://bugs.python.org/issue36848 7 msgs #25541: Wrong usage of sockaddr_un struct for abstract namespace unix https://bugs.python.org/issue25541 6 msgs Issues closed (85) ================== #1054: scriptsinstall target fails in alternate build dir https://bugs.python.org/issue1054 closed by SilentGhost #6911: Document changes in asynchat https://bugs.python.org/issue6911 closed by CuriousLearner #11001: Various obvious errors in cookies documentation https://bugs.python.org/issue11001 closed by gregory.p.smith #14440: Close background process if IDLE closes abnormally. https://bugs.python.org/issue14440 closed by asvetlov #16024: Doc cleanup regarding path=fd, dir_fd, follow_symlinks, etc https://bugs.python.org/issue16024 closed by larry #20185: Derby #17: Convert 49 sites to Argument Clinic across 13 files https://bugs.python.org/issue20185 closed by mdk #20709: os.utime(path_to_directory): wrong documentation for Windows. https://bugs.python.org/issue20709 closed by mdk #24339: iso6937 encoding missing https://bugs.python.org/issue24339 closed by mdk #24712: Docs page's sidebar vibrates on mouse wheel scroll on Chrome. https://bugs.python.org/issue24712 closed by matrixise #25251: Unknown MS Compiler version 1900 https://bugs.python.org/issue25251 closed by anthonypjshaw #25436: argparse.ArgumentError missing error message in __repr__ https://bugs.python.org/issue25436 closed by dfortunov #26374: concurrent_futures Executor.map semantics better specified in https://bugs.python.org/issue26374 closed by bquinlan #26903: ProcessPoolExecutor(max_workers=64) crashes on Windows https://bugs.python.org/issue26903 closed by bquinlan #26978: Implement pathlib.Path.link (Using os.link) https://bugs.python.org/issue26978 closed by pitrou #27497: csv module: Add return value to DictWriter.writeheader https://bugs.python.org/issue27497 closed by vstinner #28238: In xml.etree.ElementTree findall() can't search all elements i https://bugs.python.org/issue28238 closed by scoder #28729: Exception from sqlite3 adapter masked by sqlite3.InterfaceErro https://bugs.python.org/issue28729 closed by mdk #28795: Misleading stating, that SIGINT handler is installed by defaul https://bugs.python.org/issue28795 closed by mdk #29075: Remove Windows Vista support https://bugs.python.org/issue29075 closed by vstinner #29164: make test always fail at 218/405 ( AssertionError: ',' not fou https://bugs.python.org/issue29164 closed by cheryl.sabella #30262: Don't expose sqlite3 Cache and Statement https://bugs.python.org/issue30262 closed by berker.peksag #30271: Make sqlite3 statement cache optional https://bugs.python.org/issue30271 closed by palaviv #30668: DOC: missing word in license.rst https://bugs.python.org/issue30668 closed by matrixise #30793: Parsing error on f-string-expressions containing strings with https://bugs.python.org/issue30793 closed by eric.smith #31200: address sanitizer build fails https://bugs.python.org/issue31200 closed by mdk #31589: Links for French documentation PDF is broken: LaTeX issue with https://bugs.python.org/issue31589 closed by mdk #31813: python -m ensurepip hangs https://bugs.python.org/issue31813 closed by anthonypjshaw #31855: mock_open is not compatible with read(n) (and pickle.load) https://bugs.python.org/issue31855 closed by xtreak #31873: Inconsistent capitalization of proper noun - Unicode. https://bugs.python.org/issue31873 closed by mdk #32393: nav menu jitter in old documentation https://bugs.python.org/issue32393 closed by mdk #32971: Docs on unittest.TestCase.assertRaises() should clarify contex https://bugs.python.org/issue32971 closed by anthonypjshaw #33039: int() and math.trunc don't accept objects that only define __i https://bugs.python.org/issue33039 closed by serhiy.storchaka #33530: Implement Happy Eyeball in asyncio https://bugs.python.org/issue33530 closed by asvetlov #33948: doc truncated lines in PDF https://bugs.python.org/issue33948 closed by mdk #34214: Pylint recusion stack overflow abort https://bugs.python.org/issue34214 closed by cheryl.sabella #34600: python3 regression ElementTree.iterparse() unable to capture c https://bugs.python.org/issue34600 closed by scoder #34820: binascii.c:1578:1: error: the control flow of function ‘bina https://bugs.python.org/issue34820 closed by anthonypjshaw #34848: range.index only takes one argument when it's documented as ta https://bugs.python.org/issue34848 closed by vstinner #35125: asyncio shield: remove inner callback on outer cancellation https://bugs.python.org/issue35125 closed by asvetlov #35354: Generator functions stack overflow https://bugs.python.org/issue35354 closed by Mark.Shannon #35393: Typo in documentation https://bugs.python.org/issue35393 closed by cheryl.sabella #35900: Add pickler hook for the user to customize the serialization o https://bugs.python.org/issue35900 closed by pitrou #36166: DOC: Fix markup on function parameter on datamodel.rst https://bugs.python.org/issue36166 closed by cheryl.sabella #36189: DOC: Correct word in tutorial introduction https://bugs.python.org/issue36189 closed by cheryl.sabella #36239: gettext: GNUTranslations doesn't parse properly comments in de https://bugs.python.org/issue36239 closed by mdk #36275: DOC: venv.create doesn't include prompt parameter https://bugs.python.org/issue36275 closed by cheryl.sabella #36533: logging regression with threading + fork are mixed in 3.7.1rc2 https://bugs.python.org/issue36533 closed by gregory.p.smith #36545: Python 3.5 OOM during test_socket on make https://bugs.python.org/issue36545 closed by dekken #36551: Optimize list comprehensions with preallocate size and protect https://bugs.python.org/issue36551 closed by anthonypjshaw #36552: Replace OverflowError with ValueError when calculating length https://bugs.python.org/issue36552 closed by serhiy.storchaka #36601: signals can be caught by any thread https://bugs.python.org/issue36601 closed by gregory.p.smith #36737: Warnings operate out of global runtime state. https://bugs.python.org/issue36737 closed by eric.snow #36766: Typos in docs and code comments https://bugs.python.org/issue36766 closed by terry.reedy #36774: f-strings: Add a !d conversion for ease of debugging https://bugs.python.org/issue36774 closed by eric.smith #36778: test_site.StartupImportTests.test_startup_imports fails if def https://bugs.python.org/issue36778 closed by vstinner #36783: No documentation for _FromXandFold C API functions https://bugs.python.org/issue36783 closed by cheryl.sabella #36784: __import__ with empty folder after importlib.invalidate_caches https://bugs.python.org/issue36784 closed by mdk #36787: Python3 regresison: String formatting of None object https://bugs.python.org/issue36787 closed by eric.smith #36798: f-strings do not support top-level := https://bugs.python.org/issue36798 closed by gvanrossum #36800: Invalid coding error hidden on Windows https://bugs.python.org/issue36800 closed by eryksun #36801: Wait for connection_lost in StreamWriter.drain https://bugs.python.org/issue36801 closed by asvetlov #36802: Revert back StreamWriter awrite/aclose but provide await write https://bugs.python.org/issue36802 closed by asvetlov #36806: Forbid creating of stream objects outside of asyncio https://bugs.python.org/issue36806 closed by asvetlov #36808: Understanding "cannot import name" exception https://bugs.python.org/issue36808 closed by eryksun #36811: Warning when compiling _elementree.c https://bugs.python.org/issue36811 closed by scoder #36812: posix_spawnp returns error when used with file_actions https://bugs.python.org/issue36812 closed by vstinner #36814: posix_spawn explicit file_actions=None throws error https://bugs.python.org/issue36814 closed by vstinner #36815: Test Failure MacOS : test_socket test_host_resolution_bad_addr https://bugs.python.org/issue36815 closed by SilentGhost #36817: Add = to f-strings for easier debugging. https://bugs.python.org/issue36817 closed by eric.smith #36828: Cannot install et-xmlfile https://bugs.python.org/issue36828 closed by xtreak #36830: Typo in collections.deque https://bugs.python.org/issue36830 closed by matrixise #36831: ElementTree.find attribute matching with empty namespace https://bugs.python.org/issue36831 closed by scoder #36832: Port zipp to zipfile https://bugs.python.org/issue36832 closed by barry #36835: Move the warnings runtime state into per-interpreter state. https://bugs.python.org/issue36835 closed by eric.snow #36836: Test Suite not working on 3.4, 3.5 on MacOS Mojave https://bugs.python.org/issue36836 closed by CuriousLearner #36838: running 'make html' from the Doc tree emits an unwelcoming err https://bugs.python.org/issue36838 closed by gregory.p.smith #36844: abiflag `m` is no longer showing when compiled with --enable-s https://bugs.python.org/issue36844 closed by Anthony Sottile #36846: range_iterator does not use __index__ https://bugs.python.org/issue36846 closed by serhiy.storchaka #36847: Segmentation fault (core dumped) Found when we import "schedul https://bugs.python.org/issue36847 closed by jay.net.in #36851: Frame stack is not cleaned after execution is finished with re https://bugs.python.org/issue36851 closed by pablogsal #36857: bisect should support descending order https://bugs.python.org/issue36857 closed by rhettinger #36861: Update to Unicode 12.1.0 https://bugs.python.org/issue36861 closed by benjamin.peterson #36864: Double Spaces in the documentation https://bugs.python.org/issue36864 closed by mdk #36872: passing negative values through modules https://bugs.python.org/issue36872 closed by SilentGhost #1160328: urllib2 post error when using httpproxy https://bugs.python.org/issue1160328 closed by martin.panter

1 0

Please merge : bpo-34848
by Srinivas Reddy Thatiparthy 10 May '19

10 May '19

Hi, The PR - https://github.com/python/cpython/pull/9877 has been approved by Raymond long ago. Please merge or comment if not mergeable. Best Regards, Srinivas Reddy Thatipathy 7768044110.

6 8

Redoing failed PR checks
by Mark Shannon 10 May '19

10 May '19

Hi, How do I redo a failed PR check? The appveyor failure for https://github.com/python/cpython/pull/13181 appears to be spurious, but there is no obvious way to redo it. BTW, this is not the first time I've seen a PR blocked by a spurious appveyor failure. Cheers, Mark.

5 4

PR Review request
by Jake Tesler 09 May '19

09 May '19

Hello Python Dev team! I submitted a Bug+PR back in February and unfortunately, nobody has given it a review yet. :/ I'm was hoping that someone could take a look and suggest changes (if needed). I realize that there are a ton of PRs and things to review across the board, but here's to hoping someone has a little time to take a look at mine! Links: Bug: https://bugs.python.org/issue36084 PR: https://github.com/python/cpython/pull/11993 Thanks so much! Cheers, -Jake Tesler

1 0

Farewell, Python 3.4
by Larry Hastings 09 May '19

09 May '19

It's with a note of sadness that I announce the final retirement of Python 3.4. The final release was back in March, but I didn't get around to actually closing and deleting the 3.4 branch until this morning. Python 3.4 introduced many features we all enjoy in modern Python--the asyncio, ensurepip, and enum packages, just to name three. It's a release I hope we all remember fondly. My eternal thanks to all the members of the release team that worked on Python 3.4: Georg Brandl Julien Palard Martin von Löwis Ned Deily Steve Dower Terry Reedy and all the engineers of the Python infrastructure team. Special thanks to Benjamin Peterson and Ned Deily, who frequently scurried around behind the scenes cleaning up the messes I cluelessly left in my wake. Having closed 3.4, I am now retired as Python 3.4 Release Manager. I regret to inform all of you that you're still stuck with me as Python 3.5 Release Manager until sometime next year. My very best wishes, //arry/

3 2

My PyCon talk on the GIL, the C-API, and subinterpreters
by Eric Snow 09 May '19

09 May '19

I just wanted to share links to my talk about the above, which I gave a week ago at PyCon. Enjoy! :) -eric video: https://youtu.be/7RlqbHCCVyc slides: https://bit.ly/2UMMJey project: https://github.com/ericsnowcurrently/multi-core-python

1 0

Definition of equality check behavior
by Jordan Adler 08 May '19

08 May '19

Hey folks! Through the course of work on the future polyfills <https://github.com/PythonCharmers/python-future/issues/432> that mimic the behavior of Py3 builtins across versions of Python, we've discovered that the equality check behavior of at least some builtin types do not match the documented core data model <https://docs.python.org/3/reference/datamodel.html#object.__eq__>. Specifically, a comparison between a primitive (int, str, float were tested) and an object of a different type always return False, instead of raising a NotImplementedError. Consider `1 == '1'` as a test case. Should the data model be adjusted to declare that primitive types are expected to fallback to False, or should the cpython primitive type's __eq__ implementation fallback to raise NotImplementedError? Reasonable people could disagree about the right approach, but my distaste for silent failures leads me to recommend that the implementation be adjusted to return NotImplementedError as a fallback, and to document that the operands should not be coerced to the same type prior to comparison (enforcing a stricter equality check). This will of course require a deprecation protocol. Alternatively some new equality operator could be used to specify the level of coercion/type checking desired (currently Python has 'is' and '=='). Jordan

6 7

Hello
by Ben Kane 08 May '19

08 May '19

My name is Ben Kane. I'm joining this mailing list and introducing myself as asked in the "welcome to python-dev" email. I've been using Python for maybe 7 years now. I started to learn it in college to help with accounting homework and I continue to use it now for personal projects ranging from small scripts to larger applications and in my day job as a LinkedIn Site Reliability Engineer. I'd like to contribute back to the language that's helped me so much; and talking with core devs at PyCon 2019 has really encouraged me. So I'll be lurking and trying to pick things up as I see them. Thanks! -- Ben Kane

4 4

Easier debugging with f-strings
by Eric V. Smith 08 May '19

08 May '19

Last fall Larry Hastings made a suggestion for adding a way to make so-called "print-based debugging" easier with f-strings. Basically the approach is that f-strings would be able to produce the text of the expression and the value of that expression, without repeating the expression in the f-sting. No more writing f'foo={foo}, bar={bar}'. foo and bar should each only be in there once each! At PyCon US 2019 I did a lightning talk about this, suggesting the syntax of !d, so that if foo="Hello", then f"{foo!d}" would produce "foo='Hello'". That is, it's the text of the expression, followed by an equal sign, followed by the repr of the expression. I have implemented this and a PR exists. Arbitrary expressions are allowed. I heard from core devs and end users after this talk, and all were positive. After that lightning talk, Larry and I talked about it some more, and for a number of reasons decided that it would make more sense if the syntax used an = sign. So we came up with f"{foo=}", which would also produce "foo='Hello'". The reasons for the change are: - Having '=' in the expression is a better mnemonic than !d. - By not using a conversion starting with !, we can compose = with the existing ! conversions, !r, !s, and the rarely used !a. - We can let the user have a little more control of the resulting string. Another thing I like about this approach over !d is that the patch is simpler, because there are fewer special cases. And because there are fewer special cases in the code, I think the proposal is easier to explain than !d (in particular how it interacts (or doesn't!) with format specifiers). There's a good rationale here, along with a PR: https://bugs.python.org/issue36817. My plan is to commit this change before 3.8b1. If anyone would like to discuss it at PyCon, I'll be around until about 10:30 am on Tuesday. I'll be in the CPython sprint room, and I'll be watching bpo, too. Eric

9 11

Can't build 3.7.3 on AIX
by Rob Boehne 08 May '19

08 May '19

Hello, I’m attempting to get a recent version of Python3 running on my UNIX build machines, but having a problem under AIX. This seems like it might be since 3.6.5, but I’ve not actually attempted anything else other than 3.6.8 which fails similarly. I’m building with the native C compiler in 64-bit mode, using OBJECT_MODE=64 exported in the environment. I have attached config.log. Is this a known issue? TIA - Rob ../../Python-3.7.3/config.guess powerpc-ibm-aix7.1.4.0 ../../Python-3.7.3/Modules/makexp_aix Modules/python.exp . libpython3.7m.a; xlc_r -L/raid/checkouts-raid/robb/zlib-dl/Release/rs6000aix_64/lib -Wl,-blibpath:/opt/IBM/xlmass/8.1.3/lib/aix61:/opt/IBM/xlc/13.1.3/lib:/usr/lib:/lib:/opt/Python-3.7/lib -q64 -Wl,-bE:Modules/python.exp -lld -o python Programs/python.o libpython3.7m.so -lintl -ldl -lm LIBPATH=/raid/checkouts-raid/robb/nepal/build-py37 ./python -E -S -m sysconfig --generate-posix-vars ;\ if test $? -ne 0 ; then \ echo "generate-posix-vars failed" ; \ rm -f ./pybuilddir.txt ; \ exit 1 ; \ fi Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python generate-posix-vars failed Makefile:603: recipe for target 'pybuilddir.txt' failed gmake: *** [pybuilddir.txt] Error 1 [cid:image002.png@01D3D0E3.DCFE6710] Rob Boehne Senior Software Architect | Datalogics, Inc. +1.312.853.8351<tel:(312)%20853-8351> | robb(a)datalogics.com<mailto:robb@datalogics.com> datalogics.com<http://www.datalogics.com/> | blogs.datalogics.com<http://blogs.datalogics.com/> Connect with us: Facebook<https://www.facebook.com/DatalogicsInc> | Twitter<https://twitter.com/DatalogicsInc> | LinkedIn<https://www.linkedin.com/company/datalogics?actionToken=p=p%3Dbiz-company-l…> | YouTube<https://www.youtube.com/user/Datalogics>

2 2

PEP 574 ready for review
by Antoine Pitrou 08 May '19

08 May '19

Hello, I've put the final touches to PEP 574 - Pickle protocol 5 with out-of-band data (*). It is now ready for review. The implementation is fully functional, as well as its PyPI backport (**), and has regression tests against Numpy. Numpy and PyArrow have their own tests against the pickle5 backport. (*) https://www.python.org/dev/peps/pep-0574/ (**) https://pypi.org/project/pickle5/ Regards Antoine.

4 5

PEP 578 accepted
by Christian Heimes 08 May '19

08 May '19

Hi, Steve and I worked on the final touch of the PEP during PyCon. PEP 578 [1] is now ready. As the BDFL delegate I'm honoured to accept Steve's PEP. There is a very slim possibility that the PEP implementation might need adjustment for Linux. I haven't got feedback from a Red Hat engineer that works on Linux Kernel Auditing team. Everybody is busy with RHEL 8 release and the Red Hat Summit in Boston. I don't anticipate any changes to the PEP itself. After some consideration I decided to accept PEP now to give Steve enough time to push and test his implementation. There is sufficient time to address issues until the first release candidate of 3.8.0. Thanks Steve, it's been a privilege to work with you! Christian [1] https://www.python.org/dev/peps/pep-0578/

1 0

[RELEASE] Python 3.8.0a4 is now available for testing
by Łukasz Langa 07 May '19

07 May '19

It's time for the LAST alpha of Python 3.8.0. Go get it here: https://www.python.org/downloads/release/python-380a4/ <https://www.python.org/downloads/release/python-380a4/> Python 3.8.0a4 is the fourth and final alpha release of Python 3.8, the next feature release of Python. During the alpha phase, Python 3.8 remains under heavy development: additional features will be added and existing features may be modified or deleted. Please keep in mind that this is a preview release and its use is not recommended for production environments. The first beta release, 3.8.0b1, is planned for 2019-05-31. The release has slipped a week because of me being overwhelmed with PyCon US this year. There was also a release blocker and a breaking change to ElementTree. Anyway, sorry for the wait! I moved the planned date of beta1 a few days to make up for it. If you have a feature you're working on and you'd like to see it in 3.8.0, NOW IS THE TIME TO ACT. Please don't wait until May 30th, get a proper review and land your change as soon as possible. Q: Can I get my feature in after that date if I ask nicely? A: Yes, of course. I will release it in Python 3.9. - Ł

1 0

Stable ABI or not for PyTypeObject?
by Jeroen Demeyer 06 May '19

06 May '19

Hello, I have a simple question for which there doesn't seem to be a good answer: is the layout of PyTypeObject considered to be part of the stable ABI? Officially, the answer is certainly "no" (see PEP 384). However, unofficially the answer might be "yes". At least, the last time that an incompatible change was made to PyTypeObject (adding tp_finalize in Python 3.4, PEP 442), care was taken not to break the ABI by using the Py_TPFLAGS_HAVE_FINALIZE flag. There is some discussion about this on https://bugs.python.org/issue32388 The implementation of PEP 590 is going to make another ABI-breaking change. So should we add a new Py_TFLAGS_HAVE_... flag for that or not? Jeroen.

3 2

"if __name__ == '__main__'" at the bottom of python unittest files
by Chris Withers 05 May '19

05 May '19

Hi All, I have a crazy idea of getting unittest.mock up to 100% code coverage. I noticed at the bottom of all of the test files in testmock/, there's a: if __name__ == '__main__': unittest.main() ...block. How would people feel about these going away? I don't *think* they're needed now that we have unittest discover, but thought I'd ask. Chris

10 25

Unicode identifiers in test files?
by Eric V. Smith 04 May '19

04 May '19

Is there a policy against using Unicode identifiers in test files? As part of adding !d to f-strings, there's a code path that's only executed if the text of the expression is non-ascii. The easiest way to exercise it, and the way I found a bug, is by using an identifier with Unicode chars. I know we have a policy against this in Lib/, but what about Lib/test/? I could work around this with exec-ing some strings, but that seems like added confusion that I'd avoid with a real Unicode identifier. Eric

4 5

Summary of Python tracker Issues
by Python tracker 03 May '19

03 May '19

ACTIVITY SUMMARY (2019-04-26 - 2019-05-03) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7081 ( -2) closed 41454 (+52) total 48535 (+50) Open issues with patches: 2820 Issues opened (41) ================== #36356: Failure to build with address sanitizer https://bugs.python.org/issue36356 reopened by vstinner #36707: The "m" ABI flag of SOABI for pymalloc is no longer needed https://bugs.python.org/issue36707 reopened by ned.deily #36737: Warnings operate out of global runtime state. https://bugs.python.org/issue36737 opened by eric.snow #36738: Add 'array_hook' for json module https://bugs.python.org/issue36738 opened by matomatical #36739: "4.6. Defining Functions" should mention nonlocal https://bugs.python.org/issue36739 opened by pbhd0815 #36743: Docs: Descript __get__ signature defined differently across th https://bugs.python.org/issue36743 opened by jdufresne #36744: functools.singledispatch: Shouldn't require a positional argum https://bugs.python.org/issue36744 opened by KevinG #36746: Create test for fcntl.lockf() https://bugs.python.org/issue36746 opened by nanjekyejoannah #36748: Optimize textio write buffering https://bugs.python.org/issue36748 opened by inada.naoki #36749: PPC64 AIX 3.x: compilation issue, linker fails to locate symbo https://bugs.python.org/issue36749 opened by vstinner #36750: test_socket leaks file descriptors on macOS https://bugs.python.org/issue36750 opened by xtreak #36751: Changes in the inspect module for PEP 570 https://bugs.python.org/issue36751 opened by pablogsal #36752: test multiprocessing: test_rapid_restart() crash on AIX https://bugs.python.org/issue36752 opened by vstinner #36753: Python modules not linking to libpython causes issues for RTLD https://bugs.python.org/issue36753 opened by reimar #36754: Remove smart quotes in pydoc text https://bugs.python.org/issue36754 opened by steve.dower #36755: [2.7] test_httplib leaked [8, 8, 8] references with OpenSSL 1. https://bugs.python.org/issue36755 opened by vstinner #36756: tkinter tk.createcommand memory leak https://bugs.python.org/issue36756 opened by WKraus #36757: uuid constructor accept invalid strings (extra dash) https://bugs.python.org/issue36757 opened by Cédric Cabessa #36758: configured libdir not correctly passed to Python executable https://bugs.python.org/issue36758 opened by michael-o #36759: datetime: astimezone() results in OSError: [Errno 22] Invalid https://bugs.python.org/issue36759 opened by Snidhi #36760: subprocess.run fails with capture_output=True and stderr=STDOU https://bugs.python.org/issue36760 opened by Joe.Borg #36761: Extended slice assignment + iterable unpacking https://bugs.python.org/issue36761 opened by wim.glenn #36762: Teach "import *" to warn when overwriting globals or builtins https://bugs.python.org/issue36762 opened by rhettinger #36763: PEP 587: Rework initialization API to prepare second version o https://bugs.python.org/issue36763 opened by vstinner #36766: Typos in docs and code comments https://bugs.python.org/issue36766 opened by xtreak #36767: Segmentation fault when running c extension on macOS https://bugs.python.org/issue36767 opened by Senhui Guo #36768: distutils.util.convert_path mangles windows paths with forward https://bugs.python.org/issue36768 opened by LordAro #36769: doc Document that fnmatch.filter supports any kind of iterable https://bugs.python.org/issue36769 opened by adelfino #36770: stdlib - shutil.make_archive - add support for different ZIP c https://bugs.python.org/issue36770 opened by owenchia #36771: Feature Request: An option to os.walk() to return os.DirEntry https://bugs.python.org/issue36771 opened by apocalyptech #36772: Let lru_cache be used as a decorator with no arguments https://bugs.python.org/issue36772 opened by rhettinger #36773: Race condition during pickle.load() https://bugs.python.org/issue36773 opened by papad #36774: f-strings: Add a !d conversion for ease of debugging https://bugs.python.org/issue36774 opened by eric.smith #36777: unittest discover throws TypeError on empty packages https://bugs.python.org/issue36777 opened by xtreak #36778: test_site.StartupImportTests.test_startup_imports fails if def https://bugs.python.org/issue36778 opened by Paul Monson #36779: time.tzname returns empty string on Windows if default codepag https://bugs.python.org/issue36779 opened by Paul Monson #36780: Interpreter exit blocks waiting for futures of shut-down Threa https://bugs.python.org/issue36780 opened by hniksic #36781: Optimize sum() for bools https://bugs.python.org/issue36781 opened by serhiy.storchaka #36782: Add tests for the datetime C API https://bugs.python.org/issue36782 opened by p-ganssle #36783: No documentation for _FromXandFold C API functions https://bugs.python.org/issue36783 opened by p-ganssle #36784: __import__ with empty folder after importlib.invalidate_caches https://bugs.python.org/issue36784 opened by xtreak Most recent 15 issues with no replies (15) ========================================== #36784: __import__ with empty folder after importlib.invalidate_caches https://bugs.python.org/issue36784 #36781: Optimize sum() for bools https://bugs.python.org/issue36781 #36780: Interpreter exit blocks waiting for futures of shut-down Threa https://bugs.python.org/issue36780 #36779: time.tzname returns empty string on Windows if default codepag https://bugs.python.org/issue36779 #36770: stdlib - shutil.make_archive - add support for different ZIP c https://bugs.python.org/issue36770 #36769: doc Document that fnmatch.filter supports any kind of iterable https://bugs.python.org/issue36769 #36768: distutils.util.convert_path mangles windows paths with forward https://bugs.python.org/issue36768 #36766: Typos in docs and code comments https://bugs.python.org/issue36766 #36758: configured libdir not correctly passed to Python executable https://bugs.python.org/issue36758 #36755: [2.7] test_httplib leaked [8, 8, 8] references with OpenSSL 1. https://bugs.python.org/issue36755 #36752: test multiprocessing: test_rapid_restart() crash on AIX https://bugs.python.org/issue36752 #36748: Optimize textio write buffering https://bugs.python.org/issue36748 #36738: Add 'array_hook' for json module https://bugs.python.org/issue36738 #36732: test_asyncio: test_huge_content_recvinto() fails randomly https://bugs.python.org/issue36732 #36728: Remove PyEval_ReInitThreads() from the public C API https://bugs.python.org/issue36728 Most recent 15 issues waiting for review (15) ============================================= #36778: test_site.StartupImportTests.test_startup_imports fails if def https://bugs.python.org/issue36778 #36774: f-strings: Add a !d conversion for ease of debugging https://bugs.python.org/issue36774 #36772: Let lru_cache be used as a decorator with no arguments https://bugs.python.org/issue36772 #36771: Feature Request: An option to os.walk() to return os.DirEntry https://bugs.python.org/issue36771 #36770: stdlib - shutil.make_archive - add support for different ZIP c https://bugs.python.org/issue36770 #36769: doc Document that fnmatch.filter supports any kind of iterable https://bugs.python.org/issue36769 #36763: PEP 587: Rework initialization API to prepare second version o https://bugs.python.org/issue36763 #36751: Changes in the inspect module for PEP 570 https://bugs.python.org/issue36751 #36749: PPC64 AIX 3.x: compilation issue, linker fails to locate symbo https://bugs.python.org/issue36749 #36748: Optimize textio write buffering https://bugs.python.org/issue36748 #36746: Create test for fcntl.lockf() https://bugs.python.org/issue36746 #36743: Docs: Descript __get__ signature defined differently across th https://bugs.python.org/issue36743 #36739: "4.6. Defining Functions" should mention nonlocal https://bugs.python.org/issue36739 #36738: Add 'array_hook' for json module https://bugs.python.org/issue36738 #36725: Reference leak regression with Python3.8a3 https://bugs.python.org/issue36725 Top 10 most discussed issues (10) ================================= #36774: f-strings: Add a !d conversion for ease of debugging https://bugs.python.org/issue36774 14 msgs #36751: Changes in the inspect module for PEP 570 https://bugs.python.org/issue36751 12 msgs #30458: [security][CVE-2019-9740][CVE-2019-9947] HTTP Header Injection https://bugs.python.org/issue30458 9 msgs #36763: PEP 587: Rework initialization API to prepare second version o https://bugs.python.org/issue36763 9 msgs #26493: Bad formatting in WinError 193 when using subprocess.check_cal https://bugs.python.org/issue26493 8 msgs #36710: Pass _PyRuntimeState as an argument rather than using the _PyR https://bugs.python.org/issue36710 8 msgs #34616: implement "Async exec" https://bugs.python.org/issue34616 7 msgs #35184: Makefile is not correctly generated when compiling pyextat wit https://bugs.python.org/issue35184 5 msgs #36356: Failure to build with address sanitizer https://bugs.python.org/issue36356 5 msgs #36540: PEP 570: Python Positional-Only Parameters https://bugs.python.org/issue36540 5 msgs Issues closed (51) ================== #5166: ElementTree and minidom don't prevent creation of not well-for https://bugs.python.org/issue5166 closed by scoder #8583: Hardcoded namespace_separator in the cElementTree.XMLParser https://bugs.python.org/issue8583 closed by scoder #10328: re.sub[n] doesn't seem to handle /Z replacements correctly in https://bugs.python.org/issue10328 closed by serhiy.storchaka #13611: Integrate ElementC14N module into xml.etree package https://bugs.python.org/issue13611 closed by scoder #14546: lll.py can't handle multiple parameters correctly https://bugs.python.org/issue14546 closed by vstinner #15388: SAX parse (ExpatParser) leaks file handle when given filename https://bugs.python.org/issue15388 closed by berker.peksag #16425: minidom replaceChild(new_child, old_child) removes new_child e https://bugs.python.org/issue16425 closed by scoder #18675: Daemon Threads can seg fault https://bugs.python.org/issue18675 closed by asvetlov #19895: Cryptic error when subclassing multiprocessing classes https://bugs.python.org/issue19895 closed by SilentGhost #21403: cElementTree's Element creation handles attrib argument differ https://bugs.python.org/issue21403 closed by scoder #24638: asyncio "loop argument must agree with future" error message c https://bugs.python.org/issue24638 closed by asvetlov #27682: wsgiref BaseHandler / SimpleHandler can raise additional error https://bugs.python.org/issue27682 closed by berker.peksag #28460: Minidom, order of attributes, datachars https://bugs.python.org/issue28460 closed by scoder #28608: Support creating hardlink using `pathlib` https://bugs.python.org/issue28608 closed by xtreak #32235: test_xml_etree test_xml_etree_c failures with 2.7 and 3.6 bran https://bugs.python.org/issue32235 closed by scoder #34442: zlib module not built on windows https://bugs.python.org/issue34442 closed by zach.ware #34602: python3 resource.setrlimit strange behaviour under macOS https://bugs.python.org/issue34602 closed by ned.deily #35329: Documentation - capitalization issue https://bugs.python.org/issue35329 closed by benjamin.peterson #35502: Memory leak in xml.etree.ElementTree.iterparse https://bugs.python.org/issue35502 closed by scoder #35726: QueueHandler formatting affects other handlers https://bugs.python.org/issue35726 closed by ned.deily #35947: Update libffi_msvc to current version of libffi https://bugs.python.org/issue35947 closed by Paul Monson #35952: test.pythoninfo prints a stack trace and exits with 1 when the https://bugs.python.org/issue35952 closed by vstinner #36004: Add datetime.fromisocalendar https://bugs.python.org/issue36004 closed by vstinner #36025: Breaking change in PyDate_FromTimeStamp API https://bugs.python.org/issue36025 closed by berker.peksag #36341: bind() on AF_UNIX socket may fail in tests run as non-root https://bugs.python.org/issue36341 closed by asvetlov #36432: Running python test suite fails on macOS 10.14.4 with resource https://bugs.python.org/issue36432 closed by ned.deily #36475: PyEval_AcquireLock() and PyEval_AcquireThread() do not handle https://bugs.python.org/issue36475 closed by vstinner #36509: Add iot layout for windows iot containers https://bugs.python.org/issue36509 closed by Paul Monson #36613: asyncio._wait() don't remove callback in case of exception https://bugs.python.org/issue36613 closed by asvetlov #36673: Comment/PI parsing support for ElementTree https://bugs.python.org/issue36673 closed by scoder #36676: Make ET.XMLParser target aware of namespace prefixes https://bugs.python.org/issue36676 closed by scoder #36699: building for riscv multilib (patch attached) https://bugs.python.org/issue36699 closed by Andreas K. Hüttel #36715: Dictionary initialization https://bugs.python.org/issue36715 closed by rhettinger #36716: Embedded Python fails to import module files with version_plat https://bugs.python.org/issue36716 closed by ecosatto #36719: regrtest --findleaks should fail if an uncollectable object is https://bugs.python.org/issue36719 closed by vstinner #36729: Delete unused text variable on tests https://bugs.python.org/issue36729 closed by serhiy.storchaka #36730: Change outdated references to macOS https://bugs.python.org/issue36730 closed by berker.peksag #36734: Modules/faulthandler.c does not compile on HP-UX due to bpo-35 https://bugs.python.org/issue36734 closed by vstinner #36735: minimize disk size of cross-compiled python3.6 https://bugs.python.org/issue36735 closed by michalgr #36736: Python crashes when calling win32file.LockFileEx https://bugs.python.org/issue36736 closed by eryksun #36740: zipimporter misses namespace packages for implicit dirs https://bugs.python.org/issue36740 closed by jaraco #36741: Variable about function and list https://bugs.python.org/issue36741 closed by xtreak #36742: urlsplit doesn't accept a NFKD hostname with a port number https://bugs.python.org/issue36742 closed by steve.dower #36745: A possible reference leak in PyObject_SetAttr() https://bugs.python.org/issue36745 closed by asvetlov #36747: Tools/scripts/setup.py is missing https://bugs.python.org/issue36747 closed by vstinner #36764: Types module doesn't have a type for _abc_data https://bugs.python.org/issue36764 closed by serhiy.storchaka #36765: Invalid grammar for f_expression https://bugs.python.org/issue36765 closed by eric.smith #36775: Rework filesystem codec implementation https://bugs.python.org/issue36775 closed by vstinner #36776: test_tools: test_lll_multiple_dirs() failed on AMD64 Windows7 https://bugs.python.org/issue36776 closed by vstinner #1613500: Write mode option for fileinput module. https://bugs.python.org/issue1613500 closed by berker.peksag #1191964: add non-blocking read and write methods to subprocess.Popen https://bugs.python.org/issue1191964 closed by josiahcarlson

1 0

Location of CI for Windows Embeddable Distro
by Gerald Wiltse 03 May '19

03 May '19

I looked through all the windows builders here and could not find any which generate this package, can anyone tell me where the work happens? I also tried searching looking for scripts and CI files which create the package in all the relevant repos and could not find those either. Is this information not public? Is it a manual build? Gerald R. Wiltse jerrywiltse(a)gmail.com

2 10

PEP 580/590 proposals summary
by Jeroen Demeyer 03 May '19

03 May '19

Hello all, If we want to have a chance of implementing PEP 580/590 in Python 3.8, we shouldn't wait too long to make a decision on which proposal to accept. As a summary, below I'll write the four proposals together with a star "score" for 3 criteria (there is no obvious best proposal, all have advantages and disadvantages): - complexity: more stars is a protocol which is simpler to document and understand. - implementation: more stars is a simpler implementation of CPython (not just of the protocol itself, but also the code using the protocol). - performance: more stars is better performance for *existing* code. I'm using a minimum of 3 stars here, since the difference is not that big between the proposals. Criteria that I am NOT considering: - The performance for *new* code or the performance of wrappers generated by Argument Clinic: all proposals score excellent here. - Complexity of implementations of external classes: this is hard to judge, since that depends a lot on what those external classes (outside of CPython) want to do. - The work to implement the proposal in CPython: this is a one-time only thing that I'm volunteering to do anyway. - Extensibility of the protocol: first of all, it's hard to define what this means exactly. Second, using Petr's idea of putting the flags inside the instance, every proposal becomes extensible at little cost. Proposals: (A) PEP 580 complexity: * implementation: ***** performance: ***** (B) compromise: PEP 580 with a single calling convention complexity: *** implementation: **** performance: **** (C) PEP 590 with a single bound method class complexity: ***** implementation: *** performance: *** (D) PEP 590 complexity: ***** implementation: * performance: **** I consider Petr's proposal (a more extensible variant of PEP 590 with flags inside the instance) a minor variation of PEP 590 for this purpose and no need to score it differently than "plain" PEP 590. I tried to do this as unbiased as possible, even though I must admit that this is not really possible. I'm considering not just the PEP and the existing implementation as written, but also ideas that haven't been implemented yet such as: - proposals (A)-(C): rationalization of classes, in particular having a single class for bound methods (just like in PyPy). - proposals (B)-(D): Mark Shannon's idea of having a dedicated vectorcall wrapper for each calling convention (one for METH_O, one for METH_VARARGS|METH_KEYWORDS, ...). - using the protocol also for slot wrappers like object.__eq__ I'm NOT considering Petr's proposal of removing support for other calling conventions like METH_VARARGS because that won't happen any time soon. Cheers, Jeroen

1 0