Mailman 3 May 2016 - docs

[issue22128] patch: steer people away from codecs.open
by Frank van Dijk 16 Apr '21

16 Apr '21

New submission from Frank van Dijk: stackoverflow.com has a zillion answers recommending the use of codecs.open() as a unicode capable drop in replacement for open(). This probably means that there is still a lot of code being written that uses codecs.open(). That's bad thing because of codecs.open()'s lack of newline conversion. A lot of that code will - have compatibility issues when it is moved between unix and windows - silently break text files on windows, leading to issues further downstream (confusing other tools, messing up revision control histories) The problem has been fixed with io.open() in 2.x and open() in 3.x. Unfortunately the 2.7 unicode HOWTO still recommends the use of codecs.open(). The 2.7 and the 3.x documentation of codecs.open() doesn't refer the reader to better alternatives. The attached patches fix that. The only downside I see is that newly written code that uses the better alternatives would be incompatible with 2.5 and older. However croaking on a small minority of systems is better than silently disrupting workflows, causing platform incompatibilities, and inviting flaky workarounds. The 2.7 patch makes the unicode HOWTO recommend io.open() instead of codecs.open(). Both patches change the codecs.open() documentation to refer to io.open() or (on 3.x) open(). Additionally I removed the "data loss" explanation from codecs.open()'s note about its lack of newline conversion. It is not particularly helpful information and it is not entirely correct (data loss could also have been avoided by doing newline conversion before encoding and after decoding) ---------- assignee: docs@python components: Documentation files: codecsopen2.patch keywords: patch messages: 224632 nosy: Frank.van.Dijk, docs@python priority: normal severity: normal status: open title: patch: steer people away from codecs.open type: behavior versions: Python 2.7, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file36234/codecsopen2.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22128> _______________________________________

7 11

[issue20364] Rename & explain sqlite3.Cursor.execute 'parameters' param
by Terry J. Reedy 14 Apr '21

14 Apr '21

New submission from Terry J. Reedy: "execute(sql[, parameters]) Executes an SQL statement. The SQL statement may be parametrized (i. e. placeholders instead of SQL literals). The sqlite3 module supports two kinds of placeholders: question marks (qmark style) and named placeholders (named style)." Experimental facts based on experiments with the code example in the doc, using 3.4.b2: 'parameters' is a single subscriptable collection parameter, sequence or dict, that might be called seq_dict. It is positional only, so whatever name is used is a dummy. Only one placeholder style can be used in a given SQL statement string. If question marks are used, seq_dict must be a sequence. If names are used, seq_dict can be either a sequence or dict or subclass thereof. A UserDict is treated as a sequence and raises KeyError(0). Possible text that encompasses the above, replacing the last sentence: "A statement may use one of two kinds of placeholders: question marks (qmark style) or named placeholders (named style). For qmark style, seq_dict must be a sequence. For named style, it can be either a sequence or dict instance. Len(seq_dict) must match the number of placeholders." After cleaning up the test file, I will verify on 2.7 and upload. ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 208908 nosy: docs@python, terry.reedy priority: normal severity: normal stage: patch review status: open title: Rename & explain sqlite3.Cursor.execute 'parameters' param type: behavior versions: Python 2.7, Python 3.3, Python 3.4 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue20364> _______________________________________

7 12

[issue21297] skipinitialspace in the csv module only skips spaces, not "whitespace" in general
by Daniel Andersson 25 Mar '21

25 Mar '21

New submission from Daniel Andersson: Regarding the `skipinitialspace` parameter to the different CSV reader dialects in the `csv` module, the official documentation asserts: When True, whitespace immediately following the delimiter is ignored. and the `help(csv)` style module documentation says: * skipinitialspace - specifies how to interpret whitespace which immediately follows a delimiter. It defaults to False, which means that whitespace immediately following a delimiter is part of the following field. "Whitespace" is a bit too general in both cases (at least a red herring in the second case), since it only skips spaces and not e.g. tabs [1]. In `Modules/_csv.c`, it more correctly describes the parameter. At line 81: int skipinitialspace; /* ignore spaces following delimiter? */ and the actual implementation at line 638: else if (c == ' ' && dialect->skipinitialspace) /* ignore space at start of field */ ; No-one will probably assume that the whole UTF-8 spectrum of "whitespace" is skipped, but at least I initially assumed that the tab character was included. [1]: http://en.wikipedia.org/wiki/Whitespace_character ---------- assignee: docs@python components: Documentation, Library (Lib) messages: 216780 nosy: Daniel.Andersson, docs@python priority: normal severity: normal status: open title: skipinitialspace in the csv module only skips spaces, not "whitespace" in general type: behavior versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue21297> _______________________________________

6 9

[issue14189] Documentation for some C APIs is missing clear specification of the type of reference they return
by Baruch Sterin 21 Mar '21

21 Mar '21

New submission from Baruch Sterin <python(a)bsterin.com>: In addition to the description text, most C API functions have a one-line, emphasized specification whether they return a new or a borrowed reference. (e.g. 'Return value: New reference.'). The following API functions are missing that. Some of them, like PyMemoryView_FromBuffer(), have descriptions that are clear, but it would still be nice to have an unambiguous statement like most other API functions have. The list has been generated automatically, so it might contain some errors. Doc/c-api/arg.rst: Py_VaBuildValue Doc/c-api/buffer.rst: PyMemoryView_FromBuffer Doc/c-api/buffer.rst: PyMemoryView_FromObject Doc/c-api/buffer.rst: PyMemoryView_GetContiguous Doc/c-api/bytearray.rst: PyByteArray_Concat Doc/c-api/bytearray.rst: PyByteArray_FromObject Doc/c-api/bytearray.rst: PyByteArray_FromStringAndSize Doc/c-api/code.rst: PyCode_New Doc/c-api/codec.rst: PyCodec_BackslashReplaceErrors Doc/c-api/codec.rst: PyCodec_Decode Doc/c-api/codec.rst: PyCodec_Decoder Doc/c-api/codec.rst: PyCodec_Encode Doc/c-api/codec.rst: PyCodec_Encoder Doc/c-api/codec.rst: PyCodec_IgnoreErrors Doc/c-api/codec.rst: PyCodec_IncrementalDecoder Doc/c-api/codec.rst: PyCodec_IncrementalEncoder Doc/c-api/codec.rst: PyCodec_LookupError Doc/c-api/codec.rst: PyCodec_ReplaceErrors Doc/c-api/codec.rst: PyCodec_StreamReader Doc/c-api/codec.rst: PyCodec_StreamWriter Doc/c-api/codec.rst: PyCodec_StrictErrors Doc/c-api/codec.rst: PyCodec_XMLCharRefReplaceErrors Doc/c-api/exceptions.rst: PyUnicodeDecodeError_Create Doc/c-api/exceptions.rst: PyUnicodeDecodeError_GetEncoding Doc/c-api/exceptions.rst: PyUnicodeDecodeError_GetObject Doc/c-api/exceptions.rst: PyUnicodeDecodeError_GetReason Doc/c-api/exceptions.rst: PyUnicodeEncodeError_Create Doc/c-api/exceptions.rst: PyUnicodeTranslateError_Create Doc/c-api/float.rst: PyFloat_GetInfo Doc/c-api/import.rst: PyImport_GetImporter Doc/c-api/import.rst: PyImport_ImportModuleNoBlock Doc/c-api/import.rst: _PyImport_FindExtension Doc/c-api/import.rst: _PyImport_FixupExtension Doc/c-api/init.rst: PyEval_GetCallStats Doc/c-api/int.rst: PyInt_FromSize_t Doc/c-api/long.rst: PyLong_FromSize_t Doc/c-api/long.rst: PyLong_FromSsize_t Doc/c-api/number.rst: PyNumber_Index Doc/c-api/number.rst: PyNumber_ToBase Doc/c-api/object.rst: PyObject_Bytes Doc/c-api/object.rst: PyObject_GenericGetAttr Doc/c-api/unicode.rst: PyUnicode_AsUTF32String Doc/c-api/unicode.rst: PyUnicode_DecodeMBCSStateful Doc/c-api/unicode.rst: PyUnicode_DecodeUTF32 Doc/c-api/unicode.rst: PyUnicode_DecodeUTF32Stateful Doc/c-api/unicode.rst: PyUnicode_DecodeUTF7 Doc/c-api/unicode.rst: PyUnicode_DecodeUTF7Stateful Doc/c-api/unicode.rst: PyUnicode_EncodeUTF32 Doc/c-api/unicode.rst: PyUnicode_EncodeUTF7 Doc/c-api/veryhigh.rst: PyEval_EvalCodeEx Doc/c-api/veryhigh.rst: PyEval_EvalFrame Doc/c-api/veryhigh.rst: PyEval_EvalFrameEx ---------- assignee: docs@python components: Documentation messages: 154877 nosy: baruch.sterin, docs@python priority: normal severity: normal status: open title: Documentation for some C APIs is missing clear specification of the type of reference they return type: behavior versions: Python 2.7 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue14189> _______________________________________

9 9

[issue15533] subprocess.Popen(cwd) documentation
by Chris Jerdonek 16 Mar '21

16 Mar '21

New submission from Chris Jerdonek: The sentence describing Popen()'s cwd argument in the subprocess documentation seems reversed to me: http://docs.python.org/dev/library/subprocess.html#subprocess.Popen It says, "If cwd is not None, the child’s current directory will be changed to cwd before it is executed. Note that this directory is not considered when searching the executable, so you can’t specify the program’s path relative to cwd." However, when cwd is not None, it seems like you *must* specify the program's path relative to cwd. For example, when running a script containing the following using `./python.exe` from a source checkout-- p = Popen(['./python.exe', '-V'], stdout=PIPE, stderr=PIPE, cwd='temp') you get an: "OSError: [Errno 2] No such file or directory." In contrast, when you *do* specify the program's path relative to cwd, it works-- p = Popen(['../python.exe', '-V'], stdout=PIPE, stderr=PIPE, cwd='temp') Issue 6374 seems to have made the same point in its second bullet, but the issue was closed without addressing that part of it. ---------- assignee: docs@python components: Documentation keywords: easy messages: 167194 nosy: cjerdonek, docs@python priority: normal severity: normal status: open title: subprocess.Popen(cwd) documentation versions: Python 2.7, Python 3.3 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue15533> _______________________________________

11 51

[issue18767] csv documentation does not note default quote constant
by Brian McLaughlin 14 Mar '21

14 Mar '21

New submission from Brian McLaughlin: Documentation should note default quoting behavior of the csv module is csv.QUOTE_MINIMAL http://hg.python.org/cpython/file/tip/Modules/_csv.c#l420 ---------- assignee: docs@python components: Documentation files: doc.patch keywords: patch messages: 195448 nosy: bemclaugh, docs@python priority: normal severity: normal status: open title: csv documentation does not note default quote constant versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file31333/doc.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue18767> _______________________________________

4 3

[issue18017] ctypes.PyDLL documentation
by Marc Brünink 12 Mar '21

12 Mar '21

New submission from Marc Brünink: The documentation for is not very clear regarding the usage of CDLL and PyDLL. Especially it is not obvious that you should use PyDLL whenever you call any function of the Python/C API. Since calling a Python/C API function without owning the GIL will most likely cause latent segfaults, I think it warrants a warning box in section 16.17.2.2 and maybe also somewhere prominent in http://docs.python.org/dev/c-api/intro.html. For section 16.17.2.2 I would put a box next the decription of CDLL saying something like: "The Python GIL is released before a function call. It is not safe to call any Pyhon/C API function within the loaded library without acquiring the GIL first. Thus, if the loaded library calls functions of the Python/C API, for example in case it creates and returns a new Python object, and does not manually acquire and release the GIL, use PyDLL instead." ---------- assignee: docs@python components: Documentation messages: 189629 nosy: Marc.Brünink, docs@python priority: normal severity: normal status: open title: ctypes.PyDLL documentation versions: Python 3.5 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue18017> _______________________________________

5 5

[issue25741] Usual Installation Directory
by Firat Ozgul 12 Mar '21

12 Mar '21

New submission from Firat Ozgul: Official documentation reads: "On Windows machines, the Python installation is usually placed in C:\Python35" However, as of Python 3.5.0, usual installation directory on Windows is %LOCALAPPDATA%\Programs\Python. ---------- assignee: docs@python components: Documentation messages: 255455 nosy: docs@python, firatozgul priority: normal severity: normal status: open title: Usual Installation Directory type: enhancement versions: Python 3.5 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue25741> _______________________________________

6 10

[issue15097] Improving wording on the thread-safeness of import
by Merlijn van Deen 09 Mar '21

09 Mar '21

New submission from Merlijn van Deen <valhallasw(a)gmail.com>: http://docs.python.org/library/threading.html#importing-in-threaded-code Currently, the documentation states "Firstly, other than in the main module, an import should not have the side effect of spawning a new thread and then waiting for that thread in any way. Failing to abide by this restriction can lead to a deadlock if the spawned thread directly or indirectly attempts to import a module." which, I think, fails to make the main point: a call to import acquires the import lock. A call to import from a second thread will thus block. As such, I would suggest rephrasing it to something like: "Firstly, an import acquires the import lock for that thread. Therefore, the import should not have the side effect of waiting for a different thread in any way, as this can lead to a deadlock if that thread directly or indirectly attempts to import a module." There are two additional points that might be interesting to note: (1) Any module can be imported. If the import causes a deadlock, that is a bad thing. Every module *will* be imported by tools such as nosetests. (1b) so: never, ever, have code that causes locks in a different thread in module level code witout 'if __name__=="__main__" ' blocks? (2) The lock is also acquired if a module has already been imported. For instance, in import sys # (1) def f(): import sys # (2) the import lock is acquired in (1) /and/ (2). Adding example code and/or a flow diagram might be a bit too much, but it does clarify how easy it is to make this mistake. See the attached for an example (both a simple example script, as well as a flow diagram explaining what happens). ---------- assignee: docs@python components: Documentation files: deadlock.py messages: 163068 nosy: docs@python, valhallasw priority: normal severity: normal status: open title: Improving wording on the thread-safeness of import type: enhancement Added file: http://bugs.python.org/file26037/deadlock.py _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue15097> _______________________________________

3 6

[issue8557] subprocess portability issue
by Dave Abrahams 05 Mar '21

05 Mar '21

New submission from Dave Abrahams <dave(a)boostpro.com>: On POSIX systems, the PATH environment variable is always used to look up directory-less executable names passed as the first argument to Popen(...), but on Windows, PATH is only considered when shell=True is also passed. Actually I think it may be slightly weirder than that when shell=False, because the following holds for me: C:\>rem ##### Prepare minimal PATH ##### C:\>set "PATH=C:\Python26\Scripts;C:\Python26;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem" C:\>rem ##### Prepare a minimal, clean environment ##### C:\>virtualenv --no-site-packages e:\zzz New python executable in e:\zzz\Scripts\python.exe Installing setuptools................done. C:\>rem ##### Show that shell=True makes the difference in determining whether PATH is respected ##### C:\>python Python 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import subprocess >>> subprocess.Popen(['python', '-c', 'import sys; print sys.executable']) <subprocess.Popen object at 0x0000000001DBE080> >>> C:\Python26\python.exe >>> subprocess.Popen(['python', '-c', 'import sys; print sys.executable'], env={'PATH':r'e:\zzz\Scripts'}) <subprocess.Popen object at 0x0000000001F05A90> >>> C:\Python26\python.exe >>> subprocess.Popen(['python', '-c', 'import sys; print sys.executable'], env={'PATH':r'e:\zzz\Scripts'}, shell=True) <subprocess.Popen object at 0x0000000001F05B00> >>> e:\zzz\Scripts\python.exe That is, it looks like the environment at the time Python is invoked is what counts unless I pass shell=True. I don't even seem to be able to override this behavior by changing os.environ: you can clear() it and pass env={} and subprocess.Popen(['python']) still succeeds. This is a very important problem for portable code and one that took me hours to suss out. I think: a) the current behavior needs to be documented b) it needs to be fixed if possible c) otherwise, shell=True should be the default ---------- assignee: docs@python components: Documentation messages: 104422 nosy: dabrahams, docs@python priority: normal severity: normal status: open title: subprocess portability issue type: behavior versions: Python 2.6 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue8557> _______________________________________

8 23