Mailman 3 September 2018 - docs

[issue34601] Typo: "which would rather raise MemoryError than give up", than or then?
by David Lin 07 Sep '18

07 Sep '18

New submission from David Lin <david.dl.lin(a)nokia-sbell.com>: https://docs.python.org/3/library/exceptions.html exception OverflowError Raised when the result of an arithmetic operation is too large to be represented. This cannot occur for integers (which would rather raise MemoryError than give up). However, for historical reasons, OverflowError is sometimes raised for integers that are outside a required range. Because of the lack of standardization of floating point exception handling in C, most floating point operations are not checked. In sentence: "which would rather raise MemoryError than give up", should we use "then" instead of "than"? Thanks, David Lin ---------- assignee: docs@python components: Documentation messages: 324720 nosy: davidlin, docs@python priority: normal severity: normal status: open title: Typo: "which would rather raise MemoryError than give up", than or then? versions: Python 3.7 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue34601> _______________________________________

3 2

Pyton 2.7.14 Multiprocessing Pool Join Hanging
by Smirnov 07 Sep '18

07 Sep '18

Hi, /usr/lib/python2.7/multiprocessing/pool.py hangs intermittently for large queue sizes. My suggested fix is to modify Pool class' join() method thus: def join(self): debug('joining pool') assert self._state in (CLOSE, TERMINATE) # moved the join for each subprocess before Pool thead joins for p in self._pool: p.join() # moved Pool thread joins after the subprocesses and add timeouts self._worker_handler.join(timeout=30) self._task_handler.join(timeout=30) self._result_handler.join(time=30) The bulk of the work is done by subprocesses. When these are done, the Pool class' threads should complete very quickly. I'm fairly certain that the hang or stall happens in the threads' join, not the subprocesses'. The timeout on the threads' joins should prevent any hanging. The only way I've manually been able to reproduce the hanging is by the below script using the *CalledProcessError exception.* Kind Regards, -Nazar -------------------------- import logging from multiprocessing import Pool, Process from time import sleep import random import pdb import sys import os from subprocess import CalledProcessError logging.basicConfig(stream=sys.stdout, level=logging.DEBUG) def f(x): pause = random.randint(1,2) #print("pid %s start, arg = %s, pause = %s" % (os.getpid(), x, pause)) if pause % 2 == 0: #pause = 9 #9999 raise CalledProcessError(1,2) sleep(0.01) #pause) #print("pid %s finish, arg = %s" % (os.getpid(), x)) def print_status(pl): print("\n") for proc in pl._pool: print("Process %s, pid %s, is_alive %s, exit code %s, unfinished tasks %s" % (proc._name, proc.pid, proc.is_alive(), proc.exitcode, pl._taskqueue.unfinished_tasks)) import subprocess def pool_join(proc_pool): """ Join all of the process pool's subprocesses to the main processes Python Pool's "join" intermittently hangs for ~1K subprocesses: STATICQA-5027 """ RUN = 0 CLOSE = 1 TERMINATE = 2 # log memory usage mem_str = subprocess.check_output(['cat', '/proc/meminfo']) logging.info('------ memory usage\n%s' % mem_str) logging.info('------ joining process pool') assert proc_pool._state in (CLOSE, TERMINATE) logging.info('------ proc_pool._worker_handler.join()') proc_pool._worker_handler.join() logging.info('------ proc_pool._task_handler.join()') proc_pool._task_handler.join() logging.info('------ proc_pool._result_handler.join()') proc_pool._result_handler.join() logging.info('------ starting to join each process in POOL') for proc in proc_pool._pool: logging.info("------ Joining process %s, pid %s, is_alive %s, exit code %s" % (proc._name, proc.pid, proc.is_alive(), proc.exitcode)) proc.join() logging.info("------- All POOL joins are done") if __name__ == '__main__': pl = Pool(6) NUM_PROC = 100 for i in range(1,NUM_PROC): res = pl.apply_async(f, (i,)) pl.close() print("Num processes = %s" % len(pl._pool)) for i in range(4): print_status(pl) sleep(2) print("before join") #pl.join() pool_join(pl) print("after join") print_status(pl)

1 0

[issue34580] sqlite doc: clarify the scope of the context manager
by Karthikeyan Singaravelan 06 Sep '18

06 Sep '18

Karthikeyan Singaravelan <tir.karthi(a)gmail.com> added the comment: Seems like an extra note is reasonable here. I have created a PR for this. Feedback welcome on the wording. Thanks. ---------- assignee: -> docs@python components: +Documentation nosy: +docs@python versions: +Python 3.8 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue34580> _______________________________________

1 0

[issue34549] unittest docs could use another header
by Nick 06 Sep '18

06 Sep '18

New submission from Nick <manifes7o(a)gmail.com>: I find myself reaching for a list of the assert methods in the unittest library often. There are several methods but no clear way to link a URL via a header if you intend to bookmark or send it out. I have been using the method above the section in the meantime https://docs.python.org/3.7/library/unittest.html#unittest.TestCase.debug Would be great to have an `Assert Methods` header or some other equivalent. Thank you! ---------- assignee: docs@python components: Documentation messages: 324385 nosy: docs@python, napsterinblue priority: normal severity: normal status: open title: unittest docs could use another header type: enhancement versions: Python 3.7 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue34549> _______________________________________

3 5

Doc - 1.4. The Module’s Method Table and Initialization Function
by Johannes Müller 04 Sep '18

04 Sep '18

Hi, It seems like there is a small bug (or a missing detail) in the example code of the main function in Chapter "1.4. The Module’s Method Table and Initialization Function <https://docs.python.org/3/extending/extending.html#the-module-s-method-tabl…>". In the example, the return value of PyImport_ImportModule is discarded. In my experience, this may cause a crash in Py_Finalize() if the return value is discarded and Py_DECREF is omitted. Is my assumption correct that the example should actually be extended as follows? > ... > > /* Optionally import the module; alternatively, > import can be deferred until the embedded script > imports it. */ > PyObject *pModule = PyImport_ImportModule("spam"); > > ... > > Py_DECREF(pModule); > if (Py_FinalizeEx() < 0) { > exit(120); > } > PyMem_RawFree(program); > return 0; Note that I assume that the developer is supposed to keep a reference of the module object, such that the garbage collection cannot free the module, while the program is using the interpreter. Is this necessary? Or can I be 100% sure that the imported module will exist and do it's job until I call Py_FinalizeEx(), even if I don't keep a reference of the module, i.e., if I decref pModule directly after importing it? Regards, Johannes PS: In my case, the module is a Python output redirector, that's not explicitly used in the executed Python scripts and snippets. It is just supposed to redirect the output of the entire Python interpreter until I call Py_Finalize.

1 0

Bug in the glossary documentation.
by Никита Люшненко 04 Sep '18

04 Sep '18

Term struct sequence, the word "ether" repeated 2 times in the row here: "elements can either be accessed either by index or as an attribute." I suggest: "elements can ether be accessed by index or as an attribute." Link - https://docs.python.org/3/glossary.html#term-struct-sequence Sorry for poor English. -- Sincerely, ZooDSS ZooDSS(a)yandex.ru

2 2

[issue13081] Crash in Windows with unknown cause
by Zachary Ware 04 Sep '18

04 Sep '18

Zachary Ware <zachary.ware(a)gmail.com> added the comment: As far as I can tell, this was an application bug in multiprocessing cleanup 7 years ago. I'm not sure there's really even anything to add to the docs for this, but if anyone disagrees or produces a patch, please reopen. ---------- nosy: +davin, pitrou, zach.ware resolution: -> works for me stage: -> resolved status: open -> closed _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue13081> _______________________________________

1 0

[issue34500] Fix ResourceWarning in difflib.py
by Mickaël Schoentgen 03 Sep '18

03 Sep '18

New submission from Mickaël Schoentgen <contact(a)tiger-222.fr>: Fix 2 unclosed files in the documentation and the difflib.py script: diff.py:77: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpnby8pemn' mode='U' encoding='UTF-8'> fromlines = open(fromfile, 'U').readlines() diff.py:78: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpnby8pemm' mode='U' encoding='UTF-8'> self.tolines = open(tofile, 'U').readlines() ---------- assignee: docs@python components: Demos and Tools, Documentation messages: 324071 nosy: David Bieber, Tiger-222, docs@python priority: normal severity: normal status: open title: Fix ResourceWarning in difflib.py versions: Python 2.7 _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue34500> _______________________________________

2 5

[issue23460] Decimals do not obey ':g' exponential notation formatting rules
by Lorenz Mende 02 Sep '18

02 Sep '18

Lorenz Mende <lorenz.mende(a)gmail.com> added the comment: I want to bring this issue up again. As the others correctly stated, either the documentation or the implementation of Decimal 'g' formatting is incorrect. For floats the implementation suits the docu: >>> '{:g}'.format(0.00001) '1e-05' For decimals: >>> '{:g}'.format(decimal.Decimal('0.00001')) '0.00001' As there is a deviation between documentation and implementation, I advise to modify one of both. ---------- nosy: +LorenzMende _______________________________________ Python tracker <report(a)bugs.python.org> <https://bugs.python.org/issue23460> _______________________________________

1 0