Mailman 3 August 2014 - docs

surprisingly poor performance of random.sample(set)
by Alex Frase Aug. 13, 2014

Aug. 13, 2014

Hi, I noticed recently that running random.sample() on a set is measurably slower than doing so on a list or tuple with the same contents. I suspect that this is due to the "implementation detail" of converting the population argument to a tuple internally; that may not matter much for a single call, but when doing i.e. permutation testing, it is not unreasonable to call random.sample() on the same population 1000s of times in a tight loop. In that scenario, letting it cast the set to a tuple inside the loop makes it ~50x slower than casting once outside the loop and passing a tuple to random.sample() instead of a set. Assuming there is no easy way for random.sample() to work efficiently with a set, would it at least be possible to mention this behavior in the documentation? Right now there is no indication that the data structure used for the population makes any difference, which may lead people to just assume that random.sample() is slow, not realizing they could get a 50x boost with a single cast outside their loop. Regards, Alex Frase

2 1

Potential vulnerabilities in python (2.7.8)
by Hádrian R Aug. 13, 2014

Aug. 13, 2014

Hi, I'm Hádrien Romero Soria - @Kaiwaiata, I am a 16 year old boy, passionate about computer security, since more than 2h searching and finding various possible vulnerabilities in source code of python.. I will tell you some vulnerabilities now, if they treat me well I will tell the other.. foolish or important things? *#* unsafe use of *strcpy()*: *python.tar\python-2.7.8\modules\zipimport.c* *lines: 83: strcpy(buf, path);* *python.tar\python-2.7.8\modules\zipimport.c* *lines: **437: **strcpy(buf, path);* *python.tar\python-2.7.8\modules\zipimport.c* *lines: 704: **strcpy(path, archive);* *#* if an attacker manages to take control of '*buf, path, archive'*, may cause a *buffer overflow*, probably if which would be directed toward *.bss *it's not too dangerous but is a vulnerability. i hope answer, thanks a lot!, HádrienR - kaiwaiata.

2 1

Error in the documentation for unittest
by Daniel Obermiller Aug. 12, 2014

Aug. 12, 2014

In the documentation for unittest here https://docs.python.org/2/library/unittest.html This segment is not entirely correct > Instead of unittest.main() > <https://docs.python.org/2/library/unittest.html#unittest.main>, there > are other ways to run the tests with a finer level of control, less terse > output, and no requirement to be run from the command line. For example, > the last two lines may be replaced with: > > suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)unittest.TextTestRunner(verbosity=2).run(suite) > > Running the revised script from the interpreter or another script produces > the following output: > > > test_choice (__main__.TestSequenceFunctions) ... ok > test_sample (__main__.TestSequenceFunctions) ... ok > test_shuffle (__main__.TestSequenceFunctions) ... ok > > ---------------------------------------------------------------------- > Ran 3 tests in 0.110s > > OK > > This is not entirely accurate - in order to get that output from the interpreter or another script I had to provide the stream as well, like so unittest.TextTestRunner(sys.stdout, verbosity=2).run(suite) Regards, Dan Obermiller

1 0

[issue19055] Clarify docs for re module: why * does not match as many repetitions as possible.
by Akira Li Aug. 11, 2014

Aug. 11, 2014

Changes by Akira Li <4kir4.1i(a)gmail.com>: Added file: http://bugs.python.org/file36341/re-docs-repetitions.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue19055> _______________________________________

1 0

[issue19055] Clarify docs for re module: why * does not match as many repetitions as possible.
by Akira Li Aug. 11, 2014

Aug. 11, 2014

Changes by Akira Li <4kir4.1i(a)gmail.com>: Removed file: http://bugs.python.org/file36340/re-docs-repetitions.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue19055> _______________________________________

1 0

[issue19055] Clarify docs for re module: why * does not match as many repetitions as possible.
by Akira Li Aug. 11, 2014

Aug. 11, 2014

Akira Li added the comment: tl;dr: added patch that clarifies Python re behavior. Please, review. --- The documented behavior is not clear: why (a|ab)* is not equivalent to (a|ab)(a|ab) for aba if the docs say "as many repetitions as are possible"? And it is not obvious (it is not the only possible behavior) e.g., POSIX [1] expects the longest match, PCRE [2] group may be atomic (/possesive quantifiers), or there is ungreedy repetition: $ echo abac | grep -o '$a\|ab$* # Basic Regular Expression aba $ echo abac | grep -Eo '(a|ab)*' # Extended Regular Expression aba $ echo abac | grep -Po '(a|ab)*' # PCRE (like Python regexes) a a $ echo abac | grep -Po '(a|ab)(a|ab)' # PCRE (two repeats) aba $ echo abac | grep -Po '(a|ab)*c' # PCRE (re-eval to match the rest) abac $ echo abAc | grep -iPo '(a|ab)*+c' # PCRE (possessive quantifiers) Ac $ echo abac | grep -Po '(a|ab)*?' # PCRE (ungreedy, zero repeats) # matches nothing where BREs, EREs are defined in [1] that says: If the pattern permits a variable number of matching characters and thus there is more than one such sequence starting at that point, *the longest such sequence is matched*. and: Consistent with the whole match being the longest of the leftmost matches, each subpattern, from left to right, *shall match the longest possible string.* Python regexes work like PCRE [2] that says: The matching process tries each alternative in turn, from left to right, and *the first one that succeeds is used*. If the alternatives are within a subpattern (defined below), "succeeds" means matching the rest of the main pattern as well as the alternative in the subpattern. It explains why (a|ab)* matches only a in aba ("the first one that succeeds") and why at the same time (a|ab)*c matches abac: (a|ab) may match ab in the latter case because the main pattern would fail otherwise i.e., the advanced definition of "succeeds" is used: ""succeeds" means matching the rest of the main pattern ...". Python docs [3] are similar but do not contain the "advanced" "succeeds" definition: REs separated by ``'|'`` are tried from left to right. When one pattern completely matches, that branch is accepted. This means that once ``A`` matches, ``B`` will not be tested further, even if it would produce a longer overall match. In other words, the ``'|'`` operator is never greedy. It explains why (a|ab) matches a in aba. '*' behavior [2]: By default, the quantifiers are "greedy", that is, they match as much as possible (up to the maximum number of permitted times), without causing the rest of the pattern to fail. and again Python docs [3] are similar: ``'*'`` Causes the resulting RE to match 0 or more repetitions of the preceding RE, as many repetitions as are possible. It implies that (a|ab)* should be equivalent to (a|ab)(a|ab) to match aba -- TWO REPETITIONS ARE MORE THAN ONE: "as many repetitions as are possible". But it matches only 'a' i.e., it works as (a|ab) -- only one repetition instead of two. In reality, (a|ab)* along works like a*. I've uploaded a documentation patch that makes Python documentation for '|' more similar to PCRE and clarifies '*' definition as R. David Murray suggested in msg198126. Please, review. [1] http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html [2] http://man7.org/linux/man-pages/man3/pcrepattern.3.html [3] http://hg.python.org/cpython/file/18a311479e8b/Doc/library/re.rst ---------- components: -Regular Expressions keywords: +patch nosy: +akira title: Regular expressions: * does not match as many repetitions as possible. -> Clarify docs for re module: why * does not match as many repetitions as possible. versions: +Python 3.5 -Python 2.7, Python 3.3 Added file: http://bugs.python.org/file36340/re-docs-repetitions.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue19055> _______________________________________

1 0

[issue22145] <> in parser spec but not lexer spec
by François-René Rideau Aug. 11, 2014

Aug. 11, 2014

New submission from François-René Rideau: As another follow up to http://bugs.python.org/issue21972 <> is mentioned in the parser spec: https://docs.python.org/3.5/reference/grammar.html But not in the lexer spec: https://docs.python.org/3.5/reference/lexical_analysis.html Either is a mistake. I suggested in issue 21972 that the former was the bug, because it referred to a joke PEP and because <> doesn't actually work as a comparator in Python 3.4. The response by loewis was that the parser documentation was correct — well then the lexer documentation is incorrect. ---------- assignee: docs@python components: Documentation messages: 224856 nosy: François-René.Rideau, docs@python, loewis priority: normal severity: normal status: open title: <> in parser spec but not lexer spec type: enhancement versions: Python 3.5 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22145> _______________________________________

3 3

[issue22112] '_UnixSelectorEventLoop' object has no attribute 'create_task'
by Daniel Greenfeld Aug. 10, 2014

Aug. 10, 2014

New submission from Daniel Greenfeld: Problem ======== The documentation on asyncio provides an example of a parallel execution of tasks. The code is at: https://docs.python.org/3/library/asyncio-task.html#example-parallel-execut… ``` python import asyncio @asyncio.coroutine def factorial(name, number): f = 1 for i in range(2, number+1): print("Task %s: Compute factorial(%s)..." % (name, i)) yield from asyncio.sleep(1) f *= i print("Task %s: factorial(%s) = %s" % (name, number, f)) loop = asyncio.get_event_loop() tasks = [ loop.create_task(factorial("A", 2)), loop.create_task(factorial("B", 3)), loop.create_task(factorial("C", 4))] loop.run_until_complete(asyncio.wait(tasks)) loop.close() ``` Unfortunately, when I try to run this sample code on Python 3.4.1, I run into an error. Specifically, the `loop.create_task()` method does not exist: ``` python Traceback (most recent call last): File "what_me_asynicio.py", line 14, in <module> loop.create_task(factorial("A", 2)), AttributeError: '_UnixSelectorEventLoop' object has no attribute 'create_task' ``` When I perform a dir() on the `loop` object, no `create_task` item is in the result. System Information ==================== Python Version: 3.4.1 Operating System: OSX 10.9.3 (Also confirmed on Python 3.4.0 in Ubuntu 14.04) ---------- assignee: docs@python components: Documentation, Macintosh, asyncio messages: 224374 nosy: docs@python, gvanrossum, haypo, pydanny, yselivanov priority: normal severity: normal status: open title: '_UnixSelectorEventLoop' object has no attribute 'create_task' versions: Python 3.4 _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22112> _______________________________________

4 10

[issue22174] property doc fixes
by diana Aug. 10, 2014

Aug. 10, 2014

New submission from diana: The property docs are a bit funky. https://docs.python.org/3/library/functions.html#property 1) docstrings have zero to do with making a read-only property. It currently says: "If given, doc will be the docstring of the property attribute. Otherwise, the property will copy fget‘s docstring (if it exists). This makes it possible to create read-only properties easily using property() as a decorator:" 2) The 'then' in this sentence is awkward (and incorrect English). "If then c is an instance of C, c.x will invoke the getter, c.x = value will invoke the setter and del c.x the deleter." 3) This sentence is missing a beginning. "turns the voltage() method into a “getter” for a read-only attribute with the same name." 4) This sentence has an extra comma (after del'ing): "fget is a function for getting an attribute value, likewise fset is a function for setting, and fdel a function for del’ing, an attribute." Attached is a patch with minimal changes to the property docs, addressing the above four issues. Okay, that's not entirely true -- I did add an example usage because the current docs don't actually show using a property attribute. >>> p = Parrot() >>> p.voltage 100000 I've also attached an "after" screenshot of the docs in case it helps with review. Cheers, --diana ---------- assignee: docs@python components: Documentation files: property_docs.patch keywords: patch messages: 225086 nosy: diana, docs@python priority: normal severity: normal status: open title: property doc fixes versions: Python 3.5 Added file: http://bugs.python.org/file36318/property_docs.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22174> _______________________________________

4 9

[issue22180] operator.setitem example no longer works in Python 3 due to lazy map
by Michael Williamson Aug. 10, 2014

Aug. 10, 2014

New submission from Michael Williamson: The Python docs for the operator module include an example using map and setitem to "Build a dictionary that maps the ordinals from 0 to 255 to their character equivalents.": d = {} keys = range(256) vals = map(chr, keys) map(operator.setitem, [d]*len(keys), keys, vals) Since map is lazy since Python 3, the dictionary d is never actually changed in this example. I'm not entirely sure what the idiomatic way to fix the example is since it strikes me as being fairly unidiomatic to begin with, but the simplest would be to call list on the result of map to force evaluation (patch attached). ---------- assignee: docs@python components: Documentation files: doc-operator-example.patch keywords: patch messages: 225141 nosy: docs@python, mwilliamson priority: normal severity: normal status: open title: operator.setitem example no longer works in Python 3 due to lazy map versions: Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file36333/doc-operator-example.patch _______________________________________ Python tracker <report(a)bugs.python.org> <http://bugs.python.org/issue22180> _______________________________________

5 6