Mailman 3 March 2015 - Python-Dev

str.lstrip bug?
by lou xiao 10 Mar '15

10 Mar '15

I find a bug in str.lstrip, when i call str.lstrip, i get this result. tiny➜ ~ python Python 2.7.5+ (default, Feb 27 2014, 19:37:08) [GCC 4.8.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> a='device_info' >>> a.lstrip('device') '_info' >>> a.lstrip('device_') 'nfo' >>> tiny➜ ~ uname -a Linux tinymint 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:20:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux tiny➜ ~

5 5

Re: [Python-Dev] Thoughts on running Python 3.5 on Windows (path, pip install --user, etc)
by Steve Dower 10 Mar '15

10 Mar '15

Paul Moore wrote: > I just thought I'd give installing Python 3.5 a go on my PC, now that > 3.5a2 has come out. I didn't get very far (see earlier message) but it prompted > me to think about how I'd use it, and what habits I'd need to change. > > I'd suggest that the "what's new in 3.5" document probably needs a section on > the new installer that explains this stuff... This is true. Right now I'm in experimentation mode, and being more aggressive about changing things than is probably a good idea (because it solicits feedback like this :) ). When things settle down I expect to end up closer to where we started, so there's not a huge amount of value in writing it all up right now. I'll get there. > First of all, I always use "all users" installs, so I have Python in "Program > Files" now. As a result, doing "pip install foo" requires elevation. As that's a > PITA, I probably need to switch to using "pip install --user". All that's fine, > and from there "py -3.5" works fine, as does "py -3.5 -m foo". But even if it > is, not every entry point has a corresponding "-m" invocation (pygments' > pygmentize command doesn't seem to, for example) I know you're already involved in this Paul, but for everyone else there's a big discussion going on at https://github.com/pypa/pip/issues/1668 about changing pip's default behaviour to handle falling back to --user automatically. > But suppose I want to put Python 3.5 on my PATH. The installer has an "add > Python to PATH" checkbox, but (if I'm reading the WiX source right, I didn't > select that on install) that doesn't add the user directory. So I need to add > that to my PATH. Is that right? And of course, that means I need to *know* the > user site directory ($env:LOCALAPPDATA\Python\Python35\Scripts), correct? Correct. There's no way to add a per-user directory to PATH from an all-users installation (except for a few approaches that I expect/hope would trigger malware detectors...) > Maybe the answer is that we simply start recommending that everyone on Windows > uses per-user installs. It makes little difference to me (beyond the fact that > when I want to look at the source of something in the stdlib, the location of > the file is a lot harder to remember than C:\Apps\Python34\Lib\whatever.py) but > I doubt it's what most people will expect. I'm okay with this. Installing for all users is really something that could be considered an advanced option rather than the default, especially since the aim (AIUI) of the all-users install is to pretend that Python was shipped with the OS. (I'd kind of like to take that further by splitting things more sensibly between Program Files, Common Files and System32, but there's very little gain from that and much MUCH pain as long as people are still expecting C:\PythonXY installs...) Cheers, Steve

5 6

Windows installer - File associations in "per user" installs
by Paul Moore 10 Mar '15

10 Mar '15

On 9 March 2015 at 15:37, Steve Dower <Steve.Dower(a)microsoft.com> wrote: >> Maybe the answer is that we simply start recommending that everyone on Windows >> uses per-user installs. It makes little difference to me (beyond the fact that >> when I want to look at the source of something in the stdlib, the location of >> the file is a lot harder to remember than C:\Apps\Python34\Lib\whatever.py) but >> I doubt it's what most people will expect. > > I'm okay with this. Installing for all users is really something that could be considered an advanced option rather than the default, especially since the aim (AIUI) of the all-users install is to pretend that Python was shipped with the OS. (I'd kind of like to take that further by splitting things more sensibly between Program Files, Common Files and System32, but there's very little gain from that and much MUCH pain as long as people are still expecting C:\PythonXY installs...) I've just tried a per-user install of Python 3.5a2. The machine in question previously had (and still has) a system install of 3.4, with "Make this Python the default" selected (so the .py extension is associated with that version and specifically the 3.4 launcher). I didn't get the option to associate .py files with 3.5 (there's *no way* I'd consider that to be advanced usage - if I'm installing Python, why wouldn't I want to associate it with .py files [1]) and I still seem to have .py associated with the 3.4 launcher, not the 3.5 one that's in my %APPDATA% folder. >cmd /c assoc .py .py=Python.File >cmd /c ftype python.file python.file="C:\WINDOWS\py.exe" "%1" %* I'm happy if a per-user install of 3.5 makes a per-user filetype association (assuming such a thing is possible, I've never tried it before) but it's absolutely not OK if we're planning on recommending an install type that doesn't create the right association. Paul [1] Given that I have 3.4 and am installing an experimental 3.5 version, it's not actually at all clear cut which version I want as my default. In all honesty, I don't think this decision is actually something that should be defaulted. Maybe the "don't make the user make any choices in the default selection" approach has gone a little too far here?

2 1

PEP 471 Final: os.scandir() merged into Python 3.5
by Victor Stinner 10 Mar '15

10 Mar '15

Hi, FYI I commited the implementation of os.scandir() written by Ben Hoyt. I hope that it will be part of Python 3.5 alpha 2 (Ben just sent the final patch today). Please test this new feature. You may benchmark here. http://bugs.python.org/issue22524 contains some benchmark tools and benchmark results of older versions of the patch. The implementation was tested on Windows and Linux. I'm now watching for buildbots to see how other platforms like os.scandir(). Bad news: OpenIndiana doesn't support d_type: the dirent structure has no d_type field. I already fixed the implementation to support this case. os.scandir() is still useful on OpenIndiana, because the stat result is cached in a DirEntry, so only one syscall is required, instead of multiple, when multiple DirEntry methods are called (ex: entry.is_dir() and not entry.is_symlink()). Victor

7 10

Re: [Python-Dev] boxing and unboxing data types
by Serhiy Storchaka 10 Mar '15

10 Mar '15

понеділок, 09-бер-2015 10:18:50 ви написали: > On Mon, Mar 9, 2015 at 10:10 AM, Serhiy Storchaka <storchaka(a)gmail.com> wrote: > > понеділок, 09-бер-2015 09:52:01 ви написали: > > > On Mon, Mar 9, 2015 at 2:07 AM, Serhiy Storchaka <storchaka(a)gmail.com> > > > > And to be ideal drop-in replacement IntEnum should override such methods > > > > as __eq__ and __hash__ (so it could be used as mapping key). If all methods > > > > should be overridden to quack as int, why not take an int? > > > > > > You're absolutely right that if *all the methods should be overrriden to > > > quack as int, then you should subclass int (the Liskov substitution > > > principle). But all methods should not be overridden — mainly the methods > > > you overrode in your patch should be exposed. Here is a list of methods on > > > int that should not be on IntFlags in my opinion (give or take a couple): > > > > > > __abs__, __add__, __delattr__, __divmod__, __float__, __floor__, > > > __floordiv__, __index__, __lshift__, __mod__, __mul__, __pos__, __pow__, > > > __radd__, __rdivmod__, __rfloordiv__, __rlshift__, __rmod__, __rmul__, > > > __round__, __rpow__, __rrshift__, __rshift__, __rsub__, __rtruediv__, > > > __sub__, __truediv__, __trunc__, conjugate, denominator, imag, numerator, > > > real. > > > > > > I don't think __index__ should be exposed either since are you really going > > > to slice a list using IntFlags? Really? > > > > Definitely __index__ should be exposed. __int__ is for lossy conversion to int > > (as in float). __index__ is for lossless conversion. > > Is it? __index__ promises lossless conversion, but __index__ is *for* > indexing. I spite of its name it is for any lossless conversion. > > __add__ should be exposed because some code can use + instead of | for > > combining flags. But it shouldn't preserve the type, because this is not > > recommended way. > > I think it should be blocked because it can lead to all kinds of weird > bugs. If the flag is already set and you add it a copy, it silently spills > over into other flags. This is a mistake that a good interface prevents. I think this is a case when backward compatibility has larger weight. > > For the same reason I think __lshift__, __rshift__, __sub__, > > __mul__, __divmod__, __floordiv__, __mod__, etc should be exposed too. So the > > majority of the methods should be exposed, and there is a risk that we loss > > something. > > I totally disagree with all of those. > > > For good compatibility with Python code IntFlags should expose also > > __subclasscheck__ or __subclasshook__. And when we are at this point, why not > > use int subclass? > > Here's another reason. What if someone wants to use an IntFlags object, > but wants to use a fixed width type for storage, say numpy.int32? Why > shouldn't they be able to do that? By using composition, you can easily > provide such an option. You can design abstract interface Flags that can be combined with int or other type. But why you want to use numpy.int32 as storage? This doesn't save much memory, because with composition the IntFlags class weighs more than int subclass.

3 5

Re: [Python-Dev] Tunning binary insertion sort algorithm in Timsort.
by nha pham 09 Mar '15

09 Mar '15

I do not know exactly, one thing I can imagine is: it turns the worst case of binary insertion sort to best case. With sorted array in range of 32 or 64 items, built from zero element. The new element you put into the sorted list has a high chance of being the smallest or the the highest of the sorted list (or nearly highest or nearly smallest) If that case happen, the old binary insertion sort will have the investigate all the list, while with my idea, it just have to compare more 1-2 times. I will try to run more test an more thinking to make sure though. On Mon, Mar 9, 2015 at 11:48 AM, nha pham <phqnha(a)gmail.com> wrote: > I do not know exactly, one thing I can imagine is: it turns the worst case > of binary insertion sort to best case. > With sorted array in range of 32 or 64 items, built from zero element. The > new element you put into the sorted list has a high chance of being the > smallest or the the highest of the sorted list (or nearly highest or nearly > smallest) > > If that case happen, the old binary insertion sort will have the > investigate all the list, while with my idea, it just have to compare more > 1-2 times. > I will try to run more test an more thinking to make sure though. > > > > On Mon, Mar 9, 2015 at 10:39 AM, Isaac Schwabacher <ischwabacher(a)wisc.edu> > wrote: > >> On 15-03-08, nha pham >> wrote: >> > >> > We can optimize the TimSort algorithm by optimizing its binary >> insertion sort. >> > >> > The current version of binary insertion sort use this idea: >> > >> > Use binary search to find a final position in sorted list for a new >> element X. Then insert X to that location. >> > >> > I suggest another idea: >> > >> > Use binary search to find a final postion in sorted list for a new >> element X. Before insert X to that location, compare X with its next >> element. >> > >> > For the next element, we already know if it is lower or bigger than X, >> so we can reduce the search area to the left side or on the right side of X >> in the sorted list. >> >> I don't understand how this is an improvement, since with binary search >> the idea is that each comparison cuts the remaining list to search in half; >> i.e., each comparison yields one bit of information. Here, you're spending >> a comparison to cut the list to search at the element you just inserted, >> which is probably not right in the middle. If you miss the middle, you're >> getting on average less than a full bit of information from your >> comparison, so you're not reducing the remaining search space by as much as >> you would be if you just compared to the element in the middle of the list. >> >> > I have applied my idea on java.util. ComparableTimSort.sort() and >> testing. The execute time is reduced by 2%-6% with array of random integer. >> >> For all that, though, experiment trumps theory... >> >> > Here is detail about algorithm and testing: >> https://github.com/nhapq/Optimize_binary_insertion_sort >> > >> > Sincerely. >> > >> > phqnha >> > >

2 1

Tunning binary insertion sort algorithm in Timsort.
by nha pham 09 Mar '15

09 Mar '15

We can optimize the TimSort algorithm by optimizing its binary insertion sort. The current version of binary insertion sort use this idea: Use binary search to find a final position in sorted list for a new element X. Then insert X to that location. I suggest another idea: Use binary search to find a final postion in sorted list for a new element X. Before insert X to that location, compare X with its next element. For the next element, we already know if it is lower or bigger than X, so we can reduce the search area to the left side or on the right side of X in the sorted list. I have applied my idea on java.util. ComparableTimSort.sort() and testing. The execute time is reduced by 2%-6% with array of random integer. Here is detail about algorithm and testing: https://github.com/nhapq/Optimize_binary_insertion_sort Sincerely. phqnha

5 4

boxing and unboxing data types
by Ethan Furman 09 Mar '15

09 Mar '15

When data is passed from Python to a native library (such as in an O/S call), how does the unboxing of data types occur? For a specific instance, os.open allows an integer whose various bits express desired behavior as `flags` -- if flags is 1, for example, the file is open write-only. If I pass an int-like object to os.open, __int__ is called; if I pass a thin wrapper over int to os.open, __int__ is not called. So the real question: anywhere in Python where an int is expected (for lower-level API work), but not directly received, should __int__ (or __index__) be called? and failure to do so is a bug? Here's my simple test code: class Wrap: def __init__(self, value): self.value = value def __int__(self): print('__int__') return self.value def __index__(self): print('__index__') return self.value class Thin(int): def __int__(self): print('__int__') return super().__int__() def __index__(self): print('__index__') return super().__index__() two = Wrap(2) [0, 1, 2][two] # __index__ # 2 import struct struct.pack('i', two) # __index__ # b'\x02\x00\x00\x00' t = Thin(1) huh = os.open('blah.txt', t) # Traceback (most recent call last): # File "<stdin>", line 1, in <module> # FileNotFoundError: [Errno 2] No such file or directory: 'blah.txt' -- ~Ethan~

7 12

Thoughts on running Python 3.5 on Windows (path, pip install --user, etc)
by Paul Moore 09 Mar '15

09 Mar '15

I just thought I'd give installing Python 3.5 a go on my PC, now that 3.5a2 has come out. I didn't get very far (see earlier message) but it prompted me to think about how I'd use it, and what habits I'd need to change. I'd suggest that the "what's new in 3.5" document probably needs a section on the new installer that explains this stuff... First of all, I always use "all users" installs, so I have Python in "Program Files" now. As a result, doing "pip install foo" requires elevation. As that's a PITA, I probably need to switch to using "pip install --user". All that's fine, and from there "py -3.5" works fine, as does "py -3.5 -m foo". But even if it is, not every entry point has a corresponding "-m" invocation (pygments' pygmentize command doesn't seem to, for example) But suppose I want to put Python 3.5 on my PATH. The installer has an "add Python to PATH" checkbox, but (if I'm reading the WiX source right, I didn't select that on install) that doesn't add the user directory. So I need to add that to my PATH. Is that right? And of course, that means I need to *know* the user site directory ($env:LOCALAPPDATA\Python\Python35\Scripts), correct? It feels to me like this might be a frustrating step backwards for Windows users who have recently (with the arrival of ensurepip) got to the point where they can just run Python with "Add to path" and then simply do pip install pygments pygmentize --help Maybe the answer is that we simply start recommending that everyone on Windows uses per-user installs. It makes little difference to me (beyond the fact that when I want to look at the source of something in the stdlib, the location of the file is a lot harder to remember than C:\Apps\Python34\Lib\whatever.py) but I doubt it's what most people will expect. I'm completely OK with the idea that we move to "Program Files" as a default location. And I have no real issue with Steve's position that the "Add to Path" option has issues that can't really be solved because of the way Windows constructs the PATH. But I know there have been a lot of people frustrated by the complicated instructions needed to get something like pygmentize working, for whom getting pip in 2.7 and 3.4 was a major improvement. So I think we need *some* documentation helping them deal with what could well seem like a step backwards in Python 3.5... If I knew what the best (recommended) answer was, I'd be happy to write it up. But I'm not sure I do (after all, the target audience is people for whom "Add C:\PythonXY\Scripts to your PATH" was a problem in the pre-3.4 days). Should I raise this as a bug against the 3.5 documentation? If so, should it be a release blocker for the final release? Paul

2 2

[RELEASED] Python 3.5.0a2 is now available
by Larry Hastings 09 Mar '15

09 Mar '15

On behalf of the Python development community and the Python 3.5 release team, I'm thrilled to announce the availability of Python 3.5.0a2. Python 3.5.0a2 is the second alpha release of Python 3.5, which will be the next major release of Python. Python 3.5 is still under heavy development, and is far from complete. This is a preview release, and its use is not recommended for production settings. Two important notes for Windows users about Python 3.5.0a2: * If you have previously installed Python 3.5.0a1, you must manually uninstall it before installing Python 3.5.0a2 (issue23612). * If installing Python 3.5.0a2 as a non-privileged user, you may need to escalate to administrator privileges to install an update to your C runtime libraries. You can find Python 3.5.0a2 here: https://www.python.org/downloads/release/python-350a2/ Happy hacking, //arry/

4 4

Asking for review for Windows issues 21518 and 22080
by Claudiu Popa 08 Mar '15

08 Mar '15

Hello, The winreg module has a function for loading a registry key under another registry key, called winreg.LoadKey. Unfortunately, the module doesn't provide a way to unload that key after the user finishes operating with it. There's a patch [1] for exporting the RegUnloadKey [2] API in winreg module as winreg.UnloadKey, similar to how RegLoadKey is exported as winreg.LoadKey. The patch is helped by another one [3], which provides a new module, test.support.windows_helper, for handling various issues on the Windows platform, such as acquiring or releasing a privilege. Unfortunately, it seems there's a dearth of reviewers for this platform. Could someone knowledgeable with Windows be so kind to review these patches? They could make a good addition for Python 3.5. Thank you very much. [1] http://bugs.python.org/issue21518 - Expose RegUnloadKey in winreg [2] https://msdn.microsoft.com/en-us/library/windows/desktop/ms724924%28v=vs.85… [3] http://bugs.python.org/issue22080 - Add windows_helper module helper /Claudiu

1 0

Optimize binary insertion sort algorithm in Timsort.
by nha pham 07 Mar '15

07 Mar '15

This describes an optimization for "binary insertion sort" (BINSORT for short). BINSORT has been implemented in Python, CyThon, and Timsort (the default Array.sort() in JAVA SE 7 and JAVA SE 8) I have read the BINSORT in Timsort http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b1… and I think that I can make a little more optimization. ================= The old BINSORT: The basic idea is to use binary search on the sorted list to find a final position for a new element X, then insert X to the sorted list. [SORTED_LIST], [X in UNSORTED_LIST] // pick X in UNSORTED_LIST index = binarySeach([SORTED_LIST]) // use binary search to find appropriate location for X in SORTED_LIST [SORTED_LIST].add(index, X) // insert X to index location ================== New BINSORT: [SORTED_LIST], [A] // A is an UNSORTED_LIST j = compare(A[i], A[i+1]) // pick A[i], compare to next element index = binarySeach([SORTED_LIST], A[i]) // use binary search to find // appropriate location for A[i] in SORTED_LIST, and remember the index [SORTED_LIST].add(index, A[i]) // insert A[i] to index location // Now for A[+1], where already know where it should be, compare to A[i] if j >= 0: // A[i] > A[i+1], so A[i+1] will be in the right side of A[i] // We only have to search on a reduced Array: index = binarySearch(SORTED_LIST[index : length(SORTED_LIST)], A[i+1]) else: // we search on the left side of of A[i] index = binarySearch(SORTED_LIST[0 : index], A[i+1]) [SORTED_LIST].add(index, A[i+1]) // insert A[i+1] to index location //repeat the loop ================== Run test. Intuitively, new BINSORT will be better if the Array become bigger, because it reduce the array to search with the cost of only 1 more comparison. I only care about small array, with the length < 100 (we have known that in Timsort, the list is divided to chunks with length 64, then apply BINSORT to them). So I make a big Array, divide them, and apply new BINSORT in each chunk, then compare to OLD BINSORT. The source code is in the bottom of this document. Here is the results: cpuinfo: model name : Intel(R) Core(TM) i3 CPU M 350 @ 2.27GHz stepping : 2 microcode : 0xc cpu MHz : 933.000 cache size : 3072 KB ----- random array: ARRAY_SIZE: 1000000 CHUNK_SIZE: 100 DATA: randint(0, 1000000) OLD BINSORT: 81.45754 new BINSORT: 5.26754 RATIO: (OLD - new) / new = 14.464 --- incremental array: ARRAY_SIZE: 1000000 CHUNK_SIZE: 100 DATA: range(0, 1000000) OLD BINSORT: 81.87927 new BINSORT: 5.41651 RATIO: (OLD - new) / new = 14.11659 --- decremental array: ARRAY_SIZE: 1000000 CHUNK_SIZE: 100 DATA: range(0, 1000000) OLD BINSORT: 81.45723 new BINSORT: 5.09823 RATIO: (OLD - new) / new = 14.97753 ---- all equal array: ARRAY_SIZE: 1000000 CHUNK_SIZE: 100 DATA: 50000 OLD BINSORT: 40.46027 new BINSORT: 5.41221 RATIO: (OLD - new) / new = 6.47573 ==================== What should we do next: - Tuning my test code (I have just graphed it). - Test other cases, and bigger array (my laptop cannot handle array more than 10^6.) - Modifying Timsort in java.util. and test if it is better. ==================== My test code, written by Python. from timeit import Timer setup ="""\ import bisect from random import randint from timeit import Timer SIZE = 1000000 CHUNK = 100 NUM_CHUNK = SIZE/CHUNK data = [] data2 = [] data3 = [] for i in range(0,SIZE): data.append(randint(0,1000000)) #data.append(i) #data = data[::-1] """ sample ="""\ for j in range(0,NUM_CHUNK): low = CHUNK*j high= low + CHUNK data2.append(data[low]) index = low for i in range(low,high): x = data[i] index = bisect.bisect_right(data2[low:], x, low, len(data2) - low-1) data2.insert(index, x) """ new ="""\ for j in range(0,NUM_CHUNK): low = CHUNK*j high= low + CHUNK data3.append(data[low]) index = low for i in range(low,high): x = data[i] if x >= data[i-1]: index = bisect.bisect_right(data3[low:len(data3) - low-1], x, index, len(data3) - low-1) else: index = bisect.bisect_right(data3[low:index], x, low, index) data3.insert(index, x) """ t2 = Timer(stmt = sample, setup=setup) a = t2.timeit(1) print a t3 = Timer(stmt = new, setup=setup) b = t3.timeit(1) print b print (str((a - b)/b)) ==================================== Nha Pham Mar 07 2015

1 0

Summary of Python tracker Issues
by Python tracker 06 Mar '15

06 Mar '15

ACTIVITY SUMMARY (2015-02-27 - 2015-03-06) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4804 (+16) closed 30555 (+45) total 35359 (+61) Open issues with patches: 2260 Issues opened (47) ================== #22079: Ensure in PyType_Ready() that base class of static type is sta http://bugs.python.org/issue22079 reopened by doko #23538: New Windows installer in 3.5.0a1 breaks compatibility with Win http://bugs.python.org/issue23538 opened by Link Mauve #23539: Content-length not set for HTTP methods expecting body when bo http://bugs.python.org/issue23539 opened by demian.brecht #23540: Proposal for asyncio: SubprocessTransport.detach() to detach a http://bugs.python.org/issue23540 opened by martius #23542: Update PEP 476 for using urllib2.build_opener() http://bugs.python.org/issue23542 opened by Shakeel Mohamed #23543: encoding error trying to save string to file http://bugs.python.org/issue23543 opened by Gravitania #23544: IDLE hangs when selecting Stack View with debug active http://bugs.python.org/issue23544 opened by Andrew.Harrington #23545: Turn on extra warnings on GCC http://bugs.python.org/issue23545 opened by serhiy.storchaka #23546: windows, IDLE and pep 397 http://bugs.python.org/issue23546 opened by Liam.Marsh #23549: heapq docs should be more precise about how to access the smal http://bugs.python.org/issue23549 opened by eli.bendersky #23550: Add to unicodedata a function to query the "Quick_Check" prope http://bugs.python.org/issue23550 opened by Hammerite #23551: IDLE to provide menu options for using PIP http://bugs.python.org/issue23551 opened by rhettinger #23552: Have timeit warn about runs that are not independent of each o http://bugs.python.org/issue23552 opened by rhettinger #23553: Reduce the number of comparisons for range checking. http://bugs.python.org/issue23553 opened by rhettinger #23554: EchoServerClientProtocol class should be called EchoServerProt http://bugs.python.org/issue23554 opened by gc #23555: behavioural change in argparse set_defaults in python 2.7.9 http://bugs.python.org/issue23555 opened by doko #23556: Scope for raise without argument is different in Python 2 and http://bugs.python.org/issue23556 opened by a3nm #23557: Misc/SpecialBuilds.txt contains outdated information about PYM http://bugs.python.org/issue23557 opened by Jakub Klama #23560: Group the docs of similar methods in stdtypes.rst http://bugs.python.org/issue23560 opened by ezio.melotti #23564: Patch fixing sanity check for ordered fd sequence in _posixsub http://bugs.python.org/issue23564 opened by Hisham Muhammad #23565: local_clear walks the list of threads without holding head_loc http://bugs.python.org/issue23565 opened by stutzbach #23566: RFE: faulthandler.register() should support file descriptors http://bugs.python.org/issue23566 opened by haypo #23567: os.stat() tuple access vs named attribute access int vs float http://bugs.python.org/issue23567 opened by gregory.p.smith #23568: unittest.mock.MagicMock doesn't support __rdivmod__ http://bugs.python.org/issue23568 opened by zkrynicki #23570: Change "with subprocess.Popen():" (context manager) to ignore http://bugs.python.org/issue23570 opened by haypo #23571: Raise SystemError if a function returns a result with an excep http://bugs.python.org/issue23571 opened by haypo #23572: functools.singledispatch fails when "not BaseClass" is True http://bugs.python.org/issue23572 opened by Sergio Pascual #23573: Avoid redundant allocations in str.find and like http://bugs.python.org/issue23573 opened by serhiy.storchaka #23574: datetime: support leap seconds http://bugs.python.org/issue23574 opened by haypo #23575: MIPS64 needs ffi's n32.S http://bugs.python.org/issue23575 opened by Simon Hoinkis #23577: Add tests for wsgiref.validate http://bugs.python.org/issue23577 opened by ashkop #23578: struct.pack error messages do not indicate which argument was http://bugs.python.org/issue23578 opened by alynn #23579: Amazon.com links http://bugs.python.org/issue23579 opened by Edrie Ddrie #23581: unittest.mock.MagicMock doesn't support matmul (@) operator http://bugs.python.org/issue23581 opened by zkrynicki #23583: IDLE: printing unicode subclasses broken (again) http://bugs.python.org/issue23583 opened by mjpieters #23584: test_doctest lineendings fails in verbose mode http://bugs.python.org/issue23584 opened by jbeck #23585: patchcheck doesn't depend on all http://bugs.python.org/issue23585 opened by rbcollins #23587: asyncio: use the new traceback.TracebackException class http://bugs.python.org/issue23587 opened by haypo #23588: Errno conflicts in ssl.SSLError http://bugs.python.org/issue23588 opened by Ben.Darnell #23589: Redundant sentence in FAQ http://bugs.python.org/issue23589 opened by fossilet #23591: Add IntFlags http://bugs.python.org/issue23591 opened by serhiy.storchaka #23592: SIGSEGV on interpreter shutdown, with daemon threads running w http://bugs.python.org/issue23592 opened by A. Skrobov #23593: Update Windows and OS X installer OpenSSL to 1.0.2 http://bugs.python.org/issue23593 opened by ned.deily #23595: Split the math module into _math (C) + math (py) http://bugs.python.org/issue23595 opened by haypo #23596: gzip argparse interface http://bugs.python.org/issue23596 opened by Antony.Lee #23597: Allow easy display of local variables in log messages? http://bugs.python.org/issue23597 opened by ncoghlan #23598: No backspace on curses on iTerm (mac) http://bugs.python.org/issue23598 opened by ragreener Most recent 15 issues with no replies (15) ========================================== #23598: No backspace on curses on iTerm (mac) http://bugs.python.org/issue23598 #23591: Add IntFlags http://bugs.python.org/issue23591 #23588: Errno conflicts in ssl.SSLError http://bugs.python.org/issue23588 #23585: patchcheck doesn't depend on all http://bugs.python.org/issue23585 #23584: test_doctest lineendings fails in verbose mode http://bugs.python.org/issue23584 #23581: unittest.mock.MagicMock doesn't support matmul (@) operator http://bugs.python.org/issue23581 #23578: struct.pack error messages do not indicate which argument was http://bugs.python.org/issue23578 #23577: Add tests for wsgiref.validate http://bugs.python.org/issue23577 #23575: MIPS64 needs ffi's n32.S http://bugs.python.org/issue23575 #23573: Avoid redundant allocations in str.find and like http://bugs.python.org/issue23573 #23572: functools.singledispatch fails when "not BaseClass" is True http://bugs.python.org/issue23572 #23568: unittest.mock.MagicMock doesn't support __rdivmod__ http://bugs.python.org/issue23568 #23565: local_clear walks the list of threads without holding head_loc http://bugs.python.org/issue23565 #23560: Group the docs of similar methods in stdtypes.rst http://bugs.python.org/issue23560 #23557: Misc/SpecialBuilds.txt contains outdated information about PYM http://bugs.python.org/issue23557 Most recent 15 issues waiting for review (15) ============================================= #23598: No backspace on curses on iTerm (mac) http://bugs.python.org/issue23598 #23596: gzip argparse interface http://bugs.python.org/issue23596 #23595: Split the math module into _math (C) + math (py) http://bugs.python.org/issue23595 #23593: Update Windows and OS X installer OpenSSL to 1.0.2 http://bugs.python.org/issue23593 #23591: Add IntFlags http://bugs.python.org/issue23591 #23589: Redundant sentence in FAQ http://bugs.python.org/issue23589 #23585: patchcheck doesn't depend on all http://bugs.python.org/issue23585 #23584: test_doctest lineendings fails in verbose mode http://bugs.python.org/issue23584 #23583: IDLE: printing unicode subclasses broken (again) http://bugs.python.org/issue23583 #23577: Add tests for wsgiref.validate http://bugs.python.org/issue23577 #23575: MIPS64 needs ffi's n32.S http://bugs.python.org/issue23575 #23574: datetime: support leap seconds http://bugs.python.org/issue23574 #23573: Avoid redundant allocations in str.find and like http://bugs.python.org/issue23573 #23571: Raise SystemError if a function returns a result with an excep http://bugs.python.org/issue23571 #23570: Change "with subprocess.Popen():" (context manager) to ignore http://bugs.python.org/issue23570 Top 10 most discussed issues (10) ================================= #23496: Steps for Android Native Build of Python 3.4.2 http://bugs.python.org/issue23496 26 msgs #23539: Content-length not set for HTTP methods expecting body when bo http://bugs.python.org/issue23539 26 msgs #23595: Split the math module into _math (C) + math (py) http://bugs.python.org/issue23595 14 msgs #17911: traceback: add a new thin class storing a traceback without st http://bugs.python.org/issue17911 13 msgs #23553: Reduce the number of comparisons for range checking. http://bugs.python.org/issue23553 12 msgs #23571: Raise SystemError if a function returns a result with an excep http://bugs.python.org/issue23571 12 msgs #22936: traceback module has no way to show locals http://bugs.python.org/issue22936 11 msgs #23512: The list of built-in functions is not alphabetical on https:// http://bugs.python.org/issue23512 10 msgs #23524: Use _set_thread_local_invalid_parameter_handler in posixmodule http://bugs.python.org/issue23524 10 msgs #23505: Urlparse insufficient validation leads to open redirect http://bugs.python.org/issue23505 9 msgs Issues closed (43) ================== #3111: multiprocessing ppc Debian/ ia64 Ubuntu compilation error http://bugs.python.org/issue3111 closed by davin #6461: multiprocessing: freezing apps on Windows http://bugs.python.org/issue6461 closed by davin #11097: MSI: Remove win32com dependency from installer generator http://bugs.python.org/issue11097 closed by steve.dower #13697: python RLock implementation unsafe with signals http://bugs.python.org/issue13697 closed by pitrou #18382: multiprocessing's overlapped PipeConnection issues on Windows http://bugs.python.org/issue18382 closed by steve.dower #19075: Add sorting algorithm visualization to turtledemo http://bugs.python.org/issue19075 closed by ethan.furman #19931: namedtuple docstrings are verbose for no added benefit http://bugs.python.org/issue19931 closed by rhettinger #19980: Improve help('non-topic') response http://bugs.python.org/issue19980 closed by serhiy.storchaka #20147: multiprocessing.Queue.get() raises queue.Empty exception if ev http://bugs.python.org/issue20147 closed by davin #20521: Cleanup for "dis" module documentation http://bugs.python.org/issue20521 closed by berker.peksag #21293: Remove "capsule hack" from object.c? http://bugs.python.org/issue21293 closed by larry #21619: Cleaning up a subprocess with a broken pipe http://bugs.python.org/issue21619 closed by serhiy.storchaka #22791: datetime.utcfromtimestamp() shoud have option for create tz aw http://bugs.python.org/issue22791 closed by belopolsky #22801: collections.Counter, when empty, doesn't raise an error with & http://bugs.python.org/issue22801 closed by rhettinger #22817: re.split fails with lookahead/behind http://bugs.python.org/issue22817 closed by serhiy.storchaka #22840: strpdate('20141110', '%Y%m%d%H%S') returns wrong date http://bugs.python.org/issue22840 closed by belopolsky #23263: Python 3 gives misleading errors when validating unicode ident http://bugs.python.org/issue23263 closed by haypo #23304: Unused Superclass in calendar.py http://bugs.python.org/issue23304 closed by serhiy.storchaka #23362: integer overflow in string translate http://bugs.python.org/issue23362 closed by python-dev #23367: integer overflow in unicodedata.normalize http://bugs.python.org/issue23367 closed by python-dev #23368: integer overflow in _PyUnicode_AsKind http://bugs.python.org/issue23368 closed by serhiy.storchaka #23387: test_urllib2 fails with HTTP Error 502: Bad Gateway http://bugs.python.org/issue23387 closed by berker.peksag #23471: 404 Not Found when downloading Python 3.4.3rc1 Documentation http://bugs.python.org/issue23471 closed by larry #23477: Increase coverage for wsgiref module http://bugs.python.org/issue23477 closed by berker.peksag #23494: adding timedelta to datetime object is not timezone aware http://bugs.python.org/issue23494 closed by belopolsky #23504: Add __all__ into types http://bugs.python.org/issue23504 closed by serhiy.storchaka #23521: OverflowError from timedelta * float in datetime.py http://bugs.python.org/issue23521 closed by belopolsky #23527: test_smtpnet uses incorrect port for STARTTLS http://bugs.python.org/issue23527 closed by berker.peksag #23541: Re module's match() fails to halt on a particular input http://bugs.python.org/issue23541 closed by serhiy.storchaka #23547: Misdirected http://bugs.python.org/issue23547 closed by benjamin.peterson #23548: TypeError in event loop finalizer, new in Python 3.4.3 http://bugs.python.org/issue23548 closed by haypo #23558: Unable to install Python 3.4.3 amd64 on Windows 8.1 http://bugs.python.org/issue23558 closed by steve.dower #23559: [doc] inconsistent range example output http://bugs.python.org/issue23559 closed by georg.brandl #23561: a little typo in dis.rst; need a non-trivial preceding dot http://bugs.python.org/issue23561 closed by python-dev #23562: file.read() followed by file.write() result in incorrect behav http://bugs.python.org/issue23562 closed by r.david.murray #23563: Faster urllib.urlparse utility functions http://bugs.python.org/issue23563 closed by serhiy.storchaka #23569: unittest.mock.MagicMock.__div__ works but __truediv__ doesn' http://bugs.python.org/issue23569 closed by ned.deily #23576: HTTPS reads can block when content length not available and ti http://bugs.python.org/issue23576 closed by pitrou #23580: Amazon.com links http://bugs.python.org/issue23580 closed by ethan.furman #23582: multiprocessing.Queue.get is not getting all the items in the http://bugs.python.org/issue23582 closed by r.david.murray #23586: Add classproperty to the builtin functions http://bugs.python.org/issue23586 closed by rhettinger #23590: Potential leak in PyFloat_AsDouble. Refcount error. http://bugs.python.org/issue23590 closed by python-dev #23594: Wrong variable name in traceback http://bugs.python.org/issue23594 closed by ned.deily

1 0

subprocess, buffered files, pipes and broken pipe errors
by Victor Stinner 06 Mar '15

06 Mar '15

Hi, => I propose to ignore BrokenPipeError on Popen.__exit__(), what do you think? A recent issue fixed subprocess.Popen.__exit__() to read the exit status of the child process, even if stdin.close() raised a BrokenPipeError: http://bugs.python.org/issue21619 When I saw the issue, I was surprised that Popen.__exit__() doesn't ignore BrokenPipeError. Popen.communicate() is designed to handle corner cases and it ignores BrokenPipeError. Popen.__exit__() only returns when the process exited. The method starts by closing all pipes. Usually the child process notices that stdin was closed and exit (programs used as "pipes" like grep, cat, wc, etc.). The problem is that Popen uses a buffered writer for Popen.stdin with a default buffer size of 8 KB. We may buffer some data and so stdin.close() will call stdin.write(). If the child process exited or closed stdin, the pipe is broken, and the write in the parent process (stdin.close() in Popen.__exit__) will raise BrokenPipeError. I propose to ignore BrokenPipeError in Popen.__exit__, as done in communicate(), for convinience: http://bugs.python.org/issue23570 Serhiy wants to keep BrokenPipeError, he wrote that file.close() should not ignore write errors (read the issue for details). I consider that BrokenPipeError on a pipe is different than a write error on a regular file. EPIPE and SIGPIPE are designed to notify that the pipe is closed and that it's now inefficient to continue to write into this pipe. Ignoring BrokenPipeError in Popen.__exit__() respects this constrain because the method closes stdin and only returns when the process exited. So the caller will not write anything into stdin anymore. What do you think? Victor

2 1

PEP 485 review (isclose())
by Guido van Rossum 06 Mar '15

06 Mar '15

I think it's time to accept PEP 485. I've re-read it once more, and it looks like the text is in great shape. (My only recommendation would be to update the Abstract to state that we're specifically adding math.isclose().) A wording question: "This implementation has a flag that lets the user select which relative tolerance test to apply -- this PEP does not suggest that that be retained, but rather than the weak test be selected." -- I think this was meant to say "... rather *that* the weak test be selected", right? (It would be nice if the sample implementation defaulted to the choice in the PEP.) However, those are just minor edits, and I hereby approve the PEP. Thanks Chris and everyone else for the fruitful discussion (and thanks especially to Chris for eventually ending the bikeshedding and writing a PEP that explains each of the choices). Congrats! -- --Guido van Rossum (python.org/~guido)

7 15

Make review
by Serhiy Storchaka 06 Mar '15

06 Mar '15

If you have ready patches that wait for review and committing, tell me. Send me no more than 5 links to issues per person (for first time) in private and I'll try to make reviews if I'm acquainted with affected modules or areas.

1 0

Re: [Python-Dev] Python-Dev Digest, Vol 140, Issue 6
by SHAHRUKH KHAN 05 Mar '15

05 Mar '15

i think it's time to rename math module to math.py module On Thu, Mar 5, 2015 at 4:30 PM, <python-dev-request(a)python.org> wrote: > Send Python-Dev mailing list submissions to > python-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-dev > or, via email, send a message with subject or body 'help' to > python-dev-request(a)python.org > > You can reach the person managing the list at > python-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-Dev digest..." > > > Today's Topics: > > 1. Re: PEP 485 review (isclose()) (Chris Barker) > 2. Re: PEP 485 review (isclose()) (Guido van Rossum) > 3. Re: PEP 485 review (isclose()) (Brett Cannon) > 4. Re: PEP 485 review (isclose()) (Guido van Rossum) > 5. Re: PEP 485 review (isclose()) (Chris Barker) > 6. Re: Any volunteers to implement PEP 479? (Guido van Rossum) > 7. Re: Any volunteers to implement PEP 479? (Ethan Furman) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 4 Mar 2015 12:12:34 -0800 > From: Chris Barker <chris.barker(a)noaa.gov> > To: Ethan Furman <ethan(a)stoneleaf.us> > Cc: Python Dev <python-dev(a)python.org> > Subject: Re: [Python-Dev] PEP 485 review (isclose()) > Message-ID: > <CALGmxEKSz6wBzq7NyfN9B0S5EBdg3Y9wJttBoBpoebu= > 8i3jFA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Tue, Mar 3, 2015 at 8:43 AM, Ethan Furman <ethan(a)stoneleaf.us> wrote: > > > On 03/03/2015 01:17 AM, Victor Stinner wrote: > > > > > > > Maybe it's time to rename the math module to _math and create a > > > math.py module, like _decimal/decimal? math.py should end with "from > > > _math import *". > > > > +1 > > > > What do folks think? If we're going to do this, I'll write isclose() in > python. And I could do the work to split it out, too, I suppose. > > -Chris > > > > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R (206) 526-6959 voice > 7600 Sand Point Way NE (206) 526-6329 fax > Seattle, WA 98115 (206) 526-6317 main reception > > Chris.Barker(a)noaa.gov >

1 0

Any volunteers to implement PEP 479?
by Guido van Rossum 05 Mar '15

05 Mar '15

There's a proof of concept patch in http://bugs.python.org/issue22906, but it doesn't have the __future__ import and probably gets other details wrong. Reference: PEP 479 -- Change StopIteration handling inside generators <https://www.python.org/dev/peps/pep-0479/> -- --Guido van Rossum (python.org/~guido)

3 3

Wg: Re: [Python-checkins] cpython (3.4): Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
by s.krah 01 Mar '15

01 Mar '15

== == == == == == Weitergeleitete Nachricht == == == == == == Absender : Stefan Krah<stefan(a)bytereef.org> Empfänger : "Victor Stinner"<victor.stinner(a)gmail.com> Datum : So, 01 Mrz 2015 18:58:43 +0000 Betreff : Re: [Python-Dev] [Python-checkins] cpython (3.4): Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer == == == == == == Weitergeleitete Nachricht == == == == == == On Mon, Feb 16, 2015 at 10:14:59PM +0100, Victor Stinner wrote: > 2015-02-16 17:34 GMT+01:00 Stefan Krah <stefan(a)bytereef.org>: > > > > On Mon, Feb 16, 2015 at 11:35:52AM +0000, serhiy.storchaka wrote: > >> diff --git a/Modules/_testbuffer.c b/Modules/_testbuffer.c > >> --- a/Modules/_testbuffer.c > >> +++ b/Modules/_testbuffer.c > >> @@ -850,7 +850,7 @@ > >> Py_ssize_t *dest; > >> Py_ssize_t x, i; > >> > >> - dest = PyMem_Malloc(len * (sizeof *dest)); > >> + dest = PyMem_New(Py_ssize_t, len); > >> if (dest == NULL) { > >> PyErr_NoMemory(); > >> return NULL; > > > > This, too, was already protected by len == ndim <= 64. > > I don't understand why you don't want to use PyMem_New() even if it > cannot overflow. PyMem_New() is more readable no? It's readable, but I don't see a reason to change code that already has an overflow analysis, especially in 3.4. As I wrote in http://bugs.python.org/issue23446#msg235770 , people need to see that one *can* make certain assumptions about PEP-3118 buffers (otherwise one would go insane with overflow checks when doing arithmetic on the buffers. So, in a sense, this commit removes information for the reader. I would prefer an "assert(len <= 64)" for documentation purposes while keeping the multiplication. Stefan Krah

1 0