Mailman 3 May 2013 - Python-Dev

2.7.5 baking
by Benjamin Peterson May 15, 2013

May 15, 2013

The long anticipated "emergency" 2.7.5 release has now been tagged. It will be publicly announced as binaries arrive. Originally, I was just going to cherrypick regression fixes onto the 2.7.4 release and release those as 2.7.5. I started to this but ran into some conflicts. Since we don't have buildbot testing of release branches, I decided it would be best to just cut from the maintenance branch. -- Regards, Benjamin

7 9

How to debug python crashes
by Philippe Fremy May 15, 2013

May 15, 2013

Hi, I have a reproducable crash on Windows XP with Python 2.7 which I would like to investigate. I have Visual Studio 2008 installed and I downloaded the pdb files. However I could not find any instructions on how to use them and was unsuccessful at getting anything out of it. I checked the developer guide but could not find anything on debugging crashes. On internet, this seems to be also an underdocumented topic. So, a few questions : - is there some documentation to help debugging crashes ? - are the pdb files released along python usable with Visual Studio and stock Python ? Or do you need a hand-compiled version ? cheers, Philippe

6 6

Pickling failure on Enums
by Ethan Furman May 15, 2013

May 15, 2013

On 05/13/2013 11:32 AM, Guido van Rossum wrote: > > But now you enter a different phase of your project, or one of your collaborators does, or perhaps you've released your code on PyPI and one of your users does. So someone tries to pickle some class instance that happens to contain an unpicklable enum. That's not a great experience. Pickling and unpickling errors are often remarkably hard to debug. (Especially the latter, so I have privately admonished Ethan to ensure that if the getframe hack doesn't work, the pickle failure should happen at pickling time, not at unpickle time.) I can get pickle failure on members created using the functional syntax with no module set; I cannot get pickle failure on those same classes; I cannot get pickle failure on class syntax enums that inherit complex types (such as the NEI class in the tests). If anybody has any insight on how to make that work, I'm all ears. -- ~Ethan~

3 6

First post
by Carlos Nepomuceno May 14, 2013

May 14, 2013

Hi guys! This is my first post on this list. I'd like have your opinion on how to safely implement WSGI on a production server. My benchmarks show no performance differences between our PHP and Python environments. I'm using mod_wsgi v3.4 with Apache 2.4. Is that ok or can it get faster? Thanks in advance. Regards, Carlos

3 2

Issue 11406: adding os.scandir(), a directory iterator returning stat-like info
by Ben Hoyt May 14, 2013

May 14, 2013

A few of us were having a discussion at http://bugs.python.org/issue11406 about adding os.scandir(): a generator version of os.listdir() to make iterating over very large directories more memory efficient. This also reflects how the OS gives things to you -- it doesn't give you a big list, but you call a function to iterate and fetch the next entry. While I think that's a good idea, I'm not sure just that much is enough of an improvement to make adding the generator version worth it. But what would make this a killer feature is making os.scandir() generate tuples of (name, stat_like_info). The Windows directory iteration functions (FindFirstFile/FindNextFile) give you the full stat information for free, and the Linux and OS X functions (opendir/readdir) give you partial file information (d_type in the dirent struct, which is basically the st_mode part of a stat, whether it's a file, directory, link, etc). Having this available at the Python level would mean we can vastly speed up functions like os.walk() that otherwise need to make an os.stat() call for every file returned. In my benchmarks of such a generator on Windows, it speeds up os.walk() by 9-10x. On Linux/OS X, it's more like 1.5-3x. In my opinion, that kind of gain is huge, especially on Windows, but also on Linux/OS X. So the idea is to add this relatively low-level function that exposes the extra information the OS gives us for free, but which os.listdir() currently throws away. Then higher-level, platform-independent functions like os.walk() could use os.scandir() to get much better performance. People over at Issue 11406 think this is a good idea. HOWEVER, there's debate over what kind of object the second element in the tuple, "stat_like_info", should be. My strong vote is for it to be a stat_result-like object, but where the fields are None if they're unknown. There would be basically three scenarios: 1) stat_result with all fields set: this would happen on Windows, where you get as much info from FindFirst/FindNext as from an os.stat() 2) stat_result with just st_mode set, and all other fields None: this would be the usual case on Linux/OS X 3) stat_result with all fields None: this would happen on systems whose readdir()/dirent doesn't have d_type, or on Linux/OS X when d_type was DT_UNKNOWN Higher-level functions like os.walk() would then check the fields they needed are not None, and only call os.stat() if needed, for example: # Build lists of files and directories in path files = [] dirs = [] for name, st in os.scandir(path): if st.st_mode is None: st = os.stat(os.path.join(path, name)) if stat.S_ISDIR(st.st_mode): dirs.append(name) else: files.append(name) Not bad for a 2-10x performance boost, right? What do folks think? Cheers, Ben. P.S. A few non-essential further notes: 1) As a Windows guy, a nice-to-have addition to os.scandir() would be a keyword arg like win_wildcard which defaulted to '*.*', but power users can pass in to utilize the wildcard feature of FindFirst/FindNext on Windows. We have plenty of other low-level functions that expose OS-specific features in the OS module, so this would be no different. But then again, it's not nearly as important as exposing the stat info. 2) I've been dabbling with this concept for a while in my BetterWalk library: https://github.com/benhoyt/betterwalk Note that the benchmarks there are old, and I've made further improvements in my local copy. The ctypes version gives speed gains for os.walk() of 2-3x on Windows, but I've also got a C version, which is giving 9-10x speed gains. I haven't yet got a Linux/OS X version written in C. 3) See also the previous python-dev thread on BetterWalk: http://mail.python.org/pipermail/python-ideas/2012-November/017944.html

12 40

PEP 379 Python launcher for Windows - behaviour for #!/usr/bin/env python line is wrong
by Paul Moore May 14, 2013

May 14, 2013

While reviewing the behaviour of Vinay's "distil" installer tool (see distutils-sig for details, but it's not relevant here) I have found what I think is a flaw in the behaviour of the py.exe launcher for Windows. To recap for people unfamiliar with the launcher, it emulates #! line interpretation on Windows, interpreting commonly used forms from Unix and launching the appropriate installed Python interpreter (finding its location from the registry, as python.exe may not be on PATH). The problem is with the interpretation of #!/usr/bin/env python. The launcher treats this the same as #!/usr/bin/python, launching the "default" Python. But that is *not* what the equivalent line does on Unix, where it launches the *currently active* Python (a crucial difference when there is an active virtualenv). The result is that a script written to run with the active Python works on Unix as expected, but can use an unexpected version of Python on Windows. This is particularly unpleasant when the program in question is an (un)installer like distil! I would propose that the behaviour of the launcher on Windows should be changed when it encounters specifically the hashbang line #!/usr/bin/env python. In that case, it should search PATH for a copy of python.exe, and if it finds one, use that. If there is no python.exe on PATH, it should fall back to the same version of Python as would have been used if the line were #!/usr/bin/python. This will mean that scripts written with #!/usr/bin/env python will behave the same on Unix and Windows in the presence of activated virtualenvs. Would people be happy with this change? If so I will open an issue on bugs.python.org. I can look at producing a patch, as well. Paul

4 8

weak refs in descriptors (http://bugs.python.org/issue17950)
by Kristján Valur Jónsson May 13, 2013

May 13, 2013

Hello python-dev. I'm working on a patch to remove reference cycles from heap-allocated classes: http://bugs.python.org/issue17950 Part of the patch involves making sure that descriptors in the class dictionary don't contain strong references to the class itself. This is item 2) in the defect description. I have implemented this via weak references and hit no issues at all when running the test suite. But I'd like to ask the oracle if there is anything I may be overlooking with this approach? Any hidden problems we might encounter? K

5 4

Tightening up the specification for locals()
by Nick Coghlan May 13, 2013

May 13, 2013

An exchange in one of the enum threads prompted me to write down something I've occasionally thought about regarding locals(): it is currently severely underspecified, and I'd like to make the current CPython behaviour part of the language/library specification. (We recently found a bug in the interaction between the __prepare__ method and lexical closures that was indirectly related to this underspecification) Specifically, rather than the current vague "post-modification of locals may not work", I would like to explicitly document the expected behaviour at module, class and function scope (as well as clearly documenting the connection between modules, classes and the single- and dual-namespace variants of exec() and eval()): * at module scope, as well as when using exec() or eval() with a single namespace, locals() must return the same thing as globals(), which must be the actual execution namespace. Subsequent execution may change the contents of the returned mapping, and changes to the returned mapping must change the execution environment. * at class scope, as well as when using exec() or eval() with separate global and local namespaces, locals() must return the specified local namespace (which may be supplied by the metaclass __prepare__ method in the case of classes). Subsequent execution may change the contents of the returned mapping, and changes to the returned mapping must change the execution environment. For classes, this mapping will not be used as the actual class namespace underlying the defined class (the class creation process will copy the contents to a fresh dictionary that is only accessible by going through the class machinery). * at function scope, locals() must return a *snapshot* of the current locals and free variables. Subsequent execution must not change the contents of the returned mapping and changes to the returned mapping must not change the execution environment. Rather than adding this low level detail to the library reference docs, I would suggest adding it to the data model section of the language reference, with a link to the appropriate section from the docs for the locals() builtin. The warning in the locals() docs would be softened to indicate that modifications won't work at function scope, but are supported at module and class scope. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 9

make a Windows installation package (.msi) for Python 3.3
by Jianfeng Mao May 10, 2013

May 10, 2013

To Python Windows Release Managers: My name is Jianfeng Mao and I am a software developer at the U2 group in Rocket Software (http://u2.rocketsoftware.com/). I am currently working on a project to embed a slightly customized Python interpreter in our product. For easy installation and setup, we hope to be able to do the standard Python installation during the installation of our software. Basically I want to create a .msi file that can be called to install the full Python if the user needs this new feature. Brian Curtin (brian(a)python.org<mailto:brian@python.org>) pointed me to Tools/msi/msi.py for the Windows MSI builder. I tried to follow the instructions in the README but couldn't make it to work after a few twists and turns. Brian mentioned that few people needs to do this and only release managers handle the packaging of Python. I have listed the steps I have done in my attempt to create the .msi file. Please let me know if I have missed anything or done anything wrong. 1. hg clone http://hg.python.org/cpython 2. cd cpython 3. hg update 3.3 4. cd tools\buildbot, edit build.bat to change the configuration from Debug to Releaes; edit external.bat, change DEBUG=1 to DEBUG=0 5. go back to cpython\ and run tools\buildbot\build.bat 6. cd PC, then do 'nmake -f icons.mak' 7. cd ..\tools\msi 8. c:\python27\python msi.py WARNING: nm did not run successfully - libpythonXX.a not built cl /O2 /D WIN32 /D NDEBUG /D _WINDOWS /MT /W3 /c msisupport.c Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86 Copyright (C) Microsoft Corporation. All rights reserved. msisupport.c link.exe /OUT:msisupport.dll /INCREMENTAL:NO /NOLOGO /DLL /SUBSYSTEM:WIN DOWS /OPT:REF /OPT:ICF msisupport.obj msi.lib kernel32.lib Creating library msisupport.lib and object msisupport.exp Traceback (most recent call last): File "msi.py", line 1336, in <module> add_files(db) File "msi.py", line 961, in add_files generate_license() File "msi.py", line 914, in generate_license raise ValueError, "Could not find "+srcdir+"/../"+pat ValueError: Could not find C:\temp\cpython/../tcl8*

2 1

PEP 0 maintenance - deferring some currently open PEPs
by Nick Coghlan May 10, 2013

May 10, 2013

I'd like to mark a few PEPs that are not currently being actively considered for 3.4 as Deferred: S 286 Enhanced Argument Tuples von Löwis S 337 Logging Usage in the Standard Library Dubner S 368 Standard image protocol and class Mastrodomenico I 396 Module Version Numbers Warsaw S 400 Deprecate codecs.StreamReader and codecs.StreamWriter Stinner S 419 Protecting cleanup statements from interruptions Colomiets I 423 Naming conventions and recipes related to packaging Bryon I 444 Python Web3 Interface McDonough, Ronacher S 3124 Overloading, Generic Functions, Interfaces, and ... Eby S 3142 Add a "while" clause to generator expressions Britton S 3143 Standard daemon process library Finney S 3145 Asynchronous I/O For subprocess.Popen Pruitt, McCreary, Carlson S 3152 Cofunctions Ewing Obviously, they can be reactivated at any time, but I think it would be beneficial to have the "Open" list more accurately reflect proposals that are currently being championed. I'd also like to mark this one as rejected by Guido at PyCon US 2013 in favour of an updated PEP 436: S 437 A DSL for specifying signatures, annotations and ... Krah Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 10

Summary of Python tracker Issues
by Python tracker May 10, 2013

May 10, 2013

ACTIVITY SUMMARY (2013-05-03 - 2013-05-10) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3963 (+10) closed 25758 (+44) total 29721 (+54) Open issues with patches: 1774 Issues opened (48) ================== #5845: rlcompleter should be enabled automatically http://bugs.python.org/issue5845 reopened by mark.dickinson #15902: imp.load_module won't accept None for the file argument for a http://bugs.python.org/issue15902 reopened by brett.cannon #17656: Python 2.7.4 breaks ZipFile extraction of zip files with unico http://bugs.python.org/issue17656 reopened by pitrou #17883: Fix buildbot testing of Tkinter http://bugs.python.org/issue17883 reopened by ezio.melotti #17895: TemporaryFile name returns an integer in python3 http://bugs.python.org/issue17895 opened by jtaylor #17896: Move Windows external libs from <src>\..\ to <src>\externals http://bugs.python.org/issue17896 opened by zach.ware #17897: Optimize unpickle prefetching http://bugs.python.org/issue17897 opened by serhiy.storchaka #17898: gettext bug while parsing plural-forms metadata http://bugs.python.org/issue17898 opened by straz #17899: os.listdir() leaks FDs if invoked on FD pointing to a non-dire http://bugs.python.org/issue17899 opened by abacabadabacaba #17900: Recursive OrderedDict pickling http://bugs.python.org/issue17900 opened by serhiy.storchaka #17901: _elementtree.TreeBuilder raises IndexError on end if construct http://bugs.python.org/issue17901 opened by Aaron.Oakley #17902: Document that _elementtree C API cannot use custom TreeBuilder http://bugs.python.org/issue17902 opened by Aaron.Oakley #17903: Python launcher for windows should search path for #!/usr/bin/ http://bugs.python.org/issue17903 opened by pmoore #17904: bytes should be listed as built-in function for 2.7 http://bugs.python.org/issue17904 opened by flox #17905: Add check for locale.h http://bugs.python.org/issue17905 opened by cavallo71 #17906: JSON should accept lone surrogates http://bugs.python.org/issue17906 opened by serhiy.storchaka #17907: Deprecate imp.new_module() in favour of types.ModuleType http://bugs.python.org/issue17907 opened by brett.cannon #17908: Unittest runner needs an option to call gc.collect() after eac http://bugs.python.org/issue17908 opened by gvanrossum #17909: Autodetecting JSON encoding http://bugs.python.org/issue17909 opened by serhiy.storchaka #17911: Extracting tracebacks does too much work http://bugs.python.org/issue17911 opened by gvanrossum #17913: stat.filemode returns "-" for sockets and unknown types http://bugs.python.org/issue17913 opened by christian.heimes #17914: add os.cpu_count() http://bugs.python.org/issue17914 opened by neologix #17915: Encoding error with sax and codecs http://bugs.python.org/issue17915 opened by sconseil #17916: Provide dis.Bytecode based equivalent of dis.distb http://bugs.python.org/issue17916 opened by ncoghlan #17917: use PyModule_AddIntMacro() instead of PyModule_AddIntConstant( http://bugs.python.org/issue17917 opened by neologix #17919: AIX POLLNVAL definition causes problems http://bugs.python.org/issue17919 opened by delhallt #17920: Documentation: "complete ordering" should be "total ordering" http://bugs.python.org/issue17920 opened by abcdef #17922: Crash in clear_weakref http://bugs.python.org/issue17922 opened by jsafrane #17923: test glob with trailing slash fail http://bugs.python.org/issue17923 opened by delhallt #17924: Deprecate stat.S_IF* integer constants http://bugs.python.org/issue17924 opened by christian.heimes #17925: asynchat.async_chat.initiate_send : del deque[0] is not safe http://bugs.python.org/issue17925 opened by Pierrick.Koch #17927: Argument copied into cell still referenced by frame http://bugs.python.org/issue17927 opened by gvanrossum #17930: Search not needed in combinations_with_replacement http://bugs.python.org/issue17930 opened by tim_one #17931: PyLong_FromPid() is not correctly defined on Windows 64-bit http://bugs.python.org/issue17931 opened by haypo #17932: Win64: possible integer overflow in iterobject.c http://bugs.python.org/issue17932 opened by haypo #17933: test_ftp failure / ftplib error formatting issue http://bugs.python.org/issue17933 opened by pitrou #17934: Add a frame method to clear expensive details http://bugs.python.org/issue17934 opened by pitrou #17936: O(n**2) behaviour when adding/removing classes http://bugs.python.org/issue17936 opened by kristjan.jonsson #17937: Collect garbage harder at shutdown http://bugs.python.org/issue17937 opened by pitrou #17939: Misleading information about slice assignment in docs http://bugs.python.org/issue17939 opened by stefanchrobot #17940: extra code in argparse.py http://bugs.python.org/issue17940 opened by aho #17941: namedtuple should support fully qualified name for more portab http://bugs.python.org/issue17941 opened by eli.bendersky #17942: IDLE Debugger: names, values misaligned http://bugs.python.org/issue17942 opened by terry.reedy #17943: AttributeError: 'long' object has no attribute 'release' in Qu http://bugs.python.org/issue17943 opened by georg.brandl #17944: Refactor test_zipfile http://bugs.python.org/issue17944 opened by serhiy.storchaka #17945: tkinter/Python 3.3.0: peer_create doesn't instantiate Text http://bugs.python.org/issue17945 opened by ghoul #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 opened by ethan.furman #17948: HTTPS and sending a big file size hangs. http://bugs.python.org/issue17948 opened by jesusvpct Most recent 15 issues with no replies (15) ========================================== #17944: Refactor test_zipfile http://bugs.python.org/issue17944 #17942: IDLE Debugger: names, values misaligned http://bugs.python.org/issue17942 #17940: extra code in argparse.py http://bugs.python.org/issue17940 #17939: Misleading information about slice assignment in docs http://bugs.python.org/issue17939 #17937: Collect garbage harder at shutdown http://bugs.python.org/issue17937 #17934: Add a frame method to clear expensive details http://bugs.python.org/issue17934 #17933: test_ftp failure / ftplib error formatting issue http://bugs.python.org/issue17933 #17931: PyLong_FromPid() is not correctly defined on Windows 64-bit http://bugs.python.org/issue17931 #17924: Deprecate stat.S_IF* integer constants http://bugs.python.org/issue17924 #17923: test glob with trailing slash fail http://bugs.python.org/issue17923 #17916: Provide dis.Bytecode based equivalent of dis.distb http://bugs.python.org/issue17916 #17909: Autodetecting JSON encoding http://bugs.python.org/issue17909 #17905: Add check for locale.h http://bugs.python.org/issue17905 #17904: bytes should be listed as built-in function for 2.7 http://bugs.python.org/issue17904 #17902: Document that _elementtree C API cannot use custom TreeBuilder http://bugs.python.org/issue17902 Most recent 15 issues waiting for review (15) ============================================= #17947: Code, test, and doc review for PEP-0435 Enum http://bugs.python.org/issue17947 #17944: Refactor test_zipfile http://bugs.python.org/issue17944 #17937: Collect garbage harder at shutdown http://bugs.python.org/issue17937 #17936: O(n**2) behaviour when adding/removing classes http://bugs.python.org/issue17936 #17932: Win64: possible integer overflow in iterobject.c http://bugs.python.org/issue17932 #17931: PyLong_FromPid() is not correctly defined on Windows 64-bit http://bugs.python.org/issue17931 #17927: Argument copied into cell still referenced by frame http://bugs.python.org/issue17927 #17925: asynchat.async_chat.initiate_send : del deque[0] is not safe http://bugs.python.org/issue17925 #17923: test glob with trailing slash fail http://bugs.python.org/issue17923 #17919: AIX POLLNVAL definition causes problems http://bugs.python.org/issue17919 #17917: use PyModule_AddIntMacro() instead of PyModule_AddIntConstant( http://bugs.python.org/issue17917 #17915: Encoding error with sax and codecs http://bugs.python.org/issue17915 #17913: stat.filemode returns "-" for sockets and unknown types http://bugs.python.org/issue17913 #17909: Autodetecting JSON encoding http://bugs.python.org/issue17909 #17906: JSON should accept lone surrogates http://bugs.python.org/issue17906 Top 10 most discussed issues (10) ================================= #5845: rlcompleter should be enabled automatically http://bugs.python.org/issue5845 24 msgs #17922: Crash in clear_weakref http://bugs.python.org/issue17922 16 msgs #17927: Argument copied into cell still referenced by frame http://bugs.python.org/issue17927 15 msgs #11406: There is no os.listdir() equivalent returning generator instea http://bugs.python.org/issue11406 14 msgs #17883: Fix buildbot testing of Tkinter http://bugs.python.org/issue17883 14 msgs #17914: add os.cpu_count() http://bugs.python.org/issue17914 13 msgs #1545463: New-style classes fail to cleanup attributes http://bugs.python.org/issue1545463 12 msgs #11016: stat module in C http://bugs.python.org/issue11016 10 msgs #17810: Implement PEP 3154 (pickle protocol 4) http://bugs.python.org/issue17810 9 msgs #17868: pprint long non-printable bytes as hexdump http://bugs.python.org/issue17868 8 msgs Issues closed (44) ================== #2262: Helping the compiler avoid memory references in PyEval_EvalFra http://bugs.python.org/issue2262 closed by pitrou #6178: Core error in Py_EvalFrameEx 2.6.2 http://bugs.python.org/issue6178 closed by pitrou #7330: PyUnicode_FromFormat: implement width and precision for %s, %S http://bugs.python.org/issue7330 closed by haypo #7855: Add test cases for ctypes/winreg for issues found in IronPytho http://bugs.python.org/issue7855 closed by ezio.melotti #9687: dbmmodule.c:dbm_contains fails on 64bit big-endian (test_dbm.p http://bugs.python.org/issue9687 closed by pitrou #10363: Embedded python, handle (memory) leak http://bugs.python.org/issue10363 closed by pitrou #11816: Refactor the dis module to provide better building blocks for http://bugs.python.org/issue11816 closed by ncoghlan #12181: SIGBUS error on OpenBSD (sparc64) http://bugs.python.org/issue12181 closed by neologix #13495: IDLE: Regressions - Two ColorDelegator instances loaded http://bugs.python.org/issue13495 closed by roger.serwy #13831: get method of multiprocessing.pool.Async should return full t http://bugs.python.org/issue13831 closed by sbt #14173: PyOS_FiniInterupts leaves signal.getsignal segfaulty http://bugs.python.org/issue14173 closed by pitrou #14187: add "function annotation" entry to Glossary http://bugs.python.org/issue14187 closed by r.david.murray #14878: Improve documentation for generator.send method http://bugs.python.org/issue14878 closed by akuchling #15528: Better support for finalization with weakrefs http://bugs.python.org/issue15528 closed by sbt #15834: 2to3 benchmark not working under Python 3 http://bugs.python.org/issue15834 closed by brett.cannon #16445: SEGFAULT when deleting Exception.message http://bugs.python.org/issue16445 closed by pitrou #16523: attrgetter and itemgetter signatures in docs need cleanup http://bugs.python.org/issue16523 closed by ezio.melotti #16584: unhandled IOError filecmp.cmpfiles() if file not readable http://bugs.python.org/issue16584 closed by terry.reedy #16601: Restarting iteration over tarfile continues from where it left http://bugs.python.org/issue16601 closed by serhiy.storchaka #16631: tarfile.extractall() doesn't extract everything if .next() was http://bugs.python.org/issue16631 closed by serhiy.storchaka #17094: sys._current_frames() reports too many/wrong stack frames http://bugs.python.org/issue17094 closed by pitrou #17115: __loader__ = None should be fine http://bugs.python.org/issue17115 closed by brett.cannon #17116: xml.parsers.expat.(errors|model) don't set the __loader__ attr http://bugs.python.org/issue17116 closed by brett.cannon #17289: readline.set_completer_delims() doesn't play well with others http://bugs.python.org/issue17289 closed by pitrou #17408: second python execution fails when embedding http://bugs.python.org/issue17408 closed by pitrou #17714: str.encode('base64') add trailing new line character. It is no http://bugs.python.org/issue17714 closed by ezio.melotti #17798: IDLE: can not edit new file names when using -e http://bugs.python.org/issue17798 closed by roger.serwy #17805: No such class: multiprocessing.pool.AsyncResult http://bugs.python.org/issue17805 closed by sbt #17807: Generator cleanup without tp_del http://bugs.python.org/issue17807 closed by pitrou #17809: FAIL: test_expanduser when $HOME ends with / http://bugs.python.org/issue17809 closed by ezio.melotti #17833: test_gdb broken PPC64 Linux http://bugs.python.org/issue17833 closed by dmalcolm #17841: Remove missing aliases from codecs documentation http://bugs.python.org/issue17841 closed by ezio.melotti #17871: Wrong signature of TextTestRunner's init function http://bugs.python.org/issue17871 closed by ezio.melotti #17877: Skip test_variable_tzname when the zoneinfo database is missin http://bugs.python.org/issue17877 closed by ezio.melotti #17910: Usage error in multiprocessing documentation http://bugs.python.org/issue17910 closed by amysyk #17912: thread states should use a doubly-linked list http://bugs.python.org/issue17912 closed by neologix #17918: failed incoming SSL connection stays open forever http://bugs.python.org/issue17918 closed by pitrou #17921: explicit empty check instead of implicit booleaness http://bugs.python.org/issue17921 closed by r.david.murray #17926: PowerLinux dbm failure in 2.7 http://bugs.python.org/issue17926 closed by pitrou #17928: PowerLinux getargs.c FETCH_SIZE endianness bug http://bugs.python.org/issue17928 closed by pitrou #17929: TypeError using tarfile.addfile() with io.StringIO replacing S http://bugs.python.org/issue17929 closed by r.david.murray #17935: Failed compile on XP buildbot http://bugs.python.org/issue17935 closed by pitrou #17938: Duplicate text in docs/reference/import statement http://bugs.python.org/issue17938 closed by ezio.melotti #17946: base64 encoding result should be str, not bytes http://bugs.python.org/issue17946 closed by r.david.murray

1 0

PEP 4XX: pyzaa "Improving Python ZIP Application Support"
by Daniel Holth May 10, 2013

May 10, 2013

https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dn… PEP: 4XX Title: Improving Python ZIP Application Support Author: Daniel Holth <dholth(a)gmail.com> Status: Draft Type: Standards Track Python-Version: 3.4 Created: 30 March 2013 Post-History: 30 March 2013, 1 April 2013 Improving Python ZIP Application Support Python has had the ability to execute directories or ZIP-format archives as scripts since version 2.6. When invoked with a zip file or directory as its first argument the interpreter adds that directory to sys.path and executes the __main__ module. These archives provide a great way to publish software that needs to be distributed as a single file script but is complex enough to need to be written as a collection of modules. This feature is not as popular as it should be, mainly because no one’s heard of it because it wasn’t promoted as part of Python 2.6, but also because Windows users don’t have a file extension (other than .py) to associate with the launcher. This PEP proposes to fix these problems by re-publicising the feature, defining the .pyz and .pyzw extensions as “Python ZIP Applications” and “Windowed Python ZIP Applications”, and providing some simple tooling to manage the format. A New Python ZIP Application Extension The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP Applications” with the platform launcher so they can be executed. A .pyz archive is a console application and a .pyzw archive is a windowed application, indicating whether the console should appear when running the app. Why not use .zip or .py? Users expect a .zip file would be opened with an archive tool, and users expect .py to be opened with a text editor. Both would be confusing for this use case. For UNIX users, .pyz applications should be prefixed with a #! line pointing to the correct Python interpreter and an optional explanation. #!/usr/bin/env python3 # This is a Python application stored in a ZIP archive. (binary contents of archive) As background, ZIP archives are defined with a footer containing relative offsets from the end of the file. They remain valid when concatenated to the end of any other file. This feature is completely standard and is how self-extracting ZIP archives and the bdist_wininst installer format work. Minimal Tooling: The pyzaa Module This PEP also proposes including a simple application for working with these archives: The Python Zip Application Archiver “pyzaa” (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract these files, compile bytecode, and can write the __main__ module if it is not present. Usage python -m pyzaa (pack | compile) python -m pyzaa pack [-o path/name] [-m module.submodule:callable] [-c] [-w] [-p interpreter] directory: ZIP the contents of directory as directory.pyz or [-w] directory.pyzw. Adds the executable flag to the archive. -c compile .pyc files and add them to the archive -p interpreter include #!interpreter as the first line of the archive -o path/name archive is written to path/name.pyz[w] instead of dirname. The extension is added if not specified. -m module.submodule:callable __main__.py is written as “import module.submodule; module.submodule.callable()” pyzaa pack will warn if the directory contains C extensions or if it doesn’t contain __main__.py. python -m pyzaa compile arcname.pyz[w] The Python files in arcname.pyz[w] are compiled and appended to the ZIP file. A standard ZIP utility or Python’s zipfile module can unpack the archives. FAQ Q. Isn’t pyzaa just a very thin wrapper over zipfile and compileall? A. Yes. Q. How does this compete with existing sdist/bdist formats? A. There is some overlap, but .pyz files are especially interesting as a way to distribute an installer. They may also prove useful as a way to deliver applications when users shouldn’t be asked to perform virtualenv + “pip install”. References [1] http://bugs.python.org/issue1739468 “Allow interpreter to execute a zip file” [2] http://bugs.python.org/issue17359 “Feature is not documented” Copyright This document has been placed into the public domain.

11 20

PEP 435 (Enums) is Accepted
by Guido van Rossum May 10, 2013

May 10, 2013

I have reviewed the latest version of PEP 435 and I see that it is very good. I hereby declare PEP 435 as Accepted. Congratulations go to Barry, Eli and Ethan for pulling it through one of the most thorough reviewing and bikeshedding processes any PEP has seen. Thanks to everyone else for the many review comments. It is a better PEP because of the reviews. Barry or Eli, you can update the PEP's status. (I also expect there will be some copy-editing still.) Ethan: the stdlib implementation should probably be assigned a bug tracker issue (if there isn't one already) and start code review now. -- --Guido van Rossum (python.org/~guido)

4 3

Help requested for issue 9285 (profile.py)
by Giampaolo Rodola' May 10, 2013

May 10, 2013

http://bugs.python.org/issue9285#msg182986 I'm stuck as I really have no clue what that error means. Any help from someone experienced with profile.py code is welcome. --- Giampaolo https://code.google.com/p/pyftpdlib/ https://code.google.com/p/psutil/ https://code.google.com/p/pysendfile/

1 0

I want contribute to the project
by Diego Tolentino May 9, 2013

May 9, 2013

Hi guys, I have 3 computer and want to contribute to the project, Windows 7, Interl I5 64bits Ubuntu 13.04, Interl I5 32bits Ubuntu 13.04, Interl I5 64bits How i can proceed? I'm nice in this world of free software and python best regards Diego Tolentino SENIOR DEVELOPER Skype: diegotolentino “Do not go where the path may lead, go instead where there is no path and leave a trail.” - Ralph Waldo Emerson

2 1

PEP 435: pickling enums created with the functional API
by Eli Bendersky May 9, 2013

May 9, 2013

One of the contended issues with PEP 435 on which Guido pronounced was the functional API, that allows created enumerations dynamically in a manner similar to namedtuple: Color = Enum('Color', 'red blue green') The biggest complaint reported against this API is interaction with pickle. As promised, I want to discuss here how we're going to address this concern. At this point, the pickle docs say that module-top-level classes can be pickled. This obviously works for the normal Enum classes, but is a problem with the functional API because the class is created dynamically and has no __module__. To solve this, the reference implementation is used the same approach as namedtuple (*). In the metaclass's __new__ (this is an excerpt, the real code has some safeguards): module_name = sys._getframe(1).f_globals['__name__'] enum_class.__module__ = module_name According to an earlier discussion, this is works on CPython, PyPy and Jython, but not on IronPython. The alternative that works everywhere is to define the Enum like this: Color = Enum('the_module.Color', 'red blue green') The reference implementation supports this as well. Some points for discussion: 1) We can say that using the functional API when pickling can happen is not recommended, but maybe a better way would be to just explain the way things are and let users decide? 2) namedtuple should also support the fully qualified name syntax. If this is agreed upon, I can create an issue. 3) Antoine mentioned that work is being done in 3.4 to enable pickling of nested classes (http://www.python.org/dev/peps/pep-3154/). If that gets implemented, I don't see a reason why Enum and namedtuple can't be adjusted to find the __qualname__ of the class they're internal to. Am I missing something? 4) Using _getframe(N) here seems like an overkill to me. What we really need is just the module in which the current execution currently is (i.e. the metaclass's __new__ in our case). Would it make sense to add a new function somewhere in the stdlib of 3.4 (in sys or inspect or ...) that just provides the current module name? It seems that all Pythons should be able to easily provide it, it's certainly a very small subset of the functionality provided by walking the callframe stack. This function can then be used for build fully qualified names for pickling of Enum and namedtuple. Moreover, it can be general even more widely - dynamic class building is quite common in Python code, and as Nick mentioned somewhere earlier, the extra power of metaclasses in the recent 3.x's will probably make it even more common. Eli (*) namedtuple uses an explicit function to build the resulting class, not a metaclass (ther's no class syntax for namedtuple).

11 24

Any script to create the installation pacakge of Python 3.3.1 on Windows and *NIX?
by Jianfeng Mao May 9, 2013

May 9, 2013

To Python-Dev committers: I am working on a project to embed a slightly customized Python interpreter in our own software. For easy installation and setup, we want to be able to do the standard Python installation as part of the installation of our product. So far I have successfully customized and built Python 3.3.1 (including the subprojects) on Windows but I can't find anything in the source distribution to allow me package the binaries/modules etc into a MSI just like the one on the download page on python.org. So I am asking for information regarding how to package Python build for installation on both Windows and *NIX platforms. Your help will be greatly appreciated. Thanks, Jianfeng

2 1

Call for testing: generator finalization
by Antoine Pitrou May 8, 2013

May 8, 2013

Hello, In http://bugs.python.org/issue17807 I've committed a patch to allow generator finalization (execution of "finally" blocks) even when a generator is part of a reference cycle. If you have some workload which is known for problems with generator finalization (or otherwise makes a heavy use of generators), it would nice to have some feedback on this change. (the commit is only on the default branch) Regards Antoine.

1 0

Re: [Python-Dev] this python string literals documentation couldn't explain me: single quote presence inside double quoted string and viceversa. Can Anyone explain me?
by Alok Nayak May 8, 2013

May 8, 2013

I asked this question here, http://stackoverflow.com/questions/16435233/this-python-string-literals-doc…, . I was advised to ask here On Wed, May 8, 2013 at 4:56 PM, Alok Nayak <aloknayak29(a)gmail.com> wrote: > > This python string literals documentation<http://docs.python.org/2/reference/lexical_analysis.html#string-literals>couldn't explain: single quote presence inside double quoted string and > viceversa. > > I think both double quoted string and single quoted string need to be > defined differently for representing the 'stringliteral' lexical definition. > > shortstringchar ::= <any source character except "\" or newline or the quote> > > here in this definition 'the quote' isn't specific whether single (') or > double ("). > > > -- > Alok Nayak > Gwalior, India > -- Alok Nayak Gwalior, India

2 2

this python string literals documentation couldn't explain me: single quote presence inside double quoted string and viceversa. Can Anyone explain me?
by Alok Nayak May 8, 2013

May 8, 2013

This python string literals documentation<http://docs.python.org/2/reference/lexical_analysis.html#string-literals>couldn't explain: single quote presence inside double quoted string and viceversa. I think both double quoted string and single quoted string need to be defined differently for representing the 'stringliteral' lexical definition. shortstringchar ::= <any source character except "\" or newline or the quote> here in this definition 'the quote' isn't specific whether single (') or double ("). I asked this question here, http://stackoverflow.com/questions/16435233/this-python-string-literals-doc…, . I was advised to ask here -- Alok Nayak Gwalior, India -- Alok Nayak Gwalior, India

1 0

All 3.x stable buildbots are red
by Victor Stinner May 7, 2013

May 7, 2013

http://buildbot.python.org/all/waterfall?category=3.x.stable x86 Windows Server 2003 [SB] 3.x: 3 tests failed, test___all__ test_gc test_ssl x86 Windows7 3.x: 3 tests failed, test___all__ test_gc test_ssl x86 Gentoo Non-Debug 3.x: 3 tests failed, test_logging test_multiprocessing test_urllib2net x86 Gentoo 3.x: 2 tests failed, test_logging test_urllib2net x86 Ubuntu Shared 3.x: 1 test failed, test_logging AMD64 Windows7 SP1 3.x: 4 tests failed, test___all__ test_gc test_logging test_ssl AMD64 OpenIndiana 3.x: 1 test failed, test_logging AMD64 Ubuntu LTS 3.x: 1 test failed, test_logging AMD64 FreeBSD 9.0 3.x: 1 test failed, test_logging Can someone please look at these failures? (I don't have time for this right now.) Victor

2 1

Fighting the theoretical randomness of "is" on immutables
by Armin Rigo May 7, 2013

May 7, 2013

Hi all, In the context PyPy, we've recently seen again the issue of "x is y" not being well-defined on immutable constants. I've tried to summarize the issues and possible solutions in a mail to pypy-dev [1] and got some answers already. Having been convinced that the core is a language design issue, I'm asking for help from people on this list. (Feel free to cross-post.) [1] http://mail.python.org/pipermail/pypy-dev/2013-May/011299.html To summarize: the issue is a combination of various optimizations that work great otherwise. For example we can store integers directly in lists of integers, so when we read them back, we need to put them into fresh W_IntObjects (equivalent of PyIntObject). We solved temporarily the issue of "I'm getting an object which isn't ``is``-identical to the one I put in!" by making all equal integers ``is``-identical. This required hacking at ``id(x)`` as well to keep the requirement ``x is y <=> id(x)==id(y)``. This is getting annoying for strings, though -- how do you compute the id() of a long string? Give a unique long integer? And if we do the same for tuples, what about their id()? The long-term solution that seems the most stable to me would be to relax the requirement ``x is y <=> id(x)==id(y)``. If we can get away with only ``x is y <= id(x)==id(y)`` then it would allow us to implement ``is`` in a consistent way (e.g. two strings with equal content would always be ``is``-identical) while keeping id() reasonable (both in terms of complexity and of size of the resulting long number). Obviously ``x is y <=> id(x)==id(y)`` would still be true if any of ``x`` or ``y`` is not an immutable "by-value" built-in type. This is clearly a language design issue though. I can't really think of a use case that would break if we relax the requirement, but I might be wrong. It seems to me that at most some modules like pickle which use id()-keyed dictionaries will fail to find some otherwise-identical objects, but would still work (even if tuples are "relaxed" in this way, you can't have cycles with only tuples). A bientôt, Armin.

6 12

PEP 435: initial values must be specified? Yes
by Ethan Furman May 7, 2013

May 7, 2013

On 05/05/2013 10:07 AM, � wrote:> I'm chiming in late, but am I the only one who's really bothered by the syntax? > > class Color(Enum): > red = 1 > green = 2 > blue = 3 No, you are not only one that's bothered by it. I tried it without assignments until I discovered that bugs are way too easy to introduce. The problem is a successful name lookup looks just like a name failure, but of course no error is raised and no new enum item is created: --> class Color(Enum): ... red, green, blue ... --> class MoreColor(Color): ... red, orange, yellow ... --> type(MoreColor.red) is MoreColor False --> MoreColor.orange <MoreColor.orange: 4> # value should be 5 About the closest you going to be able to get is something like: def e(_next=[1]): e, _next[0] = _next[0], _next[0] + 1 return e class Color(Enum): red = e() green = e() blue = e() and you can keep using `e()` for all your enumerations, since you don't care what actual value each enumeration member happens to get. -- ~Ethan~

8 22

Re: [Python-Dev] cpython: Issue #11816: multiple improvements to the dis module
by Antoine Pitrou May 6, 2013

May 6, 2013

On Mon, 6 May 2013 15:59:49 +0200 (CEST) nick.coghlan <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/f65b867ce817 > changeset: 83644:f65b867ce817 > user: Nick Coghlan <ncoghlan(a)gmail.com> > date: Mon May 06 23:59:20 2013 +1000 > summary: > Issue #11816: multiple improvements to the dis module > > * get_instructions generator > * ability to redirect output to a file > * Bytecode and Instruction abstractions > > Patch by Nick Coghlan, Ryan Kelly and Thomas Kluyver. > > files: > Doc/library/dis.rst | 269 +++++++++++++++++++----- > Doc/whatsnew/3.4.rst | 15 + > Lib/dis.py | 341 +++++++++++++++++++++--------- > Lib/test/test_dis.py | 339 +++++++++++++++++++++++++++--- > Misc/NEWS | 4 + > 5 files changed, 775 insertions(+), 193 deletions(-) Looks like you forgot to add bytecode_helper.py. Regards Antoine.

1 0

PEP 435 - requesting pronouncement
by Eli Bendersky May 6, 2013

May 6, 2013

Hello pydev, PEP 435 is ready for final review. A lot of the feedback from the last few weeks of discussions has been incorporated. Naturally, not everything could go in because some minor (mostly preference-based) issues did not reach a consensus. We do feel, however, that the end result is better than in the beginning and that Python can finally have a useful enumeration type in the standard library. I'm attaching the latest version of the PEP for convenience. If you've read previous versions, the easiest way to get acquainted with the recent changes is to go through the revision log at http://hg.python.org/peps A reference implementation for PEP 435 is available at https://bitbucket.org/stoneleaf/ref435 Kind regards and happy weekend.

13 32

PEP 435 - reference implementation discussion
by Tim Delaney May 5, 2013

May 5, 2013

Split off from the PEP 435 - requesting pronouncement thread. Think I've come up with a system that works for my auto-numbering case without knowing the internals of enum_type. Patch passes all existing test cases. The patch does two things: 1. Finds the first non-Enum class on the MRO of the new class and uses that as the enum type. 2. Instead of directly setting the _name and _value of the enum_item, it lets the Enum class do it via Enum.__init__(). Subclasses can override this. This gives Enums a 2-phase construction just like other classes. diff -r 758d43b9f732 ref435.py --- a/ref435.py Fri May 03 18:59:32 2013 -0700 +++ b/ref435.py Sun May 05 13:43:56 2013 +1000 @@ -116,7 +116,17 @@ if bases[-1] is Enum: obj_type = bases[0] else: - obj_type = bases[-1].__mro__[1] # e.g. (IntEnum, int, Enum, object) + obj_type = None + + for base in bases: + for c in base.__mro__: + if not issubclass(c, Enum): + obj_type = c + break + + if obj_type is not None: + break + else: obj_type = object # save enum items into separate mapping so they don't get baked into @@ -143,8 +153,7 @@ enum_item = object.__new__(enum_class) else: enum_item = obj_type.__new__(enum_class, value) - enum_item._value = value - enum_item._name = e + enum_item.__init__(e, value) enum_map[e] = enum_item enum_class.__aliases__ = aliases # non-unique enums names enum_class._enum_names = enum_names # enum names in definition order @@ -232,6 +241,10 @@ return enum raise ValueError("%s is not a valid %s" % (value, cls.__name__)) + def __init__(self, name, value): + self._name = name + self._value = value + def __repr__(self): return "<%s.%s: %r>" % (self.__class__.__name__, self._name, self._value) Auto-int implementation: class AutoInt(int): __slots__ = () def __new__(cls, value): if value is Ellipsis: try: i = cls._auto_number except AttributeError: i = cls._auto_number = 0 else: i = cls._auto_number = value cls._auto_number += 1 return int.__new__(cls, i) class AutoIntEnum(AutoInt, IntEnum): def __init__(self, name, value): super(AutoIntEnum, self).__init__(name, int(self)) class TestAutoIntEnum(AutoIntEnum): a = ... b = 3 c = ... class TestAutoIntEnum2(AutoIntEnum): a = ... b = ... c = ... print(TestAutoIntEnum, list(TestAutoIntEnum)) print(TestAutoIntEnum2, list(TestAutoIntEnum2)) ---------- Run ---------- <Enum 'TestAutoIntEnum'> [<TestAutoIntEnum.a: 0>, <TestAutoIntEnum.b: 3>, <TestAutoIntEnum.c: 4>] <Enum 'TestAutoIntEnum2'> [<TestAutoIntEnum2.a: 0>, <TestAutoIntEnum2.b: 1>, <TestAutoIntEnum2.c: 2>] Tim Delaney

2 4

Re: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron Swartz memorial)
by Matěj Cepl May 5, 2013

May 5, 2013

On 2013-02-26, 16:25 GMT, Terry Reedy wrote: > On 2/21/2013 4:22 PM, Matej Cepl wrote: >> as my method to commemorate Aaron Swartz, I have decided to port his >> html2text to work fully with the latest python 3.3. After some time >> dealing with various bugs, I have now in my repo >> https://github.com/mcepl/html2text (branch python3) working solution >> which works all the way to python 3.2 (inclusive; >> https://travis-ci.org/mcepl/html2text). However, the last problem >> remains. This >> >> <li>Run this command: >> <pre>ls -l *.html</pre></li> >> <li>?</li> >> >> should lead to >> >> * Run this command: >> >> ls -l *.html >> >> * ? >> >> but it doesn’t. It leads to this (with python 3.3 only) >> >> * Run this command: >> ls -l *.html >> >> * ? >> >> Does anybody know about something which changed in modules re or >> http://docs.python.org/3.3/whatsnew/changelog.html between 3.2 and >> 3.3, which could influence this script? > > Search the changelob or 3.3 misc/News for items affecting those two > modules. There are at least 4. > http://docs.python.org/3.3/whatsnew/changelog.html > > It is faintly possible that the switch from narrow/wide builds to > unified builds somehow affected that. Have you tested with 2.7/3.2 on > both narrow and wide unicode builds? So, in the end, I have went the long way and bisected cpython to find the commit which broke my tests, and it seems that the culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is clearly something Unicode related. Unfortunately, it really doesn't tell me what exactly is broken (is it a known regression) and if there is known workaround. Could anybody suggest a way how to find bugs on http://bugs.python.org related to some particular commit (plain search for 123f2dc0 didn’t find anything). Any thoughts? Matěj P.S.: Crossposting to python-devel in hope there would be somebody understanding more about that particular commit. For that I have also intentionally not trim the original messages to preserve context. -- http://www.ceplovi.cz/matej/, Jabber: mcepl<at>ceplovi.cz GPG Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC When you're happy that cut and paste actually works I think it's a sign you've been using X-Windows for too long. -- from /. discussion on poor integration between KDE and GNOME

9 11

enum discussion: can someone please summarize open issues?
by Georg Brandl May 4, 2013

May 4, 2013

Hi all, I've just read a few dozen enum-related emails, and there are so many more. I would like to form an opinion about the proposal(s), but I feel I don't know what the actual issues are anymore. In the past, somebody usually presented a summary of the issues so far, and that was a good point for late comers to get up to speed and weigh in. (It can be here or in the PEP.) It is also a good point to focus the discussion (which seems to have wandered quite far from sensible Pythonic design in places). Thanks in advance, Georg

17 70

Re: [Python-Dev] [Python-checkins] cpython: #17115, 17116: Have modules initialize the __package__ and __loader__
by Brett Cannon May 4, 2013

May 4, 2013

FYI, I'm aware this broke some buildbots and will have a look today to figure out why. On Sat, May 4, 2013 at 1:57 PM, brett.cannon <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/e39a8f8ceb9f > changeset: 83607:e39a8f8ceb9f > user: Brett Cannon <brett(a)python.org> > date: Sat May 04 13:56:58 2013 -0400 > summary: > #17115,17116: Have modules initialize the __package__ and __loader__ > attributes to None. > > The long-term goal is for people to be able to rely on these > attributes existing and checking for None to see if they have been > set. Since import itself sets these attributes when a loader does not > the only instances when the attributes are None are from someone > overloading __import__() and not using a loader or someone creating a > module from scratch. > > This patch also unifies module initialization. Before you could have > different attributes with default values depending on how the module > object was created. Now the only way to not get the same default set > of attributes is to circumvent initialization by calling > ModuleType.__new__() directly. > > files: > Doc/c-api/module.rst | 11 +- > Doc/library/importlib.rst | 2 +- > Doc/reference/import.rst | 4 +- > Doc/whatsnew/3.4.rst | 5 + > Lib/ctypes/test/__init__.py | 2 +- > Lib/doctest.py | 2 +- > Lib/importlib/_bootstrap.py | 2 +- > Lib/inspect.py | 2 +- > Lib/test/test_descr.py | 4 +- > Lib/test/test_importlib/test_api.py | 14 +- > Lib/test/test_module.py | 28 +- > Misc/NEWS | 3 + > Objects/moduleobject.c | 39 +- > Python/importlib.h | 363 ++++++++------- > Python/pythonrun.c | 3 +- > 15 files changed, 264 insertions(+), 220 deletions(-) > > > diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst > --- a/Doc/c-api/module.rst > +++ b/Doc/c-api/module.rst > @@ -35,13 +35,20 @@ > single: __name__ (module attribute) > single: __doc__ (module attribute) > single: __file__ (module attribute) > + single: __package__ (module attribute) > + single: __loader__ (module attribute) > > Return a new module object with the :attr:`__name__` attribute set to *name*. > - Only the module's :attr:`__doc__` and :attr:`__name__` attributes are filled in; > - the caller is responsible for providing a :attr:`__file__` attribute. > + The module's :attr:`__name__`, :attr:`__doc__`, :attr:`__package__`, and > + :attr:`__loader__` attributes are filled in (all but :attr:`__name__` are set > + to ``None``); the caller is responsible for providing a :attr:`__file__` > + attribute. > > .. versionadded:: 3.3 > > + .. versionchanged:: 3.4 > + :attr:`__package__` and :attr:`__loader__` are set to ``None``. > + > > .. c:function:: PyObject* PyModule_New(const char *name) > > diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst > --- a/Doc/library/importlib.rst > +++ b/Doc/library/importlib.rst > @@ -827,7 +827,7 @@ > decorator as it subsumes this functionality. > > .. versionchanged:: 3.4 > - Set ``__loader__`` if set to ``None`` as well if the attribute does not > + Set ``__loader__`` if set to ``None``, as if the attribute does not > exist. > > > diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst > --- a/Doc/reference/import.rst > +++ b/Doc/reference/import.rst > @@ -423,8 +423,8 @@ > * If the module has a ``__file__`` attribute, this is used as part of the > module's repr. > > - * If the module has no ``__file__`` but does have a ``__loader__``, then the > - loader's repr is used as part of the module's repr. > + * If the module has no ``__file__`` but does have a ``__loader__`` that is not > + ``None``, then the loader's repr is used as part of the module's repr. > > * Otherwise, just use the module's ``__name__`` in the repr. > > diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst > --- a/Doc/whatsnew/3.4.rst > +++ b/Doc/whatsnew/3.4.rst > @@ -231,3 +231,8 @@ > :exc:`NotImplementedError` blindly. This will only affect code calling > :func:`super` and falling through all the way to the ABCs. For compatibility, > catch both :exc:`NotImplementedError` or the appropriate exception as needed. > + > +* The module type now initializes the :attr:`__package__` and :attr:`__loader__` > + attributes to ``None`` by default. To determine if these attributes were set > + in a backwards-compatible fashion, use e.g. > + ``getattr(module, '__loader__', None) is not None``. > \ No newline at end of file > diff --git a/Lib/ctypes/test/__init__.py b/Lib/ctypes/test/__init__.py > --- a/Lib/ctypes/test/__init__.py > +++ b/Lib/ctypes/test/__init__.py > @@ -37,7 +37,7 @@ > > def find_package_modules(package, mask): > import fnmatch > - if (hasattr(package, "__loader__") and > + if (package.__loader__ is not None and > hasattr(package.__loader__, '_files')): > path = package.__name__.replace(".", os.path.sep) > mask = os.path.join(path, mask) > diff --git a/Lib/doctest.py b/Lib/doctest.py > --- a/Lib/doctest.py > +++ b/Lib/doctest.py > @@ -215,7 +215,7 @@ > if module_relative: > package = _normalize_module(package, 3) > filename = _module_relative_path(package, filename) > - if hasattr(package, '__loader__'): > + if getattr(package, '__loader__', None) is not None: > if hasattr(package.__loader__, 'get_data'): > file_contents = package.__loader__.get_data(filename) > file_contents = file_contents.decode(encoding) > diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py > --- a/Lib/importlib/_bootstrap.py > +++ b/Lib/importlib/_bootstrap.py > @@ -1726,7 +1726,7 @@ > module_type = type(sys) > for name, module in sys.modules.items(): > if isinstance(module, module_type): > - if not hasattr(module, '__loader__'): > + if getattr(module, '__loader__', None) is None: > if name in sys.builtin_module_names: > module.__loader__ = BuiltinImporter > elif _imp.is_frozen(name): > diff --git a/Lib/inspect.py b/Lib/inspect.py > --- a/Lib/inspect.py > +++ b/Lib/inspect.py > @@ -476,7 +476,7 @@ > if os.path.exists(filename): > return filename > # only return a non-existent filename if the module has a PEP 302 loader > - if hasattr(getmodule(object, filename), '__loader__'): > + if getattr(getmodule(object, filename), '__loader__', None) is not None: > return filename > # or it is in the linecache > if filename in linecache.cache: > diff --git a/Lib/test/test_descr.py b/Lib/test/test_descr.py > --- a/Lib/test/test_descr.py > +++ b/Lib/test/test_descr.py > @@ -2250,7 +2250,9 @@ > minstance = M("m") > minstance.b = 2 > minstance.a = 1 > - names = [x for x in dir(minstance) if x not in ["__name__", "__doc__"]] > + default_attributes = ['__name__', '__doc__', '__package__', > + '__loader__'] > + names = [x for x in dir(minstance) if x not in default_attributes] > self.assertEqual(names, ['a', 'b']) > > class M2(M): > diff --git a/Lib/test/test_importlib/test_api.py b/Lib/test/test_importlib/test_api.py > --- a/Lib/test/test_importlib/test_api.py > +++ b/Lib/test/test_importlib/test_api.py > @@ -197,14 +197,12 @@ > # Issue #17098: all modules should have __loader__ defined. > for name, module in sys.modules.items(): > if isinstance(module, types.ModuleType): > - if name in sys.builtin_module_names: > - self.assertIn(module.__loader__, > - (importlib.machinery.BuiltinImporter, > - importlib._bootstrap.BuiltinImporter)) > - elif imp.is_frozen(name): > - self.assertIn(module.__loader__, > - (importlib.machinery.FrozenImporter, > - importlib._bootstrap.FrozenImporter)) > + self.assertTrue(hasattr(module, '__loader__'), > + '{!r} lacks a __loader__ attribute'.format(name)) > + if importlib.machinery.BuiltinImporter.find_module(name): > + self.assertIsNot(module.__loader__, None) > + elif importlib.machinery.FrozenImporter.find_module(name): > + self.assertIsNot(module.__loader__, None) > > > if __name__ == '__main__': > diff --git a/Lib/test/test_module.py b/Lib/test/test_module.py > --- a/Lib/test/test_module.py > +++ b/Lib/test/test_module.py > @@ -33,7 +33,10 @@ > foo = ModuleType("foo") > self.assertEqual(foo.__name__, "foo") > self.assertEqual(foo.__doc__, None) > - self.assertEqual(foo.__dict__, {"__name__": "foo", "__doc__": None}) > + self.assertIs(foo.__loader__, None) > + self.assertIs(foo.__package__, None) > + self.assertEqual(foo.__dict__, {"__name__": "foo", "__doc__": None, > + "__loader__": None, "__package__": None}) > > def test_ascii_docstring(self): > # ASCII docstring > @@ -41,7 +44,8 @@ > self.assertEqual(foo.__name__, "foo") > self.assertEqual(foo.__doc__, "foodoc") > self.assertEqual(foo.__dict__, > - {"__name__": "foo", "__doc__": "foodoc"}) > + {"__name__": "foo", "__doc__": "foodoc", > + "__loader__": None, "__package__": None}) > > def test_unicode_docstring(self): > # Unicode docstring > @@ -49,7 +53,8 @@ > self.assertEqual(foo.__name__, "foo") > self.assertEqual(foo.__doc__, "foodoc\u1234") > self.assertEqual(foo.__dict__, > - {"__name__": "foo", "__doc__": "foodoc\u1234"}) > + {"__name__": "foo", "__doc__": "foodoc\u1234", > + "__loader__": None, "__package__": None}) > > def test_reinit(self): > # Reinitialization should not replace the __dict__ > @@ -61,7 +66,8 @@ > self.assertEqual(foo.__doc__, "foodoc") > self.assertEqual(foo.bar, 42) > self.assertEqual(foo.__dict__, > - {"__name__": "foo", "__doc__": "foodoc", "bar": 42}) > + {"__name__": "foo", "__doc__": "foodoc", "bar": 42, > + "__loader__": None, "__package__": None}) > self.assertTrue(foo.__dict__ is d) > > @unittest.expectedFailure > @@ -110,13 +116,19 @@ > m.__file__ = '/tmp/foo.py' > self.assertEqual(repr(m), "<module '?' from '/tmp/foo.py'>") > > + def test_module_repr_with_loader_as_None(self): > + m = ModuleType('foo') > + assert m.__loader__ is None > + self.assertEqual(repr(m), "<module 'foo'>") > + > def test_module_repr_with_bare_loader_but_no_name(self): > m = ModuleType('foo') > del m.__name__ > # Yes, a class not an instance. > m.__loader__ = BareLoader > + loader_repr = repr(BareLoader) > self.assertEqual( > - repr(m), "<module '?' (<class 'test.test_module.BareLoader'>)>") > + repr(m), "<module '?' ({})>".format(loader_repr)) > > def test_module_repr_with_full_loader_but_no_name(self): > # m.__loader__.module_repr() will fail because the module has no > @@ -126,15 +138,17 @@ > del m.__name__ > # Yes, a class not an instance. > m.__loader__ = FullLoader > + loader_repr = repr(FullLoader) > self.assertEqual( > - repr(m), "<module '?' (<class 'test.test_module.FullLoader'>)>") > + repr(m), "<module '?' ({})>".format(loader_repr)) > > def test_module_repr_with_bare_loader(self): > m = ModuleType('foo') > # Yes, a class not an instance. > m.__loader__ = BareLoader > + module_repr = repr(BareLoader) > self.assertEqual( > - repr(m), "<module 'foo' (<class 'test.test_module.BareLoader'>)>") > + repr(m), "<module 'foo' ({})>".format(module_repr)) > > def test_module_repr_with_full_loader(self): > m = ModuleType('foo') > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -10,6 +10,9 @@ > Core and Builtins > ----------------- > > +- Issue #17115,17116: Module initialization now includes setting __package__ and > + __loader__ attributes to None. > + > - Issue #17853: Ensure locals of a class that shadow free variables always win > over the closures. > > diff --git a/Objects/moduleobject.c b/Objects/moduleobject.c > --- a/Objects/moduleobject.c > +++ b/Objects/moduleobject.c > @@ -26,6 +26,27 @@ > }; > > > +static int > +module_init_dict(PyObject *md_dict, PyObject *name, PyObject *doc) > +{ > + if (md_dict == NULL) > + return -1; > + if (doc == NULL) > + doc = Py_None; > + > + if (PyDict_SetItemString(md_dict, "__name__", name) != 0) > + return -1; > + if (PyDict_SetItemString(md_dict, "__doc__", doc) != 0) > + return -1; > + if (PyDict_SetItemString(md_dict, "__package__", Py_None) != 0) > + return -1; > + if (PyDict_SetItemString(md_dict, "__loader__", Py_None) != 0) > + return -1; > + > + return 0; > +} > + > + > PyObject * > PyModule_NewObject(PyObject *name) > { > @@ -36,13 +57,7 @@ > m->md_def = NULL; > m->md_state = NULL; > m->md_dict = PyDict_New(); > - if (m->md_dict == NULL) > - goto fail; > - if (PyDict_SetItemString(m->md_dict, "__name__", name) != 0) > - goto fail; > - if (PyDict_SetItemString(m->md_dict, "__doc__", Py_None) != 0) > - goto fail; > - if (PyDict_SetItemString(m->md_dict, "__package__", Py_None) != 0) > + if (module_init_dict(m->md_dict, name, NULL) != 0) > goto fail; > PyObject_GC_Track(m); > return (PyObject *)m; > @@ -347,9 +362,7 @@ > return -1; > m->md_dict = dict; > } > - if (PyDict_SetItemString(dict, "__name__", name) < 0) > - return -1; > - if (PyDict_SetItemString(dict, "__doc__", doc) < 0) > + if (module_init_dict(dict, name, doc) < 0) > return -1; > return 0; > } > @@ -380,7 +393,7 @@ > if (m->md_dict != NULL) { > loader = PyDict_GetItemString(m->md_dict, "__loader__"); > } > - if (loader != NULL) { > + if (loader != NULL && loader != Py_None) { > repr = PyObject_CallMethod(loader, "module_repr", "(O)", > (PyObject *)m, NULL); > if (repr == NULL) { > @@ -404,10 +417,10 @@ > filename = PyModule_GetFilenameObject((PyObject *)m); > if (filename == NULL) { > PyErr_Clear(); > - /* There's no m.__file__, so if there was an __loader__, use that in > + /* There's no m.__file__, so if there was a __loader__, use that in > * the repr, otherwise, the only thing you can use is m.__name__ > */ > - if (loader == NULL) { > + if (loader == NULL || loader == Py_None) { > repr = PyUnicode_FromFormat("<module %R>", name); > } > else { > diff --git a/Python/importlib.h b/Python/importlib.h > --- a/Python/importlib.h > +++ b/Python/importlib.h > [stripped] > diff --git a/Python/pythonrun.c b/Python/pythonrun.c > --- a/Python/pythonrun.c > +++ b/Python/pythonrun.c > @@ -866,7 +866,8 @@ > * be set if __main__ gets further initialized later in the startup > * process. > */ > - if (PyDict_GetItemString(d, "__loader__") == NULL) { > + PyObject *loader = PyDict_GetItemString(d, "__loader__"); > + if (loader == NULL || loader == Py_None) { > PyObject *loader = PyObject_GetAttrString(interp->importlib, > "BuiltinImporter"); > if (loader == NULL) { > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins >

1 0

PyPy, Jython, & IronPython: Enum convenience function and pickleablity
by Ethan Furman May 4, 2013

May 4, 2013

In order for the Enum convenience function to be pickleable, we have this line of code in the metaclass: enum_class.__module__ = sys._getframe(1).f_globals['__name__'] This works fine for Cpython, but what about the others? -- ~Ethan~

16 41

Summary of Python tracker Issues
by Python tracker May 3, 2013

May 3, 2013

ACTIVITY SUMMARY (2013-04-26 - 2013-05-03) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 3953 ( +4) closed 25714 (+40) total 29667 (+44) Open issues with patches: 1773 Issues opened (27) ================== #17585: IDLE - regression with exit() and quit() http://bugs.python.org/issue17585 reopened by serhiy.storchaka #17825: Indentation.offset and SyntaxError.offset mismatch http://bugs.python.org/issue17825 reopened by flox #17852: Built-in module _io can loose data from buffered files at exit http://bugs.python.org/issue17852 opened by arigo #17854: symmetric difference operation applicable to more than two set http://bugs.python.org/issue17854 opened by Amit.Saha #17855: Implement introspection of logger hierarchy http://bugs.python.org/issue17855 opened by vinay.sajip #17857: sqlite modules doesn't build with 2.7.4 on Mac OS X 10.4 http://bugs.python.org/issue17857 opened by lemburg #17858: Different documentation for identical methods http://bugs.python.org/issue17858 opened by amysyk #17859: improve error message for saving ints to file http://bugs.python.org/issue17859 opened by techtonik #17860: subprocess docs lack info how to use output result http://bugs.python.org/issue17860 opened by techtonik #17861: put opcode information in one place http://bugs.python.org/issue17861 opened by benjamin.peterson #17862: itertools.chunks(iterable, size, fill=None) http://bugs.python.org/issue17862 opened by techtonik #17868: pprint long non-printable bytes as hexdump http://bugs.python.org/issue17868 opened by serhiy.storchaka #17870: Python does not provide PyLong_FromIntMax_t() or PyLong_FromUi http://bugs.python.org/issue17870 opened by Devin Jeanpierre #17871: Wrong signature of TextTestRunner's init function http://bugs.python.org/issue17871 opened by piotr.dobrogost #17872: Crash in marshal.load() with bad reader http://bugs.python.org/issue17872 opened by serhiy.storchaka #17873: _ctypes/libffi missing bits for aarch64 support http://bugs.python.org/issue17873 opened by schwab #17874: ProcessPoolExecutor in interactive shell doesn't work in Windo http://bugs.python.org/issue17874 opened by Decade #17877: Skip test_variable_tzname when the zoneinfo database is missin http://bugs.python.org/issue17877 opened by ezio.melotti #17878: There is no way to get a list of available codecs http://bugs.python.org/issue17878 opened by pmoore #17882: test_objecttypes fails for 3.2.4 on CentOS 6 http://bugs.python.org/issue17882 opened by bharper #17883: Fix buildbot testing of Tkinter http://bugs.python.org/issue17883 opened by zach.ware #17884: Try to reuse stdint.h types like int32_t http://bugs.python.org/issue17884 opened by haypo #17887: docs: summary page - generator vs iterator vs iterable http://bugs.python.org/issue17887 opened by techtonik #17888: docs: more information on documentation team http://bugs.python.org/issue17888 opened by techtonik #17890: argparse: mutually exclusive groups full of suppressed args ca http://bugs.python.org/issue17890 opened by gholms #17893: Refactor reduce protocol implementation http://bugs.python.org/issue17893 opened by alexandre.vassalotti #17894: Edits to descriptor howto http://bugs.python.org/issue17894 opened by nedbat Most recent 15 issues with no replies (15) ========================================== #17894: Edits to descriptor howto http://bugs.python.org/issue17894 #17893: Refactor reduce protocol implementation http://bugs.python.org/issue17893 #17887: docs: summary page - generator vs iterator vs iterable http://bugs.python.org/issue17887 #17883: Fix buildbot testing of Tkinter http://bugs.python.org/issue17883 #17882: test_objecttypes fails for 3.2.4 on CentOS 6 http://bugs.python.org/issue17882 #17877: Skip test_variable_tzname when the zoneinfo database is missin http://bugs.python.org/issue17877 #17873: _ctypes/libffi missing bits for aarch64 support http://bugs.python.org/issue17873 #17872: Crash in marshal.load() with bad reader http://bugs.python.org/issue17872 #17862: itertools.chunks(iterable, size, fill=None) http://bugs.python.org/issue17862 #17848: issue about compile with clang and build a shared lib http://bugs.python.org/issue17848 #17844: Add link to alternatives for bytes-to-bytes codecs http://bugs.python.org/issue17844 #17840: base64_codec uses assert for runtime validity checks http://bugs.python.org/issue17840 #17829: csv.Sniffer.snif doesn't set up the dialect properly for a csv http://bugs.python.org/issue17829 #17824: pty.spawn handles errors improperly http://bugs.python.org/issue17824 #17799: settrace docs are wrong about "c_call" events http://bugs.python.org/issue17799 Most recent 15 issues waiting for review (15) ============================================= #17894: Edits to descriptor howto http://bugs.python.org/issue17894 #17893: Refactor reduce protocol implementation http://bugs.python.org/issue17893 #17890: argparse: mutually exclusive groups full of suppressed args ca http://bugs.python.org/issue17890 #17884: Try to reuse stdint.h types like int32_t http://bugs.python.org/issue17884 #17883: Fix buildbot testing of Tkinter http://bugs.python.org/issue17883 #17877: Skip test_variable_tzname when the zoneinfo database is missin http://bugs.python.org/issue17877 #17873: _ctypes/libffi missing bits for aarch64 support http://bugs.python.org/issue17873 #17871: Wrong signature of TextTestRunner's init function http://bugs.python.org/issue17871 #17870: Python does not provide PyLong_FromIntMax_t() or PyLong_FromUi http://bugs.python.org/issue17870 #17868: pprint long non-printable bytes as hexdump http://bugs.python.org/issue17868 #17861: put opcode information in one place http://bugs.python.org/issue17861 #17858: Different documentation for identical methods http://bugs.python.org/issue17858 #17857: sqlite modules doesn't build with 2.7.4 on Mac OS X 10.4 http://bugs.python.org/issue17857 #17855: Implement introspection of logger hierarchy http://bugs.python.org/issue17855 #17844: Add link to alternatives for bytes-to-bytes codecs http://bugs.python.org/issue17844 Top 10 most discussed issues (10) ================================= #17810: Implement PEP 3154 (pickle protocol 4) http://bugs.python.org/issue17810 18 msgs #17857: sqlite modules doesn't build with 2.7.4 on Mac OS X 10.4 http://bugs.python.org/issue17857 15 msgs #17870: Python does not provide PyLong_FromIntMax_t() or PyLong_FromUi http://bugs.python.org/issue17870 13 msgs #17878: There is no way to get a list of available codecs http://bugs.python.org/issue17878 12 msgs #17884: Try to reuse stdint.h types like int32_t http://bugs.python.org/issue17884 10 msgs #12458: Tracebacks should contain the first line of continuation lines http://bugs.python.org/issue12458 9 msgs #17825: Indentation.offset and SyntaxError.offset mismatch http://bugs.python.org/issue17825 9 msgs #17852: Built-in module _io can loose data from buffered files at exit http://bugs.python.org/issue17852 9 msgs #17838: Can't assign a different value for sys.stdin in IDLE http://bugs.python.org/issue17838 8 msgs #17843: Lib/test/testbz2_bigmem.bz2 trigger virus warnings http://bugs.python.org/issue17843 8 msgs Issues closed (38) ================== #1722: Undocumented urllib functions http://bugs.python.org/issue1722 closed by orsenthil #7152: urllib2.build_opener() skips ProxyHandler http://bugs.python.org/issue7152 closed by r.david.murray #11078: Have test___all__ check for duplicates http://bugs.python.org/issue11078 closed by ezio.melotti #12596: cPickle - stored data differ for same dictionary http://bugs.python.org/issue12596 closed by alexandre.vassalotti #13721: ssl.wrap_socket on a connected but failed connection succeeds http://bugs.python.org/issue13721 closed by pitrou #14290: Importing script as module causes ImportError with pickle.load http://bugs.python.org/issue14290 closed by alexandre.vassalotti #14679: Define an __all__ for html.parser http://bugs.python.org/issue14679 closed by ezio.melotti #15535: Fix pickling efficiency of named tuples in 2.7.3 http://bugs.python.org/issue15535 closed by rhettinger #16141: Possible simplification for old-style exception handling code http://bugs.python.org/issue16141 closed by r.david.murray #17358: imp.load_module() leads to the improper caching of the 'file' http://bugs.python.org/issue17358 closed by brett.cannon #17529: fix os.sendfile() documentation regarding the type of file des http://bugs.python.org/issue17529 closed by neologix #17565: segfaults during serialization http://bugs.python.org/issue17565 closed by alexandre.vassalotti #17646: traceback.py has a lot of code duplication http://bugs.python.org/issue17646 closed by python-dev #17712: test_gdb failures http://bugs.python.org/issue17712 closed by pitrou #17802: html.HTMLParser raises UnboundLocalError: http://bugs.python.org/issue17802 closed by ezio.melotti #17804: streaming struct unpacking http://bugs.python.org/issue17804 closed by pitrou #17834: Add Heap (and DynamicHeap) classes to heapq module http://bugs.python.org/issue17834 closed by rhettinger #17842: Add base64 module tests for a bytearray argument http://bugs.python.org/issue17842 closed by serhiy.storchaka #17851: Grammar errors in threading.Lock documentation http://bugs.python.org/issue17851 closed by georg.brandl #17853: Conflict between lexical scoping and name injection in __prepa http://bugs.python.org/issue17853 closed by python-dev #17856: multiprocessing.Process.join does not block if timeout is lowe http://bugs.python.org/issue17856 closed by neologix #17863: Bad sys.stdin assignment hangs interpreter. http://bugs.python.org/issue17863 closed by python-dev #17864: IDLE won't run http://bugs.python.org/issue17864 closed by ned.deily #17865: PowerPC exponentiation and round() interaction http://bugs.python.org/issue17865 closed by mark.dickinson #17866: TestCase.assertItemsEqual exists in 2.7, not in 3.3 http://bugs.python.org/issue17866 closed by ezio.melotti #17867: Deleting __import__ from builtins can crash Python3 http://bugs.python.org/issue17867 closed by python-dev #17869: distutils - TypeError in command/build_ext.py http://bugs.python.org/issue17869 closed by giampaolo.rodola #17875: Set Intersection returns unexpected results http://bugs.python.org/issue17875 closed by mark.dickinson #17876: Doc issue with threading.Event http://bugs.python.org/issue17876 closed by r.david.murray #17879: corrupt download http://bugs.python.org/issue17879 closed by ezio.melotti #17880: `tmpnam_r' is dangerous, better use `mkstemp' http://bugs.python.org/issue17880 closed by christian.heimes #17881: plistlib.writePlist documentation clarification for file objec http://bugs.python.org/issue17881 closed by ezio.melotti #17885: multiprocessing.Process child process imports package instead http://bugs.python.org/issue17885 closed by r.david.murray #17886: spam http://bugs.python.org/issue17886 closed by benjamin.peterson #17889: argparse subparsers break without arguments http://bugs.python.org/issue17889 closed by r.david.murray #17891: Wrong MD5 calculation on really long strings and the Hashlib http://bugs.python.org/issue17891 closed by neologix #17892: Fix the name of _PyObject_CallMethodObjIdArgs http://bugs.python.org/issue17892 closed by python-dev #1727418: xmlrpclib waits indefinately http://bugs.python.org/issue1727418 closed by r.david.murray

1 0

PEP-435 reference implementation
by Ethan Furman May 3, 2013

May 3, 2013

Greetings, Eli asked me to put the reference implementation here for review. It is available at https://bitbucket.org/stoneleaf/aenum in ref435.py and test_ref435.py -- ~Ethan~

10 30

Re: [Python-Dev] enum discussion: can someone please summarize open issues?
by Ethan Furman May 2, 2013

May 2, 2013

[re-directing back to python-dev] On 04/28/2013 08:42 PM, Davis Silverman wrote: > as a not super experienced python developer, when i see Season('AUTUMN') it looks like im creating an a Season object. I > understand your resoning, that it acts like a boolean singleton, however, i feel it would confuse many, and isn't worth > it. From what i see, its meant to be a lookup sort of thing, but it doesnt really feel like one. > > Or am i completely wrong about something? As far as you are concerned, you are creating a Season object. That it happens to be a pre-existing Season object is an implementation detail, and the whole thing would work just as well if it did indeed create a brand new object (you couldn't use `is` then, but in general `is` shouldn't be used anyway and I see no reason why `is` should be encouraged with Enums; use `==`). Two examples: - the first few integers (up to 256 now, I think) are pre-created by the interpreter; when you do `int('7')` you are not getting a brand-new, never before used, integer 7 object, you're getting a cached integer 7 object. - all booleans (yup, both of them ;) are pre-created; when you ask for a True or a False, you are not getting a brand new one. Since `is` is discouraged, both of those cases could go the other way (always creating a brand new object) and properly written programs would continue to run just fine -- slower, but fine. Enums are the same: they could return brand new instances every time, and programs using `==` to compare will keep on working. That they don't is an implementation detail. The real guarantee with enums is that once you have one created you'll only ever see the original values; so class Last(Enum): X = 1 Y = 2 Z = 3 will only have three possible elements, X, Y, and Z, and X will always have the value of 1, Y will always have the vale of 2, and Z will always have the value of 3. If you try and get `Last("W")` you'll trigger an exception, just like you do with `int("house")`. -- ~Ethan~

7 16

Re: [Python-Dev] [Python-checkins] peps: Add time(), call_at(). Remove call_repeatedly(). Get rid of add_*_handler()
by Nick Coghlan May 2, 2013

May 2, 2013

On 3 May 2013 08:34, "guido.van.rossum" <python-checkins(a)python.org> wrote: > > http://hg.python.org/peps/rev/26947623fc5d > changeset: 4870:26947623fc5d > user: Guido van Rossum <guido(a)python.org> > date: Thu May 02 14:11:08 2013 -0700 > summary: > Add time(), call_at(). Remove call_repeatedly(). Get rid of add_*_handler() return value. > > files: > pep-3156.txt | 80 +++++++++++++++++++++------------------ > 1 files changed, 43 insertions(+), 37 deletions(-) > > > diff --git a/pep-3156.txt b/pep-3156.txt > --- a/pep-3156.txt > +++ b/pep-3156.txt > @@ -252,13 +252,12 @@ > implementation may choose not to implement the internet/socket > methods, and still conform to the other methods.) > > -- Resource management: ``close()``. > +- Miscellaneous: ``close()``, ``time()``. > > - Starting and stopping: ``run_forever()``, ``run_until_complete()``, > ``stop()``, ``is_running()``. > > -- Basic callbacks: ``call_soon()``, ``call_later()``, > - ``call_repeatedly()``. > +- Basic callbacks: ``call_soon()``, ``call_later()``, ``call_at()``. > > - Thread interaction: ``call_soon_threadsafe()``, > ``wrap_future()``, ``run_in_executor()``, > @@ -303,8 +302,8 @@ > Required Event Loop Methods > --------------------------- > > -Resource Management > -''''''''''''''''''' > +Miscellaneous > +''''''''''''' > > - ``close()``. Closes the event loop, releasing any resources it may > hold, such as the file descriptor used by ``epoll()`` or > @@ -313,6 +312,12 @@ > again. It may be called multiple times; subsequent calls are > no-ops. > > +- ``time()``. Returns the current time according to the event loop's > + clock. This may be ``time.time()`` or ``time.monotonic()`` or some > + other system-specific clock, but it must return a float expressing > + the time in units of approximately one second since some epoch. > + (No clock is perfect -- see PEP 418.) Should the PEP allow event loops that use decimal.Decimal? > + > Starting and Stopping > ''''''''''''''''''''' > > @@ -362,17 +367,27 @@ > ``callback(*args)`` to be called approximately ``delay`` seconds in > the future, once, unless cancelled. Returns a Handle representing > the callback, whose ``cancel()`` method can be used to cancel the > - callback. If ``delay`` is <= 0, this acts like ``call_soon()`` > - instead. Otherwise, callbacks scheduled for exactly the same time > - will be called in an undefined order. > + callback. Callbacks scheduled in the past or at exactly the same > + time will be called in an undefined order. > > -- ``call_repeatedly(interval, callback, **args)``. Like > - ``call_later()`` but calls the callback repeatedly, every (approximately) > - ``interval`` seconds, until the Handle returned is cancelled or > - the callback raises an exception. The first call is in > - approximately ``interval`` seconds. If for whatever reason the > - callback happens later than scheduled, subsequent callbacks will be > - delayed for (at least) the same amount. The ``interval`` must be > 0. > +- ``call_at(when, callback, *args)``. This is like ``call_later()``, > + but the time is expressed as an absolute time. There is a simple > + equivalency: ``loop.call_later(delay, callback, *args)`` is the same > + as ``loop.call_at(loop.time() + delay, callback, *args)``. It may be worth explicitly noting the time scales where floating point's dynamic range starts to significantly limit granularity. Cheers, Nick. > + > +Note: A previous version of this PEP defined a method named > +``call_repeatedly()``, which promised to call a callback at regular > +intervals. This has been withdrawn because the design of such a > +function is overspecified. On the one hand, a simple timer loop can > +easily be emulated using a callback that reschedules itself using > +``call_later()``; it is also easy to write coroutine containing a loop > +and a ``sleep()`` call (a toplevel function in the module, see below). > +On the other hand, due to the complexities of accurate timekeeping > +there are many traps and pitfalls here for the unaware (see PEP 418), > +and different use cases require different behavior in edge cases. It > +is impossible to offer an API for this purpose that is bullet-proof in > +all cases, so it is deemed better to let application designers decide > +for themselves what kind of timer loop to implement. > > Thread interaction > '''''''''''''''''' > @@ -656,12 +671,9 @@ > > - ``add_reader(fd, callback, *args)``. Arrange for > ``callback(*args)`` to be called whenever file descriptor ``fd`` is > - deemed ready for reading. Returns a Handle object which can be used > - to cancel the callback. (However, it is strongly preferred to use > - ``remove_reader()`` instead.) Calling ``add_reader()`` again for > - the same file descriptor implies a call to ``remove_reader()`` for > - the same file descriptor. (TBD: Since cancelling the Handle is not > - recommended, perhaps we should return None instead?) > + deemed ready for reading. Calling ``add_reader()`` again for the > + same file descriptor implies a call to ``remove_reader()`` for the > + same file descriptor. > > - ``add_writer(fd, callback, *args)``. Like ``add_reader()``, > but registers the callback for writing instead of for reading. > @@ -669,8 +681,7 @@ > - ``remove_reader(fd)``. Cancels the current read callback for file > descriptor ``fd``, if one is set. If no callback is currently set > for the file descriptor, this is a no-op and returns ``False``. > - Otherwise, it removes the callback arrangement, cancels the > - corresponding Handle, and returns ``True``. > + Otherwise, it removes the callback arrangement and returns ``True``. > > - ``remove_writer(fd)``. This is to ``add_writer()`` as > ``remove_reader()`` is to ``add_reader()``. > @@ -704,11 +715,7 @@ > '''''''''''''''' > > - ``add_signal_handler(sig, callback, *args). Whenever signal ``sig`` > - is received, arrange for ``callback(*args)`` to be called. Returns > - a Handle which can be used to cancel the signal callback. > - (Cancelling the handle causes ``remove_signal_handler()`` to be > - called the next time the signal arrives. Explicitly calling > - ``remove_signal_handler()`` is preferred.) > + is received, arrange for ``callback(*args)`` to be called. > Specifying another callback for the same signal replaces the > previous handler (only one handler can be active per signal). The > ``sig`` must be a valid sigal number defined in the ``signal`` > @@ -777,11 +784,12 @@ > Handles > ------- > > -The various methods for registering callbacks (e.g. ``call_soon()`` > -and ``add_reader()``) all return an object representing the > -registration that can be used to cancel the callback. This object is > -called a Handle (although its class name is not necessarily > -``Handle``). Handles are opaque and have only one public method: > +The various methods for registering one-off callbacks > +(``call_soon()``, ``call_later()`` and ``call_at()``) all return an > +object representing the registration that can be used to cancel the > +callback. This object is called a Handle (although its class name is > +not necessarily ``Handle``). Handles are opaque and have only one > +public method: > > - ``cancel()``. Cancel the callback. > > @@ -1354,10 +1362,6 @@ > Open Issues > =========== > > -- A ``time()`` method that returns the time according to the function > - used by the scheduler (e.g. ``time.monotonic()`` in Tulip's case)? > - What's the use case? > - > - A fuller public API for Handle? What's the use case? > > - Should we require all event loops to implement ``sock_recv()`` and > @@ -1410,6 +1414,8 @@ > - PEP 3153, while rejected, has a good write-up explaining the need > to separate transports and protocols. > > +- PEP 418 discusses the issues of timekeeping. > + > - Tulip repo: http://code.google.com/p/tulip/ > > - Nick Coghlan wrote a nice blog post with some background, thoughts > > -- > Repository URL: http://hg.python.org/peps > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins >

1 0

PEP 428: stat caching undesirable?
by Pieter Nagel May 2, 2013

May 2, 2013

Hi all, I write as a python lover for over 13 years who's always wanted something like PEP 428 in Python. I am concerned about the caching of stat() results as currently defined in the PEP. This means that all behaviour built on top of stat(), such as p.is_dir(), p.is_file(), p.st_size and the like can indefinitely hold on to stale data until restat() is called, and I consider this confusing. Perhaps in recognition of this, p.exists() is implemented differently, and it does restat() internally (although the PEP does not document this). If this behaviour is maintained, then at the very least this makes the API more complicated to document: some calls cache as a side effect, others update the cache as a side effect, and others, such as lstat(), don't cache at all. This also introduces a divergence of behaviour between os.path.isfile() and p.is_file(), that is confusing and will also need to be documented. I'm concerned about scenarios like users of the library polling, for example, for some file to appear, and being confused about why the arguably more sloppy poll for p.exists() works while a poll for p.is_file(), which expresses intent better, never terminates. In theory the caching mechanism could be further refined to only hold onto cached results for a limited amount of time, but I would argue this is unnecessary complexity, and caching should just be removed, along with restat(). Isn't the whole notion that stat() need to be cached for performance issues somewhat of a historical relic of older OS's and filesystem performance? AFAIK linux already has stat() caching as a side-effect of the filesystem layer's metadata caching. How does Windows and Mac OS fare here? Are there benchmarks proving that this is serious enough to complicate the API? If the ability to cache stat() calls is deemed important enough, how about a different API where is_file(), is_dir() and the like are added as methods on the result object that stat() returns? Then one can hold onto a stat() result as a temporary object and ask it multiple questions without doing another OS call, and is_file() etc. on the Path object can be documented as being forwarders to the stat() result just as p.st_size is currently - except that I believe they should forward to a fresh, uncached stat() call every time. I write directly to this list instead raising it to Antoine Pitrou in private just because I don't want to make extra work for him to first receive my feedback and the re-raise it on this list. If this is wrong or disrespectful, I apologize. -- Pieter Nagel

7 12

Enum: subclassing?
by Ethan Furman May 2, 2013

May 2, 2013

We may not want to /completely/ disallow subclassing. Consider: --> class StrEnum(str, Enum): ... '''string enums for Business Basic variable names''' ... --> class Vendors(StrEnum): EnumError: subclassing not allowed My point is that IntEnum, StrEnum, ListEnum, FloatEnum are all "subclasses" of Enum. To then have a subclass of that, such as Season(StrEnum), is subclassing a subclass. Now, if we do want to completely disallow it, we can ditch IntEnum and force the user to always specify the mixin type: --> class Season(str, Enum): . . . --> class Names(str, Enum): . . . But that's not very user friendly... although it's not too bad, either. One consequence of the way it is now (IntEnum, StrEnum, etc., are allowed) is that one can put methods and other non-Enum item in a base class and then inherit from that for actual implemented Enum classes. --> class StrEnum(str, Enum): ... def describe(self): ... print("Hi! I'm a %s widget!" % self.value) ... --> class Season(StrEnum): ... spring = 'green' ... summer = 'brown' ... autumn = 'red' ... winter = 'white' ... --> class Planet(StrEnum): ... mars = 'red' ... earth = 'blue' ... --> Season.summer.descripbe() Hi! I'm a brown widget! --> Planet.earth.describe() Hi! I'm a blue widget! -- ~Ethan~

8 35

noob contributions to unit tests
by Sean Felipe Wolfe May 2, 2013

May 2, 2013

Hey everybody how are you all :) I am an intermediate-level python coder looking to get help out. I've been reading over the dev guide about helping increase test coverage --> http://docs.python.org/devguide/coverage.html And also the third-party code coverage referenced in the devguide page: http://coverage.livinglogic.de/ I'm seeing that according to the coverage tool, two of my favorite libraries, urllib/urllib2, have no unit tests? Is that correct or am I reading it wrong? If that's correct it seems like a great place perhaps for me to cut my teeth and I would be excited to learn and help out here. And of course any thoughts or advice for an aspiring Python contributor would be appreciated. Of course the dev guide gives me plenty of good info. Thanks! -- A musician must make music, an artist must paint, a poet must write, if he is to be ultimately at peace with himself. - Abraham Maslow

5 5

Enumeration items: `type(EnumClass.item) is EnumClass` ?
by Ethan Furman May 1, 2013

May 1, 2013

[creating new thread] On 04/29/2013 01:30 AM, Steven D'Aprano wrote: > On Sun, Apr 28, 2013 at 11:50:16PM -0700, Ethan Furman wrote: > >> In other words, currently: >> >> class Color(Enum): >> red = 1 >> green = 2 >> blue = 3 >> >> class MoreColor(Color): >> cyan = 4 >> magenta = 5 >> yellow = 6 >> black = 7 >> >> MoreColor.red is Color.red # True > > Correct. > > >> But as soon as: >> >> type(Color.red) is Color # True >> type(MoreColor.red) is MoreColor # True > > I don't believe this is correct. As I understand it, the proposal is the > weaker guarantee: > > isinstance(Color.red, Color) # True, possibly using __instancecheck__ Words from Guido: On 04/23/2013 08:11 AM, Guido van Rossum wrote: > I gotta say, I'm with Antoine here. It's pretty natural (also coming > from other languages) to assume that the class used to define the > enums is also the type of the enum values. Certainly this is how it > works in Java and C++, and I would say it's the same in Pascal and > probably most other languages. > On 04/25/2013 02:54 PM, Guido van Rossum wrote: > I don't know what's going on, but it feels like we had this same > discussion a week ago, and I still disagree. Disregarding, the C[i] > notation, I feel quite strongly that in the following example: > > class Color(Enum): > red = 1 > white = 2 > blue = 3 > orange = 4 > > the values Color.red etc. should be instances of Color. This is how > things work in all other languages that I am aware of that let you > define enums. On 04/25/2013 03:19 PM, Guido van Rossum wrote: > I suppose you were going to propose to use isinstance() overloading, > but I honestly think that Color.red.__class__ should be the same > object as Color. On 04/25/2013 03:37 PM, Guido van Rossum wrote: > TBH I had a hard time getting over the fact that even though the class > said "a = 1", C.a is not the integer 1. But I did get over it. > Hopefully you can get over *this* weirdness. [and from the summary thread] On 04/28/2013 01:02 PM, Guido van Rossum wrote: > On Sun, Apr 28, 2013 at 12:32 PM, Ethan Furman wrote: >> >> - should enum items be of the type of the Enum class? (i.e. type(SPRING) >> is Seasons) > > IMO Yes.

9 16

enum instances
by Marco Hemmelrath May 1, 2013

May 1, 2013

First of all, hi, I'm new to this list. Following the enum discussions on this list I am kind of confused about how enums and their respective instances, i.e. the values, should behave in "normal" context. I apologize beforehand for the mass of "questions" because the following contains really many discussed things but they all kinda depend on one another, and for terminology which I might've missed. Considering we have[1]: class Color(enum): red = 1 white = 2 other = "undefined" class State(enum): idle = 0 busy = 1 idling = idle ideling = 0 together with the premises: 1. type(State.busy) == State 2. type(State) == enum 3. isinstance(State.idle, State) 4. State.idle is State.idle which should mostly be agreed on (if I didn't misinterpret). How would an enum instance (e.g. State.busy) behave in normal Python expressions? Should these instances just wrap their values and provide some simple overhead useful for enums? I'll just note down a few examples of how I think it could work and express a few thoughts on them: 1. State.busy == 1 2. State.busy == Color.red 3. int(State.Busy) is 1 4. isinstance(State.busy, int) 5. Color.other.startswith("und") 6. State.busy is not 1 7. State.busy is not Color.red 8. State.idle in State 9. 0 in State # True, False or raise? 10. State.idling is State.idle 11. State.idle == State.idling 12. State.idle is not State.idling 1. & 2. Considering that enum components contain a value this value should be accessable and comperable. If it wasn't then there would be no real need to assign a value to it because everything must be compared using the is operator anyway. In this case an entirely new syntax could be implemented but since I think this is not what we want I'll skip this. Futhermore, if enum instances should compare equal to their values but unequal to each other this creates weird circumstances like: Color.red == 1 == State.busy -but- Color.red != State.busy == 1 3. Similar to 1. an enum instance's value /should/ be accessable as an expression by itself when needed besides simple operations like compatisons. This might be really tricky because the object in question is still of type State and some existing uses might break even though I can't think of any. What would certainly be a problem in this case though is that repr() would not reveal the value but the actual enum instance. This could be problematic when the enum instance's value is not a standard type or when repr() and str() differ and repr() is what's needed. 4. Combines 1. and 2. in class relation. It could be argued that the actual value should be an instance of its type subclassed by the enum class (State) which is in turn a subclass of enum. Going further on this it could also allow various things like class EnumWithInt(enum, int) which would only allow integers as its enum identifiers. This goes slightly into the IntEnum direction but with more flexibility. 5. Related to 3. in that this also allows attribute access to the enum instance's value. Considering that the enum instance should already be an instance of its value type as well this kinda speaks for itself. 6. & 7. Obviously, these can't be true even though they compare equal (see 1. and 2.). 8. Check if an enum instance is a member of State. Since in this case the same can also be achieved by using isinstance or by comparing the instance's type, this is mostly interesting for subclassing enums because, but I won't that cover here because it would probably be too much. 9. Analog to 8. this could either return just work, simply return False because this object is obviously not a member of State or raise an exception because of a type mismatch (not an enum instance). Also interesting for subclassing enums. 10. See premise 4. on this. 11. & 12. Even though these compare equal and even if their value object is actually the same they themseves should /not/ be the same object. Someone needs to find a use case for this though because as of now I can only think of this being Pythonic. So, the most critical part of this would probably be 3. to 7. regarding the value types. I'd love to read some thoughts and comments on that. Regards, Marco [1]: http://mail.python.org/pipermail/python-dev/2013-April/125738.html

4 7

Destructors and Closing of File Objects
by Nikolaus Rath May 1, 2013

May 1, 2013

[ Note: I already asked this on http://stackoverflow.com/questions/15917502 but didn't get any satisfactory answers] Hello, The description of tempfile.NamedTemporaryFile() says: ,---- | If delete is true (the default), the file is deleted as soon as it is | closed. `---- In some circumstances, this means that the file is not deleted after the program ends. For example, when running the following test under py.test, the temporary file remains: ,---- | from __future__ import division, print_function, absolute_import | import tempfile | import unittest2 as unittest | class cache_tests(unittest.TestCase): | def setUp(self): | self.dbfile = tempfile.NamedTemporaryFile() | def test_get(self): | self.assertEqual('foo', 'foo') `---- In some way this makes sense, because this program never explicitly closes the file object. The only other way for the object to get closed would presumably be in the __del__ destructor, but here the language references states that "It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits." So everything is consistent with the documentation so far. However, I'm confused about the implications of this. If it is not guaranteed that file objects are closed on interpreter exit, can it possibly happen that some data that was successfully written to a (buffered) file object is lost even though the program exits gracefully, because it was still in the file objects buffer and the file object never got closed? Somehow that seems very unlikely and un-pythonic to me, and the open() documentation doesn't contain any such warnings either. So I (tentatively) conclude that file objects are, after all, guaranteed to be closed. But how does this magic happen, and why can't NamedTemporaryFile() use the same magic to ensure that the file is deleted? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

6 11

Enumeration items: mixed types?
by Ethan Furman May 1, 2013

May 1, 2013

This just doesn't make sense to me: --> class Stuff(Enum): ... blue = 1 ... china = 'really big country' ... random = (8273.199, 517) --> Stuff.blue.name == 'blue' --> Stuff.blue.value == 1 --> Stuff.china.name == 'china' --> Stuff.china.value == ??? --> Stuff.random.name == 'random' --> Stuff.china.value == ??? In order to make this work at all, we have to support auto-numbering, and I didn't think we were going to do that in the class syntax? -- ~Ethan~ P.S. Apologies for all the questions, I'm just hoping to get the last details hammered out so we can stop discussing Enums. ;)

6 7