Mailman 3 March 2013 - Python-ideas

Updated PEP 428 (pathlib)
by Antoine Pitrou 07 Mar '13

07 Mar '13

Hello, I've updated PEP 428 following the previous discussion. Highlights: - the operator for combining paths is now `/`: >>> p / PurePosixPath('bar') PurePosixPath('foo/bar') >>> 'bar' / p PurePosixPath('bar/foo') - the method for combining paths is now named `joinpath` - new as_uri() method to represent a path as a `file` URI http://www.python.org/dev/peps/pep-0428/ Regards Antoine.

11 42

Updated PEP 434: IDLE Enhancement Exception
by Terry Reedy 07 Mar '13

07 Mar '13

I rewrote the PEP to better focus on the specific proposal and motivation. I also refined the formatting and added something about backwards compatibility with extensions. Is there any more discussion here before we post on pydev? -------------------------------------------------------------------- PEP: 434 Title: IDLE Enhancement Exception for All Branches Version: $Revision$ Last-Modified: $Date$ Author: Todd Rovito <rovitotv(a)gmail.com>, Terry Reedy <tjreedy(a)udel.edu> BDFL-Delegate: Nick Coghlan Status: Draft Type: Informational Content-Type: text/x-rst Created: 16-Feb-2013 Post-History: 16-Feb-2013 Abstract ======== Most CPython tracker issues are classified as behavior or enhancement. Most behavior patches are backported to branches for existing versions. Enhancement patches are restricted to the default branch that becomes the next Python version. This PEP proposes that the restriction on applying enhancements be relaxed for IDLE code, residing in .../Lib/idlelib/. In practice, this would mean that IDLE developers would not have to classify or agree on the classification of a patch but could instead focus on what is best for IDLE users and future IDLE developement. It would also mean that IDLE patches would not necessarily have to be split into 'bugfix' changes and enhancement changes. The PEP would apply to changes in existing features and addition of small features, such as would require a new menu entry, but not necessarily to possible major re-writes such as switching to themed widgets or tabbed windows. Motivation ========== This PEP was prompted by controversy on both the tracker and pydev list over adding Cut, Copy, and Paste to right-click context menus (Issue 1207589, opened in 2005 [1]_; pydev thread [2]_). The features were available as keyboard shortcuts but not on the context menu. It is standard, at least on Windows, that they should be when applicable (a read-only window would only have Copy), so users do not have to shift to the keyboard after selecting text for cutting or copying or a slice point for pasting. The context menu was not documented until 10 days before the new options were added (Issue 10405 [3]_). Normally, behavior is called a bug if it conflicts with documentation judged to be correct. But if there is no doc, what is the standard? If the code is its own documentation, most IDLE issues on the tracker are enhancement issues. If we substitute reasonable user expectation, (which can, of course, be its own subject of disagreement), many more issues are behavior issues. For context menus, people disagreed on the status of the additions -- bugfix or enhancement. Even people who called it an enhancement disagreed as to whether the patch should be backported. This PEP proposes to make the status disagreement irrelevant by explicitly allowing more liberal backporting than for other stdlib modules. Rationale ========= People primarily use IDLE by running the gui application, rather than by directly importing the effectively private (undocumented) implementation modules in idlelib. Whether they use the shell, the editor, or both, we believe they will benefit more from consistency across the latest releases of current Python versions than from consistency within the bugfix releases for one Python version. This is especially true when existing behavior is clearly unsatisfactory. When people use the standard interpreter, the OS-provided frame works pretty much the same for all Python versions. If, for instance, Microsoft were to upgrade the Command Prompt gui, the improvements would be present regardless of which Python were running within it. Similarly, if one edits Python code with editor X, behaviors such as the right-click context menu and the search-replace box do not depend on the version of Python being edited or even the language being edited. The benefit for IDLE developers is mixed. On the one hand, testing more versions and possibly having to adjust a patch, especially for 2.7, is more work. (There is, of course, the option on not backporting everything. For issue 12510, some changes to calltips for classes were not included in the 2.7 patch because of issues with old-style classes [4]_.) On the other hand, bike-shedding can be an energy drain. If the obvious fix for a bug looks like an enhancement, writing a separate bugfix-only patch is more work. And making the code diverge between versions makes future multi-version patches more difficult. These issue are illustrated by the search-and-replace dialog box. It used to raise an exception for certain user entries [5]_. The uncaught exception caused IDLE to exit. At least on Windows, the exit was silent (no visible traceback) and looked like a crash if IDLE was started normally, from an icon. Was this a bug? IDLE Help (on the current Help submenu) just says "Replace... Open a search-and-replace dialog box", and a box *was* opened. It is not, in general, a bug for a library method to raise an exception. And it is not, in general, a bug for a library method to ignore an exception raised by functions it calls. So if we were to adopt the 'code = doc' philosopy in the absence of detailed docs, one might say 'No'. However, IDLE exiting when it does not need to is definitely obnoxious. So four of us agreed that it should be prevented. But there was still the question of what to do instead? Catch the exception? Just not raise the exception? Beep? Display an error message box? Or try to do something useful with the user's entry? Would replacing a 'crash' with useful behavior be an enhancement, limited to future Python releases? Should IDLE developers have to ask that? Backwards Compatibility ======================= For IDLE, there are three types of users who might be concerned about back compatibility. First are people who run IDLE as an application. We have already discussed them above. Second are people who import one of the idlelib modules. As far as we know, this is only done to start the IDLE application, and we do not propose breaking such use. Otherwise, the modules are undocumented and effectively private implementations. If an IDLE module were defined as public, documented, and perhaps moved to the tkinter package, it would then follow the normal rules. (Documenting the private interfaces for the benefit of people working on the IDLE code is a separate issue.) Third are people who write IDLE extensions. The guaranteed extension interface is given in idlelib/extension.txt. This should be respected at least in existing versions, and not frivolously changed in future versions. But there is a warning that "The extension cannot assume much about this [EditorWindow] argument." This guarantee should rarely be an issue with patches, and the issue is not specific to 'enhancement' versus 'bugfix' patches. As is happens, after the context menu patch was applied, it came up that extensions that added items to the context menu (rare) would be broken because the patch a) added a new item to standard rmenu_specs and b) expected every rmenu_spec to be lengthened. It is not clear whether this violates the guarantee, but there is a second patch that fixes assumption b). It should be applied when it is clear that the first patch will not have to be reverted. References ========== .. [1] IDLE: Right Click Context Menu, Foord, Michael (http://bugs.python.org/issue1207589) .. [2] Cut/Copy/Paste items in IDLE right click context menu (http://mail.python.org/pipermail/python-dev/2012-November/122514.html) .. [3] IDLE breakpoint facility undocumented, Daily, Ned (http://bugs.python.org/issue10405) .. [4] IDLE: calltips mishandle raw strings and other examples, Reedy, Terry (http://bugs.python.org/issue12510) .. [5] IDLE: replace ending with '\' causes crash, Reedy, Terry (http://bugs.python.org/issue13052) Copyright ========= This document has been placed in the public domain. -- Terry Jan Reedy

1 1

BufferedIO and detach
by Robert Collins 06 Mar '13

06 Mar '13

There doesn't seem to be a way to safely use detach() on stdin - I'd like to get down to the raw stream, but after calling detach(), the initial BufferedIOReader is unusable - so you cannot retrieve any buffered content) - and unless you detach(), you can't guarantee that the buffer will ever be empty. I presume I'm missing something, but if there was a read([n], buffered_only=False) call, which you could invoke with buffered_only=True, then it would be possible to get out of this situation. -Rob -- Robert Collins <rbtcollins(a)hp.com> Distinguished Technologist HP Cloud Services

7 35

One-line "try with" statement
by Alan Johnson 05 Mar '13

05 Mar '13

It seems to me that one of the intended uses of the with statement was to automate simple initialization and deinitialization, which often accompanies a try block. It wouldn't be a game changing thing by any means, but has anybody ever thought about allowing a "try with" statement on one line? So instead of: try: with context_manager(): … bunch of code … except: … exception handler … you would have: try with context_manager(): … bunch of code … except: … exception handler … I envision the two examples being equivalent, the principle benefits being readability and one less indention level for the with block code. So a similar justification to "lower < x < upper" idiom. With standard 4 space indentation, existing with statements at the top of try blocks wouldn't even be any closer to the right margin. I'm no expert in Python interpreters, but it seems like a simple one-step internal conversion whenever this proposed syntax is encountered. But obviously, it would involve that change to every interpreter in existence with no actual new functionality, so I'm sensitive to that. Anyway, just a thought. -- Alan Johnson Cofounder | Breakrs.com 347-630-2036 | alan(a)breakrs.com

11 11

Proposal for Algorithms Library
by Akshit Agarwal 04 Mar '13

04 Mar '13

I am new to Python Community but I am using Python from around 1 year and I love to do coding on Python. Now I want to introduce an idea that I think should be there in Python which is I want to start working on a *"Algorithms Library"* which would be containing all basic Algorithms in its Intial Phase and then we can include all Algorithms which are listed in Introduction to Algorithms by CLRS and further extending to all possible algorithms which should be included. Implementing this will be very good for Python as Algorithms are used everywhere and developers have to spent a lot of their time in implementing the common algorithms which will be reduced as they will then be able to use them directly importing from Python which will also increase Python user's. I had just started contributing in Open Source and had contributed in SymPy's Symbols Function but now I want to do the above stated work as a Project in "*Google Summer Of Code 2013 in Python*". I need help from Community that how should I start working on it to do this in GSOC 2013. Akshit Agarwal

3 2

speeding up shutil.copy*()
by Charles-François Natali 04 Mar '13

04 Mar '13

shutil.copy*() use copyfileobj(): """ while 1: buf = fsrc.read(length) if not buf: break fdst.write(buf) """ This allocates and frees a lot of buffers, and could be optimized with readinto(). Unfortunately, I don't think we can change copyfileobj(), because it might be passed objects that don't implement readinto(). By implementing it directly in copyfile() (it would probably be better to expose it in shutil to make it available to tarfile & Co), there's a modest improvement: $ dd if=/dev/zero of=/tmp/foo bs=1M count=100 Without patch: $ ./python -m timeit -s "import shutil" "shutil.copyfile('/tmp/foo', '/dev/null')" 10 loops, best of 3: 218 msec per loop With readinto(): $ ./python -m timeit -s "import shutil" "shutil.copyfile('/tmp/foo', '/dev/null')" 10 loops, best of 3: 202 msec per loop (I'm using /dev/null as target because my hdd is really slow: other benchmarks are welcome, just beware that /tmp might be tmpfs). I've also written a dirty patch to use sendfile(). Here, the improvement is really significant: With sendfile(): $ ./python -m timeit -s "import shutil" "shutil.copyfile('/tmp/foo', '/dev/null')" 100 loops, best of 3: 5.39 msec per loop Thoughts? cf

6 11

Experimental package
by João Bernardo 03 Mar '13

03 Mar '13

I was thinking about "yield from" and how many times I wanted to use the feature, but because of backward compatibilities, I couldn't. It's sad that it may take possibly 5 or 10 years to see this statement being used in real programs... The PEP 380<http://www.python.org/dev/peps/pep-0380/>was proposed while Python3.1 was still alpha or beta and it took two and a half years to be accepted, but once it was, Python3.2 was already released. I don't know if this was proposed before, but why can't Python have an "__experimental__" magic package for this kind of stuff? The __future__ thing already add features that are sure to be included on a future release, but why those that *may *become official have to suffer that much? What if an experimental feature becomes rejected? from __experimental__ import foo DeprecationWarning: "foo" will be removed on Python 4.0 Note that* the idea is not for people to use experimental features* when they appear, but to have a fallback for when the feature becomes official! As side effect, the feature will be more tested and become more mature when released. -- João Bernardo

5 7

pep8 clarification, conditional top-level class/function leading newlines
by Dima Tisnek 03 Mar '13

03 Mar '13

Hi, I'm trying to figure out how to space following code according to pep-8: try: import x class A: def foo(self): # magic using x pass except ImportError: import y # different magic, using y typical conditions are try/except and if/elif/else, though I can imagine a true hacker to wrap top-level definitions in with x, possibly even for/else, while/else as well ;-) PEP-8 states to separate top-level class and functions by 2 blank lines and methods in a class by 1 blank line. This case falls into the crack, it's neither strictly top-level, nor a sub-level. option1: semantical, 2 lines before conditional top-levels option2: legalist, 1 line before any indented top-level pep8 tool only accepts option2 I think I would prefer option1; or explicitly leave it up user, then I can call option1 pep-8 compliant and someone else call option2 pep-8 compliant as well. What do you think or prefer? Perhaps this was discussed ages ago and I can't find the traces? Thanks, d.

4 3

OrderedDict for kwargs and class statement namespace
by Eric Snow 02 Mar '13

02 Mar '13

Just wanted to put out some feelers for the feasibility of these two features: * have the **kwargs param be an OrderedDict rather than a dict * have class definitions happen relative to an OrderedDict by default rather than a dict, and still overridable by a metaclass's __prepare__(). Both of these will need OrderedDict in C, which is getting close (issue #16991). -eric

10 28

string.format() default variable assignment
by 김용빈 01 Mar '13

01 Mar '13

why we bother with '{variable}'.format(variable=variable) ? can we just '{variable}.format()' ? if variable is exist, then assign it. if variable is not exist, then raise error I am not language expert. so sorry if this is not a good idea, or already discussed.

5 4