Mailman 3 September 2015 - Python-Dev

[RELEASED] Python 3.5.0rc3 is now available
by Larry Hastings Sept. 8, 2015

Sept. 8, 2015

On behalf of the Python development community and the Python 3.5 release team, I'm relieved to announce the availability of Python 3.5.0rc3, also known as Python 3.5.0 Release Candidate 3. The next release of Python 3.5 will be Python 3.5.0 final. There should be few (or no) changes to Python 3.5 between rc3 and final. This is a preview release, and its use is not recommended for production settings. You can find Python 3.5.0rc3 here: https://www.python.org/downloads/release/… [View More]

1 0

Critique of PEP 502 (String Interpolation Redux)
by Guido van Rossum Sept. 8, 2015

Sept. 8, 2015

The text of the PEP has too much on motivation and rationale: maybe that would be suitable for an informative PEP. The proposal itself is under-specified. But the real weakness cannot be fixed by improving the text: it is in the key characteristic of the proposal, which wants to have its cake and eat it, too. This is initially likable: to recipients who don't know about e-strings they behave just like regular strings; but recipients who *do* know about them can extract the template and the … [View More]

2 3

PyParallel update
by Trent Nelson Sept. 6, 2015

Sept. 6, 2015

[CC'ing python-dev@ for those that are curious; please drop and keep follow-up discussion to python-ideas@] Hi folks, I've made a lot of progress on PyParallel since the PyCon dev summit (https://speakerdeck.com/trent/pyparallel-pycon-2015-language-summit); I fixed the outstanding breakage with generators, exceptions and whatnot. I got the "instantaneous Wiki search server" working[1] and implemented the entire TechEmpower Frameworks Benchmark Suite[2], including a PyParallel-friendly pyodbc … [View More]module, allowing database connections and querying in parallel. [1]: https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/examples/wiki… [2]: https://github.com/pyparallel/pyparallel/blob/branches/3.3-px/examples/tefb… I set up a landing page for the project: http://pyparallel.org And there was some good discussion on reddit earlier this week: https://www.reddit.com/r/programming/comments/3jhv80/pyparallel_an_experime… I've put together some documentation on the project, its aims, and the key parts of the solution regarding the parallelism through simple client/server paradigms. This documentation is available directly on the github landing page for the project: https://github.com/pyparallel/pyparallel Writing that documentation forced me to formalize (or at least commit) to the restrictions/trade-offs that PyParallel would introduce, and I'm pretty happy I was basically able to boil it down into a single rule: Don't persist parallel objects. That keeps the mental model very simple. You don't need to worry about locking or ownership or races or anything like that. Just don't persist parallel objects, that's the only thing you have to remember. It's actually really easy to convert existing C code or Python code into something that is suitable for calling from within a parallel callback by just ensuring that rule isn't violated. It took about four hours to figure out how NumPy allocated stuff and add in the necessary PyParallel-aware tweaks, and not that much longer for pyodbc. (Most stuff "just works", though.) (The ABI changes would mean this is a Python 4.x type of thing; there are fancy ways we could avoid ABI changes and get this working on Python 3.x, but, eh, I like the 4.x target. It's realistic.) The other thing that clicked is that asyncio and PyParallel would actually work really well together for exploiting client-driven parallelism (PyParallel really is only suited to server-oriented parallelism at the moment, i.e. serving HTTP requests in parallel). With asyncio, though, you could keep the main-thread/single-thread client-drives-computation paradigm, but have it actually dispatch work to parallel.server() objects behind the scenes. For example, in order to process all files in a directory in parallel, asyncio would request a directory listing (i.e. issue a GET /) which the PyParallel HTTP server would return, it would then create non-blocking client connections to the same server and invoke whatever HTTP method is desired to do the file processing. You can either choose to write the new results from within the parallel context (which could then be accessed as normal files via HTTP), or you could have PyParallel return json/bytes, which could then be aggregated by asyncio. Everything is within the same process, so you get all the benefits that provides (free access to anything within scope, like large data structures, from within parallel contexts). You can synchronously call back into the main thread from a parallel thread, too, if you wanted to update a complex data structure directly. The other interesting thing that documentation highlights is the advantage of the split brain "main thread vs parallel thread" GC and non-GC allocators. I'm not sure if I've ever extolled the virtue of such an approach on paper or in e-mail. It's pretty neat though and allows us to avoid a whole raft of problems that need to be solved when you have a single GC/memory model. Next steps: once 3.5 is tagged, I'm going to bite the bullet and rebase. That'll require a bit of churn, so if there's enough interest from others, I figured we'd use the opportunity to at least get it building again on POSIX (Linux/OSX/FreeBSD). From there people can start implementing the missing bits for implementing the parallel machinery behind the scenes. The parallel interpreter thread changes I made are platform agnostic, the implementation just happens to be on Windows at the moment; don't let the Windows-only thing detract from what's actually being pitched: a (working, demonstrably-performant) solution to "Python's GIL problem". Regards, Trent. [View Less]

1 0

PEPs and PEP 8 changes
by Jim J. Jewett Sept. 6, 2015

Sept. 6, 2015

PEP 498 is only the latest PEP where part of the concern is fear that it may encourage certain types of bad code. Would it be reasonable to ask PEPs to start including a section on any recommended changes to PEP8? (e.g., "If an embedded expression doesn't fit on a single line, factor it out to a named variable.") I realize that there will be times when best practices (or common mistakes) aren't obvious in advance, but I'm a bit uncomfortable with "PEP 8 will probably grow advice"... if we … [View More]

3 2

PEP 501 Shell Command Examples
by Nikolaus Rath Sept. 6, 2015

Sept. 6, 2015

Hi Nick, You are giving runcommand(sh(i"cat {filename}")) as an example that avoids injection attacks. While this is true, I think this is still a terrible anti-pattern[1] that should not be entombed in a PEP as a positive example. Could you consider removing it? (It doubly wastes resources by pointlessly calling a shell, and then by parsing & quoting the argument only for the shell to do the same in reverse). Best, -Nikolaus -- GPG encrypted emails preferred. Key id: … [View More]

2 2

Critique of PEP 501 (General purpose string interpolation)
by Guido van Rossum Sept. 5, 2015

Sept. 5, 2015

I think it's too much effort for too little gain. The motivation feels very weak; surely writing os.system("echo " + message_from_user) is just as easy (as is the %s spelling), so the security issue can hardly be blamed on PEP 498. I also don't think that the current way to address such security issues is a big deal: - The subprocess module is complex for other reasons, and a simpler wrapper could easily be made; - Database wrappers have forever included their own solution for safely … [View More]

2 2

Should we use getentropy() for os.urandom()?
by haypo s Sept. 5, 2015

Sept. 5, 2015

Hi, I followed discussions on the new systems getrandom() on Linux and getentropy() on OpenBSD. I wanted to use them in Python to avoid the need of a file descriptor to read /dev/urandom. Linux getrandom() is also more secure than /dev/urandom because it blocks until /dev/urandom is feeded with enough entropy. getentropy() and getrandom() are now used in Python 2.7.10, Python 3.4 and newer. Today, an issue was reported on Solaris because os.urandom() is much slower with Python 2.7.10: https:… [View More]

3 2

Summary of Python tracker Issues
by Python tracker Sept. 4, 2015

Sept. 4, 2015

ACTIVITY SUMMARY (2015-08-28 - 2015-09-04) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5057 (+15) closed 31704 (+33) total 36761 (+48) Open issues with patches: 2257 Issues opened (28) ================== #24954: No way to generate or parse timezone as produced by datetime.i http://bugs.python.org/issue24954 opened by gvanrossum #24955: … [View More]

1 0

tp_finalize vs tp_del sematics
by Valentine Sinitsyn Sept. 4, 2015

Sept. 4, 2015

Hi everybody, I'm trying to get sense of PEP-0442 [1]. Most of the looks clear, however I wasn't able to answer myself one simple question: why it wasn't possible to implement proposed CI disposal scheme on top of tp_del? Common sense suggests that tp_del and tp_finalize have different semantics. For instance, with tp_finalize there is a guarantee that the object will be in a safe state, as documented at [2]. However, tp_del is not documented, and I have only vague idea of its guarantees.… [View More]

3 11

Creating server-side clones on hg.python.org is broken right now
by Larry Hastings Sept. 4, 2015

Sept. 4, 2015

Howdy. I went here: https://hg.python.org/cpython/ I clicked on "Server-side clone". I filled out the form. I clicked on the "clone" button. I was rewarded with this error message: error An error occurred while processing your request: no such method: rclone HALP PLS FX, //arry/

1 0